Audio notification display method and electronic device

Abstract

The present application provides a method for displaying an audio notification and an electronic device. The electronic device may be a PC, etc. The PC may play audio from multiple applications and / or web pages at the same time. When a call request is received, the PC may aggregately display a prompt window of the call request and an audio operation window corresponding to the application and / or web page in a non-focus state on a display screen according to the application window and / or web page displayed in a focus state. Each audio operation window includes one or more controls, so that the user can control the playback of each audio through the audio operation window. For example, the user can pause the playback of audio through the audio operation window to ensure that the call process is not disturbed by other output audios. After the user ends the call, other audio operation windows except the prompt window of the call request may be aggregated and displayed on the PC display screen to facilitate the user to quickly resume the audio playback, thereby realizing fast switching of audio playback, convenient operation, and improving the user's operating experience.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a method for displaying an audio notification and an electronic device. Background Art

[0002] With the development of the digital intelligent era and electronic devices, the types and quantities of electronic devices used by users are increasing, and the functions of each electronic device are becoming more and more abundant. Users can use different functions of an electronic device at the same time.

[0003] For example, taking the multi-screen collaboration process of a personal computer (PC) and a mobile phone as an example, in one possible scenario, the user may simultaneously run one or more applications that can output corresponding audio on the PC, or open one or more web pages that can output audio. In addition, the PC may also simultaneously run an application with an instant messaging function, or run an application with an instant messaging function on the mobile phone, such as application, Applications such as mobile phone applications, free call applications, games, etc. that have instant communication functions such as voice calls and video calls. When a user plays multiple audios from different applications at the same time through a PC, if the PC receives an instant call request such as a voice call or a video call, a prompt window related to the instant communication will pop up on the PC display screen. The prompt window is used to remind the user to promptly handle the currently received instant call request such as a voice call or a video call.

[0004] In the above scenario, the audio played by the PC will affect the user's instant communications such as voice calls and video calls. The user needs to first trigger the audio window or web page to be paused to be displayed in focus, and then click the play control in the focused audio window or web page to pause the audio output by the application window. If the user opens many application windows or web pages that can output audio, that is, when the PC outputs multiple audios at the same time, the user will have to perform more steps. Therefore, when the PC receives an instant call request such as a voice call or video call, how to provide users with a convenient method to control the playback of multiple audios is a problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a method for displaying an audio notification and an electronic device. When a call request is received, the method can aggregately display a prompt window of the call request and an audio operation window corresponding to the application and / or web page in a non-focus state on the display screen of the electronic device according to the application window and / or web page displayed in a focus state, so that the user can control the playback of each audio through the audio operation window to ensure that the call process will not be disturbed by other output audios; after the user ends the call, other audio operation windows except the prompt window of the call request are aggregated and displayed on the display screen of the electronic device, so that the user can quickly resume the audio playback, thereby realizing fast switching of audio playback, convenient operation, and improving the user's operating experience.

[0006] In a first aspect, a method for displaying an audio notification is provided, which is applied to a first electronic device including a display screen, the method comprising: receiving a call request, the call request including a voice call request or a video call request; in response to the call request, determining one or more applications that output audio on the first electronic device; and displaying a call prompt window, a first audio operation window, and a second audio operation window in an aggregated manner, the first audio operation window including a first control for controlling audio playback of a first application, the second audio operation window including a second control for controlling audio playback of a second application, and the call prompt window including a third control for controlling a call, wherein the first application and the second application are applications among the one or more applications that output audio on the first electronic device.

[0007] Optionally, the "call request" may originate from an instant messaging application. The "instant messaging application" may include: application, Applications, Changlian calling applications, and game applications with voice calling, video calling and other calling functions. When the instant messaging application receives a call request sent by other friends, an audio operation window will pop up in time to remind the user that the call request needs to be processed.

[0008] Alternatively, the "call request" may also come from a phone application, a short message application, etc. For example, when the electronic device is a mobile phone, a tablet, etc., a call made by other contacts, a call initiated by a short message application, etc. may be received.

[0009] It should be understood that in the introduction of the subsequent embodiments, the voice call request, video call request, etc. of any of the above applications are collectively referred to as "call request".

[0010] Optionally, the application corresponding to the call request may be a local application installed and actually running on the first electronic device (e.g., a PC); or, in a multi-screen collaborative connection scenario between the first electronic device (e.g., a PC) and the second electronic device (e.g., a mobile phone), the application corresponding to the call request may also be an application installed and actually running on the second electronic device (e.g., a mobile phone), which is not limited to this in the embodiments of the present application.

[0011] Optionally, the "one or more applications" mentioned in this application may be an application on the first electronic device (such as a PC) that can output audio. For example, a music application is running on the first electronic device, and the music application can output audio such as songs, but may not be outputting audio at present.

[0012] Alternatively, the "one or more applications" may be one or more applications that are playing audio on the first electronic device (e.g., a PC). For example, music application A and music application B are running on the first electronic device, and both music application A and music application B can output audio such as songs, but currently music application A is outputting audio, and music application B is not outputting audio. It should be understood that the above different scenarios can all be included in the protection scope of the embodiments of the present application, and can all be applied to the method for displaying audio notifications provided in the embodiments of the present application, which will not be described in detail later.

[0013] Through the above-mentioned audio notification display method, the first electronic device (such as a PC) can play the audio output by one or more applications. When the first electronic device (such as a PC) receives a call request, in response to the call request, the first electronic device (such as a PC) can determine the one or more applications currently playing the audio, as well as the application window displayed in the focus state on the display screen, and automatically pop up the audio operation window of one or more applications. Among them, each audio operation window may include one or more operation controls. The user can quickly control the audio playback through the controls in the audio operation window, avoiding the user from performing cumbersome operations to pause the audio playback, allowing the user to answer the call as soon as possible, and can ensure that the user's call process is not disturbed by other audio output by the PC, thereby improving the user's operating experience.

[0014] In combination with the first aspect, in certain implementations of the first aspect, a user operation on the first control is received, and in response to the operation on the first control, playing of the audio of the first application is paused; a user operation on the second control is received, and in response to the operation on the second control, playing of the audio of the second application is paused; and a user operation on the third control is received, and in response to the operation on the third control, a call is answered.

[0015] Through the above process, the user can quickly control the playback of each application audio through the controls in each audio operation window, for example, click the playback control to pause the corresponding audio. This avoids the user having to perform cumbersome operations to pause the audio, allowing the user to answer the call as soon as possible, and ensuring that the user's call process is not disturbed by other audio output from the PC.

[0016] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, in response to the operation of ending the call, a fourth audio operation window and a fifth audio operation window are displayed, the fourth audio operation window includes a fourth control, and the fourth control is used to control the audio playback of the first application, and the fifth audio window includes a fifth control, and the fifth control is used to control the audio playback of the second application.

[0017] When the user ends the call, one or more audio operation windows may automatically pop up on the display screen of the first electronic device (eg, PC), allowing the user to re-control one or more applications and / or web pages to continue playing the audio.

[0018] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, a user operation on the fourth control is received, and in response to the operation on the fourth control, the audio of the first application is resumed; and a user operation on the fifth control is received, and in response to the operation on the fifth control, the audio of the second application is resumed.

[0019] It should be understood that the fourth audio operation window and the aforementioned first audio operation window are both used to control the audio playback of the first application, and may display different or the same controls. For example, the first audio operation window displays the play control, and the fourth audio operation window displays the pause control, and they may both be referred to as the "audio operation window of the first application."

[0020] Through the above process, when the user ends the call, the first electronic device (such as a PC) can display other audio operation windows in addition to the call prompt window, and the user can quickly resume audio playback through other audio operation windows. For example, the user can click the pause control to resume the application and / or web page audio playback, and can quickly restore the first electronic device (such as a PC) to the audio playback state before receiving the call request, thereby realizing fast switching of audio playback, convenient operation, and improving the user's operating experience.

[0021] Optionally, the application, number of windows, display position, display size, etc. corresponding to the "other audio operation windows" may be consistent with the audio operation window displayed when the first electronic device (such as a PC) receives a call request, or, the application, number of windows, display position, display size, etc. corresponding to the "other audio operation windows" may be different from the audio operation window displayed when the PC receives a call request, and the embodiments of the present application are not limited to this.

[0022] Exemplarily, during the process of the user answering the call, the application running on the first electronic device (e.g., PC) may not change. For example, during the process of the user answering the call, the application running status of the first electronic device (e.g., PC) is not closed or changed. When the call is ended, the first electronic device (e.g., PC) continues to run the first application, the second application, and the third application. Accordingly, in response to the operation of ending the call, the first electronic device (e.g., PC) aggregately displays the audio operation window (i.e., the fourth audio operation window) for controlling the first application and the audio operation window (i.e., the fifth audio operation window) for controlling the second application.

[0023] Alternatively, while the user is answering the call, the application running on the first electronic device (such as a PC) may also change. For example, while the user is answering the call, the first application is closed, and the fourth application is run, and the fourth application is also an application that can output audio on the first electronic device (such as a PC). Accordingly, in response to the operation of ending the call, the first electronic device (such as a PC) can aggregate the display for controlling the audio operation window of the second application (i.e., the fifth audio operation window) and the audio operation window for controlling the fourth application. The above different audio notification display methods all fall within the scope of protection of this application.

[0024] It should be understood that in the process of displaying an audio notification on a first electronic device (such as a PC), in addition to determining the display position of the call reminder window, the first audio operation window, and the second audio operation window according to the method described above, the display size of the call reminder window, the first audio operation window, and the second audio operation window can also be determined according to a variety of possible methods. For example, the display size of the call reminder window, the first audio operation window, and the second audio operation window can be determined according to the window of the third application in the focus state, or the display size of the call reminder window, the first audio operation window, and the second audio operation window of a preset size can be displayed according to a preset size, and the embodiments of the present application are not limited to this.

[0025] Through the above method, the applicable scenarios of the display method of audio notifications in the present application are enriched. No matter what operations the user performs during a call, or whether other applications are closed or re-run, this method can be used to display an audio notification after the call ends, based on the application running on the first electronic device (such as a PC), quickly resume audio playback, or achieve fast switching of audio playback. The operation is convenient and the user's operating experience is improved.

[0026] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the aggregated display of the call reminder window, the first audio operation window and the second audio operation window includes: according to the display position of the window of the third application, the call reminder window, the first audio operation window and the second audio operation window are aggregated and displayed, and the window of the third application is displayed as a focus state on the display screen.

[0027] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the call prompt window, the first audio operation window and the second audio operation window are aggregated and displayed according to the display position of the window of the third application, including: at any border of the window of the third application, the call prompt window, the first audio operation window and the second audio operation window are aggregated and displayed; or, inside the window of the third application, the call prompt window, the first audio operation window and the second audio operation window are aggregated and displayed.

[0028] It should be understood that the above method can be specifically combined with the position and size of the remaining display area on the display screen of the first electronic device (such as a PC) to make a judgment, and the embodiments of the present application are not limited to this.

[0029] Exemplarily, the call prompt window, the first audio operation window, and the second audio operation window may be aggregated and displayed in an area outside the window border of the third application in the focus state, that is, the call prompt window, the first audio operation window, and the second audio operation window may not overlap with the window of the third application in the focus state. This display method can avoid blocking the window of the third application in the focus state, facilitate user operation, and make it easier for users to view more interface content on the display screen of the first electronic device.

[0030] Alternatively, the call prompt window, the first audio operation window, and the second audio operation window may be aggregated and displayed inside the window of the third application in the focus state. In other words, the call prompt window, the first audio operation window, and the second audio operation window may have an overlapping area with the window of the third application in the focus state, such as a partial overlapping area or a full overlapping area. This display method can provide a more eye-catching display effect for the user, making the prompt more obvious.

[0031] It should be understood that when the user is using the first electronic device (e.g., PC), the application window in focus may change continuously. When the application window in focus changes, in response to the call request, the number of windows automatically popped up by the first electronic device (e.g., PC), the application corresponding to the window, and the window display content may also change adaptively.

[0032] Through the above scheme, when the first electronic device (such as a PC) receives a call request, it can detect the application window in the focus state and the application window in the non-focus state on the display screen, and aggregate and display the audio operation window and the call request prompt window around the application window in the focus state. In other words, when the application window in the focus state and / or the application window in the non-focus state on the display screen of the first electronic device (such as a PC) are different, the number, type, display position, and display size of the audio operation windows that pop up automatically will also be different.

[0033] When it is determined that there is no application window in focus on the display screen of the first electronic device, the first electronic device (such as a PC) may also aggregate all current audio tasks through a synchronization module, and aggregately display the call reminder window, the first audio operation window, and the second audio operation window at a preset position on the display screen.

[0034] Optionally, the preset position may be the upper boundary of the display area of ​​the display screen of the first electronic device (e.g., PC) and located at the center of the width direction of the display screen. Alternatively, the preset position may be the lower boundary of the display area of ​​the display screen of the first electronic device (e.g., PC) and located at the center of the width direction of the display screen. Alternatively, the preset position may be a fixed position of the upper left corner of the display screen of the first electronic device (e.g., PC). Alternatively, the preset position may be a fixed position of the lower right corner of the display screen of the first electronic device (e.g., PC), etc., which is not limited in the embodiments of the present application.

[0035] In another possible display method, the call reminder window that automatically pops up on the first electronic device (such as a PC), the first audio operation window for controlling the audio playback of the first application, and the second audio operation window for controlling the audio playback of the second application can be displayed in the form of "one window" on the display screen of the first electronic device (such as a PC).

[0036] Optionally, the "one window" may include a first control, a second control and a third control at the same time; or, the one operation window may include two controls, wherein control 1 may simultaneously control the audio playback of the first application and the audio playback of the second application, and control 2 may be used to control answering calls; or, the one operation window may include one control, which may simultaneously control the audio playback of the first application and the audio playback of the second application, and control answering calls, and the embodiments of the present application are not limited to this.

[0037] Alternatively, two windows may automatically pop up on the display screen of the first electronic device (such as a PC), and window 1 is used to control the audio playback of the first application and the audio playback of the second application, etc. It should be understood that the above multiple possible audio notification display methods all fall within the scope of protection of the embodiments of the present application, and examples are not given one by one here.

[0038] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, when multiple web pages are displayed in the window of the third application displayed in the focus state on the display screen, after the first electronic device receives the call request, the method also includes: detecting a first web page that outputs audio in the window of the third application, the first web page being displayed in a non-focus state in the window of the third application; and displaying a sixth audio operation window according to the display position of the window of the third application, the sixth audio operation window including a sixth control for controlling the audio playback of the first web page.

[0039] In combination with the first aspect and the above implementations, in some implementations of the first aspect, the third application is a browser application.

[0040] Exemplarily, when the third application is a browser application, two or more web pages capable of outputting audio may be displayed in the window of the third application. The focus monitoring module of the first electronic device may monitor the application window in focus on the display screen, and the switching process of the web page interface in the window, for example, to determine that the first web page in the window of the third application outputs audio on the first electronic device, and the first web page is in a non-focus state. Accordingly, the first electronic device may also aggregate and display the audio operation window corresponding to the first web page to control the audio playback of the first web page. Among them, the display of the audio operation window corresponding to the first web page may refer to the display rules of the aforementioned call prompt window, the first audio operation window and the second audio operation window.

[0041] In combination with the first aspect and the above-mentioned implementations, in some implementations of the first aspect, the first application and / or the second application and / or the third application are run on a second electronic device, and the second electronic device is connected to the first electronic device.

[0042] It should be noted that the first electronic device (such as a PC) can directly install the above-mentioned audio notification display method without distinguishing the application source of one or more applications that output audio, and determine the display method of the audio notification only based on the application window in focus displayed on the display screen and / or the web page displayed in the application window.

[0043] In another possible implementation, after receiving a call request, the first electronic device (e.g., PC) can determine whether one or more applications that output audio are all from local applications of the first electronic device (e.g., PC), or include applications from a second electronic device that is connected to the first electronic device. For example, a scenario of a second electronic device in a multi-screen collaborative scenario (e.g., a mobile phone and a PC establishing a multi-screen collaborative connection).

[0044] In this multi-screen collaboration scenario, the first electronic device (such as a PC) can also determine the source of all applications that can output audio, such as determining whether the one or more applications include applications running in the foreground or background of the second electronic device (such as a mobile phone). When the display screen of the first electronic device (such as a PC) includes a multi-screen collaboration main window and the multi-screen collaboration main window is displayed in focus, after receiving the call request, it can be: aggregated and displayed in the multi-screen collaboration main window, the call prompt window, the first audio operation window and the second audio operation window. Or, according to the display position of the multi-screen collaboration main window, the call prompt window, the first audio operation window and the second audio operation window are aggregated and displayed. Alternatively, it is detected whether the one or more applications include an application derived from the second electronic device. When the one or more applications include a fifth application derived from the second electronic device, the audio operation window of the fifth application is displayed in the multi-screen collaboration main window, and according to the display position of the multi-screen collaboration main window, the call prompt window, the first audio operation window and the second audio operation window are aggregated and displayed. Alternatively, when the call request originates from the second electronic device, a call prompt interface corresponding to the call request is displayed in the multi-screen collaboration main window, and the first audio operation window and the second audio operation window are aggregated and displayed according to the display position of the multi-screen collaboration main window.

[0045] To summarize, when a user is using an electronic device such as a PC, the PC can play audio from multiple applications and / or web pages at the same time. The multiple applications may include local applications of the PC, and may also include applications on the mobile phone side that establish a multi-screen collaborative connection with the PC. Specifically, when a call request is received, the PC can display the application window and / or web page displayed in the focus state, for example, display the audio operation window corresponding to the non-focus application and / or web page on the display screen, so that the user can quickly control the playback of each audio through the audio operation window. For example, the user pauses the audio playback through the audio operation window and then answers the call, ensuring that the call process will not be disturbed by other output audio, avoiding the user from having to perform cumbersome operations to pause the audio playback, simplifying the operation process, and improving the user's operating experience.

[0046] Secondly, after the user ends the call, other audio operation windows in addition to the call request prompt window may automatically pop up on the PC display screen. The user can quickly resume audio playback through other audio operation windows. For example, the user can click the pause control to resume the application and / or web page audio playback, and can quickly restore the PC to the audio playback state before receiving the call request, thereby realizing fast switching of audio playback, convenient operation, and improving the user's operating experience.

[0047] Furthermore, during the multi-screen collaboration process of PC and mobile phone, the audio operation window that pops up on the PC can avoid users switching to mobile phones to handle call requests. Users do not need to actively switch the currently used PC to the mobile phone, which can achieve the continuity of operation tasks and improve the user experience to a certain extent.

[0048] According to a second aspect, a first electronic device is provided, which includes a display screen, one or more processors, one or more memories, a module with multiple applications installed, and the memory stores one or more programs, wherein the one or more programs are stored in the memory, and when the one or more programs are executed by the processor, the electronic device performs the following steps: receiving a call request, wherein the call request includes a voice call request or a video call request; in response to the call request, determining one or more applications that output audio on the first electronic device; and displaying a call prompt window, a first audio operation window, and a second audio operation window in an aggregated manner, wherein the first audio operation window includes a first control for controlling the audio playback of the first application, the second audio operation window includes a second control for controlling the audio playback of the second application, and the call prompt window includes a third control for controlling the call, wherein the first application and the second application are applications among the one or more applications that output audio on the first electronic device.

[0049] In combination with the second aspect, in certain implementations of the second aspect, when the one or more programs are executed by the processor, the electronic device performs the following steps: receiving an operation of the first control by the user, and pausing playing the audio of the first application in response to the operation of the first control; receiving an operation of the second control by the user, and pausing playing the audio of the second application in response to the operation of the second control; receiving an operation of the third control by the user, and answering a call in response to the operation of the third control.

[0050] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, when the one or more programs are executed by the processor, the electronic device performs the following steps: in response to the operation of ending the call, displaying a fourth audio operation window and a fifth audio operation window, the fourth audio operation window including a fourth control, the fourth control is used to control the audio playback of the first application, and the fifth audio window includes a fifth control, the fifth control is used to control the audio playback of the second application.

[0051] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, when the one or more programs are executed by the processor, the electronic device performs the following steps: receiving a user operation on the fourth control, and in response to the operation on the fourth control, resuming playing the audio of the first application; receiving a user operation on the fifth control, and in response to the operation on the fifth control, resuming playing the audio of the second application.

[0052] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, when the one or more programs are executed by the processor, the electronic device performs the following steps: according to the display position of the window of the third application, the call reminder window, the first audio operation window and the second audio operation window are aggregated and displayed, and the window of the third application is displayed as a focus state on the display screen.

[0053] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, when the one or more programs are executed by the processor, the electronic device performs the following steps: at any border of the window of the third application, the call reminder window, the first audio operation window and the second audio operation window are aggregated and displayed; or, inside the window of the third application, the call reminder window, the first audio operation window and the second audio operation window are aggregated and displayed.

[0054] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, when the one or more programs are executed by the processor, the electronic device performs the following steps: detecting a first web page that outputs audio in the window of the third application, and the first web page is displayed as a non-focus state in the window of the third application; and displaying a sixth audio operation window according to the display position of the window of the third application, and the sixth audio operation window includes a sixth control for controlling the audio playback of the first web page.

[0055] In combination with the second aspect and the above implementations, in some implementations of the second aspect, the third application is a browser application.

[0056] In combination with the second aspect and the above-mentioned implementations, in some implementations of the second aspect, the first application and / or the second application and / or the third application are run on a second electronic device, and the second electronic device is connected to the first electronic device.

[0057] In a third aspect, an electronic device is provided, comprising: a touch display screen; one or more processors; a memory; a plurality of applications; and one or more computer programs. The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device executes the display method of the audio notification executed by the first electronic device in any possible implementation of any of the above aspects.

[0058] In a fourth aspect, a graphical user interface system on an electronic device is provided, wherein the electronic device has a display screen, one or more memories, and one or more processors, wherein the one or more processors are used to execute one or more computer programs stored in the one or more memories, and the graphical user interface system includes a graphical user interface displayed when the electronic device executes the method described in any one of the first aspect and the first aspect.

[0059] In a fifth aspect, a device is provided, which is included in an electronic device, and the device has the function of implementing the behavior of the first electronic device in the method described in any one of the first and second aspects. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a display module or unit, a detection module or unit, a processing module or unit, etc.

[0060] In a sixth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes a method that the first electronic device can execute as described in the first aspect and any one of the first aspects.

[0061] In a seventh aspect, a computer program product is provided. When the computer program product is run on an electronic device, the electronic device executes the first aspect or any one of the methods of the first aspect that the first electronic device can execute. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is a diagram of an example of a graphical user interface showing audio notifications on a PC.

[0063] Figure 2 is another example of a graphical user interface diagram showing an audio notification on a PC.

[0064] Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0065] Figure 4 This is a software structure block diagram of a mobile phone and a PC in a multi-screen collaboration scenario in an embodiment of the present application.

[0066] Figure 5 This is a schematic diagram of an example of displaying an audio notification on a PC provided in an embodiment of the present application.

[0067] Figure 6 This is a schematic diagram of an example of an audio notification displayed on a PC after a call ends, provided in an embodiment of the present application.

[0068] Figure 7 This is another schematic diagram of displaying an audio notification on a PC provided in an embodiment of the present application.

[0069] Figure 8 This is another schematic diagram of an example of an audio notification displayed on a PC after a call ends, provided in an embodiment of the present application.

[0070] Fig. 9 This is another schematic diagram of displaying an audio notification on a PC provided in an embodiment of the present application.

[0071] Fig.10 This is another schematic diagram of displaying an audio notification on a PC after a call ends, provided in an embodiment of the present application.

[0072] Fig.11 This is a schematic diagram of another example of displaying an audio notification on a PC provided in an embodiment of the present application.

[0073] Fig.12 This is a schematic diagram of another example of displaying an audio notification on a PC after a call ends, provided in an embodiment of the present application.

[0074] Fig.13 This is a schematic diagram of an example of displaying an audio notification on a PC during a multi-screen collaboration process provided by an embodiment of the present application.

[0075] Fig.14 This is another schematic diagram of displaying an audio notification on a PC during a multi-screen collaboration process provided by an embodiment of the present application.

[0076] Fig.15 This is a schematic diagram of an example of an audio notification displayed on a PC after a call ends, provided in an embodiment of the present application.

[0077] Fig.16 This is another schematic diagram of displaying an audio notification on a PC during a multi-screen collaboration process provided by an embodiment of the present application.

[0078] Fig.17 This is another schematic diagram of displaying an audio notification on a PC after a call ends, provided in an embodiment of the present application.

[0079] Fig.18 This is a schematic flowchart of a method for displaying an audio notification provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] For ease of understanding, the following will take mobile phones, PCs and other electronic devices as examples, combined with the accompanying drawings, to specifically explain the display process of audio notifications on different electronic devices such as mobile phones and PCs.

[0081] Figure 1 A diagram of a graphical user interface (GUI) for displaying audio notifications on a PC is shown.

[0082] In a possible scenario, when a user uses a PC independently, he may run one or more applications installed on the PC at the same time, each application may display a different application window on the display screen of the PC, and each application may output corresponding audio. At the same time, the user may also open one or more web page interfaces through the browser application of the PC, and the one or more web page interfaces may include an interface capable of outputting audio.

[0083] For example, Figure 1 In the scenario shown, it is assumed that the user runs a music application installed on the PC and is playing song 2, and a window 10 of the music application is displayed on the display screen of the PC. Optionally, the window 10 of the music application may display the following: Figure 1 The interface shown may include information about the user name, names of multiple song lists such as private FM (frequency modulation, FM), favorites list, my music, and recently played, as well as multiple song lists such as song list 1 and song list 2 included in the interface corresponding to the "Daily Recommendation" menu. The embodiment of the present application does not limit the display content of the window 10 of the music application.

[0084] In addition, assuming that the user also opens the online playback webpage of song 1 through the browser application, accordingly, the online playback webpage of song 1 can be displayed on the PC display screen and in the window 20 of the browser application. The interface may include the song name of song 1, singer picture, song playback progress bar, lyrics display interface and other contents, which are not limited to this in the embodiment of the present application.

[0085] Assume that the current user is playing song 2 through the music app. Figure 1 In the window 10 of the music application shown, the option for song 2 includes a play control 101, indicating that the music output by the music application of the current PC is song 2. At the same time, the user is playing song 1 through the window 20 of the browser application, that is, Figure 1 In the window 20 of the browser application shown, a playback control 20 - 2 for song 1 is displayed.

[0086] For the scenario introduced above, when the window 10 of the music application and the window 20 of the browser application are displayed on the display screen of the PC, the application icon 10-1 corresponding to the window 10 of the music application and the application icon 20-1 corresponding to the window 20 of the browser application can be displayed in the bottom navigation area of ​​the PC display screen.

[0087] It should be understood that when the PC has different software systems, the application icon 10 - 1 and the application icon 20 - 1 may be displayed at different positions on the display screen, or displayed in different ways, which is not limited in this embodiment of the present application.

[0088] It should also be understood that a PC is generally provided with a plurality of audio devices, for example, a PC may include different audio devices such as headphones, microphones, and speakers. Among them, a device that can output and play audio is called an "audio playback device", and the audio playback device on a PC is not limited to being a built-in device or an external device of the PC. For example, a PC is connected to headphones through a headphone jack, and the headphones can be used as an external audio playback device of the PC, and the speakers are built-in audio playback devices of the PC. The embodiments of the present application do not limit the number and types of audio playback devices of the PC.

[0089] It should also be understood that the PC can output and play multiple audios at the same time, or the PC can also play audios from different sources simultaneously through different audio playback devices or the same audio playback device.

[0090] Exemplary, combined Figure 1In the scenario shown, the user can set the PC to play song 2 currently being played by the music application and song 1 output by the browser application window 20 through the headphones connected to the headphone jack at the same time. Alternatively, the user can also set the PC to play song 2 currently being played by the music application through the headphones connected to the headphone jack, and simultaneously play song 1 output by the browser application window 20 through the PC's speakers. Alternatively, the user can set the PC to play song 2 currently being played by the music application and song 1 output by the browser application window 20 through the speakers at the same time, etc. When the PC outputs multiple audios simultaneously, the embodiment of the present application does not limit the types of audio playback devices corresponding to the multiple audios.

[0091] For the above scenario, when the PC is playing one or more audios, if the PC is also running an instant messaging application, such as application, When a PC receives a call request from an instant messaging application, a prompt window will pop up on the PC display to remind the user that there is a call request that needs to be processed.

[0092] For example, Figure 1 As shown, the PC runs As the instant messaging application, When the application receives a call request from the user "Miss Ma", in response to the call request, a prompt window 30 may automatically pop up in the lower right corner area (or other preset fixed area) of the PC display screen, and the prompt window 30 is used to remind the user that there is a call request that needs to be processed. In addition, the application icon 30-1 of the WeChat application may be displayed in the bottom navigation area of ​​the display screen, and the application icon 30-1 of the WeChat application may flash in different colors to remind the user that there is a call from Instant messaging for apps.

[0093] Alternatively, if Figure 1 As shown, the prompt window 30 may display the image of the user "Miss Ma", the remark name, the hang-up control 30-2 and the answering control 30-3 and other contents. The user may click the answering control 30-3 to make a voice call with the user "Miss Ma" according to his / her needs, or may click the hang-up control 30-2 to reject the call request from the user "Miss Ma".

[0094] When the user wants to answer a call from "Miss Ma", in order to ensure that the call is not disturbed and affected by other audio, the user first needs to pause the audio currently playing on the PC, and then click the answer control 30-3 to make a voice call with the user "Miss Ma". Figure 1 In the scenario shown, the user may need to click any area of ​​the window 20 of the browser application to make the window 20 of the browser application display in the focus state, and then click the play control 20-2 of song 1 in the window 20 of the browser application to pause the song 1. In addition, the user also needs to click any area of ​​the window 10 of the music application to make the window 10 of the music application display in the focus state, and then click the play control 101 in the window 10 of the music application to pause the song 2 being played by the music application.

[0095] In the above process, if the user switches the window 10 of the music application or the window 20 of the browser application to a minimized state or back to a background running state, then when the user needs to browse or operate again, more steps may be required to call out the window 10 of the music application or the window 20 of the browser application, and then sequentially pause the audio output by the window 10 of the music application or the window 20 of the browser application. This process is cumbersome to operate. When the user opens too many application windows or many web pages, the user needs to sequentially pause the audio of different application windows or web pages before answering the voice call. The user will have more operation steps, which reduces the user's operating experience.

[0096] It should be noted here that in the description of the embodiments of the present application, the different user operations such as "click" and "double-click" may be operations performed by a user through a PC's peripheral input devices such as a mouse and keyboard; or, for electronic devices such as PCs and mobile phones with a touch screen, "user operation" may also be an operation performed by a user through fingers on the touch screen, such as touch operation, click operation, double-click operation, sliding operation, etc. The embodiments of the present application do not limit the type and method of user operations, and for the sake of simplicity, they will not be elaborated later.

[0097] It should also be noted that the "focus state" in the embodiment of the present application can be understood as the window corresponding to the application being in the display state of the top layer, and the middle window corresponding to the application in the focus state can directly receive user operations. For example, Figure 1In the scenario shown, the window 20 of the browser application is displayed in the focus state, so the window 20 of the browser application is displayed at the upper level of the window 10 of the music application, and the user can directly perform the desired operation on a certain control in the window 20 of the browser application. If the user wants to switch the window 10 of the music application to the focus state, the user can click any position of the window 10 of the music application to make the window 10 of the music application displayed at the upper level of the window 20 of the browser application, and the embodiment of the present application will not be repeated.

[0098] Figure 2 is another example of a graphical user interface diagram showing an audio notification on a PC.

[0099] In another possible scenario, users can use mobile phones and PCs for multi-screen collaboration. During the multi-screen collaboration process between mobile phones and PCs, users can run one or more applications installed on the mobile phone side on the PC, and one or more mobile phone windows will be displayed on the PC display. In addition, users can also run one or more applications installed locally on the PC at the same time, or open one or more web pages through the PC's browser application. In the scenario of multi-screen collaboration between mobile phones and PCs, the PC can output multiple audios at the same time, and the multiple audios output may include audio output from applications on the mobile phone side, and may also include audio output from applications and / or web pages on the PC side.

[0100] For example, Figure 2 Figure (a) shows a scenario of multi-screen collaboration between a mobile phone and a PC, wherein the mobile phone can display the main interface as shown in the figure in unlocked mode, and the main interface may include a top status bar display area, a weather clock component, and multiple applications (application, App), etc. Optionally, the top status bar display area may display the operator's communication signal icon, wireless fidelity (wireless fidelity, Wi-Fi) network icon, battery icon, and current clock information, etc., and multiple applications may include phone calls, messages, Huawei music, and WeChat, etc. It should be understood that the main interface may also include other more applications or display more content, which is not limited in the embodiments of the present application.

[0101] like Figure 2As shown in Figure (a), the main window 40 of multi-screen collaboration is displayed on the display screen of the PC, and the display content of the main window 40 of multi-screen collaboration can be consistent with the display content of the main interface of the mobile phone. Assuming that the user runs the music application of the main window 40 of multi-screen collaboration, a sub-window can be independently displayed on the PC - the window 10 of the music application, and the user also opens the online playback webpage of song 1 through the local browser application of the PC. Correspondingly, the window 20 of the browser application can be displayed on the display screen of the PC, and the online playback webpage of song 1 is displayed in the window 20 of the browser application.

[0102] Optionally, in the embodiment of the present application, if the user starts and runs multiple mobile phone applications in the main window 40 of the multi-screen collaboration, each application can be displayed on the PC display screen in the form of an independent sub-window, or the running interface of different applications can be displayed in the main window 40 of the multi-screen collaboration in the form of tabs. For example, when the user runs the music application on the mobile phone in the main window 40 of the multi-screen collaboration, the running interface of the music application can be as follows: Figure 2 In addition to the window 10 displaying an independent music application as shown in Figure (a), the running interface of the music application can also be displayed in the main window 40 of the multi-screen collaboration. The embodiment of the present application does not limit the window display rules during the multi-screen collaboration process between the mobile phone and the PC.

[0103] It should be understood that Figure 2 In the multi-screen collaboration scenario shown in Figure (a), if the window 10 of the music application on the mobile phone side, the window 20 of the browser application and the main window 40 of the multi-screen collaboration are displayed on the PC at the same time, accordingly, the application icon 10-1 corresponding to the window 10 of the music application, the application icon 20-1 corresponding to the window 20 of the browser application and the icon 40-1 corresponding to the main window 40 of the multi-screen collaboration can be displayed in the bottom navigation area of ​​the PC display screen.

[0104] For example, when the user sets the window 10 of the music application to play song 2 and sets the window 20 of the browser application to play song 1 online, that is, when the PC plays multiple audios at the same time, if the WeChat application on the mobile phone receives a call request from the user "Miss Ma", in response to the call request, the mobile phone can display the following Figure 2 The voice call interface shown in Figure (b) includes an image of the user "Miss Ma", a remark name, a hang-up control 202-1, and an answer control 202-2.

[0105] At the same time, in response to the call request, the main window 40 of the multi-screen collaboration on the PC can pop up a floating prompt window 40-2 on the currently displayed main interface, and the prompt window 40-2 can display the image of the user "Miss Ma", the remark name, the hang-up control and the answering control, etc., to prompt the user on the PC of the currently received call request. Alternatively, in response to the call request, the main window 40 of the multi-screen collaboration on the PC can also be consistent with the display interface of the mobile phone, and both display the content included in the voice call interface, which is not limited in the embodiment of the present application.

[0106] Optionally, when a call request from user "Miss Ma" is received on the mobile phone, in response to the call request, a prompt window 30 may be automatically popped up in the lower right corner area (or other preset fixed area) of the PC display screen, and the prompt window 30 is used to remind the user that there is a call request that needs to be processed. In addition, in response to the call request, the bottom navigation area of ​​the PC display screen may automatically pop up the application icon 30-1 of the WeChat application, and the application icon 30-1 of the WeChat application may present a flashing state of different colors. The user can click the answer control 30-3 according to his own needs to talk to the user "Miss Ma", or click the hang-up control 30-2 to reject the call request from the user "Miss Ma".

[0107] When the user expects to answer a call from "Miss Ma", in order to ensure that the call is not disturbed or affected by other audios and is not mixed with other audios, the user needs to first pause each of the multiple audios currently playing on the PC, and then click the answering control 30-3 in the prompt window 30 to make a voice call with the user "Miss Ma"; or, click the answering control 40-3 in the main window 40 of the multi-screen collaboration to make a voice call with the user "Miss Ma"; or, the user can follow the prompt on the PC and click the answering control 202-2 on the voice call interface on the mobile phone to make a call with the user "Miss Ma". The embodiment of the present application does not limit the way in which the user answers the call.

[0108] In the above implementation, when the user pauses multiple audios currently playing on the PC, and Figure 1Similar to the scenario introduced, the user needs to perform multiple operations, first triggering the application window to be operated to be displayed in the focus state, and then clicking the play control in the application window in the focus state to pause the audio output by the application window. If the user opens too many application windows that can output audio, or opens many web pages that can output audio, that is, when the PC outputs multiple audios at the same time, the user wants to pause the playback of one or more audios among multiple audios, and the user needs to perform more operation steps. In particular, when the user switches one or more windows that can output audio to a minimized state or back to a background running state, if the user wants to pause the playback of multiple audios, more steps and operations may be required. The process is cumbersome and affects the user experience.

[0109] In addition, Figure 2 In the multi-screen collaboration scenario shown in Figure (b), when a call request is received from the user "Miss Ma", the PC display screen displays a prompt window 40-2 in the multi-screen collaboration window 40, and at the same time displays a prompt window 30 in the lower right corner area of ​​the display screen (or other preset fixed area), and both prompt window 30 and prompt window 40-2 are used to remind the user that there are instant messaging messages that need to be processed, and will be displayed repeatedly on the PC display screen.

[0110] Furthermore, in another possible scenario, if the user sees a prompt window on the PC display screen while using the PC, he can pick up the mobile phone and click the answering control 202-2 of the voice call interface on the mobile phone according to the prompt content on the PC to talk to the user "Miss Ma". Although this implementation process can also avoid the interference of multiple audios output on the PC and does not require the user to perform the operation of pausing multiple audios on the PC, the process relies on the user to actively switch the currently used device to the mobile phone and use the mobile phone to answer the call, which cannot achieve the continuity of the operation task, and to some extent also affects the user's experience.

[0111] In summary, whether it is Figure 1 The user uses multiple local applications on the PC. Figure 2 In the scenario of multi-screen collaboration between PC and mobile phone shown, when the user is using the PC and the PC outputs multiple audios at the same time, if a call request is received on the PC, in order to avoid interference from the multiple audios output simultaneously by the PC, the user needs to perform cumbersome operations or steps to pause the playback of multiple audios. The cumbersome operation process or display method will affect the user experience.

[0112] In response to the above problems, an embodiment of the present application will provide a method for displaying an audio notification and an electronic device. When the electronic device receives a sudden instant messaging request or message, the method for displaying the audio notification can provide different audio notification display schemes for different scenarios, and can provide users with a convenient way to switch the audio of the electronic device, thereby improving the user experience.

[0113] The technical solution in the embodiment of the present application will be described below in conjunction with the accompanying drawings in the embodiment of the present application. Before introducing the technical solution in the embodiment of the present application, the hardware structure and software structure of the electronic device to which the display method of the audio notification provided in the embodiment of the present application can be applied are first introduced.

[0114] The audio notification display method provided in the embodiment of the present application can be applied to electronic devices such as mobile phones, tablets, PCs, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiment of the present application does not impose any restrictions on the specific types of electronic devices.

[0115] Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0116] It should be understood that the electronic devices such as PCs or mobile phones in the embodiments of the present application may include Figure 3 The electronic device 100 shown may include a partial structure or the entire structure, or may also include more other structures not shown, or may include fewer structures, which is not limited in the embodiments of the present application.

[0117] For example, Figure 3As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0118] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0119] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0120] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0121] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0122] In the embodiment of the present application, the electronic device 100 can be Figure 1 The PC used by the user in the scenario shown can also be Figure 2 The PC or mobile phone in the multi-screen collaboration scenario shown. The processor 110 of the electronic device 100 stores a computer program or instruction corresponding to the display method for implementing the audio notification.

[0123] Exemplarily, taking a PC as an example, the PC can play audio from multiple applications and / or web pages at the same time. When a call request is received, the PC can detect the application window and / or web page in focus and the application window and / or web page in non-focus on the display screen, and aggregate and display the audio operation window and the call request prompt window around the application window in focus. In addition, when the application window and / or web page in focus and the application window and / or web page in non-focus on the PC display screen are different, the number, type, display position, and display size of the audio operation windows that automatically pop up on the PC will also vary.

[0124] Specifically, through the display method of the audio notification provided by the embodiment of the present application, the PC can aggregate and display the audio operation window and the call request prompt window around the application window in the focus state in any of the following possible ways according to the application window and / or web page displayed in the focus state, for example:

[0125] 1) Around the application window in the focused state, the audio operation window corresponding to the application and / or webpage in the non-focused state, the audio operation window corresponding to the application and / or webpage in the focused state, and the prompt window for the call request are displayed in an aggregate manner;

[0126] 2) Around the application window in the focused state, the audio operation window and the call request prompt window corresponding to the application and / or webpage in the non-focus state are preferentially aggregated and displayed. When the available display area of ​​the display screen is sufficient, the audio operation window corresponding to the application and / or webpage in the focused state is displayed;

[0127] 3) Around the application window in focus, only the audio operation window and call request prompt window corresponding to the application and / or web page in non-focus state are aggregated and displayed, and the audio operation window corresponding to the application and / or web page in focus state is not displayed.

[0128] This display method allows users to quickly control the playback of each audio through the audio operation window. For example, the user can pause the audio playback through the audio operation window and then answer the call, ensuring that the call process will not be disturbed by other output audios, avoiding the user from having to perform tedious operations to pause the audio playback, simplifying the operation process and improving the user's operating experience.

[0129] In addition, after the user ends the call, other audio operation windows in addition to the call request prompt window may automatically pop up on the PC display screen. The user can quickly resume audio playback through other audio operation windows. For example, the user can click the pause control to resume the application and / or web page audio playback, and can quickly restore the PC to the audio playback state before receiving the call request, thereby realizing fast switching of audio playback, convenient operation, and improving the user's operating experience.

[0130] It should be noted that during the multi-screen collaboration between PC and mobile phone, the PC can play audio from multiple applications and / or web pages at the same time. The multiple applications can include local applications of the PC and applications on the mobile phone.

[0131] Specifically, through the audio notification display method provided by the embodiment of the present application, when a multi-screen collaborative window in focus is displayed on the PC, the audio operation window of the application on the mobile phone side can be displayed in the multi-screen collaborative window, and the audio operation window of the local application of the PC can be displayed around the multi-screen collaborative window.

[0132] When the multi-screen collaboration window is in a non-focus state or in the background running state, the audio operation window of the application on the mobile phone side and the audio operation window of the local application on the PC can be displayed around the application window in the focus state, so that the user can quickly control the playback of each audio through the audio operation window. For example, the user can pause the audio playback through the audio operation window and then answer the call, ensuring that the call process will not be disturbed by other output audios, avoiding the user from performing tedious operations to pause the audio playback, simplifying the operation process and improving the user's operating experience.

[0133] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0134] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0135] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0136] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0137] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0138] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0139] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0140] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0141] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0142] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0143] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0144] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0145] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0146] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G / 6G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0147] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0148] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0149] For example, in a possible scenario of the embodiment of the present application, Figure 2 The mobile phone and PC shown in the figure can establish a multi-screen collaborative connection. The embodiment of the present application does not limit the method of connecting the multi-screen collaborative connection between the mobile phone and the PC.

[0150] Optionally, in the embodiment of the present application, the mobile phone and the PC, as two different electronic devices, may both include a wireless communication module 160 .

[0151] In a possible implementation, the mobile phone and the PC may be devices connected to the same Wi-Fi network; or, a Bluetooth connection may be established between the mobile phone and the PC; or, a connection may be established between the mobile phone and the PC based on NFC technology such as touch-and-pay; or, a connection may be established between the mobile phone and the PC based on wireless communication methods such as infrared technology (IR), etc. Accordingly, the mobile phone and the PC may communicate in a corresponding manner to support the transmission of data or information in the multi-screen collaboration process. The embodiments of the present application do not limit the communication method between the mobile phone and the PC.

[0152] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with the network and other devices through wireless communication technology.

[0153] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0154] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), MiniLED, MicroLED, Micro-OLED, quantum dot light-emitting diodes (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0155] It should be understood that in the embodiment of the present application, electronic devices such as mobile phones and PCs may all have a display screen 194 .

[0156] For example, Figure 2 Taking the multi-screen collaboration process of the mobile phone and PC as an example, the user may use the PC to play audio from multiple applications. The multiple applications may include local applications of the PC and applications on the mobile phone. The PC display 194 can display the corresponding application interface or application window for the user.

[0157] When the PC receives a call request, in response to the received call request, the PC can determine the number, type and display content of the windows that automatically pop up on the display screen 194 according to whether there is an application window and / or web page displayed in a focused state on the current display screen, and the application of the PC outputting audio, etc. When the display screen 194 includes an application window in a focused state, the PC can determine the display position and / or display size of each audio operation window on the display screen 194 according to the display position and / or display size of the application window in the focused state. For example, the PC can aggregate and display the audio operation window and the call request prompt window around the application window in the focused state. The specific display method can be found in the detailed description of the embodiment, which will not be repeated here. In addition, after the user ends the call, other audio operation windows except the call prompt window can be automatically aggregated and displayed on the display screen 194 of the PC, so that the user can quickly control the application and / or web page to continue playing audio.

[0158] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0159] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0160] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0161] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0162] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0163] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0164] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0165] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0166] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0167] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0168] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0169] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0170] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0171] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0172] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0173] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0174] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0175] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0176] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0177] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0178] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0179] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0180] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0181] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0182] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0183] Optionally, in an embodiment of the present application, if the PC and the mobile phone are electronic devices with touch screens, then the PC and the mobile phone may have the touch sensor 180K, which can detect user operations such as touch and click, generate operation events, and pass the operation events to the processor 110. The processor 110 determines that the operation event includes relevant parameters such as the coordinates of the touch point and the touch status, and responds according to the operation event.

[0184] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0185] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0186] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0187] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0188] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0189] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplified.

[0190] Figure 4 This is a software structure diagram of a mobile phone and a PC in a multi-screen collaboration scenario in an embodiment of the present application. The software structure diagram shows the software structure of the PC and the possible software structure of the mobile phone, and the software structure diagram can support the multi-screen collaboration process between the mobile phone and the PC.

[0191] Optionally, the PC and the mobile phone in the embodiment of the present application can both be system, with Figure 4 The layered architecture shown may include several layers, each layer has a clear role and division of labor, and the layers communicate with each other through software interfaces.

[0192] Specifically, for the software structure of the mobile phone, For example, the hierarchical structure divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. From top to bottom, the system includes the application (APP) layer, application framework (framework) layer, system library, kernel layer and network transmission layer.

[0193] The application layer can include a series of application packages. Figure 4 As shown, the application layer may include applications such as memo, music, video, and settings, which are not limited in this embodiment of the present application.

[0194] The application framework layer may include a variety of service programs or some predefined functions. The application framework layer may provide an application programming interface (API) and a programming framework for the application programs in the application layer.

[0195] In the embodiments of the present application, Figure 4 As shown, the application framework layer may include a content provider, a view system, a notification manager, a window manager, an audio task monitoring module, an audio playback control module, a focus monitoring module, a multi-screen framework service module, an information synchronization module, and the like.

[0196] The content provider is used to store and obtain data and make the data accessible to the application program. The data may include different types of data such as video, image, audio, etc.

[0197] The view system includes visual controls provided to users, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build the display interface of an application. The display interface can be composed of one or more views. For example, for a display interface including icons of different applications, it can include a view for displaying text and a view for displaying pictures.

[0198] The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages. The displayed notification information can automatically disappear after a short stay without user interaction, or accept the user's closing operation, etc. The message manager can also transmit messages between different applications of the mobile phone, or transmit messages between the mobile phone and the PC, etc., which will not be described in detail in the embodiments of the present application.

[0199] The window manager is used to manage the window programs of electronic devices. Specifically, in the multi-screen collaboration process of the mobile phone and the PC, the window manager of the mobile phone can obtain the size of the application window to be displayed on the mobile phone, obtain the interface content or display data of the application window to be displayed, etc. It should be understood that the application window to be displayed on the mobile phone can include the window being displayed on the interface of the mobile phone, and can also include the windows of one or more application programs running in the background of the mobile phone, etc.

[0200] It should be understood that in the embodiments of the present application, an audio task monitoring module, an audio playback control module, a focus monitoring module, a multi-screen framework service module, an information synchronization module, etc. may exist in a PC or a mobile phone, and each module may have the following functions.

[0201] The audio task monitoring module can be responsible for monitoring the current audio task of the electronic device. For example, in the multi-screen collaboration process of a mobile phone and a PC, both the mobile phone and the PC have the audio task monitoring module, which can monitor whether there is an audio task.

[0202] For example, taking a PC as an example, when a music application of the PC plays a song, the audio task monitoring module records the music application in the audio application list. When the music application is exited or closed, or no longer plays songs for a long time, the PC can remove the music application from the audio application list.

[0203] For example, taking PC as an example, in order to realize the monitoring of audio tasks, the audio task monitoring module of PC can obtain the audio stream effects (SFX) from the audio processing object (Audio Processing Objects, APO) framework of the audio system. SFX can take the audio data output by the application as input, and after processing, output it to the mode effects (MFX) for mixing. When the audio information of the application passes through SFX, the audio task monitoring module can record the identification of the application, and then record the application information. Therefore, PC can accurately know which application the audio currently being output comes from.

[0204] The audio playback control module can control the playback of one or more audios.

[0205] For example, taking a PC as an example, the audio playback control module can control the audio playback of one or more applications. When a user closes or pauses an audio task of an application, the SFX corresponding to the audio task can configure the audio data output by the application to a mute mode; when the user turns down the volume, the audio playback control module can reduce the volume of the audio data output by the PC, thereby controlling the playback of one or more audios.

[0206] The focus monitoring module can monitor the switching process between the application window in focus and the application window in focus on the display screen of the electronic device, and detect the switching process between the web page in focus and the web page in focus in a certain application window on the display screen.

[0207] Exemplarily, taking a PC as an example, the focus monitoring module can monitor the application window in focus on the PC's display screen and record the information of the application window in focus, such as which of the music application window and the browser application window on the display screen is in focus; when the user switches the application window in focus, the focus monitoring module can monitor the change process of the focus window.

[0208] Or, further, for a browser application, a plurality of web pages can be displayed in the form of tabs in a window of a browser application. In this scenario, the focus monitoring module can monitor the web page displayed in the focus state in the window of the browser application. In an embodiment of the present application, a "web page in focus state" can be understood as a state in which the window of the browser application is in focus and the web page is displayed at the top level in the window of the browser application, and other web pages are displayed at non-top levels in the window of the browser application.

[0209] The multi-screen framework service module can be used to realize the multi-screen collaboration process between mobile phones and PCs.

[0210] Optionally, in an embodiment of the present application, a "multi-screen framework service layer" can be set between the application layer and the application framework layer, or a "multi-screen framework service module" can be set in the application framework layer. In other words, the multi-screen framework service module can also be divided into the application framework layer, which is not limited to the embodiment of the present application.

[0211] Exemplarily, the multi-screen framework service module can be further used to implement window management, window status monitoring and synchronization, window coordinate synchronization, reverse control and other functions in the multi-screen collaboration process, and each function can be implemented by a corresponding module. In other words, the multi-screen framework service module can further include a window status monitoring and synchronization module, a window coordinate synchronization module, a reverse control module, etc.

[0212] Among them, the window status monitoring and synchronization unit can monitor the running status of the application on the mobile phone, or monitor the running status changes of the application in a coordinated mobile phone window sent by the PC, the user's operations in the mobile phone window, etc., and respond to keep the interface content of the mobile phone application and the interface content of the mobile phone window corresponding to the application on the PC synchronized.

[0213] The window coordinate synchronization unit can determine the coordinates of the mobile phone window to be displayed on the PC display screen. When a mobile phone window displayed on the PC is enlarged, reduced, hidden, minimized, etc., the window coordinate synchronization unit can obtain the new position coordinates of the window after being updated on the PC.

[0214] The reverse control unit can cooperate with the window status monitoring and synchronization unit, the window coordinate synchronization unit, etc. After monitoring the user's operation events, the mobile phone can send the operation results to the PC to achieve information synchronization; or it can accept the user's operation events sent by the PC, and respond according to the user's operation events to keep the interface content of the mobile phone application and the interface content of the mobile phone window corresponding to the application on the PC synchronized.

[0215] The information synchronization module can pause or continue playing a certain audio task according to the user's operation. In addition, the information synchronization module can also synchronize the phone audio on the mobile phone side to the PC side, and the audio playback control module plays the phone audio data.

[0216] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0217] Among them, the surface manager is used to manage the display subsystem of the mobile phone and provide the fusion of two-dimensional and three-dimensional layers for multiple applications.

[0218] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0219] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0220] A 2D graphics engine is a drawing engine for 2D drawings.

[0221] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0222] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, Bluetooth drivers, wireless fidelity (WIFI) drivers, near-field communication (NFC) drivers, etc.

[0223] The network transport layer can be used for communication and data transmission between different devices. For example, the Bluetooth module can establish a Bluetooth channel between the mobile phone and the PC to transmit data or messages, instructions, etc. through the Bluetooth channel. I will not go into details here.

[0224] In the embodiments of the present application, the communication between the mobile phone and the PC may be carried out in a variety of different ways. For example, the connection between the mobile phone and the PC may include a wired connection or a wireless connection and other different connection methods.

[0225] Optionally, the wired connection between the mobile phone and the PC may be connected via a USB data cable, or the wireless connection between the mobile phone 100 and the PC may be established by establishing a Wi-Fi connection. In the embodiment of the present application, when the mobile phone and the PC are wirelessly connected, the kernel layer of the mobile phone may include a wireless fidelity (WIFI) module for connecting the mobile phone to perform a wireless connection.

[0226] Alternatively, with the help of the near field communication (NFC) function supported by the mobile phone and the PC, the mobile phone and the PC can be connected by the "touch and pay" function. When the mobile phone and the PC are connected by NFC, the kernel layer can also include a near-field communication (NFC) module for connecting the mobile phone and the PC through NFC.

[0227] Alternatively, you can connect via Bluetooth QR code scanning on your mobile phone and PC.

[0228] Alternatively, as communication technology develops, communication bandwidth and speed gradually increase, and data may be transmitted between mobile phones and PCs without establishing a near-field communication connection. For example, with the popularization of high-speed communication methods such as the fifth generation (5G) mobile communication system in the future, mobile phones and PCs may be able to display the window of the mobile phone on the PC through 5G communication, for example, by installing different or the same applications on the mobile phone and PC, and transmitting data through the 5G communication network.

[0229] It should be understood that the embodiment of the present application does not limit the method of establishing a connection between the mobile phone and the PC.

[0230] also, Figure 4 It also shows that the PC participates in the network transmission layer, multi-screen framework service module, audio task monitoring module, audio playback control module, focus monitoring module, information synchronization module, etc. of the embodiment of the present application. After the PC and the mobile phone are connected, the network transmission layer is used to establish a connection with the mobile phone, and the multi-screen framework service module is used to display one or more windows of the mobile phone, such as the multi-screen collaborative main window of the mobile phone, sub-windows of different applications, etc.

[0231] It should be understood that the PC may include a multi-layer architecture, such as a pre-research layer, a PC software layer, an operating system (OS) layer, and a basic read-only memory (ROM) multi-screen framework layer, etc., wherein the PC software layer can run an executable file (exe). For example, in the present application, the PC may pre-install an exe program for a multi-screen framework service module, and the exe program of the multi-screen framework service module may be embodied in the form of an App on the PC, or the PC may have the function of displaying a window of a mobile phone by running the exe program of the multi-screen framework service module.

[0232] Alternatively, the PC may also have the same multi-layer architecture as the mobile phone, which is not limited in the embodiments of the present application.

[0233] Exemplarily, taking the multi-screen collaboration process between a mobile phone and a PC as an example, this application refers to the functional module of the PC that provides a window for displaying a mobile phone as a "multi-screen framework service module". The following will focus on the multi-screen framework service module used to display windows for mobile phones on the PC.

[0234] The multi-screen framework service module of the PC can provide the functions of related services corresponding to the multi-screen framework service module of the mobile phone. The multi-screen framework service module can also include a multi-window manager, a window status monitoring and synchronization unit, a mobile phone window display, a reverse control unit, etc.

[0235] Among them, the multi-window manager can receive the window display data sent by the mobile phone, and display one or more mobile phone windows according to the window display data. The window display data may include the coordinate information of the mobile phone window, the content information of the display interface of the mobile phone window, etc. The multi-window manager can determine to display one or more windows of the mobile phone on the display screen of the PC according to the coordinate information of the mobile phone window. The transmission of window display data can be based on the aforementioned Figure 4 Any communication method introduced in the relevant content will not be repeated here.

[0236] It should be understood that in the embodiment of the present application, the application window of the mobile phone displayed on the PC can be referred to as a "mobile phone window", where the window in the mobile phone window used to display the main interface of the mobile phone is called a "main window" or a "multi-screen collaborative window", and the window on the PC that is independent of the "main window (or multi-screen collaborative window)" and is used to display the interface of an application running on the mobile phone is called a "sub-window" or a "window of the XXX application".

[0237] For example, based on the time when a window is displayed on the PC display screen, it can be determined that the first window displayed is the main window, and the second window, third window, etc. displayed after the first window are sub-windows. This embodiment of the present application is not limited to this.

[0238] It should also be understood that the "collaboration" in the "multi-screen collaborative window" referred to in the embodiments of the present application does not limit the display of the main window on the PC and the mobile phone interface to be consistent. In other words, the display content of the multi-screen collaborative window and the main interface of the mobile phone can be consistent or different. In addition, when the user runs any application of the mobile phone on the PC, the application can run in the background of the mobile phone without affecting the display content of the mobile phone screen. The embodiments of the present application do not limit the display content of the mobile phone screen.

[0239] For ease of understanding, the following embodiments of the present application will be described in detail with Figure 3 and Figure 4 Taking the electronic device 100 of the structure shown as an example, such as a PC or a mobile phone, the display method of the audio notification provided in the embodiment of the present application will be specifically described below in combination with the accompanying drawings and application scenarios.

[0240] In a possible scenario, when a user uses a PC independently, he can run one or more applications installed on the PC at the same time, and each application can output corresponding audio. At the same time, the user can also open one or more web pages through a browser application of the PC, and the one or more web pages can include an interface capable of outputting audio.

[0241] It should be understood that in the embodiments of the present application, "using the PC independently" means that the applications currently used or run by the user are all local applications from the PC. Specifically, "using the PC independently" may include a scenario where the PC is not connected to any other electronic device, and the user uses the PC independently and runs one or more local applications of the PC. It may also include a scenario where the PC is connected to other electronic devices, and the user only runs local applications of the PC during the use of the PC. The embodiments of the present application are not limited to this.

[0242] Optionally, the "call request" in the embodiment of the present application may come from an instant messaging application. Optionally, the "instant messaging application" may include application, Applications, Changlian calling applications, and game applications with voice calling, video calling and other calling functions. When the instant messaging application receives a call request sent by other friends, an audio operation window will pop up in time to remind the user that the call request needs to be processed.

[0243] Alternatively, the "call request" may also come from a phone application, a short message application, or the like of the electronic device. For example, when the electronic device is a mobile phone, a tablet, or the like, a call made by another contact, a call initiated through a short message application, or the like may be received.

[0244] It should be understood that the source of the "call request" in the embodiment of the present application is not limited to the "instant messaging applications" listed above, as well as various other possible applications such as telephone applications and short message applications, and the embodiment of the present application does not limit this.

[0245] In the introduction of the subsequent embodiments, the voice call request, video call request, etc. of any of the above applications are collectively referred to as "call request".

[0246] In one possible implementation, if the PC receives a call request while the user is using the PC, in response to the call request, the PC can determine the number of windows that need to be automatically popped up and the applications to which the windows correspond based on one or more applications that can currently output audio and the application in focus on the display screen, and display them to the user.

[0247] Figure 5 This is a schematic diagram of an example of displaying an audio notification on a PC provided in an embodiment of the present application.

[0248] Assume that a user runs multiple applications installed on the PC at the same time while using the PC independently: a music application, a browser application, and an instant messaging application. For example, Figure 5 As shown in Figure (a), the music application and the browser application are running in the foreground, and the window 10 of the music application and the window 20 of the browser application can be displayed on the display screen of the PC. The window 10 of the music application displays the running interface of the music application, and the window 20 of the browser application displays the online playback webpage of song 1. The instant messaging application is running in the background, and the window of the instant messaging application running in the background may not be displayed on the display screen of the PC. The bottom navigation area of ​​the PC display screen can correspondingly display the application icon 10-1 corresponding to the window 10 of the music application and the application icon 20-1 corresponding to the window 20 of the browser application.

[0249] exist Figure 5 In the scenario shown in FIG. 1 (a), the window 20 of the browser application is displayed one level above the window 10 of the music application and is in focus. When a call request from an instant messaging application is received, in response to the call request received from the instant messaging application, the display screen of the PC may display the following Figure 5 The audio notification interface shown in FIG. (b) in FIG. 1 shows an automatically pop-up prompt window 30 and an audio operation window 10-2 of a music application. The prompt window 30 is used to remind the user that there is a call request that needs to be processed, and the audio operation window 10-2 of the music application may include one or more controls to facilitate the user to control the playback of song 2 through the audio operation window 10-2 of the music application.

[0250] Optionally, the display position and / or display size of the prompt window 30 and the audio operation window 10 - 2 of the music application may be adaptively adjusted according to the display position and / or display size of the window 20 of the browser application.

[0251] In a possible display mode, the prompt window 30 and the audio operation window 10 - 2 of the music application can be displayed around the window 20 of the browser application in the focus state.

[0252] Optionally, the prompt window 30 and the audio operation window 10-2 of the music application can be displayed at any border position of the focused browser application window 20. In this display mode, the prompt window 30 and the audio operation window 10-2 of the music application are displayed near the focused window, which can facilitate user operation.

[0253] Optionally, the prompt window 30 and the audio operation window 10-2 of the music application may be displayed in an area outside the frame of the window 20 of the browser application in the focus state, that is, the prompt window 30 and the audio operation window 10-2 of the music application may not overlap with the window 20 of the browser application in the focus state. This display method can prevent the prompt window 30 and the audio operation window 10-2 of the music application from blocking the window 20 of the browser application in the focus state, making it easier for users to view more interface content.

[0254] Alternatively, the prompt window 30 and part of the audio operation window 10-2 of the music application are displayed in the window 20 of the browser application in the focus state. In other words, the prompt window 30, the audio operation window 10-2 of the music application and the window 20 of the browser application in the focus state may have overlapping areas. This display method can provide a more eye-catching display effect for the user, making the prompt more obvious.

[0255] For example, Figure 5 As shown in Figure (b), if the window 20 of the browser application in the focused state is displayed in the lower right corner area of ​​the PC display screen, when a call request is received, in response to the call request, the PC can determine the prompt window 30 to be displayed, and at the same time detects that the music application is playing song 2 and the window 10 of the music application is in the non-focused state. Therefore, the PC can determine that the audio operation window associated with the non-focused music application also needs to be automatically popped up.

[0256] In this scenario, after the PC determines that the number of windows that need to be automatically popped up is two and determines the application associated with each of the two audio operation windows, the display position of each of the two audio operation windows can be determined according to the display position of the window 20 of the browser application in the focus state. For example, Figure 5As shown in Figure (b), the two pop-up audio operation windows can be displayed close to the upper border of the window 20 of the browser application in the focus state, and the two audio operation windows are arranged horizontally. The two audio operation windows have the same horizontal display width and the same vertical display height, and the sum of the horizontal display widths of the two audio operation windows is equal to the horizontal display width of the window 20 of the browser application.

[0257] In another possible display manner, the PC may determine the display position and / or display size of one or more audio operation windows according to the display content and display area on the display screen.

[0258] For example, Figure 5 As shown in FIG. 2( c ), assuming that the PC determines that the remaining display area on the left side is the largest in the display area outside the window 20 of the browser application in the focus state, the PC can determine to display the two automatically popped-up audio operation windows at the position of the left border of the window 20 of the browser application in the focus state.

[0259] Optionally, the two audio operation windows can be arranged vertically, and the lower border of the audio operation window 10-2 of the music application displayed at the bottom of the two audio operation windows can be kept flush with the lower border of the browser application window 20. The prompt window 30 and the audio operation window 10-2 of the music application have the same horizontal display width and different vertical display heights.

[0260] It should be understood that this display method can ensure that multiple windows on the display screen have a neat arrangement style, providing users with a better visual experience.

[0261] In yet another possible display mode, one or more audio operation windows that pop up automatically on the PC have a preset position and / or display size.

[0262] Optionally, the preset position may be the upper boundary of the display area of ​​the PC display and located at the center of the width direction of the display. Alternatively, the preset position may be the lower boundary of the display area of ​​the PC display and located at the center of the width direction of the display. Alternatively, the preset position may be a fixed position at the upper left corner of the PC display. Alternatively, the preset position may be a fixed position at the lower right corner of the PC display, etc., which is not limited in the embodiments of the present application.

[0263] For example, Figure 5 As shown in Figure (d), the prompt window 30 and the audio operation window 10-2 of the music application can be displayed in the top middle area of ​​the PC display screen, close to the upper boundary of the display area of ​​the display screen, and the prompt window 30 and the audio operation window 10-2 of the music application have the same horizontal display width and vertical display height.

[0264] It should be understood that when the number of audio operation windows that automatically pop up on the PC is three, the three windows can be arranged side by side and displayed in the top middle area of ​​the display screen. When the number of audio operation windows that automatically pop up on the PC is more, the number of rows of window arrangement can be further increased to continue to arrange and display more windows. For the sake of simplicity, examples will not be given here one by one.

[0265] Optionally, for different applications, the audio operation window corresponding to each application may have a preset display size, and the display size may include a "horizontal display width" and a "vertical display height". Exemplarily, the system determines the horizontal display width and / or vertical display height of the audio operation window of each application according to the type of controls to be displayed, the number of controls, etc. corresponding to each application.

[0266] It should be noted here that in the embodiments of the present application, display dimensions such as "horizontal display width", "vertical display height" or "height", "width" can be expressed in pixels (px), and the height of the pixel directly affects the resolution of the display screen. For example, assuming that the resolution of the PC display screen is 1680×1050, it can be understood that the number of pixels included in the horizontal display width of the display screen is fixed. Therefore, the size of any window displayed on the PC display screen, the size of the application icon, etc. can be expressed as the number of horizontal pixels × the number of vertical pixels, which will not be repeated later.

[0267] Optionally, display contents such as the types of controls and / or the number of controls included in the prompt window 30 and the audio operation window 10 - 2 of the music application may be preset.

[0268] For example, Figure 5 As shown in Figure (b), the prompt window 30 is used to prompt the user that there is a call request that needs to be processed. The prompt window 30 may include one or more controls such as the user's "Miss Ma" image, note name, telephone number, answer control, hang up control, etc., so that the user can view the source of the call request and click the answer control and hang up control to perform quick operations. The embodiment of the present application does not limit the display content of the prompt window 30.

[0269] For example, Figure 5As shown in Figure (b), the audio operation window 10-2 of the music application may include one or more controls such as a play control, a previous song control, a next song control, a favorite control, and a share control. It should be understood that when the user is preparing to answer a call, he or she generally wants to pause the playback of song 2 to avoid audio interference. Therefore, the controls included in the audio operation window 10-2 of the music application can facilitate the user to quickly perform the pause operation. Optionally, the display content such as the type of controls and / or the number of controls included in the prompt window 30 and the audio operation window 10-2 of the music application can be manually selected by the user.

[0270] For example, the developer can provide the user with a setting entry in the music application, through which the user can manually select the type and / or number of controls that are desired to be displayed in the window when the audio operation window 10-2 of the music application actively pops up on the PC. The embodiment of the present application does not limit the way in which the user manually sets the controls in each audio operation window.

[0271] When the PC displays Figure 5 When the audio notification interface is shown in Figure (b), (c), or (d), the user can control the audio playback through the prompt window 30 and the audio operation window 10-2 of the music application according to his or her needs.

[0272] For example, the user can click the play control in the audio operation window 10-2 of the music application to pause the process of the music application playing song 2; and can also click the play control 20-2 in the window 20 of the browser application in the focus state to pause the process of the browser application playing song 1 online. After pausing all audio playback of the PC, the user can click the answer control in the prompt window 30 to talk to "Miss Ma", and the call audio will not be disturbed by the PC playing songs, etc., which will not be described in detail in the embodiment of the present application.

[0273] In the above process, the PC can determine the display position of the prompt window 30 and the audio operation window 10-2 of the music application according to the display position of the window 20 of the browser application in the focus state. Considering that the user can directly click the play control 20-2 in the window 20 of the browser application in the focus state to control the audio playback, the PC may not display the audio operation window for the browser application in the focus state. Thereby reducing the number of audio operation windows displayed on the screen, saving the screen display area, and presenting a more streamlined display effect to the user.

[0274] It should be noted here that the embodiment of the present application does not limit the display duration of one or more audio operation windows that automatically pop up on the display screen of the PC.

[0275] Optionally, the display duration of each audio operation window may be a preset duration, for example, the window automatically disappears after being displayed for 5 seconds.

[0276] Alternatively, the display duration of each audio operation window may be a fixed duration set by the user, for example, the user sets the fixed duration to 3 seconds.

[0277] Alternatively, the display duration of each audio operation window may be related to the user's operation.

[0278] For example, when the PC detects that the user clicks the play control in the audio operation window 10-2 of the music application, Figure 5 The audio operation window 10-2 of the music application displayed on any audio notification interface shown in Figures (b), (c) and (d) can disappear automatically; when the PC does not detect the user's operation of any control in the audio operation window 10-2 of the music application, the audio operation window 10-2 of the music application can be displayed for a preset time (for example, 5 seconds) and then disappear automatically. This is not limited to the embodiments of the present application.

[0279] Through the above-introduced method for displaying audio notifications, users can play audio output by one or more applications through the PC in a scenario where the PC is used independently. When the PC receives a call request, in response to the call request, the PC can determine the one or more applications currently playing the audio, as well as the application window displayed in a focus state on the display screen, and automatically pop up the audio operation window of one or more applications, which can be displayed around the application window displayed in a focus state. Among them, each audio operation window can include one or more operation controls. Users can quickly control the playback of audio through the controls in the audio operation window, avoiding the need for users to perform cumbersome operations before pausing the playback of audio, allowing users to answer the call as soon as possible, and can ensure that the user's call process is not disturbed by other audio output by the PC, thereby improving the user's operating experience.

[0280] In addition, the method for displaying audio notifications provided in the embodiment of the present application can display all applications and / or web pages currently involved in audio output on the PC in the form of corresponding audio operation windows, in addition to when the PC receives a call request. When the user ends the call, the PC can also automatically pop up one or more audio operation windows to facilitate the user to re-control one or more applications and / or web pages to continue playing audio.

[0281] Figure 6 This is a schematic diagram of an example of an audio notification displayed on a PC after a call ends, provided in an embodiment of the present application.

[0282] For example, if the PC receives a call request from an instant messaging application, the following is displayed: Figure 5 In any of the audio notification interfaces shown in (b), (c) or (d) in FIG. 1 , the user clicks the play control in the audio operation window 10-2 of the music application to pause the playing of song 2; clicks the play control 20-2 in the window 20 of the browser application to pause the playing of song 1, and then clicks the answer control in the prompt window 30 to talk to "Miss Ma". After the user ends the call with "Miss Ma", the PC can further display Figure 6 The audio notification interface shown.

[0283] Optionally, in Figure 6 The audio notification interface shown shows an audio operation window 10 - 2 of the music application, and the audio operation window 10 - 2 of the music application can be displayed around the window 20 of the browser application in the focus state.

[0284] Exemplarily, the audio operation window 10-2 of the music application can be displayed at any border position of the window 20 of the browser application in the focus state, for example, Figure 5 The audio operation window 10-2 of the music application shown in Figure (b), (c) or (d) remains consistent, has the same display position and / or display size, and the embodiments of the present application are not limited to this.

[0285] In the above display mode, after the user ends the call, the audio operation window 10-2 of the music application pops up again and is displayed near the application window in the focus state, which can facilitate user operation and continue to control the music application to play song 2.

[0286] It should be noted here that users Figure 5 In the audio operation window 10-2 of the music application shown in (b), (c) or (d), the play control is clicked and the song 2 has been paused, so Figure 6 When the audio operation window 10-2 of the music application pops up again, the original playback control in the window is displayed in the "pause playback" state, which is referred to as the "pause control" in the embodiment of the present application. Similarly, when the user pauses the playback of song 1, the original playback control 20-2 is also displayed as the pause control 20-4 in the window 20 of the browser application in the focus state, and the original playback control 101 displayed in the window 10 of the music application is also changed to the pause control 102. The display form of the control is not described in detail in the subsequent embodiments.

[0287] Exemplarily, the user can click the pause control in the audio operation window 10-2 of the music application, and the PC continues to play song 2 of the music application; click the pause control 20-4 in the window 20 of the browser application, and the PC continues to play song 1 output by the web page of the browser application.

[0288] It should be understood that the above embodiment takes the example of a user pausing the audio by clicking the playback control included in a certain audio operation window to avoid multiple audios interfering with the user's voice call. In addition, each audio operation window may also include a "volume adjustment control", "mute control", etc., and the user may also switch the audio to silent playback through the "volume adjustment control", "mute control", etc. of the audio operation window. When the user ends the voice call, the corresponding audio operation window may pop up again to end the silent playback of the audio. The process may refer to the above embodiment, and the embodiment of the present application is not limited to this.

[0289] In the above implementation process, when the user ends the call, the PC can display other audio operation windows in addition to the call prompt window, and the user can quickly resume audio playback through other audio operation windows. For example, the user can click the pause control to resume the application and / or web page audio playback, and can quickly restore the PC to the audio playback state before receiving the call request, thereby realizing fast switching of audio playback, convenient operation, and improving the user's operating experience.

[0290] It should be noted that the application window in focus may change constantly when the user is using the PC. When the application window in focus changes, in response to the call request, the window automatically popped up by the PC, the window display content, and the application corresponding to the window may also change adaptively.

[0291] Figure 7 This is another schematic diagram of displaying an audio notification on a PC provided in an embodiment of the present application.

[0292] Exemplarily, the present application embodiment continues with Figure 5 Take the scenario described in Figure (a) of FIG. 1 as an example, assuming that the user runs multiple applications installed on the PC at the same time while using the PC independently: a music application, a browser application, and an instant messaging application. The music application and the browser application are running in the foreground, and the display screen displays the window 10 of the music application and the window 20 of the browser application, and the window 20 of the browser application is displayed in the focus state. The window of the instant messaging application running in the background is not displayed on the display screen.

[0293] For example, when the user executes Figure 7 In the operation shown in FIG. 1 (a), when any display area of ​​the window 10 of the music application is clicked, the display screen of the PC can be displayed as shown in FIG. Figure 7 As shown in Figure (b) of the figure. Figure 5 In Figure (a), Figure 7In Figure (b), the window 10 of the music application is switched to be displayed in the focus state (shown in the figure as a window with a thick black solid line border), and the window 20 of the browser application is switched to be displayed in the non-focus state. The window 10 of the music application is displayed at the upper level of the window 20 of the browser application.

[0294] exist Figure 7 In the scenario shown in FIG. 1 (b), if the PC receives a call request from an instant messaging application, in response to the received call request, the display screen of the PC may further display the following Figure 7 The interface shown in FIG. (c) in FIG. 1 automatically pops up a prompt window 30 and an audio operation window 20-3 of the browser application. The prompt window 30 is used to remind the user that there is a call request that needs to be processed, and the audio operation window 20-3 of the browser application may include one or more controls that can receive user operations, so that the user can control the playback of song 1 through the controls of the audio operation window 20-3 of the browser application.

[0295] In a possible display manner, the prompt window 30 and the audio operation window 20 - 3 of the browser application may be close to the window 10 of the music application displayed in the focus state.

[0296] Optionally, the prompt window 30 and the audio operation window 20 - 3 of the browser application can be displayed at any border position of the window 10 of the music application in the focus state to facilitate user operation.

[0297] Optionally, the prompt window 30 and the audio operation window 20 - 3 of the browser application may have an overlapping area with the window 10 of the music application in the focus state, or may be displayed in an area outside the window 10 of the music application without an overlapping area.

[0298] Optionally, the PC may adaptively adjust the display position and / or display size of the prompt window 30 and the audio operation window 20-3 of the browser application according to the display position and / or display size of the window 10 of the music application in focus.

[0299] It should be understood that the above multiple possible window display modes can refer to Figure 5 For the sake of brevity, the relevant descriptions in are not repeated here.

[0300] For example, Figure 7As shown in FIG. (c), the window 10 of the music application in the focus state is displayed in the upper left corner area of ​​the display screen and close to the border of the PC display screen. In response to the received call request, the PC can determine that the audio from the browser application is currently being played, and the window 20 of the browser application is in a non-focus state, and then it can be determined that the audio operation window 20-3 of the browser application needs to be displayed. In addition, it can also be determined that the prompt window 30 of the instant call request needs to be displayed to the user in a timely manner.

[0301] Optionally, the prompt window 30 and the audio operation window 20-3 of the browser application can be displayed close to the right border of the window 10 of the music application in the focus state, the prompt window 30 and the audio operation window 20-3 of the browser application can be arranged vertically up and down, and the prompt window 30 and the audio operation window 20-3 of the browser application can have the same horizontal display width and vertical display height.

[0302] Optionally, the prompt window 30 can also be Figure 5 Figure (b) and Figure (c) have the same display content, that is, they display the same type and the same number of controls.

[0303] Optionally, the audio operation window 20-3 of the browser application may be different from Figure 5 In the audio operation windows 10-2 of the music application shown in (b) and (c), the two audio operation windows may include different numbers and / or different types of controls, which is not limited in this embodiment of the present application.

[0304] For example, Figure 5 The audio operation window 10-2 of the music application shown in (b) and (c) may include a share control, a previous song control, a play control, a next song control, and a favorite control. In contrast, Figure 7 The audio operation window 20-3 of the browser application shown in Figure (c) may include a sharing control, a previous song control, a play control, a next song control, a download control, a volume adjustment control, and the like.

[0305] It should be understood that even if the current PC's browser application and music application both output music audio, when a user plays song 1 online through a browser's web page, the user generally does not need to log in to a personal account to achieve online playback of the song. Therefore, the "Collection Control" may not be included in the audio operation window 20-3 of the browser application, and the "Download Control" may be displayed in the audio operation window 20-3 of the browser application, so that the user can perform operations such as downloading songs according to their needs.

[0306] When the PC displays Figure 7In the interface shown in FIG. 7 (c), the user can quickly control the playback of each audio according to his or her needs through the prompt window 30 and the audio operation window 20-3 of the browser application.

[0307] For example, the user can click the play control in the audio operation window 20-3 of the browser application, and the PC pauses playing song 1; click the play control 101 in the music application window 10, and the PC pauses playing song 2. The user can then click the answer control in the prompt window 30 to talk to "Miss Ma". In this way, the call audio will not be disturbed by the song played by the PC. Please refer to the previous text for related descriptions, which will not be repeated here for simplicity.

[0308] Through the above Figure 5 and Figure 7 From the comparison, it can be seen that in the method for displaying audio notifications provided in the embodiment of the present application, after receiving a call request, the PC can detect the application window in focus and the application window in non-focus on the display screen, and aggregate and display the audio operation window and the prompt window of the call request around the application window in focus. In other words, when the application window in focus and / or the application window in non-focus on the PC display screen are different, the number, type, display position, and display size of the audio operation windows that pop up automatically will also be different. Similarly, when the user ends a call, the PC can automatically pop up one or more audio operation windows, making it convenient for the user to quickly control the application and / or web page to continue playing audio.

[0309] Figure 8 This is another schematic diagram of an example of an audio notification displayed on a PC after a call ends, provided in an embodiment of the present application.

[0310] For example, if the PC receives a call request from an instant messaging application, the following is displayed: Figure 7 In the audio notification interface shown in FIG. (c), the user clicks the play control in the audio operation window 20-3 of the browser application to pause the playing of song 1; clicks the play control 101 in the music application window 10, and the PC pauses the playing of song 2. Then clicks the answer control in the prompt window 30 to talk to "Miss Ma". After the user ends the call with "Miss Ma", the PC can further display Figure 8 The audio notification interface shown.

[0311] Optionally, in Figure 8 The audio notification interface shown shows: an audio operation window 20 - 3 of a browser application that pops up automatically on the PC, and the audio operation window 20 - 3 of the browser application can be displayed around the window 20 of the browser application in the focus state.

[0312] Exemplarily, the audio operation window 20-3 of the browser application can be displayed at the upper border position of the window 20 of the browser application in the focus state, compared to Figure 7 As shown in Figure (c), the display position of the audio operation window 20-3 of the browser application can be moved upward to keep it flush with the long border of the window 10 of the music application in the focus state, and the display size remains consistent, which is not limited in this embodiment of the present application.

[0313] Exemplarily, the user can click the pause control of the audio operation window 20-3 of the browser application, and the PC continues to play song 1 output from the web page of the browser application; click the pause control 102 in the window 10 of the music application in focus, and the PC continues to play song 2 of the music application.

[0314] pass Figure 6 and Figure 8 In contrast, when the application window in focus on the display changes, the audio operation window that pops up automatically on the PC will also change after the user ends the call. For example, the PC can re-detect the application window in focus on the current PC display and aggregate and display the audio operation window around the application window in focus. The specific display method can be found in the above description and will not be repeated here.

[0315] In another possible scenario, the user may open multiple web pages capable of outputting audio through a browser application installed on a PC. The multiple web pages may be displayed as tabs in a window of a browser application, or the output web page may be displayed as an independent sub-window.

[0316] When a user opens multiple web pages capable of outputting audio through a browser application, the embodiment of the present application also provides a corresponding method for displaying audio notifications. The PC can detect and identify the multiple web pages capable of outputting audio corresponding to the browser application. If a call request is received, the PC can determine the number of audio operation windows that need to pop up automatically, the corresponding application source, etc. based on the application window in focus and / or the web page in focus.

[0317] It should be noted here that for a browser application, when multiple web pages are displayed in the form of tabs in a window of a browser application, the "web page in focus" can be understood as the window of the browser application being in focus, and the web page is displayed at the top level in the window of the browser application, and other web pages are displayed at non-top levels in the window of the browser application. When multiple web pages are displayed in the form of independent sub-windows, the "web page in focus" can be understood as the sub-window of the browser application corresponding to the web page is in focus. This scenario can refer to the window in focus described in the aforementioned embodiment, and will not be described in detail in subsequent embodiments.

[0318] Fig. 9 This is another schematic diagram of displaying an audio notification on a PC provided in an embodiment of the present application.

[0319] For example, Fig. 9 As shown in Figure (a), it is assumed that the user runs multiple applications installed on the PC at the same time while using the PC independently: music application, browser application, and instant messaging application. Among them, the music application and the browser application are running in the foreground, and the display screen displays the window 10 of the music application and the window 20 of the browser application, and the window 20 of the browser application is displayed in the focus state. The browser application opens the online playback webpage of song 1 and the FM (frequency modulation, FM) radio (FM frequency is XX Hz) online playback webpage, and both webpages are displayed in the window 20 of the browser application in the form of tabs. Among them, the online playback webpage of song 1 is displayed in the browser application window in the focus state, and the FM radio online playback webpage is displayed in the browser application window in the non-focus state. The window of the instant messaging application running in the background is not displayed on the display screen.

[0320] At this time, if a call request is received, in response to the call request, the display screen of the PC may display the following Fig. 9 The audio notification interface shown in Figure (b) is shown in Figure 1. Fig. 9 As shown in Figure (b), the audio notification interface displays: an automatically pop-up prompt window 30, an audio operation window 10-2 of the music application, and an audio operation window 20-5 of the browser application FM radio web page. The audio operation window 20-5 of the browser application FM radio web page is used for users to control the online playback of FM radio. For example, the audio operation window 20-5 of the browser application FM radio web page can display a pause control, and the user can click the pause control to pause the audio of the FM radio. The relevant description of the prompt window 30 and the audio operation window 10-2 of the music application can be referred to above and will not be repeated here. In other words, when the browser application opens multiple web pages that can output audio, the PC can detect whether the above web pages are in focus in the window of the browser application, and display the audio operation window corresponding to the non-focus web page on the display screen.

[0321] Optionally, the display positions of the prompt window 30, the audio operation window 10-2 of the music application, and the audio operation window 20-5 of the browser application FM radio web page can refer to Figure 5 , Figure 7 The relevant descriptions in the aforementioned embodiments will not be repeated here.

[0322] Similarly, when the user ends the call, the PC can further display Fig.10 The audio notification interface shown.

[0323] Fig.10 This is another example of a schematic diagram of displaying an audio notification on a PC after a call is ended provided by an embodiment of the present application. When the user ends the call, the PC can automatically pop up an audio operation window 10-2 of a music application and an audio operation window 20-5 of an FM radio webpage of a browser application.

[0324] For example, the user can click the pause control in the audio operation window 10-2 of the music application, and the PC continues to play song 2 of the music application; click the play control in the audio operation window 20-5 of the browser application FM radio webpage, and the PC continues to play the FM radio; click the pause control 20-4 in the window 20 of the browser application in the focused state, and the PC's browser application continues to play song 1.

[0325] From this, it can be seen that when the browser application on the PC outputs audio through multiple web pages respectively, the PC can also detect the source of each audio. After the user ends the call, the audio operation window that automatically pops up on the PC may include the audio operation window corresponding to the non-focus application and / or web page. The PC may not display the audio operation window corresponding to the focus application and / or web page, which is convenient for the user to quickly restore one or more applications and / or web pages to continue playing audio, simplifying the operation process and improving the user's operating experience.

[0326] It should be noted that when a user is using a PC, the user's operations may cause the multiple web pages included in the browser application in focus to change continuously. When the web page in focus displayed in the window of the browser application changes, the window automatically popped up by the PC, the window display content, and the application corresponding to the window may also change adaptively.

[0327] Fig.11 This is a schematic diagram of another example of displaying an audio notification on a PC provided in an embodiment of the present application.

[0328] For example, suppose that a user runs multiple applications installed on the PC at the same time while using the PC independently: a music application, a browser application, and an instant messaging application. The music application and the browser application are running in the foreground, and the display screen displays a window 10 of the music application and a window 20 of the browser application, and the window 20 of the browser application is in focus. The online playback webpage of song 1 is displayed in the browser application window in a focused state, and the FM radio online playback webpage is displayed in the browser application window in a non-focused state. The instant messaging application is running in the background, and the window of the instant messaging application is not displayed on the display screen. When the user executes the following command: Fig.11 In the operation shown in FIG. (a), when the tab corresponding to the FM radio online playback webpage in the window 20 of the browser application is clicked, in response to the user operation, such as Fig.11As shown in FIG. 2( b ), the web page displayed in the window 20 of the browser application is switched from the online play web page of song 1 to the online play web page of FM radio. As a result, the online play web page of FM radio is switched to the focus state, and the online play web page of song 1 is switched to the non-focus state.

[0329] Optionally, the "shadow state of the tab area" indicates that the web page corresponding to the tab is in focus, such as Fig.11 As shown in FIG. 2( b ), the tab area corresponding to the FM radio online broadcasting webpage in the tab area at the top of the window 20 of the browser application may be displayed as a shadow.

[0330] Optionally, the FM radio online play webpage in the window of the browser application can be as follows: Fig.11 As shown in Figure (b), one or more controls such as playback controls, HD version listening controls, sharing controls, and FM radio station lists and other content related to the FM radio playback process are displayed, which is not limited to this embodiment of the present application.

[0331] For example, when the PC receives a call request, in response to the call request, the display screen of the PC may display the following: Fig.11 The audio notification interface shown in FIG. (c) of FIG. 1 shows an automatically pop-up window on the audio notification interface: a prompt window 30, an audio operation window 10-2 of the music application, and an audio operation window 20-3 of the browser application. The user can quickly control the playback of each audio according to his or her needs through the controls in the audio operation window 10-2 of the music application, the audio operation window 20-3 of the browser application, and the window 20 of the browser application in the focus state.

[0332] For example, the user can click the play control in the audio operation window 10-2 of the music application to pause the PC playing song 2; click the play control in the audio operation window 20-3 of the browser application to pause the PC playing song 1; click the play control 20-6 of the FM radio online playback in the window 20 of the browser application to pause the PC playing FM radio. The user can also click the answer control in the prompt window 30 to talk to "Miss Ma", so that the call audio will not be disturbed by the songs played by the PC. For other related descriptions, please refer to the previous text, which will not be repeated here for simplicity.

[0333] Exemplarily, the audio operation window 10-2 of the music application and the audio operation window 20-3 of the browser application are arranged up and down, the prompt window 30 and the two audio operation windows can have the same horizontal display width, and the sum of the vertical display heights of the two audio operation windows is equal to the vertical display height of the prompt window 30. This display mode is convenient for user operation, and can make the automatically popped-up prompt window 30 and the two audio operation windows have a concise and neat arrangement, providing users with a more beautiful visual effect. For other related descriptions, please refer to the previous text, which will not be repeated here.

[0334] Similarly, when the user ends the call, the PC can further display Fig.12 The audio notification interface shown.

[0335] Fig.12 FIG. 1 is a schematic diagram of another example of an audio notification displayed on a PC after a call ends provided by an embodiment of the present application. Fig.12 The audio notification interface shown shows: the audio operation window 10-2 of the music application and the audio operation window 20-3 of the browser application that are automatically popped up by the PC, and the audio operation window 10-2 of the music application and the audio operation window 20-3 of the browser application are displayed around the window 20 of the browser application in the focus state.

[0336] For example, Fig.12 As shown, the audio operation window 10-2 of the music application and the audio operation window 20-3 of the browser application are displayed at the upper border position of the window 20 of the browser application in the focus state, and the display position and display size are compared with Fig.11 Figure (c) can remain unchanged and still be arranged up and down, and this embodiment of the present application is not limited to this.

[0337] In the above display method, before and after the user's voice call, the audio operation window maintains the same display position and / or display size, which is more in line with the user's usage habits and will not cause major visual changes to the user, thereby bringing the user a better usage and operation experience.

[0338] For example, the user can click the pause control of the audio operation window 10-2 of the music application, and the PC continues to play song 2 of the music application; click the pause control of the audio operation window 20-3 of the browser application, and the PC continues to play song 1 of the music application; click the pause control 20-7 of the online FM radio web page interface in the window 20 of the browser application in the focused state, and the PC continues to play the FM radio.

[0339] From this, it can be seen that when the web page in focus displayed in the browser application is different, the audio operation window that automatically pops up on the PC after the user ends the call will also be different.

[0340] In summary, through the above Figures 5 to 12 The embodiments and methods for displaying audio notifications are introduced, and users can play audio from multiple applications and / or web pages simultaneously when using a PC. When a call request is received, the PC can aggregate and display the audio operation windows corresponding to the non-focused applications and / or web pages on the display screen according to the application windows and / or web pages displayed in the focus state, so that users can quickly control the playback of each audio through the audio operation window. For example, the user can pause the audio playback through the audio operation window and then answer the call, ensuring that the call process will not be disturbed by the audio output of other applications, avoiding the need for users to perform cumbersome operations to pause the audio playback, simplifying the operation process, and improving the user's operation experience.

[0341] In addition, after the user ends the call, other audio operation windows in addition to the call request prompt window may automatically pop up on the PC display screen. The user can quickly resume audio playback through other audio operation windows. For example, the user can click the pause control to resume the application and / or web page audio playback, and can quickly restore the PC to the audio playback state before receiving the call request, thereby realizing fast switching of audio playback, convenient operation, and improving the user's operating experience.

[0342] It should be noted that in Figures 5 to 12 In the embodiments introduced in, other audio operation windows other than the call prompt window are displayed after the user ends the call as an example for explanation, wherein the applications, number of windows, display positions, display sizes, etc. corresponding to the "other audio operation windows" may be consistent with the audio operation window displayed when the PC receives a call request, or, the applications, number of windows, display positions, display sizes, etc. corresponding to the "other audio operation windows" may be different from the audio operation window displayed when the PC receives a call request, and the embodiments of the present application are not limited to this.

[0343] For example, the aforementioned Figures 5 to 12 In the embodiment described in , in response to a call request, the PC can detect that the applications that are outputting audio include application A, application B, and application C, and then can display the audio operation windows and call prompt window of application A, application B, and application C. If the user does not close or change the application running status of the PC during the call, the PC also displays the audio operation windows of application A, application B, and application C after the call ends.

[0344] If the user closes application A while answering a call, runs application D, and application D is also an application that can output audio, accordingly, after the call ends, the PC can display the audio operation windows of application B, application C, and application D. Different audio display methods fall within the scope of protection of this application, and the subsequent embodiments will not be repeated. In another possible scenario, the user can use a mobile phone and a PC for multi-screen collaboration. During the multi-screen collaboration process between a mobile phone and a PC, the user can run one or more applications installed on the mobile phone side on the PC, and one or more mobile phone windows will be displayed on the display screen of the PC. In addition, the user can also run one or more applications installed locally on the PC at the same time, or open one or more web page interfaces through the browser application of the PC. In the scenario of multi-screen collaboration between a mobile phone and a PC, the PC can output multiple audios at the same time, and the multiple audios may include audio from applications on the mobile phone side, and may also include audio from applications and / or web page interfaces on the PC side.

[0345] Fig.13 This is a schematic diagram of an example of displaying an audio notification on a PC during a multi-screen collaboration process provided by an embodiment of the present application.

[0346] For example, Fig.13 Figure (a) shows a scenario of multi-screen collaboration between a mobile phone and a PC. In unlocked mode, the mobile phone can display the main interface shown in the figure. The PC display screen displays the main window 40 of multi-screen collaboration, the window 10 of the music application, and the window 20 of the browser application. The main window 40 of the multi-screen collaboration is displayed in the focus state, and the window 20 of the browser application is displayed at the upper level of the window 10 of the music application. The display content of the main window 40 of the multi-screen collaboration can be consistent with the display content of the main interface of the mobile phone. For simplicity, it is not repeated here.

[0347] Correspondingly, the application icon 10-1 corresponding to the window 10 of the music application, the application icon 20-1 corresponding to the window 20 of the browser application, and the icon 40-1 corresponding to the main window 40 of the multi-screen collaboration can be displayed in the bottom navigation area of ​​the PC display screen.

[0348] Optionally, any one of the music application and the browser application may be an application installed on the mobile phone side or a local application of the PC, which is not limited in the embodiments of the present application.

[0349] In one possible scenario, Fig.13 In Figure (a), it is assumed that the music application and the browser application are both local applications of the PC.

[0350] For example, the user executes Fig.13As shown in FIG. (a), click the minimize control in the control area of ​​the main window 40 of the multi-screen collaboration, and in response to the user's operation, the PC displays Fig.13 In the interface shown in Figure (b), the main window 40 of the multi-screen collaboration runs in the background and is not displayed on the PC's display screen. The window 10 of the music application and the window 20 of the browser application are displayed in the foreground of the interface, and the window 20 of the browser application is displayed in focus. The application window displayed in focus is represented by a thick black solid line border in the figure.

[0351] It should be understood that in the process of multi-screen collaboration between PC and mobile phone, the instant messaging application can be a local application installed on the PC or an application installed on the mobile phone. Users can use the application on the mobile phone side on the PC, and this embodiment of the application does not limit this.

[0352] In another possible scenario, assuming that the instant messaging application is an application installed on the mobile phone, when the instant messaging application on the mobile phone receives a call request from the user "Miss Ma", in response to the call request, the mobile phone may display the following Fig.13 Optionally, the call prompt interface may include one or more contents such as an image of the user "Miss Ma", a remark name, a hang-up control 202-1 and an answer control 202-2, which is not limited in this embodiment of the present application.

[0353] In response to the call request, the display screen of the PC may display the following Fig.13 The audio notification interface shown in Figure (c) in FIG. 1 shows: an automatically pop-up prompt window 30 and an audio operation window 10-2 of the music application.

[0354] In one possible display mode, when there is an application window in a focused state on the PC display screen, the automatically popped-up prompt window 30 and the audio operation window 10-2 of the music application can be displayed around the window 20 of the browser application in the focused state.

[0355] Optionally, the prompt window 30 and the audio operation window 10 - 2 of the music application can be displayed at any border of the window 20 of the browser application in the focus state to facilitate user operation.

[0356] Optionally, the prompt window 30 and the audio operation window 10-2 of the music application may have an overlapping area with the window 20 of the browser application in the focus state, or may be displayed in an area outside the window 20 of the browser application without an overlapping area.

[0357] Optionally, the PC may adaptively adjust the display position and / or display size of the prompt window 30 and the audio operation window 10-2 of the music application according to the display position and / or display size of the window 20 of the browser application in the focus state.

[0358] It should be understood that the above various possible window display modes can refer to the relevant description of the aforementioned embodiments, and for the sake of simplicity, they are not repeated here.

[0359] For example, Fig.13 As shown in Figure (c), the prompt window 30 and the audio operation window 10-2 of the music application can be displayed vertically in the right border of the window 20 of the browser application. The prompt window 30 and the audio operation window 10-2 of the music application have the same horizontal display width and vertical display height, which is not limited in this embodiment of the present application.

[0360] It should be understood that Fig.13 In the scenario shown in Figure (c), when the PC receives a call request, in response to the call request, the PC can first detect the application window in focus to determine whether the main window of multi-screen collaboration is displayed on the PC display. When the main window of multi-screen collaboration is not displayed on the PC display, a prompt window of the call request and an audio operation window corresponding to the non-focus application are automatically popped up, and they are aggregated and displayed around the application window in focus for user operation.

[0361] When the PC displays Fig.13 When the audio notification interface shown in Figure (c) is displayed, the user can quickly control the audio playback through the audio operation window 10-2 of the music application according to their needs.

[0362] For example, the user can click the play control in the audio operation window 10-2 of the music application, and the PC pauses playing song 2; click the play control 20-2 in the window 20 of the browser application in the focus state, and the PC pauses playing song 1. The user can then click the answer control in the prompt window 30 to talk to "Miss Ma", so that the call audio will not be disturbed by the song played by the PC. Please refer to the previous text for related descriptions, which will not be repeated here for simplicity.

[0363] Fig.14 This is another schematic diagram of displaying an audio notification on a PC during a multi-screen collaboration process provided by an embodiment of the present application.

[0364] For example, Fig.14 Figure (a) and Fig.13 Figure (b) in the figure is a scenario of multi-screen collaboration between a mobile phone and a PC. Assume that before the PC receives a call request, the user performs the following operations: Fig.14The operation shown in FIG. 1 (a) is to click the minimize control in the control area of ​​the window 20 of the browser application in the focus state, and in response to the user's operation, the PC displays Fig.14 In the interface shown in FIG. (b), only the window 10 of the music application is displayed on the interface, and the window 10 of the music application is in the focus state. The main window 40 of the multi-screen collaboration and the window 20 of the browser application are in the background running state.

[0365] When the instant messaging application on the mobile phone receives a call request from the user "Miss Ma", in response to the call request, the mobile phone may display the following Fig.14 In response to the call request, the display screen of the PC may display the following: Fig.14 The audio notification interface shown in FIG. 7 (c) automatically pops up an audio operation window of a non-focus application, such as a prompt window 30 and an audio operation window 20-3 of a browser application.

[0366] In one possible display method, the audio operation window of the non-focus application that pops up automatically may have a preset display strategy. For example, the automatically popped-up prompt window 30 and the audio operation window 20-3 of the browser application may be displayed at a preset position and / or have a preset display size.

[0367] For example, Fig.14 As shown in Figure (c), the prompt window 30 and the audio operation window 20-3 of the browser application can be displayed in an upper and lower arrangement in the lower right corner display area of ​​the PC display screen, and the prompt window 30 and the audio operation window 20-3 of the browser application have the same horizontal display width and different vertical display heights, which is not limited in this embodiment of the present application.

[0368] Optionally, the prompt window 30 and the audio operation window 20-3 of the browser application may also be displayed with reference to the aforementioned multiple embodiments and the possible multiple display modes described in the accompanying drawings. For example, the prompt window 30 and the audio operation window 20-3 of the browser application are arranged horizontally and displayed in the middle position of the bottom navigation area of ​​the PC display screen, or the prompt window 30 and the audio operation window 20-3 of the browser application are displayed around the window 10 of the music application in the focus state, etc. Please refer to the aforementioned introduction. For the sake of simplicity, they are not listed one by one here.

[0369] It should also be understood that, in contrast Fig.13 Figure (c) and Fig.14As shown in Figure (c), in a multi-screen collaboration scenario between a mobile phone and a PC, when the application windows in focus on the PC display are different, if the PC receives a call request, in response to the call request, the window that automatically pops up on the PC, the window display content, and the application corresponding to the window may also change adaptively.

[0370] When the PC displays Fig.14 In the audio notification interface shown in Figure (c), the user can quickly control the playback of each audio according to their needs through the audio operation window 10-2 of the music application and the audio operation window 20-3 of the browser application.

[0371] For example, the user can click the play control in the audio operation window 20-3 of the browser application, and the PC pauses playing song 1; click the play control 101 in the music application window 10, and the PC pauses playing song 2. The user can then click the answer control in the prompt window 30 to talk to "Miss Ma". In this way, the call audio will not be disturbed by the song played by the PC. Please refer to the previous text for related descriptions, which will not be repeated here for simplicity.

[0372] When the user ends a call, the PC may also automatically pop up one or more audio operation windows, allowing the user to quickly resume the application and / or web page to continue playing the audio.

[0373] Fig.15 This is a schematic diagram of an example of an audio notification displayed on a PC after a call ends, provided in an embodiment of the present application.

[0374] For example, if the PC receives a call request from an instant messaging application, the following is displayed: Fig.14 In the audio notification interface shown in FIG. (c) of FIG. 1 , the user can click the play control of the audio operation window 20-3 of the browser application displayed on the audio notification interface, and the PC pauses playing song 1; click the play control 101 in the music application window 10, and the PC pauses playing song 2. Click the answer control of the prompt window 30 to talk to "Miss Ma". After the user ends the call with "Miss Ma", the PC can further display Fig.15 The audio notification interface shown.

[0375] Optionally, in Fig.15 The audio notification interface shown shows: an audio operation window 20-3 of the browser application that pops up automatically, and the audio operation window 20-3 of the browser application can be displayed at a preset position such as the lower right corner of the display screen, which is not limited in the embodiment of the present application.

[0376] Exemplarily, the user can click the pause control of the audio operation window 20-3 of the browser application, and the PC continues to play the song 1 output from the online web page.

[0377] It should be understood that the above-mentioned display method can be consistent with the display strategy of the automatic pop-up window when the PC receives a call request, following the preset position and / or display size. This display method is more in line with the user's usage habits and will not cause a large visual change to the user, thereby bringing the user a better visual experience and operation experience.

[0378] In another possible scenario, during the multi-screen collaboration process between a mobile phone and a PC, the main window of the multi-screen collaboration may be displayed on the PC display. Assuming that the main window of the multi-screen collaboration is displayed in the foreground and is in focus, the different sources corresponding to the audio played by the PC, for example, the multiple audios output by the PC may include audio from applications on the mobile phone side, and may also include audio from applications and / or web pages on the PC side. When the PC receives a call request, in response to the call request, one or more audio operation windows may automatically pop up on the PC display, and one or more audio operation windows may correspond to different display modes.

[0379] Fig.16 This is another schematic diagram of displaying an audio notification on a PC during a multi-screen collaboration process provided by an embodiment of the present application.

[0380] For example, Fig.16 Figure (a) shows a scenario of multi-screen collaboration between a mobile phone and a PC. The mobile phone displays the main interface in unlocked mode, and the PC display screen displays: a main window 40 of multi-screen collaboration, a window 10 of a music application, and a window 20 of a browser application in focus. The display content of the main window 40 of multi-screen collaboration can be consistent with the display content of the main interface of the mobile phone, and for simplicity, it is not repeated here.

[0381] For example, the user executes Fig.16 Operation 1 shown in FIG. (a) of FIG. 1 is to click the minimize control in the control area of ​​the window 20 of the browser application in the focus state, and then execute the following Fig.16 In operation 2 shown in FIG. (a), the minimize control in the control area of ​​the window 10 of the music application is clicked. In response to the user's operation, the display screen of the PC may further display the following Fig.16 In the interface shown in Figure (b), only the main window 40 of the multi-screen collaboration is displayed, and the main window 40 of the multi-screen collaboration is displayed in the focus state, as shown by a thick black solid line border in the figure.

[0382] It should be understood that in this scenario, the window 10 of the music application and the window 20 of the browser application on the PC display are both in the background running state and can continue to play the corresponding audio.

[0383] It should also be understood that the music application, browser application, and instant messaging application running in the background can be an application on the mobile phone side or a local application installed on the PC. When the music application, browser application, instant messaging application, etc. have different sources, if the PC receives a call request, in response to the call request, the audio operation window that automatically pops up on the PC display screen can correspond to different display modes.

[0384] The following will introduce different scenarios:

[0385] Scenario 1: Music applications, browser applications, and instant messaging applications are all installed on the PC

[0386] In a possible scenario, when the music application, browser application, and instant messaging application are all local applications installed on the PC, the process can refer to the aforementioned Figures 5 to 12 In the process introduced, the main window 40 of the multi-screen collaboration is used as the application window in the focus state, and the number of audio operation windows that need to pop up automatically, the display content, display position, display size, etc. of each audio operation window are determined based on the main window 40 of the multi-screen collaboration in the focus state.

[0387] For example, Fig.16 In the scenario shown in Figure (b), when the PC receives a call request, in response to the call request, the PC display screen may display Fig.16 The audio notification interface shown in Figure (c) in FIG. Exemplarily, the audio notification interface displays: an automatically pop-up prompt window 30, an audio operation window 20-3 of the browser application, and an audio operation window 10-2 of the music application. Among them, the prompt window 30, the audio operation window 10-2 of the music application, and the audio operation window 20-3 of the browser application are displayed around the main window 40 of the multi-screen collaboration in the focus state. For example, the prompt window 30, the audio operation window 10-2 of the music application, and the audio operation window 20-3 of the browser application can be displayed in an up-and-down arrangement at the left border position of the main window 40 of the multi-screen collaboration, and the upper border of the prompt window 30 located at the top of the three audio operation windows arranged up and down is flush with the upper border of the main window 40 of the multi-screen collaboration, and the prompt window 30, the audio operation window 10-2 of the music application, and the audio operation window 20-3 of the browser application have equal horizontal display width and vertical display height, which is not limited in the embodiment of the present application.

[0388] Scenario 2: Music apps and browser apps are installed on the PC, and instant messaging apps are installed on the phone

[0389] In a possible scenario, when the music application and the browser application are local applications installed on the PC, and the instant messaging application is a mobile application, the process can also refer to the above Figures 5 to 12 In the process introduced, the main window 40 of the multi-screen collaboration is used as the application window in the focus state, and the display strategies such as the number of audio operation windows that need to pop up automatically, the display content, display position, and display size of each audio operation window are determined according to the main window 40 of the multi-screen collaboration in the focus state.

[0390] For example, Fig.16 In the scenario shown in Figure (b), when the PC receives a call request, in response to the call request, the PC display screen may display Fig.16 The audio notification interface shown in Figure (c) is described above and will not be repeated here.

[0391] In another possible display mode, since the instant messaging application is an application installed on the mobile phone side, when the instant messaging application on the mobile phone side receives a call request, the display interface in the main window 40 of the multi-screen collaboration can be consistent with the display interface of the mobile phone. In other words, the main window 40 of the multi-screen collaboration can replace the prompt window 30 in the aforementioned embodiment to remind the user that there is a call request that needs to be processed.

[0392] For example, Fig.16 In the scenario shown in Figure (b), when the PC receives a call request, in response to the call request, the PC display screen may display Fig.16 The audio notification interface shown in Figure (d) in FIG. Optionally, the audio notification interface displays: an audio operation window 10-2 of the music application and an audio operation window 20-3 of the browser application that are automatically popped up by the PC, and the audio operation window 10-2 of the music application and the audio operation window 20-3 of the browser application can be displayed in an up-and-down arrangement at the left border position of the main window 40 of the multi-screen collaboration in the focus state, and the upper border of the audio operation window 20-3 of the browser application located above the two audio operation windows arranged up and down is kept flush with the upper border of the main window 40 of the multi-screen collaboration, and the audio operation window 10-2 of the music application and the audio operation window 20-3 of the browser application have equal horizontal display width and vertical display height.

[0393] In addition, in response to the call request, a call prompt interface is displayed in the main window 40 of the multi-screen collaboration on the PC display screen. The content included in the call prompt interface and the call prompt interface displayed on the mobile phone can be consistent. For example, one or more contents such as the image of the user "Miss Ma", the remark name, the hang-up control 202-1 and the answering control 202-2 can be displayed. The embodiment of the present application is not limited to this.

[0394] Scenario 3: Browser applications are installed on the PC, while music applications and instant messaging applications are installed on the mobile phone.

[0395] In one possible scenario, when browser applications are local applications installed on the PC, and music applications and instant messaging applications are mobile phone applications, when the PC receives a call request, the audio operation window corresponding to the mobile phone application that pops up on the PC can be displayed in the main window 40 of the multi-screen collaboration, and the audio operation windows corresponding to the browser applications installed on the PC can be displayed around the main window 40 of the multi-screen collaboration in the focus state.

[0396] For example, Fig.16 In the scenario shown in Figure (b), when the PC receives a call request, in response to the received call request, the PC display screen may display Fig.16 Optionally, the PC automatically pops up the audio operation window 20-3 of the browser application, and the audio operation window 20-3 of the browser application can be displayed at the bottom border position of the multi-screen collaboration main window 40 in the focus state, and has the same horizontal display width as the multi-screen collaboration main window 40.

[0397] In addition, in response to the call request, the audio operation window 10-2 and the prompt window 30 of the music application are displayed in the main window 40 of the multi-screen collaboration of the PC display screen, and the prompt window 30 and the audio operation window 10-2 of the music application have equal horizontal display widths, which is not limited to this in the embodiment of the present application.

[0398] In the above-mentioned display method, during the multi-screen collaboration process between the mobile phone and the PC, the applications corresponding to the multiple audios output by the PC may include applications on the mobile phone side, and may also include local applications on the PC. When the main window 40 of the multi-screen collaboration in the focus state is displayed on the PC display, if a call request is received, the PC can display the audio operation window corresponding to the application on the mobile phone side in the multi-screen collaboration window 40, and display the audio operation window corresponding to the local application of the PC in the display area outside the multi-screen collaboration window 40. The display method of this audio notification can remind the user that there is a call request that needs to be processed, and avoids the problem of repeated display on the PC display screen. It can reduce the number of audio operation windows displayed on the screen, save screen display area, present a more streamlined display effect to the user, and improve the user's visual experience.

[0399] Scenario 4: Browser applications, music applications, and instant messaging applications are all mobile applications

[0400] In one possible scenario, when the browser application, music application, and instant messaging application are all mobile-side applications, when the PC receives a call request, all audio operation windows corresponding to the mobile-side applications that pop up on the PC can be displayed in the main window 40 of the multi-screen collaboration.

[0401] For example, Fig.16 In the scenario shown in Figure (b), when the browser application, music application, and instant messaging application are all mobile-side applications, when the PC receives a call request, in response to the call request, the PC display can display Fig.16 Optionally, the audio operation window 10-3 of the music application, the voice call prompt window 30-4 and the audio operation window 20-3 of the browser application that pop up automatically can be displayed in the main window 40 of the multi-screen collaboration.

[0402] Optionally, the audio operation window 10-3 of the music application, the voice call prompt window 30-4 and the audio operation window 20-3 of the browser application can be displayed in the main window 40 of multi-screen collaboration in an up-down arrangement, and the audio operation window 10-3 of the music application, the voice call prompt window 30-4 and the audio operation window 20-3 of the browser application can have equal horizontal display widths.

[0403] Optionally, the audio operation window 10-3 of the music application, the voice call prompt window 30-4 and the audio operation window 20-3 of the browser application may have a different display style and / or different display content than on a PC display screen.

[0404] It should be understood that the main window 40 of the multi-screen collaboration may generally be displayed as a vertically long rectangular frame style, and the aspect ratio is consistent with the screen size of the mobile phone. For example, the horizontal display width of the main window 40 of the multi-screen collaboration is generally smaller than the vertical display height. Therefore, when the audio operation window of a certain application is displayed in the main window 40 of the multi-screen collaboration, the vertically long rectangular frame style of the main window 40 of the multi-screen collaboration can be adapted to adjust the display style and / or display content to better meet the user's visual requirements and improve the user's visual experience.

[0405] For example, Fig.16 As shown in FIG. 1(f), the audio operation window 10-3 of the music application may be different from the aforementioned embodiment (eg Fig.16 The display style and / or display content of the audio operation window 10-2 of the music application as shown in Figure (e). Optionally, the audio operation window 10-3 of the music application can display the name of the song 2 currently being played, the singer's name, the singer's photo, a playback progress bar, and more or less content.

[0406] Optionally, the voice call prompt window 30-4 may include the same number and / or type of controls as the prompt window 30 described above, and controls such as the answering control and the hang-up control may have different display positions and / or display sizes to better adapt to the vertically long rectangular frame style of the multi-screen collaborative main window 40.

[0407] Optionally, the audio operation window 20-3 of the browser application can be the same as the above-mentioned embodiment (for example Fig.16 The display style and / or display content listed in the audio operation window 20-3) of the browser application shown in (e) are consistent, and for the sake of simplicity, they are not repeated here.

[0408] It should also be understood that in the above scenario 4, when the main window 40 of the multi-screen collaboration is displayed in the focus state, the audio operation window corresponding to each application automatically popped up by the PC can be displayed in the main window 40 of the multi-screen collaboration.

[0409] Optionally, when the browser application, music application, and instant messaging application are not all mobile applications, or are all local applications of the PC, that is, for possible scenarios such as Scenario 1-Scenario 3 listed above, one or more audio operation windows can be displayed in the main window 40 of the multi-screen collaboration according to the display method introduced in Scenario 4, so as to reasonably utilize the display area of ​​the PC display screen, avoid affecting the user's use of other display areas on the display screen, save screen display area, avoid the problem of repeated display on the PC display screen, and present a more streamlined display effect to the user.

[0410] For any of the possible scenarios listed above, when the PC screen displays Fig.16 When viewing the audio notification interfaces shown in Figures (c), (d), (e) and (f), users can quickly control the playback of each audio through the audio operation window according to their needs.

[0411] For example, Fig.16 Taking (f) in the figure as an example, the user can click the play control in the audio operation window 10-3 of the music application, and the PC will pause playing song 2; click the play control in the audio operation window 20-3 of the browser application, and the PC will pause playing the song. The user can then click the answer control in the voice call prompt window 30-4 to talk to "Miss Ma", so that the call audio will not be disturbed by the song played by the PC. Please refer to the previous text for related descriptions, which will not be repeated here for simplicity.

[0412] When the user ends a call, the PC may also automatically pop up one or more audio operation windows, allowing the user to quickly resume the application and / or web page to continue playing the audio.

[0413] Fig.17 This is another schematic diagram of displaying an audio notification on a PC after a call ends, provided in an embodiment of the present application.

[0414] For example, the user clicks the play control in the audio operation window 10-3 of the music application, and the PC pauses playing song 2; clicks the play control in the audio operation window 20-3 of the browser application, and the PC pauses playing the song. The user then clicks the answer control in the voice call prompt window 30-4 to talk to "Miss Ma". After the user ends the call with "Miss Ma", the PC can further display Fig.17 The audio notification interface shown.

[0415] Optionally, in Fig.17 The audio notification interface shown includes: an audio operation window 10-3 of the music application, an audio operation window 20-3 of the browser application, and the audio operation window 10-3 of the music application and the audio operation window 20-3 of the browser application can be displayed in an upper and lower arrangement in the main window 40 of the multi-screen collaboration.

[0416] Exemplarily, the user can click the pause control of the audio operation window 10-3 of the music application, and the PC continues to play song 2 of the music application; click the pause control of the audio operation window 20-3 of the browser application, and the PC continues to play song 1 output by the browser application online web page.

[0417] In summary, through the above Figures 13 to 17 In the embodiment and the method for displaying audio notifications introduced, during the multi-screen collaboration between a PC and a mobile phone, the PC can simultaneously play audio from multiple applications and / or web pages, and the multiple applications can include local applications of the PC and applications on the mobile phone side. Specifically, when a multi-screen collaboration window in focus is displayed on the PC, the audio operation window of the application on the mobile phone side can be displayed in the multi-screen collaboration window, and the audio operation window of the local application on the PC can be displayed around the multi-screen collaboration window; when the multi-screen collaboration window is in a non-focus state or in a background running state, the audio operation window of the application on the mobile phone side and the audio operation window of the local application on the PC can be displayed around the application window in focus, so that the user can quickly control the playback of each audio through the audio operation window, ensure that the call process will not be disturbed by other output audios, avoid the user from having to perform cumbersome operations to pause the audio playback, simplify the operation process, and improve the user's operation experience.

[0418] In addition, the display method of this audio notification avoids the problem of repeated display on the PC display, can reduce the number of audio operation windows displayed on the screen, save screen display area, present a more streamlined display effect to users, and improve the user's visual experience.

[0419] Secondly, after the user ends the call, other audio operation windows except the call request prompt window can be automatically popped up and aggregated on the PC display screen. The user can quickly resume audio playback through other audio operation windows. For example, the user can click the pause control to resume the application and / or web page audio playback, which can quickly restore the PC to the audio playback state before receiving the call request, realize the fast switching of audio playback, convenient operation, and improve the user's operating experience.

[0420] Furthermore, during the above-mentioned multi-screen collaboration process, the audio operation window that pops up on the PC can avoid the user switching to other electronic devices (such as mobile phones) to handle call requests. The user does not need to actively switch the currently used PC to the mobile phone, which can achieve the continuity of the operation task and improve the user experience to a certain extent.

[0421] against Figures 5 to 17 The application scenario introduced introduces the possible display process and implementation method of audio notifications on a PC from the user interaction level. The following will introduce the underlying implementation process corresponding to the display method of audio notifications provided in the embodiment of the present application from the underlying implementation strategy level.

[0422] Fig.18 This is a schematic flowchart of a method for displaying an audio notification provided in an embodiment of the present application.

[0423] It should be understood that the display process of the audio notification provided in the embodiment of the present application can occur in the above-mentioned Figures 5 to 12 The user can also use the PC independently. Figures 13 to 17 In the multi-screen collaboration process of the PC and the mobile phone introduced, the embodiment of the present application does not limit the applicable scenarios of the display method of the audio notification.

[0424] Specifically, the PC described in the above embodiment is used as the first electronic device, and the mobile phone is used as the second electronic device, wherein the first electronic device and the second electronic device may have Figure 3 The hardware structure shown and Figure 4 The software structure shown below Fig.18 This section describes the underlying implementation process of the audio notification display method.

[0425] like Fig.18 As shown, the method 1800 may include the following steps:

[0426] 1801. A first electronic device receives a call request, where the call request includes a voice call request or a video call request.

[0427] Optionally, the call request may come from application, Any application with voice call and video call functions, such as mobile phone applications, call applications, phone applications, short message applications, games, etc. It should be understood that when such applications receive call requests from other devices or friends, a call prompt window will pop up in time to remind the user that the call request needs to be processed, and this embodiment of the application will not be described in detail.

[0428] Optionally, the application corresponding to the call request may be a local application installed and actually running on the first electronic device (e.g., a PC); or, in a multi-screen collaborative connection scenario between the first electronic device (e.g., a PC) and the second electronic device (e.g., a mobile phone), the application corresponding to the call request may also be an application installed and actually running on the second electronic device (e.g., a mobile phone), which is not limited to this in the embodiments of the present application.

[0429] 1802. In response to the call request, determine one or more applications that output audio on the first electronic device.

[0430] Optionally, the "one or more applications" in the embodiment of the present application may be an application on the first electronic device (such as a PC) that can output audio. For example, a music application is running on the first electronic device, and the music application can output audio such as songs, but may not be outputting audio at present.

[0431] Alternatively, the "one or more applications" may be one or more applications that are playing audio on the first electronic device (e.g., a PC). For example, music application A and music application B are running on the first electronic device, and both music application A and music application B can output audio such as songs, but currently music application A is outputting audio, and music application B is not outputting audio. It should be understood that the above different scenarios can all be included in the scope of the embodiments of the present application, and can all be applied to the method for displaying audio notifications provided in the embodiments of the present application, which will not be described in detail later.

[0432] Exemplarily, in an embodiment of the present application, it is assumed that the current first electronic device (e.g., a PC) is running the application corresponding to the call request, the first application, the second application, etc., wherein the first application and the second application are applications among the one or more applications that output audio on the first electronic device.

[0433] In a possible implementation, when a user is using a first electronic device (e.g., a PC), an audio task monitoring module of the first electronic device (e.g., a PC) can monitor and record the running status of each application, the process information or task information of the application being executed, etc. Correspondingly, the first electronic device (e.g., a PC) can obtain the display status of each of the one or more running applications based on the process information or task information recorded by the audio task monitoring module, such as which application windows are displayed on the display screen of the first electronic device, and which application window is displayed as the focus state.

[0434] There are two scenarios for whether the display includes the application window in the focus state:

[0435] Scenario 1: The display shows a window of a third application in focus

[0436] When it is determined that the window of the third application is displayed as the focus state, the first electronic device (such as a PC) can summarize all current audio tasks through the synchronization module and display all audio tasks on the display screen in the form of an audio operation window. Execute the following steps 1803:

[0437] 1803, a call reminder window, a first audio operation window, and a second audio operation window are displayed in an aggregated manner, wherein the first audio operation window includes a first control for controlling the audio playback of the first application, the second audio operation window includes a second control for controlling the audio playback of the second application, and the call reminder window includes a third control for controlling the call.

[0438] In one possible display method, when the display screen includes a window of a third application in a focused state, the first electronic device (such as a PC) can aggregate and display the call reminder window, the first audio operation window and the second audio operation window according to the display position of the window of the third application.

[0439] Optionally, the first electronic device (eg, PC) may aggregately display the call prompt window, the first audio operation window, and the second audio operation window at any border of the window of the third application.

[0440] Exemplarily, the call prompt window, the first audio operation window, and the second audio operation window may be aggregated and displayed in an area outside the window border of the third application in the focus state, that is, the call prompt window, the first audio operation window, and the second audio operation window may not overlap with the window of the third application in the focus state. This display method can avoid blocking the window of the third application in the focus state, facilitate user operation, and make it easier for users to view more interface content on the display screen of the first electronic device.

[0441] Optionally, the first electronic device (eg, PC) may also aggregate and display the call prompt window, the first audio operation window, and the second audio operation window inside the window of the third application.

[0442] Exemplarily, the call prompt window, the first audio operation window, and the second audio operation window can be aggregated and displayed inside the window of the third application in the focus state. In other words, the call prompt window, the first audio operation window, and the second audio operation window can have an overlapping area with the window of the third application in the focus state, such as a partial overlapping area, or a full overlapping area, which is not limited in this embodiment of the present application. This display method can provide users with a more eye-catching display effect, making the prompt more obvious.

[0443] It should be understood that the above method can be specifically combined with the position and size of the remaining display area on the display screen of the first electronic device (such as a PC) to make a judgment, and the embodiments of the present application are not limited to this.

[0444] In one possible scenario, when multiple web pages are displayed in the window of a third application that is in focus on the display screen, the audio task monitoring module of the first electronic device (such as a PC) can also monitor whether each web page of the third application can output audio or is outputting audio; and monitor the display status of the web page that outputs audio, such as whether it is displayed in a focus state or a non-focus state.

[0445] Optionally, after receiving the call request, the first electronic device can also detect the first web page that outputs audio in the window of the third application, and the first web page is displayed as a non-focus state in the window of the third application; and according to the display position of the window of the third application, a sixth audio operation window is displayed, and the sixth audio operation window includes a sixth control for controlling the audio playback of the first web page.

[0446] Optionally, the third application may be a browser application.

[0447] Exemplarily, when the third application is a browser application, two or more web pages capable of outputting audio may be displayed in the window of the third application. The focus monitoring module of the first electronic device may monitor the application window in focus on the display screen, as well as the switching process of the web page interface in the window. For example, it is determined that the first web page in the window of the third application outputs audio on the first electronic device, and the first web page is in a non-focus state. Accordingly, the first electronic device may also aggregate and display the audio operation window corresponding to the first web page to control the audio playback of the first web page. Among them, the display of the audio operation window corresponding to the first web page may refer to the display rules of the aforementioned call prompt window, the first audio operation window and the second audio operation window, which will not be repeated here.

[0448] In another possible scenario, when the user is using the first electronic device (e.g., PC), the application window in focus may change continuously. When the application window in focus changes, in response to the call request, the number of windows automatically popped up by the first electronic device (e.g., PC), the application corresponding to the window, and the window display content may also change adaptively.

[0449] Through the above scheme, when the first electronic device (such as a PC) receives a call request, it can detect the application window in the focus state and the application window in the non-focus state on the display screen, and aggregate and display the audio operation window and the call request prompt window around the application window in the focus state. In other words, when the application window in the focus state and / or the application window in the non-focus state on the display screen of the first electronic device (such as a PC) are different, the number, type, display position, and display size of the audio operation windows that pop up automatically will also be different.

[0450] Scenario 2: The application window without focus on the display

[0451] When it is determined that there is no application window in focus on the display screen of the first electronic device, the first electronic device (such as a PC) can also summarize all current audio tasks through the synchronization module and display all audio tasks on the display screen in the form of an audio operation window. Execute the following steps 1804:

[0452] 1804. The call prompt window, the first audio operation window and the second audio operation window are displayed in aggregate at a preset position on the display screen.

[0453] In one possible display method, the call prompt window, the first audio operation window and the second audio operation window that automatically pop up on the first electronic device (such as a PC) have preset positions.

[0454] It should be noted that, for scenario 1, when the first electronic device (such as a PC) displays a window of a third application in focus, the call reminder window, the first audio operation window and the second audio operation window can also be displayed in a cluster at a preset position on the display screen according to the display method of step 1804. This method also falls within the protection scope of the embodiments of the present application.

[0455] Optionally, the preset position may be the upper boundary of the display area of ​​the display screen of the first electronic device (e.g., PC) and located at the center of the width direction of the display screen. Alternatively, the preset position may be the lower boundary of the display area of ​​the display screen of the first electronic device (e.g., PC) and located at the center of the width direction of the display screen. Alternatively, the preset position may be a fixed position of the upper left corner of the display screen of the first electronic device (e.g., PC). Alternatively, the preset position may be a fixed position of the lower right corner of the display screen of the first electronic device (e.g., PC), etc., which is not limited in the embodiments of the present application.

[0456] In another possible display method, the call reminder window that pops up automatically, the first audio operation window for controlling the audio playback of the first application, and the second audio operation window for controlling the audio playback of the second application can be displayed in the form of "one window" on the display screen of the first electronic device (such as a PC).

[0457] Optionally, the "one window" may include a first control, a second control and a third control at the same time; or, the one operation window may include two controls, wherein control 1 may simultaneously control the audio playback of the first application and the audio playback of the second application, and control 2 may be used to control answering calls; or, the one operation window may include one control, which may simultaneously control the audio playback of the first application and the audio playback of the second application, and control answering calls, and the embodiments of the present application are not limited to this.

[0458] Alternatively, two windows may automatically pop up on the display screen of the first electronic device (such as a PC), and window 1 is used to control the audio playback of the first application and the audio playback of the second application, etc. It should be understood that the above multiple possible audio notification display methods all fall within the scope of protection of the embodiments of the present application, and examples are not given one by one here.

[0459] It can be seen from the above steps that when the first electronic device (e.g., PC) receives a call request, all applications and / or web pages currently involved in audio output of the first electronic device (e.g., PC) can be displayed in the form of corresponding audio operation windows, and each audio operation window can include one or more operation controls. The user can quickly control the audio playback through the controls in each audio operation window. Therefore, the method 1800 can also include the following steps 1805:

[0460] 1805. Receive an operation on the first control by a user, and in response to the operation on the first control, pause playing the audio of the first application; receive an operation on the second control by a user, and in response to the operation on the second control, pause playing the audio of the second application; receive an operation on the third control by a user, and in response to the operation on the third control, answer the call.

[0461] Specifically, users can quickly control the playback of each application audio through the controls in each audio operation window, for example, clicking the playback control to pause the corresponding audio. This avoids the need for users to perform cumbersome operations to pause the audio playback, allowing users to answer calls as quickly as possible and ensuring that the user's call process is not disturbed by other audio output from the PC.

[0462] When the user ends the call, one or more audio operation windows may automatically pop up on the display screen of the first electronic device (e.g., PC), so that the user can re-control one or more applications and / or web pages to continue playing the audio. Accordingly, the method 1800 may also include the following steps 1806:

[0463] 1806. In response to the operation of ending the call, a fourth audio operation window and a fifth audio operation window are displayed, the fourth audio operation window includes a fourth control, the fourth control is used to control the audio playback of the first application, and the fifth audio window includes a fifth control, the fifth control is used to control the audio playback of the second application.

[0464] 1807, receiving the user's operation on the fourth control, and in response to the operation on the fourth control, resuming the playing of the audio of the first application; receiving the user's operation on the fifth control, and in response to the operation on the fifth control, resuming the playing of the audio of the second application.

[0465] It should be understood that the fourth audio operation window and the aforementioned first audio operation window are both used to control the audio playback of the first application, and may display different or the same controls. For example, the first audio operation window displays the play control, and the fourth audio operation window displays the pause control. They may both be referred to as the "audio operation window of the first application", which is not limited in this embodiment of the present application.

[0466] It should also be understood that when the user ends the call, the information synchronization module of the first electronic device (such as a PC) can stop synchronizing the audio data of the call, and the call data stops playing. The audio information is then aggregated through the audio task monitoring module and the synchronization module, and the audio operation window of each application included in the target application set is automatically displayed on the display screen. The user's operation is then detected, and according to the user's operation, the audio playback control module releases the mute state of other applications and resumes the original audio playback.

[0467] In the above implementation process, after the user ends the call, the first electronic device (such as a PC) can display other audio operation windows in addition to the call prompt window, and the user can quickly resume audio playback through other audio operation windows. For example, the user can click the pause control to resume the application and / or web page audio playback, and can quickly restore the first electronic device (such as a PC) to the audio playback state before receiving the call request, thereby realizing fast switching of audio playback, convenient operation, and improving the user's operating experience.

[0468] Optionally, the application, number of windows, display position, display size, etc. corresponding to the "other audio operation windows" may be consistent with the audio operation window displayed when the first electronic device (such as a PC) receives a call request, or, the application, number of windows, display position, display size, etc. corresponding to the "other audio operation windows" may be different from the audio operation window displayed when the PC receives a call request, and the embodiments of the present application are not limited to this.

[0469] Exemplarily, during the process of the user answering the call, the application running on the first electronic device (e.g., PC) may not change. For example, during the process of the user answering the call, the application running status of the first electronic device (e.g., PC) is not closed or changed. When the call is ended, the first electronic device (e.g., PC) continues to run the first application, the second application, and the third application. Accordingly, in response to the operation of ending the call, the first electronic device (e.g., PC) aggregately displays the audio operation window (i.e., the fourth audio operation window) for controlling the first application and the audio operation window (i.e., the fifth audio operation window) for controlling the second application.

[0470] Alternatively, while the user is answering the call, the application running on the first electronic device (such as a PC) may also change. For example, while the user is answering the call, the first application is closed, and the fourth application is run, and the fourth application is also an application that can output audio on the first electronic device (such as a PC). Accordingly, in response to the operation of ending the call, the first electronic device (such as a PC) can aggregate the display for controlling the audio operation window of the second application (i.e., the fifth audio operation window) and the audio operation window for controlling the fourth application. The above different audio notification display methods all fall within the scope of protection of this application.

[0471] It should be understood that in the process of displaying an audio notification on a first electronic device (such as a PC), in addition to determining the display position of the call reminder window, the first audio operation window, and the second audio operation window according to the method described above, the display size of the call reminder window, the first audio operation window, and the second audio operation window can also be determined according to a variety of possible methods. For example, the display size of the call reminder window, the first audio operation window, and the second audio operation window can be determined according to the window of the third application in the focus state, or the display size of the call reminder window, the first audio operation window, and the second audio operation window of a preset size can be displayed according to a preset size, and the embodiments of the present application are not limited to this.

[0472] It should be noted that the first electronic device (such as a PC) may not distinguish the application source of one or more applications that output audio, and directly install the above steps 1801-1807, and only determine the display method of the audio notification based on the application window in focus displayed on the display screen and / or the web page displayed in the application window.

[0473] In another possible implementation, after receiving a call request, the first electronic device (e.g., PC) can determine whether one or more applications that output audio are all from local applications of the first electronic device (e.g., PC), or include applications from a second electronic device that is connected to the first electronic device. For example, a scenario of a second electronic device in a multi-screen collaborative scenario (e.g., a mobile phone and a PC establishing a multi-screen collaborative connection).

[0474] In this scenario, the first electronic device (such as a PC) may further perform the following steps:

[0475] Step 1, determining the sources of all applications capable of outputting audio, for example, determining whether the one or more applications include an application running in the foreground or background of the second electronic device (eg, a mobile phone).

[0476] Step 2: Determine whether the display screen of the first electronic device (eg, PC) includes a window of an application in a focus state.

[0477] Step 3. When the display screen of the first electronic device (e.g., PC) does not include any application window displayed in a focus state, the call reminder window, the first audio operation window, and the second audio operation window are automatically displayed on the display screen according to a preset position and / or display size.

[0478] Step 4: When the display screen of the first electronic device (such as a PC) includes a multi-screen collaboration main window and the multi-screen collaboration main window is displayed in a focus state, after receiving the call request, multiple windows can be aggregated and displayed in the following different ways:

[0479] (1) The call prompt window, the first audio operation window, and the second audio operation window are aggregated and displayed in the multi-screen collaboration main window.

[0480] Exemplarily, when the multi-screen collaboration main window is the focus window on the PC, the call prompt window and the audio operation window of each application can be aggregated and displayed in the multi-screen collaboration main window.

[0481] (2) According to the display position of the multi-screen collaboration main window, the call prompt window, the first audio operation window and the second audio operation window are aggregated and displayed.

[0482] Exemplarily, according to the implementation process of the aforementioned steps 1801 to 1804, only the multi-screen collaboration main window is used as a focus window, and the call prompt window and the audio operation window of each application are aggregated around the multi-screen collaboration main window, such as any border position, etc. Please refer to the aforementioned introduction and no further details will be given here.

[0483] (3) Detect whether the one or more applications include an application originated from the second electronic device. When the one or more applications include a fifth application originated from the second electronic device, display the audio operation window of the fifth application in the multi-screen collaborative main window, and aggregate and display the call prompt window, the first audio operation window, and the second audio operation window according to the display position of the multi-screen collaborative main window.

[0484] (4) When the call request originates from the second electronic device, a call prompt interface corresponding to the call request is displayed in the multi-screen collaboration main window, and the first audio operation window and the second audio operation window are aggregated and displayed according to the display position of the multi-screen collaboration main window.

[0485] It should be understood that the audio notification display method provided in the embodiment of the present application may include all the steps described above, or only some of the steps described above. In other words, some steps of the above process may be optional steps, and the embodiment of the present application is not limited to this.

[0486] To summarize, when a user is using an electronic device such as a PC, the PC can play audio from multiple applications and / or web pages at the same time. The multiple applications may include local applications of the PC, and may also include applications on the mobile phone side that establish a multi-screen collaborative connection with the PC. Specifically, when a call request is received, the PC can display the application window and / or web page displayed in the focus state, for example, display the audio operation window corresponding to the non-focus application and / or web page on the display screen, so that the user can quickly control the playback of each audio through the audio operation window. For example, the user pauses the audio playback through the audio operation window and then answers the call, ensuring that the call process will not be disturbed by other output audio, avoiding the user from having to perform cumbersome operations to pause the audio playback, simplifying the operation process, and improving the user's operating experience.

[0487] Secondly, after the user ends the call, other audio operation windows in addition to the call request prompt window may automatically pop up on the PC display screen. The user can quickly resume audio playback through other audio operation windows. For example, the user can click the pause control to resume the application and / or web page audio playback, and can quickly restore the PC to the audio playback state before receiving the call request, thereby realizing fast switching of audio playback, convenient operation, and improving the user's operating experience.

[0488] Furthermore, during the multi-screen collaboration process of PC and mobile phone, the audio operation window that pops up on the PC can avoid users switching to mobile phones to handle call requests. Users do not need to actively switch the currently used PC to the mobile phone, which can achieve the continuity of operation tasks and improve the user experience to a certain extent.

[0489] It should be noted that, in the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0490] It should be understood that the methods, situations, categories and divisions of the embodiments in the present application are only for the convenience of description and should not constitute special limitations. The features of various methods, categories, situations and embodiments can be combined without contradiction.

[0491] It should also be understood that the terms "first", "second" and "third" in the embodiments of the present application are only for distinction and should not constitute any limitation to the present application. For example, the "first application", "second application", etc. in the embodiments of the present application represent different applications; "first audio operation window", "second audio operation window", "third audio operation window", etc. represent audio operation windows corresponding to different applications and / or web pages, and each audio operation window may include different numbers and types of controls.

[0492] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0493] It should also be noted that in the embodiments of the present application, the "preset position", "preset size", etc. can be implemented by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in the electronic device, and the present application does not limit the specific implementation method. For example, in the embodiments of the present application, the "preset position" can represent a fixed position on the PC display screen, such as the lower right corner of the PC display screen close to the boundary of the display area of ​​the display screen.

[0494] It is understandable that, in order to realize the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of the present application.

[0495] In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0496] In the case of dividing each functional module according to each function, a possible composition diagram of the electronic device involved in the above embodiment, the electronic device may include: a display unit, a detection unit and a processing unit. The display unit, the detection unit and the processing unit cooperate with each other to support the electronic device to perform the above steps, etc., and / or other processes of the technology described in this article.

[0497] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0498] The electronic device provided in this embodiment is used to execute the above-mentioned audio notification display method, and thus can achieve the same effect as the above-mentioned implementation method.

[0499] In the case of using integrated units, the electronic device may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the electronic device, for example, it can be used to support the electronic device to execute the steps performed by the above-mentioned display unit, detection unit and processing unit. The storage module can be used to support the electronic device to execute stored program codes and data, etc. The communication module can be used to support the communication between the electronic device and other devices.

[0500] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0501] In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device (eg, the first electronic device and the second electronic device) involved in this embodiment may be a device having Figure 3 Device of the structure shown.

[0502] This embodiment also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method for displaying audio notifications in the above-mentioned embodiment.

[0503] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the method for displaying audio notifications in the above-mentioned embodiment.

[0504] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory to enable the chip to execute the audio notification display method in the above-mentioned method embodiments.

[0505] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0506] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0507] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0508] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0509] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0510] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0511] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for displaying an audio notification, characterized in that: Applied to a first electronic device including a display screen, the method includes: receiving a call request, wherein the call request includes a voice call request or a video call request; In response to the call request, determining one or more applications that output audio on the first electronic device; Aggregate and display a call prompt window, a first audio operation window, and a second audio operation window, wherein the first audio operation window includes a first control for controlling audio playback of a first application, the second audio operation window includes a second control for controlling audio playback of a second application, and the call prompt window includes a third control for controlling a call, wherein the first application and the second application are applications in the one or more applications that output audio on the first electronic device; receiving an operation of the first control by a user, and pausing playing of the audio of the first application in response to the operation of the first control; receiving an operation of the second control by a user, and pausing playing of the audio of the second application in response to the operation of the second control; An operation of the third control by a user is received, and in response to the operation of the third control, a call is answered.

2. The method according to claim 1, characterized in that The method further comprises: In response to the operation of ending the call, a fourth audio operation window and a fifth audio operation window are displayed, the fourth audio operation window includes a fourth control, the fourth control is used to control the audio playback of the first application, and the fifth audio window includes a fifth control, the fifth control is used to control the audio playback of the second application.

3. The method according to claim 2, characterized in that The method further comprises: receiving an operation of the fourth control by a user, and resuming playing the audio of the first application in response to the operation of the fourth control; An operation of the fifth control by a user is received, and in response to the operation of the fifth control, the audio of the second application is resumed to be played.

4. The method according to any one of claims 1 to 3, characterized in that The aggregate display of the call prompt window, the first audio operation window and the second audio operation window includes: According to the display position of the window of the third application, the call prompt window, the first audio operation window and the second audio operation window are aggregated and displayed, and the window of the third application is displayed as a focus state on the display screen.

5. The method according to claim 4, characterized in that The step of displaying the call prompt window, the first audio operation window, and the second audio operation window in an aggregate manner according to the display position of the window of the third application includes: At any border of the window of the third application, the call prompt window, the first audio operation window and the second audio operation window are aggregated and displayed; or, The call prompt window, the first audio operation window and the second audio operation window are displayed in an aggregate manner inside the window of the third application.

6. The method according to any one of claims 1 to 3, characterized in that When a plurality of web pages are displayed in the window of the third application in the focus state on the display screen, after the first electronic device receives the call request, the method further includes: Detecting a first webpage that outputs audio in the window of the third application, where the first webpage is displayed in a non-focus state in the window of the third application; A sixth audio operation window is displayed according to the display position of the window of the third application, wherein the sixth audio operation window includes a sixth control for controlling the audio playback of the first webpage.

7. The method according to claim 6, characterized in that The third application is a browser application.

8. The method according to any one of claims 1 to 3, characterized in that The first application and / or the second application and / or the third application are run on a second electronic device, the second electronic device is connected to the first electronic device to realize multi-screen collaboration, the main window of the multi-screen collaboration is displayed on the display screen of the first electronic device, and the display content of the main window of the multi-screen collaboration is consistent with the display content of the main interface of the second electronic device.

9. An electronic device, characterized in that: include: Display screen; one or more processors; one or more memories; Modules with multiple applications installed; The memory stores one or more programs, and when the one or more programs are executed by the processor, the electronic device executes the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 8.

11. A computer program product, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 8.

Images