Interface display method, electronic device, chip system and computer program product

Abstract

The invention relates to the field of terminals, in particular to an interface display method, electronic equipment, a chip system and a computer program product. According to the method, a user can trigger a resident setting control (a first control) of the virtual assistant by clicking a navigation bar in a window of the virtual assistant, and the virtual assistant can be set to be in a resident state by operating the resident setting control. Through the method, the user can autonomously set the window of the virtual assistant, so that the virtual assistant can adapt to various application scenes, and the experience of the user when the virtual assistant is used can be improved.

Description

Technical Field

[0001] This application relates to the field of terminals, and more particularly to an interface display method, electronic device, chip system, and computer program product. Background Technology

[0002] A virtual assistant is an interactive service implemented using artificial intelligence technology. When a user activates a virtual assistant while using applications on their electronic device, the device can collaboratively run the virtual assistant and other applications, meaning the virtual assistant and other applications run simultaneously in the foreground. However, during this collaborative operation, the user cannot flexibly interact with the virtual assistant's window, impacting the user experience. Summary of the Invention

[0003] This application provides an interface display method, electronic device, chip system, and computer program product, which solves the problem in the prior art that users cannot flexibly operate application windows.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] Firstly, a method for displaying an interface is provided, applied to an electronic device, including:

[0006] Display a first interface, the first interface including a first window of the first application;

[0007] In response to a user's first operation to wake up the second application, a second interface is displayed, the second interface including the first window and a second window of the second application, the second window being displayed floating on the first window; the second window includes a first navigation bar;

[0008] In response to a second user action on the first navigation bar, a third interface is displayed; the third interface includes the first window, the second window, and the first control.

[0009] In response to a third operation by the user on the first control, a fourth interface is displayed; the fourth interface includes the first window and a third window of the second application; wherein the third window has a different form than the first window.

[0010] For example, Figure 18 When interface 32 is designated as the first interface, Figure 17Interface 31 is designated as the second interface. The first application can be email, and the second application can be a virtual assistant. Window 901 of the virtual assistant can be the second window, and navigation bar 1302 can be the first navigation bar. Interface 37 can be the third interface, control 1701 can be the first control, and the third operation can be clicking or long-pressing the first control. Interface 38 can be the fourth interface, and window 1702 can be the third window.

[0011] Among them, such as Figures 3 to 5 As shown, the first operation can be pressing the power button, voice activation, or clicking / long-pressing the navigation bar in the first application interface (such as...). Figure 5 (Navigation bar 501 in the middle).

[0012] In this embodiment, by clicking the navigation bar in the virtual assistant window, the user can trigger the virtual assistant's persistent settings control (first control). By manipulating the persistent settings control, the virtual assistant can be set to a persistent state. This method allows users to customize the virtual assistant window, enabling it to adapt to various application scenarios and thus improving the user experience.

[0013] In one implementation of the first aspect, the method further includes:

[0014] In response to a fourth user action on the second window, a fifth interface is displayed; the fifth interface includes the first window and the third window of the second application.

[0015] In some cases, the fourth action can be dragging the second window.

[0016] For example, Figure 18 Interface 39 can be referred to as the fifth interface, and window 1702 can be referred to as the third window.

[0017] In this way, users can choose whether to keep the virtual assistant running, enabling it to adapt to various application scenarios and thus improve the user experience.

[0018] In one implementation of the first aspect, the fifth interface further includes a second control; the method further includes:

[0019] In response to the user's fifth operation on the second control, the second interface is displayed.

[0020] For example, Figure 18 Control 1801 in the text can be referred to as the second control.

[0021] In some implementations, if the user does not operate or clicks any position on the screen other than control 1801 within a preset time after the electronic device switches from interface 31 to interface 39, the electronic device hides control 1801 and keeps the floating window 1702 of the virtual assistant displayed. That is, the electronic device performs collaborative processing of the virtual assistant and the background application by default, and the virtual assistant and the background application run in the foreground at the same time.

[0022] In other implementations, after the electronic device hides the control 1801, the user can manually click to re-expand the control 1801. For example, after the electronic device hides the control 1801, if the user clicks / drags the floating window 1702, or clicks / long-presses the navigation bar 1703 in the floating window 1702, the electronic device will redisplay the control 1801.

[0023] In this embodiment of the application, the virtual assistant's persistent state setting control enables users to independently set the persistent state of the virtual assistant, thereby improving the user's flexibility and experience when using the virtual assistant.

[0024] In one implementation of the first aspect, the third window includes a third control; the method further includes:

[0025] In response to the user's sixth operation on the third control, the first interface is displayed.

[0026] For example, Figure 18 The cancel control 1704 in the code can be a third-party control.

[0027] Optionally, the sixth action can be clicking the third control.

[0028] In this embodiment of the application, by setting a third control, the user can quickly close the virtual assistant by operating the third control, thereby improving the user's experience when using the virtual assistant.

[0029] In one implementation of the first aspect, the third window includes a fourth control; the method further includes:

[0030] In response to the user's seventh operation on the fourth control, a sixth interface is displayed; the third window is displayed in full screen on the sixth interface.

[0031] For example, Figure 18 The magnification control 1705 in the diagram is designated as the fourth control.

[0032] In this embodiment, by setting a fourth control, the user can display the virtual assistant window in full screen by operating the fourth control, thereby improving the user's experience when using the virtual assistant.

[0033] In one implementation of the first aspect, the method further includes:

[0034] In response to the user's eighth operation on the first window in the second interface, a seventh interface is displayed; the seventh interface includes the first window and a fourth window of the second application, the fourth window having a different form from the second window.

[0035] The eighth operation can be clicking on the first window or swiping the screen in the first window.

[0036] For example, Figure 11 Interface 34 can be referred to as the seventh interface. The changes in the form of the fourth window relative to the second window can include: window shrinking and hiding the outer aperture, etc.

[0037] Optionally, the fourth window is smaller than the second window. In this implementation, when the virtual assistant runs in conjunction with other applications, the virtual assistant's window shrinks as the user interacts with the background application's interface, making it easier for the user to operate and thus improving the user experience when using the virtual assistant.

[0038] When the virtual assistant is active, if the user interacts with a background application, the electronic device can automatically perform collaborative processing between the virtual assistant and the background application. This means the virtual assistant doesn't end the session, and both the virtual assistant and the background application remain in the foreground. Furthermore, the electronic device can automatically adjust the virtual assistant's window appearance, such as shrinking the window, to reduce its impact on user interaction. Additionally, the electronic device can further enhance the user experience by automatically adjusting the virtual assistant's window appearance, such as hiding the outer aperture, to indicate to the user that the current focus is on the background application.

[0039] In one implementation of the first aspect, the first window in the seventh interface displays a first screen, and the seventh interface includes a fifth control for triggering a task matching the first screen; the method further includes:

[0040] In response to the user's ninth operation on the first window, the first screen of the first window in the seventh interface is switched to the second screen;

[0041] The fifth control is updated to a sixth control, which is used to trigger a task that matches the second screen.

[0042] Optionally, when the focus is on the background application, the virtual assistant's session does not end while the user is operating the background application. That is, the virtual assistant continues to run in the foreground and does not exit, and the virtual assistant's window can remain out of focus.

[0043] For example, Figure 11Control 902 in the code can be referred to as the fifth control. For example... Figure 11 As shown in (b), the electronic device displays interface 34. At this time, the virtual assistant is out of focus, and the virtual assistant window is in a one-state out-of-focus state, as shown in window 1101 in interface 34. When the user clicks the exit control 1102 of the background application, in response to the user's operation, the electronic device prioritizes the return operation of the background application and displays interface 35. Comparing interfaces 34 and 35, the interface of the background application has changed, but the virtual assistant window 1101 is still displayed on the interface, and its form has not changed; it is still in a one-state out-of-focus window state. At this time, the electronic device has not closed the virtual assistant, and the virtual assistant is still in an out-of-focus state.

[0044] However, due to the change in the background application's interface, the virtual assistant's control 902 changes accordingly. As shown in interface 35, the virtual assistant's control 902 disappears (equivalent to the sixth control disappearing).

[0045] In some implementations, to facilitate user operation, the shape of the virtual assistant's controls may remain unchanged during the virtual assistant's focus loss and focus gain processes, but it can change according to the changes in the background application's interface.

[0046] In one implementation of the first aspect, the method further includes:

[0047] In response to the user's ninth operation on the content displayed in the first window of the second interface, the fifth control is updated to a seventh control, which is used to trigger a task that matches the display content pointed to by the ninth operation.

[0048] For example, Figure 12 Control 1301 can be designated as the fifth control, and control 1302 as the seventh control. The ninth operation can be clicking or selecting content to display. For example... Figure 12 As shown in (a), the prompt text for the virtual assistant's control 1301 is "Polish the article" and "Summary Summary," and the virtual assistant is in a defocused state at this time. When the user clicks or long-presses the text in the background application, in response to the user's operation, the electronic device displays as shown in (a). Figure 12 The interface shown in (b) is as follows. Figure 12 As shown in (b), the prompt text of the virtual assistant control 1302 is "Navigate to this address" and "Add this address", that is, the prompt text of the virtual assistant control has changed, and correspondingly, the service indicated by the virtual assistant control has also changed.

[0049] In one implementation, the tooltip text of the virtual assistant's control can change according to changes in the background application. For example, ... Figures 3 to 5As shown, when the background application is a desktop application, the tooltip text for the controls in the virtual assistant's display area that provide services matching the current interface is "Organize Desktop" and "New Feature Introduction". Figure 9 and Figure 10 As shown, when the background application is email, the tooltip text for the controls in the virtual assistant's display area that provide services matching the current interface is "Polish Article" and "Summary Summary".

[0050] In another implementation, when the interface of the same background application changes, the tooltip text of the virtual assistant's controls can change according to the change in the background application's interface. For example, ... Figure 11 As shown, when the background application interface switches from interface 34 to interface 35, the controls for "polishing the article" and "summary summary" disappear.

[0051] In another implementation, the tooltip text of the virtual assistant's controls can change according to changes in the user's input area. For example, ... Figure 12 As shown, when a user taps an area in the background application, the electronic device responds to that user action by... Figure 12 (a) in the middle switch to Figure 12 In (b), the tooltip text for the controls in the virtual assistant's display area that provide services matching the current interface changes from "polish the article" and "summarize the summary" to "navigate to this address" and "add this address".

[0052] It should be noted that the size of the virtual assistant's controls can be adjusted according to the amount of its prompt text.

[0053] By using controls in the virtual assistant's display area to provide services that match the current interface, quick services can be offered to users, thus improving the user experience.

[0054] In one implementation of the first aspect, the method further includes:

[0055] In response to the user's tenth operation on the fourth window, the second interface is displayed.

[0056] Optionally, the tenth action can be clicking the fourth window, interacting with the virtual assistant via voice, or entering text in the fourth window.

[0057] It is understood that the fourth window represents the virtual assistant when it is out of focus, while the second window in the second interface represents the virtual assistant when it is in focus. In this embodiment, when the fourth window is operated on, the virtual assistant regains focus. In this case, the window changes from the fourth window to the second window, which is equivalent to the virtual assistant's window changing from an out-of-focus state to a focused state. In this way, the user is prompted that the current focus has returned to the virtual assistant, thereby improving the user experience.

[0058] In one implementation of the first aspect, the method further includes:

[0059] In response to the user's tenth operation on the fourth window, the first animation effect of the fourth window is played at the first moment;

[0060] At a second moment, the second interface is displayed; wherein the second moment is after the first moment.

[0061] Optionally, the first animation effect includes at least one of the following: the navigation bar in the fourth window is highlighted for a preset duration; the shape of the fourth window changes to the shape of the second window; a halo appears at the edge of the fourth window, and the halo rotates a preset number of times.

[0062] In this embodiment, the virtual assistant's window changes from a out-of-focus state to a focused state. This change in the virtual assistant's window prompts the user to switch the current focus to the virtual assistant. Furthermore, animation effects enhance the prompt during the change, improving the user experience.

[0063] It should be noted that when the virtual assistant changes from being unwakeable to being woken up, such as... Figure 3 (a) Figure 4 (a) and Figure 5 In the process shown in (a), the navigation bar in the virtual assistant's window can be displayed in grayscale. Since the virtual assistant's window changes from hidden to visible when it is activated, and the window includes an outer aperture, these display features can indicate to the user that the virtual assistant has been activated. Therefore, in this case, visual cues via a highlighted navigation bar are unnecessary. However, when the virtual assistant refocuses, as shown in (a), the navigation bar appears to be in grayscale. Figure 13 and Figure 15 As shown in the process, since the virtual assistant's window remains displayed during this process, only its appearance changes, the navigation bar in the virtual assistant's window can be highlighted for a period of time and then displayed in grayscale to enhance the prompt for the virtual assistant to refocus. Of course, it is understood that other methods can be used in practical applications, and this application embodiment does not specifically limit this.

[0064] When an electronic device is performing collaborative processing between a virtual assistant and a background application—meaning both are running in the foreground—if the virtual assistant refocuses from a defocused state, the electronic device can automatically adjust the virtual assistant's window appearance, such as restoring its size, to increase the user's operating area. Additionally, the electronic device can further enhance the user experience by automatically adjusting the virtual assistant's window appearance, such as displaying a dynamic effect of the window's outer aperture or highlighting the navigation bar, to indicate to the user that the current focus has returned to the virtual assistant.

[0065] In one implementation of the first aspect, the method further includes:

[0066] In response to the user's eleventh action on the second interface, the first interface is displayed.

[0067] Optionally, the eleventh operation can be a click operation on a blank area in the first window of the second interface. It should be noted that the blank area in the above example refers to the part of the background application's display area where no controls or content are displayed. Besides the above examples, in some implementations, the user can also exit the virtual assistant using physical buttons (such as volume buttons, power buttons, etc.). In other implementations, the user can also exit the virtual assistant using preset gestures (such as clicking the screen three times consecutively or drawing a preset shape on the screen). This application does not specifically limit this aspect.

[0068] Secondly, an electronic device is provided, comprising:

[0069] One or more processors;

[0070] One or more memory units;

[0071] The memory stores a computer program that, when executed by the processor, causes the electronic device to perform the method as described in any of the first aspects.

[0072] Thirdly, a chip system is provided, the chip system including a processor coupled to a memory, the processor being configured to run a computer program stored in the memory to implement the method as described in any of the first aspects.

[0073] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by one or more processors, implements the method as described in any of the first aspects.

[0074] Fifthly, a computer program product is provided, the computer program product including computer program code, which, when run on an electronic device, causes the electronic device to perform the method as described in any of the first aspects.

[0075] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0076] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0077] Figure 2 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application;

[0078] Figure 3 This is a schematic diagram illustrating a scenario where an electronic device wakes up a virtual assistant, as provided in an embodiment of this application.

[0079] Figure 4 This is a schematic diagram illustrating a scenario where an electronic device wakes up a virtual assistant, as provided in another embodiment of this application.

[0080] Figure 5 This is a schematic diagram illustrating a scenario where an electronic device wakes up a virtual assistant, as provided in another embodiment of this application.

[0081] Figure 6 A schematic diagram of the window form of the virtual assistant provided in the embodiments of this application;

[0082] Figure 7 A schematic diagram illustrating a scenario of a virtual assistant performing tasks, as provided in an embodiment of this application;

[0083] Figure 8 A schematic diagram illustrating the window mode switching of the virtual assistant provided in this application embodiment;

[0084] Figure 9 A schematic diagram illustrating a scenario where a virtual assistant ends a session, as provided in an embodiment of this application.

[0085] Figure 10 This is a schematic diagram illustrating a scenario where a virtual assistant is used in conjunction with other applications, as provided in the embodiments of this application.

[0086] Figure 11 A schematic diagram illustrating a scenario where the virtual assistant window loses focus, as provided in an embodiment of this application.

[0087] Figure 12 A schematic diagram illustrating the switching of control states provided in an embodiment of this application;

[0088] Figure 13 A schematic diagram illustrating the scenario where the virtual assistant window gains focus, as provided in this application embodiment;

[0089] Figure 14 A schematic diagram of a virtual assistant window losing focus, provided in another embodiment of this application;

[0090] Figure 15 A schematic diagram of a virtual assistant window being focused, provided in another embodiment of this application;

[0091] Figure 16 This is a schematic diagram illustrating the exit scenario of the virtual assistant provided in an embodiment of this application;

[0092] Figure 17A schematic diagram illustrating a persistent scenario for the virtual assistant provided in this application embodiment;

[0093] Figure 18 A schematic diagram illustrating a persistent scenario for a virtual assistant provided in another embodiment of this application;

[0094] Figure 19 A schematic diagram illustrating a persistent scenario for a virtual assistant provided in another embodiment of this application;

[0095] Figure 20 A schematic diagram illustrating a persistent scenario for a virtual assistant provided in another embodiment of this application;

[0096] Figure 21 This is a schematic diagram of a persistent scenario for a virtual assistant provided in another embodiment of this application. Detailed Implementation

[0097] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limiting purposes, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details.

[0098] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0099] It should also be understood that in the embodiments of this application, "one or more" refers to one, two, or more; "and / or" describes the relationship between the associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0100] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," "fourth," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0101] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0102] The interface display method provided in this application can be applied to electronic devices. Electronic devices include terminal devices, which can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablets, smart screens, computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments of this application do not limit the specific technologies or device forms used in the electronic devices.

[0103] See Figure 1This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a touch sensor 180K, an ambient light sensor 180L, etc.

[0104] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0105] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0106] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0107] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0108] External memory 120 generally refers to external storage. In the embodiments of this application, external storage refers to storage other than the memory of electronic devices and the cache of processors. This storage is generally non-volatile memory.

[0109] Internal memory 121, also known as "RAM," can be used to store executable program code for a computer, including instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a given function (such as sound playback, image playback, etc.).

[0110] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays screens 194, where N is a positive integer greater than 1. In some embodiments, electronic device 100 displays a user interface through the displays screens 194.

[0111] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0112] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0113] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0114] Electronic device 100 also includes various sensors that can convert different physical signals into electrical signals. For example, pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. Gyroscope sensor 180B can be used to determine the motion posture of electronic device 100. Barometric pressure sensor 180C is used to measure air pressure. Magnetic sensor 180D includes a Hall sensor. Accelerometer sensor 180E can detect the magnitude of acceleration of electronic device 100 in various directions (generally three axes). Distance sensor 180F is used to measure distance. Electronic device 100 can measure distance using infrared or laser. Proximity sensor 180G may include, for example, a light-emitting diode (LED) and a photodetector, such as a photodiode. Ambient light sensor 180L is used to sense ambient light brightness. Electronic device 100 can adaptively adjust the brightness of display screen 194 according to the sensed ambient light brightness. Fingerprint sensor 180H is used to collect fingerprints. Electronic device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application lock, fingerprint photography, fingerprint answering of calls, etc. Temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. The bone conduction sensor 180M can acquire vibration signals.

[0115] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0116] For example, in this embodiment of the application, the touch sensor 180K can detect the user's click operation on the application icon, and pass the detected click operation to the application processor to determine that the click operation is used to start or run the application, and then execute the running operation of the application.

[0117] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0118] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0119] The above is a detailed description of the embodiments of this application using electronic device 100 as an example. It should be understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on electronic device 100. Electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0120] In addition, an operating system runs on top of the aforementioned components. Examples include iOS, Android (an open-source operating system), and Windows. Applications can be installed and run on this operating system.

[0121] The operating system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0122] Figure 2 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0123] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system libraries and Android runtime, and the kernel layer.

[0124] The application layer can include a series of application packages.

[0125] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, shopping, Bluetooth, music, video, and SMS.

[0126] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0127] like Figure 2 As shown, the application framework layer may include a window manager, content provider, view system, resource manager, notification manager, etc.

[0128] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0129] The content provider is used to store and retrieve data, and to make this data accessible to applications. The data may include monitoring data acquired by the sensor module 180 (such as acceleration data acquired by the accelerometer 180E), video, images, audio, phone calls made and received, browsing history and bookmarks, phone books, etc.

[0130] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0131] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0132] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and so on.

[0133] The notification manager allows applications to display notifications in the status bar. These can be used to convey informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify of download completion or message alerts (such as travel notifications). The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0134] In this embodiment of the application, the application framework layer may include a camera access interface, wherein the camera access interface is used to provide an application programming interface and a programming framework for camera applications.

[0135] The Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0136] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0137] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0138] The system layer can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0139] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0140] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0141] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0142] A 2D graphics engine is a graphics engine for 2D drawing.

[0143] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver. The camera driver is used to drive the camera to capture images.

[0144] Electronic devices often incorporate virtual assistants, which are interactive services implemented using artificial intelligence technology. These assistants can mimic the role of a human helper, quickly handling everyday tasks. Virtual assistants include voice assistants and large-scale assistants. Voice assistants utilize speech recognition technology to understand and respond to user voice commands, providing functions such as information retrieval, schedule management, and device control. Large-scale assistants can handle complex tasks and data, providing functions such as text generation, image recognition, and data analysis. These types of assistants can meet diverse user interaction needs. The virtual assistant discussed in this application can be any type, and this application makes no limitation on it.

[0145] In one example, a user can wake up the virtual assistant using the power button. See also Figure 3 This is a schematic diagram of a scenario where an electronic device wakes up a virtual assistant, as provided in an embodiment of this application.

[0146] like Figure 3 As shown in (a), when the electronic device displays interface 11, the user presses the power button 301 of the electronic device. In response to the user's operation, the electronic device activates the virtual assistant and displays interface 12, thus waking up the virtual assistant. Interface 11 can be a desktop interface. Interface 12 can include a window 302 and controls 303.

[0147] like Figure 3 As shown in (b), when the electronic device displays interface 12, the user slides down window 302. In response to this user action, the electronic device closes the virtual assistant and displays interface 11.

[0148] In another example, a user can wake up the virtual assistant with their voice. See also Figure 4 This is a schematic diagram of a scenario where an electronic device wakes up a virtual assistant, according to another embodiment of this application.

[0149] like Figure 4 As shown in (a), when the electronic device displays interface 11, the user says "YOYO" to the electronic device. In response to this voice command, the electronic device activates the virtual assistant and displays interface 12, thus activating the virtual assistant.

[0150] In some implementations, users can wake up the virtual assistant using a specific wake word. For example... Figure 4As shown in (a) above, the wake word is "YOYO". It should be noted that the wake word "YOYO" is only an example. The wake word can be the name of the virtual assistant in the electronic device, a statement containing the name of the virtual assistant (such as "Hey, YOYO"), or a wake word set by the user (such as "Little Assistant"). The wake word in this embodiment of the application is not specifically limited.

[0151] In other implementations, users can wake up the virtual assistant through natural dialogue. For example, if a user says "hello" to the electronic device, the device will activate the virtual assistant in response to this natural dialogue, thus waking up the virtual assistant. It should be noted that the natural dialogue used to wake up the virtual assistant can be set at the factory or can be set by the user. This application does not impose any specific limitations on this.

[0152] like Figure 4 As shown in (b), when the electronic device displays interface 12, the user says "close" to the electronic device. In response to the voice command, the electronic device closes the virtual assistant and displays interface 11.

[0153] In one implementation, the user can also enter the text "Close" in window 302 of interface 12. In response to the user's input, the electronic device recognizes the text entered by the user, closes the virtual assistant, and displays interface 11.

[0154] In another example, users can activate the virtual assistant via the navigation bar on the interface. See also Figure 5 This is a schematic diagram of a scenario where an electronic device wakes up a virtual assistant, according to another embodiment of this application.

[0155] like Figure 5 As shown in (a), the electronic device displays interface 11, which includes a navigation bar 501. When the user clicks or long-presses the navigation bar 501, the electronic device activates the virtual assistant and displays interface 12 in response to the user's operation, thus activating the virtual assistant.

[0156] like Figure 5 As shown in (b), the electronic device displays interface 12, which includes window 302. When the user enters "close" in window 302, the electronic device recognizes the entered text, closes the virtual assistant, and displays interface 11 in response to the user's input.

[0157] In one implementation, while the electronic device displays interface 12, the user can also say "close" to the electronic device. In response to this voice command, the electronic device closes the virtual assistant and displays interface 11.

[0158] It should be noted that, Figures 3 to 5Three different ways to wake up the virtual assistant are shown. In practical applications, other methods can also be used to wake up the virtual assistant. This application does not limit the specific methods.

[0159] After the virtual assistant is activated, the electronic device runs the virtual assistant and other applications in the foreground. In this embodiment, for ease of explanation, the applications running in the foreground simultaneously with the virtual assistant are referred to as background applications, such as... Figures 3 to 5 The desktop application shown is the background application, such as... Figures 9 to 15 The email application shown is the background application.

[0160] It is understandable that there can be one or more background applications. For example, when an electronic device displays application A and application B in split screen, if a virtual assistant is activated, the electronic device will run the virtual assistant, application A, and application B in the foreground. Application A and application B can both be referred to as background applications.

[0161] After activating the virtual assistant, the user can be prompted that the current focus of the operation is on the virtual assistant through interface changes or some interface elements.

[0162] In one implementation, such as Figures 3 to 5 As shown, after activating the virtual assistant, the electronic device switches from interface 11 to interface 12. The desktop in interface 12 is darker than the desktop in interface 11, while the areas of window 303 and control 302 in interface 12 remain bright. This contrast in brightness between the display areas corresponding to the virtual assistant and the background application in the interface serves to indicate to the user that the current focus is on the virtual assistant.

[0163] In another implementation, the virtual assistant's window may include an outer aperture to indicate to the user that the current focus is on the virtual assistant.

[0164] For example, see Figure 6 This is a schematic diagram of the window form of the virtual assistant provided in the embodiments of this application.

[0165] After waking up the virtual assistant, the following will be displayed: Figure 6 The interface shown in (a) includes a window 601. Window 601 includes an outer halo, which is a single color. The outer halo makes window 601 more prominent, indicating to the user that the current focus is on the virtual assistant.

[0166] After activating the virtual assistant, it can also display things like... Figure 6 The interface shown in (b) includes a window 601. Window 601 includes an outer halo, which may contain multiple colors. Using a multi-colored outer halo makes window 601 more prominent, indicating to the user that the current focus is on the virtual assistant.

[0167] It should be noted that, Figure 6 Only two examples of the outer aperture are shown. In practical applications, the outer aperture can also be other forms, such as other colors or other shapes. This application does not specifically limit this.

[0168] In another implementation, when the virtual assistant is first displayed after it is activated, animation effects can be added, such as highlighting the navigation bar 501 for a few seconds, or playing the flowing animation effect of the outer halo of window 601 several times, so as to prompt the user that the current focus of operation is on the virtual assistant.

[0169] It should be noted that, in addition to the above examples, other methods can be used to enhance the user's current focus, and this application embodiment does not specifically limit this.

[0170] The virtual assistant's display area may include virtual assistant controls, which provide services matching the current interface. When a user clicks a virtual assistant control, in response to the user's action, the virtual assistant can perform the task corresponding to the service indicated by the control.

[0171] by Figures 3 to 5 For example, with the background application being the desktop, the display area of ​​the virtual assistant can include the virtual assistant's control 303. The prompt text on control 303 can be "Organize Desktop" or "New Feature Introduction," etc. When the user clicks the "Organize Desktop" control 303, in response to the user's action, the virtual assistant organizes the icons on the desktop application. When the user clicks the "New Feature Introduction" control 303, in response to the user's action, the virtual assistant can display the text introducing the new feature in window 302.

[0172] In one implementation, the tooltip text of the virtual assistant's control can change according to changes in the background application. For example, ... Figures 3 to 5 As shown, when the background application is a desktop application, the tooltip text for the controls in the virtual assistant's display area that provide services matching the current interface is "Organize Desktop" and "New Feature Introduction". Figure 9 and Figure 10 As shown, when the background application is email, the tooltip text for the controls in the virtual assistant's display area that provide services matching the current interface is "Polish Article" and "Summary Summary".

[0173] In another implementation, when the interface of the same background application changes, the tooltip text of the virtual assistant's controls can change according to the change in the background application's interface. For example, ... Figure 11 As shown, when the background application interface switches from interface 34 to interface 35, the controls for "polishing the article" and "summary summary" disappear.

[0174] In another implementation, the tooltip text of the virtual assistant's controls can change according to changes in the user's input area. For example, ... Figure 12 As shown, when a user taps an area in the background application, the electronic device responds to that user action by... Figure 12 (a) in the middle switch to Figure 12 In (b), the tooltip text for the controls in the virtual assistant's display area that provide services matching the current interface changes from "polish the article" and "summarize the summary" to "navigate to this address" and "add this address".

[0175] It should be noted that the size of the virtual assistant's controls can be adjusted according to the amount of its prompt text.

[0176] By using controls in the virtual assistant's display area to provide services that match the current interface, quick services can be offered to users, thus improving the user experience.

[0177] In some implementations, after waking up the virtual assistant, the user can input a command to make the virtual assistant execute the task corresponding to that command.

[0178] See one example. Figure 7 This is a schematic diagram of a scenario where a virtual assistant performs tasks according to an embodiment of this application.

[0179] like Figure 7 As shown in (a), after waking up the virtual assistant, the electronic device displays interface 21, which includes a window 701 for the virtual assistant. When the user says "What are you doing?" to the electronic device, in response to the voice command, the electronic device displays interface 22, which includes a window 702 for the virtual assistant. The window 702 displays the message "I'm waiting for you to chat. Is there anything new you want to share with me today?" from the virtual assistant.

[0180] It should be noted that when the electronic device displays interface 22, the electronic device can also play the virtual assistant's reply message "I'm waiting for you to chat. Is there anything new you want to share with me today?", so that the user can interact with the virtual assistant by voice.

[0181] like Figure 7 As shown in (b), when the electronic device displays interface 22, the user says to the electronic device, "Help me search for today's weather." In response to this voice command, the electronic device displays interface 23. Interface 23 includes a virtual assistant window 703, in which the virtual assistant's reply is displayed: "Today's weather: sunny, 25℃, light breeze, suitable for wearing short sleeves."

[0182] It should be noted that when the electronic device displays interface 23, the electronic device can also play the message "Today's weather: sunny, 25℃, light breeze, suitable for wearing short sleeves" via voice, so that the user can interact with the virtual assistant via voice.

[0183] The virtual assistant's window shape can adaptively adjust based on the interactive content. For example... Figure 7 As shown in (a), when switching from interface 21 to interface 22, window 701 switches to window 702 accordingly. This means the virtual assistant's window height and display area increase, and additional prompts and operation controls are added. Figure 7 As shown in (b), when switching from interface 22 to interface 23, window 702 switches to window 703, that is, the height of the virtual assistant window increases and the display area increases.

[0184] It should be noted that users can also interact with the virtual assistant by entering text in the virtual assistant's window, which will not be described in detail in this application embodiment.

[0185] In other implementations, after waking up the virtual assistant, the user can also manually switch the window appearance of the virtual assistant.

[0186] See one example. Figure 8 This is a schematic diagram of the window mode switching of the virtual assistant provided in the embodiments of this application.

[0187] like Figure 8 As shown in (a), when the electronic device displays interface 22, the user slides up the top of the virtual assistant's window 702, and in response to the user's operation, the electronic device displays interface 23. Switching from interface 22 to interface 23, correspondingly, switching from window 702 to window 703, the window shape of the virtual assistant changes.

[0188] like Figure 8 As shown in (b), when the electronic device displays interface 22, the user slides down the virtual assistant window 702, and in response to the user's operation, the electronic device displays interface 21. Switching from interface 22 to interface 21, correspondingly, switching from window 702 to window 701, the window form of the virtual assistant changes.

[0189] like Figure 8 As shown in (c), when the electronic device displays interface 22, window 702 in interface 22 may include an exit control 801. When the user clicks the exit control 801, in response to the user's operation, the electronic device exits the virtual assistant and displays interface 24. Switching from interface 22 to interface 24, the virtual assistant window disappears accordingly.

[0190] It should be noted that users can also exit the virtual assistant by quickly swiping down. For example, when the electronic device displays interface 22, if the user quickly swipes down window 702, the electronic device will close the virtual assistant and display interface 24 in response to the user's action.

[0191] like Figure 7 and Figure 8 As shown, the window appearance of the virtual assistant can be switched according to user operations. For ease of explanation, in this embodiment, the interface state when the virtual assistant is not activated can be referred to as the zero state (or Bar state, unactivated state, dormant state, etc.), such as... Figures 3 to 5 Interface 11 and Figure 7 and Figure 8 Interface 24. The window state when the virtual assistant is first activated can be called a state (or first state, awakened state, etc.), such as Figures 1 to 3 The shape of window 302 shown in interface 12, and Figure 7 and Figure 8 The form of window 701 shown in interface 21. The form of the virtual assistant window after adaptive adjustment according to user operation can be called a two-state (or second state, dynamic panel state, etc.), such as Figure 7 and Figure 8 The shape of window 702 shown in interface 22 and the shape of window 703 shown in interface 23.

[0192] It should be noted that, from Figures 3 to 5 as well as Figures 7 to 8 As can be seen from the example, the virtual assistant's window state can switch between zero and one state, or between one state and two state, or from two state to zero state, depending on the user's operation.

[0193] In some application scenarios, if the user interacts with a background application after waking up the virtual assistant, the virtual assistant will end the session and the electronic device will close the virtual assistant.

[0194] For example, see Figure 9 This is a schematic diagram of a scenario where a virtual assistant ends a session, as provided in an embodiment of this application.

[0195] like Figure 9 As shown, the electronic device displays interface 31, in which the virtual assistant is in an awakened state. Interface 31 may include the virtual assistant's window 901 and control 902. When the user slides the display area of ​​the background application, in response to the user's operation, the electronic device closes the virtual assistant and displays interface 32. As shown in interface 32, the virtual assistant's window 901 and control 902 disappear.

[0196] like Figure 9 In the scenario shown, the virtual assistant and the background application do not support collaborative use, which affects the user experience.

[0197] In this application embodiment, a scheme is provided for virtual assistants to work together with other applications.

[0198] In one embodiment, the electronic device can automatically perform collaborative processing between the virtual assistant and other applications. In this embodiment, when application A is running in the foreground of the electronic device, if the virtual assistant is activated, and the user performs an operation on application A, the electronic device can automatically perform collaborative processing between the virtual assistant and application A. That is, the virtual assistant will not end the session, and both the virtual assistant and application A will run in the foreground.

[0199] For example, see Figure 10 This is a schematic diagram illustrating a scenario where the virtual assistant provided in this application is used in conjunction with other applications.

[0200] like Figure 10 As shown, the electronic device displays interface 31, at which time the virtual assistant is in an awake state. Interface 31 may include the virtual assistant's window 901 and control 902. When the user slides the display area of ​​the background application, in response to the user's operation, the electronic device performs collaborative processing between the virtual assistant and the background application. That is, the electronic device does not close the virtual assistant, so that the virtual assistant and the background application run simultaneously in the foreground, and the electronic device displays interface 32. As shown in interface 32, the virtual assistant's window 901 and control 902 are still displayed on the interface.

[0201] and Figure 9 Compared to the scenario shown, Figure 10 In the scenario shown, through the collaborative processing of the virtual assistant with other applications, after the virtual assistant is woken up in another application, it is also possible to operate on the page in that application again. That is, the background application and the virtual assistant can operate collaboratively, and for the updated page, the virtual assistant can provide service recommendations that match it, thereby improving the user experience when using the virtual assistant.

[0202] like Figure 10 In the scenario shown, when the virtual assistant is used in conjunction with other applications, the virtual assistant's window may interfere with the user's operation of the background application, thus affecting the user experience.

[0203] To address the aforementioned issues, in one implementation, the electronic device can reduce the impact of the virtual assistant's window on user operations by adjusting the window shape of the virtual assistant after it goes out of focus.

[0204] It should be noted that, in the embodiments of this application, if the user's focus switches from the background application to the virtual assistant, it can be said that the virtual assistant gains focus; if the user's focus switches from the virtual assistant to the background application, it can be said that the virtual assistant loses focus.

[0205] As an example of one-state defocusing, see Figure 11 This is a schematic diagram of a scenario where the virtual assistant window loses focus, as provided in the embodiments of this application.

[0206] like Figure 11 As shown in (a), the electronic device displays interface 31. At this time, the virtual assistant is awake and in focus. The virtual assistant's window is in a focused state, as shown by window 901 in interface 31. When the user slides the display area of ​​the background application, in response to the user's operation, the virtual assistant loses focus, and the electronic device displays interface 34. Interface 34 may include the virtual assistant's window 902, which is in a defocused state. At this time, the electronic device does not close the virtual assistant; the virtual assistant and the background application are running simultaneously in the foreground. However, comparing interfaces 31 and 34, the virtual assistant's window shape has changed; the window is shrunk, and the outer aperture is hidden.

[0207] It should be noted that, Figure 11 (a) shows one way in which the window shape of the virtual assistant changes when it loses focus. In practical applications, other ways of changing the window can also be used, such as minimizing it to an icon, collapsing it, or hiding it in the sidebar. This application does not specifically limit this.

[0208] Optionally, when the focus is on the background application, the virtual assistant's session does not end while the user is operating the background application. That is, the virtual assistant continues to run in the foreground and does not exit, and the virtual assistant's window can remain out of focus.

[0209] like Figure 11 As shown in (b), the electronic device displays interface 34. At this time, the virtual assistant is out of focus, and the virtual assistant window is in a one-state out-of-focus state, as shown in window 1101 in interface 34. When the user clicks the exit control 1102 of the background application, in response to the user's operation, the electronic device prioritizes the return operation of the background application and displays interface 35. Comparing interfaces 34 and 35, the interface of the background application has changed, but the virtual assistant window 1101 is still displayed on the interface, and its form has not changed; it is still in a one-state out-of-focus window state. At this time, the electronic device has not closed the virtual assistant, and the virtual assistant is still in an out-of-focus state.

[0210] However, due to the change in the background application's interface, the virtual assistant's control 902 changes accordingly, as shown in interface 35, where the virtual assistant's control 902 disappears.

[0211] In some implementations, to facilitate user operation, the shape of the virtual assistant's controls may remain unchanged during the virtual assistant's focus transitions, but it can change according to changes in the background application's interface. For example... Figure 11 As shown in (b), when the background application interface changes, the virtual assistant remains out of focus, the virtual assistant window remains out of focus, and the virtual assistant controls disappear.

[0212] For example Figure 12 This is a schematic diagram illustrating the control state switching provided in an embodiment of this application. For example... Figure 12 As shown in (a), the prompt text for the virtual assistant's control 1301 is "Polish the article" and "Summary Summary," and the virtual assistant is in a defocused state at this time. When the user clicks or long-presses the text in the background application, in response to the user's operation, the electronic device displays as shown in (a). Figure 12 The interface shown in (b) is as follows. Figure 12 As shown in (b), the prompt text of the virtual assistant control 1302 is "Navigate to this address" and "Add this address", that is, the prompt text of the virtual assistant control has changed, and correspondingly, the service indicated by the virtual assistant control has also changed.

[0213] If the virtual assistant is out of focus, the virtual window can regain focus if the user inputs commands or interacts with its window. In this case, the electronic device can adjust the virtual assistant's window appearance to indicate to the user that the current focus is on the virtual assistant.

[0214] As an example of one-state focus acquisition, see Figure 13 This is a schematic diagram of a scenario where the virtual assistant window is focused, as provided in the embodiments of this application.

[0215] like Figure 13 As shown, the electronic device displays interface 34. At this time, the virtual assistant is in a defocused state, and the virtual assistant's window is in a defocused state, such as the form of window 1101. When the user clicks on window 1101, enters text in window 1101, or speaks a voice command to the electronic device, the virtual assistant regains focus in response to the user's operation, and the electronic device displays interface 36. As shown in interface 36, the virtual assistant's window 1301 is displayed. Compared with window 1101, the form of window 1301 has changed, that is, the window becomes larger (such as returning to the size of the focused state), an outer aperture is added, and the navigation bar 1302 is highlighted. Through the change in the form of the virtual assistant's window, the user can be prompted to switch the current operation focus to the virtual assistant. Afterwards, the electronic device displays interface 31, which displays window 901. Compared with window 1301 in interface 36 during the display animation process, the navigation bar 1302 of window 901 is grayed out.

[0216] Optionally, when the virtual assistant refocuses, a motion effect can be displayed, such as playing a flowing motion effect of the outer aperture several times and / or highlighting the navigation bar for a few seconds.

[0217] In this embodiment, the virtual assistant's window changes from window 1101 in interface 34 to window 1301 in interface 36, and then from window 1301 in interface 36 to window 901 in interface 31. This change in the virtual assistant's window prompts the user to switch the current focus to the virtual assistant. Furthermore, animated effects enhance the prompts during the transition, improving the user experience.

[0218] It should be noted that when the virtual assistant changes from being unwakeable to being woken up, such as... Figure 3 (a) Figure 4 (a) and Figure 5 In the process shown in (a), the navigation bar in the virtual assistant's window can be displayed in grayscale. Since the virtual assistant's window changes from hidden to visible when it is activated, and the window includes an outer aperture, these display features can indicate to the user that the virtual assistant has been activated. Therefore, in this case, visual cues via a highlighted navigation bar are unnecessary. However, when the virtual assistant refocuses, as shown in (a), the navigation bar appears to be in grayscale. Figure 13 and Figure 15 As shown in the process, since the virtual assistant's window remains displayed during this process, only its appearance changes, the navigation bar in the virtual assistant's window can be highlighted for a period of time and then displayed in grayscale to enhance the prompt for the virtual assistant to refocus. Of course, it is understood that other methods can be used in practical applications, and this application embodiment does not specifically limit this.

[0219] As an example of two-state defocusing, see Figure 14 This is a schematic diagram of a virtual assistant window losing focus, provided in another embodiment of this application.

[0220] like Figure 14 As shown in (a), the electronic device displays interface 41. At this time, the virtual assistant is in focus, and the virtual assistant window is in a two-state focused state, as shown in window 1401 on interface 41. When the user slides the display area of ​​the background application, in response to the user's operation, the virtual assistant loses focus, and the electronic device displays interface 42. Interface 42 may include the virtual assistant window 1402, which is in a two-state out-of-focus state. At this time, the electronic device does not close the virtual assistant, and the virtual assistant and the background application are running in the foreground simultaneously. However, comparing interface 41 and interface 42, the window form of the virtual assistant has changed, that is, the outer aperture is hidden.

[0221] In this implementation, when the virtual assistant's window is in a binary state, it indicates that interaction has occurred between the user and the virtual assistant. In this case, the interaction information can be retained, i.e., the binary state of the virtual assistant's window can be preserved, which facilitates subsequent user operations. Furthermore, hiding the outer halo of the binary window can prompt the user to switch the current focus to the background application.

[0222] It should be noted that, Figure 14 (a) shows one way in which the window shape of the virtual assistant changes when it is out of focus in two states. In practical applications, other ways of changing the window can also be used, such as shrinking the window or switching to one state. This application does not specifically limit this.

[0223] Optionally, when the focus is on the background application, the virtual assistant's session does not end while the user is operating the background application. That is, the virtual assistant continues to run in the foreground and does not exit, and the virtual assistant's window can remain out of focus.

[0224] like Figure 14 As shown in (b), the electronic device displays interface 42. At this time, the virtual assistant is out of focus, and the virtual assistant window is in a two-state out-of-focus state, as shown in window 1402 on interface 42. When the user clicks the exit control 1403 of the background application, in response to the user's operation, the electronic device prioritizes the background application's return operation and displays interface 43. Comparing interface 42 and interface 43, the background application's interface has changed, but the virtual assistant window 1402 is still displayed on the interface, and its form has not changed; it remains in a two-state out-of-focus state. At this time, the electronic device has not closed the virtual assistant, and the virtual assistant is still in an out-of-focus state.

[0225] Understandable Figure 11 and Figure 14 These are the out-of-focus forms of a one-state window and a two-state window, respectively. The execution principle is the same for both, only the window forms of the one-state window and the two-state window are different.

[0226] If the virtual assistant is out of focus, the virtual window can regain focus if the user inputs commands or interacts with its window. In this case, the electronic device can adjust the virtual assistant's window appearance to indicate to the user that the current focus is on the virtual assistant.

[0227] As an example of two-state focusing, see Figure 15 This is a schematic diagram of a virtual assistant window being focused, provided in another embodiment of this application.

[0228] like Figure 15As shown, the electronic device displays interface 42. At this time, the virtual assistant is in a defocused state, and the virtual assistant's window is in a two-state defocused state, such as the form of window 1402. When the user clicks on window 1402, the user enters text in window 1402, or the user speaks a voice command to the electronic device, in response to the user's operation, the virtual assistant regains focus, and the electronic device displays interface 44. As shown in interface 44, the virtual assistant's window 1501 is displayed. Compared with window 1402, the form of window 1501 has changed, namely, an outer aperture has been added, and the navigation bar 1502 is highlighted. Through the change in the form of the virtual assistant's window, the user can be prompted to switch the current operation focus to the virtual assistant. Afterwards, the electronic device displays interface 41, and interface 41 displays window 1401. Compared with window 1501 in interface 44 during the display animation process, the navigation bar 1502 of window 1401 is grayed out.

[0229] Optionally, when the virtual assistant refocuses, a motion effect can be displayed, such as playing a flowing motion effect of the outer aperture several times and / or highlighting the navigation bar for a few seconds.

[0230] In this embodiment, the window 1402 in interface 42 changes to window 1501 in interface 44, and then the window 1501 in interface 44 changes back to window 1401 in interface 41. The change in the virtual assistant's window prompts the user to switch the current focus to the virtual assistant.

[0231] like Figure 11 and Figure 14 As shown in the scenario, when the virtual assistant is active, if the user interacts with a background application, the electronic device can automatically perform collaborative processing between the virtual assistant and the background application. This means the virtual assistant doesn't end the session, and both the virtual assistant and the background application run in the foreground. Furthermore, the electronic device can automatically adjust the virtual assistant's window appearance, such as shrinking the window, to reduce its impact on user operations. Additionally, the electronic device can further enhance the user experience by automatically adjusting the virtual assistant's window appearance, such as hiding the outer aperture, to indicate to the user that the current focus is on the background application.

[0232] like Figure 13 and Figure 15 As shown in the scenario, when an electronic device is performing collaborative processing between a virtual assistant and a background application—that is, when both the virtual assistant and the background application are running in the foreground—if the virtual assistant refocuses from a defocused state, the electronic device can automatically adjust the virtual assistant's window appearance, such as restoring its size, to increase the user's operating area. Furthermore, the electronic device can also automatically adjust the virtual assistant's window appearance, such as displaying a dynamic effect of the window's outer aperture or highlighting the navigation bar, to prompt the user that the current focus has returned to the virtual assistant, thereby improving the user experience.

[0233] In another embodiment, the user can choose whether to perform collaborative processing between the virtual assistant and other applications. In this embodiment, when application A is running in the foreground of the electronic device, after waking up the virtual assistant, the user can manually choose whether to have the virtual assistant and the background application run simultaneously in the foreground of the electronic device.

[0234] As an example of a user choosing to opt out of the virtual assistant, see Figure 16 This is a schematic diagram of the exit scenario of the virtual assistant provided in the embodiments of this application.

[0235] like Figure 16 As shown, the electronic device displays interface 31, at which point the virtual assistant is in focus. When the user clicks on a blank area in interface 31, in response to the user's action, the electronic device closes the virtual assistant and displays interface 32. As shown in interface 32, the virtual assistant's window 901 and control 902 disappear.

[0236] It should be noted that the blank area in the above examples refers to the part of the background application's display area where no controls or content are displayed. Besides the examples above, in some implementations, users can also exit the virtual assistant using physical buttons (such as volume buttons, power buttons, etc.). In other implementations, users can also exit the virtual assistant using preset gestures (such as tapping the screen three times consecutively or drawing a preset shape on the screen). This application does not specifically limit these methods.

[0237] As an example of user-selected collaborative processing, see Figure 17 This is a schematic diagram of a persistent scenario for the virtual assistant provided in this application embodiment.

[0238] like Figure 17 As shown, the electronic device displays interface 31. At this time, the virtual assistant is in focus. Interface 31 displays the virtual assistant's window 901 and controls 902. Window 901 includes a navigation bar 1302. When the user clicks or long-presses the navigation bar 1302, in response to the user's action, the electronic device displays interface 37. Interface 37 may include controls 1701. At this time, controls 1701 are in their first display state, indicating that the virtual assistant is not currently in a persistent state, prompting the user to choose to keep the virtual assistant persistent. Keeping the virtual assistant persistent means that the virtual assistant is not closed, and the virtual assistant and the background application run simultaneously in the foreground.

[0239] When a user clicks control 1701, in response to the user's action, the electronic device performs collaborative processing between the virtual assistant and the background application, displaying interface 38. Interface 38 may include a floating window 1702 for the virtual assistant, which may include a control 902 for the virtual assistant and a navigation bar 1703. The navigation bar 1703 may be highlighted to indicate to the user that the virtual assistant is currently in a persistent state, meaning that the virtual assistant and the background application are running simultaneously in the foreground.

[0240] Optionally, the virtual assistant's floating window 1701 can adaptively adjust its size based on the user's interaction with the virtual assistant.

[0241] It should be noted that, Figure 17 The image shows the persistent scene of the virtual assistant window in a one-state condition, and the persistent scene of the virtual assistant window in a two-state condition. Figure 17 The scenarios shown are the same, and will not be described again in the embodiments of this application.

[0242] For example, Figure 18 When interface 32 is designated as the first interface, Figure 17 Interface 31 is designated as the second interface. The first application can be email, and the second application can be a virtual assistant. Window 901 of the virtual assistant can be the second window, and navigation bar 1302 can be the first navigation bar. Interface 37 can be the third interface, control 1701 can be the first control, and the third operation can be clicking or long-pressing the first control. Interface 38 can be the fourth interface, and window 1702 can be the third window.

[0243] Figure 17 In the illustrated embodiment, by clicking the navigation bar in the virtual assistant window, the user can trigger the virtual assistant's persistent setting control (control 1701). By manipulating the persistent setting control, the virtual assistant can be set to a persistent state. This method allows users to customize the virtual assistant window, enabling it to adapt to various application scenarios and thus improving the user experience.

[0244] As another example of user-selected collaborative processing, see Figure 18 This is a schematic diagram of a persistent scenario for a virtual assistant provided in another embodiment of this application.

[0245] like Figure 18As shown in (a), the electronic device displays 31, at which point the assistant is in focus. When the user drags the virtual assistant's window 901 upwards, in response to the user's action, the electronic device performs collaborative processing between the virtual assistant and the background application, displaying interface 39. Interface 39 may include a floating window 1702 for the virtual assistant and a control 1801. Control 1801 represents a second display state, indicating that the virtual assistant is currently in a persistent state, prompting the user to choose to de-persist the virtual assistant.

[0246] In some implementations, if a user clicks control 1801, in response to the user's action, the electronic device displays interface 31, i.e., the virtual assistant's window, and returns to a previous state.

[0247] In some implementations, if the user does not operate or clicks any position on the screen other than control 1801 within a preset time after the electronic device switches from interface 31 to interface 39, the electronic device hides control 1801 and keeps the floating window 1702 of the virtual assistant displayed. That is, the electronic device performs collaborative processing of the virtual assistant and the background application by default, and the virtual assistant and the background application run in the foreground at the same time.

[0248] In other implementations, after the electronic device hides the control 1801, the user can manually click to re-expand the control 1801. For example, after the electronic device hides the control 1801, if the user clicks / drags the floating window 1702, or clicks / long-presses the navigation bar 1703 in the floating window 1702, the electronic device will redisplay the control 1801.

[0249] In this embodiment of the application, the virtual assistant's persistent state setting control enables users to independently set the persistent state of the virtual assistant, thereby improving the user's flexibility and experience when using the virtual assistant.

[0250] As shown in interface 39, the virtual assistant's floating window 1702 may include a cancel control 1704 and a zoom control 1705. The cancel control 1704 is used to close the virtual assistant. The zoom control 1705 is used to zoom in on the floating window 1702.

[0251] In this embodiment, a cancel control is provided, allowing users to quickly close the virtual assistant by activating it, thus improving the user experience. A zoom control is also provided, allowing users to display the virtual assistant window in full screen, further enhancing the user experience.

[0252] like Figure 18 As shown in (b), when the user clicks the cancel control 1704 of the floating window 1702 in interface 39, the electronic device closes the virtual assistant and displays interface 32 in response to the user's operation.

[0253] In some implementations, when a user clicks the magnification control 1705 of the floating window 1702 in interface 39, the electronic device displays the floating window 1702 in full screen in response to the user's operation.

[0254] For example, Figure 18 Interface 39 can be referred to as the fifth interface, and window 1702 can be referred to as the third window.

[0255] In this way, users can choose whether to keep the virtual assistant running, enabling it to adapt to various application scenarios and thus improve the user experience.

[0256] In scenarios where a virtual assistant is always active, there are various forms of controls used to set the virtual assistant to be always active. Below are some examples of possible implementations.

[0257] In one implementation, see Figure 19 This is a schematic diagram of a persistent scenario for a virtual assistant provided in another embodiment of this application.

[0258] like Figure 19 Figure (a) shows a scenario illustration of setting a virtual assistant to be always active through user operation. This process is similar to... Figure 17 The process shown is based on the same principle, and will not be repeated here.

[0259] and Figure 17 Compared to the process shown, Figure 19 In interface 51 shown in (a), control 1901 and Figure 17 The controls 1701 in interface 37 have the same function (both are used to set the virtual assistant's persistent state), but they are different in form (the control icons and prompt text are different).

[0260] like Figure 19 Figure (b) illustrates a scenario where the virtual assistant is deactivated through user interaction. This process is similar to... Figure 18 The process shown in (a) is based on the same principle and will not be repeated here.

[0261] and Figure 18 Compared to the process shown in (a) above, Figure 19 In interface 52 shown in (b), control 1902 and Figure 18 The control 1801 in interface 39 shown in (a) has the same function (both are used to cancel the virtual assistant's persistent state), but the forms are different (the control icons and prompt text are different).

[0262] Understandable Figure 17 Control 1701 and Figure 18The control 1801 matches in the table, and its form, icon, etc. are consistent. Figure 19 Control 1901 in (a) and Figure 19 The control 1902 in (b) matches in form, icon and other similarities.

[0263] In another implementation, see Figure 20 This is a schematic diagram of a persistent scenario for a virtual assistant provided in another embodiment of this application.

[0264] like Figure 20 Figure (a) shows a scenario illustration of setting a virtual assistant to be always active through user operation. This process is similar to... Figure 17 The process shown is based on the same principle, and will not be repeated here.

[0265] and Figure 17 Compared to the process shown, Figure 20 In interface 61 shown in (a), control 2001 and Figure 17 The control 1701 in interface 37 shown has the same function, only the form is different.

[0266] like Figure 20 Figure (b) illustrates a scenario where the virtual assistant is deactivated through user interaction. This process is similar to... Figure 18 The process shown in (a) is based on the same principle and will not be repeated here.

[0267] and Figure 18 Compared to the process shown in (a) above, Figure 20 In interface 52 shown in (b), control 2002 and Figure 18 The controls 1801 in interface 39 shown in (a) have the same function but different forms.

[0268] Understandable Figure 20 Control 2001 in (a) and Figure 20 The control in (b) matches the 2002 version, with identical form, icon, etc. It is consistent with... Figure 17 compared to, Figure 20 The controls used to set the persistent state of the virtual assistant do not include prompt text.

[0269] In another implementation, see Figure 21 This is a schematic diagram of a persistent scenario for a virtual assistant provided in another embodiment of this application.

[0270] like Figure 21 Figure (a) shows a scenario illustration of setting a virtual assistant to be always active through user operation. This process is similar to... Figure 17 The process shown is based on the same principle, and will not be repeated here.

[0271] and Figure 17Compared to the process shown, Figure 21 In interface 71 shown in (a), control 2101 and Figure 17 The control 1701 in interface 37 shown has the same function, only the form is different.

[0272] like Figure 21 Figure (b) illustrates a scenario where the virtual assistant is deactivated through user interaction. This process is similar to... Figure 18 The process shown in (a) is based on the same principle and will not be repeated here.

[0273] and Figure 18 Compared to the process shown in (a) above, Figure 21 In interface 72 shown in (b), control 2102 and Figure 18 The controls 1801 in interface 39 shown in (a) have the same function but different forms.

[0274] Understandable Figure 21 Control 2101 in (a) and Figure 21 The control 2102 in (b) matches in form, icon, etc. Figure 19 compared to, Figure 20 The controls used to set the persistent state of the virtual assistant do not include prompt text, but include a cancel button 2103 and a zoom button 2102.

[0275] In some implementations, when a user clicks the zoom button 2102 on interface 71, the electronic device displays a floating window 1702 of the virtual assistant in full screen in response to the user's operation.

[0276] In some implementations, when a user clicks the cancel button 2103 on interface 71, the electronic device closes the virtual assistant and displays interface 32 in response to the user's operation.

[0277] Figure 21 In the scenario shown, the control used to set the persistent state of the virtual assistant includes a number of control buttons, which allows users to quickly set up the virtual assistant, thereby improving the user experience.

[0278] It should be noted that in some implementations, the form of the control used to set the persistent state of the virtual assistant can be consistent with the user's usage habits of the electronic device. For example, the icon used for the control to set the persistent state of the virtual assistant can be consistent with the icons used in other applications or settings on the electronic device. This allows users to adapt to the persistent setting of the virtual assistant more quickly, thereby improving the user experience.

[0279] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0280] This application also provides a computer-readable storage medium storing a computer program that, when run on an electronic device, can implement the steps in the above-described method embodiments.

[0281] This application also provides a computer program product that, when run on an electronic device or a wireless router, enables the electronic device to perform the steps described in the various method embodiments above.

[0282] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to the first device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0283] This application also provides a chip, which includes a processor coupled to a memory. The processor calls a computer program stored in the memory to implement the steps of any method embodiment of this application. The chip can be a single chip or a chip module composed of multiple chips.

[0284] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0285] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0286] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for displaying an interface, characterized in that, Applied to electronic devices, including: Display a first interface, the first interface including a first window of the first application; In response to a user's first operation to wake up the second application, a second interface is displayed, the second interface including the first window and a second window of the second application, the second window being displayed floating on the first window; the second window includes a first navigation bar; In response to a second user action on the first navigation bar, a third interface is displayed; the third interface includes the first window, the second window, and the first control. In response to a third operation by the user on the first control, a fourth interface is displayed; the fourth interface includes the first window and a third window of the second application; wherein the third window has a different form than the first window.

2. The method according to claim 1, characterized in that, The method further includes: In response to a fourth user action on the second window, a fifth interface is displayed; the fifth interface includes the first window and the third window of the second application.

3. The method according to claim 1 or 2, characterized in that, The fifth interface also includes a second control; the method further includes: In response to the user's fifth operation on the second control, the second interface is displayed.

4. The method according to any one of claims 1 to 3, characterized in that, The third window includes a third control; the method further includes: In response to the user's sixth operation on the third control, the first interface is displayed.

5. The method according to any one of claims 1 to 4, characterized in that, The third window includes a fourth control; the method further includes: In response to the user's seventh operation on the fourth control, a sixth interface is displayed; the third window is displayed in full screen on the sixth interface.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: In response to the user's eighth operation on the first window in the second interface, a seventh interface is displayed; the seventh interface includes the first window and a fourth window of the second application, the fourth window having a different form from the second window.

7. The method according to claim 6, characterized in that, The first window in the seventh interface displays a first screen, and the seventh interface includes a fifth control, which is used to trigger a task matching the first screen; the method further includes: In response to the user's ninth operation on the first window, the first screen of the first window in the seventh interface is switched to the second screen; The fifth control is updated to a sixth control, which is used to trigger a task that matches the second screen.

8. The method according to claim 7, characterized in that, The method further includes: In response to the user's ninth operation on the content displayed in the first window of the second interface, the fifth control is updated to a seventh control, which is used to trigger a task that matches the display content pointed to by the ninth operation.

9. The method according to any one of claims 6 to 8, characterized in that, The method further includes: In response to the user's tenth operation on the fourth window, the second interface is displayed.

10. The method according to any one of claims 6 to 9, characterized in that, The method further includes: In response to the user's tenth operation on the fourth window, the first animation effect of the fourth window is played at the first moment; At a second moment, the second interface is displayed; wherein the second moment is after the first moment.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: In response to the user's eleventh action on the second interface, the first interface is displayed.

12. The method according to any one of claims 6 to 10, characterized in that, The fourth window is smaller than the second window.

13. An electronic device, characterized in that, include: One or more processors; One or more memory units; The memory stores a computer program that, when executed by the processor, causes the electronic device to perform the method as described in any one of claims 1 to 12.

14. A chip system, characterized in that, The chip system includes a processor coupled to a memory, the processor being configured to run a computer program stored in the memory to implement the method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by one or more processors, implements the method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that, The computer program product includes computer program code that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 12.

Images