Interface display method and electronic equipment

Abstract

The invention provides an interface display method and electronic equipment. The method can be applied to the electronic equipment, the electronic equipment comprises a display screen, the display screen has a first refresh rate, and the method comprises the steps that in response to a first instruction, a first window is displayed on the display screen, a first interface is displayed in the first window, the first instruction is used for creating a first virtual screen, and the first virtual screen has the first refresh rate; the first interface is mapped to a first window from an interface of a first virtual screen, the first virtual screen corresponds to first extended display identification data EDID, the first EDID is different from at least one second EDID historically stored in the electronic equipment, and each second EDID corresponds to external equipment historically connected or a second virtual screen historically created. According to the interface display method disclosed by the embodiment of the invention, in the process that the electronic equipment displays the virtual screen interface based on the window created by the display screen, the problem that the display screen flashes black can be avoided, and the use experience of a user is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an interface display method and an electronic device. Background Technology

[0002] Electronic devices can enable multi-screen collaborative work by expanding the desktop workspace of the primary display from the secondary display (such as a physical screen or a virtual screen) during use.

[0003] The operating system of electronic devices (such as Windows) typically has a display connection configuration memory function, that is, the operating system has the ability to restore the historical display configuration of the main display based on the same display interface and Extended Display Identification Data (EDID).

[0004] In multi-screen collaboration on an electronic device, if the main display A's refresh rate is P1, and the device creates a virtual display B for the first time, setting its refresh rate to P2 (P2 can be the same as or different from P1), the operating system will use the display connection configuration memory function to store the main display A's refresh rate P1 and the virtual display's refresh rate P2 as historical display configurations. If the device then changes the main display A's refresh rate to P3 (P3 is different from P1), and subsequently creates another virtual display B, the operating system, recognizing the EDID corresponding to virtual display B, will use the historical display configuration to set the main display A's refresh rate to P1 and the virtual display B's refresh rate to P2. This inconsistency between the main display A's refresh rate P1 and refresh rate P3 will cause the main display A to experience screen flickering. Summary of the Invention

[0005] This application provides an interface display method and an electronic device that can avoid screen flickering during the process of displaying a virtual screen interface in a window created based on the display screen, thereby improving the user experience.

[0006] In a first aspect, embodiments of this application provide an interface display method applied to an electronic device. The electronic device includes a display screen having a first refresh rate. The method includes: in response to a first instruction, displaying a first window on the display screen and displaying a first interface in the first window, wherein the first instruction is used to create a first virtual screen, the first virtual screen has a first refresh rate, the first interface is mapped from the interface of the first virtual screen to the first window, the first virtual screen corresponds to a first extended display identification data EDID, the first EDID is different from at least one second EDID historically stored in the electronic device, each second EDID corresponds to a historically connected external device, or a historically created second virtual screen.

[0007] The interface display method provided in this application embodiment can set a new EDID for the currently created virtual screen each time the electronic device creates a virtual screen, which is different from the historically stored EDID. In this way, the electronic device avoids the problem of the screen flickering due to the inconsistency between the historical refresh rate and the current refresh rate caused by obtaining the historical display configuration corresponding to the historically stored EDID based on the display connection configuration memory function and resetting the current refresh rate of the display based on the historical refresh rate in the historical display configuration, thus improving the user experience.

[0008] In one implementation, before displaying a first window on the display screen in response to a first instruction, and before displaying a first interface in the first window, the method further includes: running a virtual assistant application and displaying a first program interface in the virtual assistant application; displaying first response information in response to a program installation instruction entered in the first program interface, wherein the first response information includes at least one program installation control corresponding to the program installation instruction; and generating a first instruction in response to a first operation on a target program installation control among the at least one program installation controls, wherein the first instruction is further used to execute a first installation process based on a first virtual screen, and the first installation process is used to install a first installation program corresponding to the target program installation control. This implementation illustrates an application scenario where an electronic device displays a virtual screen interface on a display screen. In this scenario, the virtual assistant application can automatically create a first virtual screen and execute a first installation process based on the first instruction. Thus, the virtual assistant application can automatically install applications without affecting other user operations during the application installation process, improving the user experience. Furthermore, in this scenario, the electronic device does not reset the current refresh rate of the display screen based on its historical refresh rate, avoiding screen flickering issues.

[0009] In one implementation, before displaying a first window on the display screen in response to a first instruction, and before displaying a first interface in the first window, the method further includes: running a virtual assistant application and displaying a first program interface in the virtual assistant application; displaying second response information in response to a program uninstallation instruction input in the first program interface, wherein the second response information includes at least one program uninstallation control corresponding to the program uninstallation instruction; and generating a first instruction in response to a second operation on a target program uninstallation control in the at least one program uninstallation control, wherein the first instruction is further used to execute a first uninstallation process based on the first virtual screen, and the first uninstallation process is used to uninstall the first uninstallation program corresponding to the target program uninstallation control. This implementation demonstrates another application scenario where an electronic device displays a virtual screen interface on a display screen. In this scenario, the virtual assistant application can automatically create a first virtual screen and execute a first uninstallation process based on the first instruction. Thus, the virtual assistant application can automatically uninstall the application without affecting other user operations during the application uninstallation process, improving the user experience. Furthermore, in this scenario, the electronic device does not reset the current refresh rate of the display screen based on its historical refresh rate, avoiding screen flickering issues.

[0010] In one implementation, responding to a first instruction, a first window is displayed on the screen, and a first interface is displayed in the first window. This includes: responding to the first instruction, obtaining the first resolution and first refresh rate of the screen; initializing the screen configuration information corresponding to the first virtual screen to configure the first resolution, first refresh rate, and first EDID in the screen configuration information; creating the first virtual screen based on the configured screen configuration information; displaying the first window on the screen, and mapping the interface of the first virtual screen to the first window to display the first interface in the first window. By using this implementation, by setting a different first EDID each time the screen configuration information is set, the electronic device creates a new first virtual screen by default, without resetting the current refresh rate of the screen based on the screen's historical refresh rate, thus avoiding the screen flickering problem.

[0011] In one implementation, in response to a first instruction, obtaining the first resolution and first refresh rate of the display screen includes: in response to a first instruction sent by a virtual assistant application to a window application, the window application obtains first display information of the display screen, the first display information including the first resolution and the first refresh rate; the window application sends the first display information to a virtual screen display driver; and the virtual screen display driver obtains the first resolution and first refresh rate based on the first display information. This implementation demonstrates a specific method by which an electronic device maintains consistency between the resolution and refresh rate of the display screen and the resolution and refresh rate of the virtual screen. This avoids the virtual screen triggering changes in the resolution and refresh rate of the display screen during subsequent mode switching, thus preventing the display screen from flickering black.

[0012] In one implementation, the screen configuration information corresponding to the first virtual screen is initialized to configure a first resolution, a first refresh rate, and a first EDID. This includes: the virtual screen display driver configuring the screen configuration information based on the obtained first resolution and first refresh rate; and the virtual screen display driver modifying its built-in EDID table to generate a first EDID and configuring the screen configuration information based on the first EDID. This implementation allows the electronic device to set a new EDID for each newly created virtual screen, different from the historically stored EDIDs. This avoids the problem of screen flickering caused by the inconsistency between the historical and current refresh rates, which arises from retrieving the historical display configuration corresponding to the historical EDID based on the display connection configuration memory function and then resetting the current refresh rate of the display based on the historical refresh rate in the historical display configuration. This improves the user experience.

[0013] In one implementation, creating a first virtual screen based on configured screen configuration information includes: in a preset screen creation mode, the virtual screen display driver creates the first virtual screen based on the configured screen configuration information. The preset screen creation mode includes an extended screen mode or a duplicated screen mode. In the extended screen mode, the display screen and the first virtual screen have different display content; in the duplicated screen mode, the display screen and the first virtual screen have the same display content. Using this implementation, the electronic device can support multiple preset screen creation modes to ensure that no screen flickering occurs when creating a virtual screen in any preset screen creation mode.

[0014] In one implementation, after creating the first virtual screen based on the configured screen configuration information, the method further includes: if the preset screen creation mode is extended screen mode, responding to a first mode switching command, the virtual screen display driver switches the first virtual screen from extended screen mode to duplicate screen mode; if the preset screen creation mode is duplicate screen mode, responding to a second mode switching command, the virtual screen display driver switches the first virtual screen from duplicate screen mode to extended screen mode. Using this implementation, the electronic device can support switching between multiple virtual screen creation modes, so that the display screen will not flicker black during any virtual screen creation mode switching process.

[0015] In one implementation, after creating the first virtual screen based on the configured screen configuration information, the process further includes: the virtual screen display driver sending a first message to the Windows application, indicating that the first virtual screen has been successfully created; and the Windows application sending a second message to the virtual assistant application, also indicating that the first virtual screen has been successfully created. This implementation illustrates the specific way the virtual screen display driver and the virtual assistant application interact. In this way, the virtual assistant application can determine that the first virtual screen has been successfully created and then execute subsequent installation or uninstallation processes to achieve automatic installation or uninstallation of the application.

[0016] In one implementation, after displaying a first window on the screen in response to a first instruction, and displaying a first interface within the first window, the method further includes: if a first instruction is generated based on a first operation on the target program installation control, displaying a first installation interface corresponding to the first installation program on the first interface; performing a first simulated click operation on the first installation interface based on a user interface agent (UI Agent); and completing a first installation process based on the first simulated click operation to install the first installation program on the electronic device. Using this implementation, the electronic device can automatically install applications, improving the user experience.

[0017] In one implementation, after displaying a first window on the screen in response to a first instruction, and displaying a first interface within the first window, the method further includes: if a first instruction is generated based on a second operation on the target program uninstallation control, displaying a first uninstallation interface corresponding to the first uninstallation program on the first interface; performing a second simulated click operation on the first uninstallation interface based on a user interface agent (UI Agent); and completing the first uninstallation process based on the second simulated click operation to uninstall the first uninstallation program from the electronic device. Using this implementation, the electronic device can automatically uninstall applications, improving the user experience.

[0018] In one implementation, displaying a first installation interface corresponding to the first installer on a first virtual screen includes: in response to a second message, the virtual assistant application sends a third message to the window application, the third message triggering the window application to execute the first installation process; in response to the third message, the window application executes the first installation process to capture the first installation interface based on a hook mechanism; and the window application sends the first installation interface to the virtual assistant application. This implementation illustrates a specific method by which the virtual assistant application obtains the first installation interface. Thus, after obtaining the first installation interface, the virtual assistant application can execute subsequent installation processes to achieve automatic application installation and improve the user experience.

[0019] In one implementation, a first simulated click operation is performed on the first installation interface based on a user interface agent (UI Agent). This includes: upon receiving the first installation interface, the virtual assistant application performs the first simulated click operation on the first installation interface based on the UI Agent. Using this implementation, the virtual assistant application can automatically execute the installation process based on the UI Agent after obtaining the first installation interface, thereby achieving automatic application installation and improving the user experience.

[0020] In one implementation, displaying a first uninstallation interface corresponding to a first uninstaller on a first virtual screen includes: in response to a second message, the virtual assistant application sends a fourth message to the window application, the fourth message triggering the window application to execute the first uninstallation process; in response to the fourth message, the window application executes the first uninstallation process to capture the first uninstallation interface based on a hook mechanism; and the window application sends the first uninstallation interface to the virtual assistant application. This implementation demonstrates a specific method by which the virtual assistant application obtains the first uninstallation interface. Thus, after obtaining the first uninstallation interface, the virtual assistant application can execute subsequent uninstallation processes to achieve automatic uninstallation of the application and improve the user experience.

[0021] In one implementation, a second simulated click operation is performed on the first uninstallation interface based on a user interface agent (UI Agent). This includes: upon receiving the first uninstallation interface, the virtual assistant application performs the second simulated click operation on the first uninstallation interface based on the UI Agent. Using this implementation, the virtual assistant application can automatically execute the uninstallation process based on the UI Agent after obtaining the first uninstallation interface, thereby achieving automatic application uninstallation and improving the user experience.

[0022] Secondly, embodiments of this application provide an interface display device applied to an electronic device. The electronic device includes a display screen with a first refresh rate. The device includes a display module, which is configured to display a first window on the display screen and a first interface in the first window in response to a first instruction. The first instruction is used to create a first virtual screen, which has a first refresh rate. The first interface is mapped from the interface of the first virtual screen to the first window. The first virtual screen corresponds to a first extended display identification data EDID. The first EDID is different from at least one second EDID historically stored in the electronic device. Each second EDID corresponds to a historically connected external device or a historically created second virtual screen.

[0023] The interface display device provided in this application embodiment can set a new EDID for the currently created virtual screen each time the electronic device creates a virtual screen, which is different from the historically stored EDID. In this way, the electronic device avoids the problem of the screen flickering black caused by the inconsistency between the historical refresh rate and the current refresh rate, which occurs when the historical refresh rate is not consistent with the current refresh rate, by obtaining the historical display configuration corresponding to the historically stored EDID based on the display connection configuration memory function and resetting the current refresh rate of the display based on the historical refresh rate in the historical display configuration. This improves the user experience.

[0024] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device performs the interface display method provided by the first aspect and any possible implementation thereof.

[0025] Fourthly, embodiments of this application provide a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform an interface display method as provided in the first aspect and any possible implementation thereof.

[0026] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the interface display method provided by the first aspect and any possible implementation thereof.

[0027] It is understood that the beneficial effects that the technical solutions provided in the third to fifth aspects above can achieve can be referred to the beneficial effects in the first aspect and any of its possible implementations, and will not be repeated here. Attached Figure Description

[0028] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of a multi-screen collaborative work scenario; Figure 2 This is a schematic diagram of an abnormal scenario in multi-screen collaborative work. Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the software structure of the electronic device provided in the embodiments of this application; Figure 5 This is the first flowchart of the interface display method provided in the embodiments of this application; Figure 6 This is a schematic diagram of the first scene of the electronic device displaying the first window provided in the embodiments of this application; Figure 7 This is an interactive schematic diagram of the electronic device displaying the first response information provided in the embodiments of this application; Figure 8 This is the first interactive diagram in the process of the electronic device displaying the first window provided in the embodiments of this application; Figure 9 This is the second interactive diagram in the process of the electronic device displaying the first window provided in the embodiments of this application; Figure 10 This is an interactive diagram of the pre-configured window application for an electronic device provided in the embodiments of this application; Figure 11 This is a schematic diagram of a scenario in which an electronic device provided in an embodiment of this application performs a first simulated click operation; Figure 12 This is an interactive diagram of an electronic device performing a first simulated click operation, as provided in an embodiment of this application. Figure 13 This is a schematic diagram illustrating a scenario where the electronic device provided in this application completes the installation of the first installation program; Figure 14 This is the second flowchart of the interface display method provided in the embodiments of this application; Figure 15 This is a schematic diagram of a second scene of an electronic device displaying a first window, as provided in an embodiment of this application. Figure 16 This is the third interactive diagram in the process of the electronic device displaying the first window provided in the embodiments of this application; Figure 17This is the fourth interactive diagram in the process of the electronic device displaying the first window provided in the embodiments of this application; Figure 18 This is a schematic diagram of a scenario in which an electronic device provided in an embodiment of this application performs a second simulated click operation; Figure 19 This is an interactive diagram of an electronic device performing a second simulated click operation, as provided in an embodiment of this application. Figure 20 This is a schematic diagram of a scenario where an electronic device provided in this application completes the uninstallation of the first uninstallation program. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are all within the protection scope of this application.

[0031] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0032] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0033] The application scenarios of the embodiments of this application will be described below with reference to the accompanying drawings.

[0034] The method provided in this application can be applied to multi-screen collaborative work scenarios. Multi-screen collaborative work refers to the use of electronic devices to extend the desktop workspace of the main display screen (e.g., a physical screen or a virtual screen) from the display screen. This method can achieve the effect of processing multiple tasks simultaneously, displaying multiple sets of information, and improving operational efficiency in the same working environment.

[0035] Figure 1This is a schematic diagram of a multi-screen collaborative work scenario.

[0036] Multi-screen collaboration can include, for example, Figure 1 The first electronic device shown in A (e.g., a personal computer (PC) 1) works in conjunction with the main display screen 2 and the external display screen 3.

[0037] The external display screen 3 can be connected to the first electronic device 1 via at least one of a High Definition Multimedia Interface (HDMI), a Display Port (DP), or a Universal Serial Bus (USB) interface to achieve collaborative operation with the main display screen 2.

[0038] Multi-screen collaboration can also include, for example, Figure 1 The main display screen 2 of the first electronic device 1 shown in B works in conjunction with the wireless display screen 4.

[0039] The wireless display screen 4 can be connected to the first electronic device 1, for example, via a wireless projection protocol (Miracast).

[0040] Multi-screen collaboration can also include, for example, Figure 1 The main display screen 2 of the first electronic device 1, shown in C, works in conjunction with the virtual screen.

[0041] Specifically, the first electronic device 1 achieves collaborative work with the virtual screen by establishing a virtual screen window 5 on the main display screen 2 and mapping the interface of the virtual screen into the virtual screen window 5.

[0042] It should be noted that the virtual screen is an additional display screen simulated by the operating system of the first electronic device 1 through software. Based on the main display screen 2 and the simulated additional display screen, the first electronic device 1 can be assumed to have multiple display screens connected, thereby realizing multi-display screen management functions.

[0043] However, since the display areas of the main display screen 2 and the virtual screen are independent display areas, the first electronic device 1 typically cannot display the virtual screen's interface on the main display screen 2. Therefore, the first electronic device 1 can, for example, use windowing tools, screen mirroring tools, or image capture tools to display the virtual screen's interface as a window on the main display screen 2. That is, the first electronic device 1 can display the virtual screen's interface based on the virtual screen window 5, enabling the main display screen 2 to work collaboratively with the virtual screen.

[0044] The following problems exist in the above-mentioned multi-screen collaborative work scenarios.

[0045] For example Figure 1 The multi-screen working scenario of the main display screen 2 and the virtual screen shown in C is illustrated by example.

[0046] Figure 2 This is a schematic diagram of an abnormal scenario for multi-screen collaborative work.

[0047] like Figure 2 As shown, when the operating system of the first electronic device 1 is a Windows operating system, the first electronic device 1 can implement the display screen connection configuration memory function based on the Windows operating system.

[0048] Among them, the display connection configuration memory function of the Windows operating system refers to the operating system's ability to restore the historical display configuration of the main display screen 2 based on the same display interface and Extended Display Identification Data (EDID).

[0049] For example, if the refresh rate of the main display screen 2 of the first electronic device 1 is P1, and the first electronic device 1 sets the refresh rate of the virtual screen to P2 when it first creates the virtual screen (P2 can be the same as or different from P1), then the Windows operating system of the first electronic device 1 will store the refresh rate P1 of the main display screen 2 and the refresh rate P2 of the virtual screen as historical display configurations based on the display connection configuration memory function.

[0050] If the first electronic device 1 modifies the refresh rate of the main display screen 2 to P3 (P3 is different from P1), and then creates a virtual screen again after modifying the refresh rate of the main display screen 2, the Windows operating system of the first electronic device 1 will recognize the EDID corresponding to the virtual screen, and will enable the historical display configuration to set the refresh rate of the main display screen 2 to P1 and the refresh rate of the virtual screen to P2. In this way, since the refresh rate P1 and refresh rate P3 of the main display screen 2 are inconsistent, the main display screen 2 will experience screen flickering.

[0051] To address the issue of screen flickering during the display of a virtual interface via a window created by an electronic device, this application provides an interface display method. This interface display method can be applied to electronic devices.

[0052] The electronic devices provided in this application include, but are not limited to, PCs, mobile phones, tablets, personal digital assistants (PDAs), workstations, large-screen devices (e.g., smart screens, smart TVs), wearable devices (e.g., smart bracelets, smartwatches), handheld game consoles, home game consoles, virtual reality (VR) devices, augmented reality (AR) devices, mixed reality devices, and in-vehicle smart terminals. This application does not limit the specific technologies or device forms used in the electronic devices. The electronic devices involved in this application can run Windows®, Android®, Harmony OS®, iOS®, or other operating systems, and this application does not impose any restrictions on this.

[0053] Figure 3 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0054] like Figure 3 As shown, the electronic device 900 may include a processor 910, a memory 920, a Universal Serial Bus (USB) interface 930, a charging management module 940, a power management module 941, a battery 942, an antenna 01, a wireless communication module 960, an audio module 970, a speaker 970A, a microphone 970C, a headphone jack 970D, a sensor module 980, buttons 990, a camera 992, a display screen 993, etc. The sensor module 980 may include a touch sensor 980A, a proximity sensor 980G, a fingerprint sensor 980H, a temperature sensor 980J, etc.

[0055] The processor 910 may include one or more processing units, such as a central processing unit (CPU), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0056] In this embodiment of the application, the processor 910 can be used to implement the interface display method.

[0057] In some embodiments, the processor 910 may include one or more interfaces. Interfaces may include a Mobile Industry Processor Interface (MIPI), and / or a Universal Serial Bus (USB) interface, etc.

[0058] The MIPI interface can be used to connect the processor 910 to peripheral devices such as the display screen 993 and the camera 992. The MIPI interface includes a Camera Serial Interface (CSI) and a Display Serial Interface (DSI). In some embodiments, the processor 910 and the camera 992 communicate via the CSI interface to enable the electronic device 900 to capture images. The processor 910 and the display screen 993 communicate via the DSI interface to enable the electronic device 900 to display images.

[0059] In this embodiment, the processor 910 can interact with the display screen 993 via the MIPI interface to display a virtual screen on the display screen 993.

[0060] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0061] In this embodiment of the application, the electronic device 900 can be connected to an external device via a USB interface.

[0062] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0063] The wireless communication function of electronic device 900 can be realized through antenna 01, wireless communication module 960, modem processor and baseband processor.

[0064] The wireless communication module 960 can provide solutions for wireless communication applications on electronic devices 900, including Wireless Local Area Networks (WLANs) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR) technology, and other wireless communication technologies.

[0065] The wireless communication module 960 can be one or more devices integrating at least one communication processing module. The wireless communication module 960 receives electromagnetic waves via antenna 01, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 910. The wireless communication module 960 can also receive signals to be transmitted from processor 910, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 01.

[0066] In some embodiments, the antenna 01 of the electronic device 900 is coupled to the wireless communication module 960, enabling the electronic device 900 to communicate with networks and other devices via wireless communication technology.

[0067] In this embodiment, the electronic device 900 needs to implement the installation and uninstallation functions of the application based on the virtual assistant application under the condition that WLAN is enabled, thereby creating a virtual screen.

[0068] Electronic device 900 implements display functions through a GPU, a display screen 993, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 993 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 910 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0069] Display screen 993 is used to display images, videos, etc.

[0070] In this embodiment of the application, the display screen 993 can be used to display a first window, so as to display the interface of the first virtual screen based on the first window.

[0071] The software system of electronic device 900 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses a layered Windows system as an example to exemplify the software structure of electronic device 900.

[0072] Figure 4 This is a schematic diagram of the software structure of the electronic device provided in the embodiments of this application.

[0073] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces.

[0074] For example, the Windows system is divided into user mode and kernel mode. User mode can be divided from top to bottom into the application layer, system services and subsystem layer. Kernel mode can be divided from top to bottom into the Windows Executable, kernel and device driver layer, and hardware abstraction layer.

[0075] It should be noted that Windows systems can also have other software structure division methods. This application only uses the above division method as an example for illustration.

[0076] The application layer can include virtual assistant applications and window applications.

[0077] The virtual assistant application includes a native control module and a user interface agent (UI Agent) module.

[0078] The local control module can be a software module used to call the Windows operating system's built-in application programming interface (API), drivers, and system services. This module features fast response speed, high privacy, and no dependence on network connection.

[0079] The UI Agent module is an intelligent interaction module that can autonomously understand user needs and complete tasks by simulating human user interfaces.

[0080] The Windows application includes a window management module, a hook management module, an image capture module, a keyboard and mouse operation module, and a virtual screen driver module.

[0081] The window management module can be used to manage the display, hiding, hierarchy, and status of window programs. It may include a Remote Procedure Call (RPC) service module, a window display management module, an application installation / uninstallation control module, a keyboard and mouse control module, a virtual screen control module, an image capture control module, and a window position management module.

[0082] An RPC service module can be, for example, a GRPC service module, which can be used to provide remote procedure call services to enable communication between modules. This application does not limit the specific type of RPC service module.

[0083] The RPC service module can include an application installation / uninstallation response module, an application screenshot response module, a keyboard and mouse response module, and an application exception response module. The RPC service module can establish an inter-process communication link with the UI Agent module, creating a GRPC communication server to receive and process application installation / uninstallation requests, keyboard and mouse operation requests, application screenshot requests, and application operation exception requests from the UI Agent module.

[0084] The window display management module can be used to configure window size, position, and style strategies for different scenarios, and control the opening and closing of windows according to commands from the UI Agent module.

[0085] The install / uninstall application control module is used to initiate the installation and uninstallation of an app based on the location of the app's installation / uninstallation package and command-line parameters.

[0086] The keyboard and mouse control module is used to execute mouse click operations based on the mouse position information.

[0087] The virtual screen control module is used to control the creation or exit of virtual screens and to control the start and stop of virtual screen image capture.

[0088] The image capture control module is used to establish a virtual screen image capture link in order to control the start and stop of virtual screen capture.

[0089] The window position management module is used to cache App window information that has been moved to the virtual screen and move the App window information back to the main screen when needed.

[0090] The Hook management module includes the injector and the Hook module.

[0091] Hooking is a technique that uses mechanisms such as function calls, message passing, and system-level APIs to insert custom logic before, during, or after the execution of target code in the operating system, application, or other components. In this way, hooking can modify the default operating mechanisms of the operating system, application, or other components.

[0092] The injector includes an injection control module, a Hook installation / uninstallation control module, and an inter-process communication (IPC) module.

[0093] An injector is used to inject its own modules into a target process to access the target process's virtual address space. The injector is also used to establish a communication link with the injected module and to initiate hook installation / uninstallation requests to the target process.

[0094] The Hook module includes an API Hook module, an interactive window recognition module, an interactive window position control module, and an IPC module.

[0095] The Hook module is used to hook API calls related to the target process's window, identify interactive windows, and move them to a virtual screen.

[0096] The image capture module includes a virtual screen capture module and an App window capture module.

[0097] The virtual screen capture module is used to establish a virtual screen image capture link and send the virtual screen image to the Windows application.

[0098] The App window capture module is used to establish an App window capture link and provide support for App window screenshots.

[0099] The keyboard and mouse operation module includes a virtual keyboard and mouse operation interface and a physical mouse operation interface.

[0100] The virtual keyboard and mouse operation interface is used to provide a virtual mouse click interface for apps that support background keyboard and mouse operations, so that the UI Agent module can perform operations on the app interface.

[0101] The physical mouse operation interface is used for apps that do not support background keyboard and mouse operations. It provides a physical mouse click interface so that the UI Agent can temporarily control the mouse to perform operations on the app interface.

[0102] The virtual screen driver module includes a driver loading / unloading module, a virtual screen creation / destruction module, a virtual screen information management module, and a virtual screen mode control module.

[0103] The driver loading / unloading module is used to manage the loading / unloading of virtual screen drivers.

[0104] The virtual screen creation / destruction module provides the ability to create and destroy virtual screens, and supports changing the attributes of virtual screens (such as type, resolution, etc.).

[0105] The virtual screen information management module is used to save the resolution, refresh rate, position information, and size information of the virtual screen, and provides an external query interface.

[0106] The virtual screen mode control module provides an interface for setting the virtual screen to extended mode or duplicate mode.

[0107] The kernel and device driver layer includes at least the display driver, camera driver, audio driver, sensor driver, and virtual screen display driver.

[0108] It should be noted that the specific interaction methods of the above modules and their sub-modules will be exemplified in subsequent embodiments of this application, and will not be elaborated upon here.

[0109] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 900. In other embodiments of this application, the electronic device 900 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0110] Figure 5 This is the first flowchart of the interface display method provided in the embodiments of this application.

[0111] Figure 6 This is a schematic diagram of the first scene of the electronic device displaying the first window provided in the embodiments of this application.

[0112] In one implementation, based on such Figure 5 The flowchart shown illustrates that the interface display method includes the following steps S11-S18. Based on steps S11-S18, the electronic device 900 can display a first window on the display screen, and display a first interface in the first window. The first interface is mapped from the interface of the first virtual screen to the first window. In this embodiment, the electronic device 900 can automatically complete the installation of the first installation program on the first interface.

[0113] In step S11, the electronic device 900 displays a desktop interface 11 on the display screen 10 and displays a virtual assistant application icon 12 on the desktop interface 11.

[0114] like Figure 6 As shown in A, the display screen 10 has a first resolution and a first refresh rate.

[0115] For example, the first resolution of display 10 is 1920×1080, and the first refresh rate is 60Hz.

[0116] It should be noted that the values ​​of the first resolution and the first refresh rate shown in the embodiments of this application are for illustrative purposes only, and the specific values ​​of the first resolution and the first refresh rate are not limited in the embodiments of this application.

[0117] In step S12, in response to the click operation on the virtual assistant application icon 12, the electronic device 900 runs the virtual assistant application and displays the first program interface 13 in the virtual assistant application.

[0118] like Figure 6As shown in B, for example, a user can click on the virtual assistant application icon 12 with the mouse. In response to the click on the virtual assistant application icon 12, the electronic device 900 runs the virtual assistant application and displays the first program interface 13 in the virtual assistant application.

[0119] The first program interface 13 may include a new dialog control 14 and a first dialog interface 15 corresponding to the new dialog control 14. The first program interface 13 may also include a recent dialog control, a smart search control, a writing assistant control, a reading assistant control, and other controls.

[0120] The first dialogue interface 15 includes an input box 16. The input box 16 supports multiple input methods, including voice input and keyboard input. Users can input user commands based on the input box 16.

[0121] This application embodiment does not limit the types of controls included in the first program interface 13 or the specific interface form of the first program interface 13.

[0122] In step S13, in response to the input of a program installation command on the first program interface 13, the electronic device 900 displays first response information, wherein the first response information includes at least one program installation control corresponding to the program installation command.

[0123] like Figure 6 As shown in C, for example, a user can enter a program installation command 17 in input box 16. The command content of program installation command 17 is, for example, "Install chat application". In response to program installation command 17, electronic device 900 displays first response information 18 on the first dialogue interface 15. The first response information 18 may include at least one program installation control corresponding to program installation command 17. The program installation control includes, for example, a first program installation control 19 for installing the K1 version of the chat application, and a second program installation control 20 for installing the K2 version of the chat application.

[0124] The process of electronic device 900 displaying the first response information 18 is explained in detail below.

[0125] Figure 7 This is an interactive schematic diagram of the electronic device displaying the first response information provided in the embodiments of this application.

[0126] like Figure 7 As shown, step S13 includes the following steps S131-S134.

[0127] In step S131, in response to the program installation command, the electronic device 900 sends the program installation command to the first server.

[0128] In one implementation, step S131 includes step S1311.

[0129] In step S1311, in response to the program installation instruction, the virtual assistant application sends the program installation instruction to the content understanding service in the first server.

[0130] The first server can be a cloud server, and content understanding services can be deployed on the cloud server.

[0131] Content understanding services include, for example, multimodal content understanding services in multimodal scenarios. This application does not limit the specific service type of the content understanding service.

[0132] Multimodal scenarios refer to scenarios where cloud servers process and understand data modalities from various sources or of different types. Data modalities include, but are not limited to, visual modalities, auditory modalities, text modalities, and sensor modalities. Specifically, visual modalities are visual information acquired through image and video data, such as visual feature information extracted from images based on image recognition technology. Auditory modalities are auditory information acquired through audio and speech data, such as text information obtained by converting speech using speech recognition technology. Text modalities are information acquired through natural language text, such as text information obtained by classifying text using natural language processing technology. Sensor modalities are data information acquired based on various sensors built into electronic devices, such as temperature, pressure, and location information.

[0133] Multimodal content understanding service is an artificial intelligence service that can process and analyze data of different modalities based on the deep learning models set within it, and perform content recognition processing on this data.

[0134] In this embodiment, the program installation instruction is a text modality, and the multimodal content understanding service has the ability to recognize the program installation instruction in multiple ways.

[0135] Deep learning models may include at least one of face detection models, optical character recognition models, object detection models, image scene recognition models, text recognition models, and address recognition models. This application does not limit the number or type of deep learning models set in the content understanding service.

[0136] Based on the deep learning model in the multimodal content understanding service, the multimodal content understanding service can perform content recognition processing on the text elements in the program installation instructions to obtain the first content recognition result.

[0137] The first content recognition result can be in JavaScript Object Notation (JSON) data format. The first content recognition result can also be in other data formats; however, this application embodiment does not limit the data format of the first content recognition result.

[0138] JSON data format is a lightweight data-interchange format that supports various data types, including numbers, strings, booleans, arrays, and objects, and can be used for data transmission. The initial content recognition result of JSON data format can include multiple parts related to scene type, image tags, and entity type.

[0139] After obtaining the first content recognition result, the multimodal content understanding service can send the first content recognition result to the electronic device 900.

[0140] In step S132, the electronic device 900 receives the first content recognition result sent by the first server based on the intent framework.

[0141] In one implementation, step S132 includes step S1321.

[0142] Step S1321: The intent framework receives the first content recognition result sent by the content understanding service.

[0143] The intent framework can be part of the operating system of the electronic device 900, including multiple modules such as speech recognition, semantic understanding, and speech synthesis. This framework enables users to interact with the electronic device 900 through voice commands and keyboard instructions, thereby enabling the electronic device 900 to provide program installation controls.

[0144] In step S133, the electronic device 900 determines a first list corresponding to the first content recognition result based on the intent framework, wherein the first list includes at least one program installation control.

[0145] In one implementation, step S133 includes steps S1331-S1334.

[0146] In step S1331, the intent framework sends the first content recognition result to the intent query service.

[0147] The intent query service can exist as an application (Android application package, APK), or as an operating system service, system-level software, microservice, or other forms. The intent query service can be a service component embedded in the intent framework or a service component independent of the intent framework; this application embodiment does not limit the specific configuration method of the intent query service.

[0148] In one implementation, before sending the first content recognition result to the intent query service, the intent framework can initialize the intent query service to put it into a working state. After the intent query service enters the working state and completes the initialization process, it can send a notification message to the intent framework to enable the intent framework to perform subsequent operations.

[0149] The intent query service may include a first service plugin and a second service plugin.

[0150] In one implementation, step S1331 includes step S1331a.

[0151] In step S1331a, the intent framework sends the first content recognition result to the first service plugin.

[0152] The first service plugin can be an intent prediction plugin, used to predict user intent based on the first content recognition result. This application embodiment does not limit the type of plugin for the first service plugin.

[0153] An intent prediction plugin is a software component used to identify and predict user intents, enabling functions such as natural language understanding, intent recognition, entity extraction, and context management.

[0154] Step S1332: The intent query service performs an intent query on the first content recognition result to obtain at least one first target intent corresponding to the first content recognition result.

[0155] In one implementation, step S1332 includes steps S1332a-S1332c.

[0156] In step S1332a, the first service plugin performs an intent query on the first content recognition result to obtain at least one first target intent corresponding to the first content recognition result.

[0157] Specifically, the first service plugin matches the first content recognition result with the first preset configuration to perform intent query and obtain at least one first target intent corresponding to the first content recognition result, wherein the first preset configuration includes at least one user intent.

[0158] The first service plugin can obtain a first preset configuration containing multiple user intents from a cloud server during the idle period of the electronic device 900. For example, the first service plugin can obtain the first preset configuration when the electronic device 900 is charging and the screen is off. This application embodiment does not limit the specific method by which the first service plugin obtains the first preset configuration.

[0159] For example, the first target intent can be in JSON data format. The first target intent can also be in other data formats, and this application embodiment does not limit the data format of the first target intent.

[0160] In this embodiment, the first content recognition result is related to the installation of the application, and the first target intent may include, for example, the intent to install the application.

[0161] In this embodiment, the first content recognition result may correspond to multiple first target intentions. This application embodiment only uses the first target intention including the intention to install an application as an example for illustration. This application embodiment does not limit the specific form of the first target intention.

[0162] In step S1332b, the first service plugin sends at least one first target intent to the intent framework.

[0163] Further based on the aforementioned example, the first service plugin sends the application installation intent to the intent framework.

[0164] In step S1332c, the intent framework sends the application intent to the second service plugin.

[0165] Further based on the aforementioned example, the intent framework sends the application installation intent to the second service plugin.

[0166] It should be noted that, in another implementation, the first service plugin can also directly send the first target intent to the second service plugin. This application embodiment does not limit the specific method by which the second service plugin obtains the first target intent.

[0167] Step S1333: The intent query service obtains a first list that matches the first target intent.

[0168] In one implementation, step S1333 includes step S1333a.

[0169] Step S1333a: The second service plugin obtains a first list that matches the first target intent.

[0170] The second service plugin can be a program installation control recommendation plugin, used to recommend program installation controls based on the first target intent. This application embodiment does not limit the type of plugin for the second service plugin.

[0171] The program installation control recommendation plugin is a software component used to recommend programs based on user intent. It can achieve functions such as assisting decision-making and optimizing resource allocation.

[0172] In one implementation, the second service plugin matches the first target intent with the second preset configuration to obtain a first list corresponding to the first target intent, wherein the second preset configuration includes at least one preset list. The second service plugin can obtain the second preset configuration containing multiple preset lists from a cloud server during the idle period of the electronic device 900. Each preset list may include multiple recommended programs. For example, the second service plugin can obtain the second preset configuration when the electronic device 900 is charging and the screen is off. This application embodiment does not limit the specific method by which the second service plugin obtains the second preset configuration.

[0173] It should be noted that the first preset configuration and the second preset configuration can be deployed in different services of the same cloud server, or they can be deployed on different cloud servers. For example, the first preset configuration can be deployed in the decision cloud platform, and the second preset configuration can be deployed on the cloud server corresponding to the intent framework. This application embodiment does not limit the specific deployment method of the first and second preset configurations.

[0174] For example, the first list includes at least one program installation control, such as a first program installation control 19 for installing the K1 version of the chat application, and a second program installation control 20 for installing the K2 version of the chat application.

[0175] This application embodiment does not limit the specific method by which the second service plugin obtains the first list.

[0176] In step S1334, the intent query service sends the first list to the intent framework.

[0177] In one implementation, step S1334 includes step S1334a.

[0178] In step S1334a, the second service plugin sends the first list to the intent framework.

[0179] In step S134, the electronic device 900 displays the first response information based on the first list.

[0180] In one implementation, step S134 includes steps S1341-S1342.

[0181] In step S1341, the intent framework sends the first list to the virtual assistant application.

[0182] In step S1342, the virtual assistant application obtains at least one program installation control from the first list and sends the program installation control to the display, so that the electronic device 900 displays the first response information 18, and then displays the program installation control in the first response information 18.

[0183] In step S14, in response to a first operation on a target program installation control in at least one program installation control, the electronic device 900 generates a first installation instruction.

[0184] In one implementation, step S14 includes step S141.

[0185] In step S141, in response to a first operation on a target program installation control in at least one program installation control, the virtual assistant application generates a first installation instruction.

[0186] Further as Figure 6 As shown in C, the user can use the first program installation control 19 for installing the K1 version of the chat application as the target program installation control. In response to the click operation of the installation sub-control 21 in the first program installation control 19, the virtual assistant application generates a first installation instruction.

[0187] In step S15, in response to the first installation command, the electronic device 900 displays a first window on the display screen and displays a first interface in the first window. The first installation command is used to create a first virtual screen. The resolution of the first virtual screen is the same as the resolution of the display screen, and the refresh rate of the first virtual screen is the same as the refresh rate of the display screen. The first interface is mapped from the interface of the first virtual screen to the first window.

[0188] It should be noted that the first installation instruction is also used to instruct the electronic device 900 to execute a first installation process based on the first virtual screen in subsequent steps. The first installation process, for example, involves subsequent steps S16-S18, and is used to install the first installation program corresponding to the target program installation control.

[0189] like Figure 6 As shown in D, after the electronic device 900 generates the first installation instruction, the electronic device 900 downloads the K1 version chat application based on the first program installation control 19, and displays the installation progress bar 22 on the installation sub-control 21.

[0190] like Figure 6 As shown in E, after the installation progress bar 22 changes from 0 to 100%, the electronic device 900 displays an installation prompt 23 on the installation sub-control 21. For example, the prompt 23 may read "Installing".

[0191] like Figure 6 As shown in F, after displaying the installation prompt 23, the electronic device 900 displays the first window 24 and displays the first interface 25 in the first window 24.

[0192] Since the first interface 25 is mapped from the interface of the first virtual screen to the first window 24, and the resolution of the first virtual screen is the same as the resolution of the display screen, and the refresh rate of the first virtual screen is the same as the refresh rate of the display screen, based on the aforementioned example, the first resolution of the display screen 10 is 1920×1080, and the first refresh rate is 60Hz. Therefore, the first interface 25 has a first resolution, specifically 1920×1080, and a first refresh rate, specifically 60Hz.

[0193] The following describes the specific process of the electronic device 900 displaying the first window 24.

[0194] In one implementation, step S15 includes steps S151-S154.

[0195] In step S151, in response to the first installation command, the electronic device 900 determines whether the first installation program corresponding to the first installation command supports automatic installation.

[0196] Figure 8 This is the first interactive diagram in the process of displaying the first window on the electronic device provided in the embodiments of this application.

[0197] like Figure 8 As shown, in one implementation, step S151 includes steps S1511-S1515.

[0198] In step S1511, in response to the first installation instruction, the virtual assistant application downloads the first installer from the application market application.

[0199] It should be noted that the application market application can be an application built into the operating system or a third-party application. This application embodiment does not limit the specific type of application market application.

[0200] In step S1512, the virtual assistant application sends a first query request to the UI Agent module based on the local control module. The first query request is used to query whether the first installer supports automatic installation.

[0201] In step S1513, the virtual assistant application sends a first query request to the UI Agent cloud server based on the UI Agent module.

[0202] The UI Agent cloud server deploys configuration files for multiple installers that support automatic installation. In response to the first query request, the UI Agent cloud server sends the first configuration file to the UI Agent module. This first configuration file configures the installation path for the first installer.

[0203] In step S1514, the virtual assistant application receives the first configuration file sent by the UI Agent cloud server in response to the first query request based on the UI Agent module, and after receiving the first configuration file, sends a first notification message to the local control module. The first notification message is used to notify the local control module that the first installer supports automatic installation.

[0204] In one implementation, the UI Agent module specifically deploys a configuration acquisition module and an inference module.

[0205] The configuration acquisition module can be used to store the received configuration files. For example, the configuration file can be a prompt word configuration file, with prompt words including, for example, "Start automatically on boot" or "Install now".

[0206] The inference module can be used to store the specific installation path inferred from the configuration file. The installation path may include, for example, the path of the installation control corresponding to the first confirmation box after clicking the "Automatically start on boot" prompt, and then the path of the installation control corresponding to the "Install now" prompt.

[0207] The installation path can be obtained by the UI Agent based on the inference model stored locally and stored in the inference module, or it can be obtained by the UI Agent interacting with the UI Agent cloud server and receiving the inference result fed back by the UI Agent cloud server. This application embodiment does not limit the specific method of obtaining the installation path in the inference module.

[0208] In step S1515, the virtual assistant application responds to the first notification message based on the native control module to determine that the first installer supports automatic installation.

[0209] In this way, the virtual assistant application can determine whether the first installer supports automatic installation based on the received first configuration file and the inferred installation path.

[0210] In step S152, if the electronic device 900 determines that the first installation program supports automatic installation, the electronic device 900 obtains the first resolution and the first refresh rate of the display screen.

[0211] Figure 9 This is the second interactive diagram in the process of displaying the first window on the electronic device provided in the embodiments of this application.

[0212] like Figure 9 As shown, in one implementation, step S152 includes steps S1521-S1523.

[0213] In step S1521, if the virtual assistant application determines that the first installer supports automatic installation, the virtual assistant application sends a first installation request to the window application.

[0214] It should be noted that in order for the Windows application to respond accurately to the first installation request and execute subsequent processes, the electronic device 900 needs to pre-configure the Windows application.

[0215] Figure 10 This is an interactive diagram of the pre-configured window application for an electronic device provided in the embodiments of this application.

[0216] like Figure 10 As shown, in one implementation, electronic device 900 can pre-configure the Windows application based on the following steps S001-S004.

[0217] Step S001: After the electronic device 900 is powered on, a window application is launched based on the SystemCapTools.

[0218] Step S002: The Windows application starts the RPC service module and installs the virtual screen driver based on the virtual screen driver module.

[0219] It should be noted that steps S001-S002 can be executed before steps S11-S18, or after step S12 and before step S13. This application embodiment does not limit the specific execution order of steps S001-S002.

[0220] In step S1522, the Windows application responds to the first installation request by obtaining the first display information of the display screen, which includes the first resolution and the first refresh rate.

[0221] It should be noted that the first display information may also include other display parameters of the display screen besides the first resolution and the first refresh rate. This application embodiment does not limit the type of parameters in the first display information.

[0222] In one implementation, the Windows application responds to the first installation request based on the install / uninstall application response module in the RPC service module, and obtains the first display information of the display screen based on the virtual screen driver module.

[0223] In step S1523, the Windows application sends the first display information to the virtual screen display driver.

[0224] In one implementation, the Windows application sends the first display information to the virtual screen display driver based on the virtual screen driver module.

[0225] In step S1524, the virtual screen display driver obtains the first resolution and the first refresh rate based on the first display information.

[0226] In step S153, the electronic device 900 initializes the screen configuration information corresponding to the first virtual screen, so as to configure the first resolution, the first refresh rate, and the first EDID in the screen configuration information.

[0227] In one implementation, step S153 includes steps S1531-S1532.

[0228] Step S1531: The virtual screen display driver configures screen configuration information based on the obtained first resolution and first refresh rate.

[0229] In step S1532, the virtual screen display driver modifies the built-in EDID table to generate a first EDID and configures the screen configuration information based on the first EDID.

[0230] The first EDID is different from at least one second EDID historically stored in the electronic device 900. Each second EDID corresponds to a historically connected external device, or a historically created second virtual screen.

[0231] Specifically, EDID is a standard data format defined by the Video Electronics Standards Association (VESA). It can include a basic 128-bit binary data block, such as a header field, a product ID field, an EDID version field, a basic display parameters field, a color characteristics field, a standard timings field, a detailed timing descriptors field, and an extension flag field.

[0232] EDID may also include one or more extended 128-bit binary extension blocks to describe more display information fields such as color space and refresh rate extension.

[0233] It should be noted that the embodiments of this application only provide illustrative examples of the fields in EDID. EDID may also include other types of fields, such as Manufacturer ID, Produce Code, Serial Number, and Week / Year of Manufacture. The embodiments of this application do not limit this.

[0234] Each physical screen or virtual screen needs to be configured with an EDID so that the electronic device 900 can read it based on the operating system; otherwise, the electronic device 900 cannot configure the parameters of the physical screen or virtual screen.

[0235] The EDID of a physical screen is usually written into an electrically erasable programmable read-only memory (EEPROM) chip by the display manufacturer during the production process of the physical screen. However, a virtual screen does not have a real EDID. Therefore, the EDID of a virtual screen is usually generated by the virtual screen display driver based on the built-in EDID table.

[0236] Thus, if the electronic device 900 creates a virtual screen based on the built-in EDID table every time, the electronic device 900 may encounter the aforementioned problems when creating a virtual screen for the first time. Figure 2 The abnormal scenarios involved.

[0237] Therefore, in this embodiment of the application, the virtual screen display driver configures a different first EDID each time it configures screen configuration information, so that the electronic device 900 can recognize the created first virtual screen as the first virtual screen created.

[0238] In one implementation, the virtual screen display driver can modify the Product ID field in the built-in EDID table each time to generate a different first EDID.

[0239] It should be noted that the virtual screen display driver can also modify other fields in the EDID table, such as the Manufacturer ID field, Produce Code field, Serial Number field, and Week / Year of Manufacture field, to generate different first EDIDs. This application embodiment does not limit the specific method by which the virtual screen display driver generates different EDIDs.

[0240] In other words, each time the electronic device 900 creates the first virtual screen, it sets a new EDID for the currently created first virtual screen that is different from the historically stored EDID. In this way, the electronic device 900 avoids the problem of the screen flickering black caused by the inconsistency between the historical refresh rate and the current refresh rate, which would occur when the historical refresh rate is not consistent with the current refresh rate, based on the screen connection configuration memory function to obtain the historical display configuration corresponding to the historical display configuration. This improves the user experience.

[0241] In other embodiments, before creating the first virtual screen, the electronic device 900 obtains the second refresh rate corresponding to the display screen. If the second refresh rate of the display screen is different from the first refresh rate when creating the first virtual screen, the second refresh rate is changed to the first refresh rate after the display screen disconnects from the first virtual screen. This ensures that when the electronic device 900 creates the first virtual screen again in subsequent steps, the display screen has the first refresh rate, thus avoiding the problem of the display screen flickering black.

[0242] In other words, besides modifying the first EDID, electronic devices can also modify the display refresh rate after the first virtual screen is disconnected to avoid the display flickering problem. However, this embodiment requires preventing the display from switching to another refresh rate (i.e., a refresh rate different from the first refresh rate) during the process from the previous creation of the first virtual screen to the next creation of the first virtual screen.

[0243] The subsequent embodiments of this application will be exemplified by the process of creating a first virtual screen based on modifying the first EDID in the virtual screen display driver.

[0244] In step S154, the electronic device 900 creates a first virtual screen based on the configured screen configuration information, displays a first window, and displays a first interface in the first window.

[0245] In one implementation, step S154 includes steps S1541-S1545.

[0246] Step S1541: The virtual screen display driver creates the first virtual screen based on the configured screen configuration information.

[0247] In one implementation, the virtual screen display driver creates a first virtual screen based on a preset screen creation mode.

[0248] For example, in the application installation scenario, the default screen creation mode can be extended screen mode.

[0249] It should be noted that when the interface display method shown in this embodiment is applied to other scenarios, the preset screen creation mode can also be a screen copy mode.

[0250] Among them, the extended screen mode is a mode in which the display screen 10 and the first virtual screen have different display content, and the duplicate screen mode is a mode in which the display screen 10 and the first virtual screen have the same display content.

[0251] In this embodiment of the application, the first virtual screen supports multiple screen creation modes, and this embodiment of the application does not limit the type of screen creation mode corresponding to the first virtual screen.

[0252] In one implementation, when the preset screen creation mode is extended screen mode, the virtual screen display driver switches the first virtual screen from extended screen mode to duplicate screen mode in response to a first mode switching command. When the preset screen creation mode is duplicate screen mode, the virtual screen display driver switches the first virtual screen from duplicate screen mode to extended screen mode in response to a second mode switching command.

[0253] It should be noted that both the first mode switching command and the second mode switching command can be generated based on the mode switching operation performed by the user on the settings interface of the display screen 10. The specific operation method of the mode switching operation will not be described in detail in this embodiment.

[0254] In this embodiment, the first virtual screen not only supports multiple screen creation modes, but also supports switching between different screen creation modes. Furthermore, during the switching between different screen creation modes, since the first virtual screen in extended screen mode, the first virtual screen in duplicate screen mode, and the display screen 10 all have the same resolution and refresh rate, the display screen 10 will not experience a blackout problem during the mode switching process.

[0255] Specifically, if the first refresh rate of the display screen 10 is 60Hz, and the created first virtual screen is in copy screen mode, corresponding to a first refresh rate of 60Hz, then when the first virtual screen is switched from copy screen mode to extended screen mode, since the first virtual screen has a new first EDID, the electronic device 900 will recognize the first virtual screen as a newly created screen and support setting this screen to a different refresh rate than the display screen. That is, the first virtual screen can correspond to a second refresh rate of 120Hz. At this time, since the first refresh rate of the display screen 10 is different from the second refresh rate of the first virtual screen, and the display screen supports multiple refresh rates, the operating system of the electronic device 900 can automatically switch the first refresh rate of the display screen 10 to the second refresh rate to adapt to the first virtual screen. This will cause the display screen 10 to experience a flickering black problem. This embodiment of the application ensures that the display screen 10 will not experience a flickering black problem during the mode switching process of the first virtual screen by keeping the first virtual screen in extended screen mode, the first virtual screen in copy screen mode, and the display screen 10 at the same resolution and refresh rate.

[0256] In step S1542, the virtual screen display driver sends a first message to the window application, which indicates that the first virtual screen has been successfully created.

[0257] In one implementation, the virtual screen display driver sends a first message to the virtual screen driver module.

[0258] In step S1543, the Windows application responds to the first message and creates the first window.

[0259] In one implementation, the Windows application creates the first window based on the window display management module and the window position management module.

[0260] In step S1544, the window application maps the interface of the first virtual screen to the first window to display the first interface in the first window.

[0261] In one implementation, step S1544 includes steps S1544a-S1544b.

[0262] In step S1544a, the window application controls the image capture module to establish a virtual screen image capture link based on the first capture instruction, so as to capture the interface of the first virtual screen.

[0263] In step S1544b, the window application sends the interface of the first virtual screen to the first window based on the image capture module, so as to display the first interface in the first window.

[0264] In step S1545, the Windows application establishes an interactive relationship with the first installer based on the Hook mechanism.

[0265] In one implementation, step S1545 includes steps S1545a-S1545b.

[0266] In step S1545a, the Windows application starts the Hook management module and, based on the injector, performs an injection operation on the WindowsHook module in the first installer to install the Hook module in the first installer.

[0267] In other words, the Hook management module injects the code logic in the Hook module into the process of the first installer based on the injection operation. In this way, the Hook module can intercept and modify the function calls or message transmission process in the first installer.

[0268] In step S1545b, after the first installer completes the installation of the Hook module, the first installer runs the API Hook module, the interactive window recognition module, the interactive window position control module, and the IPC module based on the Hook module, so that the application installation window displayed by the first installer can be displayed on the first virtual screen.

[0269] In one implementation, the Hook module can intercept window-related APIs in the first installer based on the API Hook module. Further, the Hook module uses an interactive window recognition module to determine whether the intercepted window-related APIs correspond to the application installation window. Further, if the Hook module determines that it has intercepted an API corresponding to the application installation window based on the interactive window recognition module, the Hook module processes the API corresponding to the application installation window using an interactive window position control module and an IPC module, so that the application installation window can be moved to the first virtual screen and displayed on the first virtual screen.

[0270] In step S1546, the Windows application sends a second message to the virtual assistant application. The second message indicates that the first virtual screen has been successfully created and that the Windows application has established an interactive relationship with the first installer based on the Hook mechanism.

[0271] In step S16, the electronic device 900 displays the first installation interface corresponding to the first installation program on the first interface.

[0272] In one implementation, step S16 includes steps S161-S163.

[0273] In step S161, in response to the second message, the virtual assistant application sends a third message to the window application, which triggers the window application to execute the first installation process.

[0274] In step S162, in response to the third message, the Windows application executes the first installation process to capture the first installation interface based on the Hook mechanism.

[0275] In one implementation, step S162 includes steps S1621-S1622.

[0276] In step S1621, in response to the third message, the window application controls the image capture module to establish a window capture link based on the second capture instruction, so as to capture the application installation window of the first installer.

[0277] The process of the image capture module capturing the application installation window of the first installer refers to the process of taking a screenshot of the application installation window.

[0278] In step S1622, the Windows application sends the application installation window to the first window based on the image capture module, so as to generate the first installation interface corresponding to the application installation window on the first interface.

[0279] In step S17, the electronic device 900 performs a first simulated click operation on the first installation interface based on the UI Agent.

[0280] Figure 11 This is a schematic diagram of a scenario in which an electronic device provided in an embodiment of this application performs a first simulated click operation.

[0281] like Figure 6 China F and Figure 11 As shown in A, after the electronic device 900 displays the first interface 25 in the first window 24, it can display the first installation interface 26 on the first interface 25.

[0282] like Figure 11 As shown in B, the electronic device 900 can select the first confirmation box 27 adjacent to the "Automatic Startup" prompt based on the first simulated click operation.

[0283] like Figure 11 As shown in C, the electronic device 900 can further select the installation control 28 corresponding to the "Install Now" prompt based on the first simulated click operation.

[0284] The following describes the specific process of the electronic device 900 performing the first simulated click operation.

[0285] Figure 12 This is an interactive diagram of an electronic device performing a first simulated click operation, as provided in an embodiment of this application.

[0286] like Figure 12 As shown, in one implementation, step S17 includes steps S171-S173.

[0287] In step S171, the Windows application sends the first installation interface to the virtual assistant application.

[0288] In step S172, upon receiving the first installation interface, the virtual assistant application determines whether it is currently in an application installation state or an application uninstallation state.

[0289] In step S173, when the virtual assistant application determines that the current state is an application installation state, it performs a first simulated click operation on the first installation interface based on the UI Agent.

[0290] In one implementation, step S173 includes steps S1731-S1735.

[0291] In step S1731, when the virtual assistant application determines that it is currently in the state of installing an application, the virtual assistant application parses the first configuration file based on the UI Agent module and sends the parsed first configuration file and the first installation interface to the UI Agent cloud server.

[0292] In this way, the UI Agent cloud server can use the inference model deployed therein to infer the parsed first configuration file and the first installation interface to obtain the installation path corresponding to the first installer. The installation path can be generated based on the interface coordinates corresponding to each prompt word.

[0293] In step S1732, the virtual assistant application receives the inference results sent by the UI Agent cloud server based on the UI Agent module. The inference results include the installation path.

[0294] In step S1733, after obtaining the reasoning result, the virtual assistant application sends a keyboard and mouse click request to the window application.

[0295] In step S1734, the window application responds to the keyboard and mouse click request and provides a virtual keyboard and mouse operation interface to the UI Agent module in the virtual assistant application based on the keyboard and mouse operation module.

[0296] In step S1735, the virtual assistant application performs a first simulated click operation on the installation path in the inference result based on the UI Agent module.

[0297] In step S18, the electronic device 900 completes the first installation process based on the first simulated click operation to complete the installation of the first installation program.

[0298] In one implementation, the UI Agent module in the virtual assistant application completes the first installation process based on a first simulated click operation to complete the installation of the first installer.

[0299] Figure 13 This is a schematic diagram of a scenario in which the electronic device provided in this application completes the installation of the first installation program.

[0300] like Figure 11 C and Figure 13 As shown in A, after the electronic device 900 completes the first simulated click operation, it cancels the display of the first installation interface 26 on the first interface 25.

[0301] like Figure 13 As shown in B, after canceling the display of the first installation interface 26, the electronic device 900 cancels the display of the first window 24 and displays the chat application icon 29 on the desktop interface 11.

[0302] Based on the above process, during the display of the virtual screen interface created by the electronic device 900 based on the display screen 10, the problem of the display screen 10 flickering black is avoided, thus improving the user experience. Furthermore, the electronic device 900 automatically completes the installation of the first installation program on the first interface 25. In the application installation scenario, this embodiment not only improves the user's visual experience but also avoids manual operation by the user.

[0303] Figure 14 This is the second flowchart of the interface display method provided in the embodiments of this application.

[0304] Figure 15 This is a second scene diagram of the electronic device displaying the first window provided in the embodiments of this application.

[0305] In one implementation, based on such Figure 14 The flowchart shown illustrates that the interface display method includes the following steps S21-S28. Based on steps S21-S28, the electronic device 900 can display a first window on the display screen, and display a first interface in the first window. The first interface is mapped from the interface of the first virtual screen to the first window. In this embodiment, the electronic device 900 can automatically uninstall the first uninstallation program on the first interface.

[0306] In step S21, the electronic device 900 displays a desktop interface 11 on the display screen 10 and displays a virtual assistant application icon 12 on the desktop interface 11.

[0307] like Figure 15 As shown in A, the display screen 10 has a first resolution and a first refresh rate.

[0308] For example, the first resolution of display 10 is 1920×1080, and the first refresh rate is 60Hz.

[0309] In step S22, in response to the click operation on the virtual assistant application icon 12, the electronic device 900 runs the virtual assistant application and displays the first program interface 13 in the virtual assistant application.

[0310] like Figure 15 As shown in B, for example, a user can click on the virtual assistant application icon 12 with the mouse. In response to the click on the virtual assistant application icon 12, the electronic device 900 runs the virtual assistant application and displays the first program interface 13 in the virtual assistant application.

[0311] The control settings in the first program interface 13 can be found in the aforementioned embodiments, and this application does not limit them.

[0312] In step S23, in response to the input of a program uninstallation command on the first program interface 13, the electronic device 900 displays second response information, wherein the second response information includes at least one program uninstallation control corresponding to the program uninstallation command.

[0313] like Figure 15 As shown in C, for example, a user can enter an uninstallation command 30 in input box 16. The command content of the uninstallation command 30 is, for example, "Uninstall browser application". In response to the uninstallation command 30, the electronic device 900 displays a second response information 31 on the first dialog interface 15. The second response information 31 may include at least one uninstallation control corresponding to the uninstallation command. The uninstallation control includes, for example, a first uninstallation control 32 for uninstalling the F1 version of the browser application, and a second uninstallation control 33 for uninstalling the F2 version of the browser application.

[0314] The process of the electronic device 900 displaying the second response information can be referred to in the foregoing embodiments, and will not be repeated here.

[0315] In step S24, in response to a second operation on a target program uninstallation control in at least one program uninstallation control, the electronic device 900 generates a first uninstallation instruction.

[0316] Further as Figure 15 As shown in C, the user can use the first program uninstallation control 32 for uninstalling the F1 version browser application as the target program uninstallation control. In response to the click operation of the uninstallation sub-control 34 in the first program uninstallation control 32, the virtual assistant application generates a first uninstallation instruction.

[0317] In step S25, in response to the first unload command, the electronic device 900 displays a first window on the display screen and displays a first interface in the first window. The first unload command is used to create a first virtual screen. The resolution of the first virtual screen is the same as the resolution of the display screen, and the refresh rate of the first virtual screen is the same as the refresh rate of the display screen. The first interface is mapped from the interface of the first virtual screen to the first window.

[0318] It should be noted that the first uninstallation instruction is also used to instruct the electronic device 900 to execute a first uninstallation process based on the first virtual screen in subsequent steps. The first uninstallation process, for example, involves subsequent steps S26-S28, and is used to uninstall the first uninstallation program corresponding to the target program uninstallation control.

[0319] like Figure 15 As shown in D, after the electronic device 900 generates the first unload command, the electronic device 900 displays the first window 24 and displays the first interface 25 in the first window 24.

[0320] Since the first interface 25 is mapped from the interface of the first virtual screen to the first window 24, and the resolution of the first virtual screen is the same as the resolution of the display screen, and the refresh rate of the first virtual screen is the same as the refresh rate of the display screen, based on the aforementioned example, the first resolution of the display screen 10 is 1920×1080, and the first refresh rate is 60Hz. Therefore, the first interface 25 has a first resolution, specifically 1920×1080, and a first refresh rate, specifically 60Hz.

[0321] The following describes the specific process of the electronic device 900 displaying the first window 24.

[0322] In one implementation, step S25 includes steps S251-S254.

[0323] In step S251, in response to the first uninstallation command, the electronic device 900 determines whether the first uninstallation program corresponding to the first uninstallation command supports automatic uninstallation.

[0324] Figure 16 This is the third interactive diagram in the process of displaying the first window on the electronic device provided in the embodiments of this application.

[0325] like Figure 16 As shown, in one implementation, step S251 includes steps S2511-S2514.

[0326] In step S2511, in response to the first uninstallation command, the virtual assistant application sends a second query request to the UIAgent module based on the local control module. The second query request is used to query whether the first uninstallation program supports automatic uninstallation.

[0327] In step S2512, the virtual assistant application sends a second query request to the UI Agent cloud server based on the UI Agent module.

[0328] The UI Agent cloud server deploys configuration files for multiple uninstallers that support automatic uninstallation. This allows the UI Agent cloud server to respond to a second query request by sending a second configuration file to the UI Agent module. The second configuration file is used to configure the uninstallation path corresponding to the first uninstaller.

[0329] In step S2513, the virtual assistant application receives the second configuration file sent by the UI Agent cloud server in response to the second query request based on the UI Agent module, and after receiving the second configuration file, sends a second notification message to the local control module. The second notification message is used to notify the local control module that the first uninstaller supports automatic uninstallation.

[0330] In one implementation, the UI Agent module specifically deploys a configuration acquisition module and an inference module.

[0331] The configuration acquisition module can be used to store the received configuration files. These configuration files can be, for example, prompt word configuration files, with prompt words including, for example, "Uninstall now".

[0332] The inference module can be used to store specific uninstallation paths inferred from the configuration file. The uninstallation path may include, for example, the path of the uninstallation control corresponding to clicking the "Uninstall Now" prompt.

[0333] The uninstallation path can be obtained by the UI Agent based on the inference model stored locally and stored in the inference module, or it can be obtained by the UI Agent interacting with the UI Agent cloud server and receiving the inference result fed back by the UI Agent cloud server. This application embodiment does not limit the specific method of obtaining the uninstallation path in the inference module.

[0334] In step S2514, the virtual assistant application responds to the second notification message based on the local control module to determine that the first uninstaller supports automatic uninstallation.

[0335] In this way, the virtual assistant application can determine whether the first uninstaller supports automatic uninstallation based on the received second configuration file and the inferred uninstallation path.

[0336] In step S252, if the electronic device 900 determines that the first uninstallation program supports automatic uninstallation, the electronic device 900 obtains the first resolution and the first refresh rate of the display screen.

[0337] Figure 17 This is the fourth interactive diagram in the process of displaying the first window on the electronic device provided in this application embodiment.

[0338] like Figure 17 As shown, in one implementation, step S252 includes steps S2521-S2523.

[0339] In step S2521, if the virtual assistant application determines that the first uninstaller supports automatic uninstallation, the virtual assistant application sends a first uninstallation request to the window application.

[0340] It should be noted that in order for the Windows application to respond accurately to the first uninstallation request and to execute subsequent processes, the electronic device 900 needs to pre-configure the Windows application.

[0341] The process of pre-configuring the Windows application required by the electronic device 900 can be referred to the foregoing embodiments, and will not be repeated here.

[0342] In step S2522, the Windows application responds to the first uninstallation request by obtaining the first display information of the display screen, which includes the first resolution and the first refresh rate.

[0343] In one implementation, the Windows application responds to the first uninstallation request based on the install / uninstall application response module in the RPC service module, and obtains the first display information of the display screen based on the virtual screen driver module.

[0344] In step S2523, the Windows application sends the first display information to the virtual screen display driver.

[0345] In one implementation, the Windows application sends the first display information to the virtual screen display driver based on the virtual screen driver module.

[0346] In step S2524, the virtual screen display driver obtains the first resolution and the first refresh rate based on the first display information.

[0347] In step S253, the electronic device 900 initializes the screen configuration information corresponding to the first virtual screen, so as to configure the first resolution, the first refresh rate, and the first EDID in the screen configuration information.

[0348] In one implementation, step S253 includes steps S2531-S2532.

[0349] Step S2531: The virtual screen display driver configures screen configuration information based on the obtained first resolution and first refresh rate.

[0350] In step S2532, the virtual screen display driver modifies the built-in EDID table to generate a first EDID and configures the screen configuration information based on the first EDID.

[0351] The first EDID is different from at least one second EDID historically stored in the electronic device 900. Each second EDID corresponds to a historically connected external device, or a historically created second virtual screen.

[0352] It should be noted that the parameters of the first EDID in this embodiment are different from those in the previous embodiments. This is because this embodiment and the previous embodiments correspond to two different virtual screen creation processes, and will not generate the same first virtual screen.

[0353] In step S254, the electronic device 900 creates a first virtual screen based on the configured screen configuration information, displays a first window, and displays a first interface in the first window.

[0354] In one implementation, step S254 includes steps S2541-S2545.

[0355] Step S2541: The virtual screen display driver creates the first virtual screen based on the configured screen configuration information.

[0356] In one implementation, the virtual screen display driver creates a first virtual screen based on a preset screen creation mode.

[0357] For example, in the program uninstallation scenario, the default screen creation mode can be extended screen mode.

[0358] In step S2542, the virtual screen display driver sends a first message to the window application, which indicates that the first virtual screen has been successfully created.

[0359] In one implementation, the virtual screen display driver sends a first message to the virtual screen driver module.

[0360] In step S2543, the Windows application responds to the first message and creates the first window.

[0361] In step S2544, the window application maps the interface of the first virtual screen to the first window to display the first interface in the first window.

[0362] In one implementation, step S2544 includes steps S2544a-S2544b.

[0363] In step S2544a, the Windows application controls the image capture module to establish a virtual screen image capture link based on the first capture instruction, so as to capture the interface of the first virtual screen.

[0364] In step S2544b, the window application sends the interface of the first virtual screen to the first window based on the image capture module, so as to display the first interface in the first window.

[0365] In step S2545, the Windows application establishes an interactive relationship with the first uninstaller based on the Hook mechanism.

[0366] In one implementation, step S2545 includes steps S2545a-S2545c.

[0367] In step S2545a, the Windows application starts the Hook management module and performs an injection operation on the WindowsHook module in the first uninstaller based on the injector.

[0368] In step S2545b, after the injection operation is completed, the Windows application installs the Hook module in the first uninstaller based on the Hook management module.

[0369] In step S2545c, after the first uninstaller completes the installation of the Hook module, the first uninstaller runs the API Hook module, the interactive window recognition module, the interactive window position control module, and the IPC module based on the Hook module, so that the application uninstallation window displayed by the first uninstaller can be displayed on the first virtual screen.

[0370] In step S2546, the Windows application sends a second message to the virtual assistant application. The second message indicates that the first virtual screen has been successfully created and that the Windows application has established an interactive relationship with the first uninstaller based on the Hook mechanism.

[0371] In step S26, the electronic device 900 displays the first uninstallation interface corresponding to the first uninstallation program on the first interface.

[0372] In one implementation, step S26 includes steps S261-S263.

[0373] In step S261, in response to the second message, the virtual assistant application sends a fourth message to the window application, which triggers the window application to execute the first uninstallation process.

[0374] In step S262, in response to the fourth message, the Windows application executes the first uninstallation process to capture the first uninstallation interface based on the Hook mechanism.

[0375] In one implementation, step S262 includes steps S2621-S2622.

[0376] In step S2621, in response to the fourth message, the window application controls the image capture module to establish a window capture link based on the third capture instruction, so as to capture the application uninstallation window of the first uninstaller.

[0377] The process of the image capture module capturing the application uninstallation window of the first uninstallation program refers to the process of taking a screenshot of the application uninstallation window.

[0378] In step S2622, the window application sends the application uninstallation window to the first window based on the image capture module, so as to generate the first uninstallation interface corresponding to the application uninstallation window on the first interface.

[0379] In step S27, the electronic device 900 performs a second simulated click operation on the first uninstallation interface based on the UI Agent.

[0380] Figure 18 This is a schematic diagram of a scenario in which an electronic device provided in an embodiment of this application performs a first simulated click operation.

[0381] like Figure 15 China D and Figure 18As shown in A, after the electronic device 900 displays the first interface 25 in the first window 24, it can display the first uninstallation interface 35 on the first interface 25.

[0382] like Figure 18 As shown in B, the electronic device 900 can select the uninstall control 36 corresponding to the "Uninstall Now" prompt based on the second simulated click operation.

[0383] The following describes the specific process of electronic device 900 performing two simulated click operations.

[0384] Figure 19 This is an interactive diagram of an electronic device performing a second simulated click operation, as provided in an embodiment of this application.

[0385] like Figure 19 As shown, in one implementation, step S27 includes steps S271-S273.

[0386] In step S271, the Windows application sends the first uninstallation interface to the virtual assistant application.

[0387] In step S272, when the virtual assistant application receives the first uninstallation interface, it determines whether the current state is an application installation state or an application uninstallation state.

[0388] In step S273, when the virtual assistant application determines that the current state is an uninstallation state, it performs a second simulated click operation on the uninstallation interface based on the UI Agent.

[0389] In one implementation, step S273 includes steps S2731-S2735.

[0390] In step S2731, when the virtual assistant application determines that the current state is an uninstalled application, the virtual assistant application parses the second configuration file based on the UI Agent module and sends the parsed second configuration file and the first uninstallation interface to the UI Agent cloud server.

[0391] In this way, the UI Agent cloud server can use the inference model deployed therein to infer the parsed second configuration file and the first uninstallation interface to obtain the uninstallation path corresponding to the first uninstallation program. The uninstallation path can be generated based on the interface coordinates corresponding to each prompt word.

[0392] In step S2732, the virtual assistant application receives the inference results sent by the UI Agent cloud server based on the UI Agent module. The inference results include the uninstallation path.

[0393] In step S2733, after obtaining the reasoning result, the virtual assistant application sends a keyboard and mouse click request to the window application.

[0394] In step S2734, the window application responds to the keyboard and mouse click request and provides a virtual keyboard and mouse operation interface to the UI Agent module in the virtual assistant application based on the keyboard and mouse operation module.

[0395] In step S2735, the virtual assistant application performs a second simulated click operation based on the uninstallation path in the inference result from the UI Agent module.

[0396] In step S28, the electronic device 900 completes the first uninstallation process based on the second simulated click operation to uninstall the first uninstallation program.

[0397] In one implementation, the UI Agent module in the virtual assistant application completes the first uninstallation process based on a first simulated click operation, thereby uninstalling the first uninstaller.

[0398] Figure 20 This is a schematic diagram of a scenario where an electronic device provided in this application completes the uninstallation of the first uninstallation program.

[0399] like Figure 18 B and Figure 20 As shown in A, after the electronic device 900 completes the second simulated click operation, it cancels the display of the first uninstallation interface 35 on the first interface 25.

[0400] like Figure 20 As shown in B, after canceling the display of the first uninstallation interface 35, the electronic device 900 cancels the display of the first window 24 and cancels the display of the browser application icon 37 on the desktop interface 11.

[0401] Based on the above process, during the display of the virtual screen interface created by the electronic device 900 based on the display screen 10, the problem of the display screen 10 flickering black is avoided, thus improving the user experience. Furthermore, the electronic device 900 automatically uninstalls the first uninstallation program on the first interface 25. In the application uninstallation scenario, this embodiment not only improves the user's visual experience but also avoids manual operation by the user.

[0402] The foregoing primarily describes the solutions provided in the embodiments of this application from the perspective of electronic devices. It is understood that, in order to achieve the aforementioned functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the interface display method steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed through hardware or by software-driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0403] This application embodiment can divide the above-described electronic device into functional modules or functional units according to the above method examples. For example, each function can be divided into its own functional modules or functional units, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module or functional unit. The module or unit division in this application embodiment is illustrative and represents only one logical functional division; other division methods may be used in actual implementation.

[0404] This application provides an interface display device applied to the electronic device described in the above embodiments. The electronic device includes a display screen with a first refresh rate. The device includes a display module, which is configured to display a first window on the display screen and a first interface in the first window in response to a first instruction. The first instruction is used to create a first virtual screen, which has a first refresh rate. The first interface is mapped from the interface of the first virtual screen to the first window. The first virtual screen corresponds to a first EDID, which is different from at least one second EDID historically stored in the electronic device. Each second EDID corresponds to a historically connected external device or a historically created second virtual screen.

[0405] The interface display device provided in this application embodiment can set a new EDID for the currently created virtual screen each time the electronic device creates a virtual screen, which is different from the historically stored EDID. In this way, the electronic device avoids the problem of the screen flickering black caused by the inconsistency between the historical refresh rate and the current refresh rate, which occurs when the historical refresh rate is not consistent with the current refresh rate, by obtaining the historical display configuration corresponding to the historically stored EDID based on the display connection configuration memory function and resetting the current refresh rate of the display based on the historical refresh rate in the historical display configuration. This improves the user experience.

[0406] This application also provides an electronic device, which may include a display screen, a memory, a processor, and a communication module. These devices can be connected via one or more communication buses. The display screen may include a display panel and a touch sensor, wherein the display panel displays images, and the touch sensor transmits detected touch operations to an application processor to determine the type of touch event, providing visual output related to the touch operation through the display panel. The processor may include one or more processing units, such as a central processing unit, a graphics processing unit, an image signal processor, a controller, a video codec, a digital signal processor, and / or a neural network processor. Different processing units may be independent devices or integrated into one or more processors. The memory is coupled to the processor and is used to store various software programs and / or computer instructions. The memory may include volatile memory and / or non-volatile memory. When the processor executes computer instructions, the electronic device can perform the various functions or steps in the above method embodiments.

[0407] This application also provides a chip system including at least one processor and at least one interface circuit. The processor and the interface circuit are interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit can be used to send signals to other devices. Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the steps in the above method embodiments. Of course, the chip system may also include other discrete devices, and this application does not specifically limit this.

[0408] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform the various functions or steps described in the method embodiments.

[0409] This application also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps described in the above method embodiments.

[0410] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0411] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0412] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0413] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0414] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0415] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for displaying an interface, characterized in that, Applied to an electronic device, the electronic device including a display screen having a first refresh rate, the method includes: In response to a first instruction, a first window is displayed on the display screen, and a first interface is displayed in the first window, wherein the first instruction is used to create a first virtual screen, the first virtual screen has the first refresh rate, the first interface is mapped from the interface of the first virtual screen to the first window, the first virtual screen corresponds to a first extended display identification data EDID, the first EDID is different from at least one second EDID historically stored by the electronic device, each second EDID corresponds to a historically connected external device, or a historically created second virtual screen.

2. The interface display method according to claim 1, characterized in that, Before displaying the first window on the display screen in response to the first instruction, and before displaying the first interface in the first window, the method further includes: Run the virtual assistant application and display the first program interface of the virtual assistant application; In response to a program installation command input on the first program interface, a first response message is displayed, wherein the first response message includes at least one program installation control corresponding to the program installation command; In response to a first operation on a target program installation control in the at least one program installation control, the first instruction is generated, and the first instruction is further configured to execute a first installation process based on the first virtual screen, the first installation process being configured to install a first installation program corresponding to the target program installation control.

3. The interface display method according to claim 1, characterized in that, Before displaying the first window on the display screen in response to the first instruction, and before displaying the first interface in the first window, the method further includes: Run the virtual assistant application and display the first program interface within the virtual assistant application; In response to a program uninstallation command input on the first program interface, a second response message is displayed, wherein the second response message includes at least one program uninstallation control corresponding to the program uninstallation command; In response to a second operation on a target program uninstallation control in the at least one program uninstallation control, the first instruction is generated. The first instruction is further configured to execute a first uninstallation process based on the first virtual screen. The first uninstallation process is configured to uninstall a first uninstallation program corresponding to the target program uninstallation control.

4. The interface display method according to claim 2 or 3, characterized in that, The step of responding to a first instruction by displaying a first window on the display screen and displaying a first interface in the first window includes: In response to the first instruction, the first resolution and the first refresh rate of the display screen are obtained; The screen configuration information corresponding to the first virtual screen is initialized and configured to include the first resolution, the first refresh rate, and the first EDID. The first virtual screen is created based on the configured screen configuration information; The first window is displayed on the display screen, and the interface of the first virtual screen is mapped onto the first window to display the first interface in the first window.

5. The interface display method according to claim 4, characterized in that, The step of responding to the first instruction and obtaining the first resolution and the first refresh rate of the display screen includes: In response to the first instruction sent by the virtual assistant application to the window application, the window application obtains first display information of the display screen, the first display information including the first resolution and the first refresh rate; The window application sends the first display information to the virtual screen display driver; The virtual screen display driver obtains the first resolution and the first refresh rate based on the first display information.

6. The interface display method according to claim 5, characterized in that, The initialization configuration of the screen configuration information corresponding to the first virtual screen, including configuring the first resolution, the first refresh rate, and the first EDID in the screen configuration information, includes: The virtual screen display driver configures the screen configuration information based on the obtained first resolution and first refresh rate; The virtual screen display driver modifies the built-in EDID table to generate the first EDID, and configures the screen configuration information based on the first EDID.

7. The interface display method according to claim 6, characterized in that, Creating the first virtual screen based on the configured screen configuration information includes: In a preset screen creation mode, the virtual screen display driver creates the first virtual screen based on the configured screen configuration information. The preset screen creation mode includes an extended screen mode or a duplicate screen mode. In the extended screen mode, the display screen and the first virtual screen have different display content. In the duplicate screen mode, the display screen and the first virtual screen have the same display content.

8. The interface display method according to claim 7, characterized in that, After creating the first virtual screen based on the configured screen configuration information, the process further includes: The virtual screen display driver sends a first message to the window application, the first message indicating that the first virtual screen has been successfully created. The window application sends a second message to the virtual assistant application, the second message indicating that the first virtual screen has been successfully created.

9. The interface display method according to claim 7, characterized in that, After creating the first virtual screen based on the configured screen configuration information, the process further includes: When the preset screen creation mode is the extended screen mode, in response to the first mode switching command, the virtual screen display driver switches the first virtual screen from the extended screen mode to the duplicated screen mode; When the preset screen creation mode is the copy screen mode, in response to the second mode switching command, the virtual screen display driver switches the first virtual screen from the copy screen mode to the extended screen mode.

10. The interface display method according to claim 8, characterized in that, After responding to the first instruction, displaying the first window on the display screen, and displaying the first interface in the first window, the method further includes: When a first instruction is generated based on the first operation on the target program installation control, the first installation interface corresponding to the first installation program is displayed on the first interface; The UI Agent performs a first simulated click operation on the first installation interface. The first installation process is completed based on the first simulated click operation to install the first installation program in the electronic device.

11. The interface display method according to claim 8, characterized in that, After responding to the first instruction, displaying the first window on the display screen, and displaying the first interface in the first window, the method further includes: When a first instruction is generated based on a second operation on the target program uninstallation control, the first uninstallation interface corresponding to the first uninstallation program is displayed on the first interface; The UI Agent performs a second simulated click operation on the first uninstallation interface; The first uninstallation process is completed based on the second simulated click operation to uninstall the first uninstallation program from the electronic device.

12. The interface display method according to claim 10, characterized in that, The step of displaying the first installation interface corresponding to the first installation program on the first virtual screen includes: In response to the second message, the virtual assistant application sends a third message to the window application, the third message being used to trigger the window application to execute the first installation process; In response to the third message, the Windows application executes the first installation process to capture the first installation interface based on a hook mechanism; The Windows application sends the first installation interface to the virtual assistant application.

13. The interface display method according to claim 12, characterized in that, The UIAgent, a user interface-based intelligent agent, performs a first simulated click operation on the first installation interface, including: Upon receiving the first installation interface, the virtual assistant application executes the first simulated click operation on the first installation interface based on the UI Agent.

14. The interface display method according to claim 11, characterized in that, Displaying the first uninstallation interface corresponding to the first uninstallation program on the first virtual screen includes: In response to the second message, the virtual assistant application sends a fourth message to the window application, the fourth message being used to trigger the window application to execute the first uninstallation process; In response to the fourth message, the window application executes the first uninstallation process to capture the first uninstallation interface based on the hook mechanism; The window application sends the first uninstallation interface to the virtual assistant application.

15. The interface display method according to claim 14, characterized in that, The UIAgent, a user interface-based intelligent agent, performs a second simulated click operation on the first uninstallation interface, including: Upon receiving the first uninstallation interface, the virtual assistant application performs the second simulated click operation on the first uninstallation interface based on the UI Agent.

16. An electronic device, characterized in that, include: The electronic device includes a processor and a memory, the memory storing program instructions that, when executed by the processor, cause the electronic device to perform the interface display method as described in any one of claims 1-15.

17. A computer-readable storage medium, characterized in that, include: Computer instructions that, when executed on an electronic device, cause the electronic device to perform the interface display method as described in any one of claims 1-15.

Images