Interface display method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an interface display method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the steps of calling a hardware synthesizer to obtain first data in response to a triggering operation for a preset control of a first application program; calling a hardware synthesizer to send the first data to a wayland synthesizer through a wayland client side; calling a wayland synthesizer to obtain second data, and merging the first data and the second data to obtain third data; and calling a wayland synthesizer to send the third data to a display of the terminal, so that the display renders and displays a second interface. According to the method, the data for displaying the interface of the second system is sent to the wayland synthesizer of the first system, and the wayland synthesizer of the first system sends the data to the display for displaying, so that under the condition that the application program of the second system has an anti-screenshot mechanism, the anti-screenshot mechanism is bypassed, and screenshot is carried out through the first system; the technical problem that the user cannot perform screenshot in the scene is solved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to an interface display method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of technology, people frequently use smart devices for work, study, entertainment, and other activities. For example, they use smart devices to browse information, chat, and so on. The content being browsed is usually displayed to the user through the smart device's screen. During this process, users often want to save the content currently displayed on the screen for storage or sharing. To achieve this, existing technologies provide a screenshot function, which allows users to save the content on the screen as a screenshot image.

[0003] However, in real-world applications, some operating systems (such as Android) allow applications to request anti-screenshot permissions. When an application obtains this permission, users are unable to take screenshots of the application's interface, which can cause some inconvenience for users. Summary of the Invention

[0004] The purpose of this application is to at least solve one of the aforementioned technical defects. The technical solution provided by the embodiments of this application is as follows: In a first aspect, embodiments of this application provide an interface display method applied to a terminal, wherein the terminal simultaneously runs a first system and a second system. The first system includes a Wayland synthesizer, and the second system includes a hardware synthesizer and a Wayland client. The method specifically includes: In response to a trigger operation on a preset control of the first application, a hardware compositor is invoked to obtain first data, which is used to render a first interface of the first application displayed in response to the trigger operation; the first application is installed in the second system. The hardware synthesizer is invoked to send the first data to the Wayland synthesizer via the Wayland client; The Wayland compositor is invoked to obtain the second data, and the first and second data are merged to obtain the third data; the second data is used to describe the preset display rules of the first system; the third data is used to render the second interface, which includes the first interface displayed under the preset display rules. The Wayland compositor is invoked to send third-party data to the terminal's display, causing the display to render and show the second interface.

[0005] Secondly, embodiments of this application provide an interface display device, including: The rendering module is used to call the hardware compositor to obtain first data in response to a trigger operation of a preset control of the first application. The first data is used to render the first interface of the first application displayed in response to the trigger operation; the first application is installed in the second system. The data transmission module is used to call the hardware synthesizer to send the first data to the Wayland synthesizer through the Wayland client; The data merging module is used to call the Wayland compositor to obtain second data, and merge the first data and the second data to obtain third data; the second data is used to describe the preset display rules of the first system; the third data is used to render the second interface, which includes the first interface displayed under the preset display rules. The interface display module is used to call the Wayland compositor to send third-party data to the terminal's display, so that the display can render and display the second interface.

[0006] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory; The processor executes a computer program to implement the method provided in the first aspect embodiment or any alternative embodiment of the first aspect.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method provided in the first aspect embodiment or any optional embodiment of the first aspect.

[0008] The beneficial effects of the technical solutions provided in this application are: First, the solution provided in this application, when it is necessary to display the interface of the first application installed on the second system, does not send the relevant display data to the display screen by the hardware synthesizer of the second system, but instead sends the data to the Wayland synthesizer of the first system, thus bypassing the relevant mechanisms of the second system; Secondly, the solution provided in this application embodiment combines second data related to the preset display rules of the first system with the first data sent from the hardware synthesizer of the second system through the Wayland synthesizer of the first system, and sends the synthesized third data to the display, so that the terminal's display can display the relevant interface of the second system under the preset display rules of the first system. This allows for the complete display of the relevant interface information of the second system while bypassing the relevant mechanisms of the second system.

[0009] The solution provided in this application provides a way to bypass the "screenshot prevention mechanism" in the second system's application and take screenshots through the first system, by sending the data used to display the interface of the second system to the Wayland synthesizer of the first system, and then having the Wayland synthesizer of the first system send it to the display. This solves the technical problem that users cannot take screenshots in the above-mentioned scenarios. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0011] Figure 1 A flowchart illustrating an interface display method provided in an embodiment of this application; Figure 2 This is an example diagram of the second interface of a first application installed on a second system, as shown in one embodiment of this application. Figure 3 This is an example diagram of the third interface when the terminal has finished starting up, as shown in one embodiment of this application. Figure 4 This is a schematic diagram of an interface display method in one example of an embodiment of this application; Figure 5 A structural block diagram of an interface display device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0013] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0015] The following is an explanation of some of the concepts appearing in the embodiments of this application: Wayland compositor: A core component of modern Linux graphical interfaces, it can be understood as a window manager that integrates display server functionality, responsible for compositing the window images of various applications into the final screen image seen. Its specific functions include, but are not limited to, the following: Window composition and management: This function manages the position, size, and stacking order of windows, and composites all window content into the final desktop image. Different compositors have different window management styles; for example, Hyprland uses dynamic tiling to efficiently utilize screen space.

[0016] Input event handling: As the hub for input events (such as mouse and keyboard operations), it receives events from the system hardware and accurately dispatches them to the currently focused application window.

[0017] Provide visual effects: Many compositors support visual effects such as window transparency, shadows, and animations.

[0018] Graphics performance optimization: Improve the overall graphics performance and response speed of the system by reducing unnecessary rendering and compositing operations.

[0019] Protocol extension support: In addition to the core protocol, the synthesizer can implement various extensions of the Wayland protocol, such as XDGShell, which provides applications with richer ways to interact with the synthesizer. A Wayland client is any graphical application that uses the Wayland protocol to communicate with a Wayland compositor. In the Wayland architecture, the client does not directly control the display hardware; instead, it requests the compositor to display its content via the Wayland protocol. Its specific functions may include, but are not limited to, the following: 1. Graphical content provision: Rendering the application's own user interface, managing the application's internal graphical state, and updating and submitting new graphical content as needed.

[0020] 2. Protocol Communication: Establish a Wayland connection with the synthesizer, comply with the Wayland core protocol and its extensions, and implement the necessary Wayland interfaces.

[0021] 3. Input event handling: Receive and process input events such as mouse and keyboard inputs, update the application state based on user interaction, and provide visual feedback.

[0022] 4. Window Management Collaboration: Negotiates window behavior with the compositor via protocols such as XDG Shell, responds to window state changes (maximization, minimization, etc.), and adheres to the desktop environment's appearance specifications. Hardware compositor: Used for hardware overlay or hardware-accelerated compositing, it is a type of hardware dedicated to the graphics processing unit (GPU) used to efficiently combine multiple image layers. It can combine multiple image layers into a final image with the highest efficiency and lowest power consumption, thus providing users with a smooth, power-saving, and tear-free visual experience.

[0023] Linux Containers: An operating system-level virtualization technology that allows multiple isolated user-space instances to run simultaneously on a single Linux host. These instances are called containers. Containers can package only the application and its dependencies and utilize the host kernel to create an isolated runtime environment.

[0024] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0025] Figure 1This application provides a flowchart illustrating an interface display method, the execution subject of which can be a terminal (e.g., a computer, a mobile phone, etc.). Figure 1 As shown, the method may include: Step S101: In response to a trigger operation of a preset control for the first application, a hardware compositor is invoked to obtain first data. The first data is used to render the first interface of the first application displayed in response to the trigger operation. The first application is installed in the second system.

[0026] In the embodiments of this application, the first system can be a system with the same kernel architecture as the second system. For example, when the second system uses a Linux kernel architecture (such as Android), the first system should also use a Linux kernel architecture (such as Yuanxin system) to ensure that data can be shared between the two systems. The first application can be any APP (Application) installed on the second system. The first interface can be the interface displayed on the screen when the second system is running normally and independently on the terminal and the first application is running. The first data can be data used to render a visual interface, which is dynamic information that drives changes in the content and appearance of the interface, and is technically usually referred to as "state" or "view model".

[0027] Specifically, users can interact with the terminal by triggering relevant operations, such as "clicking the display" or "clicking a button on the terminal." After each triggered operation, the terminal responds accordingly to display the corresponding interface on its display. To display the corresponding interface, rendering data needs to be sent to the display so that it can render the data to generate the user-visible screen content (i.e., the interface). Therefore, in this embodiment, when a user interacts with the terminal by executing relevant triggered operations, the terminal generates the corresponding user-rendered interface data in real time through the corresponding compositor. Taking a first application installed in the second system as an example, when a triggered operation is performed on the first application, since the first application is installed in the second system, the hardware compositor in the second system will immediately generate the interface data generated after the triggered operation is performed on the first application, so that the result of this triggered operation can be displayed on the display later.

[0028] Step S102: The hardware synthesizer is invoked to send the first data to the Wayland synthesizer through the Wayland client.

[0029] Specifically, after generating the first data, the hardware synthesizer needs to send the first data to the Wayland synthesizer of the first system so that the Wayland synthesizer of the first system can send the data to the display. However, due to the structural differences between the hardware synthesizer and the Wayland synthesizer, they cannot directly transmit or share data. Therefore, in this embodiment, the Wayland client is chosen as the "bridge" between the two to realize data sharing between the two synthesizers. That is, after the hardware synthesizer generates the first data, it can send the first data to the Wayland client, and then the Wayland client sends the first data to the Wayland synthesizer of the first system.

[0030] Step S103: Call the Wayland compositor to obtain the second data, and merge the first data and the second data to obtain the third data; the second data is used to describe the preset display rules of the first system; the third data is used to render the second interface, which includes the first interface displayed under the preset display rules.

[0031] In the embodiments of this application, the preset display rules may include the resolution of the interface display and the UI elements of the interface display. For example, in the embodiments of this application, all interfaces will adopt the UI elements of the first system (such as battery level, time, Wi-Fi connection status, etc., which can be freely set according to actual needs). When the displayed interface is the desktop, the arrangement of each application icon can also be limited (such as the maximum number of application icons displayed per row, etc.). The embodiments of this application do not limit this.

[0032] Optionally, in this embodiment, the Wayland client actually acquires the first data by calling relevant interfaces. For example, in existing technical solutions, the Wayland client directly calls the display interface to send the first data to the display to show the second interface. However, in this embodiment, the interface it calls is rewritten so that it calls the Wayland synthesizer interface to send the first data to the Wayland synthesizer instead of directly to the display.

[0033] It should be noted that the purpose of this application is to bypass some of the "mechanisms" of the second system. The way to bypass this is to display the interface of the second system through the first system. Since the second system in this application embodiment is not running "normally" on the terminal, the "normal" one should be the first system.

[0034] Specifically, when the Wayland compositor of the first system receives the first data, it knows that the second system has a first interface that needs to be displayed on the screen. Since the interface obtained by directly rendering the second data is the interface of the second system running independently, while the system that the terminal is currently actually "normally" running is the first system, it is necessary to display the interface originally displayed by the second system using the style of the first system. Specifically, when the first data is received, the Wayland compositor can obtain the second data used to characterize the preset display rules of the first system, and then merge the first data and the second data to obtain the third data. The merged third data is used to render the first interface (i.e., the second interface) of the second system displayed under the preset display rules of the first system.

[0035] Step S104: Call the Wayland compositor to send the third data to the terminal's display so that the display can render and display the second interface.

[0036] Specifically, after the Wayland synthesizer merges the first data and the second data to obtain the third data, it can send the third data to the terminal's display. The terminal's display can then render the third data, so that the first interface of the second system is displayed in the form of the first system's preset display rules.

[0037] The solution provided in this application embodiment, firstly, when it is necessary to display the interface of the first application installed on the second system, does not have the hardware synthesizer of the second system send the relevant data for display to the display to the screen, but instead sends the data to the Wayland synthesizer of the first system, thus bypassing the relevant mechanisms of the second system; Secondly, the solution provided in this application embodiment combines second data related to the preset display rules of the first system with the first data sent from the hardware synthesizer of the second system through the Wayland synthesizer of the first system, and sends the synthesized third data to the display, so that the terminal's display can display the relevant interface of the second system under the preset display rules of the first system. This allows for the complete display of the relevant interface information of the second system while bypassing the relevant mechanisms of the second system.

[0038] The solution provided in this application provides a way to bypass the "screenshot prevention mechanism" in the second system's application and take screenshots through the first system, by sending the data used to display the interface of the second system to the Wayland synthesizer of the first system, and then having the Wayland synthesizer of the first system send it to the display. This solves the technical problem that users cannot take screenshots in the above-mentioned scenarios.

[0039] Based on the above embodiments, as an optional embodiment, the triggering operation includes a screenshot operation; After obtaining the third data, the method further includes: The Wayland synthesizer is invoked to parse the third data and determine its format. The Wayland synthesizer is invoked to construct the corresponding image matrix for the third data based on the format of the third data. Determine the target format of the image file stored in the terminal, call the Wayland synthesizer to encode the image matrix based on the target format, obtain the image file in the target format, and store the image file in the target format in the terminal.

[0040] In the embodiments of this application, the data format can be a format used to characterize image elements, such as pixel formats: RGB, BGR, RGBA, ARGB, grayscale, YUV, etc., which are not limited in this embodiment. The target format can be the format of the final stored image file, such as common image file formats like PNG and BMP. The image matrix can be constructed according to the data format; for example, for pixel data, one-dimensional data can be reshaped into a two-dimensional matrix according to the display resolution, etc. The encoding of the image matrix mainly includes two parts. One part is "image file structure creation." For example, the structure of an image file can include: a file header (containing format identifiers, version information, etc.), an information header (including metadata such as image size, color depth, compression method, etc.), pixel data (including the encoded image content), and a file footer (end marker). The other part is the specific "encoding process." The encoding process can use different methods. For example, when the target format is PNG or BMP, it can be directly stored or a compression algorithm can be used; when the target format is JPEG, lossy encoding (discrete cosine transform + quantization + entropy encoding), etc.

[0041] Specifically, when a user interacts with the first application, it may be necessary to save information from a certain first interface generated by the first application. The common method for saving this information is through screenshotting, which involves capturing and saving the content displayed on the current screen as an image file. In the application scenario of this application embodiment, some first applications installed on the second system have an "anti-screenshot mechanism" (such as...). Figure 2 As shown, Figure 2The second interface shown in the image involves bank account information, which is usually set to be unscreenable by default due to account security concerns. That is, when the terminal runs the second system alone and runs the first application within that system, screenshots of the second interfaces of these first applications cannot be stored on the terminal, making screenshotting impossible. Therefore, to solve this problem, the solution in this application embodiment is to run both the first and second systems simultaneously on the terminal. When the first application installed on the second system is run, its interface is sent to the display via the first system. In this way, the second system does not receive any "screenshot" requests during this process, and therefore cannot trigger its "anti-screenshot mechanism," thus enabling screenshotting of the second interfaces of the relevant first applications.

[0042] When a user needs to trigger a screenshot operation, they can do so by simultaneously clicking one or more preset buttons (such as side buttons) on the terminal, or by triggering a corresponding gesture operation on the display, etc., which are not limited in this embodiment. After the screenshot operation is triggered, the Wayland synthesizer will consider the second interface corresponding to the third data that will be output when the operation is triggered as the screenshot that the user needs to save. Therefore, it needs to convert the second interface corresponding to the third data into an image and store it in the terminal to achieve the purpose of taking a screenshot. Specifically, the third-party data can be parsed first to determine its format. Then, a suitable image matrix can be constructed based on the format of the third-party data. Different formats of third-party data will use different methods to construct different image matrices. After the image matrix is ​​constructed, different compression methods are selected to further encode it according to the image format to be stored on the terminal (e.g., when saving as a PNG image, a lossless compression algorithm can be used to store the pixel array, while when saving as a JPEG image, a lossy compression algorithm can be used to discard some visual information that is not sensitive to the human eye to significantly reduce the file size). For example, each value in the matrix can be mapped to a color spectrum (e.g., blue represents a low value, red represents a high value), etc. This embodiment of the application does not limit this. After encoding, the corresponding format image file can be obtained. At this time, storing the image file on the terminal completes the screenshot.

[0043] It should be noted that the above-described embodiments of this application provide only an exemplary screenshot process. Any improvements and modifications made without departing from the principles of the embodiments of this application should also fall within the protection scope of this application.

[0044] Based on the above embodiments, as an optional embodiment, the first system and the second system are started in the following manner: In response to a startup operation targeting the terminal, the first system is started; Create an isolated environment in the first system, and then start the second system in the isolated environment.

[0045] In the embodiments of this application, the isolated environment can also be called a "container", which can be implemented by a Linux container. Its core is to isolate the second system so that the triggered screenshot operation is not monitored by it, thereby bypassing its "anti-screenshot mechanism".

[0046] Specifically, when a user performs a startup trigger operation on the terminal (startup could be by clicking a preset "power on" button on the terminal, etc.), the terminal will first start the first system it has installed. After the first system starts up, it will further create an isolated environment in the first system and start the second system in the isolated environment. In this way, the second system runs in the isolated environment of the first system. This startup method not only successfully runs two systems at the same time, but the terminal also believes that the only system it is "normally" running is the first system. Subsequent interaction processes will be based on the relevant mechanisms of the first system, thus bypassing the relevant mechanisms of the second system.

[0047] It should be noted that the startup method of the terminal in this application embodiment can be pre-set (such as setting a corresponding script), that is, the above-mentioned startup method will be used every time it is started.

[0048] Based on the above embodiments, as an optional embodiment, both the first system and the second system adopt the Linux kernel architecture; Creating an isolated environment in the first system specifically includes: At least one Linux kernel module for running Linux Container tools is loaded into the first system. The Linux Container tools are used to create an isolated environment in the first system. The Linux Container tool is run based on at least one Linux kernel module, and an isolated environment is created in the first system using the Linux Container tool.

[0049] In the embodiments of this application, a Linux kernel module can be a piece of code that can be dynamically loaded into or unloaded from the kernel during Linux kernel runtime, thereby extending kernel functionality without requiring system restart, modification of kernel source code, or recompilation of the entire kernel.

[0050] Specifically, after the first system boots up, in order to create an isolated environment for running the second system, a Linux container tool for creating the isolated environment needs to be prepared. This tool requires the support of corresponding Linux kernel modules to run. Therefore, one or more corresponding Linux kernel modules need to be loaded first, and then the tool needs to be run with the support of the Linux kernel modules to complete the creation of the isolated environment. The following will explain the process in several segments using a specific example (taking the second system as an Android system and Anbox compatibility layer (an open-source compatibility layer that can run Android applications on a regular GNU / Linux system like running local applications)): I. Preliminary Preparations: Environment Dependencies and Module Loading 1. Linux kernel module loading: Anbox requires two core kernel modules, ashmem_linux (for efficient inter-process data exchange) and binder_linux (to simulate Android's Binder IPC mechanism), to enable communication between Android and the host.

[0051] 2. Anbox service starts: Anbox relies on system services to manage the container lifecycle. The service anbox-container-manager is started, which is responsible for managing the creation / destruction of LXC (Linux Containers), maintaining the Android system image, and handling resource mapping (network, graphics, input, etc.) between the host and the container.

[0052] II. Container Initialization: Creating an LXC Container Anbox creates an isolated environment based on LXC. The container initialization steps are as follows: 1. Generate a temporary LXC configuration file (path: / var / lib / anbox / lxc / config) 2. Mount the Android image of Anbox with a union file system. 3. Start the LXC container. Anbox uses the LXC tool to create and start the container. III. Android System Startup: Initialization Process within the Container After the container starts, it automatically executes the Android system's initialization script (similar to the startup process of a real Android device): 1. Start the init process: The first process inside the container is Android's init program (path: / init), which is responsible for parsing the init.rc script and initializing system services. 2. Start the Binder and Ashmem services servicemanager: Manages the registration and querying of Binder IPC services.

[0053] ashmemd: Manages anonymous shared memory regions for use by processes such as graphics rendering and media processing.

[0054] 3. Start the graphics service (SurfaceFlinger) SurfaceFlinger: Android's graphics compositing service, responsible for combining application windows, status bars, and other layers into the final image.

[0055] Anbox uses the libhybris library to forward SurfaceFlinger's OpenGL ES rendering commands to the host's GPU driver, thus achieving hardware acceleration.

[0056] 4. Start the system service (SystemServer). Start the core services of the Android framework layer, including: ActivityManagerService: Manages the application lifecycle (starting / stopping Activities).

[0057] PackageManagerService: Manages application installation and permissions.

[0058] WindowManagerService: Manages window layout and input events (mouse / keyboard events are forwarded via Anbox).

[0059] 5. Launch the desktop application (Launcher) Once the system services are ready, Anbox's default desktop application (org.anbox.appmgr) will start automatically, allowing users to manage and launch other Android applications.

[0060] Based on the above embodiments, as an optional embodiment, after starting the second system in the isolated environment, the method further includes: The third interface is displayed, which includes icons of various applications installed on the first system and various application icons on the second system, and the application icons are arranged according to the preset display rules of the first system.

[0061] In embodiments of this application, an application icon can be a unique visual symbol or graphic representing a specific application, software, or program. It serves as the entry point for a user to launch the application from an interface on a terminal (such as the desktop, start menu, or application library).

[0062] Specifically, when both the first and second systems have finished booting up, the terminal will display a third interface indicating boot completion. This third interface can include all application icons from both the first and second systems (such as...). Figure 3 As shown, the icons include those for applications such as "Calendar," "Clock," "Settings," "Camera," "File Manager," "Enterprise App Store," "Calculator," "Compass," "Music," "Video," "Photos," "Notepad," "Sogou Input Method," and "China Merchants Bank." Below these are icons for applications such as "Phone," "Messages," "Contacts," and "Browser." All icons are arranged according to the first system's preset display rules (e.g., ...). Figure 3 To allow a maximum of four application icons to be displayed in the same row. Optionally, the third interface may also include UI framework elements from the first system (such as...). Figure 3 The top left corner displays the time, the top right corner displays the Wi-Fi connection indicator, the current "Airplane Mode" indicator, the battery level indicator, and below are quick operation controls, as well as the first system's preset desktop background.

[0063] Optionally, there may be multiple third interfaces in this application embodiment. When the terminal is a mobile device such as a mobile phone, all application icons of the first system and the second system can be displayed through multiple third interfaces. Users can switch between the various third interfaces by triggering operations such as swiping. However, the other parts of each third interface are completely the same except for the application icons (i.e., UI framework elements, desktop background, etc.).

[0064] Based on the above embodiments, as an optional embodiment, the method further includes: For each application icon installed on the second system in the third interface, a preset mark is set around the application icon; the preset mark is used to indicate that the application icon is installed on the second system.

[0065] Specifically, in some terminals, users may install two identical applications on the first and second systems. In this case, after the terminal starts, the third interface will display two identical application icons, making it difficult for users to distinguish which system the application they are launching originates from. To solve this problem, this application embodiment can mark the application icons from the second system. Specifically, a preset marker is set around each application icon from the second system. In this way, even if two identical applications are installed on the first and second systems respectively, users can distinguish the system location where the application is installed by the presence or absence of the preset marker, thus improving the user experience.

[0066] Based on the above embodiments, as an optional embodiment, the method further includes: In response to a trigger operation on a preset control of the second application, the Wayland compositor is invoked to obtain fourth data. The fourth data is used to render the fourth interface of the second application displayed in response to the trigger operation. The fourth interface is displayed under preset display rules. The second application is installed on the first system. The Wayland compositor is invoked to send the fourth data to the display so that the display can render and display the fourth interface.

[0067] Specifically, when a user interacts with the second application installed in the first system, the triggering operation is detected by the Wayland compositor of the first system. At this time, the Wayland compositor knows that the first system has an interface that needs to be displayed on the screen. Since the Wayland compositor itself serves the first system, it can directly obtain the fourth data of the fourth interface of the second application used for display and send it to the terminal's screen for display. The terminal's screen can then render the fourth data and display the corresponding fourth interface. The interface displayed in the fourth interface is naturally the interface of the second application displayed under the preset display rules of the first system.

[0068] The following will combine Figure 4 This paper provides an overall flowchart of the screenshot method provided in the embodiments of this application, such as... Figure 4 As shown: First, users can start the terminal by clicking the "Power On" button. After the terminal starts, the first installed system will be launched first. After the first system is launched, an isolated environment (also known as a "container") will be created in the first system. Then, the second system will be launched in the isolated environment.

[0069] After the second system starts up, it will send the first data (the first data here is used to render the first interface when the second system starts up, excluding the UI framework elements of the second system) to the Wayland client of the second system through its own hardware compositor (corresponding to HardwareCompositor in the figure). The Wayland client will send the first data to the Wayland compositor of the first system through its preset interface.

[0070] Then, the Wayland synthesizer of the first system further synthesizes the second data with the first data transmitted from the second system to obtain the third data, and outputs the third data to the display so that the display shows the second interface. The second interface includes icons of each application installed on the first system, icons of each application installed on the second system, and UI framework elements of the first system, and the icons of each application are arranged according to the row and column sorting rules specified by the first system (such as a maximum of 4 application icons per row).

[0071] At this time, the terminal is running both the first and second systems simultaneously. Users can select the application to run by clicking the corresponding application icon on the monitor.

[0072] If a user wants to run an application installed on the first system, they can click the application's icon displayed on the screen. At this time, the Wayland compositor of the first system can directly send the initial data for rendering the application's initial interface to the screen so that the screen can display the initial interface. Afterwards, the user can interact with the application by clicking on the screen. During the interaction, the Wayland compositor of the first system will send the initial data for rendering the application interface to the screen in real time so that the corresponding application interface is displayed on the screen in real time.

[0073] If the user wants to run an application installed on the second system, they can click the application's icon displayed on the screen. The second system's hardware compositor will then send the initial data for rendering the application's interface to the first system's Wayland compositor via a preset interface in the Wayland client. The first system's Wayland compositor will then composite the initial data from the second system with the second data used to display the first system's UI framework elements. The composited data is then sent to the screen, allowing the screen to display the application's initial interface, including the first system's UI framework elements. The user can then interact with the application by clicking on the screen. During this interaction, the first system's Wayland compositor will continuously composite the first and second data sent from the second system with the application's interface, sending the composited data back to the screen in real-time, ensuring the screen displays the application interface, including the first system's UI framework elements.

[0074] When a user needs to take a screenshot, they can click (or trigger) the relevant preset button on the terminal. After the Wayland synthesizer of the first system detects the trigger operation, it will copy the third data that is about to be output to the display, encode the copied third data to obtain the corresponding image file, and then store the image file in the terminal to realize the screenshot of the application interface. At the same time, the first data is sent to the display so that the display shows the application interface.

[0075] Figure 5 A structural block diagram of an interface display device provided in an embodiment of this application is shown below. Figure 5 As shown, the interface display device 500 may include: a dataset acquisition module 501, a dataset analysis module 502, and a fault prediction module 503, wherein, The rendering module 501 is used to call the hardware compositor to obtain first data in response to a trigger operation of a preset control for the first application. The first data is used to render the first interface of the first application displayed in response to the trigger operation. The first application is installed in the second system. The data transmission module 502 is used to call the hardware synthesizer to send the first data to the Wayland synthesizer through the Wayland client; The data merging module 503 is used to call the Wayland compositor to obtain the second data, and merge the first data and the second data to obtain the third data; the second data is used to describe the preset display rules of the first system; the third data is used to render the second interface, which includes the first interface displayed under the preset display rules. The interface display module 504 is used to call the Wayland synthesizer to send the third data to the terminal's display so that the display can render and display the second interface.

[0076] The solution provided in this application embodiment, firstly, when it is necessary to display the interface of the first application installed on the second system, does not have the hardware synthesizer of the second system send the relevant data for display to the display to the screen, but instead sends the data to the Wayland synthesizer of the first system, thus bypassing the relevant mechanisms of the second system; Secondly, the solution provided in this application embodiment combines second data related to the preset display rules of the first system with the first data sent from the hardware synthesizer of the second system through the Wayland synthesizer of the first system, and sends the synthesized third data to the display, so that the terminal's display can display the relevant interface of the second system under the preset display rules of the first system. This allows for the complete display of the relevant interface information of the second system while bypassing the relevant mechanisms of the second system.

[0077] The solution provided in this application provides a way to bypass the "screenshot prevention mechanism" in the second system's application and take screenshots through the first system, by sending the data used to display the interface of the second system to the Wayland synthesizer of the first system, and then having the Wayland synthesizer of the first system send it to the display. This solves the technical problem that users cannot take screenshots in the above-mentioned scenarios.

[0078] Based on the above embodiments, as an optional embodiment, the device further includes a screenshot module, specifically used for: The Wayland synthesizer is invoked to parse the third data and determine its format. The Wayland synthesizer is invoked to construct the corresponding image matrix for the third data based on the format of the third data. Determine the target format of the image file stored in the terminal, call the Wayland synthesizer to encode the image matrix based on the target format, obtain the image file in the target format, and store the image file in the target format in the terminal.

[0079] Based on the above embodiments, as an optional embodiment, the device further includes a startup module, specifically used for: In response to a startup operation targeting the terminal, the first system is started; Create an isolated environment in the first system, and then start the second system in the isolated environment.

[0080] Based on the above embodiments, as an optional embodiment, both the first system and the second system adopt the Linux kernel architecture; The startup module is specifically used for: At least one Linux kernel module for running Linux Container tools is loaded into the first system. The Linux Container tools are used to create an isolated environment in the first system. The Linux Container tool is run based on at least one Linux kernel module, and an isolated environment is created in the first system using the Linux Container tool.

[0081] Based on the above embodiments, as an optional embodiment, the interface display module is further used for: The third interface is displayed, which includes icons of various applications installed on the first system and various application icons on the second system, and the application icons are arranged according to the preset display rules of the first system.

[0082] Based on the above embodiments, as an optional embodiment, the device further includes a preset marker adding module, specifically used for: For each application icon installed on the second system in the third interface, a preset mark is set around the application icon; the preset mark is used to indicate that the application icon is installed on the second system.

[0083] Based on the above embodiments, as an optional embodiment, the interface display module can also be used for: In response to a trigger operation on a preset control of the second application, the Wayland compositor is invoked to obtain fourth data. The fourth data is used to render the fourth interface of the second application displayed in response to the trigger operation. The fourth interface is displayed under preset display rules. The second application is installed on the first system. The Wayland compositor is invoked to send the fourth data to the display so that the display can render and display the fourth interface.

[0084] The following is for reference. Figure 6 It illustrates an electronic device suitable for implementing embodiments of this application (e.g., performing...). Figure 1 The diagram shows the structure of the terminal device or server 600 of the method shown. The electronic devices in the embodiments of this application may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (e.g., vehicle navigation terminals), wearable devices, etc., as well as fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0085] The electronic device includes a memory and a processor. The memory stores a program for executing the methods described in the various method embodiments above. The processor is configured to execute the program stored in the memory. The processor may be referred to as processing device 601 as described below. The memory may include at least one of read-only memory (ROM) 602, random access memory (RAM) 603, and storage device 608 as described below, as follows: like Figure 6As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0086] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0087] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 609, or installed from storage device 608, or installed from ROM 602. When the computer program is executed by processing device 601, it performs the functions defined in the methods of embodiments of this application.

[0088] It should be noted that the computer-readable storage medium described above in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0089] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0090] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0091] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: In response to a trigger operation on a preset control of a first application, a hardware compositor is invoked to obtain first data. The first data is used to render a first interface of the first application displayed in response to the trigger operation. The first application is installed on a second system. The hardware compositor sends the first data to the Wayland compositor via a Wayland client. The Wayland compositor is invoked to obtain second data, and the first and second data are merged to obtain third data. The second data is used to adjust the first interface based on the preset display rules of the first system. The third data is used to render a second interface, which includes the first interface displayed under the preset display rules. The third data is sent to the terminal's display so that the display renders and displays the second interface.

[0092] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0094] The modules or units described in the embodiments of this application can be implemented in software or hardware. The names of modules or units do not necessarily limit the specific unit; for example, a first constraint acquisition module can also be described as a "module for acquiring the first constraint".

[0095] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0096] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0097] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0098] The above description is only a partial embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An interface presentation method, characterized by, The application is applied to a terminal, the terminal runs a first system and a second system simultaneously, the first system comprises a wayland compositor, the second system comprises a hardware compositor and a wayland client, and the interface display method comprises: In response to a trigger operation of a preset control of a first application, the hardware compositor is called to obtain first data, the first data is used to render a first interface of the first application displayed in response to the trigger operation; the first application is installed in the second system; The hardware compositor is called to send the first data to the wayland compositor through the wayland client; The wayland compositor is called to obtain second data, and the first data and the second data are merged to obtain third data; the second data is used to describe a preset display rule of the first system; the third data is used to render a second interface, and the second interface comprises the first interface displayed under the preset display rule; The wayland compositor is called to send the third data to a display of the terminal, so that the display renders and displays the second interface.

2. The method of claim 1, wherein, The trigger operation comprises a screenshot operation; The third data is obtained, and then the method further comprises: The wayland compositor is called to analyze the third data and determine the format of the third data; The wayland compositor is called to construct a corresponding image matrix for the third data based on the format of the third data; A target format of an image file stored in the terminal is determined, the wayland compositor is called to encode the image matrix based on the target format, an image file in the target format is obtained, and the image file in the target format is stored in the terminal.

3. The method of claim 1, wherein, The first system and the second system are started in the following manner: In response to a start operation of the terminal, the first system is started; An isolated environment is created in the first system, and the second system is started in the isolated environment.

4. The method of claim 3, wherein, The first system and the second system both adopt a Linux kernel architecture; The isolated environment is created in the first system, and the method comprises: At least one Linux kernel module for running a Linux Container container tool is loaded in the first system, the Linux Container container tool is used to create the isolated environment in the first system; The Linux Container container tool is run based on the at least one Linux kernel module, and the isolated environment is created in the first system through the Linux Container container tool.

5. The method of claim 3, wherein, After the second system is started in the isolated environment, the method further comprises: A third interface is displayed, the third interface comprises application icons of the first system and application icons of the second system, and the application icons are arranged according to a preset display rule of the first system.

6. The method of claim 5, wherein, The method further comprises: For each application icon installed in the second system in the third interface, a preset mark is set around the application icon; the preset mark is used to indicate that the application icon is installed in the second system.

7. The method of claim 1, wherein, The method further comprises: In response to a triggering operation on a preset control of a second application, the wayland compositor is invoked to obtain fourth data, the fourth data being used to render a fourth interface of the second application displayed in response to the triggering operation, the fourth interface being displayed under the preset display rule; the second application being installed in the first system; The wayland compositor is invoked to send the fourth data to the display, so that the display renders and displays the fourth interface.

8. An interface presentation device, characterized by Comprise: In response to a triggering operation on a preset control of a first application, the hardware compositor is invoked to obtain first data, the first data being used to render a first interface of the first application displayed in response to the triggering operation; The first application being installed in the second system; The hardware compositor is invoked to send the first data to the wayland compositor through the wayland client; The wayland compositor is invoked to obtain second data, and the first data and the second data are merged to obtain third data; the second data being used to describe a preset display rule of the first system adjusting the first interface; The third data being used to render a second interface, the second interface comprising the first interface displayed under the preset display rule; An interface display module is configured to invoke the wayland compositor to send the third data to a display of the terminal, so that the display renders and displays the second interface.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program comprises instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1-8. The processor executes the computer program to implement the method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-7.

Images