Display method and device of virtual desktop

Abstract

The application provides a virtual desktop display method, which comprises the following steps: creating a view proxy container instance; binding the view proxy container instance to a desktop wallpaper rendering thread of a desktop wallpaper service engine; loading picture data into the desktop wallpaper rendering thread through the view proxy container instance; and rendering the picture data through the desktop wallpaper rendering thread to display a virtual picture on a desktop. According to the technical scheme provided by the application, the desktop can support various dynamic pictures such as games and live broadcasts as wallpapers, and the playability is good.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, and computer-readable storage medium for displaying a virtual desktop. Background Technology

[0002] With the development of computer technology, screen-equipped electronic products such as mobile phones, computers, and tablets have become everyday necessities. Users can set their favorite desktop wallpapers, such as a static image. Operating systems generally provide wallpaper services. However, the wallpaper services provided by current operating systems do not support rendering games or live streams, resulting in limited usability.

[0003] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, computer device, and computer-readable storage medium for displaying a virtual desktop, in order to solve the above-mentioned problems.

[0005] The first aspect of this application provides a method for displaying a virtual desktop, the method comprising:

[0006] Create a view proxy container instance;

[0007] Bind the view proxy container instance to the desktop wallpaper rendering thread of the desktop wallpaper service engine;

[0008] The view proxy container instance loads the screen data into the desktop wallpaper rendering thread; and

[0009] The screen data is rendered through the desktop wallpaper rendering thread to display a virtual screen on the desktop.

[0010] Optionally, the method further includes:

[0011] Listen for interactive events through the desktop wallpaper service engine;

[0012] The desktop wallpaper service engine provides the event information of the interactive events to the view proxy container instance.

[0013] The event information of the interactive event is provided to the target engine through the view proxy container instance, so that the target engine executes the corresponding preset logic and updates the screen data based on the drawing layer of the target engine;

[0014] The updated screen data is provided to the desktop wallpaper rendering thread through the view proxy container instance;

[0015] The updated screen data is rendered through the desktop wallpaper rendering thread to update the desktop wallpaper.

[0016] Optionally, the method further includes:

[0017] By switching the rendering layer of the target engine, the screen of other games or live streams can be used as the wallpaper of the desktop.

[0018] Optionally, the method further includes:

[0019] Provide a preliminary background image during the resource package loading process;

[0020] Generate a target background image based on the current wallpaper status of the desktop and the prepared background image;

[0021] A background texture is generated based on the target background image using the view proxy container instance;

[0022] The background texture is provided to the desktop wallpaper rendering thread through the view proxy container instance; and

[0023] The background texture is rendered using the desktop wallpaper rendering thread, and the rendering result is displayed as wallpaper on the desktop.

[0024] Optionally, the method further includes:

[0025] Once the first frame of the target object has been drawn, switch to having the target engine's drawing layer provide the data.

[0026] Optionally, the method further includes:

[0027] The image data drawn by the target engine is rendered into the target texture, which is associated with the FBO;

[0028] Render the video frame data in the target video onto a shared texture, which is a texture used by multiple threads;

[0029] A texture blending operation is performed on the target texture and the shared texture to obtain a target blended texture;

[0030] Display the content of the target blended texture on the desktop.

[0031] Optionally, the step of performing a texture blending operation on the target texture and the shared texture to obtain a target blended texture includes:

[0032] The target texture is overlaid on the shared texture to obtain the target blended texture.

[0033] A second aspect of this application provides a virtual desktop display device, the device comprising:

[0034] Create a module for creating view proxy container instances;

[0035] The binding module is used to bind the view proxy container instance to the desktop wallpaper rendering thread of the desktop wallpaper service engine;

[0036] The loading module is used to load the image to be drawn into the desktop wallpaper rendering thread through the view proxy container instance; and

[0037] The rendering module is used to render the image data through the desktop wallpaper rendering thread to display a virtual image on the desktop.

[0038] A third aspect of this application provides a method for displaying a virtual desktop, the method comprising:

[0039] Acquire screen data of the target application, including video screen data or game screen data;

[0040] In response to a preset operation, a virtual screen corresponding to the screen data is displayed on the desktop.

[0041] Optionally, displaying the virtual screen corresponding to the screen data on the desktop includes the steps of the virtual desktop display method described in the third aspect.

[0042] A fourth aspect of this application provides a computer device, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described virtual desktop display method.

[0043] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the above-described virtual desktop display method.

[0044] The virtual desktop display method, apparatus, computer device, and computer-readable storage medium provided in this application embodiment take over the desktop wallpaper rendering thread and bind the view proxy container instance to the desktop wallpaper rendering thread, so that the rendering operation of the target engine (such as a game engine) can be implemented in the desktop wallpaper rendering thread. This is equivalent to wrapping a drawing layer of the target engine in the desktop wallpaper rendering thread, allowing the image data of the target engine's drawing layer to be rendered in the desktop wallpaper rendering thread, and the rendering result of the image data to be displayed as wallpaper on the desktop. Therefore, in this application embodiment, the desktop supports using real-time game or live stream footage as wallpaper, offering good playability. Attached Figure Description

[0045] Figure 1 This diagram schematically illustrates the operating environment of a recording system according to an embodiment of this application.

[0046] Figure 2 A flowchart illustrating a method for displaying a virtual desktop according to Embodiment 1 of this application is shown schematically.

[0047] Figure 3 The drawing process is illustrated schematically;

[0048] Figure 4 This diagram illustrates the loading and rendering process.

[0049] Figure 5 The flowchart illustrating the additional steps of the virtual desktop display method according to Embodiment 1 of this application is shown in the schematic diagram.

[0050] Figure 6 This diagram illustrates the information flow of interactive clicks.

[0051] Figure 7 This illustration shows the desktop wallpaper during the game loading process;

[0052] Figure 8 The flowchart illustrating the additional steps of the virtual desktop display method according to Embodiment 1 of this application is shown in the schematic diagram.

[0053] Figure 9 The flowchart illustrating the additional steps of the virtual desktop display method according to Embodiment 1 of this application is shown in the schematic diagram.

[0054] Figure 10 This diagram illustrates the process of simultaneously loading videos and games as desktop wallpapers;

[0055] Figure 11 A block diagram schematically illustrates a virtual desktop display device according to Embodiment 2 of this application;

[0056] Figure 12A schematic diagram illustrating the hardware architecture of a computer device according to Embodiment 3 of this application is shown; and

[0057] Figure 13 A flowchart illustrating a method for displaying a virtual desktop according to Embodiment 5 of this application is shown schematically. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0059] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0060] In the description of this application, it should be understood that the numerical labels before the steps do not indicate the order of the steps, but are only used to facilitate the description of this application and to distinguish each step, and therefore should not be construed as a limitation of this application.

[0061] Explanation of terms used in this application:

[0062] OpenGL ES (OpenGL for Embedded Systems) is a subset of the OpenGL 3D graphics API, designed for embedded devices such as mobile phones and PDAs. OpenGL (Open Graphics Library) refers to a professional graphics programming interface that defines a cross-programming language, cross-platform programming interface specification. It is used for 3D graphics (and also 2D), and is a powerful and easy-to-use low-level graphics library.

[0063] Android is a free and open-source operating system based on the Linux kernel. It can be used in mobile devices such as smartphones and tablets, and is led and developed by Google and the Open Handset Alliance.

[0064] An FBO (Frame Buffer Object) is a container that can have buffers added to it, including textures and render buffer objects.

[0065] A texture is a photograph or a series of data that can be used to add details to an object.

[0066] An API (Application Programming Interface) is a collection of definitions, procedures, and protocols. The main function of an API is to provide a common set of functionalities. Developing applications by calling API functions can reduce programming tasks. APIs also serve as middleware, enabling data sharing across various platforms.

[0067] A Surface is a handle to a raw buffer, which can be used to draw on the raw buffer.

[0068] SurfaceView encapsulates and manages a Surface. You can use SurfaceView to perform drawing operations on the Surface.

[0069] GLSurfaceView (view proxy container), which inherits from SurfaceView (a wrapper around SurfaceView), allows the use of OperaGL in Android.

[0070] GLThread is a rendering thread that comes with GLSurfaceView and can be used to perform OperaGL drawing tasks.

[0071] The onDrawFrame function is called to refresh the screen each time it is called, so that the latest frame represented by the current element is drawn using OpenGL.

[0072] WallpaperService is responsible for displaying live wallpapers.

[0073] WallpaperService.Engine (desktop wallpaper service engine) is an inner class of WallpaperService. All the drawing processes for displaying dynamic wallpapers are completed by this class.

[0074] WallpaperManagerService is used to manage the operation and switching of wallpapers. It provides an interface for manipulating wallpapers to the outside world through the WallpaperManager class, which mainly reflects Android's management method for wallpapers.

[0075] To facilitate understanding of the technical solutions provided in the embodiments of this application by those skilled in the art, the relevant technologies are described below:

[0076] Research revealed that the Android system provides Wallpaper Service, exposing drawing handles to developers and allowing them to customize desktop rendering content. Dynamic desktops can be implemented using Wallpaper Service + MediaPlayer. However, Android devices suffer from severe fragmentation, making video size adaptation costly. Furthermore, relying solely on video content limits interactive content creation, resulting in poor sustainability, low playability, and a limited business model. Additionally, existing mini-games offer rich rendering content, such as rendering virtual idol products onto the desktop, but the Android desktop does not support game rendering.

[0077] Therefore, this application provides a virtual desktop display solution. Based on this, the desktop (such as an Android virtual desktop) can run interactive games or live streaming scenarios, and can also dynamically switch between different games or live streaming rooms, improving interactivity and playability. See below for details.

[0078] Figure 1 The diagram illustrates the environment in which a virtual desktop display method according to an embodiment of this application is operated.

[0079] Mobile terminal 2 may include any type of computing device using the Android operating system, such as mobile devices, tablets, laptops, virtual reality devices, gaming devices, set-top boxes, in-vehicle terminals, and smart TVs.

[0080] Mobile terminal 2 can be used to run games or live streaming applications. The game application provides functions such as game login, operation, and payment. The game application can be an online game or a local offline application on the mobile terminal. The live streaming application can also provide functions such as login, sending gifts, sending bullet comments, and redirecting to product links.

[0081] In mobile terminal 2, game or live stream footage can be configured as desktop wallpaper and allows for interaction between the game or live stream program and the user. That is, the dynamic footage of the game or live stream program can be displayed to the user in real time as desktop wallpaper. In some embodiments, the content displayed as desktop wallpaper may include the game or live stream login page, live stream footage, comments, and / or the like. Of course, audio can also be output with the user's permission.

[0082] Example 1

[0083] Figure 2A flowchart illustrating a method for displaying a virtual desktop according to Embodiment 1 of this application is shown schematically.

[0084] like Figure 2 As shown, the method for displaying the virtual desktop may include steps S200 to S206, wherein:

[0085] Step S200: Create a view proxy container instance.

[0086] Step S202: Bind the view proxy container instance to the desktop wallpaper rendering thread of the desktop wallpaper service engine.

[0087] Step S204: Load the screen data into the desktop wallpaper rendering thread through the view proxy container instance.

[0088] Step S206: Render the image data through the desktop wallpaper rendering thread to display a virtual image on the desktop.

[0089] The virtual desktop display method provided in this application takes over the desktop wallpaper rendering thread of WallpaperService.Engine and binds the GLSurfaceView instance to the desktop wallpaper rendering thread, thereby enabling the rendering operation of the target engine (such as a game engine or live streaming engine) to be implemented in the desktop wallpaper rendering thread. Essentially, a drawing layer of the target engine is wrapped in the desktop wallpaper rendering thread, and the image data of the target engine's drawing layer can be rendered in the desktop wallpaper rendering thread, with the rendering result displayed as wallpaper on the desktop. Therefore, in this application embodiment, the Android desktop supports using dynamic game footage as wallpaper on the mobile terminal 2, as well as dynamic footage from live streaming or other applications, offering good playability.

[0090] The following combination Figure 2 The steps in steps S200 to S206 and other newly added steps are described in detail.

[0091] It should be noted that the technical solutions provided in this application are implemented in the Android operating system environment and depend on some services in Android operation, such as system wallpaper service and related services.

[0092] First, a wallpaper service instance is created, and a callback is made to create the corresponding WallpaperService.Engine and Surface handle. The wallpaper service is a system wallpaper service provided by the Android system. Then, based on the aforementioned WallpaperService.Engine and Surface handle, support for subsequent game or live streaming desktops is provided.

[0093] A desktop provides a surface handle, and the Surface callback lifecycle is completed when the Wallpaper Service instance is created, meaning the desktop's GLThread is already created. However, the target engine has some preliminary validation logic that is not in the GLThread thread; subsequent image rendering only occurs in the GLThread thread, creating a gap and preventing subsequent image binding. One solution is to render the game or live stream image off-screen and then synchronously render it onto the desktop. However, since the Wallpaper Service only provides one on-screen WallpaperService.Engine and one rendering thread, it cannot provide the target engine with a second independent rendering thread, making the above solution impractical. Therefore, this embodiment adopts the following approach: it takes over the desktop wallpaper rendering thread of WallpaperService.Engine, proxies the target engine's rendering thread, and transfers the work of the target engine's rendering thread to the desktop wallpaper rendering thread. See below for details.

[0094] Step S200 Create a view proxy container (GLSurfaceView) instance.

[0095] This GLSurfaceView instance specifies a Surface handle. This Surface handle is provided by WallpaperService.Engine.

[0096] GLSurfaceView instances can be used to provide game / live streaming canvas containers that provide proxies, as well as for distributing interactive events.

[0097] Step S202 The view proxy container instance is bound to the desktop wallpaper rendering thread of the desktop wallpaper service engine.

[0098] Once this GLSurfaceView instance is bound to the desktop wallpaper rendering engine, the rendering of games / live streams can be delegated to the desktop wallpaper rendering thread. In other words, the wallpaper rendering thread proxies the work of the target engine's rendering thread.

[0099] Step S204 The view proxy container instance loads the screen data into the desktop wallpaper rendering thread.

[0100] In the example application, the desktop wallpaper rendering engine requests frame data from the GLSurfaceView instance every certain period of time (e.g., 16.7 milliseconds) by calling the onDrawFrame method. Specifically, the GLSurfaceView instance delegates the onDrawFrame event callback from the system to the target engine. Therefore, the target engine needs to provide frame data for one frame every 16.7 milliseconds so that the GLSurfaceView instance can respond to the desktop wallpaper rendering engine's request and provide the aforementioned frame data to the desktop wallpaper rendering thread.

[0101] Step S206 The desktop wallpaper rendering thread renders the image data to display a virtual image on the desktop.

[0102] The virtual screen can be a game scene, a live stream, or a screen from another third-party application.

[0103] The desktop wallpaper rendering thread (WallpaperGLRender) uses the onDrawFrame function to draw using OpenGL ES instructions, pushing the rendered image to the Swap buffer for display on the screen. For example... Figure 3 As shown, the operation process can be as follows: use the compiled vertex program → activate the texture unit → bind the texture ID → bind the vertex data → draw and render → push the data to the Swap buffer.

[0104] This virtual screen serves as the desktop wallpaper and updates in real time as the game or live stream progresses, such as every 16.7 milliseconds. Of course, in some embodiments, to save power, it can update at longer intervals.

[0105] To make this embodiment easier to understand, combined with Figure 4 The following provides an exemplary game screen loading and rendering process.

[0106] S400, callback creates WallpaperService.Engine and Surface handle;

[0107] S402, Create a GLSurfaceView instance; where the Surface handle returned by the GLSurfaceView instance is provided by WallpaperService.Engine;

[0108] S404, binds a GLSurfaceView instance and a WallpaperGLRender;

[0109] S406 uses the onDrawFrame function to retrieve and render game screen data from a GLSurfaceView instance every 16.7 milliseconds; this data is provided by the game engine's rendering layer (GameGL Render). During loading, other wallpapers can be displayed first. If a loading error occurs, the process exits or displays an error message.

[0110] In optional embodiments, interactive functionality with the user may also be provided. For example... Figure 5 As shown, the method further includes:

[0111] Step S500: Listen for interactive events through the desktop wallpaper service engine;

[0112] Step S502: The desktop wallpaper service engine provides the event information of the interactive event to the view proxy container instance;

[0113] Step S504: The event information of the interactive event is provided to the target engine through the view proxy container instance, so that the target engine executes the corresponding preset logic and updates the screen data based on the drawing layer of the target engine;

[0114] Step S506: Provide the updated screen data to the desktop wallpaper rendering thread through the view proxy container instance;

[0115] Step S508: Render the updated screen data through the desktop wallpaper rendering thread to update the desktop wallpaper.

[0116] The interaction event can be an onTouch event. OnTouch events can include the following: ACTION_DOWN (press screen), ACTION_MOVE (move gesture), ACTION_UP (leave screen), and ACTION_CANCEL (cancel gesture). Other triggering events are also possible. Since GLSurfaceView instances provide the ability to dispatch interaction events, the interaction event can be dispatched to the GLSurfaceView instance within the onTouch event function callback of the actual click source, WallpaperService.Engine.

[0117] like Figure 6The game desktop interaction flow shown is as follows: When the user clicks the screen, the system desktop (WallpaperService.Engine) receives the click event and notifies the game proxy container (GLSurfaceView instance). The GLSurfaceView instance obtains the coordinate information (X / Y coordinates relative to the screen) and event type (press, release, swipe, etc.) of the click event, then assembles and distributes this information to the target engine. Based on this information, the target engine renders the corresponding business logic and generates new screen data, which is then passed to WallpaperGLRender for rendering. WallpaperGLRender updates the desktop wallpaper based on the rendering results to achieve dynamic interactive effects. Figure 1 As shown, when a user clicks "Go Out and Explore," new streets and other scenes will be displayed.

[0118] In an optional embodiment, other games or other live streams can also be used as desktop wallpapers, and the method further includes:

[0119] By switching the rendering layer of the target engine, the screen of other games or live streams can be used as the desktop wallpaper, which is convenient and efficient.

[0120] While waiting for the resource package to load, the desktop may appear as a black screen due to the lack of visuals, resulting in a poor user experience. In a typical client-side container, since the container is a system interface activity (activity component), native views can be added to display the loading status. However, without an activity on the desktop, components cannot be added. In this embodiment, OpenGL instructions can be used to directly render and display the loading waiting state (e.g., ...). Figure 7 (As shown). Details are as follows.

[0121] In an optional embodiment, to avoid a black screen during the resource package loading process, such as Figure 8 As shown, the method further includes: step S800, providing a preliminary background image during resource package loading; step S802, generating a target background image based on the current wallpaper running state of the desktop and the preliminary background image; step S804, generating a background texture based on the target background image through the view proxy container instance; step S806, providing the background texture to the desktop wallpaper rendering thread through the view proxy container instance; and step S808, rendering the background texture through the desktop wallpaper rendering thread to display the rendering result as wallpaper on the desktop. Through steps S800 to S808, during the loading process of resource packages (code, etc.) in games or live streams, WallpaperGLRender renders the waiting process, improving the user experience.

[0122] Furthermore, when displaying a loading process, to improve user experience through seamless screen transitions, the method further includes:

[0123] Once the first frame of the target object (game or live stream) has been rendered, switch to having the target engine's rendering layer provide the data.

[0124] To further enhance the user experience, desktop wallpapers can also be set to include videos and games / live streams. Details are as follows.

[0125] In optional embodiments, such as Figure 9 As shown, the method may further include:

[0126] Step S900: Render the image data drawn by the target engine into the target texture, wherein the target texture is associated with the FBO.

[0127] Step S902: Render the video frame data from the target video onto a shared texture, which is a texture used by multiple threads. The video frames can be real-time footage from the live stream or pre-defined frames.

[0128] Step S904: Perform a texture blending operation on the target texture and the shared texture to obtain a target blended texture.

[0129] Step S906: Display the content of the target blended texture on the desktop.

[0130] Furthermore, in order to achieve the visual effect of a game or live streaming scene floating above the video and to provide interaction, step S906 may include: overlaying the target texture on the shared texture to obtain the target blended texture.

[0131] Combination Figure 10 The following provides a process for implementing both videos and games as desktop wallpapers to help understand the above optional embodiments.

[0132] Step S1000: Decode the target video through the video decoding sub-thread and generate a shared texture.

[0133] The video decoding sub-thread can periodically notify WallpaperGLRender to render based on the bitrate of the target video.

[0134] The shared texture is a texture that can be used by multiple threads.

[0135] Step S1002: Intercept game rendering and render the target engine's rendering data onto the target texture.

[0136] The target texture is attached to the off-screen FBO of the screen.

[0137] Step S1004: Perform texture blending on the shared texture and the target texture.

[0138] Texture blending can be done by first rendering the video frame data (shared texture), and then rendering the game frame data (target texture). Using this blending method, the game frame will be superimposed on top of the target video.

[0139] Step S1006: Draw the composite off-screen FBO onto the screen.

[0140] Example 2

[0141] Figure 11 The diagram schematically illustrates a block diagram of a virtual desktop display device according to Embodiment 2 of this application. This virtual desktop display device can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program modules referred to in the embodiments of this application are a series of computer program instruction segments capable of performing specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 11 As shown, the display device 1100 of the virtual desktop may include a creation module 1110, a binding module 1120, a loading module 1130, and a rendering module 1140, wherein:

[0142] Create module 1110 to create a view proxy container instance;

[0143] Binding module 1120 is used to bind the view proxy container instance to the desktop wallpaper rendering thread of the desktop wallpaper service engine;

[0144] Loading module 1130 is used to load the image to be drawn into the desktop wallpaper rendering thread through the view proxy container instance; and

[0145] The rendering module 1140 is used to render the image data through the desktop wallpaper rendering thread to display a virtual image on the desktop.

[0146] In an optional embodiment, the display device 1100 of the virtual desktop may include an interaction module (unidentified) for:

[0147] Listen for interactive events through the desktop wallpaper service engine;

[0148] The desktop wallpaper service engine provides the event information of the interactive events to the view proxy container instance.

[0149] The event information of the interactive event is provided to the target engine through the view proxy container instance, so that the target engine executes the corresponding preset logic and updates the screen data based on the drawing layer of the target engine;

[0150] The updated screen data is provided to the desktop wallpaper rendering thread through the view proxy container instance;

[0151] The updated screen data is rendered through the desktop wallpaper rendering thread to update the desktop wallpaper.

[0152] In an optional embodiment, the display device 1100 of the virtual desktop may include a switching module (unidentified) for:

[0153] By switching the rendering layer of the target engine, the screen of other games or live streams can be used as the wallpaper of the desktop.

[0154] In an optional embodiment, the display device 1100 of the virtual desktop may include a background transition module (unidentified) for:

[0155] Provide a preliminary background image during the resource package loading process;

[0156] Generate a target background image based on the current wallpaper status of the desktop and the prepared background image;

[0157] A background texture is generated based on the target background image using the view proxy container instance;

[0158] The background texture is provided to the desktop wallpaper rendering thread through the view proxy container instance; and

[0159] The background texture is rendered using the desktop wallpaper rendering thread, and the rendering result is displayed as wallpaper on the desktop.

[0160] In an optional embodiment, the display device 1100 of the virtual desktop may include a switching module (unidentified) for:

[0161] Once the first frame of the target object has been drawn, switch to having the target engine's drawing layer provide the data.

[0162] In an optional embodiment, the display device 1100 of the virtual desktop may include a display module (unidentified) for:

[0163] The image data drawn by the target engine is rendered into the target texture, which is associated with the FBO;

[0164] Render the video frame data in the target video onto a shared texture, which is a texture used by multiple threads;

[0165] A texture blending operation is performed on the target texture and the shared texture to obtain a target blended texture;

[0166] Display the content of the target blended texture on the desktop.

[0167] In an optional embodiment, the display module is further configured to:

[0168] The target texture is overlaid on the shared texture to obtain the target blended texture.

[0169] Example 3

[0170] Figure 12 This illustration schematically shows a hardware architecture diagram of a computer device 10000 suitable for implementing a virtual desktop display method according to Embodiment 3 of this application. The computer device 10000 can be used as a mobile terminal 2 or part of a mobile terminal 2. In this embodiment, the computer device 10000 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. For example, it can be a smartphone, tablet computer, laptop, personal computer, virtual device, set-top box, television, projector, vehicle terminal, etc. Figure 12 As shown, the computer device 10000 includes, but is not limited to, a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other via a system bus. Wherein:

[0171] The memory 10010 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of the computer device 10000, such as the hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 10000. Of course, the memory 10010 may also include both the internal storage module and the external storage device of the computer device 10000. In this embodiment, the memory 10010 is typically used to store the operating system and various application software installed on the computer device 10000, such as program code for virtual desktop display methods. In addition, the memory 10010 can also be used to temporarily store various types of data that have been output or will be output.

[0172] In some embodiments, processor 10020 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 10020 is typically used to control the overall operation of computer device 10000, such as performing control and processing related to data interaction or communication with computer device 10000. In this embodiment, processor 10020 is used to run program code stored in memory 10010 or process data.

[0173] Network interface 10030 may include a wireless network interface or a wired network interface, which is typically used to establish a communication link between computer device 10000 and other computer devices. For example, network interface 10030 is used to connect computer device 10000 to an external terminal via a network, establishing a data transmission channel and communication link between computer device 10000 and the external terminal. The network may be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.

[0174] It should be pointed out that, Figure 12 Only computer devices with components 10010-10030 are shown; however, it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0175] In this embodiment, the method for displaying the virtual desktop stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (processor 10020 in this embodiment) to complete the embodiment of this application.

[0176] Example 4

[0177] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the virtual desktop display method in the embodiments.

[0178] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device. Of course, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer-readable storage medium is typically used to store the operating system and various application software installed on the computer device, such as the program code of the virtual desktop display method in this embodiment. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.

[0179] Example 5

[0180] Figure 13 A flowchart illustrating a method for displaying a virtual desktop according to Embodiment 5 of this application is shown schematically.

[0181] like Figure 13 As shown, the method for displaying the virtual desktop may include steps S1300 to S1302, wherein:

[0182] Step S1300: Obtain screen data of the target application, wherein the screen data includes video screen data or game screen data;

[0183] Step S1302: In response to a preset operation, display the virtual screen corresponding to the screen data on the desktop.

[0184] While watching videos or playing games, users may find a particular frame, several frames, or even a real-time frame suitable as wallpaper. In this case, users can trigger specific controls to set a designated frame or real-time frame of the video or game as their desktop wallpaper, thus increasing wallpaper usability, improving user experience, and also helping to promote live streaming and games.

[0185] It should be noted that displaying the virtual screen corresponding to the screen data on the desktop can be achieved in the manner described in Example 1.

[0186] For example, step S1302 may include:

[0187] In response to a preset action, create a view proxy container instance;

[0188] Bind the view proxy container instance to the desktop wallpaper rendering thread of the desktop wallpaper service engine;

[0189] The view proxy container instance loads the screen data into the desktop wallpaper rendering thread;

[0190] The screen data is rendered through the desktop wallpaper rendering thread to display a virtual screen on the desktop.

[0191] Furthermore, interactive features such as link redirection and likes can be added to wallpaper submissions based on live video streaming, games, etc.

[0192] Specific implementation steps may include:

[0193] Listen for interactive events through the desktop wallpaper service engine;

[0194] The desktop wallpaper service engine provides the event information of the interactive events to the view proxy container instance.

[0195] The event information of the interactive event is provided to the target engine through the view proxy container instance, so that the target engine executes the corresponding preset logic and updates the screen data based on the drawing layer of the target engine;

[0196] The updated screen data is provided to the desktop wallpaper rendering thread through the view proxy container instance;

[0197] The updated screen data is rendered through the desktop wallpaper rendering thread to update the desktop wallpaper.

[0198] Furthermore, a switching function can be provided for wallpapers based on live video streaming, games, etc., such as switching the wallpaper from a live video stream to a game wallpaper. Specifically, this can be achieved by switching the rendering layer of the target engine to use footage from other games or live streams as the desktop wallpaper.

[0199] Furthermore, wallpapers combining videos and games can also be provided. Specific implementation methods include: rendering the image data drawn by the target engine onto a target texture, the target texture being associated with an FBO; rendering the video image data from the target video onto a shared texture, the shared texture being a texture used by multiple threads; performing a texture blending operation on the target texture and the shared texture to obtain a target blended texture; and displaying the content of the target blended texture on the desktop.

[0200] It should be noted that the technical content and effects of this embodiment can be referred to the content of Embodiment 1, and will not be repeated here.

[0201] The method of this embodiment can be stored in code form in the computer device of Embodiment 3 and the computer-readable storage medium of Embodiment 4.

[0202] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0203] It should be noted that the above are merely preferred embodiments of this application and do not limit the scope of patent protection of this application. Any equivalent structural or procedural changes made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of this application.

Claims

1. A method for displaying a virtual desktop, characterized in that, The method includes: Create a view proxy container instance; Bind the view proxy container instance to the desktop wallpaper rendering thread of the desktop wallpaper service engine; The view proxy container instance loads the screen data into the desktop wallpaper rendering thread; and The screen data is rendered through the desktop wallpaper rendering thread to display a virtual screen on the desktop.

2. The method for displaying a virtual desktop according to claim 1, characterized in that, The method further includes: Listen for interactive events through the desktop wallpaper service engine; The desktop wallpaper service engine provides the event information of the interactive events to the view proxy container instance. The event information of the interaction event is provided to the target engine through the view proxy container instance, so that the target engine executes the corresponding preset logic and updates the screen data based on the drawing layer of the target engine; The updated screen data is provided to the desktop wallpaper rendering thread through the view proxy container instance; The updated screen data is rendered through the desktop wallpaper rendering thread to update the desktop wallpaper.

3. The method for displaying a virtual desktop according to claim 2, characterized in that, The method further includes: By switching the rendering layer of the target engine, the screen of other games or live streams can be used as the wallpaper of the desktop.

4. The method for displaying a virtual desktop according to claim 1, characterized in that, The method further includes: Provide a preliminary background image during the resource package loading process; Generate a target background image based on the current wallpaper status of the desktop and the prepared background image; A background texture is generated based on the target background image using the view proxy container instance; The background texture is provided to the desktop wallpaper rendering thread through the view proxy container instance; and The background texture is rendered using the desktop wallpaper rendering thread, and the rendering result is displayed as wallpaper on the desktop.

5. The method for displaying a virtual desktop according to claim 4, characterized in that, The method further includes: Once the first frame of the target object has been drawn, switch to having the target engine's drawing layer provide the data.

6. The method for displaying a virtual desktop according to any one of claims 1 to 5, characterized in that, The method further includes: The image data drawn by the target engine is rendered into the target texture, which is associated with the FBO; Render the video frame data in the target video onto a shared texture, which is a texture used by multiple threads; A texture blending operation is performed on the target texture and the shared texture to obtain a target blended texture; Display the content of the target blended texture on the desktop.

7. The method for displaying a virtual desktop according to claim 6, characterized in that, The step of performing a texture blending operation on the target texture and the shared texture to obtain a target blended texture includes: The target texture is overlaid on the shared texture to obtain the target blended texture.

8. A display device for a virtual desktop, characterized in that, The device includes: Create a module for creating view proxy container instances; The binding module is used to bind the view proxy container instance to the desktop wallpaper rendering thread of the desktop wallpaper service engine; The loading module is used to load the image to be drawn into the desktop wallpaper rendering thread through the view proxy container instance; and The rendering module is used to render screen data through the desktop wallpaper rendering thread to display a virtual image on the desktop.

9. A method for displaying a virtual desktop, characterized in that, The method includes: Acquire screen data of the target application, including video screen data or game screen data; In response to a preset operation, a virtual screen corresponding to the screen data is displayed on the desktop; The process of displaying the virtual screen corresponding to the screen data on the desktop includes: The steps of the virtual desktop display method according to any one of claims 1 to 7.

10. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it performs the steps of the method for displaying a virtual desktop as described in any one of claims 1 to 7 or 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the virtual desktop display method according to any one of claims 1 to 7 or 9.

Images