Desktop rendering method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a desktop rendering method and device, a storage medium and electronic equipment, and the method comprises the steps: determining an interface element of a to-be-rendered initial desktop; wherein the interface elements comprise background elements and foreground elements; the background elements and the foreground elements are subjected to three-dimensional conversion to obtain three-dimensional scene representation of the background elements and the foreground elements in the three-dimensional scene, and then illumination calculation can be carried out based on the three-dimensional scene representation and the light source attribute parameters of the background elements to obtain an illumination map; according to the mode of applying the light source attribute parameters of the background elements to the three-dimensional scene representation, the problems of visual separation and incoherent illumination between the background elements and the foreground elements are solved, and the immersion of the desktop environment and the user experience are effectively improved.

Description

Desktop rendering methods and devices, storage media, electronic devices Technical Field

[0001] This application relates to the field of rendering technology, and more specifically, to a desktop rendering method and apparatus, storage medium and electronic device. Background Technology

[0002] In the digital age, the desktop is not only a bridge between the operating system and the user, but also a key component of personalized and immersive experiences.

[0003] Current desktop rendering technology generally separates foreground elements (such as desktop icons and desktop windows) and background elements (such as desktop wallpapers) into two independent visual layers, each applying predefined lighting effects. This separate processing method results in a lack of visual coherence in the desktop environment.

[0004] To address these issues, live wallpaper solutions have emerged. While live wallpapers can provide dynamic visual effects, these effects are limited to the wallpaper itself and cannot affect desktop icons and windows. Therefore, they still struggle to solve the problem of a flat, fragmented, and disjointed visual experience in the desktop environment. Summary of the Invention

[0005] This application provides a desktop rendering method, apparatus, storage medium, and electronic device to at least solve the problem in related technologies where the independent processing of lighting foreground and background elements results in a flat visual experience and a lack of visual coherence in the desktop environment.

[0006] According to one embodiment of this application, a desktop rendering method is provided, comprising: determining interface elements of an initial desktop to be rendered; wherein the interface elements include: background elements and foreground elements; performing 3D transformation on the background elements and the foreground elements respectively to obtain 3D scene representations of the background elements and the foreground elements in a 3D scene; performing lighting calculations based on the 3D scene representations and light source attribute parameters of the background elements to obtain a light map; wherein the light source attribute parameters are used to render the lighting in the 3D scene; and rendering the light map as an overlay layer of the initial desktop to obtain a target desktop.

[0007] According to another embodiment of this application, a desktop rendering apparatus is provided, comprising: a determining module, configured to determine interface elements of an initial desktop to be rendered; wherein the interface elements include: background elements and foreground elements; a conversion module, configured to perform three-dimensional conversion on the background elements and the foreground elements respectively to obtain a three-dimensional scene representation of the background elements and the foreground elements in a three-dimensional scene; a lighting calculation module, configured to perform lighting calculation based on the three-dimensional scene representation and the light source attribute parameters of the background elements to obtain a light map; wherein the light source attribute parameters are used to render the lighting in the three-dimensional scene; and a rendering module, configured to render the light map as an overlay layer of the initial desktop to obtain a target desktop.

[0008] According to yet another embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0009] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0010] According to yet another embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0011] According to this application, the interface elements of the initial desktop to be rendered are determined; wherein, the interface elements include: background elements and foreground elements; and the background elements and foreground elements are respectively subjected to 3D transformation to obtain the 3D scene representation of the background elements and foreground elements in the 3D scene. Then, lighting calculation can be performed based on the 3D scene representation and the light source attribute parameters of the background elements to obtain a light map. This method of applying the light source attribute parameters of the background elements to the 3D scene representation overcomes the problems of visual separation and inconsistent lighting between background elements and foreground elements, and effectively improves the immersiveness of the desktop environment and the user experience. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0014] Figure 1 is a hardware structure block diagram of a computer device for a desktop rendering method according to an embodiment of this application;

[0015] Figure 2 is a flowchart of one of the desktop rendering methods according to an embodiment of this application;

[0016] Figure 3 is a second flowchart of a desktop rendering method according to an embodiment of this application;

[0017] Figure 4 is a third flowchart of a desktop rendering method according to an embodiment of this application;

[0018] Figure 5 is a schematic diagram of a three-dimensional scene of a desktop rendering method according to an embodiment of this application;

[0019] Figure 6 is a fourth flowchart of a desktop rendering method according to an embodiment of this application;

[0020] Figure 7 is a fifth flowchart of a desktop rendering method according to an embodiment of this application;

[0021] Figure 8 is a comparative diagram of rendering effects according to an embodiment of this application;

[0022] Figure 9 is a structural block diagram of a desktop rendering apparatus according to an embodiment of this application. Detailed Implementation

[0023] The embodiments of this application will be described in detail below with reference to the accompanying drawings and examples.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0025] The methods and embodiments provided in this application can be executed in a computer device or similar computing device. Taking a computer device as an example, FIG1 is a hardware structure block diagram of a computer device for a desktop rendering method according to an embodiment of this application. As shown in FIG1, the computer device may include one or more (only one is shown in FIG1) processors 102 (processors 102 may include, but are not limited to, processing devices such as microprocessors or programmable logic devices) and a memory 104 for storing data. The computer device may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that the structure shown in FIG1 is only illustrative and does not limit the structure of the computer device. For example, the computer device may also include more or fewer components than shown in FIG1, or have a different configuration than shown in FIG1.

[0026] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the desktop rendering method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, which can be connected to computer devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0027] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer equipment. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0028] This embodiment provides a desktop rendering method applied to the aforementioned computer device. Figure 2 is a flowchart of the desktop rendering method according to an embodiment of this application. As shown in Figure 2, the process includes the following steps:

[0029] Step S202: Determine the interface elements of the initial desktop to be rendered; wherein, the interface elements include: background elements and foreground elements.

[0030] It should be noted that the initial desktop refers to a desktop environment without any additional visual effects applied. The initial desktop can include all interface elements, such as desktop wallpaper, desktop icons, desktop windows, and other visual components.

[0031] Background elements are elements located at the bottom layer of the desktop, providing a visual foundation for the desktop. Background elements can be desktop wallpaper, ground textures in game maps, or background patterns in application interfaces. When determining background elements, they can be obtained through APIs (Application Programming Interfaces) or image processing functions provided by the operating system of the computer device.

[0032] Foreground elements are UI elements that can interact with the user directly or indirectly. They can be placed on top of background elements and include icons, buttons, text boxes, menus, windows, etc. Foreground elements can also be determined using APIs (Application Programming Interfaces) or image processing functions provided by the operating system of the computer device.

[0033] Optionally, to ensure accurate acquisition and 3D processing of interface elements on all major operating systems (Windows, macOS, Linux, etc.), corresponding APIs can be set for different operating systems to take into account the API differences of different operating systems and ensure the accuracy and consistency of data collection.

[0034] Step S204: Perform 3D transformation on the background elements and foreground elements respectively to obtain the 3D scene representation of the background elements and foreground elements in the 3D scene;

[0035] It should be noted that 3D transformation refers to the process of mathematically modeling and data describing background and foreground elements in a 3D space. By converting background and foreground elements into a 3D scene representation, subsequent lighting calculations can be performed.

[0036] In an exemplary embodiment, a first textured polygon obtained by mapping a background element is acquired, and a first depth coordinate of the first textured polygon is determined to obtain a background representation; a first textured polygon obtained by mapping a first foreground element is acquired, and a second depth coordinate of the first textured polygon is determined to obtain a first foreground representation; and a three-dimensional polyhedron obtained by mapping a second foreground element is acquired, and a third depth coordinate of the three-dimensional polyhedron is determined to obtain a second foreground representation; wherein, the three-dimensional scene representation includes: a background representation, a first foreground representation, and a second foreground representation.

[0037] Understandably, the background element can first be converted into a first polygon, and then texture mapping can be performed on the first polygon, that is, mapping the texture data of the background element to the first polygon, thus obtaining the first texture polygon. Further, a depth coordinate is set for the first texture polygon, labeled as the first depth coordinate. The coordinate value of the first depth coordinate can be adaptively set according to the actual desktop environment, scene, etc. In this embodiment, the coordinate value of the first depth coordinate can be set to Z=0, indicating that the background element is located at the bottom layer of all elements, acting as the background plane of the scene.

[0038] For the first foreground element (such as a desktop icon), it is converted into a second polygon. Then, texture mapping is performed on the first polygon, that is, the texture data of the first foreground element is mapped to the second polygon to obtain the second texture polygon. A depth coordinate is further assigned to the second texture polygon, labeled as the second depth coordinate. The coordinate value of the second depth coordinate can be adaptively set according to the actual desktop environment, scene, etc. In this embodiment, the coordinate value of the second depth coordinate can be assigned a small Z-axis offset (e.g., Z = 0.1) to make it visually appear in front of the background element.

[0039] For the second foreground element (such as the desktop window), it is transformed into a 3D polyhedron. The geometry, size, and texture data of the 3D polyhedron are all based on the geometry of the desktop window. A depth coordinate is assigned to the 3D polyhedron, labeled as the third depth coordinate. The value of the third depth coordinate is greater than the value of the second depth coordinate, ensuring that the desktop window is positioned above the desktop icons. By assigning different depth coordinates to different elements, a layered 3D scene is created. This ensures that foreground elements correctly occlude background elements, while also establishing correct occlusion relationships between multiple foreground elements.

[0040] In the above embodiments, by precisely setting the depth coordinates of the background and foreground elements, a detailed and layered 3D scene representation is constructed, providing the necessary data foundation for subsequent lighting rendering.

[0041] Step S206: Perform lighting calculations based on the light source attribute parameters of the 3D scene representation and background elements to obtain a light map; wherein, the light source attribute parameters are used to render the lighting in the 3D scene;

[0042] The light source attribute parameters include information such as the light source's position, direction, color, intensity, radius, and light attenuation characteristics in three-dimensional space. These light source attribute parameters are crucial for illuminating each interface element in the scene; they determine the direction, color, and intensity of the light, thus affecting the realism and aesthetics of the final rendered image.

[0043] It's important to note that the goal of lighting calculations is to determine how each object in a 3D scene is affected by light sources, thereby generating a lightmap. The lightmap provides lighting and shadow information for each interface element on the initial desktop (desktop icons, desktop windows, desktop wallpaper, etc.).

[0044] Step S208: Render the lightmap as an overlay of the initial desktop to obtain the target desktop.

[0045] Understandably, a lightmap (an image containing lighting, shadows, and reflection effects) can be rendered as a semi-transparent overlay on the initial desktop, and then the initial desktop and the overlay can be blended together to obtain the target desktop.

[0046] Alternatively, in Windows systems, the Desktop Window Manager (DWM) API can be used to render a lightmap onto an overlay window as an overlay for the initial desktop. This means that each pixel in the lightmap corresponds to the same location on the desktop screen, but its background is completely transparent, allowing the desktop elements below to be seen through the lightmap. The overlay window is set to semi-transparent, typically neither completely transparent nor completely opaque. This setting allows the lightmap to blend with the desktop elements below, rather than simply obscuring them.

[0047] Through the above steps S202-S208, the interface elements of the initial desktop to be rendered are determined. The interface elements include background elements and foreground elements. The background elements and foreground elements are then transformed into three dimensions to obtain the three-dimensional scene representations of the background elements and foreground elements in the three-dimensional scene. Then, lighting calculations can be performed based on the three-dimensional scene representation and the light source attribute parameters of the background elements to obtain a light map. This method of applying the light source attribute parameters of the background elements to the three-dimensional scene representation overcomes the problems of visual separation and inconsistent lighting between background elements and foreground elements, effectively improving the immersiveness of the desktop environment and the user experience.

[0048] Optionally, in order to perform lighting calculations to obtain a lightmap, in addition to determining the 3D scene representations of the background and foreground elements, it is also necessary to determine the light source attribute parameters. In an exemplary embodiment, before performing lighting calculations based on the 3D scene representation and the light source attribute parameters of the background elements to obtain the lightmap, the desktop rendering method further includes: if the background element includes a desktop wallpaper, obtaining the wallpaper image file of the desktop wallpaper; extracting the metadata of the wallpaper image file from the wallpaper image file; determining whether light source metadata exists in the metadata; if light source metadata exists, extracting the first light source intensity, the first light source position, and the first light source color from the light source metadata, wherein the light source attribute parameters include: the first light source intensity, the first light source position, and the first light source color.

[0049] It should be noted that when the background element is a desktop wallpaper, the wallpaper image file can be obtained. This can be achieved, for example, through the operating system interface or by directly accessing the wallpaper image file in the file system.

[0050] Furthermore, metadata can be extracted from wallpaper image files. When reading metadata, standard metadata areas, such as EXIF ​​(Exchangeable Image File Format), can be read, or custom metadata fields can be searched.

[0051] The system searches for specific light source metadata within the extracted metadata. Light source metadata can be specifically defined, containing parameters such as light source location, color, and intensity. If light source metadata is detected, relevant parameters of the light source are extracted from it. These parameters may include the first light source intensity, the first light source location, and the first light source color; these define the basic characteristics of the light source and will be used in subsequent lighting calculations.

[0052] In the above embodiments, the information within the wallpaper image file can be fully utilized to determine the light source attribute information, thereby accurately simulating lighting conditions and providing users with a more natural and immersive desktop experience.

[0053] In an exemplary embodiment, after extracting the metadata of the wallpaper image file from the wallpaper image file, the method further includes: dividing the desktop wallpaper into multiple wallpaper areas if the metadata does not contain light source metadata; determining the light source wallpaper area among the multiple wallpaper areas; and determining the second light source intensity, second light source position, and second light source color of the light source wallpaper area, wherein the light source attribute parameters include: second light source intensity, second light source position, and second light source color.

[0054] Optionally, if the light source metadata is not found in the metadata of the wallpaper image file, the desktop wallpaper image can be segmented into multiple consecutive small regions, forming a series of wallpaper areas. This can be achieved through uniform grid partitioning, edge-based segmentation, or adaptive segmentation algorithms, aiming to decompose the image into analyzable parts.

[0055] Furthermore, the average brightness of each wallpaper area is calculated. Areas with higher brightness may contain light sources or strongly reflective objects, and are potential light source locations. By comparing the brightness of each area, areas with significantly higher brightness are identified and marked as light source wallpaper areas.

[0056] Once the wallpaper area for the second light source is determined, the intensity, location, and color of the second light source can be determined. Optionally, when determining the location of the second light source, the geometric center of the wallpaper area can be used as the position of the light source in the 3D scene. This is because in desktop wallpapers, light sources are often concentrated in a small point or area, and the center of that area represents the actual location of the light source.

[0057] When determining the color of the second light source, statistical analysis of the pixel colors in the light source wallpaper area can be performed, such as calculating the average RGB value, which can be used as the color of the second light source.

[0058] When determining the intensity of the second light source, it can be estimated by analyzing the brightness and size of the wallpaper area containing the light source, as well as the contrast of the surrounding area.

[0059] In the above embodiments, when there is no light source metadata in the metadata of the wallpaper image file, the light source is automatically identified through image content analysis, so that high-quality lighting effects can be generated even in the absence of light source metadata, ensuring a unified visual harmony and immersive experience for the desktop environment.

[0060] In an exemplary embodiment, lighting calculations are performed based on the light source attribute parameters of the three-dimensional scene representation and background elements to obtain a light map, including: rendering the three-dimensional scene representation to a preset geometric buffer to obtain multiple pixel representations of the three-dimensional scene representation in the screen coordinate system; filtering the multiple pixel representations to obtain a target pixel representation; and performing lighting calculations based on the target pixel representation and light source attribute parameters to obtain a light map.

[0061] Understandably, this involves transforming all interface elements, such as desktop wallpapers, desktop icons, and desktop windows, into a 3D scene representation. This means converting each element into a 3D geometry with position, size, and Z-order (stack order). For example, an icon can be represented as a textured plane, while a window can be modeled as a semi-transparent cube.

[0062] Optionally, the geometry buffer is a special buffer containing multiple pixels. The 3D scene representation in the 3D scene is rendered into the geometry buffer, realizing the projection from 3D space to the 2D screen coordinate system. Each pixel in the geometry buffer is assigned information such as its position, normal, and material properties in the 3D scene, thus obtaining a multi-pixel representation.

[0063] For each pixel representation in the geometry buffer, it is layered and filtered. Only pixel representations belonging to the current processing layer (such as the desktop icon layer) are selected as target pixel representations for lighting calculations. Pixel representations not input for the current processing layer are skipped at this stage, thus avoiding over-rendering and improving rendering efficiency.

[0064] Optionally, during layered processing, the target pixel representation is defined as those pixels that are visible in the current rendering layer and are not completely occluded. This ensures that lighting calculations are performed only on the interface elements of the current layer, resulting in more accurate lighting effects while also saving computational resources.

[0065] In some embodiments, when performing lighting calculations based on target pixel representations and light source attribute parameters, the target pixel representations can be traversed, and the lighting and shadows on each pixel can be calculated according to the depth value, surface normal, and material properties of each target pixel representation, as well as the light source attribute parameters (light source position, intensity, and color) of the background element. During lighting calculations, direct lighting calculations, ambient lighting calculations, shadow casting calculations, specular reflection calculations, and diffuse reflection calculations can be performed. The degree to which each target pixel representation is affected by lighting is calculated based on its relative position and angle to the light source, as well as material properties (such as specular reflectivity and roughness). The results of the lighting calculations can be stored in a separate texture, i.e., a lightmap. This texture contains the final color value of each pixel after taking into account lighting and shadow effects, and will be used as the basis for subsequent compositing with real-time desktop images.

[0066] In the above embodiments, by combining a geometric buffer, efficient lighting calculations and lightmap generation are achieved. This process ensures that the light sources in desktop elements and wallpapers create a natural and unified visual effect. At the same time, by avoiding redundant lighting calculations for occluded pixels, rendering efficiency and performance are greatly improved.

[0067] In one exemplary embodiment, filtering multiple pixel representations to obtain a target pixel representation includes: determining a layer identifier corresponding to each of the multiple pixel representations; the layer identifier is used to indicate the relative position of the multiple pixel representations in the screen coordinate system; based on the layer identifier, performing layer processing on the multiple pixel representations to obtain multiple layer pixel representations corresponding to different layers; for any one of the different layers, determining the target pixel representation of any layer based on the depth value of the multiple layer pixel representations under any one layer.

[0068] The hierarchy identifier is an additional attribute represented by each pixel, indicating the relative position or depth of its corresponding UI element on the desktop. Optionally, the hierarchy identifier can be determined by the stacking order of UI elements.

[0069] It's important to note that when the 3D scene representation is rendered to the geometry buffer, each pixel representation corresponds to a layer identifier. This layer identifier is stored in the geometry buffer along with other attributes of that pixel representation (such as position, normal, color, etc.). Depending on the layer identifier, the pixel representations in the geometry buffer are separated into different layers. This is equivalent to splitting the entire desktop environment into multiple independent layers according to the depth order of the interface elements, with each layer containing the layered pixel representations of the interface elements at that level.

[0070] Furthermore, the pixel representations of each layer can be processed sequentially according to the order of their hierarchical identifiers. Interface elements corresponding to lower-level identifiers (such as desktop wallpapers) are processed first, followed by middle-level elements, and finally the highest-level interface elements (i.e., those closest to the user). When determining the hierarchical pixel representation for each layer, the depth value stored in the geometric buffer for each layer's pixel representation can be used. When processing any given layer, the depth value at any given layer is compared to determine whether the pixel representation is occluded by pixels at the same or higher levels. For example, if, at layer A, a pixel representation has a depth value greater than a set depth threshold, it means it is further back in 3D space and has a greater chance of being occluded by preceding pixels.

[0071] Optionally, for level B, pixel representations with depth values ​​less than a depth threshold at level B can be identified as target pixel representations. Target pixel representations are not occluded by pixels at the same or higher levels, and therefore can directly participate in lighting calculations. Occluded pixels will not undergo lighting calculations at this stage, thereby reducing over-rendering and improving rendering efficiency.

[0072] In the above embodiments, the use of layer identifiers and depth values ​​enables effective filtering to determine the target pixel representation. This process is not only key to achieving efficient deferred shading technology, but also a technical guarantee for ensuring that the lighting effects of the desktop environment are both aesthetically pleasing and realistic. Through meticulous filtering and layering, unnecessary calculations can be avoided, rendering performance can be improved, and the consistency and high quality of lighting and shadow effects can be maintained, bringing users an immersive desktop experience.

[0073] In one exemplary embodiment, determining a target pixel representation at any given level based on depth values ​​of multiple layer pixel representations at any given level includes: determining a target depth value at any given level based on the depth values ​​of multiple layer pixel representations at any given level; the target depth value is a reference value determined based on the multiple depth values; determining the difference between the multiple depth values ​​at any given level and the target depth value, and determining layer pixel representations whose differences are greater than a preset difference threshold as filtered pixel representations; and determining the target pixel representation at any given level based on the filtered pixel representations.

[0074] The target depth value is a predetermined baseline value. This baseline value is used to filter the pixels at the current layer to determine which pixels are the target pixel representations, i.e., those pixels that should participate in lighting calculations.

[0075] Optionally, for any given level, the depth values ​​represented by pixels from all levels within that level can be read. The target depth value can be the minimum of all depth values ​​at that level, or the average of depth values ​​within a certain range. The goal is to find a baseline depth value that can represent the foremost part of the current level.

[0076] Optionally, each depth value is compared with the target depth value, and the difference between the two is calculated. A pre-set difference threshold is used to determine whether a pixel is close enough to the target depth value, thus determining whether it might be occluded. If the difference between the depth value of a layer pixel representation and the target depth value is greater than the pre-set difference threshold, this layer pixel representation is considered a filtered pixel representation. This means that it may be located behind the current layer and occluded by preceding pixels, and therefore should not be used as a target pixel for lighting calculations. Further, by excluding all filtered pixel representations, layer pixel representations whose depth values ​​are close to or equal to the target depth value (i.e., the difference is less than or equal to the pre-set difference threshold) are retained as target pixel representations.

[0077] In the above embodiments, the target pixel representation involved in lighting calculations can be determined efficiently and accurately using depth values. This process not only improves the efficiency of lighting rendering and reduces unnecessary calculations, but also ensures the realism of the lighting effects.

[0078] To better understand the process of the above desktop rendering method, the implementation flow of the above desktop rendering method will be described below in conjunction with optional embodiments, but this is not intended to limit the technical solution of the embodiments of this application.

[0079] Current desktop rendering technologies visually separate desktop icons, windows, and wallpapers, failing to create a unified, dynamic, and physically believable lighting environment. This results in a flat user experience and a lack of immersion. To address this, this application provides a desktop rendering method applicable to desktop rendering systems. Figure 3 shows a flowchart of the desktop rendering process corresponding to the system.

[0080] Figure 3 shows that the desktop rendering system involves multiple subsystems, and its overall architecture includes four core subsystems: the interface element tracking subsystem, the scene composition subsystem, the dynamic lighting and rendering engine, and the overlay compositing subsystem. The interface element tracking subsystem is configured to obtain element information of one or more interface elements from the operating system API, such as geometric data (position, size) and Z-order data (stack order). The Z-order defines the depth relationship between interface elements. Specifically, the interface element tracking subsystem collects geometric data of all desktop windows and icons from the host operating system, such as screen space coordinates (x, y) and dimensions (width, height), which is crucial for the subsequent accurate calculation of lightmaps.

[0081] Optionally, the operating system API can differ for different platforms to enable the collection of element information on different platforms. For example, the Windows API may include: EnumWindows, GetTopWindow, GetWindowRect, IFolderView, etc.; the macOS API may include: NSWindow.windowNumbers(options:), SWindow.frame, etc.; and the Linux API may include: XQueryTree, XGetGeometry, etc.

[0082] Scene composition subsystem: configured to generate a 3D scene representation of interface elements based on geometric data and Z-order data.

[0083] Dynamic lighting and rendering engine: Configured to render lighting and shadow effects on a 3D scene representation based on virtual light sources derived from the desktop wallpaper (i.e., light source attribute parameters determined by the desktop wallpaper).

[0084] Overlay Compositing Subsystem: Configured to blend the effect layers generated by the rendering engine with the initial desktop to ultimately present a uniform lighting effect.

[0085] The desktop rendering system can run in a continuous loop. Referring to Figure 4, which is a flowchart of a desktop rendering method in one embodiment, the specific steps are as follows:

[0086] In step S401, the interface element tracking subsystem polls the operating system's API to detect changes in interface elements (e.g., window movement, icon renaming, etc.). Once a change is detected, in step S402, the scene composition subsystem updates the 3D scene representation. Subsequently, in step S403, the dynamic lighting and rendering engine recalculates the lighting effects based on the updated scene and light sources derived from the desktop wallpaper, generating a light / shadow map. Finally, in step S404, the overlay compositing subsystem applies the newly generated light map to the screen, blending it with the initial desktop. This loop runs at a sufficiently high frame rate (e.g., 30-60 FPS) to create a real-time, interactive visual experience for the user.

[0087] Referring to Figure 5, the scene composition subsystem transforms 2D data into a 3D scene image. As shown in Figure 5, the scene composition subsystem can convert 2D data obtained from the tracking subsystem into a 3D scene image usable by the rendering engine. The desktop wallpaper 510 is treated as a large, textured quadrilateral located on the Z=0 plane of the scene. The desktop icon 520 is represented as a textured quadrilateral. Its position in 3D space corresponds to its 2D screen coordinates and is given a small Z-axis offset (e.g., Z=0.1) to visually position it in front of the wallpaper. The desktop application window 530 is modeled as a stretched, semi-transparent prism or cube. Its front corresponds to the window's content, while its sides provide depth, allowing it to cast more volumetric shadows. The Z-axis position of each prism is determined by the window Z-order (Z>0.1) provided by the tracking subsystem, ensuring the correct stacking relationship between windows. This process of transforming flat interface elements into 3D geometry with volume and depth is the foundation for achieving unified lighting effects.

[0088] Furthermore, since desktop icons are essentially simple two-dimensional textures, they lack the surface normals and material properties required for realistic lighting calculations. To address this issue, this application proposes the DNMI algorithm. Referring to Figure 6, this algorithm generates these data in real-time and programmatically. The algorithm consists of two parts: normal generation and material inference. For normal generation, for a given icon texture, it is first converted into a grayscale image, and then a Sobel or similar gradient filter is applied to the luminance channel. The generated gradient map is a normal map. This gives the planar icon a subtle yet believable three-dimensional surface structure, as if it were a physical object with minute undulations.

[0089] For material inference, the color and saturation histograms of the original icon texture can be analyzed. High-saturation, high-brightness areas (e.g., the highlights of a metallic badge) may be inferred to have higher specular reflectivity. Conversely, low-saturation, medium-brightness areas (e.g., the paper-like texture of a folder icon) may be inferred to have more diffuse or coarser characteristics. Furthermore, these inferred properties are combined into a material parameter map (such as a texture containing channels for specular reflectivity, roughness, and metallicity).

[0090] Moving further into the dynamic lighting and rendering engine, a typical desktop scene can contain dozens of overlapping windows and icons. Traditional rendering methods waste significant computational resources due to the repeated calculation of lighting for occluded pixels. To address this performance issue, this application proposes the HZDS (Hierarchical Z-Partition Deferred Shading) rendering pipeline. Referring to Figure 7, which illustrates the rendering process, the dynamic lighting and rendering engine first renders the scene's geometry (the quadrilaterals of icons, the prisms of windows) into a G-buffer (geometry buffer). The G-buffer stores the pixel representation, normals, and material properties of each geometry. Crucially, it also writes a unique hierarchy identifier for each geometry, which can be derived from Z-order information provided by the operating system.

[0091] Furthermore, the pixel representations in the G-buffer are processed hierarchically using a hierarchy identifier. First, the pixel representations belonging to the lowest layer (e.g., desktop wallpaper) are processed, and their lightmaps are calculated. Then, it moves to the next layer (e.g., desktop icon layer) and performs lighting calculations on the pixels belonging to that layer. During this process, lower-layer pixels occluded by pixels in the current layer are completely skipped without any lighting calculations.

[0092] Finally, the dynamic lighting and rendering engine outputs a lightmap containing only calculated lighting, shadows, and reflections, with a transparent background. The overlay compositing subsystem renders this light / shadow map as a top-level, semi-transparent, clickable overlay that covers the entire screen. On Windows, this can be achieved by creating a layered window using the desktop window manager's API. The desktop window manager is responsible for compositing the initial desktop and the lighting overlay into a single image sent to the monitor, ensuring a seamless, flicker-free blend with the rest of the desktop.

[0093] Referring to Figure 8, a comparison is shown between a standard desktop (Figure 8A) and a desktop rendered using this application (Figure 8B), which highlights the shadows cast by the light source in the desktop wallpaper on desktop icons and desktop windows and the resulting specular reflections.

[0094] The desktop rendering method provided in this application unifies the entire desktop environment into a single, dynamically lit 3D scene, creating a visually stunning and physically believable desktop experience, completely resolving the flatness and disjointedness of traditional desktops. It effectively solves the rendering performance problems in complex, layered desktop scenes, ensuring a smooth, real-time interactive experience. It enables realistic lighting rendering of any common 2D interface element without requiring any pre-made 3D models or data. Furthermore, the content-derived light source mechanism allows any ordinary image to serve as a source of dynamic lighting.

[0095] Based on the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0096] This embodiment also provides a desktop rendering apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0097] Figure 9 is a structural block diagram of a desktop rendering apparatus according to an embodiment of the present application. As shown in Figure 9, the apparatus includes:

[0098] The determination module 92 is used to determine the interface elements of the initial desktop to be rendered; wherein, the interface elements include: background elements and foreground elements;

[0099] The conversion module 94 is used to perform three-dimensional conversion on the background element and the foreground element respectively to obtain a three-dimensional scene representation of the background element and the foreground element in a three-dimensional scene;

[0100] The lighting calculation module 96 is used to perform lighting calculations based on the three-dimensional scene representation and the light source attribute parameters of the background elements to obtain a light map; wherein, the light source attribute parameters are used to render the lighting in the three-dimensional scene;

[0101] The rendering module 98 is used to render the light map as an overlay layer of the initial desktop to obtain the target desktop.

[0102] According to the above device, the interface elements of the initial desktop to be rendered are determined; wherein, the interface elements include: background elements and foreground elements; and the background elements and foreground elements are respectively transformed into three dimensions to obtain the three-dimensional scene representation of the background elements and foreground elements in the three-dimensional scene. Then, lighting calculation can be performed based on the three-dimensional scene representation and the light source attribute parameters of the background elements to obtain a light map. This method of applying the light source attribute parameters of the background elements to the three-dimensional scene representation overcomes the problems of visual separation and inconsistent lighting between background elements and foreground elements, effectively improving the immersiveness of the desktop environment and the user experience.

[0103] In an exemplary embodiment, the conversion module 94 is further configured to: acquire a first textured polygon obtained by mapping the background element, and determine a first depth coordinate of the first textured polygon to obtain a background representation; acquire a first textured polygon obtained by mapping a first foreground element among the foreground elements, and determine a second depth coordinate of the first textured polygon to obtain a first foreground representation; and acquire a three-dimensional polyhedron obtained by mapping a second foreground element among the foreground elements, and determine a third depth coordinate of the three-dimensional polyhedron to obtain a second foreground representation; wherein the three-dimensional scene representation includes: the background representation, the first foreground representation, and the second foreground representation.

[0104] In an exemplary embodiment, the apparatus further includes an extraction module, configured to: acquire a wallpaper image file of the desktop wallpaper when the background element includes a desktop wallpaper; extract metadata of the wallpaper image file from the wallpaper image file; determine whether light source metadata exists in the metadata; and, if the light source metadata exists, extract a first light source intensity, a first light source position, and a first light source color from the light source metadata, wherein the light source attribute parameters include: the first light source intensity, the first light source position, and the first light source color.

[0105] In an exemplary embodiment, the extraction module is further configured to, in the absence of the light source metadata in the metadata, divide the desktop wallpaper into multiple wallpaper areas; determine the light source wallpaper area among the multiple wallpaper areas; and determine the second light source intensity, second light source position, and second light source color of the light source wallpaper area, wherein the light source attribute parameters include: the second light source intensity, the second light source position, and the second light source color.

[0106] In an exemplary embodiment, the lighting calculation module 96 is further configured to render the three-dimensional scene representation to a preset geometric buffer to obtain multiple pixel representations of the three-dimensional scene representation in the screen coordinate system; filter the multiple pixel representations to obtain a target pixel representation; and perform lighting calculations based on the target pixel representation and the light source attribute parameters to obtain a lighting map.

[0107] In an exemplary embodiment, the illumination calculation module 96 is further configured to determine the layer identifiers corresponding to the plurality of pixel representations respectively; the layer identifiers are used to indicate the relative positions of the interface elements to which the plurality of pixel representations belong; based on the layer identifiers, the plurality of pixel representations are subjected to layer processing to obtain a plurality of layer pixel representations corresponding to different layers respectively; for any one of the different layers, based on the depth values ​​of the plurality of layer pixel representations under the arbitrary layer, the target pixel representation under the arbitrary layer is determined.

[0108] In an exemplary embodiment, the illumination calculation module 96 is further configured to determine a target depth value at the arbitrary level based on the depth values ​​of multiple layer pixel representations at the arbitrary level; the target depth value is a reference value determined based on the multiple depth values; determine the difference between the multiple depth values ​​at the arbitrary level and the target depth value, and determine the layer pixel representations whose difference is greater than a preset difference threshold as filtered pixel representations; and determine the target pixel representation at the arbitrary level based on the filtered pixel representations.

[0109] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0110] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run.

[0111] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0112] S1, determine the interface elements of the initial desktop to be rendered; wherein, the interface elements include: background elements and foreground elements;

[0113] S2, Perform 3D transformation on the background element and the foreground element respectively to obtain the 3D scene representation of the background element and the foreground element in the 3D scene;

[0114] S3, perform lighting calculations based on the three-dimensional scene representation and the light source attribute parameters of the background elements to obtain a light map; wherein, the light source attribute parameters are used to render the lighting in the three-dimensional scene;

[0115] S4, render the light map as an overlay layer of the initial desktop to obtain the target desktop.

[0116] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0117] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0118] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0119] Optionally, in this embodiment, the processor may be configured to perform the following steps according to a computer program:

[0120] S1, determine the interface elements of the initial desktop to be rendered; wherein, the interface elements include: background elements and foreground elements;

[0121] S2, Perform 3D transformation on the background element and the foreground element respectively to obtain the 3D scene representation of the background element and the foreground element in the 3D scene;

[0122] S3, perform lighting calculations based on the three-dimensional scene representation and the light source attribute parameters of the background elements to obtain a light map; wherein, the light source attribute parameters are used to render the lighting in the three-dimensional scene;

[0123] S4, render the light map as an overlay layer of the initial desktop to obtain the target desktop.

[0124] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0125] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0126] Embodiments of this application also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0127] Optionally, in this embodiment, the processor may be configured to perform the following steps according to a computer program:

[0128] S1, determine the interface elements of the initial desktop to be rendered; wherein, the interface elements include: background elements and foreground elements;

[0129] S2, Perform 3D transformation on the background element and the foreground element respectively to obtain the 3D scene representation of the background element and the foreground element in the 3D scene;

[0130] S3, perform lighting calculations based on the three-dimensional scene representation and the light source attribute parameters of the background elements to obtain a light map; wherein, the light source attribute parameters are used to render the lighting in the three-dimensional scene;

[0131] S4, render the light map as an overlay layer of the initial desktop to obtain the target desktop.

[0132] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0133] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0134] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A desktop rendering method, characterized in that, include: The interface elements of the initial desktop to be rendered are determined; wherein the interface elements include: background elements and foreground elements; the background elements and the foreground elements are respectively subjected to 3D transformation to obtain the 3D scene representation of the background elements and the foreground elements in the 3D scene; lighting calculation is performed based on the 3D scene representation and the light source attribute parameters of the background elements to obtain a light map; wherein the light source attribute parameters are used to render the lighting in the 3D scene; the light map is rendered as an overlay layer of the initial desktop to obtain the target desktop.

2. The method according to claim 1, characterized in that, Performing 3D transformations on the background element and the foreground element respectively to obtain 3D scene representations of the background element and the foreground element in a 3D scene includes: obtaining a first textured polygon obtained by mapping the background element, and determining a first depth coordinate of the first textured polygon to obtain a background representation; obtaining a first textured polygon obtained by mapping a first foreground element among the foreground elements, and determining a second depth coordinate of the first textured polygon to obtain a first foreground representation; and obtaining a 3D polyhedron obtained by mapping a second foreground element among the foreground elements, and determining a third depth coordinate of the 3D polyhedron to obtain a second foreground representation; wherein the 3D scene representation includes: the background representation, the first foreground representation, and the second foreground representation.

3. The method according to claim 1, characterized in that, Before performing lighting calculations based on the 3D scene representation and the light source attribute parameters of the background element to obtain a light map, the method further includes: if the background element includes a desktop wallpaper, obtaining the wallpaper image file of the desktop wallpaper; extracting the metadata of the wallpaper image file from the wallpaper image file; determining whether light source metadata exists in the metadata; if the light source metadata exists, extracting a first light source intensity, a first light source position, and a first light source color from the light source metadata, wherein the light source attribute parameters include: the first light source intensity, the first light source position, and the first light source color.

4. The method according to claim 3, characterized in that, After extracting the metadata of the wallpaper image file from the wallpaper image file, the method further includes: dividing the desktop wallpaper into multiple wallpaper areas if the light source metadata is not present in the metadata; determining the light source wallpaper area among the multiple wallpaper areas; and determining the second light source intensity, second light source position, and second light source color of the light source wallpaper area, wherein the light source attribute parameters include the second light source intensity, the second light source position, and the second light source color.

5. The method according to claim 1, characterized in that, Lighting calculations are performed based on the 3D scene representation and the light source attribute parameters of the background elements to obtain a light map. This includes: rendering the 3D scene representation to a preset geometric buffer to obtain multiple pixel representations of the 3D scene representation in the screen coordinate system; filtering the multiple pixel representations to obtain a target pixel representation; and performing lighting calculations based on the target pixel representation and the light source attribute parameters to obtain a light map.

6. The method according to claim 5, characterized in that, Filtering the plurality of pixel representations to obtain target pixel representations includes: determining the hierarchical identifiers corresponding to the plurality of pixel representations respectively; the hierarchical identifiers are used to indicate the relative positions of the interface elements to which the plurality of pixel representations belong; Based on the hierarchical identifier, the multiple pixel representations are processed in layers to obtain multiple hierarchical pixel representations corresponding to different layers; For any given level in different layers, the target pixel representation of that level is determined based on the depth values ​​of the pixel representations of multiple levels under that given level.

7. The method according to claim 6, characterized in that, Determining a target pixel representation at any given level based on the depth values ​​of multiple layer pixel representations at that level includes: determining a target depth value at that level based on the depth values ​​of multiple layer pixel representations at that level; the target depth value being a reference value determined based on the multiple depth values; determining the differences between the multiple depth values ​​at that level and the target depth value, and identifying layer pixel representations whose differences are greater than a preset difference threshold as filtered pixel representations; and determining the target pixel representation at that level based on the filtered pixel representations.

8. A desktop rendering device, characterized in that, include: A determination module is used to determine the interface elements of the initial desktop to be rendered; wherein the interface elements include: background elements and foreground elements; a transformation module is used to perform 3D transformations on the background elements and the foreground elements respectively to obtain 3D scene representations of the background elements and the foreground elements in a 3D scene; a lighting calculation module is used to perform lighting calculations based on the 3D scene representation and the light source attribute parameters of the background elements to obtain a lightmap; wherein the light source attribute parameters are used to render the lighting in the 3D scene; a rendering module is used to render the lightmap as an overlay layer of the initial desktop to obtain the target desktop.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to perform the method according to any one of claims 1 to 7.

Images