Graph drawing method and related apparatus

Abstract

The present application provides a graph drawing method and a related apparatus. The method can comprise: an electronic device can construct primitive SDFs and store the primitive SDFs as a global acceleration structure, and perform screen visibility determination and spatial visibility determination for pixel points in a screen by means of a gaze tracking method, so that finally visible primitives at the pixel points can be determined. Moreover, the dynamic effect of the primitives can be calculated by means of gaze tracking, and color values of the pixel points are finally determined. Finally, the electronic device can call a drawcall instruction on the basis of the calculation result to draw a first image frame. In this way, the drawing effect of the electronic device can be improved, and the power consumption of the electronic device can be reduced.

Description

A graphic drawing method and related apparatus

[0001] This application claims priority to Chinese Patent Application No. 202311839884.5, filed on December 27, 2023, entitled "A Graphic Drawing Method and Related Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminals and image rendering, and in particular to a graphics rendering method and related apparatus. Background Technology

[0003] A window is one of the most important graphical user interfaces (GUIs) in an operating system. A window can be understood as a visual graphical interface of the operating system; for example, the desktop of electronic devices such as computers and mobile phones is a window interface. Windows provide a bridge between the user and the operating system, allowing the user to interact with the operating system intuitively through a graphical interface. Windows are classified into planar windows (or traditional windows) and spatial windows based on their dimensions. Planar windows primarily feature two-dimensional, planar effects. Spatial windows, while maintaining two-dimensional planar effects, incorporate three-dimensional and realistic physical simulation effects (e.g., blur effects, lighting effects, dynamic wallpapers, etc.).

[0004] With the development of operating systems and human-computer interaction theory, spatial windows will be increasingly used and displayed in electronic devices. Currently, traditional graphics rendering pipelines used for drawing planar windows are insufficient for efficiently and with low power consumption to render 3D effects and realistic physical simulations within spatial windows.

[0005] Therefore, how to efficiently and with low power consumption draw spatial windows in electronic devices is an urgent problem to be solved. Summary of the Invention

[0006] This application provides a graphics drawing method and related apparatus. Through the graphics drawing method provided in the embodiments of this application, electronic devices can draw spatial windows efficiently and with low power consumption.

[0007] In a first aspect, this application provides a graphics drawing method, which may include: an electronic device acquiring drawing data of a first image frame, wherein the first image frame is an image frame to be drawn in a target application, and the drawing data is used to describe N controls contained in the first image frame and the attributes of the N controls; the electronic device constructing N primitive symbol distance fields (SDFs) corresponding to the N controls, wherein the N controls include a first control, and the N primitive SDFs include a first primitive SDF corresponding to the first control, wherein the first primitive SDF is used to describe the positional relationship between any point in space and the first control; the electronic device determining the occlusion relationship of the N controls through line-of-sight tracking based on the N primitive SDFs; the electronic device drawing the first image frame based on the occlusion relationship of the N controls; and the electronic device displaying the first image frame.

[0008] In this way, when drawing image frames, electronic devices do not need to use traditional triangle-based controls, reducing data volume and repetitive drawing. After determining the occlusion relationship, the electronic device can draw the first image frame with a single drawing instruction based on the occlusion relationship. When the first image frame contains animation effects, it can also be called a spatial viewport. Regardless of whether the first image frame is a planar viewport or a spatial viewport, the electronic device can draw the first image frame efficiently and with low power consumption.

[0009] In conjunction with the first aspect, in one possible implementation, the electronic device constructs a distance field SDF (Site Distance Field) of N primitive symbols based on N controls. Specifically, this may include: the electronic device constructing a distance equation from any point in space to the first control based on the geometric analytical expression corresponding to the geometry of the first control. The distance equation is the first primitive SDF.

[0010] Among them, the geometric expression corresponding to the geometry of the first control can be the Bézier curve expression.

[0011] In this way, the electronic device can construct the first primitive SDF through the geometric analytical expression corresponding to the first control.

[0012] In conjunction with the first aspect, in one possible implementation, the electronic device constructs N primitive symbol distance fields (SDFs) based on N controls. Specifically, this may include: the electronic device constructing a primitive bounding box surrounding the first control, and dividing the primitive bounding box into k rectangles, where the center points of the k rectangles represent k coordinate points with the center of the first control as the origin; the electronic device calculating the distances from the k coordinate points to the surface of the first control, obtaining k distance values; and the electronic device generating a texture image based on the k distance values, where the texture image is the first primitive SDF. In this way, the electronic device can construct the first primitive SDF corresponding to the first control, and this first primitive SDF is the texture image.

[0013] In conjunction with the first aspect, in one possible implementation, the electronic device constructs a distance field (SDF) of N primitive symbols based on N controls. Specifically, this may include: the electronic device constructing a first neural network, where the input of the first neural network is the coordinates of any point in space, and the output of the first neural network is the distance from any point in space to the surface of the first control. The first neural network is the first primitive SDF. In this way, the electronic device can construct the first primitive SDF corresponding to the first control, and this first primitive SDF is the first neural network.

[0014] In conjunction with the first aspect, in one possible implementation, the method may further include: the electronic device clusters the N primitive SDFs according to clustering rules to obtain a global acceleration structure, the global acceleration structure containing the attributes of the N primitive SDFs; the electronic device stores the global acceleration structure. In this way, the electronic device can more conveniently call or query the attributes of each primitive SDF.

[0015] In conjunction with the first aspect, in one possible implementation, the occlusion relationship includes screen visibility. The electronic device determines the occlusion relationship of N controls based on N primitive SDFs through gaze tracking. Specifically, this may include: the electronic device constructing a first ray starting from a first pixel on the display screen, with the direction of the first ray perpendicular to the display screen; the electronic device querying a global acceleration structure to determine that one or more controls exist in the direction of the first ray, and the N controls include one or more controls; the electronic device determining the nearest distance between the first pixel and one or more controls based on the primitive SDFs corresponding to the one or more controls; and the electronic device determining that the control visible to the first pixel is the second control among the one or more controls based on the nearest distance between the first pixel and the one or more controls. In this way, the electronic device can determine the control ultimately visible to each pixel on the display screen.

[0016] In conjunction with the first aspect, in one possible implementation, the occlusion relationship includes spatial visibility. The electronic device determines the occlusion relationship of N controls based on N primitive SDFs through gaze tracking. Specifically, this may include: the electronic device constructing a second ray with a first point on a third control as the starting point and a second point on a fourth control as the ending point; the N controls include both the third and fourth controls; the electronic device querying the global acceleration structure to determine the existence of a fifth control in the direction of the second ray; the electronic device determining the distance between the first point and the fifth control based on the primitive SDF corresponding to the fifth control; and the electronic device moving the second ray based on the distance between the first point and the fifth control until the starting point of the second ray is moved to the second point, at which point the electronic device determines that there is no occlusion between the third and fourth controls. In this way, the electronic device can determine the occlusion relationship between each control in the first image frame.

[0017] In conjunction with the first aspect, in one possible implementation, after the electronic device determines the occlusion relationship of the N controls based on the N primitive SDFs through gaze tracking, the method may further include: the electronic device determining from the drawing data that the sixth control among the N controls has a shadow effect, and the electronic device calculating the shadow effect. In this way, the electronic device can determine the shadow effect in the first image frame.

[0018] In conjunction with the first aspect, in one possible implementation, the electronic device calculates the shadow effect, which specifically includes: the electronic device determining the shadow radius and shadow path of the sixth control, and constructing a shadow SDF based on the shadow radius and shadow path; the electronic device calculating the nearest distance from the second pixel to the shadow of the sixth control based on the shadow SDF; the electronic device determining the shadow density of the second pixel based on the nearest distance from the second pixel to the shadow of the sixth control, and the shadow radius; and the electronic device determining the color value of the second pixel based on the shadow density. In this way, the electronic device can determine the impact of the shadow effect on the color value of each pixel in the display screen.

[0019] In conjunction with the first aspect, in one possible implementation, after the electronic device determines the occlusion relationship of N controls based on N primitive SDFs through gaze tracking, the method may further include: the electronic device determining from the drawing data that the seventh control among the N controls has refraction and reflection effects, and the electronic device calculating the refraction and reflection effects. In this way, the electronic device can determine the refraction and reflection effects in the first image frame.

[0020] In conjunction with the first aspect, in one possible implementation, the electronic device calculates the refraction and reflection effects, specifically including: the electronic device constructs a third ray starting from a third pixel point, where the third pixel point is a pixel on the display screen of the electronic device, and the direction of the third ray is perpendicular to the display screen; the electronic device determines, based on eye-tracking, the third point on the surface of the seventh control where the third ray reaches; the electronic device determines the reflection and refraction directions of the third ray at the third point; the electronic device queries the environmental texture based on the reflection direction and determines the color value of the environment as a first color value; the electronic device moves the third ray based on the refraction direction, so that the third ray reaches a fourth point in the background image; the electronic device retrieves the second color value of the fourth point in the background image; the electronic device determines the color value of the third pixel based on the first and second color values. In this way, the electronic device can determine the impact of the refraction and reflection effects on the color value of each pixel point on the display screen.

[0021] In conjunction with the first aspect, in one possible implementation, before the electronic device constructs the corresponding N primitive SDFs based on the N controls, the method may further include: the electronic device identifying the controls with animation effects among the N controls.

[0022] In conjunction with the first aspect, in one possible implementation, after the electronic device identifies the controls with animation effects among the N controls, the method may further include: the electronic device constructing corresponding primitive SDFs based on the controls with animation effects among the N controls; the electronic device determining the occlusion relationship between the controls with animation effects through line-of-sight tracking, and drawing the controls with animation effects based on the occlusion relationship; the electronic device drawing the N controls without animation effects based on a rasterization drawing method; the electronic device fusing the drawing results of the controls with animation effects and the drawing results of the controls without animation effects to obtain a first image frame, and the electronic device displaying the first image frame.

[0023] In this way, electronic devices can draw controls with and without animation effects in parallel, which can improve the efficiency of drawing the first image frame.

[0024] In a second aspect, an electronic device is provided, which may include a display screen, one or more processors, and one or more memories; wherein the display screen, one or more memories are coupled to one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by one or more processors, cause the electronic device to perform the methods involved in any possible implementation of the first aspect.

[0025] Thirdly, an electronic device is provided, which may include one or more functional modules for the methods involved in any possible implementation of the first aspect.

[0026] Fourthly, a chip system is provided for use in an electronic device, the chip system including one or more processors for invoking computer instructions to cause the electronic device to perform the methods involved in any possible implementation of the first aspect.

[0027] Fifthly, a computationally readable storage medium is provided, including instructions that, when executed on an electronic device, cause the electronic device to perform the methods involved in any possible implementation of the first aspect.

[0028] In a sixth aspect, a computer program product is provided that, when the program product is run on an electronic device, causes the electronic device to perform the methods involved in any possible implementation of the first aspect. Attached Figure Description

[0029] Figure 1A is a schematic diagram of a planar window provided in an embodiment of this application;

[0030] Figure 1B is a schematic diagram of a set of spatial windows provided in an embodiment of this application;

[0031] Figure 2 is a schematic diagram comparing the relationships between graphic elements in a planar view and a spatial view provided in an embodiment of this application;

[0032] Figure 3 is a schematic diagram of the drawing process of a window with a blur effect provided in an embodiment of this application;

[0033] Figure 4 is a schematic diagram of the drawing process of a window with a shadow effect provided in an embodiment of this application;

[0034] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0035] Figure 6 is a schematic diagram of the software framework of an electronic device provided in an embodiment of this application;

[0036] Figure 7 is a flowchart illustrating a graphic drawing method provided in an embodiment of this application;

[0037] Figure 8 is a schematic diagram of various representations of the primitive symbol distance field SDF provided in the embodiments of this application;

[0038] Figure 9 is a schematic diagram of the analytical expression of the circular primitive SDF provided in the embodiment of this application;

[0039] Figure 10 is a schematic diagram of the discrete representation of the circular primitive SDF provided in the embodiment of this application;

[0040] Figure 11 is a schematic diagram of the neural network representation of the circular primitive SDF provided in an embodiment of this application;

[0041] Figure 12 is a schematic diagram of a global acceleration structure for primitive SDF provided in an embodiment of this application;

[0042] Figure 13 is a schematic diagram of screen visibility determination provided in an embodiment of this application;

[0043] Figure 14 is a schematic diagram of spatial visibility determination provided in an embodiment of this application;

[0044] Figure 15 is a schematic diagram of shadow effect calculation provided in an embodiment of this application;

[0045] Figure 16 is a schematic diagram of refraction and reflection effect calculation provided in an embodiment of this application;

[0046] Figure 17 is a comparative diagram of different drawing processes provided in the embodiments of this application;

[0047] Figure 18 is a schematic diagram of the structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. The terms “first” and “second” are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include one or more of that feature. “First” and “second,” etc., are used to distinguish different objects, not to describe a particular order of objects. For example, a first object and a second object are used to distinguish different objects, not to describe a particular order of objects.

[0050] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

[0051] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or related scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0052] The term "and / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone.

[0053] In this embodiment, the electronic device 100 is capable of displaying both planar and spatial windows. The electronic device 100 may include, but is not limited to, devices with drawing and displaying spatial windows, such as mobile phones, tablets, and computers. This embodiment does not limit the specific form or type of the electronic device 100.

[0054] The term "user interface (UI)" used in the following embodiments of this application refers to the medium interface through which an application or operating system interacts and exchanges information with the user. It realizes the conversion between the internal form of information and the form that the user can accept. The user interface is source code written in a specific computer language such as Java or Extensible Markup Language (XML). The interface source code is parsed and rendered on the electronic device, ultimately presenting content that the user can recognize. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be visible interface elements such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets displayed on the screen of an electronic device.

[0055] In this embodiment, a window can also be referred to as a user interface. A window is rendered and presented by an electronic device through a graphics pipeline. A window may contain window animation effects. Window animation effects are a collective term for the window's visual effects and animation effects. Visual effects are primarily static effects, rendered by graphics. Animation effects are primarily dynamic effects, dynamically presented by graphics. Visual effects and animation effects are the core of window interaction. Visual effects can be simply referred to as visual effects, and animation effects can be simply referred to as animation effects.

[0056] A planar view primarily includes two-dimensional visual effects. For example, as shown in Figure 1A, the desktop 1000 displayed by the electronic device is a planar view. Desktop 1000 only contains two-dimensional planar effects. It is understood that this application is merely an example illustrating a desktop that is currently a planar view. In future developments, the desktop of an electronic device may evolve into a spatial view, and this application does not limit this.

[0057] The spatial view, based on a two-dimensional planar effect, combines three-dimensional and realistic physical simulation effects (e.g., blur effects, lighting effects, dynamic wallpapers, etc.). The spatial view can contain visual and motion effects. For example, as shown in Figure 1B(a), the electronic device can display a user interface 10A when charging. This user interface 10A can be a spatial view, and it can include controls 101 and 102. Control 101 can include a glowing motion effect, and control 102 can include a glowing bubble motion effect. As shown in Figure 1B(b), the electronic device can display a user interface 10B. This user interface 10B can be a spatial view, and it can include control 103. Control 103 can have a blur effect. As shown in Figure 1B(c), the electronic device can display a user interface 10C. This user interface 10C can be a spatial view, and it can include control 104. Control 104 can have lighting effects. Finally, as shown in Figure 1B(d), the electronic device can display a user interface 10D. The user interface 10D can be a spatial window, and the user interface 10D can include a control 105. The control 105 can be a three-dimensional sphere.

[0058] It is understood that the spatial windows shown in Figure 1B, such as user interfaces 10A, 10B, 10C, and 10D, are merely examples. This application does not limit the specific content contained in the spatial windows or their specific display format.

[0059] As shown in Figure 2(a), the relationships between primitives in a planar view are independent. For example, primitive 1 and primitive 2 are independent of each other, and only local information of primitive 1 is needed when drawing primitive 1 in the planar view. Similarly, only local information of primitive 2 is needed when drawing primitive 2 in the planar view. As shown in Figure 2(b), the relationships between primitives in a spatial view are interdependent. For example, primitive 1 and primitive 2 are interdependent, and global information is needed when drawing primitive 1 and primitive 2 in the spatial view.

[0060] In some embodiments, the rendering pipeline for drawing animations in a spatial window is based on a traditional rasterization rendering process. When rendering a spatial window, the electronic device first uses triangles to represent primitives within the window. For example, a rectangular primitive requires two triangles. A circular primitive may require over 100 triangles. Then, the electronic device uses a triangular rasterization method to determine visibility and renders the primitives in the spatial window from bottom to top, with upper-layer primitives covering lower-layer primitives. The final spatial window animation is formed by superimposing multiple drawcall instructions. This process requires additional read / write operations to video memory when the electronic device calls drawcall instructions multiple times. The high number of drawcall instructions in the electronic device system, while not computationally expensive per instruction, results in a waste of computational resources due to the inherent power consumption of each instruction. Furthermore, the bandwidth consumption between multiple drawcall instructions, which communicate via video memory, increases linearly with the number of drawcall instructions.

[0061] Figure 3 illustrates the drawing process of a window with a blurred effect. As shown in Figure 3, ①, the electronic device first retrieves the control to be drawn with a blurred effect from the framebuffer. Then, as shown in Figure 3, ②, the electronic device copies the control to the framebuffer used for drawing. This framebuffer can also be called an off-screen buffer. As shown in Figures 3, ④, ⑤, and ⑥, the electronic device downsamples the control multiple times, changing its resolution from 2048*3072 to 128*256. Then, as shown in Figure 3, ⑦, the electronic device blurs the control horizontally, and as shown in Figure 3, ⑧, the electronic device blurs the control vertically. Next, as shown in Figure 3, ⑨, the electronic device upsamples the blurred control to obtain a blurred control with a resolution of 2048*3072. Then, as shown in Figure 3, ⑩, the electronic device copies the blurred control from Figure 3, ⑨, to the framebuffer. Finally, as shown in Figure 3... As shown in the figure, the electronic device can overlay the drawn control 301 (which can be called the search control), control 302 (which can be called the common control), and control 303 (which can be called the notification control) on the basis of the control shown in Figure 3 (i.e., control 304, which can also be called the background control).

[0062] Figure 4 illustrates the rendering process of a window with a shadow effect. As shown in Figure 4, firstly, the electronic device retrieves the control (control 401) to be drawn with a shadow effect from the framebuffer. Then, as shown in Figure 4, the electronic device copies the shadow portion of control 401, i.e., primitive 402, to the off-screen buffer. As shown in Figures 4, 5, and 6, the electronic device downsamples primitive 402 multiple times, changing its resolution from 512*512 to 64*64. Then, as shown in Figure 4, 6, the electronic device blurs primitive 402 horizontally, and as shown in Figure 4, 7, the electronic device blurs primitive 402 vertically. Finally, as shown in Figure 4, 8, the electronic device upsamples the blurred primitive 402 to obtain primitive 404 with a shadow effect and a resolution of 512*512. Then, as shown in Figure 4(9), the electronic device copies the blurred primitives 404 and 403 from Figure 4(8) into the frame buffer. Finally, as shown in Figure 4(10), the electronic device can overlay other controls from the calculator interface onto the controls shown in Figure 4(9), resulting in the final calculator interface visible to the user.

[0063] The blur effect shown in Figure 3 and the shadow effect shown in Figure 4 both require the electronic device to invoke multiple drawcall commands during the rendering phase. For example, Table 1 shows relevant statistics for the blur effect shown in Figure 3 and the shadow effect shown in Figure 4 during the rendering phase.

[0064] Table 1

[0065] As shown in Table 1, the electronic device draws the diagram in Figure 3. The user interface with a blurred effect shown in Figure 4 requires 35 drawcall instructions to render, and the additional video memory access between these drawcalls consumes 20 Mbps of bandwidth, resulting in a frame rate of only 45 FPS. To render the user interface with a shadow effect shown in Figure 4(10), the electronic device requires 135 drawcall instructions, and the additional video memory access between these drawcalls consumes 1 Mbps of bandwidth, resulting in a frame rate of only 57 FPS.

[0066] Thus, as can be seen from Table 1 above, drawing spatial windows based on the traditional rasterization rendering process results in a relatively high number of draw calls and bandwidth consumption, and may cause the target scene (e.g., game scene) to fail to reach the expected target frame rate (e.g., 90 FPS).

[0067] The high power consumption of electronic devices drawing spatial windows using traditional rasterization rendering processes is due to the following reasons: 1. Simple primitives: Primitives are represented by triangles, and some primitives require many triangles, resulting in high cost per draw call. 2. Redundant drawing: When drawing primitives, upper-layer primitives may occlude lower-layer primitives, leading to wasted computation and reduced efficiency per draw call. 3. Local access: A single draw call can only access local information of primitives, not global information between primitives. 4. Separate rendering: Multiple animations in the spatial window are rendered separately, resulting in a lack of data between multiple animations and repeated draw calls.

[0068] To improve the performance of window rendering and reduce the power consumption of electronic devices during window rendering, embodiments of this application provide a graphics rendering method and an electronic device. In the graphics rendering method provided in this application, improvements to primitive representation and the rendering process effectively reduce the number of draw calls, memory accesses, and repetitive rendering when rendering spatial windows. Before introducing the graphics rendering method provided in this application, the hardware structure and software framework of the electronic device provided in this application will be described below.

[0069] Figure 5 is a schematic diagram of the structure of the electronic device 100 provided in the embodiment of this application.

[0070] As shown in Figure 5, the electronic device 100 may include a central processing unit (CPU) 501, a memory 502, a display 503, and a graphics processing unit (GPU) 504. Optionally, the electronic device 100 may also include a communication bus for connecting the various components.

[0071] It is understood that the various components in electronic device 100 can also be coupled to each other via other connectors, which may include various interfaces, transmission lines, or buses. The various components in electronic device 100 can also be connected radially with CPU 501 as the center. In the embodiments of this application, coupling refers to mutual electrical connection or communication, including direct connection or indirect connection via other devices.

[0072] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0073] In this application example, the CPU 501 and GPU 504 in the electronic device 100 can be located on the same chip or they can be separate chips. This application embodiment does not limit this.

[0074] The CPU 501 may include an application program 5011 and an operating system 5012. The CPU 501 can run the operating system 5012 and the application program 5011. The application program 5011 may be a graphics application, such as a game application or a video application. The operating system 5012 may include system graphics library interfaces and drivers, such as user-mode and / or kernel-mode drivers for the graphics library. The system graphics library interface includes, but is not limited to, OpenGL ES (Open Graphics Library for Embedded Systems), the Kronos Platform Graphics Interface, or Vulkan (a cross-platform graphics application programming interface). The user-mode and / or kernel-mode drivers for the graphics library can generate instruction streams for rendering graphics or image frames, as well as the necessary rendering data. The instruction stream may contain a series of instructions, which are typically calls to the system graphics library interface.

[0075] The operating system 5012 may also include a layer display composition module. This layer display composition module can be used to send the drawing results obtained by the GPU 504 based on drawing instructions to the display 503.

[0076] The memory 502 can store instruction streams and rendering data generated by the user-mode driver or kernel driver of the graphics library.

[0077] The CPU504 graphics processing unit may include a motion rendering pipeline and a cache. The motion rendering pipeline may include a primitive processing module, a basic motion rendering pipeline, a higher-order motion rendering pipeline, and a blending module. The motion rendering pipeline can be used to render rendering data based on drawcall instructions in the instruction stream. Specifically, the primitive processing module can represent primitives in the rendering data as signed distance fields (SDFs). The basic motion rendering pipeline can be used to render SDFs of primitives without animation. The higher-order motion rendering pipeline can be used to render SDFs of primitives with animation. The blending module can be used to blend the rendering results from the basic motion rendering pipeline and the rendering results from the higher-order motion rendering pipeline. The blending module can store the rendering results in a cache.

[0078] In the embodiments of this application, SDF can also be called a directed distance field. SDF is a function used to describe spatial location. When transmitting the coordinates of a point in space, SDF returns the shortest distance from that point to a surface (curve or surface) in space. The sign of the returned value indicates whether the point is inside or outside the surface.

[0079] Primitive SDF can be used to describe the positional relationship between each point in space and primitives. There are several ways to express this primitive SDF, which will be introduced in detail below, and will not be elaborated here.

[0080] The display 503 can be used to display various images generated by the electronic device 100. These images can be the GUI of the operating system 5012 or image data (including still images and video data) processed by the GPU 504. In this embodiment, the display 503 can be used to display received drawing results.

[0081] Figure 6 is a schematic diagram of the software framework of the electronic device 100 provided in an embodiment of this application.

[0082] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the system (e.g., the operating system 5012 shown in Figure 5) is divided into four layers, from top to bottom: the application layer, the application framework layer, the runtime and system libraries, and the kernel layer.

[0083] The application layer can include a series of application packages.

[0084] As shown in Figure 6, the application package can include game applications, video applications, and so on.

[0085] In some instances, the application package may also include applications (also known as apps) such as camera, gallery, calendar, calling, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0086] It is understandable that the application layer may also include the application 5011 shown in Figure 5.

[0087] The application framework layer (or simply the framework layer) provides application programming interfaces (APIs) and a programming framework for applications within the application layer. The application framework layer includes a set of predefined functions.

[0088] As shown in Figure 6, the application framework layer can include a rendering system (RS). The rendering system can include a unified rendering service and a discrete rendering service. The discrete rendering service can be used to generate discrete rendering instructions and rendering data. These discrete rendering instructions can be sent to the GPU. Based on these discrete rendering instructions and rendering data, the GPU can generate a basic geometric skeleton composed of triangles using a vertex shader. Then, the basic motion pipeline in the GPU can fill the image using a pixel shader and then apply textures. The unified rendering service can be used to generate unified rendering instructions and rendering data. Based on the unified rendering instructions and rendering data, the GPU's motion rendering pipeline can first generate one or more primitive SDFs from the rendering data, then draw one or more primitive SDFs using a single drawcall instruction, and obtain the rendering result.

[0089] A system library can include multiple functional modules. For example, a graphics device interface (GPU) and a graphics processing library (GPU). The rendering system can call the GPU through the GPU. The GPU can include the Open Graphics Library (OpenGL), a 2D graphics engine (or simply a 2D engine), a 3D graphics engine, a compositor, and a frame buffer. OpenGL is an application programming interface / function library, and it can also include OpenGL ES. The 2D engine is the drawing engine for rendering. The 3D engine is the drawing engine for 3D rendering. The compositor can provide the blending of 2D and 3D layers for multiple applications. The frame buffer, also called a frame buffer, is used to cache drawn image frames.

[0090] In some embodiments, the graphics processing library may include the system library graphics library interface shown in Figure 5.

[0091] The driver layer is the layer between hardware and software. It contains at least a graphics processor driver (GPU driver) and a display controller driver.

[0092] In some embodiments, the driver layer may further include a camera driver, an audio driver, a sensor driver, and a virtual card driver.

[0093] In some embodiments, the driver layer may also be referred to as the kernel layer.

[0094] In some embodiments, the driver layer may further include the graphics library kernel-mode driver shown in FIG5.

[0095] The following example, using a scene of capturing a photograph, illustrates the workflow of the software and hardware of the electronic device 100.

[0096] When the touch sensor receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, etc.). The raw input event is stored in the kernel layer. The application framework layer retrieves the raw input event from the kernel layer and identifies the control corresponding to the input event. Taking a single touch operation as an example, where the corresponding control is the camera application icon, the camera application calls the application framework layer's interface to launch the camera application, and then calls the kernel layer to launch the camera driver, capturing still images or videos through the camera.

[0097] The following describes in detail a graphics rendering method provided in this application embodiment, in conjunction with the hardware and software architecture of the exemplary electronic device 100 described above. The graphics rendering method provided in this application may include: First, the electronic device 100 can acquire rendering data of a first image frame, where the first image frame is the image frame to be rendered in the target application. The electronic device 100 can construct N primitive SDFs based on the rendering data. Then, the electronic device 100 can use gaze tracking to determine the screen visibility of the N primitive SDFs. Next, the electronic device 100 can render the N primitive SDFs based on the screen visibility of the N primitive SDFs, obtaining the rendering result of the first image frame. Finally, the electronic device 100 can send the rendering result to the display and display the first image frame.

[0098] Optionally, the electronic device 100 can also construct a global acceleration structure from the N primitive SDFs and store the global acceleration structure.

[0099] In this way, electronic device 100 represents primitives using SDF, avoiding the excessive data volume caused by using triangles to represent primitives in traditional drawing pipelines. By constructing N primitives into a global acceleration structure, the electronic device can easily access global information during drawing. The electronic device uses eye tracking to determine the screen visibility of primitives, and only needs to call a drawcall to obtain the drawing result. Furthermore, electronic device 100 does not require additional video memory access during drawing, which reduces bandwidth consumption.

[0100] Figure 7 is a flowchart illustrating a graphics drawing method provided in an embodiment of this application. As shown in Figure 7, a graphics drawing method provided in an embodiment of this application may include the following steps:

[0101] S701, Electronic device 100 acquires the drawing data of the first image frame, the first image frame being the image frame to be drawn in the target application.

[0102] The target application can be a game application or a video application installed on an electronic device.

[0103] When the target application is launched or the user interface is refreshed, it can send drawing data of an image frame to be drawn to the electronic device 100. This image frame to be drawn can be a first image frame. The drawing data is used to describe the controls contained in the first image frame and the attributes of the controls. For example, the attributes of the controls may include, but are not limited to: the name of the control, the shape of the control, the size of the control, the position of the control, the color of the control, etc. In some instances, the position of the control can be represented as the coordinates of the control in the first image frame. Optionally, in other embodiments, the position of the control can be represented as the distance from the control to the top of the first image frame and the distance from the control to the left border of the first image frame.

[0104] Alternatively, in one possible implementation, the drawing data is in an Extensible Markup Language (XML) file, which the target application can distribute.

[0105] S702, Electronic device 100 constructs a distance field SDF of N primitive symbols based on the drawing data.

[0106] The multiple controls described by the drawing data can be represented by N primitives. One or more controls can correspond to one primitive. Then, the electronic device 100 can construct N primitive symbols into N primitive symbol distance fields (SDFs).

[0107] It is understood that each control may include one or more interface elements, such as text, images, icons, etc. In some instances, each interface element contained in a control may also be called a control or a sub-control, and this application embodiment does not limit this.

[0108] N is a positive number. For example, N can be 1, N can be 2, N can be 3, etc. The specific value of N is not limited in the embodiments of this application.

[0109] In the embodiments of this application, primitive SDF can be expressed in various ways, such as analytical expression, discretized expression, neural network expression, etc.

[0110] For example, as shown in Figure 8, the analytical expression of the original primitive of the control can include the original geometric expression of the graphic, such as the Bézier curve expression. Then, the electronic device 100 can construct an equation for the distance from any point on the display screen to the original primitive based on the geometric expression of the original primitive. This equation is the analytical expression of the SDF of the primitive corresponding to the control.

[0111] For example, when the control is the circular control 801 shown in Figure 8, the electronic device 100 can express the circular control 801 using a geometric analytical expression of the circle. Then, the electronic device 100 can construct Equation 1, which is the distance from any point on the display screen to the surface of the circular control 801, based on the geometric expression of the circle. This Equation 1 is the analytical expression of the primitive SDF corresponding to the circular control 801.

[0112] For example, when the control is a rectangular control 802 as shown in Figure 8, the electronic device 100 can express the rectangular control 802 using a geometric analytical expression of the rectangle. Then, the electronic device 100 can construct Equation 2, which is the distance from any point on the display screen to the surface of the rectangular control 802, based on the geometric expression of the rectangle. This Equation 2 is the analytical expression of the primitive SDF corresponding to the rectangular control 802.

[0113] When the control is the triangular control 803 shown in Figure 8, the electronic device 100 can express the triangular control 803 using the geometric analytical expression of a triangle. Then, the electronic device 100 can construct Equation 3, which is the distance from any point on the display screen to the surface of the triangular control 803, based on the geometric expression of the triangle. This Equation 3 is the analytical expression of the primitive SDF corresponding to the triangular control 803.

[0114] When the control is the elliptical control 804 shown in Figure 8, the electronic device 100 can express the elliptical control 804 using the geometric analytical expression of an ellipse. Then, the electronic device 100 can construct Equation 4, which is the distance from any point on the display screen to the surface of the elliptical control 804, based on the geometric expression of the ellipse. Equation 4 is the analytical expression of the primitive SDF corresponding to the circular control 801.

[0115] Optionally, the electronic device 100 can also use Bézier curve analytical expressions to represent the circular control 801, rectangular control 802, triangular control 803, and elliptical control 804 shown in Figure 8. It is understood that the Bézier curve analytical expressions corresponding to the circular control 801, rectangular control 802, triangular control 803, and elliptical control 804 are different. When the electronic device 100 represents the circular control 801, rectangular control 802, triangular control 803, and elliptical control 804 shown in Figure 8 using Bézier curve analytical expressions, these controls can achieve the effect of scalable vector graphics.

[0116] Optionally, as shown in Figure 8, the electronic device 100 can also represent the primitive SDFs corresponding to the circular control 801, rectangular control 802, triangular control 803, and elliptical control 804 as discretized texture images. For example, the primitive SDF corresponding to the circular control 801 can be represented as texture image 805. The primitive SDF corresponding to the rectangular control 802 can be represented as texture image 806. The primitive SDF corresponding to the triangular control 803 can be represented as texture image 807. The primitive SDF corresponding to the elliptical control 804 can be represented as texture image 808.

[0117] Texture images 805, 806, 807, and 808 contain the original primitive and texture lines outside and inside the original primitive. Multiple points on each texture line are equidistant from the surface of the original primitive.

[0118] Optionally, the electronic device 100 can also represent primitive SDFs via a neural network. For example, as shown in FIG8, the SDF primitives corresponding to the texture image 809 can also be represented as a neural network 810. It is understood that the neural network 810 shown in FIG8 is merely an example, and the specific structure of the neural network 810 is not limited in this embodiment.

[0119] The following section will use a circular control as an example to detail the various ways to represent the SDF primitives corresponding to the circular control. For the geometric expressions of rectangles, triangles, and ellipses, please refer to the descriptions in relevant textbooks; they will not be elaborated upon here.

[0120] As shown in Figure 9, the distance from any point P0(x0, y0) in space to the circle is d0. The radius of the circle is r, and the origin of the coordinate system is O. The relationship between d0 and x0, y0, and r can be expressed as shown in Equation 1:

[0121] The electronic device 100 can express the primitive SDF of the circular control as the equation shown in Formula 1 above. In one possible implementation, the solution process of Formula 1 can be completed at runtime, and the specific calculation can be performed by the shader in the GPU of the electronic device 100.

[0122] The primitive SDF of the circular primitive shown in Figure 9 can also be expressed through discretization. As shown in Figure 10, the original circular primitive is covered by a primitive bounding box. The primitive bounding box is uniformly divided into 8*8 small rectangles, the center points of which represent 8*8 fixed coordinates with the center of the circle as the origin in the circular coordinate system. The electronic device 100 can use the primitive SDF analytical expression shown in Formula 1 to calculate the nearest distance from the 8*8 fixed coordinates to the circular surface. The electronic device 100 can save the calculated distance as the pixel value corresponding to the texture image. This texture image can be seen in the texture image 805 shown in Figure 8. For points outside the original primitive, the nearest distance to the surface of the original primitive is a positive value. For points inside the original primitive, the nearest distance to the surface of the original primitive is a negative value. For points on the surface of the original primitive, the nearest distance to the surface of the original primitive is 0. For example, for the discrete coordinate point P0 shown in Figure 10, which is outside the original primitive, the nearest distance from the discrete coordinate point P0 to the original primitive is 1. In this embodiment of the application, the unit of the closest distance can be a pixel.

[0123] The electronic device 100 can generate the texture image 805 shown in Figure 8 based on the pixel values ​​corresponding to the texture image, that is, the nearest distances from the 8*8 fixed coordinates shown in Figure 10 to the circular surface. Optionally, the texture image stored by the electronic device 100 may include 8*8 pixels, each of which stores the distance from the center point of one of the 8*8 rectangles in Figure 10 to the original primitive surface.

[0124] For example, the primitive SDF corresponding to the circular primitive can also be expressed as a neural network as shown in Figure 11. As shown in Figure 11, in this neural network expression corresponding to the primitive SDF, the input of the neural network can be x, y, 1. Here, x and y can be the horizontal and vertical coordinates of any point on the display screen, respectively. The network weights of the neural network are w, the activation function is f, and the skip connections are g. When x, y, 1 are input into the neural network, the neural network can output the nearest distance L from the coordinate point (x, y) to the surface of the original primitive.

[0125] It is understood that the specific values ​​of network weights w, activation function f, and skip connections g in the neural network shown in Figure 11 are not limited in the embodiments of this application. The specific structure of the neural network shown in Figure 11 is also not limited in the embodiments of this application.

[0126] S703 and electronic device 100 construct a global acceleration structure from N primitive SDFs.

[0127] The original primitives in an N primitive SDF can include various primitive shape types, such as circles, rectangles, triangles, ellipses, etc. Each primitive shape type can have multiple instances, for example, 10 circles, 10 rectangles, etc. That is, an N primitive SDF can contain multiple circular primitive SDFs and multiple rectangular primitive SDFs; this embodiment does not limit this. However, if the N primitives are processed separately, there may be a large amount of duplicate data. This is detrimental to the GPU's ability to transfer and read rendering data when drawing the first image frame.

[0128] The electronic device 100 can construct a storage structure for all scenarios, used to store N primitive SDFs. This storage structure can be a global acceleration structure, and the name of this storage structure is not limited in the embodiments of this application.

[0129] Figure 12 exemplifies a global acceleration structure for primitive SDFs. As shown in Figure 12, this global acceleration structure can be divided into two parts: a global relation structure and a global attribute structure. The global relation structure clusters primitive instances from N primitive SDFs, organizing the N primitive SDFs together according to specific clustering rules. Clusters can form tree-like structures (or hierarchical structures) based on different granularities, meaning a cluster can contain several sub-clusters. Clustering rules can include, but are not limited to, scene space partitioning based on grids or K-dimensional trees (kd-trees), screen space partitioning based on tiles, etc.

[0130] The global attribute structure is the actual storage format of primitive SDF data. All primitive SDFs of the same type share the same attribute structure. For example, as shown in Figure 12, taking a circular primitive SDF (also known as a circular instance) as an example, all circles have two attributes: center and radius. The difference lies in the specific values ​​of the center and radius in different circles. Primitive SDFs of the same type are stored in contiguous storage space to facilitate efficient retrieval when the GPU renders primitive SDFs. Since there are three ways to express primitive SDFs, there are also three ways to express the global attribute structure. For primitive SDFs expressed analytically, the attributes of the primitive SDF can be composed of a set of parameters. For example, the attributes of a circular primitive SDF may include the center and radius. For primitive SDFs expressed discretely, the attributes of the primitive SDF can be composed of distance field textures, such as the texture image 805 shown in Figure 5. For primitive SDFs expressed by a neural network, the attributes of the primitive SDF can be composed of network weights (e.g., w1, ..., wn). Optionally, the attributes of primitive SDFs in analytical, discrete, and neural network representations can all include a transformation matrix. This transformation matrix can be used to describe the spatial transformation attributes of the primitive SDF, such as translation, rotation, and scaling.

[0131] Understandably, the electronic device 100 can store N primitive SDFs according to the attribute structure corresponding to any one of the three expression methods: analytical primitive SDF, discrete primitive SDF, and neural network primitive SDF.

[0132] S704, Electronic device 100 uses a line-tracking algorithm to draw N primitive SDFs in the global acceleration structure.

[0133] Electronic device 100 can calculate and determine two visibility criteria for the SDF of N primitives using a gaze-tracking algorithm. These two visibility criteria can include screen visibility criteria and spatial visibility criteria. Screen visibility criteria can be used to determine the primitives visible to each pixel on the screen. Spatial visibility criteria can be used to determine whether there is object occlusion between any two points in space.

[0134] Figure 13 exemplarily illustrates a schematic diagram of screen visibility determination. As shown in Figure 13, pixel P8 is any pixel on the screen. Electronic devices typically draw graphical interfaces based on orthographic projection. Electronic device 100 constructs a ray T1 starting from pixel P8, with the direction of ray T1 perpendicular to the screen. Then, electronic device 100 can traverse the global attribute structure of the SDF of N primitives in the global acceleration structure and calculate the nearest distances from pixel P8 to primitive A and primitive B, respectively (P8, P5) and (P8, P4). Among all distances, the distance (P8, P5) is the shortest, indicating that there is no occlusion within this distance. Therefore, electronic device 100 can move ray T1 forward by (P8, P5) to point P1. The distance from point P1 to the nearest primitive, namely primitive A, is (P1, P6). There is no occlusion within this distance (P1, P6). Electronic device 100 can move ray T1 forward by the distance (P1, P6) to point P2. The distance from point P2 to the nearest primitive, i.e., primitive A, is (P2, P7). There is no occlusion within this distance (P2, P7). Electronic device 100 can move ray T1 forward a distance (P2, P7) to point P3. The distance from point P3 to the nearest primitive, i.e., primitive B, is (P3, P4). There is no occlusion within this distance (P3, P4). Electronic device 100 can move ray T1 forward a distance (P3, P4) to point P4. The distance from point P4 to primitive B is 0. Electronic device 100 can determine that ray T1 hits primitive B. Therefore, electronic device 100 can determine that the primitive visible to pixel P8 is primitive B.

[0135] It is understandable that the SDF of N primitives can include the SDF of primitive A and the SDF of primitive B.

[0136] In this way, the electronic device 100 can determine the visible primitives of each pixel on its screen according to the screen visibility determination steps shown in Figure 13 above. For example, if the visible primitive of pixel P8 is primitive B, then the electronic device 100 can draw primitive B at pixel P8.

[0137] In one possible implementation, after the electronic device 100 determines that the visible primitive at pixel P8 on its screen is primitive B according to the screen visibility determination steps shown in Figure 13, the electronic device 100 also needs to determine whether there is occlusion between primitive A and primitive B. If there is no occlusion, the electronic device 100 can draw primitive B at pixel P8. If there is occlusion, the electronic device 100 can determine that the visible object at pixel P8 is the occluding object.

[0138] Understandably, when electronic device 100 performs screen visibility and spatial visibility determination, it can access the SDFs of N primitives according to a global acceleration structure. For example, taking the global acceleration structure of N primitive SDFs obtained by electronic device 100 based on grid tile screen spatial partitioning and clustering as an example, when electronic device 100 performs screen visibility or spatial visibility determination, if the primitive corresponding to the root cluster does not occlude primitive 1, then the primitives corresponding to the sub-clusters (also called child nodes) under the root cluster will not occlude primitive 1 either. Therefore, electronic device 100 does not need to access and calculate whether the primitives corresponding to the sub-clusters under the root cluster will occlude primitive 1. This global acceleration structure reduces the computational load of the electronic device.

[0139] Figure 14 exemplarily illustrates a schematic diagram of spatial visibility determination. As shown in Figure 14, point P10 is a location point on the surface of primitive A. Point P14 is a location point on the surface of primitive B. Electronic device 100 constructs a ray T2 with point P10 as the starting point and point P14 as the ending point. Then, electronic device 100 can traverse the global attribute structure of the SDF of N primitives in the global acceleration structure and calculate the nearest distances from point P10 to primitive C and primitive B as (P10, P15) and (P10, P14), respectively. Among all distances, the distance (P10, P15) is the shortest, indicating that there is no occlusion within this distance. Therefore, electronic device 100 can move ray T2 forward by (P10, P15) to point P11. The distance from point P11 to the nearest primitive, namely primitive C, is (P11, P16). There is no occlusion within this distance (P11, P16). Electronic device 100 can move ray T2 forward a distance (P11, P16) to point P12. The distance from point P12 to the nearest primitive, primitive C, is (P12, P17). There is no occlusion within this distance (P12, P17). Electronic device 100 can move ray T2 forward a distance (P12, P17) to point P13. The distance from point P13 to the nearest primitive, primitive B, is (P13, P14). There is no occlusion within this distance (P13, P14). Electronic device 100 can move ray T2 forward a distance (P13, P14) to point P14. The distance from point P14 to primitive B is 0. Electronic device 100 can determine that ray T2 hits primitive B. Since ray T2 did not find any other primitives between points P10 and P14, electronic device 100 can determine that the area between points P10 and P14 is visible.

[0140] Then, based on the screen visibility determination results and spatial visibility determination results, the electronic device 100 can draw N primitive SDFs, that is, draw the original primitives corresponding to the N primitive SDFs. Since the electronic device 100 has determined the spatial relationship between the N primitives based on the screen visibility determination and spatial visibility determination, the electronic device 100 can draw all the primitives by calling a drawcall instruction once.

[0141] S705, electronic device 100, generates animation effects for drawing N primitive SDFs.

[0142] Electronic device 100 can read whether N primitives have animation effects from the drawing data. If the i-th primitive in the SDF of the N primitives has animation effects, then electronic device 100 can perform animation effect calculation on the i-th primitive. Then, electronic device 100 can draw all primitives and their animation effects by calling a drawcall instruction once.

[0143] Optionally, in one possible implementation, the N primitives may include primitive A, which has a shadow effect, and the electronic device 100 needs to calculate the shadow effect of primitive A. That is, the electronic device 100 needs to determine whether the color of the pixels in the space will be affected by the shadow effect of primitive A.

[0144] Figure 15 exemplarily illustrates a schematic diagram of the shadow effect of primitive A calculated by electronic device 100. As shown in Figure 15, primitive A has a shadow animation effect, and there is a pixel P(x, y) in space. Electronic device 100 needs to determine whether the shadow effect of primitive A will be reflected on pixel P. First, electronic device 100 needs to determine the positional relationship between pixel P and primitive A. There are two positional relationships between pixel P and primitive A: first, pixel P is located inside primitive A; second, pixel P is located outside primitive A. After determining the screen visibility determination result of pixel P through a gaze tracking algorithm, electronic device 100 can determine that pixel P is located outside primitive A. Then, electronic device 100 needs to further determine whether pixel P has a shadow. Electronic device 100 can perform shadow calculation to determine the shadow path and shadow radius Q. As shown in Figure 15, the shadow path can be the solid line with a single arrow shown in Figure 15(a). The shadow radius Q can be the dashed line with a double arrow shown in Figure 15(a). To determine the shadow effect of pixel P, electronic device 100 can use the shadow path as a primitive and construct a primitive SDF for the shadow path. Then, electronic device 100 can calculate the nearest distance d1 from pixel P to the shadow path. Based on the relationship between this nearest distance d1 and the shadow radius Q, electronic device 100 can determine whether the pixel has a shadow effect. Specifically, if the nearest distance d1 is greater than the shadow radius Q, electronic device 100 can determine that the shadow density of pixel P is 0, meaning that pixel P has no shadow effect. If the nearest distance d1 is less than or equal to the shadow radius Q, electronic device 100 can determine that pixel P is located in a shadow region. Then, electronic device 100 needs to further determine the shadow density of pixel P.

[0145] In one possible implementation, the electronic device 100 can calculate the shadow density of pixel P using a distance attenuation method. As shown in Figure 15(b), y represents the shadow density value, and the value of y ranges from greater than or equal to 0 to less than or equal to 1. Specifically, y = 0 indicates the maximum shadow density, and y = 1 indicates the minimum shadow density. The intermediate values ​​between 0 and 1 constitute a smooth transition between light and dark. The independent variable x is the ratio of the nearest distance d1 to the shadow radius Q. Since the nearest distance d1 is less than or equal to the shadow radius Q, the value of the independent variable x ranges from greater than or equal to 0 to less than or equal to 1. The electronic device 100 can substitute the ratio of the nearest distance d1 to the shadow radius Q into the function F(x) = y to determine the value of y. Then, the electronic device 100 can multiply the original primitive color at pixel P by y to darken it, obtaining the final color value of pixel P.

[0146] Optionally, in one possible implementation, the N primitives may include primitive D, which has Fresnel refraction and reflection effects. Electronic device 100 needs to calculate the Fresnel refraction and reflection effects of primitive D. That is, electronic device 100 needs to determine whether the color of a pixel in space is affected by the Fresnel refraction and reflection effects of primitive D.

[0147] Figure 16 exemplarily illustrates a schematic diagram of the Fresnel refraction and reflection effects of primitive D calculated by electronic device 100. As shown in Figure 16, primitive D exhibits both Fresnel refraction and reflection effects. A screen pixel P30 exists on the screen of electronic device 100. Electronic device 100 can emit a ray from pixel P30, and by line-of-sight tracking, it can be determined that the ray reaches point P31 on the surface of primitive D. Electronic device 100 can determine the reflection direction of the ray at point P31 based on the Fresnel formula. After determining the reflection direction, the electronic device can query the environmental texture based on the reflection direction to obtain the environmental color value 1. The environmental texture is a panoramic image that can store the 3D environment within a 2D image. Electronic device 100 can determine the color value for any 2D reflection direction based on this environmental texture.

[0148] As shown in Figure 16, the electronic device 100 can also calculate the refraction direction of the ray at point P31 based on Fresnel's formula and discovery. Then, the electronic device 100 can continue to move the ray along the refraction direction, and the ray can reach point P32 on the inner surface of the primitive D. The ray can be refracted at point P32 to the outside of the primitive D and reach a point on the background image, namely point P33. The electronic device 100 can extract the color value 2 of point P33 from the background image as the refraction color. The background image is a conventional two-dimensional image, the content of which can be art materials or real photos, etc., and this application embodiment does not limit it. For example, the background image can be the background control 304 shown in Figure 3. The image specifically corresponding to the background control 304 is the image shown in Figure 1 or Figure 2 in Figure 3.

[0149] Based on the above reflection and refraction calculations, the final color value of point P30 is 3 = color value 1 * u1 + v1 + color value 2 * u2 + v2. Here, "*" represents multiplication, and u1, u2, v1, and v2 are linear parameters used to attenuate, amplify, or shift the color. Finally, the electronic device 100 can determine that the color value of pixel P30 is color value 3.

[0150] S706, Electronic device 100 displays the first image frame.

[0151] After the electronic device 100 draws all the graphic elements and their animations by calling a drawcall instruction once, it obtains the drawing result. The electronic device 100 can then display the drawing result and show the first image frame.

[0152] In one possible implementation, before executing step S702, the electronic device 100 can first determine the animated primitives and non-animated primitives among the N primitives described in the drawing data. For the non-animated primitives, the electronic device 100 can draw them based on a traditional rasterization drawing process. For the animated primitives, the electronic device 100 can draw them according to steps S702-S705. Then, the electronic device 100 can fuse the drawing result based on the traditional rasterization drawing process with the drawing result according to steps S702-S705 to obtain a first image frame, which the electronic device can then display. For example, when the electronic device 100 determines that the i-th primitive has an animation effect, it can construct a primitive SDF for the i-th primitive and build a global acceleration structure. Then, it calculates the animation effect of the primitive SDF, and finally, the electronic device 100 can draw the i-th primitive. Once the electronic device 100 determines that the j-th primitive does not have animation effects, the electronic device 100 can represent the j-th primitive with a triangle, and then perform vertex calculation, rasterization, pixel coloring, etc., to obtain the drawing result of the j-th primitive.

[0153] In one possible implementation, for primitives without animation effects, the electronic device 100 can draw the primitives using the basic animation pipeline shown in Figure 5. This basic animation pipeline can draw the primitives based on a conventional rasterization rendering process. For primitives with animation effects, the electronic device 100 can construct a primitive SDF using the primitive processing module shown in Figure 5. Then, the electronic device 100 draws the primitives with animation effects using the higher-order animation pipeline shown in Figure 5. Finally, the electronic device 100 can fuse the drawing results from the basic animation pipeline and the higher-order animation pipeline using the fusion module shown in Figure 5, and send the fused drawing result to the layer compositing display module. The layer compositing display module then displays the fused drawing result, i.e., the first image frame, on the display 503.

[0154] It is understood that in the embodiments of this application, the screen may also be referred to as a display screen or monitor, or a touch screen, touch control screen, etc., and the embodiments of this application do not limit it in this way.

[0155] The graphics rendering method provided in this application allows the electronic device 100 to construct a primitive SDF (Software Rendering Function) and store it in a globally accelerated structure. By using gaze tracking to determine the screen visibility and spatial visibility of pixels on the screen, the ultimately visible primitives within a pixel can be identified. Furthermore, gaze tracking can calculate the animation effects of the primitives, ultimately determining the pixel's color value. Finally, the electronic device 100 can invoke a drawcall instruction once based on the aforementioned calculation results to render the first image frame. This improves the rendering effect of the electronic device 100 and reduces its power consumption.

[0156] For example, as shown in Figure 17, for the same control to be drawn, the electronic device 100 needs to represent primitives with triangles through the traditional rasterization drawing process. Then, the electronic device 100 needs to perform vertex calculation, rasterization, pixel coloring, etc. for each control, call the drawcall instruction multiple times, and additional video memory read and write operations are required to draw the final displayed user interface.

[0157] As shown in Figure 17, the electronic device 100 draws the same control to be drawn using the graphics drawing method provided in this embodiment of the application, constructing control 1, control 2, control 3, and control 4 of the control to be drawn into primitive SDFs respectively. Then, the electronic device can store the constructed multiple primitive SDFs into a global acceleration structure. The electronic device can finally determine the visible primitives and color values ​​of each pixel on the screen of the electronic device 100 through gaze calculation. Then, the electronic device 100 can draw the final displayed user interface in a single draw call.

[0158] As shown in Figure 17, using the graphics drawing method provided in this embodiment, the electronic device 100 only needs to call the drawcall instruction once to draw the user interface, and no additional video memory read / write is required. This improves the drawing effect of the electronic device 100 and reduces its power consumption.

[0159] In this embodiment, the first image frame can be the image frame finally displayed by the electronic device as shown in FIG. 17. The N controls can include controls 1, 2, 3, and 4 as shown in FIG. 17. The first control can be control 1. The first ray can be ray T1, and the first pixel can be pixel P8. One or more controls can include controls corresponding to primitive A and primitive B. The second control can be the control corresponding to primitive B. The second ray can be ray T2. The first point can be point P10, and the second point can be point P14. The third control can be the control corresponding to primitive A. The fourth control can be the control corresponding to primitive B. The fifth control can be the control corresponding to primitive C. The sixth control can be the control corresponding to primitive A in FIG. 15. The second pixel can be pixel P shown in FIG. 15. The seventh control can be the control corresponding to primitive D in FIG. 16. The third pixel can be screen pixel P30. The third point can be point P31, and the fourth point can be point P33.

[0160] The exemplary electronic device 1000 provided in the embodiments of this application will be introduced first below.

[0161] Figure 18 is a schematic diagram of the structure of the electronic device 1000 provided in an embodiment of this application.

[0162] The following detailed description uses electronic device 1000 as an example. It should be understood that electronic device 1000 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0163] Electronic device 1000 may include: processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0164] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 1000. In other embodiments of this application, the electronic device 1000 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0165] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0166] The controller can be the nerve center and command center of the electronic device 1000. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0167] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0168] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0169] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces.

[0170] The I2S interface can be used for audio communication.

[0171] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals.

[0172] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0173] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 1000 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 1000 to display images.

[0174] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0175] The SIM interface can be used to communicate with the SIM card interface 195 to transmit data to or read data from the SIM card.

[0176] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, or USB Type-C port. USB port 130 can be used to connect a charger to charge electronic device 1000, and can also be used for data transfer between electronic device 1000 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0177] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 1000. In other embodiments of this application, the electronic device 1000 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0178] The charging management module 140 is used to receive charging input from the charger. The charger can be a wireless charger or a wired charger.

[0179] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc.

[0180] The wireless communication function of the electronic device 1000 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0181] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 1000 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0182] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 1000. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0183] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0184] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 1000, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0185] In some embodiments, antenna 1 of electronic device 1000 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 1000 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0186] Electronic device 1000 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0187] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 1000 may include one or N displays 194, where N is a positive integer greater than 1.

[0188] Electronic device 1000 can achieve shooting function through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0189] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0190] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 1000 may include one or N cameras 193, where N is a positive integer greater than 1.

[0191] A digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals.

[0192] Video codecs are used to compress or decompress digital video. Electronic device 1000 can support one or more video codecs. Thus, electronic device 1000 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0193] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0194] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 1000. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0195] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 1000 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as facial recognition, fingerprint recognition, mobile payment, etc.). The data storage area may store data created during the use of electronic device 1000 (such as facial information template data, fingerprint information templates, etc.). Furthermore, internal memory 121 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0196] Electronic device 1000 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0197] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0198] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. Electronic device 1000 can listen to music or make hands-free calls through the speaker 170A.

[0199] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 1000 answers a phone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0200] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 1000 may have at least one microphone 170C. In some embodiments, electronic device 1000 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 1000 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0201] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0202] The pressure sensor 180A is used to sense pressure signals and can convert pressure signals into electrical signals.

[0203] The gyroscope sensor 180B can be used to determine the motion attitude of electronic device 1000.

[0204] The 180C barometric pressure sensor is used to measure barometric pressure.

[0205] The magnetic sensor 180D includes a Hall sensor. The electronic device 1000 can use the magnetic sensor 180D to detect the opening and closing of the flip cover.

[0206] The 180E accelerometer can detect the magnitude of acceleration in various directions (typically three axes) of an electronic device 1000.

[0207] The distance sensor 180F is used to measure distance. The electronic device 1000 can measure distance via infrared or laser.

[0208] The proximity light sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode.

[0209] The 180L ambient light sensor is used to detect ambient light intensity.

[0210] The fingerprint sensor 180H is used to collect fingerprints.

[0211] The 180J temperature sensor is used to detect temperature.

[0212] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 1000, in a different position than display screen 194.

[0213] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. The electronic device 1000 can receive button input and generate key signal inputs related to user settings and function control of the electronic device 1000.

[0214] Motor 191 can generate vibration alerts.

[0215] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0216] The SIM card interface 195 is used to connect the SIM card.

[0217] The electronic device 1000 can perform the steps executed by the electronic device 100 and achieve the kinetic energy achieved by the electronic device 100. The specific steps executed by the electronic device 100 and the kinetic energy achieved are described above and will not be repeated here.

[0218] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0219] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

Claims

1. A method for drawing graphics, characterized in that, include: The electronic device acquires the drawing data of a first image frame, which is the image frame to be drawn in the target application. The drawing data is used to describe the N controls contained in the first image frame and the attributes of the N controls. The electronic device constructs N primitive symbol distance fields (SDFs) based on the N controls. The N controls include a first control, and the N primitive SDFs include a first primitive SDF corresponding to the first control. The first primitive SDF is used to describe the positional relationship between any point in space and the first control. The electronic device determines the occlusion relationship of the N controls based on the SDF of the N primitives through eye tracking; The electronic device draws the first image frame based on the occlusion relationship of the N controls; The electronic device displays the first image frame.

2. The method according to claim 1, characterized in that, The electronic device constructs N primitive symbol distance fields (SDFs) based on the N controls, specifically including: The electronic device constructs a distance equation from any point in space to the first control based on the geometric analytical expression corresponding to the geometry of the first control. The distance equation is the first primitive SDF.

3. The method according to claim 1, characterized in that, The electronic device constructs N primitive symbol distance fields (SDFs) based on the N controls, specifically including: The electronic device constructs a bounding box of primitives that surrounds the first control, and divides the bounding box of primitives into k rectangles, where the center point of the k rectangles represents k coordinate points with the center of the first control as the origin of the coordinate system; The electronic device calculates the distances from the k coordinate points to the surface of the first control, and obtains k distance values; The electronic device generates a texture image based on the k distance values, and the texture image is the first primitive SDF.

4. The method according to claim 1, characterized in that, The electronic device constructs N primitive symbol distance fields (SDFs) based on the N controls, specifically including: The electronic device constructs a first neural network. The input of the first neural network is the coordinates of any point in space, and the output of the first neural network is the distance from any point in space to the surface of the first control. The first neural network is the first primitive SDF.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: The electronic device clusters the N primitive SDFs according to clustering rules to obtain a global acceleration structure, which contains the attributes of the N primitive SDFs. The electronic device stores the global acceleration structure.

6. The method according to claim 5, characterized in that, The occlusion relationship includes screen visibility; the electronic device determines the occlusion relationship of the N controls based on the SDF of the N primitives through gaze tracking, specifically including: The electronic device constructs a first ray starting from a first pixel in the display screen of the electronic device, and the direction of the first ray is perpendicular to the display screen; The electronic device queries the global acceleration structure to determine that there are one or more controls in the direction of the first ray, and the N controls include the one or more controls; The electronic device determines the nearest distance between the first pixel and the one or more controls based on the SDF of the primitives corresponding to the one or more controls; The electronic device determines that the control visible to the first pixel is the second control among the one or more controls based on the nearest distance between the first pixel and the one or more controls.

7. The method according to claim 5, characterized in that, The occlusion relationship includes spatial visibility; the electronic device determines the occlusion relationship of the N controls based on the SDF of the N primitives through gaze tracking, specifically including: The electronic device constructs a second ray with the first point on the third control as the starting point and the second point on the fourth control as the ending point, and the N controls include the third control and the fourth control; The electronic device queries the global acceleration structure to determine that a fifth control exists in the direction of the second ray; The electronic device determines the distance between the first point and the fifth control based on the SDF of the primitive corresponding to the fifth control; Based on the distance between the first point and the fifth control, the electronic device moves the second ray until the starting point of the second ray is moved to the second point, and the electronic device determines that there is no obstruction between the third control and the fourth control.

8. The method according to claim 7, characterized in that, After the electronic device determines the occlusion relationship of the N controls based on the N primitive SDFs through gaze tracking, the method further includes: The electronic device determines from the drawing data that the sixth control among the N controls has a shadow effect, and the electronic device calculates the shadow effect.

9. The method according to claim 8, characterized in that, The electronic device calculates the shadow effect, specifically including: The electronic device determines the shadow radius and shadow path of the sixth control, and constructs a shadow SDF based on the shadow radius and shadow path; The electronic device calculates the nearest distance from the second pixel to the shadow of the sixth control based on the shadow SDF; The electronic device determines the shadow density of the second pixel based on the nearest distance from the shadow of the second pixel to the shadow of the sixth control, and the shadow radius; The electronic device determines the color value of the second pixel based on the shadow density.

10. The method according to claim 7, characterized in that, After the electronic device determines the occlusion relationship of the N controls based on the N primitive SDFs through gaze tracking, the method further includes: The electronic device determines from the drawing data that the seventh control among the N controls has a refraction effect and a reflection effect, and the electronic device calculates the refraction effect and the reflection effect.

11. The method according to claim 10, characterized in that, The electronic device calculates the refraction effect and the reflection effect, specifically including: The electronic device constructs a third ray starting from a third pixel, where the third pixel is a pixel in the display screen of the electronic device, and the direction of the third ray is perpendicular to the display screen. The electronic device determines the third point on the surface of the seventh control based on eye-tracking; The electronic device determines the reflection and refraction directions of the third ray at the third point; The electronic device queries the environmental texture based on the reflection direction and determines the color value of the environment as a first color value. The electronic device moves the third ray based on the refraction direction, and the third ray moves to the fourth point in the background image; The electronic device extracts the second color value of the fourth point in the background image; The electronic device determines the color value of the third pixel based on the first color value and the second color value.

12. An electronic device, characterized in that, The device includes a display screen, one or more processors, and one or more memories; wherein the display screen, the one or more memories, and the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, the computer program code including computer instructions, which, when executed by the one or more processors, cause the method as described in any one of claims 1-11 to be performed.

13. A chip system applied to an electronic device, the chip system comprising one or more processors, characterized in that, The processor is used to invoke computer instructions to cause the execution of the method as described in any one of claims 1-11.

14. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, they cause the method as described in any one of claims 1-11 to be performed.

Images