Graph drawing method and related apparatus

Abstract

The present application provides a graph drawing method and a related apparatus. The method can comprise: an electronic device can construct primitive SDFs and store the primitive SDFs as a global acceleration structure, and perform screen visibility determination and spatial visibility determination for pixel points in a screen by means of a gaze tracking method, so that finally visible primitives at the pixel points can be determined. Moreover, the dynamic effect of the primitives can be calculated by means of gaze tracking, and color values of the pixel points are finally determined. Finally, the electronic device can call a drawcall instruction on the basis of the calculation result to draw a first image frame. In this way, the drawing effect of the electronic device can be improved, and the power consumption of the electronic device can be reduced.

Description

A graphics drawing method and related device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on December 27, 2023, with application number 202311839884.5 and application name “A Graphics Drawing Method and Related Devices”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminals and image drawing, and in particular to a graphics drawing method and related devices. Background Art

[0003] A window is one of the most important graphical user interfaces (GUIs) in an operating system. A window can be understood as the visual graphical interface of an operating system. For example, the desktop on electronic devices such as computers and mobile phones is a window interface. A window provides a bridge between the user and the operating system, allowing the user to interact with the operating system intuitively through a graphical interface. Windows are divided into two-dimensional windows (or traditional windows) and spatial windows based on their dimensions. Flat windows mainly feature two-dimensional effects. Spatial windows combine three-dimensional and real-world physical simulation effects (for example, blur effects, light and shadow effects, dynamic wallpapers, etc.) with the two-dimensional effects.

[0004] With the development of operating systems and human-computer interaction theory, spatial windows will be increasingly used and displayed in electronic devices. Currently, traditional graphics rendering pipelines used to draw two-dimensional windows have difficulty efficiently and power-efficiently rendering the three-dimensional effects and realistic physical simulation effects in spatial windows.

[0005] Therefore, how electronic devices can draw spatial windows efficiently and with low power consumption is an urgent problem to be solved. Summary of the Invention

[0006] The present application provides a graphics drawing method and related devices. Through the graphics drawing method provided in the embodiments of the present application, an electronic device can draw a spatial window with high efficiency and low power consumption.

[0007] In a first aspect, the present application provides a graphics drawing method, which may include: an electronic device obtaining drawing data of a first image frame, where the first image frame is an image frame to be drawn in a target application, and the drawing data is used to describe N controls contained in the first image frame and the properties of the N controls; the electronic device constructs corresponding N graphic element symbol distance fields (SDFs) based on the N controls, wherein the N controls include a first control, the N graphic element SDFs include a first graphic element SDF corresponding to the first control, and the first graphic element SDF is used to describe the positional relationship between any point in space and the first control; the electronic device determines the occlusion relationship of the N controls based on the N graphic element SDFs through line of sight tracking, and the electronic device draws the first image frame based on the occlusion relationship of the N controls; the electronic device displays the first image frame.

[0008] In this way, when drawing an image frame, the electronic device does not need to use traditional triangles to express controls, which can reduce the amount of data and repeated drawing. After the electronic device determines the occlusion relationship, it can call a drawing instruction once according to the occlusion relationship to draw the first image frame. When the first image frame has a motion effect, the first image frame can also be called a spatial window. Regardless of whether the first image frame is a plane window or a spatial window, the electronic device can draw the first image frame efficiently and with low power consumption.

[0009] In combination with the first aspect, in a possible implementation method, the electronic device constructs N corresponding graphic element symbol distance fields SDF based on N controls, which may specifically include: the electronic device constructs a distance equation from any point in space to the first control based on the geometric analytical expression corresponding to the geometric shape of the first control, and the distance equation is the first graphic element SDF.

[0010] The geometric expression corresponding to the geometric shape of the first control may be a Bezier curve expression.

[0011] In this way, the electronic device can construct the first primitive SDF using the geometric analytical expression corresponding to the first control.

[0012] In conjunction with the first aspect, in one possible implementation, an electronic device constructs N corresponding primitive symbol distance fields (SDFs) based on N controls. Specifically, the electronic device may include: constructing a primitive bounding box that encloses a first control, and dividing the primitive bounding box into k rectangles, where the center points of the k rectangles represent k coordinate points with the center of the first control as the coordinate origin; the electronic device calculates the distances from the k coordinate points to the surface of the first control to obtain k distance values; and the electronic device generates a texture image based on the k distance values, where the texture image is the first primitive SDF. In this way, the electronic device can construct a first primitive SDF corresponding to the first control, and the first primitive SDF is the texture image.

[0013] In conjunction with the first aspect, in one possible implementation, the electronic device constructs N corresponding primitive symbol distance fields (SDFs) based on N controls. Specifically, the electronic device may construct a first neural network, where the input of the first neural network is the coordinates of any point in space, the output of the first neural network is the distance from any point in space to the surface of the first control, and the first neural network is a first primitive SDF. In this way, the electronic device can construct a first primitive SDF corresponding to the first control, and the first primitive SDF is the first neural network.

[0014] In conjunction with the first aspect, in one possible implementation, the method may further include: the electronic device clustering N primitive SDFs according to a clustering rule to obtain a global acceleration structure, wherein the global acceleration structure includes attributes of the N primitive SDFs; and the electronic device storing the global acceleration structure. This allows the electronic device to more conveniently call or query the attributes of each primitive SDF.

[0015] In combination with the first aspect, in a possible implementation, the occlusion relationship includes screen visibility; the electronic device determines the occlusion relationship of N controls through line of sight tracking based on N graphic primitives SDF, which may specifically include: the electronic device constructs a first ray with the first pixel point in the display screen of the electronic device as the starting point, and the direction of the first ray is perpendicular to the display screen; the electronic device queries the global acceleration structure to determine that there are one or more controls in the direction of the first ray, and the N controls include one or more controls; the electronic device determines the closest distance between the first pixel point and the one or more controls based on the graphic primitives SDF corresponding to the one or more controls; the electronic device determines that the control visible to the first pixel point is the second control among the one or more controls based on the closest distance between the first pixel point and the one or more controls. In this way, the electronic device can determine the control that is ultimately visible to each pixel point in the display screen.

[0016] In combination with the first aspect, in a possible implementation, the occlusion relationship includes spatial visibility; the electronic device determines the occlusion relationship of N controls through line of sight tracking based on N graphic primitives SDF, which may specifically include: the electronic device constructs a second ray with the first point on the third control as the starting point and the second point on the fourth control as the end point, and the N controls include the third control and the fourth control; the electronic device queries the global acceleration structure to determine that there is a fifth control in the direction of the second ray; the electronic device determines the distance between the first point and the fifth control based on the graphic primitive SDF corresponding to the fifth control; based on the distance between the first point and the fifth control, the electronic device moves the second ray until the electronic device moves the starting point of the second ray to the second point, and the electronic device determines that there is no occlusion between the third control and the fourth control. In this way, the electronic device can determine the occlusion relationship between each control in the first image frame.

[0017] In conjunction with the first aspect, in one possible implementation, after the electronic device determines the occlusion relationship between the N controls through line of sight tracking based on the N primitive SDFs, the method may further include: the electronic device determining from the drawing data that a sixth control among the N controls has a shadow effect, and the electronic device calculating the shadow effect. In this way, the electronic device can determine the shadow effect in the first image frame.

[0018] In conjunction with the first aspect, in one possible implementation, the electronic device calculates a shadow effect, which may specifically include: the electronic device determining the shadow radius and shadow path of the sixth control, and constructing a shadow SDF based on the shadow radius and shadow path; the electronic device calculating the closest distance from the second pixel to the shadow of the sixth control based on the shadow SDF; the electronic device determining the shadow density of the second pixel based on the closest distance from the second pixel to the shadow of the sixth control and the shadow radius; and the electronic device determining the color value of the second pixel based on the shadow density. In this way, the electronic device can determine the impact of the shadow effect on the color value of each pixel on the display screen.

[0019] In conjunction with the first aspect, in one possible implementation, after the electronic device determines the occlusion relationship between N controls through line of sight tracking based on the N primitive SDFs, the method may further include: the electronic device determining from the drawing data that a seventh control among the N controls has a refraction effect and a reflection effect, and the electronic device calculating the refraction effect and the reflection effect. In this way, the electronic device can determine the refraction effect and the reflection effect in the first image frame.

[0020] In conjunction with the first aspect, in one possible implementation, the electronic device calculates the refraction and reflection effects, which may specifically include: the electronic device constructs a third ray with a third pixel point as a starting point, where the third pixel point is a pixel point on the display screen of the electronic device, and the direction of the third ray is perpendicular to the display screen; the electronic device determines, based on line of sight tracking, that the third ray reaches a third point on the surface of the seventh control; the electronic device determines the reflection and refraction directions of the third ray at the third point; the electronic device queries the environmental texture based on the reflection direction and determines that the color value of the environment is a first color value; the electronic device moves the third ray based on the refraction direction, so that the third ray reaches a fourth point in the background image; the electronic device retrieves the second color value of the fourth point in the background image; and the electronic device determines the color value of the third pixel based on the first and second color values. In this way, the electronic device can determine the impact of the refraction and reflection effects on the color value of each pixel point on the display screen.

[0021] In combination with the first aspect, in a possible implementation, before the electronic device constructs corresponding N graphic primitives SDF based on the N controls, the method may further include: the electronic device determining controls with motion effects among the N controls.

[0022] In combination with the first aspect, in a possible implementation, after the electronic device determines the controls with motion effects among N controls, the method may further include: the electronic device constructs a corresponding graphic primitive SDF based on the controls with motion effects among the N controls, the electronic device determines the occlusion relationship between the controls with motion effects through line of sight tracking, and draws the controls with motion effects based on the occlusion relationship; the electronic device draws N controls without motion effects based on a rasterization drawing method; the electronic device fuses the drawing results of the controls with motion effects and the drawing results of the controls without motion effects to obtain a first image frame, and the electronic device displays the first image frame.

[0023] In this way, the electronic device can draw the controls with motion effects and the controls without motion effects in parallel, which can improve the efficiency of drawing the first image frame.

[0024] In a second aspect, an electronic device is provided, which may include a display screen, one or more processors and one or more memories; wherein the display screen, one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the method involved in any possible implementation method of the first aspect.

[0025] According to a third aspect, an electronic device is provided. The electronic device may include one or more functional modules, and the one or more functional modules are used for the method involved in any possible implementation of the first aspect.

[0026] In a fourth aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method involved in any possible implementation of the first aspect.

[0027] In a fifth aspect, a computer-readable storage medium is provided, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute the method involved in any possible implementation of the first aspect.

[0028] In a sixth aspect, a computer program product is provided. When the program product is run on an electronic device, the electronic device executes the method involved in any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] FIG1A is a schematic diagram of a flat window provided in an embodiment of the present application;

[0030] FIG1B is a schematic diagram of a set of spatial windows provided in an embodiment of the present application;

[0031] FIG2 is a schematic diagram comparing the relationship between graphic elements in a planar window and a spatial window provided by an embodiment of the present application;

[0032] FIG3 is a schematic diagram of a process for drawing a window with a blur effect provided by an embodiment of the present application;

[0033] FIG4 is a schematic diagram of a process for drawing a window with a shadow effect provided by an embodiment of the present application;

[0034] FIG5 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0035] FIG6 is a schematic diagram of a software framework of an electronic device provided in an embodiment of the present application;

[0036] FIG7 is a flow chart of a graphics drawing method provided in an embodiment of the present application;

[0037] FIG8 is a schematic diagram of various representations of a primitive symbol distance field SDF provided by an embodiment of the present application;

[0038] FIG9 is a schematic diagram showing an analytical expression of a circular primitive SDF provided in an embodiment of the present application;

[0039] FIG10 is a schematic diagram of a discrete expression of a circular primitive SDF provided in an embodiment of the present application;

[0040] FIG11 is a schematic diagram of a neural network expression of a circular primitive SDF provided in an embodiment of the present application;

[0041] FIG12 is a schematic diagram of a global acceleration structure of a primitive SDF provided in an embodiment of the present application;

[0042] FIG13 is a schematic diagram of screen visibility determination provided by an embodiment of the present application;

[0043] FIG14 is a schematic diagram of spatial visibility determination provided by an embodiment of the present application;

[0044] FIG15 is a schematic diagram of a shadow effect calculation provided by an embodiment of the present application;

[0045] FIG16 is a schematic diagram of calculating a refraction effect and a reflection effect provided by an embodiment of the present application;

[0046] FIG17 is a schematic diagram comparing different drawing processes provided in embodiments of the present application;

[0047] FIG18 is a schematic structural diagram of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0049] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of this application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless the context clearly indicates otherwise. The terms "first" and "second" are used for descriptive purposes only and are not to be understood as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. "First" and "second" etc. are used to distinguish different objects, rather than to describe a specific order of objects. For example, the first object and the second object are used to distinguish different objects, rather than to describe a specific order of objects.

[0050] In the description of the embodiments of this application, unless otherwise specified, "a plurality" means two or more. For example, a plurality of processing units refers to two or more processing units; a plurality of systems refers to two or more systems.

[0051] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0052] The term "and / or" in this application is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0053] In the embodiments of the present application, electronic device 100 is capable of displaying both planar windows and spatial windows. Electronic device 100 may include, but is not limited to, mobile phones, tablet computers, and other devices capable of drawing and displaying spatial windows. The embodiments of the present application do not limit the specific form or type of electronic device 100.

[0054] The term "user interface (UI)" in the following embodiments of this application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that the user can recognize. The commonly used form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0055] In the embodiments of the present application, a window may also be referred to as a user interface. A window is drawn and presented by an electronic device through a graphics pipeline. A window may include window animation effects. Window animation effects are a general term for the visual effects and animation effects of a window. Visual effects are primarily static effects, presented by graphics rendering. Animation effects are primarily dynamic effects, presented dynamically by graphics. Visual effects and animation effects are the core of window interaction. Visual effects can be referred to as visual effects, and animation effects can be referred to as animation effects.

[0056] A flat window primarily includes two-dimensional visual effects. For example, as shown in FIG1A , the electronic device displays a desktop 1000, which is a flat window. Desktop 1000 only includes two-dimensional plane effects. It is understood that this application only uses a desktop that is currently a flat window as an example. In the future, the desktop of an electronic device may also develop into a spatial window, and this embodiment of the application does not limit this.

[0057] The spatial window combines three-dimensional and real-world physical simulation effects (e.g., blur effects, light and shadow effects, dynamic wallpapers, etc.) with two-dimensional graphic effects. The spatial window can include visual and motion effects. For example, as shown in Figure 1B (a), an electronic device can display user interface 10A while charging. User interface 10A can be a spatial window, which can include controls 101 and 102. Control 101 can include a luminous motion effect, and control 102 can include a luminous bubble motion effect. As shown in Figure 1B (b), the electronic device can display user interface 10B. User interface 10B can be a spatial window, which can include control 103. Control 103 can have a blur effect. As shown in Figure 1B (c), the electronic device can display user interface 10C. User interface 10C can be a spatial window, which can include control 104. Control 104 can have light and shadow effects. As shown in Figure 1B (d), the electronic device can display user interface 10D. The user interface 10D may be a spatial window, and the user interface 10D may include a control 105. The control 105 may be a three-dimensional sphere.

[0058] It is understood that the spatial windows shown in FIG1B , such as user interface 10A, user interface 10B, user interface 10C, and user interface 10D, are merely examples. The present application does not limit the specific content contained in the spatial windows or the specific display format of the spatial windows.

[0059] As shown in Figure 2 (a), the relationships between primitives in the two-dimensional viewport are independent. For example, primitive 1 and primitive 2 are independent of each other. When drawing primitive 1 in the two-dimensional viewport, only the local information of primitive 1 is required. When drawing primitive 2 in the two-dimensional viewport, the local information of primitive 2 is also required. As shown in Figure 2 (b), the relationships between primitives in the spatial viewport are interdependent. For example, primitive 1 and primitive 2 are interdependent. When drawing primitive 1 and primitive 2 in the spatial viewport, global information is required.

[0060] In some embodiments, the drawing pipeline for drawing motion effects in the spatial window is based on a traditional rasterization drawing process. When drawing the spatial window, first, the electronic device uses triangles to represent the primitives in the spatial window. For example, a rectangular primitive requires two triangles to represent. A circular primitive may require more than 100 triangles to represent. Then, the electronic device uses a triangle rasterization method to make visibility decisions, drawing the primitives in the spatial window from the bottom up, with the upper primitives covering the lower primitives. By calling multiple drawcall instructions, the final window motion effect of the spatial window is superimposed. In this way, when the electronic device calls the drawcall instruction multiple times for drawing, additional reading and writing of the video memory is required. The number of drawcall instructions in the electronic device system is too high. Although the computational complexity of a single drawcall instruction is not high, each drawcall itself has a certain basic power consumption. The high number of drawcall instructions causes a waste of computing resources. In addition, multiple drawcall instructions communicate with each other through the video memory, and the bandwidth consumption increases linearly with the number of drawcalls.

[0061] FIG3 exemplarily shows the drawing process of a window with a blur effect. As shown in FIG3 ①, first, the electronic device needs to take out the control that needs to be drawn with a blur effect from the frame buffer. Then, as shown in FIG3 ②, the electronic device can copy the control to the frame buffer used for drawing. The frame buffer used for drawing can also be called an off-screen buffer. As shown in FIG3 ③, FIG4, FIG5, and FIG6, the electronic device can downsample the control multiple times to change the resolution of the control from 2048*3072 to 128*256. Then, as shown in FIG3 ⑦, the electronic device can blur the control horizontally, and as shown in FIG3 ⑧, the electronic device can blur the control vertically. Then, as shown in FIG3 ⑨, the electronic device upsamples the blurred control to obtain a control with a resolution of 2048*3072 and a blur effect. Then, as shown in FIG3 ⑩, the electronic device copies the control with a blur effect in FIG3 ⑨ to the frame buffer. Finally, as shown in FIG3 As shown in the figure, the electronic device can superimpose the drawn control 301 (which can be called a search control), control 302 (which can be called a common control), and control 303 (which can be called a notification control) on the control with a blurred effect (i.e., control 304, which can also be called a background control) on the basis of the control shown in Figure ⑩ in Figure 3.

[0062] FIG4 exemplifies the process of drawing a window with a shadow effect. As shown in FIG4 ①, the electronic device first retrieves the control (i.e., control 401) to be drawn with a shadow effect from the frame buffer. Then, as shown in FIG4 ②, the electronic device can copy the shadowed portion of control 401, i.e., primitive 402, to the off-screen buffer. As shown in FIG4 ③, ④, and ⑤, the electronic device can downsample primitive 402 multiple times, reducing the resolution of primitive 402 from 512*512 to 64*64. Then, as shown in FIG4 ⑥, the electronic device can blur primitive 402 horizontally, and as shown in FIG4 ⑦, the electronic device can blur primitive 402 vertically. Then, as shown in FIG4 ⑧, the electronic device upsamples the blurred primitive 402 to obtain a primitive 404 with a shadow effect and a resolution of 512*512. Then, as shown in Figure 4 (9), the electronic device copies the blurred graphics element 404 and graphics element 403 in Figure 4 (8) to the frame buffer. Finally, as shown in Figure 4 (10), the electronic device can overlay other controls in the calculator interface on top of the controls shown in Figure 4 (9), thus forming the final calculator interface visible to the user.

[0063] In the drawing phase, the electronic device needs to call multiple drawcall instructions for both the blur effect shown in FIG3 and the shadow effect shown in FIG4 . For example, Table 1 shows relevant statistical data for the blur effect shown in FIG3 and the shadow effect shown in FIG4 in the drawing phase.

[0064] Table 1

[0065] As shown in Table 1, the electronic device draws the To render the user interface with a blur effect shown in the figure, the electronic device needs to call 35 drawcalls. The additional memory accesses between drawcalls consume 20MB of bandwidth, resulting in a frame rate of only 45 FPS. To render the user interface with a shadow effect shown in Figure 4 (Figure 10), the electronic device needs to call 135 drawcalls. The additional memory accesses between drawcalls consume 1MB of bandwidth, resulting in a frame rate of only 57 FPS.

[0066] Thus, it can be seen from Table 1 above that based on the traditional rasterization drawing process to draw the spatial window, the number of drawcalls and bandwidth consumption are relatively high, and it will cause the target scene (for example, the game scene) to fail to reach the expected target frame rate (for example, 90FPS).

[0067] The reasons why electronic devices use traditional rasterization drawing processes to draw spatial windows, resulting in high power consumption, are as follows: 1. Simple primitives: Primitives are represented by triangles, and some primitives require more triangles, which results in a high cost for a single drawcall. 2. Redundant drawing: When drawing primitives, upper-layer primitives will obscure lower-layer primitives, resulting in wasted computation and reduced efficiency of a single drawcall. 3. Local access: A single drawcall can only access local information of a primitive and cannot access global information between primitives. 4. Separate rendering: Multiple animations in the spatial window are rendered separately, resulting in a lack of data between multiple animations and repeated drawcalls.

[0068] In order to improve the performance of drawing windows and reduce the power consumption of electronic devices when drawing windows, an embodiment of the present application provides a graphics drawing method and an electronic device. In the graphics drawing method provided in an embodiment of the present application, by improving the primitive representation and drawing process, the number of drawcalls, the number of video memory accesses, and repeated drawing when drawing spatial windows can be effectively reduced. Before introducing the graphics drawing method provided in an embodiment of the present application, the hardware structure and software framework of the electronic device provided in an embodiment of the present application are first introduced below.

[0069] FIG5 is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application.

[0070] As shown in Figure 5, the electronic device 100 may include a central processing unit (CPU) 501, a memory 502, a display 503, and a graphics processing unit (GPU) 504. Optionally, the electronic device 100 may further include a communication bus for connecting various components.

[0071] It is understood that the various components in the electronic device 100 may also be coupled via other connectors, which may include various interfaces, transmission lines, or buses. The various components in the electronic device 100 may also be connected in a radial manner centered around the CPU 501. In the embodiments of the present application, coupling refers to being electrically connected or communicating with each other, including direct connection or indirect connection through other devices.

[0072] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0073] In the example of the present application, the CPU 501 and GPU 504 in the electronic device 100 can be located on the same chip or can be independent chips, which is not limited in the embodiment of the present application.

[0074] Among them, CPU501 may include an application 5011 and an operating system 5012. CPU501 can run the operating system 5012 and the application 5011. The application 5011 can be a graphics application, such as a game application or a video application, etc. The operating system 5012 may include a system graphics library interface and a driver, such as a graphics library user mode driver and / or a graphics library kernel mode driver. Among them, the system graphics library interface includes but is not limited to: system graphics libraries such as the Open Graphics Library for Embedded Systems (OpenGL ES), the Kronos Platform Graphics Interface (Khronos Platform Graphics Interface), or Vulkan (a cross-platform drawing application program interface). The graphics library user mode driver and / or the graphics library kernel mode driver can generate an instruction stream for rendering graphics or image frames, as well as the required related rendering data. The instruction stream can include a series of instructions, which are generally call instructions for the system graphics library interface.

[0075] The operating system 5012 may further include a layer display synthesis module that can be used to send the rendering result obtained by the GPU 504 based on the rendering instruction to the display 503 .

[0076] The memory 502 may store instruction streams and rendering data generated by a graphics library user-mode driver or a graphics library kernel driver.

[0077] The graphics processor CPU504 may include a motion rendering pipeline and a cache. Among them, the motion rendering pipeline may include a primitive processing module, a basic motion pipeline, a high-order motion pipeline, and a fusion module. The motion rendering pipeline can be used to draw rendering data based on the drawcall instruction in the instruction stream. Specifically, the primitive processing module can represent the primitives in the rendering data as primitive signed distance fields (SDF). The basic motion pipeline can be used to draw primitives SDF without motion. The high-order motion pipeline can be used to draw primitives SDF with motion. The fusion module can be used to fuse the drawing results of the basic motion pipeline and the drawing results of the high-order motion rendering pipeline. The fusion module can store the drawing results in the cache.

[0078] In the present application, an SDF is also referred to as a signed distance field. An SDF is a function that describes a spatial position. When given the coordinates of a point in space, the SDF returns the shortest distance between that point and a surface (curve or surface) in space. The sign of the return value indicates whether the point is inside or outside the surface.

[0079] The primitive SDF can be used to describe the positional relationship between each point in space and the primitive. There are many ways to express this primitive SDF, which can be described in detail below and will not be repeated here.

[0080] The display 503 can be used to display various images generated by the electronic device 100, which can be a GUI of the operating system 5012, or image data (including still images and video data) processed by the GPU 504. In the embodiment of the present application, the display 503 can be used to display the received rendering results.

[0081] FIG6 is a schematic diagram of the software framework of the electronic device 100 provided in an embodiment of the present application.

[0082] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the system (e.g., operating system 5012 shown in FIG5 ) is divided into four layers: the application layer, the application framework layer, the runtime and system libraries layer, and the kernel layer.

[0083] The application layer can include a series of application packages.

[0084] As shown in FIG6 , the application package may include game applications, video applications, and the like.

[0085] In some instances, the application package may also include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications (also referred to as applications).

[0086] It can be understood that the application layer can also include the application 5011 shown in Figure 5.

[0087] The application framework layer (also referred to as the framework layer) provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0088] As shown in Figure 6, the application framework layer may include a rendering system (RS). The rendering system may include a unified rendering service and a separate rendering service. The separate rendering service may be used to generate separate rendering instructions and rendering data. The separate rendering instructions may be sent to the GPU. The GPU may generate a basic geometric skeleton composed of triangles through a vertex shader based on the separate rendering instructions and rendering data. Then, the basic motion effect pipeline in the GPU may be filled with color through a pixel shader and then mapped. The unified rendering service may be used to generate unified rendering instructions and rendering data. The motion effect drawing pipeline in the GPU may first generate one or more primitive SDFs from the rendering data based on the unified rendering instructions and rendering data, and then draw one or more primitive SDFs through a drawcall instruction, and obtain the drawing results.

[0089] The system library may include multiple functional modules. For example, a graphics device interface and a graphics processing library. The rendering system may call the graphics processing library through the graphics device interface. The graphics processing library may include an open graphics library (OpenGL), a two-dimensional (2D) graphics engine (which may be referred to as a 2D engine), a three-dimensional (3D) graphics engine, a compositor, and a frame buffer. OpenGL is an application programming interface / function library, and OpenGL may also include OpenGL ES. A 2D engine is a drawing engine for drawing. A 3D engine is a drawing engine for 3D drawing. The compositor may provide fusion of 2D layers and 3D layers for multiple applications. A frame buffer may also be referred to as a frame buffer area, which may be used to cache drawn image frames.

[0090] In some embodiments, the graphics processing library may include the system library graphics library interface shown in FIG. 5 .

[0091] The driver layer is the layer between hardware and software. The driver layer includes at least the graphics processor driver (GPU driver) and the display controller driver (displaycontrollerdriver).

[0092] In some embodiments, the driver layer may further include a camera driver, an audio driver, a sensor driver, and a virtual card driver.

[0093] In some embodiments, the driver layer may also be referred to as a kernel layer.

[0094] In some embodiments, the driver layer may further include the graphics library kernel mode driver shown in FIG. 5 .

[0095] The following describes the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0096] When the touch sensor receives a touch, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, and other information). The raw input event is stored in the kernel layer. The application framework layer retrieves the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a single-click and the control corresponding to the camera application icon, the camera application calls the application framework layer's interface to start the camera application, which then calls the kernel layer to start the camera driver and capture still images or video through the camera.

[0097] The following is a detailed description of a graphics drawing method provided in an embodiment of the present application in conjunction with the software and hardware architecture of the exemplary electronic device 100 described above. The graphics drawing method provided in the present application may include: first, the electronic device 100 may obtain drawing data of a first image frame, and the first image frame is an image frame to be drawn in the target application. The electronic device 100 may construct N primitives SDF based on the drawing data. Then, the electronic device 100 may use eye tracking to determine the screen visibility of the N primitives SDF. Next, the electronic device 100 may draw N primitives SDF based on the screen visibility of the N primitives SDF to obtain the drawing result of the first image frame. Finally, the electronic device 100 may send the drawing result for display and display the first image frame.

[0098] Optionally, the electronic device 100 may also construct the N primitive SDFs into a global acceleration structure and store the global acceleration structure.

[0099] In this way, the electronic device 100 uses SDF to represent primitives, avoiding the excessive data size caused by the traditional rendering pipeline using triangles to represent primitives. By constructing N primitives into a global acceleration structure, the electronic device can easily access global information during rendering. The electronic device uses eye tracking to determine the screen visibility of the primitive, and only a single drawcall is required to obtain the rendering result. Furthermore, the electronic device 100 does not require additional video memory access during rendering, which can reduce bandwidth consumption.

[0100] FIG7 is a flow chart of a method for drawing a graphic provided by an embodiment of the present application. As shown in FIG7 , a method for drawing a graphic provided by an embodiment of the present application may include the following steps:

[0101] S701: The electronic device 100 obtains drawing data of a first image frame, where the first image frame is an image frame to be drawn in a target application.

[0102] The target application may be a game application or a video application installed in the electronic device.

[0103] When the target application is started or the user interface is refreshed, the drawing data of the image frame to be drawn can be sent to the electronic device 100. The image frame to be drawn can be the first image frame. The drawing data is used to describe the controls contained in the first image frame and the properties of the controls. For example, the properties of the controls may include but are not limited to: the name of the control, the shape of the control, the size of the control, the position of the control, the color of the control, etc. In some instances, the position of the control can be expressed as the coordinates of the control in the first image frame. Optionally, in other embodiments, the position of the control can be expressed as the distance from the control to the top of the first image frame and the distance from the control to the left border of the first image frame.

[0104] Optionally, in a possible implementation, the drawing data is in an extensible markup language (XML) file, and the target application may send the XML file containing the drawing data.

[0105] S702: The electronic device 100 constructs N primitive signed distance fields (SDFs) based on the drawing data.

[0106] The multiple controls described by the drawing data can be represented by N primitives. One control or multiple controls can correspond to one primitive. Then, the electronic device 100 can construct N primitive signed distance fields (SDFs).

[0107] It is understandable that each control may include one or more interface elements, such as text, pictures, icons, etc. In some instances, each interface element included in a control may also be referred to as a control or a sub-control, which is not limited in this embodiment of the present application.

[0108] N is a positive number. For example, N can be 1, N can be 2, N can be 3, and so on. The embodiment of the present application does not limit the specific value of N.

[0109] In the embodiment of the present application, the primitive SDF can be expressed in a variety of ways, such as analytical expression, discretization expression, neural network expression, etc.

[0110] For example, as shown in FIG8 , the analytical expression of the primitive of the control can include the geometric expression of the original graphic, such as the analytical expression of a Bezier curve. Then, electronic device 100 can construct an equation for the distance from any point on the display screen to the primitive based on the geometric expression of the primitive. This equation is the analytical expression of the primitive SDF corresponding to the control.

[0111] For example, when the control is the circular control 801 shown in FIG8 , the electronic device 100 can express the circular control 801 using a geometric expression of the circle. Then, based on the geometric expression of the circle, the electronic device 100 can construct Equation 1 for the distance from any point on the display screen to the surface of the circular control 801. Equation 1 is the analytical expression of the primitive SDF corresponding to the circular control 801.

[0112] For another example, when the control is the rectangular control 802 shown in FIG8 , the electronic device 100 can express the rectangular control 802 using a geometrical expression of the rectangle. Then, based on the geometrical expression of the rectangle, the electronic device 100 can construct Equation 2 for the distance from any point on the display screen to the surface of the rectangular control 802. Equation 2 is the analytical expression of the primitive SDF corresponding to the rectangular control 802.

[0113] When the control is the triangle control 803 shown in FIG8 , the electronic device 100 can express the triangle control 803 using a geometric analytical expression of the triangle. Then, based on the geometric expression of the triangle, the electronic device 100 can construct Equation 3 for the distance from any point on the display screen to the surface of the triangle control 803. Equation 3 is the analytical expression of the primitive SDF corresponding to the triangle control 803.

[0114] When the control is the elliptical control 804 shown in FIG8 , the electronic device 100 can express the elliptical control 804 using the geometrical expression of the ellipse. Then, based on the geometrical expression of the ellipse, the electronic device 100 can construct Equation 4 for the distance from any point on the display screen to the surface of the elliptical control 804. Equation 4 is the analytical expression of the primitive SDF corresponding to the circular control 801.

[0115] Optionally, the electronic device 100 can also use Bezier curve analytical expressions to express the circular control 801, rectangular control 802, triangular control 803, and elliptical control 804 shown in Figure 8. It is understandable that the Bezier curve analytical expressions corresponding to the circular control 801, rectangular control 802, triangular control 803, and elliptical control 804 are different. When the electronic device 100 expresses the circular control 801, rectangular control 802, triangular control 803, and elliptical control 804 shown in Figure 8 using Bezier curve analytical expressions, the circular control 801, rectangular control 802, triangular control 803, and elliptical control 804 can achieve the effect of scalable vector graphics.

[0116] Optionally, as shown in FIG8 , electronic device 100 may further express the primitive SDFs corresponding to circular control 801, rectangular control 802, triangular control 803, and elliptical control 804 as discretized texture images. For example, the primitive SDF corresponding to circular control 801 may be expressed as texture image 805. The primitive SDF corresponding to rectangular control 802 may be expressed as texture image 806. The primitive SDF corresponding to triangular control 803 may be expressed as texture image 807. The primitive SDF corresponding to elliptical control 804 may be expressed as texture image 808.

[0117] Texture images 805, 806, 807, and 808 include original primitives and texture lines outside and inside the original primitives. Multiple points on each circle of texture lines are equidistant from the surface of the original primitive.

[0118] Optionally, electronic device 100 may also express primitives (SDFs) using a neural network. For example, as shown in FIG8 , the SDF primitives corresponding to texture image 809 may also be expressed as a neural network 810. It should be understood that the neural network 810 shown in FIG8 is merely an example, and the specific structure of the neural network 810 is not limited in this embodiment of the present application.

[0119] The following section uses a circle control as an example to explain various ways to express the corresponding primitive SDF. For geometric expressions for rectangles, triangles, and ellipses, refer to relevant textbooks and will not be repeated here.

[0120] As shown in Figure 9, the distance from any point P0 (x0, y0) in space to the circle is d0. The radius of the circle is r, and the center O is the coordinate origin. The relationship between d0 and x0, y0, and r can be expressed as Formula 1:

[0121] The electronic device 100 can express the primitive SDF of the circular control as the equation shown in the above formula 1. In one possible implementation, the solution process of the formula 1 can be completed at runtime, and the specific calculation can be performed by the shader in the GPU in the electronic device 100.

[0122] The primitive SDF of the circular primitive shown in Figure 9 can also be expressed through discretization. As shown in Figure 10, the original circular primitive is covered by the primitive bounding box. The primitive bounding box is evenly divided into 8*8 small rectangles. The center points of these small rectangles represent 8*8 fixed coordinates in the circular coordinate system, with the center of the circle as the coordinate origin. Electronic device 100 can use the primitive SDF analytical expression shown in Formula 1 to calculate the closest distance from the 8*8 fixed coordinates to the circular surface. Electronic device 100 can save the calculated distance as the corresponding pixel value of the texture image. The texture image can be seen as texture image 805 shown in Figure 8. For points outside the original primitive, the closest distance to the primitive surface is positive. For points inside the primitive, the closest distance to the primitive surface is negative. For points on the primitive surface, the closest distance to the primitive surface is 0. For example, the discrete coordinate point P0 shown in Figure 10 is outside the primitive, and the closest distance to the primitive is 1. In the embodiment of the present application, the unit of the closest distance may be pixels.

[0123] The electronic device 100 can generate the texture image 805 shown in FIG8 based on the pixel values ​​corresponding to the texture image, that is, the closest distance from the 8*8 fixed coordinates to the circular surface shown in FIG10. Optionally, the texture image stored by the electronic device 100 can include 8*8 pixels, and the 8*8 pixels respectively store the distance from the center point of the 8*8 rectangle in FIG10 to the original primitive surface.

[0124] For example, the primitive SDF corresponding to the circular primitive can also be expressed as a neural network as shown in Figure 11. As shown in Figure 11, in the neural network expression corresponding to the primitive SDF, the input of the neural network can be x, y, 1. Among them, x and y can be the horizontal coordinate and vertical coordinate of any point on the display screen, respectively. The network weight of the neural network is w, the activation function is f, and the jump connection is g. When x, y, 1 are input into the neural network, the neural network can output the closest distance L from the coordinate point (x, y) to the original primitive surface.

[0125] It is understood that the embodiments of the present application do not limit the specific values ​​of the network weight w, the activation function f, and the jump connection g in the neural network shown in Figure 11. The embodiments of the present application also do not limit the specific structure of the neural network shown in Figure 11.

[0126] S703: The electronic device 100 constructs a global acceleration structure using N primitive SDFs.

[0127] The original primitives in the N primitive SDFs can include a variety of primitive shape types, such as circles, rectangles, triangles, and ellipses. Each primitive shape type can have multiple instances, such as 10 circles, 10 rectangles, and so on. That is, the N primitive SDFs can contain multiple circular primitive SDFs and multiple rectangular primitive SDFs, although this is not a limitation in the present embodiment. However, if the N primitives are processed separately, a large amount of duplicate data may exist, which would hinder the GPU from transmitting and reading the drawing data when drawing the first image frame.

[0128] The electronic device 100 can construct a storage structure for all scenarios to store N primitive SDFs. This storage structure can be called a global acceleration structure, and the embodiment of the present application does not limit the name of this storage structure.

[0129] FIG12 exemplarily shows a global acceleration structure of a primitive SDF. As shown in FIG12 , the global acceleration structure can be divided into two parts, a global relationship structure and a global attribute structure. Among them, the global relationship structure is a clustering of primitive instances in N primitive SDFs, and the N primitive SDFs are organized together according to specific clustering rules. Clustering can form a tree structure (or hierarchical structure) according to different granularities, that is, a cluster can contain several subclusters. Clustering rules may include but are not limited to scene space division based on a grid or a K-dimensional tree (kd-tree), screen space division based on grid tiles, and the like.

[0130] Among them, the global attribute structure is the actual storage format of the primitive SDF data. All primitive SDFs of the same type share the same attribute structure. For example, as shown in Figure 12, taking the circular primitive SDF (also known as a circular instance) as an example, all circles have two attributes: center and radius. The difference is that the specific values ​​of the center and radius are different in different circles. Primitive SDFs of the same type are stored in a continuous storage space to facilitate efficient retrieval when the GPU draws the primitive SDF. Since there are three ways to express primitive SDFs, there are also three ways to express the global attribute structure. For a primitive SDF expressed in an analytical form, the attributes of the primitive SDF can be composed of a set of parameters. For example, the attributes of a circular primitive SDF may include the center, radius, etc. For a primitive SDF expressed in a discrete form, the attributes of the primitive SDF can be composed of a distance field texture, such as the texture image 805 shown in Figure 5. For a primitive SDF expressed in a neural network, the attributes of the primitive SDF can be composed of network weights (e.g., w1, ..., wn). Optionally, the attributes of the primitive SDFs expressed analytically, discretely, and neurally can also include a transformation matrix. This transformation matrix can be used to describe the spatial transformation properties of the primitive SDF, such as translation, rotation, and scaling.

[0131] It is understandable that the electronic device 100 can store N primitive SDFs according to the attribute structure corresponding to any one of the three expressions: primitive SDF expressed in analytical form, primitive SDF expressed in discrete form, and primitive SDF expressed in neural network form.

[0132] S704: The electronic device 100 draws N primitives SDF in the global acceleration structure using a gaze tracking algorithm.

[0133] The electronic device 100 can calculate and determine two types of visibility determinations for the N primitive SDFs using a gaze tracking algorithm. These two types of visibility determinations can include screen visibility determination and spatial visibility determination. The screen visibility determination can be used to determine which primitives are visible to each pixel on the screen. The spatial visibility determination can be used to determine whether any two points in space are blocked by an object.

[0134] Figure 13 illustrates a schematic diagram for determining screen visibility. As shown in Figure 13, pixel P8 is any pixel on the screen. Electronic devices typically draw graphical interfaces based on orthogonal projection. The electronic device 100 constructs a ray T1 starting from pixel P8, with the direction of ray T1 perpendicular to the screen. The electronic device 100 then traverses the global attribute structure of the N primitive SDFs in the global acceleration structure and calculates the closest distances from pixel P8 to primitives A and B as (P8, P5) and (P8, P4), respectively. Of all the distances, (P8, P5) is the shortest, indicating that there is no obstruction within the distance (P8, P5). Therefore, the electronic device 100 can move ray T1 forward by (P8, P5) to point P1. The distance from point P1 to the nearest primitive, primitive A, is (P1, P6). There is no obstruction within the distance (P1, P6). The electronic device 100 can move ray T1 forward by a distance (P1, P6) to point P2. The distance from point P2 to the nearest primitive, that is, primitive A, is (P2, P7). There is no obstruction within the distance (P2, P7). The electronic device 100 can move the ray T1 forward by a distance (P2, P7) to reach point P3. The distance from point P3 to the nearest primitive, that is, primitive B, is (P3, P4). There is no obstruction within the distance (P3, P4). The electronic device 100 can move the ray T1 forward by a distance (P3, P4) to reach point P4. The distance from point P4 to primitive B is 0. The electronic device 100 can determine that the ray T1 hits primitive B. Therefore, the electronic device 100 can determine that the primitive visible to pixel point P8 is primitive B.

[0135] It can be understood that the N primitive SDFs may include the primitive SDF of primitive A and the primitive SDF of primitive B.

[0136] In this way, the electronic device 100 can determine the graphic element visible at each pixel on the screen of the electronic device 100 according to the screen visibility determination steps shown in FIG13 . For example, if the graphic element visible at pixel P8 is graphic element B, the electronic device 100 can draw graphic element B at pixel P8.

[0137] In one possible implementation, after the electronic device 100 determines that the image element visible at pixel P8 on the screen of the electronic device 100 is image element B according to the screen visibility determination steps shown in FIG13 , the electronic device 100 further needs to determine whether there is occlusion between image elements A and B. If there is no occlusion, the electronic device 100 can draw image element B at pixel P8. If there is occlusion, the electronic device 100 can determine that the object visible at pixel P8 is the occluding object.

[0138] It is understandable that when the electronic device 100 makes a screen visibility determination and a spatial visibility determination, the electronic device 100 can access the primitive SDFs corresponding to the N primitives according to the global acceleration structure. For example, taking the global acceleration structure of the electronic device 100 based on the screen space division and clustering of the grid tile to obtain N primitive SDFs as an example, when the electronic device 100 makes a screen visibility determination or a spatial visibility determination, the primitive SDF corresponding to the primitive, if the primitive corresponding to the root cluster does not block primitive 1, then the primitive corresponding to the sub-cluster under the root cluster (also referred to as a sub-node) will not block primitive 1. Then the electronic device 100 does not need to access and calculate whether the primitive corresponding to the sub-cluster under the root cluster will block primitive 1. Through this global acceleration structure, the amount of calculation of the electronic device can be reduced.

[0139] FIG14 exemplarily shows a schematic diagram of spatial visibility determination. As shown in FIG14 , point P10 is a position point on the surface of primitive A. Point P14 is a position point on the surface of primitive B. The electronic device 100 constructs a ray T2 with point P10 as the starting point and point P14 as the end point. Then, the electronic device 100 can traverse the global attribute structure of the N primitive SDFs in the global acceleration structure and calculate the closest distances from point P10 to primitive C and primitive B as (P10, P15) and (P10, P14) respectively. Among all distances, the distance (P10, P15) is the shortest, which means that there is no occlusion within the distance (P10, P15). Therefore, the electronic device 100 can move ray T2 forward (P10, P15) to point P11. The distance from point P11 to the nearest primitive, that is, primitive C, is (P11, P16). There is no occlusion within the distance (P11, P16). The electronic device 100 can move ray T2 forward by a distance (P11, P16) to reach point P12. The distance from point P12 to the nearest primitive, namely primitive C, is (P12, P17). There is no obstruction within the distance (P12, P17). The electronic device 100 can move ray T2 forward by a distance (P12, P17) to reach point P13. The distance from point P13 to the nearest primitive, namely primitive B, is (P13, P14). There is no obstruction within the distance (P13, P14). The electronic device 100 can move ray T2 forward by a distance (P13, P14) to reach point P14. The distance from point P14 to primitive B is 0. The electronic device 100 can determine that ray T2 hits primitive B. Since ray T2 does not find other primitives between points P10 and P14, the electronic device 100 can determine that the area between points P10 and P14 is visible.

[0140] Then, based on the screen visibility determination results and the spatial visibility determination results, the electronic device 100 can draw N primitive SDFs, that is, draw the original primitives corresponding to the N primitive SDFs. Since the electronic device 100 has determined the spatial relationship between the N primitives based on the screen visibility determination and the spatial visibility determination, the electronic device 100 can draw all the primitives by calling a single drawcall instruction.

[0141] S705: The electronic device 100 draws the dynamic effects of N graphic primitives SDF.

[0142] The electronic device 100 can read from the drawing data whether N primitives have animation effects. If the i-th primitive in the N primitive SDF has animation effects, the electronic device 100 can calculate the animation effect for the i-th primitive. Then, the electronic device 100 can draw all primitives and their animation effects by calling a single drawcall instruction.

[0143] Optionally, in a possible implementation, the N graphic elements may include graphic element A, which has a shadow effect, and the electronic device 100 needs to calculate the shadow effect of graphic element A. That is, the electronic device 100 needs to confirm whether the color of the pixel point in the space is affected by the shadow effect of graphic element A.

[0144] Figure 15 illustrates an exemplary diagram of electronic device 100 calculating the shadow effect of primitive A. As shown in Figure 15 , primitive A has a shadow animation, and a pixel point P(x, y) exists in space. Electronic device 100 needs to determine whether the shadow effect of primitive A will be reflected on pixel point P. First, electronic device 100 needs to determine the positional relationship between pixel point P and primitive A. There are two possible positional relationships between pixel point P and primitive A: the first is that pixel point P is located inside primitive A; the second is that pixel point P is located outside primitive A. After determining the screen visibility result of pixel point P using an eye tracking algorithm, electronic device 100 can determine that pixel point P is located outside primitive A. Next, electronic device 100 needs to further determine whether pixel point P has a shadow. Electronic device 100 can perform shadow calculations to determine the shadow path and shadow radius Q. As shown in Figure 15 , the shadow path can be the solid line with a single arrow shown in Figure 15 (a). The shadow radius Q can be the dashed line with a double arrow shown in Figure 15 (a). In order to determine the shadow effect of the pixel point P, the electronic device 100 can use the shadow path as a primitive and construct a primitive SDF for the shadow path. Then, the electronic device 100 can calculate the closest distance d1 from the pixel point P to the shadow path. Based on the relationship between the closest distance d1 and the shadow radius Q, the electronic device 100 can determine whether the pixel point has a shadow effect. Specifically, when the closest distance d1 is greater than the shadow radius Q, the electronic device 100 can determine that the shadow concentration of the pixel point P is 0, that is, the pixel point P has no shadow effect. When the closest distance d1 is less than or equal to the shadow radius Q, the electronic device 100 can determine that the pixel point P is located in the shadow area. Then the electronic device 100 needs to further determine the shadow concentration of the pixel point P.

[0145] In one possible implementation, the electronic device 100 can calculate the shadow density of the pixel point P by the distance attenuation method. As shown in (b) of Figure 15, y represents the shadow density value, and the value range of y is greater than or equal to 0 and less than or equal to 1. Among them, when y = 0, it means that the shadow density is the maximum; when y = 1, it means that the shadow density is the minimum. The intermediate value between 0 and 1 constitutes a smooth transition between light and dark. The independent variable x is the ratio of the nearest distance d1 to the shadow radius Q. Since the nearest distance d1 is less than or equal to the shadow radius Q, the value range of the independent variable x is greater than or equal to 0 and less than or equal to 1. The electronic device 100 can substitute the ratio of the nearest distance d1 to the shadow radius Q into the function F(x) = y to determine the value of y. Then, the electronic device 100 can multiply the original primitive color at the pixel point P by y to darken it and obtain the final color value of the pixel point P.

[0146] Optionally, in one possible implementation, the N primitives may include a primitive D, which has a Fresnel refraction effect and a reflection effect. The electronic device 100 needs to calculate the Fresnel refraction effect and the reflection effect of the primitive D. In other words, the electronic device 100 needs to determine whether the color of a pixel in space is affected by the Fresnel refraction effect and the reflection effect of the primitive D.

[0147] Figure 16 exemplarily shows a schematic diagram of the electronic device 100 calculating the Fresnel refraction effect and reflection effect of the image element D. As shown in Figure 16, the image element D has a Fresnel refraction effect and a reflection effect. There is a screen pixel point P30 on the screen of the electronic device 100. The electronic device 100 can emit a ray from the pixel point P30, and through line of sight tracking, it can be determined that the ray can reach the point P31 on the surface of the image element D. The electronic device 100 can determine the reflection direction of the ray at the point P31 based on the Fresnel formula. After determining the reflection direction, the electronic device can query the environment texture based on the reflection direction to obtain the color value 1 of the environment. The environment texture is a panoramic image that can save a 3D environment in a 2D image. The electronic device 100 can determine the color value of any 2D reflection direction based on the environment texture.

[0148] As shown in Figure 16, the electronic device 100 can also calculate the refraction direction of the ray at point P31 based on the Fresnel formula and discovery. Then, the electronic device 100 can continue to move the ray along the refraction direction, and the ray can reach point P32 on the inner surface of the graphic element D. The ray can be refracted to the outside of the graphic element D at point P32 and reach a point on the background image, namely point P33. The electronic device 100 can take out the color value 2 of point P33 from the background image as the refracted color. The background image is a conventional two-dimensional image, and its content can be art materials or real photos, etc., which is not limited in this embodiment of the present application. For example, the background image can be the background control 304 shown in Figure 3. The image specifically corresponding to the background control 304 is the image shown in Figure ① or Figure ② in Figure 3.

[0149] Based on the above reflection and refraction calculations, the final color value 3 at point P30 = color value 1*u1+v1+color value 2*u2+v2. The "*" represents a multiplication sign, and u1, u2, v1, and v2 are linear parameters used to attenuate, amplify, or shift the color. Finally, electronic device 100 determines the color value of pixel P30 as color value 3.

[0150] S706: The electronic device 100 displays the first image frame.

[0151] After the electronic device 100 draws all the graphics elements and their animations by calling a drawcall instruction once, a drawing result is obtained, and the electronic device 100 can display the drawing result and the first image frame.

[0152] In one possible implementation, before the electronic device 100 executes step S702, the electronic device 100 may first determine the graphics elements with motion effects and the graphics elements without motion effects among the N graphics elements described in the drawing data. For graphics elements without motion effects, the electronic device 100 may draw the graphics elements without motion effects based on the traditional rasterization drawing process. For graphics elements with motion effects, the electronic device 100 may draw the graphics elements with motion effects according to the above steps S702-S705. Then, the electronic device 100 may fuse the drawing results drawn based on the traditional rasterization drawing process with the drawing results drawn according to the above steps S702-S705 to obtain a first image frame, and the electronic device may display the first image frame. For example, when the electronic device 100 determines that the i-th graphics element has motion effects, the electronic device 100 may construct the i-th graphics element into a graphics element SDF and a global acceleration structure, and then calculate the motion effects of the graphics element SDF. Finally, the electronic device 100 may draw the i-th graphics element. When the electronic device 100 determines that the jth primitive does not have an animation effect, the electronic device 100 may represent the jth primitive with a triangle, and then perform vertex calculation, rasterization, pixel shading, etc. to obtain a drawing result of the jth primitive.

[0153] In one possible implementation, for graphics elements without motion effects, the electronic device 100 can draw the graphics elements without motion effects through the basic motion effect pipeline shown in Figure 5. The basic motion effect pipeline can draw the graphics elements without motion effects based on the traditional rasterization drawing process. For graphics elements with motion effects, the electronic device 100 can construct the graphics elements with motion effects into graphics element SDF through the graphics element processing module shown in Figure 5. Then, the electronic device 100 draws the graphics elements with motion effects through the high-order motion effect pipeline shown in Figure 5. Finally, the electronic device 100 can fuse the drawing results of the basic motion effect pipeline and the drawing results of the high-order motion effect pipeline through the fusion module shown in Figure 5, and send the fused drawing results to the layer synthesis display module, and the layer synthesis display module sends the fused drawing results, that is, the first image frame, to the display 503.

[0154] It can be understood that in the embodiment of the present application, the screen can also be called a display screen or a display, or a touch screen, a touch screen, etc., and the embodiment of the present application does not limit this.

[0155] Through the graphics drawing method provided by the embodiment of the present application, the electronic device 100 can construct a primitive SDF, store the primitive SDF in a global acceleration structure, and perform screen visibility determination and spatial visibility determination of pixels on the screen by means of line of sight tracking. In this way, the primitive that is ultimately visible in the pixel can be determined. And the motion effect of the primitive can also be calculated by line of sight tracking, and the color value of the pixel can be ultimately determined. Finally, the electronic device 100 can call a drawcall instruction based on the aforementioned calculation results to draw the first image frame. In this way, the drawing effect of the electronic device 100 can be improved and the power consumption of the electronic device can be reduced.

[0156] Exemplarily, as shown in FIG17 , for the same control to be drawn, the electronic device 100 needs to use triangles to represent primitives based on the traditional rasterization drawing process. Then, the electronic device 100 needs to perform vertex calculation, rasterization, pixel shading, etc. for each control, call the drawcall instruction multiple times, and require additional video memory reading and writing to draw the final displayed user interface.

[0157] As shown in Figure 17, the electronic device 100 draws the same control to be drawn using the graphics drawing method provided in an embodiment of the present application, and constructs control 1, control 2, control 3, and control 4 of the control to be drawn into primitive SDFs. The electronic device can then store the constructed multiple primitive SDFs into a global acceleration structure. The electronic device can ultimately determine the visible primitives and color values ​​of each pixel on the screen of the electronic device 100 through line of sight calculation, and then the electronic device 100 can draw the final displayed user interface with a single drawcall.

[0158] As shown in Figure 17, through the graphics drawing method provided by the embodiment of the present application, the electronic device 100 only needs to call the drawcall instruction once to draw the user interface, and no additional video memory reading and writing is required. In this way, the drawing effect of the electronic device 100 can be improved and the power consumption of the electronic device can be reduced.

[0159] In an embodiment of the present application, the first image frame may be the image frame ultimately displayed by the electronic device shown in FIG17 . The N controls may include Control 1, Control 2, Control 3, and Control 4 shown in FIG17 . The first control may be Control 1. The first ray may be Ray T1, and the first pixel may be Pixel P8. The one or more controls may include controls corresponding to Graphics Element A and Graphics Element B. The second control may be the control corresponding to Graphics Element B. The second ray may be Ray T2. The first point may be Point P10, and the second point may be Point P14. The third control may be the control corresponding to Graphics Element A. The fourth control may be the control corresponding to Graphics Element B. The fifth control may be the control corresponding to Graphics Element C. The sixth control may be the control corresponding to Graphics Element A in FIG15 . The second pixel may be Pixel P shown in FIG15 . The seventh control may be the control corresponding to Graphics Element D in FIG16 . The third pixel may be Screen Pixel P30. The third point may be Point P31, and the fourth point may be Point P33.

[0160] The following first introduces an exemplary electronic device 1000 provided in an embodiment of the present application.

[0161] FIG18 is a schematic structural diagram of an electronic device 1000 provided in an embodiment of the present application.

[0162] The following embodiments are described in detail using electronic device 1000 as an example. It should be understood that electronic device 1000 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different component configuration. The various components shown in the figure may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0163] The electronic device 1000 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0164] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 1000. In other embodiments of the present application, the electronic device 1000 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0165] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0166] The controller may be the nerve center and command center of the electronic device 1000. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0167] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0168] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0169] The I2C interface is a bidirectional synchronous serial bus that includes a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C bus interfaces. The processor 110 can be coupled to the touch sensor 180K, charger, flash, camera 193, etc. through different I2C bus interfaces.

[0170] The I2S interface can be used for audio communication.

[0171] The PCM interface can also be used for audio communication to sample, quantize and encode analog signals.

[0172] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface, enabling the function of playing music through Bluetooth headphones.

[0173] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display 194 and the camera 193. MIPI interfaces include the camera serial interface (CSI) and the display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the camera function of the electronic device 1000. The processor 110 and the display 194 communicate via the DSI interface to implement the display function of the electronic device 1000.

[0174] The GPIO interface can be configured via software. The GPIO interface can be configured as either a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display 194, wireless communication module 160, audio module 170, sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0175] The SIM interface can be used to communicate with the SIM card interface 195 to implement the function of transmitting data to the SIM card or reading data in the SIM card.

[0176] USB interface 130 is an interface that complies with USB standards and specifications, and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. USB interface 130 can be used to connect a charger to charge electronic device 1000, or to transfer data between electronic device 1000 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as AR devices.

[0177] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 1000. In other embodiments of the present application, the electronic device 1000 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0178] The charging management module 140 is configured to receive charging input from a charger, which may be a wireless charger or a wired charger.

[0179] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0180] The wireless communication function of the electronic device 1000 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0181] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 1000 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0182] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 1000. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0183] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0184] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 1000. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0185] In some embodiments, antenna 1 of electronic device 1000 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 1000 can communicate with a network and other devices via wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0186] Electronic device 1000 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0187] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 1000 may include one or N display screens 194, where N is a positive integer greater than one.

[0188] The electronic device 1000 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0189] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and color. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0190] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 1000 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0191] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals.

[0192] Video codecs are used to compress or decompress digital video. Electronic device 1000 may support one or more video codecs. This allows electronic device 1000 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0193] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 1000, such as image recognition, face recognition, speech recognition, and text comprehension.

[0194] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 1000. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0195] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 1000 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, applications required for at least one function (such as face recognition function, fingerprint recognition function, mobile payment function, etc.), etc. The data storage area can store data created during the use of the electronic device 1000 (such as face information template data, fingerprint information template, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0196] The electronic device 1000 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0197] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0198] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 1000 can listen to music or listen to hands-free calls through the speaker 170A.

[0199] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the electronic device 1000 receives a call or a voice message, the user can place the receiver 170B close to the ear to hear the voice.

[0200] Microphone 170C, also known as "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 1000 can be provided with at least one microphone 170C. In other embodiments, the electronic device 1000 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 1000 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the source of sound, realize directional recording function, etc.

[0201] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0202] The pressure sensor 180A is used to sense pressure signals and convert the pressure signals into electrical signals.

[0203] The gyro sensor 180B may be used to determine the motion posture of the electronic device 1000 .

[0204] The air pressure sensor 180C is used to measure air pressure.

[0205] The magnetic sensor 180D includes a Hall sensor, and the electronic device 1000 can use the magnetic sensor 180D to detect whether the flip cover is opened or closed.

[0206] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 1000 in various directions (generally three axes).

[0207] The distance sensor 180F is used to measure distance. The electronic device 1000 can measure distance by infrared or laser.

[0208] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector such as a photodiode.

[0209] The ambient light sensor 180L is used to sense the brightness of the ambient light.

[0210] The fingerprint sensor 180H is used to collect fingerprints.

[0211] The temperature sensor 180J is used to detect temperature.

[0212] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 1000, in a location different from that of the display screen 194.

[0213] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 1000 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 1000.

[0214] Motor 191 can generate vibration prompts.

[0215] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0216] The SIM card interface 195 is used to connect a SIM card.

[0217] The electronic device 1000 can execute the steps executed by the electronic device 100 and realize the kinetic energy realized by the electronic device 100. The steps executed and the kinetic energy realized by the electronic device 100 are described above and will not be repeated here.

[0218] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0219] As used in the above embodiments, the term “when…” may be interpreted to mean “if…” or “after…” or “in response to determining…” or “in response to detecting…”, depending on the context. Similarly, the phrases “upon determining…” or “if (stated condition or event) is detected” may be interpreted to mean “if determining…” or “in response to determining…” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

Claims

1. A method for drawing a graph, characterized in that, Including: The electronic device obtains the rendering data of the first image frame, where the first image frame is the image frame to be rendered in the target application, and the rendering data is used to describe N controls included in the first image frame and the attributes of the N controls; The electronic device constructs corresponding N primitive symbol distance fields SDFs based on the N controls, where among the N controls, there is a first control, and among the N primitive SDFs, there is a first primitive SDF corresponding to the first control, and the first primitive SDF is used to describe the positional relationship between any point in space and the first control; The electronic device determines the occlusion relationship of the N controls through line-of-sight tracking based on the N primitive SDFs; The electronic device renders the first image frame based on the occlusion relationship of the N controls; The electronic device displays the first image frame.

2. The method according to claim 1, characterized in that The electronic device constructs corresponding N primitive symbol distance fields SDFs based on the N controls, specifically including: The electronic device constructs an equation for the distance from any point in space to the first control based on the geometric analytical formula corresponding to the geometric shape of the first control, and the distance equation is the first primitive SDF.

3. The method according to claim 1, wherein The electronic device constructs corresponding N primitive symbol distance fields SDFs based on the N controls, specifically including: The electronic device constructs a primitive bounding box enclosing the first control and divides the primitive bounding box into k rectangles, and the center points of the k rectangles represent k coordinate points with the center of the first control as the coordinate origin; The electronic device calculates the distances from the k coordinate points to the surface of the first control to obtain k distance values; The electronic device generates a texture image based on the k distance values, and the texture image is the first primitive SDF.

4. The method according to claim 1, characterized in that, The electronic device constructs corresponding N primitive symbol distance fields SDFs based on the N controls, specifically including: The electronic device constructs a first neural network, where the input of the first neural network is the coordinate of any point in space, and the output of the first neural network is the distance from any point in space to the surface of the first control, and the first neural network is the first primitive SDF.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: The electronic device clusters the N primitive SDFs according to a clustering rule to obtain a global acceleration structure, and the global acceleration structure contains the attributes of the N primitive SDFs; The electronic device stores the global acceleration structure.

6. The method according to claim 5, characterized in that, The occlusion relationship includes screen visibility; the electronic device determines the occlusion relationship of the N controls through line-of-sight tracking based on the N primitive SDFs, specifically including: The electronic device constructs a first ray starting from a first pixel point in the display screen of the electronic device, and the direction of the first ray is perpendicular to the display screen; The electronic device queries the global acceleration structure to determine that there is one or more controls in the direction of the first ray, and the N controls include the one or more controls; The electronic device determines the closest distance between the first pixel point and the one or more controls based on the primitive SDFs corresponding to the one or more controls; The electronic device determines that the second control among the one or more controls is the control visible to the first pixel point based on the shortest distance between the first pixel point and the one or more controls.

7. The method according to claim 5, characterized in that, The occlusion relationship includes spatial visibility; the electronic device determines the occlusion relationship of the N controls through line-of-sight tracking based on the N primitive SDFs, specifically including: The electronic device constructs a second ray with a first point on the third control as the starting point and a second point on the fourth control as the ending point, where the third control and the fourth control are included in the N controls. The electronic device queries the global acceleration structure and determines that there is a fifth control in the direction of the second ray. The electronic device determines the distance between the first point and the fifth control based on the primitive SDF corresponding to the fifth control. Based on the distance between the first point and the fifth control, the electronic device moves the second ray until the electronic device moves the starting point of the second ray to the second point, and the electronic device determines that there is no occlusion between the third control and the fourth control.

8. The method according to claim 7, characterized in that After the electronic device determines the occlusion relationship of the N controls through line-of-sight tracking based on the N primitive SDFs, the method further includes: The electronic device determines that the sixth control among the N controls has a shadow effect from the rendering data, and the electronic device calculates the shadow effect.

9. The method according to claim 8, wherein The electronic device calculates the shadow effect, specifically including: The electronic device determines the shadow radius and shadow path of the sixth control, and constructs a shadow SDF based on the shadow radius and shadow path. The electronic device calculates the shortest distance from the second pixel point to the shadow of the sixth control based on the shadow SDF. The electronic device determines the shadow concentration of the second pixel point based on the shortest distance from the second pixel point to the shadow of the sixth control and the shadow radius. The electronic device determines the color value of the second pixel point based on the shadow concentration.

10. The method according to claim 7, wherein After the electronic device determines the occlusion relationship of the N controls through line-of-sight tracking based on the N primitive SDFs, the method further includes: The electronic device determines that the seventh control among the N controls has a refraction effect and a reflection effect from the rendering data, and the electronic device calculates the refraction effect and the reflection effect.

11. The method according to claim 10, wherein The electronic device calculates the refraction effect and the reflection effect, specifically including: The electronic device constructs a third ray with a third pixel point as the starting point, where the third pixel point is a pixel point in the display screen of the electronic device, and the direction of the third ray is perpendicular to the display screen. The electronic device determines the third point where the third ray reaches the surface of the seventh control based on line-of-sight tracking. The electronic device determines the reflection direction and refraction direction of the third ray at the third point. The electronic device queries the environmental texture based on the reflection direction and determines that the color value of the environment is the first color value. The electronic device moves the third ray based on the refraction direction, and the third ray reaches the fourth point in the background image. The electronic device extracts the second color value of the fourth point in the background image; The electronic device determines the color value of the third pixel based on the first color value and the second color value.

12. An electronic device, characterized in that, It includes a display screen, one or more processors, and one or more memories; wherein, the display screen, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the method described in any one of claims 1-11 is performed.

13. A chip system, the chip system is applied to an electronic device, the chip system includes one or more processors, characterized in that, The processor is used to call computer instructions to cause the execution of the method described in any one of claims 1-11.

14. A computer-readable storage medium, comprising instructions, characterized in that, When the instruction runs on the electronic device, the method described in any one of claims 1-11 is performed.

Images