Rendering optimization method, electronic equipment and computer readable storage medium

Abstract

The invention provides a rendering optimization method, electronic equipment and a computer readable storage medium, belongs to the technical field of electronic equipment, is applied to the electronic equipment, and comprises the following steps: determining first polygonal mesh data corresponding to an object model in a scene, the first polygonal mesh data being used for describing the object model; creating a bounding box corresponding to the object model based on the first polygon mesh data; determining a pixel range covered by the bounding box when the bounding box is projected to a screen of the electronic equipment, obtaining a relationship between the number of pixels covered by the bounding box and the number of pixels of the screen, and determining a distance between an observation view angle in the scene and the object model; determining a multi-level of detail (LOD) level corresponding to the object model based on the distance and the relationship, and updating the first polygonal mesh data into second polygonal mesh data based on the LOD level; and rendering the object model based on the second polygon mesh data. According to the method, the calculation consumption of light tracking can be effectively reduced, and vivid image quality is generated.

Description

Rendering optimization method, electronic device and computer-readable storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 4, 2022, with application number 202210934834.4 and application name “Rendering Optimization Method, Electronic Device and Computer-readable Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of electronic technology, and in particular to a rendering optimization method, an electronic device, and a computer-readable storage medium. Background Art

[0003] With the advancement of computer technology, more and more applications, such as games and videos, require more realistic graphics, specifically special effects scenes. These scenes, such as virtual simulation, virtual reality, and augmented reality, utilize graphics rendering technology to enhance the realism of images and provide a better user experience. Therefore, improving image realism has become a major concern in the industry.

[0004] Ray tracing technology, based on the laws of physics, has garnered significant attention due to its ability to produce rendering results that approximate the real world. As hardware-accelerated real-time ray tracing technology becomes increasingly widespread in personal computers (PCs) and consoles, Vulkan-based real-time ray tracing for mobile devices is now supported by numerous mobile chip products. However, even with hardware acceleration, ray tracing remains a heavy-load technology on mobile devices. It still requires a significant amount of computation, significantly increasing chip power consumption and affecting the frame rate of electronic devices.

[0005] Summary of the Invention

[0006] In view of this, the present invention provides a rendering optimization method, an electronic device and a computer-readable storage medium, which can reduce the amount of calculation during the image rendering process, reduce the chip power consumption load while maintaining the special effects running frame rate.

[0007] Some embodiments of the present application provide a rendering optimization method. The following describes the present application from multiple aspects, and the implementation methods and beneficial effects of the following aspects can be referenced to each other.

[0008] In a first aspect, the present invention provides a rendering optimization method, which is applied to an electronic device, and the method includes: determining first polygonal mesh data corresponding to an object model in a scene, the first polygonal mesh data being used to describe the object model; creating a bounding box corresponding to the object model based on the first polygonal mesh data; determining the pixel range covered by the bounding box when it is projected onto the screen of the electronic device, and obtaining the relationship between the number of pixels covered by the bounding box and the number of pixels on the screen, and determining the distance between the observation angle in the scene and the object model; determining the multi-level of detail LOD level corresponding to the object model based on the distance and the relationship, and updating the first polygonal mesh data to second polygonal mesh data based on the LOD level; and rendering the object model based on the second polygonal mesh data.

[0009] According to the rendering optimization method of the embodiment of the present application, the LOD level of the model's simplification is determined by calculating the proportion of the model's visible range on the screen and observing the viewing angle and the distance of the model, thereby realizing dynamic adjustment of the LOD level. While maintaining the geometric shape of the model, the number of triangular meshes constituting the model is reduced, which can effectively reduce the computational consumption of ray tracing and more efficiently generate realistic image quality.

[0010] As an embodiment of the first aspect of the present application, determining the LOD level corresponding to the object model based on distance and relationship includes: performing weighted sum calculations on the distance and relationship respectively to obtain the sum result, and determining the LOD level based on the sum result.

[0011] As an embodiment of the first aspect of the present application, determining the distance between the observation perspective and the object model in the scene includes: obtaining the first world coordinates of the observation perspective in the world space, and the second world coordinates of the object model in the world space, and determining the distance between the observation perspective and the object model based on the first world coordinates and the second world coordinates.

[0012] As an embodiment of the first aspect of the present application, updating first polygon mesh information to second polygon mesh information based on an LOD level includes: using a polygon mesh reduction algorithm to remove some polygon mesh data from the first polygon mesh information based on the LOD level to obtain the second polygon mesh data. The LOD level obtained by this algorithm can be dynamically adjusted based on the real scene to minimize the amount of computation while ensuring the realism of the image.

[0013] As an embodiment of the first aspect of the present application, rendering an object model based on second polygonal mesh data includes: when the electronic device determines that the rendering method corresponding to the object model is ray tracing rendering; obtaining second polygonal mesh information; based on the second polygonal mesh information, using a ray tracing shader to calculate the intersection of the light and the object model in the scene, and shading the object model based on the intersection.

[0014] As an embodiment of the first aspect of the present application, the method also includes: when the electronic device determines that the rendering mode corresponding to the object model is normal rendering; obtaining the first polygonal mesh information corresponding to the object model in the scene, and using a rasterized shader to color the object model.

[0015] As an embodiment of the first aspect of the present application, a bounding box corresponding to the object model is created based on the first polygonal mesh information, including: determining the maximum and minimum values ​​of the object model in the X direction, Y direction and Z direction in its own coordinates based on the first polygonal mesh information; and determining the bounding box based on the maximum and minimum values.

[0016] As an embodiment of the first aspect of the present application, the relationship between the number of pixels covered by the bounding box and the number of pixels on the screen is: the ratio of the number of pixels covered by the bounding box to the number of pixels on the screen.

[0017] As an embodiment of the first aspect of the present application, the weight of the distance is set between 0.1-0.5, and the weight range of the ratio is between 0.5-0.9.

[0018] As an embodiment of the first aspect of the present application, the weight of the distance is set to 0.3, and the weight of the ratio is set to 0.7. This ratio is more conducive to minimizing the amount of calculation while ensuring the realism of the picture.

[0019] As an embodiment of the first aspect of the present application, the first polygon mesh information includes the number of polygons corresponding to the object model that has not been simplified at the LOD level, the vertex coordinates and vertex indexes of the polygons.

[0020] As an embodiment of the first aspect of the present application, the second polygon mesh information includes the number of polygons corresponding to the simplified object model based on the LOD level, the vertex coordinates and vertex indexes of the polygons.

[0021] As an embodiment of the first aspect of the present application, the polygon is a triangle.

[0022] In a second aspect, the present application provides an electronic device, comprising:

[0023] A mesh data acquisition module, configured to determine first polygonal mesh data corresponding to an object model in a scene, wherein the first polygonal mesh data is used to describe the object model;

[0024] A bounding box creation module, configured to create a bounding box corresponding to the object model based on the first polygonal mesh data;

[0025] a ratio determination module, configured to determine a pixel range covered by the bounding box when projected onto a screen of the electronic device, obtain a relationship between the number of pixels covered by the bounding box and the number of pixels on the screen, and determine a distance between an observation angle and an object model in the scene;

[0026] An LOD calculation module is configured to determine a level of detail (LOD) corresponding to the object model based on the distance and the relationship, and update the first polygon mesh data to the second polygon mesh data based on the LOD level;

[0027] The rendering module is used to render the object model based on the second polygonal mesh data.

[0028] According to the electronic device of the embodiment of the present application, the simplification level LOD of the model is determined by calculating the proportion of the model in the visible range of the screen and observing the viewing angle and the distance of the model, thereby realizing dynamic adjustment of the LOD level. While maintaining the geometric shape of the model, the number of triangular meshes constituting the model is reduced, which can effectively reduce the computational consumption of ray tracing and more efficiently generate realistic image quality.

[0029] As an embodiment of the second aspect of the present application, the LOD calculation module is further configured to perform a weighted summation of the distance and the relationship to obtain a summation result, and determine the LOD level based on the summation result. The LOD level obtained by this algorithm can be dynamically adjusted according to the real scene to minimize the amount of calculation while ensuring the realism of the image.

[0030] As an embodiment of the second aspect of the present application, the ratio determination module is further configured to: obtain a first world coordinate of the observation angle in the world space, and a second world coordinate of the object model in the world space,

[0031] The distance between the viewing angle and the object model is determined based on the first world coordinate and the second world coordinate.

[0032] As an embodiment of the second aspect of the present application, the LOD calculation module is used to:

[0033] According to the LOD level, a polygon mesh reduction algorithm is used to remove part of the polygon mesh data from the first polygon mesh information to obtain the second polygon mesh data.

[0034] As an embodiment of the second aspect of the present application, the rendering module is configured to:

[0035] When the rendering module determines that the rendering method corresponding to the object model is ray tracing rendering;

[0036] Get the second polygon mesh information;

[0037] Based on the second polygonal mesh information, a ray tracing shader is used to calculate the intersection point between the light and the object model in the scene, and the object model is shaded based on the intersection point.

[0038] As an embodiment of the second aspect of the present application, the rendering module is further configured to:

[0039] When the rendering module determines that the rendering mode corresponding to the object model is normal rendering;

[0040] The first polygon mesh information corresponding to the object model in the scene is obtained, and the object model is shaded using a rasterized shader.

[0041] As an embodiment of the second aspect of the present application, the bounding box creation module is used to: determine the maximum and minimum values ​​of the object model in the X direction, Y direction, and Z direction in its own coordinates based on the first polygonal mesh information;

[0042] Determines the bounding box based on the maximum and minimum values.

[0043] As an embodiment of the second aspect of the present application, the relationship between the number of pixels covered by the bounding box and the number of pixels on the screen is the ratio of the number of pixels covered by the bounding box to the number of pixels on the screen.

[0044] As an embodiment of the second aspect of the present application, the weight of the distance is set between 0.1-0.5, and the weight range of the ratio is between 0.5-0.9.

[0045] As an embodiment of the second aspect of the present application, the weight of the distance is set to 0.3, and the weight of the ratio is set to 0.7. This ratio is more conducive to minimizing the amount of calculation while ensuring the realism of the picture.

[0046] As an embodiment of the second aspect of the present application, the first polygon mesh information includes the number of polygons corresponding to the object model that has not been simplified at the LOD level, the vertex coordinates and vertex indexes of the polygons.

[0047] As an embodiment of the second aspect of the present application, the second polygon mesh information includes the number of polygons corresponding to the simplified object model based on the LOD level, the vertex coordinates and vertex indexes of the polygons.

[0048] As an embodiment of the second aspect of the present application, the polygon is a triangle.

[0049] In a third aspect, the present application also provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the device, and a processor for executing instructions so that the electronic device executes any one of the methods in the technical solution of the first aspect.

[0050] In a fourth aspect, the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes any one of the methods in the technical solution of the first aspect.

[0051] In a fifth aspect, the present application also provides a computer program product comprising instructions, which, when run on an electronic device, enables a processor to execute any one of the methods in the technical solution of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] FIG1 is a scene diagram of ray tracing according to an embodiment of the present application;

[0053] FIG2 is a schematic diagram of a grid structure change of an object model according to an embodiment of the present application;

[0054] FIG3 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

[0055] FIG4 is a block diagram of the software structure of an electronic device according to an embodiment of the present application;

[0056] FIG5 is a flowchart of a rendering optimization method according to an embodiment of the present application;

[0057] FIG6 is a schematic diagram of a triangular mesh constituting a sphere according to an embodiment of the present application;

[0058] FIG7 is a schematic diagram of a process for creating a bounding box according to an embodiment of the present application;

[0059] FIG8 is a schematic diagram of a bounding box projection according to an embodiment of the present application;

[0060] FIG9 is a schematic diagram of a triangle surface reduction algorithm according to an embodiment of the present application;

[0061] FIG10 is a schematic diagram of another triangle reduction algorithm according to an embodiment of the present application;

[0062] FIG11 is a simplified schematic diagram of the structure of a sphere according to an embodiment of the present application;

[0063] FIG12 is a scene diagram of intersection calculation based on ray tracing according to one embodiment of the present application;

[0064] FIG13 is another flowchart of a rendering optimization method according to an embodiment of the present application;

[0065] FIG14 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

[0066] FIG15 is a block diagram of a system on a chip according to some embodiments of the present application. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0068] In order to facilitate the understanding of the technical solution of this application, the technical problem to be solved by this application is first described below.

[0069] In order to facilitate the understanding of the technical solution of this application, the terms appearing in this application are first explained.

[0070] Multiple levels of detail (LOD) can display the details of the object model in a hierarchical manner in the world space when the distance between the camera (observation angle) and the object model (object in the scene) is different. When the observation angle is close to the object model, the LOD level with high object fineness is displayed; when the observation angle is far from the object model, the LOD level with low object fineness is displayed, thereby saving the performance overhead of the electronic equipment.

[0071] Tile: A tile is the smallest unit of planar structure in two-dimensional or three-dimensional space, that is, a plane composed of polygons. In rendering, a model in space needs to be divided into countless tiny planes. The LOD model level determines the number of these tiles.

[0072] The visual plane image refers to a plane image corresponding to the screen of the electronic device, that is, a two-dimensional (2D) plane. Pixels are correspondingly arranged on the visual plane image, that is, the rendering result of the visual plane image is presented in pixels.

[0073] Rasterization: Rasterization is the process of dividing the object model in a 3D scene into polygons, such as triangles, and converting the three-dimensional coordinates of the triangle vertices into a two-dimensional viewing plane image so that the object model corresponds to some pixels on the viewing plane image. That is, the mathematical description of the object model and the color information related to the object model are converted into pixels for corresponding positions on the screen and the colors used to fill the pixels to obtain a raster image.

[0074] Ray tracing: Ray tracing, also known as ray tracking or ray tracing, is a method for presenting three-dimensional (3D) images on a two-dimensional (2D) screen. It traces light rays interacting with optical surfaces to create a model of the object along their path. The principle is that when light strikes a surface, it typically undergoes one or more of the following phenomena: absorption, reflection, refraction, and diffuse reflection. These phenomena alter the spectrum of light, meaning that light undergoes a series of attenuations before entering the human eye. Ray tracing calculates the attenuation of light rays before entering the eye, thereby simulating the realism of virtual scenes.

[0075] Ray tracing can include forward ray tracing and backward ray tracing. Forward ray tracing refers to the propagation of light in a straight line from the light source. During the propagation process in the scene, it will interact with the surface of objects. Some light is absorbed by objects, some is projected through objects or is reflected, and some light may even reflect multiple times between multiple objects before finally entering the human eye (observation angle).

[0076] Reverse ray tracing involves emitting light from the viewing angle (e.g., the camera in the scene). After the light passes through the viewing plane corresponding to the screen, it interacts with the surfaces of objects in the scene. Some light is absorbed by the object, some passes through the object or is reflected, and some light is even reflected multiple times between multiple objects. The color information in these light paths is calculated and accumulated and rendered into the corresponding pixels to obtain the final rendering target.

[0077] In order to facilitate the understanding of the technical solution of this application, the technical problem to be solved by this application is first described.

[0078] The rendering optimization method of the embodiment of the present application is described below with reference to the accompanying drawings.

[0079] Referring to Figure 1, Figure 1 shows a scene diagram of ray tracing in an embodiment of the present application. As shown in Figure 1, the scene includes a camera 10 (observation angle), a visual plane image 20, an object model 30, a light source 40, a light ray 51, a light ray 52, and a light ray 53 located in the application scene. Among them, the visual plane image 20 corresponds to the screen of the device. The rendering result of the visual plane image 20 is in pixels, and the value of each pixel is based on the physical phenomena such as reflection, refraction, diffuse reflection, and light absorption that may occur on the surface of the object after the light passes through the pixel and collides with the object model in the scene, depending on the material of the object. Finally, the color value corresponding to the path of the light is calculated (this process is the process of ray tracing). As shown in Figure 1, light 51, light 52, and light 53 all pass through the pixels in the visual plane image 20. Light ray 51 intersects object model 30, and the reflected light directly reaches the light source, indicating that the light source can directly illuminate the point. Light ray 52 intersects the ground, and the reflected light collides with object model 30, indicating that the intersection with the ground is not directly illuminated by light source 40, thus forming a shadow. Light ray 53 does not intersect any object. The paths of these rays are ultimately calculated to obtain corresponding color values. The color value corresponding to each ray is output to the pixel of the viewing plane image 20 corresponding to that ray, resulting in the rendered target image. The final rendered image is then displayed on the display of the electronic device.

[0080] In some embodiments, the number of pixels corresponding to the viewing plane image 20 is equal to or proportional to the number of pixels corresponding to the screen. When the number of pixels corresponding to the viewing plane image and the screen is proportional, when the image is displayed on the screen, it can be appropriately enlarged or reduced according to the proportional relationship to fit the pixels corresponding to the screen.

[0081] The ray tracing described above has a significant impact on improving the realism of rendered images, for example, in terms of covering shadows, reflections, ambient occlusion, translucent materials, and global illumination. The algorithms used in these aspects require a large number of calculations of the intersections of ray vectors with the three-dimensional scene. The objects in the scene are composed of many patches, which constitute the spatial structure of the object model. Among them, a pixel in the viewing plane image (corresponding to the screen) can correspond to one or more patches, that is, the path of the light passing through the pixel may pass through one or more patches, and the color value corresponding to the pixel is calculated based on these patches (the path corresponding to the light). When there are many object models in the three-dimensional scene, or the spatial structure of the object model is complex (the number of patches is large), the number of calculated intersections between the ray and the object surface increases, and the computational efficiency increases with the increase in the amount and complexity of the three-dimensional scene spatial structure data. In real life, when the viewing angle is far from the object, many details are not very clear to the naked eye. At this time, if the spatial structure data at close range is used to calculate the intersection of light and objects, the calculation amount is large, but it does not achieve a good visual effect. As a result, the device has a large amount of calculation, low calculation efficiency, and even affects the device's image output frame rate.

[0082] Based on the above-mentioned problems, this application provides a rendering optimization method, which dynamically changes the data volume of the ray tracing acceleration structure according to the changes in the scene, optimizes the efficiency of the ray intersection calculation between the light and the object model in the scene, and improves the ray tracing calculation performance.

[0083] In conjunction with the scene shown in Figure 1, the rendering optimization method of the embodiment of the present application is described. In some embodiments, the object model can be a model of the sky, clouds, people, houses, trees, objects, etc. As shown in Figure 2, Figure 2 shows a schematic diagram of the grid structure change of the object model of the embodiment of the present application. This application is described by taking the object model as a sphere as an example. During the three-dimensional scene update process, the polygonal mesh data corresponding to the object model in the scene is determined, such as the grid structure diagram of the sphere shown in (a) of Figure 2, that is, the surface of the object model is divided into patches. Taking the patch as a triangular structure as an example, a triangular mesh is divided on the surface of the object model. The division process can refer to the division method of the prior art. After division, triangular mesh data (corresponding to the first polygonal mesh data) for describing the object model is obtained. The triangular mesh data may include the number of triangles, triangle vertex coordinates and vertex index data, etc. The triangle vertex coordinates are converted to world space (word space) through a model matrix to obtain the world coordinates of the vertex (world coordinates). Specifically, the model matrix can be a pre-set matrix. All object models to be rendered in the scene can be in the same world space (word space) converted by the same model matrix. The conversion process can refer to the prior art. Based on the world coordinates of the object model, a bounding box is established for the geometric body, such as an axis-aligned bounding box (AABB), an oriented bounding box (OBB), or a spherical bounding box, etc., which encloses the geometric body. The bounding box is rendered into the viewing plane image corresponding to the screen, and the pixel range covered by the bounding box in the viewing plane image is obtained. The proportion of the number of pixels covered by the bounding box in the number of pixels in the viewing plane image is calculated, that is, the ratio (relationship) of the number of pixels covered by the bounding box to the number of pixels on the screen is determined. The distance between the camera (observation angle) and the object model in the world space is determined, and the multi-level detail (LOD) level corresponding to the object model is determined based on the distance and the ratio. The polygon mesh data is updated according to the LOD level to obtain the triangle mesh data (second polygon mesh data) corresponding to the LOD level. Its structure can be referred to as shown in (b) of Figure 2, which is the mesh structure 60 of the simplified object model. The electronic device then performs ray tracing on the object model based on the updated triangular mesh data, that is, as shown in (b) in Figure 2, calculates the intersection of the light and the triangular mesh 61 in the model 60, calculates the color value based on the intersection, and outputs it to the corresponding pixel to obtain the rendering target.

[0084] The rendering optimization method of the present application embodiment determines the degree of model simplification by calculating the percentage of the visible area of ​​the object model on the screen during the 3D scene update process. The method then performs model simplification on the object model in parallel, reducing the number of triangle meshes that make up the model while maintaining its geometric shape. This optimizes ray intersection efficiency when the object model is far away from the viewing angle, reduces the amount of computation, and improves ray tracing performance.

[0085] The electronic device of the embodiment of the present application can be a mobile phone, a tablet computer, a laptop computer, a PC, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless electronic device in industrial control, a wireless electronic device in self-driving, a wireless electronic device in remote medical surgery, a wireless electronic device in a smart grid, a wireless electronic device in transportation safety, a wireless electronic device in a smart city, a wireless electronic device in a smart home, etc. The electronic device can be a device running HarmonyOS, Android, IOS, Windows, or other systems. Applications that need to render 3D scenes to obtain two-dimensional images, such as games, videos, and other applications, can be run in the electronic device.

[0086] The rendering optimization method of the embodiment of the present application is described below in conjunction with the specific structure of the electronic device.

[0087] Figure 3 shows a schematic diagram of the structure of the electronic device 100. As shown in Figure 3, the electronic device 300 may include a processor 310, a memory 320, a display 330, and a communication unit 340. These components may be connected via a communication bus to enable communication between the components.

[0088] It should be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 300. In other embodiments of the present application, the electronic device 300 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0089] The processor 310 may include one or more processing units. In some embodiments, as shown in FIG3 , the processor 310 may include a central processing unit (CPU) and a graphics processing unit (GPU).

[0090] In other embodiments, the processor may further include an application processor (AP), a modem processor, an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. The different processing units may be independent devices or integrated into one or more processors.

[0091] The processor 310 can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and execution.

[0092] The processor 310 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 310 is a high-speed cache memory. The memory may store instructions or data that have just been used or are recycled by the processor 310. If the processor 310 needs to use the instruction or data again, it may be directly called from the memory. For example, during real-time rendering, data corresponding to the scene to be rendered, such as vertex data, coordinate data, material, lighting, and other data corresponding to the geometric bodies of characters, houses, and objects, may be stored, thereby avoiding repeated access, reducing the waiting time of the processor 310, and thus improving the efficiency of the system.

[0093] In one embodiment of the present application, CPU 311 can run an operating system and various application programs, which can be graphics applications such as games and video players. The operating system provides a system graphics library interface. The application program uses the system graphics library interface and the driver provided by the operating system to generate an instruction stream for rendering graphics or image frames, as well as the required rendering data. The instruction stream includes a series of instructions, which are generally calls to the system graphics library interface.

[0094] GPU 312 is used to receive the graphics command stream sent by CPU 311 and generate a view plane image with some pixels covered by bounding boxes through the Graphics Rendering Pipeline. In combination with ray tracing technology, it obtains the trajectory of light in world space, that is, the surface patches where the light intersects. Finally, it calculates the color value based on the light trajectory and outputs it to the corresponding pixel to obtain the rendering target. The rendering target is then displayed on display screen 330 through processes such as layer compositing in the operating system. The graphics rendering pipeline (also known as the rendering pipeline), which can be called a rendering pipeline, pixel pipeline, or pixel pipeline, is a parallel processing unit within the graphics processor for processing graphics signals.

[0095] The main function of the graphics rendering pipeline is to generate (or render) a two-dimensional image based on a given virtual camera, three-dimensional objects, and light sources. A GPU can include multiple rendering pipelines, which can independently and concurrently process graphics signals. A rendering pipeline performs a series of operations during the rendering of graphics or image frames. For example, the operations of a rendering pipeline can be roughly categorized as: vertex processing, primitive processing, rasterization, fragment processing, and so on.

[0096] In an embodiment of the present application, the communication unit 340 may include antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, a modem processor and a baseband processor, etc., and communication with other electronic devices can be achieved through the cooperation of various components.

[0097] In some embodiments, the electronic device 300 can be connected to a cloud server via a communication unit to obtain data of objects to be rendered in a 3D scene, or can communicate with other electronic devices to obtain data of objects to be rendered, etc.

[0098] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0099] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0100] In one embodiment of the present application, when CPU311 submits the set parameters and vertex data content to GPU312, GPU312 completes the rendering of the object model in the scene based on this data, obtains the rendering target, and stores the rendering target in the frame buffer. The video controller reads the data in the frame buffer frame by frame, and finally displays it on the display screen 330 after data conversion. The user can see the special effects rendering image through the display screen 330.

[0101] The memory 320 may include an internal memory and an external memory. The internal memory may be used to store computer executable program code, which includes instructions. The internal memory may include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function (such as a game, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 300 (such as rendering target image data obtained by rendering by the CPU and GPU, etc.). In addition, the internal memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0102] In some embodiments, the processor 310 executes instructions stored in the internal memory, for example, an instruction stream for rendering graphics or image frames, so that the GPU processes the data to be rendered according to the instruction stream.

[0103] The software system of the electronic device 300 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 300.

[0104] FIG4 is a block diagram of the software structure of the electronic device 300 according to an embodiment of the present invention.

[0105] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0106] The application layer can include a series of application packages.

[0107] As shown in FIG4 , the application package may include applications such as games, videos, gallery, calendar, calls, maps, navigation, WLAN, Bluetooth, music, and short messages.

[0108] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0109] As shown in FIG4 , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0110] The window manager is used to manage window programs. The window manager can obtain the display screen size so that the final rendered image is displayed on the display screen according to the corresponding ratio.

[0111] Content providers are used to store and retrieve data and make it accessible to applications. The data may include data corresponding to object models in the game scene, such as video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0112] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0113] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0114] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0115] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0116] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0117] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0118] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0119] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0120] The 3D graphics processing library is used to implement the image rendering process based on the graphics rendering pipeline, obtain the rendering target, and further perform layer processing on the rendering target.

[0121] A 2D graphics engine is a drawing engine for 2D drawings.

[0122] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0123] The above introduces the specific structure of the electronic device to which the rendering optimization method provided in the embodiment of the present application is applied. Below, taking the electronic device being a mobile phone as an example, the process of the rendering optimization method provided in the embodiment of the present application is introduced in detail.

[0124] Refer to Figure 5, which is a flow chart of a rendering method provided in an embodiment of the present application. The flow chart shows six steps. As shown in Figure 5, the rendering method includes S501-S506. The method can be applied to an electronic device having a hardware structure as shown in Figure 3 and a software structure as shown in Figure 4. Or an electronic device with a similar hardware structure and software structure having more or fewer components than shown, or combining certain components, or splitting certain components, or a different arrangement of components.

[0125] S501: Obtain an object model to be rendered in the scene.

[0126] In an embodiment of the present application, the geometric body corresponding to the object to be rendered in the scene may include a model in a 2D or 3D scene and attribute information assigned to the model. For example, the geometric body model may be a model of the sky, white clouds, people, buildings, flowers, trees, objects, etc. These models may be drawn using a 3D modeling or 2D modeling application. The model in the scene has its own coordinates, and the attribute information of the model may include attribute information such as the model's color, material, light intensity, and light direction.

[0127] It can be understood that, for a mobile phone, the geometric model corresponding to the object to be rendered is composed of some related data. Therefore, the geometric model of this application can also be understood as the data to be rendered.

[0128] It should be understood that these data to be rendered can be stored in the mobile phone's memory during the process of downloading and installing the application on the mobile phone, or can be stored in the mobile phone's memory during the process of running the application. For example, in an online game, the mobile phone can communicate with the server in real time and obtain the data to be rendered in the game scene, so that the mobile phone can render the image based on the data to be rendered.

[0129] That is to say, the mobile phone can obtain the data to be rendered by reading local data, or it can obtain the data to be rendered from other devices in real time through communication connections with other devices. This application does not limit the specific method of obtaining the data to be rendered.

[0130] In some embodiments, the mobile phone can also provide the position and direction in the world space at which the naked eye should look at objects in the world space, that is, set the camera position and direction, and determine the projection method and projection matrix, where the projection method can be orthographic projection, perspective projection, matrix coordinates, camera position coordinates, screen size, viewing plane image size, etc.

[0131] In an embodiment of the present application, the process described in S501 can be executed by the CPU shown in FIG2 , and the corresponding data is stored. When the CPU issues a rendering instruction to the GPU, the GPU, upon receiving the instruction, executes the graphics rendering pipeline program based on the data, and completes the rendering of the object model in the scene using processes such as ray tracing technology. This is the execution of S502-S506.

[0132] S502: Create a bounding box corresponding to the object model.

[0133] In the embodiment of the present application, the bounding box may be an axis aligned bounding box (AABB), an oriented bounding box (OBB), or a spherical bounding box.

[0134] As shown in Figure 6, Figure 6 shows a schematic diagram of the triangular mesh that constitutes a sphere. As shown in Figure 6, the object model seen by the human eye is a sphere. In an electronic device, the sphere is composed of many facets. The facet is the smallest plane unit that constitutes the object model. It can be a triangle, a quadrilateral, or other shapes, usually a triangular structure. In this application, the facet is a triangular structure as an example for explanation. In other words, the object model is composed of many triangular meshes. The fixed-point coordinates and quantity information of these triangular meshes (first polygonal mesh data) can be obtained by the CPU when the scene is loaded, and the initial triangular mesh vertex coordinates of the object model in the scene are used to construct a vertex coordinate buffer, and the index data of the triangular mesh are used to construct an index buffer. These two buffers are used to create the acceleration structure (Acceleration Structure) of Vulkan ray tracing.

[0135] The following takes a sphere as an object model and creates an AABB bounding box for the sphere as an example to illustrate how to determine the vertex data of the bounding box.

[0136] Refer to Figure 7, which is a flow chart of bounding box creation in an embodiment of the present application. First, based on the sphere's own coordinates and the vertex coordinates of the triangle mesh, the maximum and minimum values ​​in the X, Y, and Z directions are calculated. Taking the symmetry of the vertices of the geometric body as an example, the equation of the symmetrical geometric body is as follows: x≤|a|; y≤|b|; z≤|c|;

[0137] As shown in Figure 7(a), based on the above geometric equation, the minimum value in the x-direction of the geometric body's own coordinates is calculated to be -a, the maximum value is a, the minimum value in the y-direction is -b, the maximum value is b, and the minimum value in the z-direction is -c, the maximum value is c. Based on the maximum and minimum values ​​of the geometric body in the x-axis, y-axis, and z-axis directions respectively, the size of the AABB bounding box is determined. As shown in Figure 7(b), the coordinates of the eight vertices of the bounding box are calculated as: (a,b,c), (a,b,-c), (a,-b,c), (a,-b,-c), (-a,b,-c), (-a,b,-c), (-a,-b,c), (-a,-b,-c), (-a,-b,-c), (-a,-b,-c). In other words, the bounding box surrounds the sphere. After determining the vertex coordinates of the bounding box, the spatial matrix M corresponding to the geometry in world space is used to apply M's spatial transformation to each vertex coordinate. The resulting world coordinates for the eight vertices of the bounding box are: (a,b,c)×M, (a,b,-c)×M, (a,-b,c)×M, (a,-b,-c)×M, (-a,b,c)×M, (-a,b,-c)×M, (-a,-b,c)×M, (-a,-b,c)×M, (-a,-b,-c)×M. After obtaining the vertex coordinates, the phone determines the vertex index. For example, to construct a triangle on a face of the bounding box, each face consists of four vertices, three of which form a triangle. Two triangles can be constructed from a face, and two points in the two triangles are reused. Therefore, vertex indexes are used to eliminate duplicate vertex data. Vertex buffers and index buffers are also constructed for the vertex data of the bounding box. This allows the GPU to retrieve vertex data from the cache when receiving rendering instructions to render the bounding box. In the embodiments of the present application, the specific calculation process for the vertex coordinate transformation of the bounding box can be referred to the prior art, and the present application will not describe it in detail.

[0138] S503 : Determine the pixel range covered by the bounding box when projected onto the screen of the electronic device, and obtain a ratio of the number of pixels covered by the bounding box to the number of pixels on the screen.

[0139] In an embodiment of the present application, the screen of an electronic device corresponds to a visual plane image. When the width and height of the visual plane image are equal to the width and height of the screen, and the resolutions of the two are the same, that is, the number of pixels of the two are equal, then the ratio of the visual plane image to the screen size is 1:1. When the electronic device displays the rendered rendering target through the screen, there is no need to scale the visual plane image. In some embodiments, the size of the visual plane image is in a certain ratio to the screen size, and the number of pixels of the two is also in a certain ratio accordingly. In this case, when displaying the rendering target, the rendered visual plane image needs to be enlarged or reduced to reasonably display the corresponding image. Since the number of pixels of the visual plane image is in a certain ratio to the number of pixels of the screen, for example, the ratio is r1, when the bounding box is projected into the visual plane image, based on the pixel range of the bounding box covering the visual plane image, the pixel range of the bounding box covering the screen can also be obtained. In the following embodiment, the pixel range covered by the bounding box after projection is first explained.

[0140] Referring to FIG8 , FIG8 shows a schematic diagram of a bounding box projection according to an embodiment of the present application. As shown in FIG8 (a), a bounding box 830 exists in world space. The process of obtaining the bounding box 830 can refer to the process of obtaining the bounding box in S502. After obtaining the camera position, the GPU adds camera coordinates (camera coordinates) in world space, i.e., the coordinates of the camera origin, to simulate the viewing angle 810 of observing the bounding box in real space, including the position in world space and the direction of observing the object. The camera coordinates can be obtained by multiplying the world coordinates of the bounding box with the view matrix. The bounding box is then observed using the camera coordinates as the viewing angle. After determining the camera coordinates, the camera coordinates are converted to screen coordinates to obtain a projected view plane image 820. The projection can use orthographic projection or perspective projection. As shown in FIG8 , using orthographic projection as an example, the bounding box 830 is projected onto the view plane image 820 in equal proportions. As shown in Figure 8(b), the dimensions of the viewing plane image 820 are width (W) × height (Height), expressed in pixels as a size (G): 9 × 8. The number of pixels covered by the bounding box after projection (H) is 5 × 5, which gives the pixel range covered by the bounding box after projection onto the viewing plane image. Finally, the ratio of the number of pixels covered by the bounding box to the number of pixels in the viewing plane image is determined as: H / G = 55 / 72. The specific steps are as follows: 1. Create a viewing plane image (texture image) with width × height of W and H, respectively, and initialize all pixels to 0; 2. Render the bounding box onto the texture image created above, with the output pixel value of the covered pixels being 1; 3. Count the number of pixels in the viewing plane image with a pixel value of 1 and divide it by the number of pixels in the viewing plane image itself, W × H (pixels with a pixel value of 0 + pixels with a pixel value of 1), to obtain the ratio r2 of the number of pixels covered by the bounding box to the number of pixels in the viewing plane image. The ratio r2 is then multiplied by the ratio r1 (the ratio of the number of pixels of the viewing plane image to the number of pixels of the screen) to obtain the ratio r3 of the number of pixels covered by the bounding box to the number of pixels of the screen.

[0141] In one embodiment of the present application, when the bounding box is a bounding sphere, the ratio of the number of pixels covered by the bounding box to the number of pixels of the viewing plane image is calculated as follows: 4πR 2 / (W × H). R represents the number of pixels within the bounding sphere radius, W represents the width of the viewing plane image resolution, and H represents the height of the viewing plane image resolution. Multiplying this ratio by L yields the ratio of the number of pixels covered by the final bounding box to the number of pixels on the screen.

[0142] In the embodiments of the present application, the relationship between the two is expressed as the ratio of the number of pixels a covered by the bounding box to the number of pixels b on the screen, i.e., a / b. In some embodiments, the relationship between the two can also be expressed as the ratio of the number of pixels a covered by the bounding box to the number of uncovered pixels c (c=b), or the relationship between the two can be expressed as b / c based on the ratio c of the number of uncovered pixels b to the number of pixels on the screen. In addition, a / b can also be expressed to the power of n, or multiples thereof, and the present application does not limit the form of expressing the relationship between the two.

[0143] In some embodiments, before projection, the identifiers of the pixels corresponding to the viewing plane image can be initialized to 0. After the GPU obtains the two-dimensional model of the bounding box, that is, after projecting it onto the viewing plane image, the pixel values ​​covered by the bounding box model are changed from 0 to 1, thereby obtaining a viewing plane image with an identifier as shown in (b) in Figure 8.

[0144] S504: Determine the distance (Distance, D) between the observation perspective and the object model in the scene.

[0145] The viewing angle refers to the camera mentioned above. The world coordinates of the object model can be calculated based on the conversion of the triangle mesh vertex coordinates of the object model with the model matrix (model coordinates). The distance between the camera and the object model can be calculated by calculating the distance D between the camera coordinates and the object model coordinates in world space (world coordinates). The camera coordinates can be coordinate values ​​set in advance based on the angle of observation required for the object in the scene, and the model matrix is ​​also a pre-set value. The specific calculation process of distance D is as follows:

[0146] 1. Take the three-dimensional vector (X, Y, Z) that identifies the world space coordinates in the model's space matrix, and the world space coordinates of the camera itself (X0, Y0, Z0);

[0147] 2. Calculate the modulus of the vector composed of two coordinates, that is, the distance D, using formula (1).

[0148] It should be noted that this step and S503 are parallel steps and there is no limitation on the order of precedence.

[0149] S505 , determining a level of detail (LOD) corresponding to the object model based on the distance and the ratio.

[0150] In one embodiment of the present application, the distance D and the ratio r3 are weighted and the weighted sum is calculated. For example, the weight of the distance D may be 0.3, and the weight of the ratio may be 0.7. The LOD level is calculated as 0.3D + 0.7 × r3.

[0151] In some embodiments, the weights can also be other proportional relationships, for example, the weight of distance D is 0.1, and the weight of ratio r3 is 0.9; or the weight of distance D is 0.5, and the weight of ratio r3 is 0.5, etc. By setting the ratio, the weights of the two can be adjusted more reasonably, which is conducive to obtaining a LOD model with a reasonable level.

[0152] S506 , updating the polygon mesh data based on the LOD level to obtain simplified polygon mesh data (second polygon mesh data).

[0153] In some embodiments of the present application, a face reduction algorithm may be used to remove some data from polygonal mesh data.

[0154] With reference to Figures 9 and 10, Figures 9 and 10 show schematic diagrams of a triangle face reduction algorithm. As shown in Figure 9, a vertex culling method can be used to remove vertices shared by triangles and then recombine the triangles. This process can refer to the process from (a) to (b) in Figure 9, where the number of triangles is reduced by removing vertices. As shown in Figure 10, an edge collapse method can be used to remove edges shared by several triangles and then recombine the triangles. This process can refer to the process from (a) to (b) in Figure 10, where the number of triangles is reduced by edge collapse. In addition, in other embodiments of the present application, other face reduction algorithms, such as face deletion, etc., can also be used to delete a whole shared face to reduce the number of triangles. This application is not limited to this.

[0155] After the face reduction algorithm, the polygonal mesh data of the object model is regenerated. Its structure can be seen in Figure 11, which shows the simplified structure of the sphere. As shown in Figure 11 (a), this structure is the original triangular mesh structure of the object model. After simplifying the facets on the sphere model through the above steps, the triangular mesh structure shown in Figure 11 (b) is obtained. It can be seen that the number of triangular meshes after simplification is significantly less than the number of triangular meshes before simplification.

[0156] The GPU obtains the vertex coordinates and index data of the triangles of the simplified object model, updates the vertex buffer data and index buffer data, and reconstructs the Vulkan ray tracing acceleration structure, including the vertex coordinates, index data, number of triangle meshes, etc. The specific implementation process is as follows:

[0157] 1. Get the GPU address of the model vertex buffer and index buffer after face reduction through the vulkan instruction vkGetBufferDeviceAddressKHR;

[0158] 2. Fill in the VkAccelerationStructureGeometryKHR and the corresponding BuildGeometryInfo data structures (that is, initialize a ray tracing geometry information structure and fill the vertex buffer and index buffer addresses obtained in the previous step into this geometry information structure);

[0159] 3. Call the vulkan instruction vkCmdBuildAccelerationStructuresKHR to build the ray tracing acceleration structure.

[0160] S507: Rendering the object model based on the simplified polygonal mesh data.

[0161] In one embodiment of the present application, before rendering an object model, the GPU may first determine whether the object model requires ray tracing rendering. If so, the GPU executes S507. By binding the ray tracing acceleration structure obtained in S506, the GPU uses a Vulkan ray tracing-supported shader to calculate the intersection of the ray and the scene to shade the object model.

[0162] Referring to Figure 12, a scene graph illustrating intersection calculations based on ray tracing is shown. As shown in Figure 12, the three-dimensional scene includes a camera 1210, a viewing plane image 1220, a light source 1230, a spherical model 1240 formed from a simplified triangular mesh, and light rays 1251 and 1252. Pixels in the viewing plane image 1220 other than those covered by the image do not intersect with the spherical model 1240. Therefore, during ray tracing, only the color values ​​corresponding to the covered pixels need to be calculated, reducing the computational effort. As shown in Figure 12, ray 1251 uses the camera origin as its starting point and the unit vector from the origin to the corresponding pixel as its emission direction. It then emits a ray toward the spherical model 1240 in the three-dimensional scene. After intersecting one of the triangular facets, the reflected ray directly reaches the light source 1230. This intersection is where the light source can illuminate. The GPU calculates the color value corresponding to this intersection based on the light intensity, material, and other factors, and outputs it to the corresponding pixel coordinates. As shown in Figure 12, after ray 1252 intersects sphere model 1240, it is reflected off the ground and then off the ground, failing to reach the light source. This indicates that the intersection point is not illuminated by the light source, and the corresponding output color value is darker than the color value of direct illumination. Based on the above principles, the intersection points between these triangle meshes and the ray are calculated, and the corresponding color values ​​are calculated and output to the corresponding pixels, resulting in the final rendered target image.

[0163] In other embodiments of the present application, when the GPU determines that the rendering mode of the sphere model is normal rendering, such as rasterization rendering, in the actual rendering process, the original triangle mesh data structure can also be used to perform rasterization rendering on the sphere model.

[0164] It should be noted that, in the above rendering process, the calculation methods mentioned, such as the calculation of the intersection shading value, etc., can be calculated using the existing technology, and this application will not describe this in detail.

[0165] It should be noted that in the above embodiments of the present application, the steps executed by the GPU may also be executed by the CPU. In some embodiments, the steps executed by the CPU may also be executed by the GPU, or both may be executed by the GPU or the CPU. This application does not limit this.

[0166] The rendering optimization method described in FIG5 above is a description of the preferred rendering of an object model. There are many object models in a scene, and the rendering methods of these object models will be different. For example, some object models require ray tracing, while some models do not require ray tracing calculations. Therefore, based on the rendering optimization method described in FIG5 , the present application can also subdivide or combine the above steps to obtain another flowchart of the rendering optimization method of the embodiment of the present application as shown in FIG13 . As shown in FIG13 , the flowchart includes S1301-S1308. Among them, these steps can be started by a rendering engine deployed in the application. The rendering engine can be a submodule in the processor.

[0167] S1301, executing scene update.

[0168] This process can be understood as acquiring object models in the scene and performing left-side transformations on them. The rendering engine then performs scene updates. Based on business logic information, it updates the model with animation and physics updates. The results of these updates are displayed as changes in the model's spatial coordinates and spatial matrix, as well as spatial coordinate changes caused by vertex displacements. The updated data is then synchronized to the model's vertex buffer. This includes triangle mesh vertex coordinates and index data. This step corresponds to S501 in Figure 5 .

[0169] S1302: Determine whether the current object model requires ray tracing.

[0170] In a scene, there are many object models, some of which require rendering and some of which do not. For object models that do not require ray tracing rendering, there is no need to perform simplified calculations on the models. For object models that require ray tracing, the processor executes S1303.

[0171] In S1303, the LOD level (LOD value) of the object model is calculated. That is, the distance of the object model from the camera in the 3D scene (world space) and the pixel ratio of the object model in the rendered screen are calculated first, and then the distance and ratio are weighted and summed to obtain the LOD level. The specific calculation process can be seen in S502-S505 in Figure 5.

[0172] S1304, model simplification. The simplification process can use a triangular mesh reduction algorithm to dynamically remove the triangular mesh of the model, such as using edge collapse and vertex merging to reduce the number of triangles and the corresponding number of vertices in the triangular mesh. This step corresponds to S506 in Figure 5.

[0173] The model is simplified based on the obtained LOD value to obtain a simplified object model.

[0174] The simplified scene in FIG13 refers to a storage area storing all object models in the scene, including simplified object model data and unsimplified object model data.

[0175] S1305: Constructing a ray tracing acceleration structure. This step corresponds to the description of the ray tracing acceleration structure in S506 in FIG5 .

[0176] When rendering a scene, the GPU needs to determine whether each model in the scene requires ray tracing rendering, that is, execute S1306.

[0177] In S1306 , the rendering mode of the model is determined, that is, whether ray tracing is required.

[0178] When it is determined that the model does not require ray tracing, the GPU executes S1307 to perform normal model rendering on the current model.

[0179] If it is determined that the model requires ray tracing, the GPU executes S1308 to perform ray tracing rendering. This step corresponds to the description of ray tracing in S507 in FIG5 .

[0180] According to the rendering optimization method of the embodiment of the present application, the LOD level of the model's simplification is determined by calculating the proportion of the model's visible range on the screen and observing the viewing angle and the distance of the model, thereby realizing dynamic adjustment of the LOD level. While maintaining the geometric shape of the model, the number of triangular meshes constituting the model is reduced, which can effectively reduce the computational consumption of ray tracing and more efficiently generate ray tracing rendering results with similar image quality.

[0181] Referring to FIG. 14 , the present application further provides an electronic device 1400 , including:

[0182] A mesh data acquisition module 1410 is configured to determine first polygon mesh data corresponding to an object model in a scene, where the first polygon mesh data is used to describe the object model.

[0183] A bounding box creation module 1420 is configured to create a bounding box corresponding to the object model based on the first polygon mesh data;

[0184] a ratio determination module 1430 for determining a pixel range covered by the bounding box when projected onto the screen of the electronic device, obtaining a relationship between the number of pixels covered by the bounding box and the number of pixels on the screen, and determining a distance between the viewing angle and the object model in the scene;

[0185] An LOD calculation module 1440 is configured to determine a level of detail (LOD) corresponding to the object model based on the distance and the relationship, and update the first polygon mesh data to the second polygon mesh data based on the LOD level;

[0186] The rendering module 1450 is configured to render the object model based on the second polygonal mesh data.

[0187] According to an embodiment of the present application, the LOD calculation module 1440 is further configured to perform weighted sum calculations on the distance and the relationship respectively to obtain a summation result, and determine the LOD level based on the summation result.

[0188] According to one embodiment of the present application, the ratio determination module 1430 is further configured to: obtain a first world coordinate of the observation angle in the world space, and a second world coordinate of the object model in the world space,

[0189] The distance between the viewing angle and the object model is determined based on the first world coordinate and the second world coordinate.

[0190] According to one embodiment of the present application, the LOD calculation module 1440 is used to:

[0191] According to the LOD level, a polygon mesh reduction algorithm is used to remove part of the polygon mesh data from the first polygon mesh information to obtain the second polygon mesh data.

[0192] According to one embodiment of the present application, the rendering module 1450 is configured to:

[0193] When the rendering module 1450 determines that the rendering method corresponding to the object model is ray tracing rendering;

[0194] Get the second polygon mesh information;

[0195] Based on the second polygonal mesh information, a ray tracing shader is used to calculate the intersection point between the light and the object model in the scene, and the object model is shaded based on the intersection point.

[0196] According to one embodiment of the present application, the rendering module 1450 is further configured to:

[0197] When the rendering module 1450 determines that the rendering mode corresponding to the object model is normal rendering;

[0198] The first polygon mesh information corresponding to the object model in the scene is obtained, and the object model is shaded using a rasterized shader.

[0199] According to one embodiment of the present application, the bounding box creation module 1420 is configured to: determine the maximum and minimum values ​​of the object model in the X direction, Y direction, and Z direction in its own coordinates based on the first polygonal mesh information;

[0200] Determines the bounding box based on the maximum and minimum values.

[0201] According to one embodiment of the present application, the relationship between the number of pixels covered by the bounding box and the number of pixels on the screen is a ratio of the number of pixels covered by the bounding box to the number of pixels on the screen.

[0202] According to one embodiment of the present application, the weight of the distance is set between 0.1-0.5, and the weight of the ratio is set between 0.5-0.9.

[0203] According to one embodiment of the present application, the weight of the distance is set to 0.3, and the weight of the ratio is set to 0.7.

[0204] According to one embodiment of the present application, the first polygon mesh information includes the number of polygons corresponding to the object model that has not been simplified by the LOD level, the vertex coordinates and vertex indexes of the polygons.

[0205] According to one embodiment of the present application, the second polygon mesh information includes the number of polygons corresponding to the simplified object model based on the LOD level, the vertex coordinates and vertex indexes of the polygons.

[0206] According to one embodiment of the present application, the polygon is a triangle.

[0207] It should be noted that the functions and effects of the above modules have been described in detail in the embodiment shown in FIG5 . For details, please refer to the corresponding description in FIG5 , and will not be repeated here.

[0208] The present application also provides an electronic device, comprising:

[0209] a memory for storing instructions to be executed by one or more processors of the device, and

[0210] The processor is used to execute the method explained in conjunction with Figures 5 to 13 in the above embodiments.

[0211] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes the method explained in Figures 5 to 13 in the above embodiments.

[0212] The present application also provides a computer program product comprising instructions, which, when executed on an electronic device, enables a processor to execute the methods shown in FIG. 5 to FIG. 13 in the above embodiments.

[0213] Referring now to FIG. 15 , a block diagram of a SoC (System on Chip) 1500 according to one embodiment of the present application is shown. In FIG. 15 , similar components have the same reference numerals. Additionally, dashed boxes indicate optional features of more advanced SoCs. In FIG. 15 , SoC 1500 includes an interconnect unit 1550 coupled to an application processor 1510; a system agent unit 1580; a bus controller unit 1590; an integrated memory controller unit 1540; a set of one or more coprocessors 1520, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 1530; and a direct memory access (DMA) unit 1560. In one embodiment, coprocessors 1520 include specialized processors, such as, for example, a network or communications processor, a compression engine, a general-purpose computing graphics processor (GPGPU), a high-throughput multi-core (MIC) processor, or an embedded processor.

[0214] The static random access memory (SRAM) unit 1530 may include one or more computer-readable media for storing data and / or instructions. The computer-readable storage medium may store instructions, specifically, temporary and permanent copies of the instructions. The instructions may include: when executed by at least one unit in the processor, causing the SoC 1500 to perform the method according to the above embodiment, specifically referring to the method explained in Figures 5 and 13 of the above embodiment, which will not be repeated here.

[0215] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0216] Program code can be applied to input instructions to perform the functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0217] Program code can be implemented with a high-level programming language or an object-oriented programming language to communicate with the processing system. Where necessary, program code can also be implemented in assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0218] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software or any combination thereof. The disclosed embodiments can also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, instructions can be distributed over a network or by other computer-readable media. Therefore, machine-readable media can include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to, floppy disks, optical disks, optical disks, compact disc read-only memories (Compact Disc Read Only Memory, CD-ROMs), magneto-optical disks, read-only memories (ROM), random access memories (RAM), erasable programmable read-only memories (EPROM), electrically erasable programmable read-only memories (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable memories for transmitting information (e.g., carrier waves, infrared signal digital signals, etc.) using the Internet in electrical, optical, acoustic or other forms of propagation signals. Accordingly, machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0219] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such a feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0220] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by this application. In addition, in order to highlight the innovative part of this application, the above-mentioned device embodiments of this application do not introduce units / modules that are not closely related to solving the technical problems raised by this application. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0221] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "including a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0222] Although the present application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the application.

Claims

1. A rendering optimization method, applied to electronic equipment, characterized in that: The method comprises: Determining first polygonal mesh data corresponding to an object model in a scene, where the first polygonal mesh data is used to describe the object model; creating a bounding box corresponding to the object model based on the first polygonal mesh data; Determining a pixel range covered by the bounding box when projected onto the screen of the electronic device, obtaining a relationship between the number of pixels covered by the bounding box and the number of pixels on the screen, and determining a distance between an observation angle in the scene and the object model; determining a level of detail (LOD) corresponding to the object model based on the distance and the relationship, and updating the first polygonal mesh data to second polygonal mesh data based on the LOD level; The object model is rendered based on the second polygonal mesh data.

2. The method according to claim 1, characterized in that Determining an LOD level corresponding to the object model based on the distance and the relationship includes: Perform weighted sum calculation on the distance and the relationship respectively to obtain a sum result, Based on the summation result, the LOD level is determined.

3. The method according to claim 1 or 2, characterized in that Determining a distance between an observation perspective in the scene and the object model includes: Obtaining the first world coordinates of the observation perspective in the world space and the second world coordinates of the object model in the world space, The distance between the viewing angle and the object model is determined based on the first world coordinate and the second world coordinate.

4. The method according to any one of claims 1 to 3, characterized in that Updating the first polygon mesh information to second polygon mesh information based on the LOD level includes: According to the LOD level, a polygon mesh reduction algorithm is used to remove part of the polygon mesh data from the first polygon mesh information to obtain the second polygon mesh data.

5. The method according to any one of claims 1 to 4, characterized in that Rendering the object model based on the second polygonal mesh data includes: When the electronic device determines that the rendering mode corresponding to the object model is ray tracing rendering; Obtaining the second polygon mesh information; Based on the second polygonal mesh information, a ray tracing shader is used to calculate the intersection of the light and the object model in the scene, and the object model is shaded based on the intersection.

6. The method according to claim 5, characterized in that Also includes: When the electronic device determines that the rendering mode corresponding to the object model is normal rendering; First polygon mesh information corresponding to the object model in the scene is obtained, and the object model is shaded using a rasterized shader.

7. The method according to any one of claims 1 to 6, characterized in that Creating a bounding box corresponding to the object model based on the first polygonal mesh information includes: Determine the maximum and minimum values ​​of the object model in the X direction, Y direction, and Z direction in its own coordinates based on the first polygonal mesh information; The bounding box is determined based on the maximum value and the minimum value.

8. The method according to any one of claims 1 to 7, characterized in that The relationship between the number of pixels covered by the bounding box and the number of pixels on the screen is: The ratio of the number of pixels covered by the bounding box to the number of pixels on the screen.

9. The method according to claim 8, characterized in that The weight of the distance is set between 0.1 and 0.5, and the weight of the ratio is set between 0.5 and 0.

9.

10. The method according to claim 9, characterized in that The weight of the distance is set to 0.3, and the weight of the ratio is set to 0.

7.

11. The method according to any one of claims 1 to 10, characterized in that The first polygon mesh information includes the number of polygons corresponding to the object model that has not been simplified by the LOD level, and the vertex coordinates and vertex indexes of the polygons.

12. The method according to any one of claims 1 to 11, characterized in that The second polygon mesh information includes the number of polygons corresponding to the object model simplified based on the LOD level, the vertex coordinates and vertex indexes of the polygons.

13. The method according to any one of claims 1 to 12, characterized in that The polygon is a triangle.

14. An electronic device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the device, and A processor is configured to execute the instructions so that the electronic device executes the method according to any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 13.

16. A computer program product comprising instructions, characterized in that When the computer program product is run on an electronic device, it enables a processor to execute the method according to any one of claims 1 to 13.

Images