Image rendering method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides an image rendering method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: carrying out image block disassembly processing on a virtual texture through a central processing unit to obtain a plurality of virtual texture image blocks of the virtual texture; determining the importance degree of each virtual texture image block, and determining a virtual texture image block to be rendered based on the importance degree of each virtual texture image block; obtaining mapping relation data between the virtual texture blocks to be rendered and the corresponding physical textures; and transmitting the to-be-rendered virtual texture blocks and the mapping relation data to a graphics processor through a central processing unit, so that the graphics processor loads physical textures corresponding to the to-be-rendered virtual texture blocks based on the mapping relation data, and performs rendering processing based on the physical textures to obtain a rendered image. According to the invention, the resource occupation in the transmission process can be reduced, and the image rendering efficiency can be improved.

Description

Technical Field

[0001] The present application relates to image rendering technology, and in particular to an image rendering method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the development of computer technology, users have increasingly higher requirements for the image quality presented by computers when using computer programs. As the image quality improves, more space is often occupied to store the loaded physical tiles.

[0003] In related technologies, in order to save space, virtual texture technology is used by the graphics processor to reduce the space occupied by loaded physical tiles. However, this method requires the graphics processor to transmit the determined texture-related data to the central processing unit, which then confirms the data and executes the subsequent image rendering process. As a result, the entire process occupies more transmission resources and has low rendering efficiency. Summary of the Invention

[0004] Embodiments of the present application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for image rendering, which can reduce resource usage during the transmission process and improve the efficiency of image rendering.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] An embodiment of the present application provides a method for image rendering, the method comprising:

[0007] Performing a block disassembly process on the virtual texture by a central processing unit to obtain a plurality of virtual texture blocks of the virtual texture;

[0008] Determining the importance of each virtual texture tile, and determining a virtual texture tile to be rendered based on the importance of each virtual texture tile;

[0009] Acquire mapping relationship data between the virtual texture block to be rendered and the corresponding physical texture;

[0010] The central processing unit transmits the virtual texture block to be rendered and the mapping relationship data to the graphics processor, so that the graphics processor loads the physical texture corresponding to the virtual texture block to be rendered based on the mapping relationship data, and performs rendering processing based on the physical texture to obtain a rendered image.

[0011] An embodiment of the present application provides an image rendering device, comprising:

[0012] A tile disassembly module is used to perform tile disassembly processing on the virtual texture through a central processing unit to obtain a plurality of virtual texture tiles of the virtual texture;

[0013] a tile determination module, configured to determine the importance of each virtual texture tile, and determine a virtual texture tile to be rendered based on the importance of each virtual texture tile;

[0014] A data acquisition module is used to acquire mapping relationship data between the virtual texture block to be rendered and the corresponding physical texture;

[0015] An image rendering module is used to transmit the virtual texture block to be rendered and the mapping relationship data to the graphics processor through the central processing unit, so that the graphics processor loads the physical texture corresponding to the virtual texture block to be rendered based on the mapping relationship data, and performs rendering processing based on the physical texture to obtain a rendered image.

[0016] In the above scheme, the tile determination module is also used to perform perspective confirmation processing on the rendering camera based on the first position of the rendering camera to obtain the field of view corresponding to the rendering camera; when the virtual texture tile is outside the field of view, the importance of the virtual texture tile is set to a null value; when the virtual texture tile is within the field of view, the importance of the virtual texture tile is determined based on the position of the virtual texture tile within the field of view.

[0017] In the above solution, the tile determination module is further configured to draw a viewing frustum, wherein the vertex of the viewing frustum is the first position of the rendering camera, and the perpendicular line of the viewing frustum is consistent with the orientation of the rendering camera; and the range included in the viewing frustum is used as the field of view corresponding to the rendering camera.

[0018] In the above scheme, the tile determination module is further used to determine the relative position between the rendering camera and the virtual texture tile based on the first position of the rendering camera and the second position of the virtual texture tile; and determine the importance of the virtual texture tile based on the relative position.

[0019] In the above scheme, the tile determination module is also used to project the virtual texture tile onto the display screen based on the relative position to obtain the area range of the display area of ​​the virtual texture tile on the display screen; and obtain the importance of the virtual texture tile that is positively correlated with the area range.

[0020] In the above scheme, the tile determination module is further used to determine the candidate physical texture corresponding to the virtual texture tile to be rendered; select the quadtree node corresponding to the importance from the quadtree corresponding to the candidate physical texture; and use the physical texture corresponding to the quadtree node as the physical texture corresponding to the virtual texture tile to be rendered.

[0021] In the above scheme, the tile determination module is also used to obtain a first difference between the virtual texture tile to be rendered corresponding to the current frame and the virtual texture tile to be rendered corresponding to the previous frame of the current frame; when the first difference exceeds a first difference threshold, the virtual texture tile to be rendered corresponding to the current frame is divided into multiple groups of virtual texture sub-tiles to be rendered; and the virtual texture tile to be rendered is updated to the virtual texture sub-tile to be rendered.

[0022] In the above scheme, the block determination module is also used to compare the virtual texture blocks to be rendered corresponding to the current frame with the virtual texture blocks to be rendered corresponding to the previous frame of the current frame to obtain a first number of virtual texture blocks to be rendered with differences; and use the first number as the first difference.

[0023] In the above scheme, the data acquisition module is also used to determine the scaling parameter and the bias parameter based on the mapping relationship between the virtual texture block to be rendered and the physical texture corresponding to the virtual texture block to be rendered; use the virtual texture block to be rendered as the index, and use the scaling parameter and the bias parameter as the index value.

[0024] In the above solution, the image rendering module is further used for the graphics processor to load the physical texture corresponding to the virtual texture block to be rendered in the mapping relationship data; and perform rendering processing based on the physical texture to obtain the rendered image.

[0025] In the above scheme, the image rendering module is also used to convert the original coordinate system where the physical texture is located into a world coordinate system through the vertex shader; convert the world coordinate system where the physical texture is located into an observation coordinate system through the vertex shader; and render the image interface based on the physical texture in the observation coordinate system and the virtual texture block to be rendered to obtain the rendered image.

[0026] In the above scheme, the image rendering module is also used for the pixel shader to perform pixel interpolation processing on the physical texture in the observation coordinate system to obtain the pixel value of each pixel point in the graphical interface; and generate the rendered image based on the pixel values ​​of multiple pixel points in the graphical interface.

[0027] An embodiment of the present application provides an electronic device, comprising:

[0028] a memory for storing computer-executable instructions;

[0029] The processor is used to implement the image rendering method provided in the embodiment of the present application when executing the computer-executable instructions stored in the memory.

[0030] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for implementing the image rendering method provided in the embodiment of the present application when executed by a processor.

[0031] An embodiment of the present application provides a computer program product, including computer-executable instructions. When the computer-executable instructions are executed by a processor, the image rendering method provided in the embodiment of the present application is implemented.

[0032] The embodiments of the present application have the following beneficial effects:

[0033] The present application uses a central processing unit to realize tile decomposition of a virtual texture and determination of virtual texture tiles to be rendered. Compared with the graphics processor decomposing the virtual texture into tiles and determining the virtual texture tiles to be rendered, and then sending the virtual texture tiles to be rendered determined by the graphics processor to the central processing unit, there is no need for the graphics processor to transmit data to the central processing unit, thereby saving the occupation of transmission resources during the image rendering process. At the same time, the central processing unit can transmit the mapping relationship data between the virtual texture and the physical texture to the graphics processor, so that the graphics processor can load the corresponding physical texture through the mapping relationship data and the virtual texture to be rendered, and then perform complex rendering processing. Compared with directly transmitting the physical texture to the graphics processor, the occupation of transmission resources during the image rendering process is saved and the efficiency of image rendering is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 1 is a schematic diagram of the architecture of an image rendering system 100 provided in an embodiment of the present application;

[0035] Figure 2 4 is a schematic structural diagram of an image rendering terminal 400 provided in an embodiment of the present application;

[0036] Figure 3A This is a schematic diagram of the image rendering process provided by the embodiment of the present application Figure 1 ;

[0037] Figure 3B This is a schematic diagram of the image rendering process provided by the embodiment of the present application Figure 2 ;

[0038] Figure 3CThis is a third flow chart of image rendering provided by an embodiment of the present application;

[0039] Figure 4 is a schematic diagram of the visual cone provided in an embodiment of the present application;

[0040] Figure 5 This is a schematic diagram of the startup interface of the virtual texture provided in an embodiment of the present application;

[0041] Figure 6 is a schematic diagram of terrain material options provided in an embodiment of the present application;

[0042] Figure 7A This is a schematic diagram of the add function provided in the embodiment of the present application Figure 1 ;

[0043] Figure 7B This is a schematic diagram of the add function provided in the embodiment of the present application Figure 2 ;

[0044] Figure 7C This is a third schematic diagram of the add function provided in an embodiment of the present application;

[0045] Figure 8 This is a schematic diagram of the basic color nodes provided in the embodiment of the present application;

[0046] Figure 9 This is a flowchart of the implementation of image rendering in a specific application scenario provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0048] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0049] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0050] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0051] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0052] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0053] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0054] 1) Graphics Processing Unit (GPU): A graphics processor is a microprocessor that performs image and graphics-related calculations on personal computers, workstations, game consoles, and some mobile devices (such as tablets and smartphones).

[0055] 2) Virtual Texture: Virtual Texture, also known as sparse texture, is a technology used to process large texture data. The main goal of this technology is to allow developers to use textures that are larger than the video memory without causing performance degradation.

[0056] 3) Terrain Rendering: Terrain rendering is an important concept in computer graphics, which involves the creation and display of terrain or terrain features (such as mountains, hills, rivers, etc.) in a 3D environment. It is a key component of many types of video games (such as simulation games, strategy games, role-playing games, etc.) and simulation software (such as geographic information systems, architectural visualization, etc.).

[0057] Terrain rendering is an important topic in computer graphics, involving the creation and rendering of large-scale 3D terrains. To make terrains appear realistic, textures are often applied to the terrain mesh. This can be achieved in a variety of ways, such as using texture mapping, texture blending, and detail texturing. However, these techniques also come with significant performance overhead, particularly due to the large number of samples and computational complexity of the materials.

[0058] To address this issue, traditional virtual texturing methods based on GPU readback were developed. Traditional virtual texturing technology divides images of various virtual texture resolutions into fixed-size blocks. The GPU readback results determine which block of each resolution to use, and only loads the resources for the corresponding block to reduce memory usage. GPU readback occurs when terrain rendering is performed, writing the virtual texture information for the current frame to be rendered to memory and then reading it back from the CPU cache. This can cause the following problems:

[0059] 1. The graphics processor needs to determine the virtual texture tiles to be rendered, and after determining the virtual texture tiles to be rendered, the virtual texture images to be rendered are sent to the central processing unit, resulting in the need to occupy transmission resources in transmitting data.

[0060] 2. Regardless of the difference between the virtual texture tile to be rendered currently and the virtual texture tile to be rendered in the previous frame, the image rendering needs to be completed within one frame, resulting in excessive load on the graphics processor.

[0061] Embodiments of the present application provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for image rendering, which can reduce resource usage during the transmission process and improve the efficiency of image rendering.

[0062] See also Figure 1 , Figure 1 4 is a schematic diagram of the architecture of the image rendering system 100 provided in an embodiment of the present application. In order to support an image rendering application, the device that executes the image rendering application may be a terminal 400.

[0063] Terminal 400 is configured to, through a central processing unit (CPU) included in terminal 400, perform tile decomposition processing on a virtual texture to obtain multiple virtual texture tiles of the virtual texture, determine the importance of each virtual texture tile, and based on the importance of each virtual texture tile, determine a virtual texture tile to be rendered, and obtain mapping relationship data between the virtual texture tile to be rendered and the corresponding physical texture. The CPU in terminal 400 then transmits the virtual texture tile to be rendered and the mapping relationship data to a graphics processor in terminal 400, so that the graphics processor loads the physical texture corresponding to the virtual texture tile to be rendered based on the mapping relationship data, performs rendering processing based on the physical texture, obtains a rendered image, and displays the rendered image on the display screen of terminal 400.

[0064] In implementing the image rendering method provided by this application, existing technologies such as artificial intelligence technology can be used. The following describes the artificial intelligence technology used in implementing the image rendering method provided in the embodiment of this application. A differential rendering framework can be used in the rendering process. The differential rendering framework uses artificial intelligence technology. Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0065] It should be noted that the image rendering method provided in the embodiment of the present application can be applied to game scenes and three-dimensional map application scenarios.

[0066] In the game scene, when the user first enters the game interface or the field of view position in the game interface changes, the central processing unit can perform tile disassembly processing on the virtual texture of the game interface, thereby obtaining multiple virtual texture tiles of the virtual texture of the game interface. The central processing unit can then determine the importance of each virtual texture tile, and then determine the virtual texture tile to be rendered based on the importance of each virtual texture tile, and obtain the mapping relationship data between the virtual texture tile to be rendered and the corresponding physical texture. The central processing unit transmits the virtual texture tile to be rendered and the mapping relationship data to the graphics processor through the central processing unit, so that the graphics processor loads the physical texture corresponding to the virtual texture tile to be rendered based on the mapping relationship data, and performs rendering processing based on the physical texture to obtain a rendered image. Finally, the obtained rendered image is displayed on the terminal display as a game interface.

[0067] In the application scenario of three-dimensional maps (such as autonomous driving scenarios), when the autonomous driving vehicle enters the three-dimensional map for the first time or when the field of view position in the three-dimensional map changes, the central processing unit can perform tile disassembly processing on the virtual texture of the three-dimensional map, thereby obtaining multiple virtual texture tiles of the virtual texture of the three-dimensional map. The central processing unit can then determine the importance of each virtual texture tile, and then determine the virtual texture tile to be rendered based on the importance of each virtual texture tile, and obtain the mapping relationship data between the virtual texture tile to be rendered and the corresponding physical texture. The central processing unit transmits the virtual texture tile to be rendered and the mapping relationship data to the graphics processor through the central processing unit, so that the graphics processor loads the physical texture corresponding to the virtual texture tile to be rendered based on the mapping relationship data, and performs rendering processing based on the physical texture to obtain a rendered image. Finally, the obtained rendered image is displayed on the display of the terminal as a display screen of the three-dimensional map.

[0068] The following describes exemplary applications of the image rendering device provided in the embodiments of the present application. The image rendering device provided in the embodiments of the present application can be implemented as various types of user terminals such as laptops, tablet computers, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), smart devices (e.g., smart speakers, smart watches, smart TVs, smart home appliances, smart voice interaction devices), vehicle terminals, aircraft, etc. The following describes exemplary applications when the device is implemented as a terminal.

[0069] See also Figure 2 , Figure 2 is a structural diagram of an image rendering terminal 400 provided in an embodiment of the present application, Figure 2 The terminal 400 shown includes: at least one central processing unit 410-1, at least one graphics processing unit 410-2, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0070] The central processing unit 410-1 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0071] Graphics processor 410-2 may be a microprocessor that performs image and graphics-related operations, including: (1) The core of the display main chip graphics card, commonly known as the GPU, whose main task is to construct and render video information input by the system. (2) Display buffer memory is used to store graphics information to be displayed and save intermediate data for graphics operations. The size and speed of the display buffer directly affect the performance of the main chip. (3) RAM D / A converter converts binary digital data into analog signals suitable for the display.

[0072] The graphics processor and image processor involved in the embodiments of the present application can be integrated into one electronic device or deployed in different electronic devices.

[0073] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0074] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0075] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0076] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0077] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0078] A network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include Bluetooth, Wi-Fi, and Universal Serial Bus (USB);

[0079] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripheral devices and displaying content and information);

[0080] The input processing module 454 is configured to detect one or more user inputs or interactions from one of the one or more input devices 432 and to translate the detected inputs or interactions.

[0081] In some embodiments, the apparatus provided in the embodiments of the present application may be implemented in software. Figure 2 The image rendering device 455 stored in the memory 450 is shown. This device can be software in the form of a program or plug-in, and includes the following software modules: a tile decomposition module 4551, a tile determination module 4552, a data acquisition module 4553, and an image rendering module 4554. These modules are logical and can be arbitrarily combined or further separated according to the functions they implement. The functions of each module will be described below.

[0082] In other embodiments, the apparatus provided in the embodiments of the present application may be implemented in hardware. As an example, the apparatus provided in the embodiments of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the image rendering method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor may be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0083] As mentioned above, the capacity processing method of the server cluster provided in the embodiment of the present application can be implemented by various types of electronic devices. Figure 3A , Figure 3AThis is a schematic diagram of the image rendering process provided by the embodiment of the present application Figure 1 , combined with Figure 3A Steps 101 to 104 are shown for explanation.

[0084] In step 101, a central processing unit (CPU) performs a block decomposition process on a virtual texture to obtain a plurality of virtual texture blocks of the virtual texture.

[0085] As an example, the central processing unit (CPU) can be the computing and control core in the terminal, and is the final execution unit for information processing and program execution. Its main function is to interpret computer instructions and process data in computer software. The CPU is the core component in the computer responsible for reading instructions, decoding instructions and executing instructions. The central processing unit mainly includes two parts, namely the controller and the arithmetic unit, which also includes a cache memory and a data and control bus that realizes the connection between them. The three core components of an electronic computer are the CPU, internal memory, and input / output devices. The functions of the central processing unit are mainly to process instructions, execute operations, control time, and process data.

[0086] As an example, the virtual texture may be a virtual texture including the entire scene, and the tile disassembly process of the virtual texture may be to divide the virtual texture including the entire scene into multiple squares of the same size, and treat the virtual texture included in each square as a virtual texture tile, or to divide the virtual texture including the entire scene into multiple triangles of the same size, and treat the virtual texture included in each triangle as a virtual texture tile.

[0087] As an example, taking a game scene as an example, if the terrain involved in the game scene is a mountain range, a virtual texture including the entire mountain range can be obtained. If the size of the virtual texture is 16x16, the process of tile decomposition of the virtual texture can be to divide the virtual texture into 16 4x4 squares, and the virtual texture included in each square is used as a virtual texture tile.

[0088] In step 102 , the importance of each virtual texture tile is determined, and based on the importance of each virtual texture tile, a virtual texture tile to be rendered is determined.

[0089] For example, since a virtual texture that includes the entire terrain requires too much cache, in actual applications, due to cache limitations, it is often impossible to load the entire terrain's virtual texture. In this case, it is necessary to use Runtime Virtual Texture technology, which can dynamically generate and update virtual textures at runtime. The following describes the application process of Runtime Virtual Texture technology in the embodiments of this application.

[0090] For example, only the virtual textures visible to the user need to be loaded. This significantly reduces cache usage compared to loading all virtual textures into the cache, allowing for higher-resolution virtual textures to be loaded without sacrificing cache performance. The following describes how to determine which virtual texture tiles need to be rendered.

[0091] As an example, the importance of each virtual texture tile may be used to determine the virtual texture tile to be rendered. The following describes a method for determining the importance of each virtual texture tile.

[0092] In some embodiments, the importance of each virtual texture tile in step 102 can be determined by Figure 3B Steps 1021 to 1023 are shown to be implemented.

[0093] In step 1021 , based on the first position of the rendering camera, a viewing angle confirmation process is performed on the rendering camera to obtain a field of view corresponding to the rendering camera.

[0094] For example, to ensure realistic service execution, the user's viewing angle can be used as the rendering camera's viewing angle. Specifically, the user's viewing position and angle can be used as the rendering camera's initial position. Based on the rendering camera's initial position, the rendering camera's viewing angle can then be verified to determine its corresponding field of view.

[0095] The following describes a method for confirming the field of view.

[0096] In some embodiments, the above step 1021 of performing viewing angle confirmation processing on the rendering camera based on the first position of the rendering camera to obtain the field of view corresponding to the rendering camera can be implemented by the following technical solutions:

[0097] Draw a viewing frustum, where the vertex of the viewing frustum is the first position of the rendering camera, and the vertical line of the viewing frustum matches the orientation of the rendering camera; the range included in the viewing frustum is used as the field of view of the rendering camera.

[0098] As an example, the human visual field is a range of a visual cone, that is, a visual cone obtained by taking the human eye as the vertex and the direction of the human eye as the perpendicular line of the visual cone. The range of the visual cone can be regarded as the range that the human eye can observe.

[0099] As an example, based on the above concept, to ensure a realistic effect, a viewing frustum of a rendering camera can be drawn. The specific process of drawing the viewing frustum of a rendering camera can be to use the position of the rendering camera as the first position, the orientation of the rendering camera as the perpendicular line of the viewing frustum, and then obtain the viewing frustum of the rendering camera. The range included in the viewing frustum is used as the field of view of the rendering camera.

[0100] As an example, the above method can reduce the rendered virtual texture while ensuring the user experience, thereby achieving the technical effect of reducing the memory occupied by image rendering.

[0101] In step 1022 , when the virtual texture tile is outside the field of view, the importance of the virtual texture tile is set to a null value.

[0102] As an example, after determining the field of view of the rendering camera, since the field of view of the rendering camera is the same as the field of view of the user, it can be determined that the virtual texture tiles outside the field of view will not be observed by the user. Therefore, the virtual texture tiles outside the field of view do not need to be loaded, and the importance of the virtual texture tiles outside the field of view can be set to a null value.

[0103] In step 1023 , when the virtual texture tile is located within the field of view, the importance of the virtual texture tile is determined based on the position of the virtual texture tile within the field of view.

[0104] For example, when a virtual texture tile is within the field of view, since the size of each virtual texture tile is the same during the above-mentioned tile decomposition process, and since objects closer to the human eye have larger volumes, the size of each virtual texture tile within the field of view of the rendering camera is different. Virtual texture tiles closer to the rendering camera occupy a larger size within the field of view, and therefore, virtual texture tiles closer to the rendering camera have a higher degree of importance.

[0105] The following describes a method for determining the importance of virtual texture tiles.

[0106] In some embodiments, determining the importance of the virtual texture tile based on the position of the virtual texture tile within the field of view in the above-mentioned step 1023 can be achieved by the following technical solution: determining the relative position between the rendering camera and the virtual texture tile based on the first position of the rendering camera and the second position of the virtual texture tile; and determining the importance of the virtual texture tile based on the relative position.

[0107] As an example, the position of the rendering camera and the position of the virtual texture tile are first determined, then the relative position of the rendering camera and the virtual texture is determined, and then the importance of the virtual texture tile is determined based on the relative position.

[0108] As an example, the first position of the rendering camera is represented by coordinates (1, 1), and the second position of the virtual texture tile is represented by coordinates (1, 4). It can be determined that the relative position between the first position of the rendering camera and the second position of the virtual texture tile is 3 unit lengths.

[0109] As an example, through the above method, the relative position between the virtual texture tile and the rendering camera can be accurately determined, thereby improving the accuracy of the determined importance of the virtual texture tile.

[0110] The following describes the process of determining the importance of virtual texture tiles based on their relative positions.

[0111] In some embodiments, the above-mentioned determination of the importance of the virtual texture tile based on the relative position can be achieved through the following technical solution: based on the relative position, the virtual texture tile is projected onto the display screen to obtain the regional range of the display area of ​​the virtual texture tile on the display screen; and the importance of the virtual texture tile is obtained which is positively correlated with the regional range.

[0112] As an example, since the picture finally displayed to the user is a whole plane map, the importance of each virtual texture tile can be determined by the size of the area occupied by each virtual texture tile in the plane map.

[0113] As an example, Figure 4 As shown, Figure 4 1023B is a schematic diagram of a viewing frustum provided in an embodiment of the present application. In practical applications, the bottom surface of the viewing frustum may be used as the display area on the display screen ultimately presented to the user. Therefore, the virtual texture tile shown in 1023A may be projected onto the bottom surface of the viewing frustum to obtain the projected area shown in 1023B.

[0114] As an example, after projecting all virtual texture tiles within the viewing frustum onto the bottom surface of the viewing frustum, virtual texture tiles closer to the rendering camera are projected onto a higher layer above the bottom surface of the viewing frustum, following the physically correct occlusion principle. This ensures that virtual texture tiles closer to the rendering camera at the bottom of the viewing frustum are not obscured by virtual texture tiles farther away. The importance of completely obscured virtual texture tiles can be set to 0.

[0115] As an example, through the above method, the accuracy of the determined importance of the virtual texture can be improved.

[0116] In step 103, mapping relationship data between the virtual texture block to be rendered and the corresponding physical texture is obtained.

[0117] As an example, since in the subsequent image rendering process, the central processing unit needs to transmit data to the graphics processor so that the graphics processor can render the image according to the received data, in order to reduce the amount of data transmitted from the central processing unit to the graphics processor, the mapping relationship data between the virtual texture block to be rendered and the corresponding physical texture can be used instead of directly transmitting the physical texture with a large amount of data.

[0118] Before obtaining the mapping data, you need to determine which physical textures will be used when rendering the image. The following describes a method for determining physical textures.

[0119] In some embodiments, before executing the above step 103 of obtaining the mapping relationship data between the virtual texture tile to be rendered and the corresponding physical texture, the following technical solution can also be executed: determining the candidate physical texture corresponding to the virtual texture tile to be rendered; selecting the quadtree node corresponding to the importance from the quadtree corresponding to the candidate physical texture; and using the physical texture corresponding to the quadtree node as the physical texture corresponding to the virtual texture tile to be rendered.

[0120] As an example, each physical texture may correspond to a quadtree structure, and the parent node of the quadtree structure may be the physical texture with the lowest resolution of the physical texture. As the quadtree structure extends, the nodes of each layer of the quadtree structure store physical textures with higher resolution than the physical textures corresponding to the nodes of the previous layer of the quadtree.

[0121] As an example, the resolution of the physical texture corresponding to the parent node 103B of the quadtree structure corresponding to the physical texture 103A is 320*480, the resolution of the physical texture corresponding to the child node 103C connected to the parent node 103B is 480*800, the resolution of the physical texture corresponding to the child node 103D connected to the child node 103C is 720*1280, and the resolution of the physical texture corresponding to the child node 103E connected to the child node 103D is 1080*1920.

[0122] As an example, since the importance of the virtual texture tile represents the area range of the virtual texture tile in the display screen, based on the principle that the larger the area range in the display screen, the higher the resolution of the corresponding physical texture, the physical texture corresponding to the quadtree node is selected as the physical texture corresponding to the virtual texture to be rendered.

[0123] As an example, if the importance of virtual texture tile 103F is 1 and the importance of virtual texture tile 103G is 2, a physical texture with a resolution of 320*480 can be selected as the physical texture of virtual texture tile 103F, and a physical texture with a resolution of 480*800 can be selected as the physical texture of virtual texture tile 103G.

[0124] For example, if the viewing angle of a frame changes significantly from the previous one, resulting in significant changes in the virtual texture tiles to be rendered, the GPU needs to render all the changed virtual texture tiles within a single frame. This can place a heavy burden on the GPU, and a GPU with poor performance may not be able to complete this task. The following describes an image rendering method for situations where the virtual texture tiles to be rendered have significant changes.

[0125] As an example, the above method can present higher-resolution physical textures in an image, thereby improving the user experience.

[0126] In some embodiments, before executing the above step 103 to obtain the mapping relationship data between the virtual texture block to be rendered and the corresponding physical texture, the following steps may also be executed: Figure 3C Steps 1031 to 1033 are shown:

[0127] In step 1031 , a first difference between a virtual texture block to be rendered corresponding to a current frame and a virtual texture block to be rendered corresponding to a frame before the current frame is obtained.

[0128] For example, since the refresh rate of the human eye is often lower than that of the screen, the human eye may not be able to discern the difference when there is a delay of one or several frames in the rendered image. Based on this, when the virtual texture tiles to be rendered vary significantly, the virtual texture tiles to be rendered can be grouped, and only the virtual texture tiles included in each group are rendered in each frame, extending the task of rendering the virtual texture tiles from one frame to multiple frames.

[0129] As an example, whether to group the virtual texture tiles to be rendered may be determined based on a first difference between the virtual texture tiles to be rendered corresponding to the current frame and the virtual texture tiles to be rendered corresponding to the frame before the current frame.

[0130] A method for determining the first difference is described below.

[0131] In some embodiments, obtaining the first difference between the virtual texture blocks to be rendered corresponding to the current frame and the virtual texture blocks to be rendered corresponding to the previous frame of the current frame in the above step 1031 can be achieved by the following technical solution: comparing the virtual texture blocks to be rendered corresponding to the current frame and the virtual texture blocks to be rendered corresponding to the previous frame of the current frame to obtain the first number of virtual texture blocks to be rendered with differences; and using the first number as the first difference.

[0132] As an example, the CPU may compare the virtual texture tiles to be rendered in the current frame with the virtual texture tiles to be rendered in the previous frame, determine the number of different virtual texture tiles to be rendered, and use the number as the first difference.

[0133] As an example, the number of virtual texture tiles included in the current frame 1031A is 100, and the number of virtual texture tiles included in the previous frame 1031B is 80. Among them, the number of virtual texture tiles in the current frame 1031A that are the same as those in the previous frame 1031B is 40. Therefore, the first number between the current frame 1031A and the previous frame 1031B is 60 (there are a total of 100 virtual texture tiles, 40 of which are the same as the previous frame and do not need to be rendered, and the number that needs to be rendered is 60). The 60 virtual texture tiles can be used as the first difference.

[0134] In step 1032 , when the first difference exceeds a first difference threshold, the virtual texture block to be rendered corresponding to the current frame is divided into multiple groups of virtual texture sub-blocks to be rendered.

[0135] As an example, when the first difference exceeds a first difference threshold, the virtual texture tiles to be rendered corresponding to the current frame may be grouped to obtain multiple groups of virtual texture sub-tiles.

[0136] As an example, for the way to divide the virtual texture tiles to be rendered, you can first sort them according to their importance, and group the first 50% of the virtual textures to be rendered in the sequence as a group, and group the last 50% of the virtual textures to be rendered in the sequence as a group.

[0137] The above is only one specific division method provided in the embodiment of the present application. In actual applications, the number of groups obtained after division can be 2 or 3. The specific number of groups can be set according to actual conditions. At the same time, the specific division method can also be random division or other methods, which are not specifically limited here.

[0138] In step 1033 , the virtual texture tile to be rendered is updated to a virtual texture sub-tile to be rendered.

[0139] As an example, after the virtual texture tile to be rendered is divided into multiple groups of virtual texture sub-tiles, the virtual texture tile to be rendered may be updated to one of the groups of virtual texture sub-tiles.

[0140] As an example, since virtual textures with higher importance show more details and are easier for users to observe, virtual texture sub-blocks with higher importance can be used as virtual texture blocks to be rendered in the current frame, and virtual texture sub-blocks with lower importance can be used as virtual texture blocks to be rendered in the next frame.

[0141] The following describes a method for obtaining mapping relationship data.

[0142] In some embodiments, the mapping relationship data includes an index and an index value. Acquiring the mapping relationship data between the virtual texture tile to be rendered and the corresponding physical texture in step 103 can be achieved by the following technical solution: determining a scaling parameter and an offset parameter based on the mapping relationship between the virtual texture tile to be rendered and the physical texture corresponding to the virtual texture tile to be rendered; using the virtual texture tile to be rendered as an index, and using the scaling parameter and the offset parameter as the index value.

[0143] As an example, first, based on the mapping relationship between the virtual texture tile to be rendered and the physical texture corresponding to the virtual texture tile to be rendered, the scaling parameter and the bias parameter are determined, wherein the goal of the scaling parameter and the bias parameter is to obtain the physical texture corresponding to the virtual texture tile through the virtual texture tile, the scaling parameter and the bias parameter.

[0144] As an example, the scaling parameter and the bias parameter may be determined by, after the coordinates of the virtual texture tile and the storage address of the physical texture of the virtual texture tile are known, calculating the scaling parameter and the bias parameter on the condition that the storage address of the physical texture of the virtual texture tile can be obtained by multiplying the coordinates of the virtual texture tile by the scaling parameter and then adding the bias parameter.

[0145] As an example, after obtaining the scaling parameter and the bias parameter, the virtual texture tile can be used as an index, and the scaling parameter and the bias parameter can be used as index values ​​to obtain the mapping relationship data between the virtual texture tile to be rendered and the corresponding physical texture.

[0146] As an example, the mapping relationship data between the virtual texture tiles to be rendered and the corresponding physical textures may include a table of all the virtual texture tiles to be rendered and scaling parameters and offset parameters corresponding to each virtual texture tile to be rendered.

[0147] As an example, through the above method, the address of the physical texture can be represented by a small amount of data, thereby reducing the occupation of transmission resources when transmitting data.

[0148] In step 104, the virtual texture block to be rendered and the mapping relationship data are transmitted to the graphics processor through the central processing unit, so that the graphics processor loads the physical texture corresponding to the virtual texture block to be rendered based on the mapping relationship data, and performs rendering processing based on the physical texture to obtain a rendered image.

[0149] As an example, after obtaining the virtual texture tiles to be rendered and the mapping relationship data, the virtual texture tiles to be rendered and the mapping relationship data can be sent to a graphics processor, and the graphics processor completes the subsequent image rendering.

[0150] The following describes the rendering process of the graphics processor after obtaining the virtual texture to be rendered and the mapping relationship data.

[0151] As an example, after obtaining the virtual texture tile to be rendered and the mapping relationship data, the graphics processor can load the physical texture of the virtual texture tile to be rendered based on the mapping relationship data and the virtual texture tile to be rendered. The specific loading process can be to find the address of the physical texture corresponding to the virtual texture tile to be rendered through the virtual texture tile to be rendered and the mapping relationship data, and then load the physical texture into the cache.

[0152] The following describes the image rendering process after the physical texture is loaded.

[0153] In some embodiments, the graphics processor includes a vertex shader, and the rendering process based on the physical texture to obtain the rendered image in the above step 104 can be implemented by the following technical solution: rendering process based on the physical texture to obtain the rendered image.

[0154] Through the vertex shader, the original coordinate system where the physical texture is located is converted into the world coordinate system; through the vertex shader, the world coordinate system where the physical texture is located is converted into the observation coordinate system; based on the physical texture in the observation coordinate system and the virtual texture block to be rendered, the image interface is rendered to obtain a rendered image.

[0155] For example, an origin coordinate system can be an object coordinate system or a local coordinate system. An origin coordinate system is a coordinate system established with an object or point in the scene as its origin. In this coordinate system, the position and orientation of an object are measured relative to this origin. The origin coordinate system is primarily used to describe the geometry and surface properties of an object, such as texture coordinates and normal vectors.

[0156] The world coordinate system can be a global coordinate system. The world coordinate system is a three-dimensional space coordinate system used to describe the position and orientation of all objects in the scene. The world coordinate system is fixed and usually has a reference point in the scene (such as the center of the scene or a specific point) as its origin. During the rendering process, all objects are converted to the world coordinate system to facilitate calculations such as position, orientation, and lighting.

[0157] The observation coordinate system can be either the camera coordinate system or the viewpoint coordinate system. The observation coordinate system is based on the camera's position and orientation. It describes the camera's position and viewing direction within the scene, as well as the position and shape of objects within the scene from the camera's perspective. During the rendering process, objects are transformed into the observation coordinate system for projection and rendering operations, ultimately generating the image we see.

[0158] As an example, the coordinates of the virtual texture to be rendered are the coordinates in the original coordinate system. In order to obtain the final image, the coordinates of the virtual texture to be rendered need to be converted into coordinates in the world coordinate system through the vertex shader, and then the coordinates of the world coordinate system are converted into coordinates in the observation coordinate system.

[0159] As an example, the vertex shader can convert the coordinates of the virtual texture to be rendered into the coordinates in the original coordinate system by multiplying each vertex by a model matrix (usually called the world matrix or model matrix), which contains the position, rotation, and scale information of the object. In this way, the vertex is converted from the original coordinate system to the world coordinate system, and its position and orientation are defined relative to the entire scene.

[0160] As an example, a vertex shader converts world coordinates to view coordinates by multiplying the vertex's world coordinates by a view matrix, which describes the camera's position and orientation. This converts the vertex from the world coordinates to the view coordinates, where its position and orientation are defined relative to the camera.

[0161] After obtaining the coordinates in the observation coordinate system, image rendering can be performed through a pixel shader. The following describes the process of image rendering through a pixel shader in a graphics processor.

[0162] In some embodiments, the graphics processor includes a pixel shader. The above-mentioned rendering processing of the image interface based on the physical texture in the observation coordinate system and the virtual texture block to be rendered can be achieved by the following technical solution: the pixel shader performs pixel interpolation processing on the physical texture in the observation coordinate system to obtain the pixel value of each pixel point in the graphics interface; based on the pixel values ​​of multiple pixels in the graphics interface, a rendered image is generated.

[0163] As an example, since the physical texture rendering process may involve operations such as expansion, scaling, and cropping, some display areas may have no pixel values. In order to avoid the situation where some positions have no pixel values, the physical texture can be processed with a pixel shader to perform pixel difference processing, and then the pixel value of each position can be obtained, and the pixel value of each position can be assigned to each pixel point to obtain the rendered image.

[0164] As an example, a pixel shader can perform pixel interpolation on a physical texture in the following ways:

[0165] (1) No interpolation is performed.

[0166] (2) Linear interpolation: An interpolation method that uses the middle point between two points, generally used in orthogonal projection.

[0167] (3) Default interpolation method: Vertices are interpolated in a perspective-corrected manner, which is used in perspective projection. In perspective projection, images are displayed in a way that objects far away are smaller and objects near are larger. Therefore, it is necessary to obtain uniform linear interpolation based on the perspective-corrected interpolation method.

[0168] The image rendering method provided by the embodiment of the present application can save the transmission resources occupied during the image rendering process and improve the efficiency of image rendering.

[0169] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0170] The image rendering method provided in the embodiment of the present application is a runtime virtual texture terrain rendering method that does not require graphics processor readback. Specifically, Figure 5 The interface shown, Figure 5 This is a schematic diagram of the startup interface of the virtual texture provided by the embodiment of the present application. Figure 5 The steps in this chapter include creating a terrain, checking the switch to use runtime virtual textures, and modifying the terrain material used by the terrain. The following will describe them in detail in conjunction with the Unreal Engine usage scenario.

[0171] First, create a terrain and check the UseLandscape VT switch for the main terrain object. Through the automatic refresh mechanism implemented in the code, the runtime virtual texture switch will be checked for all terrain streaming proxies associated with the main terrain object. In addition, the runtime virtual texture switch will be checked for all associated landscape components, enabling the virtual texture function for all components in the entire terrain with one click.

[0172] Afterwards, see Figure 6 , Figure 6 Schematic diagram of terrain material options provided in an embodiment of the present application.

[0173] exist Figure 6 In the , we modify the terrain material used by the terrain. First, we turn on the switch to use the terrain virtual texture in the material to create the corresponding shader variant.

[0174] Later you can see Figures 7A to 7C , Figure 7A This is a schematic diagram of the add function provided in the embodiment of the present application Figure 1 , Figure 7B This is a schematic diagram of the add function provided in the embodiment of the present application Figure 2 , Figure 7C This is the third schematic diagram of the add function provided in the embodiment of the present application.

[0175] exist Figure 7A In the , add a custom material function with the parameter name (LandscapeVTSwitchFloat3), where the function name of the material function is (DefaultinputFloat(V3)) and the input of the function is the input of the virtual texture (VTInputFloat).

[0176] like Figure 7B The code shown is for Material Expression Custom.

[0177] like Figure 7C As shown in the figure, a custom node that distinguishes branches through static macros is added to distinguish the virtual texture rendering channel (VTFarinputFloat(V3)) and the terrain base channel (NearinputFloat(V3)). Because the node connection branches required in these two channels are different, one branch is used for the virtual texture map that is not called in every frame, and the other branch is used for the terrain sampling virtual texture that needs to be performed in every frame.

[0178] In addition, since not all calculations in the BaseColor connection are completely static, there are some nodes that change dynamically (such as the high-altitude map that is mixed according to the distance to the camera), so you need to Figure 8 As shown, Figure 8 This is a schematic diagram of the basic color nodes provided in an embodiment of the present application.

[0179] exist Figure 8The purpose of the Custom Output Node in the Graph is to only extract the static calculations and bake them into the virtual texture. When there is a visibility layer in the weight map, please note that the mask value also needs to be baked into the virtual texture.

[0180] The following combination Figure 9 , Figure 9 This is a flowchart of the implementation of image rendering in a specific application scenario provided by an embodiment of the present application.

[0181] exist Figure 9 In the first step, you need to input terrain data. After inputting terrain data, you can get physical map (physical texture) and mapping table. The following introduces physical map and mapping table respectively.

[0182] Physical Texture: The actual sampled atlas. Each fixed-size block in the atlas corresponds to an image block of variable size in a huge map, which stores the mixed results of the terrain, including base color (BaseColor), normal (Normal), roughness (Roughness), specular (Specular), mask (Mask), etc. The multi-layer structure (multi-layer Layer) in the virtual texture management part (FVirtualTextureSpace) is mixed through the pixel shader. The mixing ratio is specified by the RGBA four channels of the mixed map, which is to pack the various mixing results here together to reduce the virtual texture address conversion overhead of the same texture coordinate.

[0183] The mapping table (Page Table), corresponding to the page table in virtual memory, contains enough information in a cache or a map to map the original default coordinate system to the coordinate system used to sample the physical texture, using a four-component offset and scale vector (ScaleOffset, where the first two components correspond to the scale parameter (Scale) of the coordinates, and the last two components correspond to the offset parameter (Offset) of the coordinates. Each index of the mapping table array corresponds to a virtual texture tile in the huge map, that is, the size of the mapping table is equal to the number of virtual texture tiles into which the huge map is divided.

[0184] Combining the above physical texture and mapping table, we can get a 4K*4K physical texture and a mapping page table to simulate a huge 512K*512K virtual texture.

[0185] exist Figure 9 In the Feedback Pass process, it is necessary to determine what to render in the virtual texture and update the mapping table.

[0186] The general implementation method is to collect screen information (staging buffer) through a (asynchronous) readback channel from GPU to CPU to determine which virtual texture tiles are currently needed (that is, the GPU determines which areas need to be rendered, and then which physical maps need to be loaded, and then the determined information is sent to the CPU); however, reading back every frame has too great an impact on transmission resources, so the virtual texture tiles to be rendered are divided in the CPU and the importance is calculated to find which virtual texture tiles need to be rendered into the physical map.

[0187] Each section (2x2) in each Landscape Component has its own virtual texture. For performance reasons, the physical texture in each virtual texture is only split into 4x4 physical texture tiles, which must also include a physical texture tile covering the entire section to ensure that when the high-resolution physical texture tile is on the physical map, the corresponding low-resolution physical texture tile can be used as an alternative. The method to obtain the virtual texture tile to be rendered can be:

[0188] First, the virtual texture tiles outside the frustum are removed through frustum culling.

[0189] The importance quadtree is then determined based on the screen-to-body ratio. Each physical texture is split into sub-images of varying resolutions. The correspondence between these sub-images and the physical texture can be represented using a quadtree. When expanding the quadtree branches, some child nodes may not have their own dedicated physical texture tiles. Therefore, these child nodes must point to the physical texture tiles of their parent node. The importance is then determined based on the distance from the camera. The closer the distance to the camera, the more important and the higher the resolution.

[0190] When there are many virtual texture tiles that need to be updated, all of them may not be updated in one frame, but may be updated in multiple frames.

[0191] After determining the virtual texture tiles to be rendered for each component, the values ​​in the mapping table need to be updated based on the actual positions of each physical texture tile in the physical cache.

[0192] When the position or perspective changes significantly, an update is required. At this time, only those virtual texture tiles that are not in the physical cache need to be updated. That is, those virtual texture tiles in the physical cache may not need to be updated, but only the scaling parameters and bias parameters in the mapping page table need to be updated.

[0193] exist Figure 9After the feedback transfer process is completed, the virtual texture rendering channel (VT RenderingPass) step is executed. In this step, only a triangle (Quad) can be rendered when rendering the image. Since the mesh material shader (Mesh Material Shader) is used for terrain rendering, and the pixel shader and vertex shader included in the graphics processing are different from those in terrain rendering, there is a lot of mesh information that the pixel shader and vertex shader cannot perceive (including texture coordinates (TexCoords[]), vertex blending weights (LayerWeights), and normals to the real world (TangentToWorld)).

[0194] The Unreal Engine 4 component system (UE4 Landscape) includes three types of coordinates: graphics coordinates (Quad UV), weight coordinates (Weight Map UV), and height coordinates (Height Map UV). Each type of coordinate covers a different range, and it is necessary to convert the terrain system coordinates when rendering the image. The following introduces the above three types of coordinates.

[0195] Quad UV (Quad refers to a square made up of two triangles), each Quad corresponds to [0, 1], and each section (Section) has 64x64 Quads; Quad UV is obtained by using Wrap simulation sampling (the sampling mode of the map can be set in the GPU, and Wrap sampling is one of them. When the texture coordinates exceed the range of [0, 1], it will divide the coordinate value by 1 and take the remainder, so that the coordinates out of range can be mapped back to the range of [0, 1], that is, the texture coordinates of each quad are obtained through wrap sampling).

[0196] Weight Map UV, each section corresponds to [0, 1], each component has 2x2 sections; Weight Map UV is used for blending weight layers, which may contain visibility layers, in which case the mask value needs to be baked. (The weight of the blended material is manually entered by the artist for each section. For example, if the section is for a river, the weight of the river material should be higher.)

[0197] Height Map UV, each component (Component) corresponds to [0, 1]; Height Map UV can be obtained through Vertex Texture Fetch. When the height is not obtained through Vertex Texture Fetch, an additional step (Load) is required to obtain the Height Map UV (Vertex Texture Fetch is a feature of sampling textures in the vertex shader, only high-end models support this feature; if this feature is used, the height of the terrain vertex can be directly obtained by sampling the height map in the vertex shader; but if this feature is not used, the height map will not be used at runtime, that is, it will not be loaded by default, which requires us to load it additionally).

[0198] exist Figure 9 In the step of executing the virtual texture rendering channel (VT Rendering Pass), the base rendering step (Base Pass Rendering) is executed. In this step, because the sampling coordinates of the virtual texture in an interface (Drawcall) are not continuous, they cannot be calculated in the vertex shader and can only be calculated in the pixel shader.

[0199] In order to deal with the seams, padding is required (equivalent to leaving a blank at the seam of two physical textures so that the connection between the two images will not be subtracted during interpolation), that is, the edge of each sub-image in the physical cache needs to render a few more extra pixels.

[0200] like Figure 9 As shown, after the basic rendering steps are performed, a rendered image can be obtained, which is output and displayed to the user.

[0201] The following continues to describe an exemplary structure of the image rendering device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the image rendering device 455 of the memory 450 may include:

[0202] A tile disassembly module 4551 is configured to perform tile disassembly processing on the virtual texture through a central processing unit to obtain a plurality of virtual texture tiles of the virtual texture;

[0203] a tile determination module 4552 for determining the importance of each virtual texture tile, and determining a virtual texture tile to be rendered based on the importance of each virtual texture tile;

[0204] The data acquisition module 4553 is used to acquire the mapping relationship data between the virtual texture block to be rendered and the corresponding physical texture;

[0205] The image rendering module 4554 is used to transmit the virtual texture block to be rendered and the mapping relationship data to the graphics processor through the central processing unit, so that the graphics processor loads the physical texture corresponding to the virtual texture block to be rendered based on the mapping relationship data, and performs rendering processing based on the physical texture to obtain a rendered image.

[0206] In some embodiments, the tile determination module 4552 is further used to perform perspective confirmation processing on the rendering camera based on the first position of the rendering camera to obtain the field of view corresponding to the rendering camera; when the virtual texture tile is outside the field of view, the importance of the virtual texture tile is set to a null value; when the virtual texture tile is within the field of view, the importance of the virtual texture tile is determined based on the position of the virtual texture tile within the field of view.

[0207] In some embodiments, the tile determination module 4552 is further configured to draw a viewing frustum, wherein the vertex of the viewing frustum is the first position of the rendering camera, and the vertical line of the viewing frustum is consistent with the orientation of the rendering camera; and the range included in the viewing frustum is used as the field of view corresponding to the rendering camera.

[0208] In some embodiments, the tile determination module 4552 is further used to determine the relative position between the rendering camera and the virtual texture tile based on the first position of the rendering camera and the second position of the virtual texture tile; and determine the importance of the virtual texture tile based on the relative position.

[0209] In some embodiments, the tile determination module 4552 is further used to project the virtual texture tile onto the display screen based on the relative position to obtain the area range of the display area of ​​the virtual texture tile on the display screen; and obtain the importance of the virtual texture tile that is positively correlated with the area range.

[0210] In some embodiments, the tile determination module 4552 is further used to determine a candidate physical texture corresponding to the virtual texture tile to be rendered; select a quadtree node corresponding to the importance from the quadtree corresponding to the candidate physical texture; and use the physical texture corresponding to the quadtree node as the physical texture corresponding to the virtual texture tile to be rendered.

[0211] In some embodiments, the tile determination module 4552 is also used to obtain a first difference between the virtual texture tile to be rendered corresponding to the current frame and the virtual texture tile to be rendered corresponding to the previous frame of the current frame; when the first difference exceeds a first difference threshold, the virtual texture tile to be rendered corresponding to the current frame is divided into multiple groups of virtual texture sub-tiles to be rendered; and the virtual texture tile to be rendered is updated to the virtual texture sub-tile to be rendered.

[0212] In some embodiments, the tile determination module 4552 is further used to compare the virtual texture tiles to be rendered corresponding to the current frame with the virtual texture tiles to be rendered corresponding to the previous frame of the current frame to obtain a first number of virtual texture tiles to be rendered that have differences; and use the first number as the first difference.

[0213] In some embodiments, the data acquisition module 4553 is also used to determine the scaling parameter and the bias parameter based on the mapping relationship between the virtual texture tile to be rendered and the physical texture corresponding to the virtual texture tile to be rendered; use the virtual texture tile to be rendered as the index, and use the scaling parameter and the bias parameter as the index value.

[0214] In some embodiments, the image rendering module 4554 is further used for the graphics processor to load the physical texture corresponding to the virtual texture block to be rendered in the mapping relationship data; and perform rendering processing based on the physical texture to obtain the rendered image.

[0215] In some embodiments, the image rendering module 4554 is also used to convert the original coordinate system where the physical texture is located into a world coordinate system through the vertex shader; convert the world coordinate system where the physical texture is located into an observation coordinate system through the vertex shader; and render the image interface based on the physical texture in the observation coordinate system and the virtual texture block to be rendered to obtain the rendered image.

[0216] In some embodiments, the image rendering module 4554 is also used by the pixel shader to perform pixel interpolation processing on the physical texture in the observation coordinate system to obtain the pixel value of each pixel point in the graphical interface; and generate the rendered image based on the pixel values ​​of multiple pixel points in the graphical interface.

[0217] An embodiment of the present application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the image rendering method described in the embodiment of the present application.

[0218] The embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the processor will execute the image rendering method provided by the embodiment of the present application, for example, Figure 3A A method of image rendering is shown.

[0219] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0220] In some embodiments, computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0221] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0222] By way of example, computer-executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed across multiple sites and interconnected by a communication network.

[0223] In summary, the following beneficial effects can be achieved through the embodiments of the present application:

[0224] The present application uses a central processing unit to realize tile decomposition of virtual textures and determine the virtual texture tiles to be rendered. Compared with the related art in which the graphics processor decomposes the virtual texture into tiles and determines the virtual texture tiles to be rendered, the virtual texture tiles to be rendered determined by the graphics processor are sent to the central processing unit. There is no need for the graphics processor to transmit data to the central processing unit, which saves the occupation of transmission resources in the image rendering process. At the same time, the central processing unit can transmit the mapping relationship between the virtual texture and the physical texture to the graphics processor, so that the graphics processor can load the corresponding physical texture through the mapping relationship, and then perform complex rendering processing. Compared with directly transmitting the physical texture to the graphics processor, it further saves the occupation of transmission resources in the image rendering process and improves the efficiency of image rendering.

[0225] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A method for image rendering, characterized in that: The method comprises: Performing a block disassembly process on the virtual texture by a central processing unit to obtain a plurality of virtual texture blocks of the virtual texture; Determining the importance of each virtual texture tile, and determining a virtual texture tile to be rendered based on the importance of each virtual texture tile; Acquire mapping relationship data between the virtual texture block to be rendered and the corresponding physical texture; The central processing unit transmits the virtual texture block to be rendered and the mapping relationship data to the graphics processor, so that the graphics processor loads the physical texture corresponding to the virtual texture block to be rendered based on the mapping relationship data, and performs rendering processing based on the physical texture to obtain a rendered image.

2. The method according to claim 1, characterized in that Determining the importance of each virtual texture tile includes: Based on the first position of the rendering camera, performing a viewing angle confirmation process on the rendering camera to obtain a field of view corresponding to the rendering camera; When the virtual texture tile is outside the field of view, setting the importance of the virtual texture tile to a null value; When the virtual texture tile is located within the field of view, the importance of the virtual texture tile is determined based on the position of the virtual texture tile within the field of view.

3. The method according to claim 2, characterized in that The performing viewing angle confirmation processing on the rendering camera based on the first position of the rendering camera to obtain a field of view corresponding to the rendering camera includes: Drawing a viewing frustum, wherein the vertex of the viewing frustum is the first position of the rendering camera, and the vertical line of the viewing frustum is consistent with the orientation of the rendering camera; The range included in the viewing frustum is used as the field of view corresponding to the rendering camera.

4. The method according to claim 2, characterized in that The determining the importance of the virtual texture tile based on the position of the virtual texture tile within the field of view includes: determining a relative position between the rendering camera and the virtual texture tile based on a first position of the rendering camera and a second position of the virtual texture tile; Based on the relative position, the importance of the virtual texture tile is determined.

5. The method according to claim 4, characterized in that The determining the importance of the virtual texture block based on the relative position includes: Based on the relative position, projecting the virtual texture tile onto a display screen to obtain an area range of a display area of ​​the virtual texture tile on the display screen; Obtaining the importance of the virtual texture block that is positively correlated with the region range.

6. The method according to claim 1, characterized in that Before obtaining the mapping relationship data between the virtual texture block to be rendered and the corresponding physical texture, the method further includes: Determining a candidate physical texture corresponding to the virtual texture tile to be rendered; Selecting a quadtree node corresponding to the importance from the quadtree corresponding to the candidate physical texture; The physical texture corresponding to the quadtree node is used as the physical texture corresponding to the virtual texture block to be rendered.

7. The method according to claim 1, characterized in that Before obtaining the mapping relationship data between the virtual texture tile to be rendered and the corresponding physical texture, the method further includes: Obtaining a first difference between a virtual texture tile to be rendered corresponding to a current frame and a virtual texture tile to be rendered corresponding to a frame before the current frame; When the first difference exceeds a first difference threshold, dividing the virtual texture block to be rendered corresponding to the current frame into multiple groups of virtual texture sub-blocks to be rendered; The virtual texture tile to be rendered is updated to the virtual texture sub-tile to be rendered.

8. The method according to claim 7, characterized in that The obtaining a first difference between a virtual texture tile to be rendered corresponding to a current frame and a virtual texture tile to be rendered corresponding to a frame before the current frame includes: Comparing the virtual texture blocks to be rendered corresponding to the current frame with the virtual texture blocks to be rendered corresponding to a frame before the current frame to obtain a first number of virtual texture blocks to be rendered that are different; The first number is taken as the first difference.

9. The method according to claim 1, characterized in that The mapping relationship data includes an index and an index value; and obtaining the mapping relationship data between the virtual texture block to be rendered and the corresponding physical texture includes: Determining a scaling parameter and a bias parameter based on a mapping relationship between the virtual texture tile to be rendered and a physical texture corresponding to the virtual texture tile to be rendered; The virtual texture tile to be rendered is used as the index, and the scaling parameter and the offset parameter are used as the index value.

10. The method according to claim 1, characterized in that The graphics processor includes a vertex shader; the rendering process based on the physical texture to obtain a rendered image includes: Converting the original coordinate system of the physical texture into a world coordinate system through the vertex shader; Converting the world coordinate system where the physical texture is located into an observation coordinate system through the vertex shader; Based on the physical texture in the observation coordinate system and the virtual texture block to be rendered, the image interface is rendered to obtain the rendered image.

11. The method according to claim 10, characterized in that The graphics processor includes a pixel shader; The rendering process is performed on the image interface based on the physical texture in the observation coordinate system and the virtual texture block to be rendered to obtain the rendered image, including: The pixel shader performs pixel interpolation processing on the physical texture in the observation coordinate system to obtain a pixel value of each pixel point in the graphical interface; The rendered image is generated based on the pixel values ​​of the plurality of pixel points in the graphical interface.

12. An image rendering device, characterized in that: The device comprises: A tile disassembly module is used to perform tile disassembly processing on the virtual texture through a central processing unit to obtain a plurality of virtual texture tiles of the virtual texture; a tile determination module, configured to determine the importance of each virtual texture tile, and determine a virtual texture tile to be rendered based on the importance of each virtual texture tile; A data acquisition module is used to acquire mapping relationship data between the virtual texture block to be rendered and the corresponding physical texture; An image rendering module is used to transmit the virtual texture block to be rendered and the mapping relationship data to the graphics processor through the central processing unit, so that the graphics processor loads the physical texture corresponding to the virtual texture block to be rendered based on the mapping relationship data, and performs rendering processing based on the physical texture to obtain a rendered image.

13. An electronic device, characterized in that: The electronic device comprises: a memory for storing computer-executable instructions; A processor, configured to implement the image rendering method according to any one of claims 1 to 11 when executing the computer-executable instructions stored in the memory.

14. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the image rendering method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising computer executable instructions, characterized in that When the computer executable instructions are executed by a processor, the image rendering method according to any one of claims 1 to 11 is implemented.

Images