A real-time plane reflection rendering method and device for mobile terminal
By generating screen space reflection coordinates and UV textures on the mobile terminal, and using MRT technology to transfer the reflection effect to the screen post-processing stage, the hardware resource pressure and unrealistic effect of real-time plane reflection rendering in the mobile terminal in the existing technology is solved, and the problem of unrealistic effect is realized. Efficient and flexible rendering effects.
Patent Information
- Application Number
- CN202210041160.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-14
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-01-14
AI Technical Summary
The prior art is difficult to achieve efficient and flexible real-time plane reflection rendering on mobile terminals, especially because reflecting CubeMap, camera rendering reflection maps and screen space reflection technologies have great pressure on hardware resources and calculation volume, and the effect is not realistic enough.
By generating screen space reflection coordinates and UV textures of the target object, the multi-rendering target (MRT) technology is used to transfer the reflection effect to the post-screen processing stage, directly synthesizing the reflection results, reducing the use of additional computing resources.
Real reflection effect is achieved, while reducing the hardware resource pressure on mobile devices when rendering three-dimensional scenes in real time, improving rendering performance and flexibility.
Smart Images

Figure CN114419235B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of computer graphics rendering, and in particular to a real-time planar reflection rendering method and apparatus for a mobile terminal, a computing device, and a computer-readable storage medium. Background Art
[0002] In a 3D scene, some important objects such as characters and buildings require more realistic lighting effects, especially plane reflection effects. When rendering, the existing technology usually uses three methods: reflection CubeMap, camera rendering reflection map, and screen space reflection. However, each of these three solutions has corresponding defects and is not suitable for implementing real-time plane reflection rendering in mobile 3D applications. In order to adapt to mobile hardware resources and have better plane reflection effects, a faster, more efficient and flexible real-time plane reflection solution is needed. Summary of the invention
[0003] In view of this, the embodiments of the present application provide a real-time planar reflection rendering method and apparatus for a mobile terminal, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.
[0004] According to a first aspect of an embodiment of the present application, a real-time plane reflection rendering method for a mobile terminal is provided, comprising:
[0005] Generate the screen space reflection coordinates of the target object according to the reflection plane and normal direction;
[0006] Generate a screen space reflection UV texture according to the screen space reflection coordinates of the target object;
[0007] Generates the reflection effect of the target object based on the screen space reflection UV texture.
[0008] According to a second aspect of an embodiment of the present application, a real-time plane reflection rendering device for a mobile terminal is provided, comprising:
[0009] A reflection coordinate generation module, used for generating screen space reflection coordinates of a target object according to a reflection plane and a normal direction;
[0010] A reflection UV texture generation module, used for generating a screen space reflection UV texture according to the screen space reflection coordinates of the target object;
[0011] The reflection effect generation module is used to generate the reflection effect of the target object according to the screen space reflection UV texture.
[0012] According to a third aspect of an embodiment of the present application, a computing device is provided, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein when the processor executes the instructions, the steps of the real-time planar reflection rendering method for a mobile terminal are implemented.
[0013] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer instructions, and when the instructions are executed by a processor, the steps of the real-time planar reflection rendering method for a mobile terminal are implemented.
[0014] Through the embodiments of the present application, the reflection UV texture is obtained by acquiring and converting the reflection coordinates of the target object, and the generation of the reflection effect is further transferred to the screen post-processing stage by means of MRT. In the screen post-processing stage, the reflection UV texture in the MRT is used to sample the normally rendered texture to obtain the reflection rendering result. The reflection effect achieved by this method is not only realistic, but also directly synthesized in the screen post-processing stage without the need for additional computing resources. This is significantly different from the existing SSPR technology, and is particularly suitable for the rendering requirements of mobile terminals, reducing the hardware resource pressure of mobile terminals when rendering three-dimensional scenes in real time. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 is a structural block diagram of a computing device provided in an embodiment of the present application;
[0016] Figure 2 It is a schematic diagram of the plane reflection rendering effect in the prior art;
[0017] Figure 3 It is a schematic diagram of a process of a real-time plane reflection rendering method for a mobile terminal provided in an embodiment of the present application;
[0018] Figure 4 is a schematic diagram of how to calculate the reflection coordinates of a target object provided by an embodiment of the present application;
[0019] Figure 5 It is a schematic diagram of an effect of plane reflection achieved by the real-time plane reflection rendering method for a mobile terminal provided in an embodiment of the present application;
[0020] Figure 6 It is a structural schematic diagram of a real-time planar reflection rendering device for a mobile terminal provided in an embodiment of the present application. DETAILED DESCRIPTION
[0021] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.
[0022] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.
[0023] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "in response to determination".
[0024] In the present application, a real-time planar reflection rendering method and apparatus for a mobile terminal, a computing device, and a computer-readable storage medium are provided, which are described in detail one by one in the following embodiments.
[0025] Figure 1 The structure block diagram of a computing device 100 according to an embodiment of the present application is shown. The components of the computing device 100 include but are not limited to a memory 110 and a processor 120. The processor 120 is connected to the memory 110 via a bus 130, and the database 150 is used to store data.
[0026] The computing device 100 also includes an access device 140 that enables the computing device 100 to communicate via one or more networks 160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 140 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.
[0027] In one embodiment of the present application, the above components of the computing device 100 and Figure 1 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 1 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.
[0028] The computing device 100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 100 may also be a mobile or stationary server.
[0029] In the prior art, in the field of 3D rendering, especially in 3D scenes of games, real-time plane reflection rendering is a common scenario. As a part of global illumination, it can ensure the correctness of the lighting environment and improve the lighting effect. Figure 2 As shown. In mainstream game engines, three technologies are usually used to realize plane reflection, namely reflection CubeMap, camera rendering reflection map, and screen space reflection. Among them, reflection CubeMap generates a hexahedral cube map for the scene, which is time-consuming and usually used in static reflection. In addition, the reflection effect in the plane area will be out of place. Camera rendering reflection map requires an additional camera to re-render the objects that need to be reflected in the scene into a texture map. The camera itself will cause a large additional overhead. Rendering the scene again will double the DrawCall, and the reflection map Figure 1 Generally, the lighting calculation will be simplified, and the effect will be poor. Screen space reflection technology (SSR) requires ray tracing or ray stepping algorithms, which are very computationally intensive and cannot reflect individual objects for optimization, which is very limited on mobile devices.
[0030] Recently, in order to solve the defects of the above-mentioned prior art, a screen space planar reflection technology (SSPR) has emerged. Compared with SSR, SSPR eliminates the high consumption of ray stepping, greatly improves performance, and does not double the drawcall compared to planar reflection. However, the idea of SSPR technology is to reconstruct the world coordinates through the depth map, then flip the coordinates through a predetermined height, and then use the flipped screen coordinates to correspond to the screen color before flipping. The implementation of these steps depends on ComputeShader technology, such as the prior art CN112233216A, "URP Rendering Pipeline-Screen Space Planar Reflection": https: / / zhuanlan.zhihu.com / p / 357714920, etc. This brings new problems. For example, ComputeShader technology is not supported by all mobile devices, and the cost of using ComputeShader to calculate the flipped coordinates is very high, which puts higher requirements on the hardware resources of the mobile terminal, and due to the disorder of ComputeShader execution, the reflected texture will flicker.
[0031] In the embodiment of the present application, in order to solve the above problems, a real-time plane reflection rendering method and apparatus for a mobile terminal, a computing device and a computer-readable storage medium are proposed. The processor 120 can execute Figure 3 The steps in the real-time plane reflection rendering method for mobile terminals are shown. Figure 3 , a flowchart of a real-time planar reflection rendering method for a mobile terminal is shown, including steps 202 to 208.
[0032] Step 302: Generate screen space reflection coordinates of the target object according to the reflection plane and the normal direction.
[0033] In a specific implementation, for a target object that needs to be reflected in real time, in the vertex shader, the coordinates of the target object after reflection projection are calculated according to the pre-input reflection plane position and normal direction. Figure 4 As shown, S is the reflection plane and P is the target object for which the reflection effect needs to be generated. In this step, the screen space coordinates of the target object P are obtained, and the coordinates of the reflected object P' are generated according to the position of the reflection plane S and the direction of the normal.
[0034] Furthermore, a new Pass channel is created in the vertex shader, and the coordinate calculation of the object P' after reflection is implemented in the Pass channel.
[0035]
[0036] In this step, by obtaining the screen space reflection coordinates of the target object, it is possible to individually control whether each object needs reflection, which is more flexible and only requires an additional Pass channel. There is no need to add new camera cropping or reflection calculation of the entire screen space, which reduces the computing pressure on mobile devices.
[0037] Step 304: Generate a screen space reflection UV texture according to the screen space reflection coordinates of the target object.
[0038] In a specific implementation, the vertex coordinates of the target object are transformed into screen space UV coordinates through projection transformation and perspective division to obtain the position of the vertex coordinates of the target object in the screen texture.
[0039]
[0040] In the field of computer graphics technology, vertex coordinates, UV coordinates, world coordinates and other coordinates are conventional technologies in the field and will not be described in detail here.
[0041] Furthermore, in the fragment shader, the screen space UV coordinates of the target object are written into the reflection space formed by the screen space reflection coordinates of the target object to generate a reflection UV texture.
[0042] Furthermore, the above reflection UV texture is written into one of the buffers of MRT, wherein the other buffer of MRT saves the texture map normally rendered in the original screen.
[0043] Multiple Render Target (MRT) technology allows a rendering program to bind multiple rendering targets in one drawing call, and then write different rendering data into different buffers.
[0044] In this step, the UV texture coordinates of the target object are obtained through coordinate conversion, and the coordinates are used together with the reflection coordinates calculated in step 302 to generate a reflection UV texture, which is written into a buffer in the MRT, thereby achieving the acquisition of the screen space reflection UV texture.
[0045] Step 306: Generate a reflection rendering effect according to the screen space reflection UV texture.
[0046] In a specific implementation, based on the screen space reflection UV texture of the MRT buffer, the screen texture of another MRT buffer is sampled to obtain the screen pixels after reflection;
[0047] Furthermore, the reflected pixels are mixed with the current screen pixels to obtain the reflection effect of the target object.
[0048] Furthermore, the above step 306 is implemented in the screen post-processing stage. The screen post-processing stage is a stage in which a series of processes are performed on the screen rendering after the screen rendering is obtained. This term is well known to those skilled in the art and will not be described in detail here.
[0049] In this step, the object sampled according to the reflection UV texture is the screen buffer texture rendered by the original scene, so the generated reflection effect is realistic. Figure 5 The reflection effect shown is achieved by the method of the embodiment of the present application. At the same time, the generation step is directly synthesized in the screen post-processing stage, without the need for additional calculation and generation of Pass channels, further reducing the resource consumption of the mobile terminal.
[0050] In the above-mentioned embodiment of the real-time planar reflection rendering method for mobile terminals, the screen space coordinates after reflection are first calculated according to the screen space coordinates of the target to be reflected, so that each object can be individually controlled whether it needs to be reflected, which is more flexible and has a small amount of calculation, reducing the calculation pressure of the mobile terminal device. The obtained screen space coordinates after reflection are further converted into UV texture coordinates, and the two coordinates are used to generate a reflective UV texture and write it into the MRT together with the normally rendered texture. In the screen post-processing stage, the reflective UV texture in the MRT is used to sample the normally rendered texture to obtain a reflection effect. The reflection effect achieved by this method is not only realistic, but also directly synthesized in the screen post-processing stage without the need for additional computing resources. This is significantly different from the existing SSPR technology, and is particularly suitable for the rendering requirements of mobile terminals, reducing the hardware resource pressure of the mobile terminal when rendering three-dimensional scenes in real time.
[0051] Corresponding to the above method embodiment, the present application also provides an embodiment of a real-time plane reflection rendering device for a mobile terminal, Figure 6 FIG. 1 is a schematic diagram showing a real-time plane reflection rendering device for a mobile terminal according to an embodiment of the present application. Figure 6 As shown, the device comprises:
[0052] A reflection coordinate generation module, used for generating screen space reflection coordinates of a target object according to a reflection plane and a normal direction;
[0053] A reflection UV texture generation module, used for generating a screen space reflection UV texture according to the screen space reflection coordinates of the target object;
[0054] The reflection effect generation module is used to generate the reflection effect of the target object according to the screen space reflection UV texture.
[0055] The above is a schematic scheme of a real-time plane reflection rendering device for a mobile terminal of this embodiment. It should be noted that the technical scheme of the real-time plane reflection rendering device for a mobile terminal and the technical scheme of the real-time plane reflection rendering method for a mobile terminal belong to the same concept, and the details not described in detail in the technical scheme of the real-time plane reflection rendering device for a mobile terminal can be found in the description of the technical scheme of the real-time plane reflection rendering method for a mobile terminal.
[0056] In one embodiment of the present application, a computing device is also provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein when the processor executes the instructions, the steps of the real-time planar reflection rendering method for a mobile terminal are implemented.
[0057] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned real-time planar reflection rendering method for mobile terminals belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned real-time planar reflection rendering method for mobile terminals.
[0058] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the real-time planar reflection rendering method for a mobile terminal as described above.
[0059] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the real-time planar reflection rendering method for mobile terminals are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the real-time planar reflection rendering method for mobile terminals.
[0060] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0061] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0062] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0063] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0064] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.
Claims
1. A real-time plane reflection rendering method for a mobile terminal, characterized in that: include: Generate the screen space reflection coordinates of the target object according to the reflection plane and normal direction; Generating a screen space reflection UV texture according to the screen space reflection coordinates of the target object, including: Transform the vertex coordinates of the target object into screen space UV coordinates; write the screen space UV coordinates of the target object into a reflection space formed by the screen space reflection coordinates of the target object to generate a reflection UV texture; write the reflection UV texture into a buffer of a multi-rendering target MRT; write the texture map normally rendered in the original screen into another buffer of the multi-rendering target MRT; According to the screen space reflection UV texture, a reflection effect of a target object is generated in a screen post-processing stage, including: according to the screen space reflection UV texture of the MRT buffer, the screen texture of another MRT buffer is sampled to obtain screen pixels after reflection; and the screen pixels after reflection are mixed with the current screen pixels to obtain a rendering result of the reflection effect.
2. The method according to claim 1, wherein generating screen space reflection coordinates of the target object further comprises: In the vertex shader, the screen space reflection coordinates of the target object are calculated in the newly created Pass channel according to the reflection plane position and normal direction.
3. A real-time plane reflection rendering device for a mobile terminal, characterized in that: include: A reflection coordinate generation module, used for generating screen space reflection coordinates of a target object according to a reflection plane and a normal direction; A reflection UV texture generation module is used to generate a screen space reflection UV texture according to the screen space reflection coordinates of the target object, including: Transform the vertex coordinates of the target object into screen space UV coordinates; write the screen space UV coordinates of the target object into a reflection space formed by the screen space reflection coordinates of the target object to generate a reflection UV texture; write the reflection UV texture into a buffer of a multi-rendering target MRT; write the texture map normally rendered in the original screen into another buffer of the multi-rendering target MRT; The reflection effect generation module is used to generate the reflection effect of the target object in the screen post-processing stage according to the screen space reflection UV texture, including: According to the screen space reflection UV texture of the MRT buffer, the screen texture of another MRT buffer is sampled to obtain the screen pixels after reflection; the screen pixels after reflection are mixed with the current screen pixels to obtain the rendering result of the reflection effect.
4. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the instructions, the steps of the method according to any one of claims 1 to 2 are implemented.
5. A computer-readable storage medium storing computer instructions, characterized in that: When the instruction is executed by a processor, the steps of the method described in any one of claims 1 to 2 are implemented.
Citation Information
Patent Citations
Game image processing method and device and electronic equipment
CN112233216A
CPU-based graphic rendering engine and realization method
CN102722861A
Real-time soft shadow realization method based on image screen space
CN104103089A
Real-time reflection method based on screen space
CN104240286A
Model rendering method, device and equipment and storage medium
CN112316420A
