A method, apparatus, device, and storage medium for reflecting texture completion

Abstract

The present application discloses a method, apparatus, device, and storage medium for reflection texture completion. The method includes: generating a reflection texture from an image obtained by a main camera based on screen space planar reflection technology; rotating the orientation of the main camera to the reflection orientation, where the original orientation and the reflection orientation of the main camera are mirror images based on the reflection plane; vertically flipping the screen of the main camera and having the main camera render a skybox once, and rendering the obtained image to a texture to obtain a sky texture in the reflection direction; mixing the reflection texture and the sky texture to obtain a completed reflection texture, improving the inherent defect that the screen space planar reflection technology cannot obtain the reflection content outside the screen, resulting in incomplete reflection texture and poor visual effects, as well as the technical problem of poor effect when using the skybox sampling result to repair the reflection texture.

Description

Technical Field

[0001] The present application relates to the technical field of electronic game rendering, and in particular, to a method, device, equipment, and storage medium for completing reflection textures. Background Art

[0002] With the continuous progress of mobile device performance and the continuous development of mobile games, the competition in image quality has become white-hot. Planar reflection, a visual effect that has long been widely used in PC and console platform games, has also become indispensable in mobile games and is very suitable for presenting visual effects such as smooth surfaces, mirrors, and water surface reflections.

[0003] Due to the limited performance of mobile platforms, it is very appropriate to choose screen space planar reflection technology to achieve the reflection visual effect of mobile games, and many games have adopted this technology. The essence of screen space planar reflection technology is to generate reflection textures based on screen images, and then the shader obtains the textures and draws the reflection content, with balanced and stable performance.

[0004] However, screen space planar reflection technology has its own limitations. Since the content outside the screen cannot be known, the incompleteness of the reflection texture is inevitable, which results in poor reflection effects at certain viewing angles. In practical applications, the incomplete part of the reflection texture is often the sky part of the scene. The skybox in the scene can be sampled one more time in the shader, and the sampling result is mixed with the reflection texture generated by the screen space planar reflection technology to complete the reflection content. However, the sampling result of the skybox is not the real reflection result, and the visual effect after mixing is not good. Summary of the Invention

[0005] The present application provides a method, device, equipment, and storage medium for completing reflection textures, which are used to improve the inherent defect that the screen space planar reflection technology cannot obtain the reflection content outside the screen, resulting in incomplete reflection textures and poor visual effects, as well as the technical problem that the effect of using the skybox sampling result to repair the reflection texture is not good.

[0006] In view of this, the first aspect of the present application provides a method for completing reflection textures, including:

[0007] Generating a reflection texture from the image obtained by the main camera based on the screen space planar reflection technology;

[0008] Rotating the orientation of the main camera to the reflection orientation, where the original orientation and the reflection orientation of the main camera are mirror images based on the reflection plane;

[0009] Vertically flipping the image of the main camera and having the main camera render the skybox once, and rendering the obtained image to a texture to obtain the sky texture in the reflection direction;

[0010] Mix the reflection texture and the sky texture to obtain a completed reflection texture.

[0011] Optionally, the vertically flipping the image of the main camera includes:

[0012] Multiply the projection matrix of the main camera by a preset parameter matrix to vertically flip the image of the main camera. The preset parameter matrix is

[0013] Optionally, the method further includes:

[0014] Restore the orientation and projection matrix of the main camera.

[0015] Optionally, the mixing the reflection texture and the sky texture to obtain a completed reflection texture includes:

[0016] Mix the reflection texture and the sky texture through the A channel to obtain a completed reflection texture. The mixing process is:

[0017] refl.rgb = lerp(reflSkybox.rgb, refl.rgb, refl.a);

[0018] In the formula, refl.rgb is the color of the reflection texture, refl.a is the value of the A channel of the reflection texture, reflSkybox.rgb is the color of the sky texture, and lerp() is a linear interpolation function.

[0019] The second aspect of the present application provides a reflection texture completion device, including:

[0020] A reflection texture generation unit, configured to generate a reflection texture from an image obtained by the main camera based on screen space planar reflection technology;

[0021] A rotation unit, configured to rotate the orientation of the main camera to a reflection orientation, where the original orientation and the reflection orientation of the main camera are mirror images based on a reflection plane;

[0022] A sky texture generation unit, configured to vertically flip the image of the main camera, and cause the main camera to render a sky box once, and render the obtained image to a texture to obtain a sky texture in the reflection direction;

[0023] A texture mixing unit, configured to mix the reflection texture and the sky texture to obtain a completed reflection texture.

[0024] Optionally, the sky texture generation unit is specifically configured to:

[0025] Multiply the projection matrix of the main camera by a preset parameter matrix to vertically flip the image of the main camera. The preset parameter matrix is

[0026] Have the main camera render a skybox once, and render the obtained image to a texture to obtain the sky texture in the reflection direction.

[0027] Optionally, the device further includes:

[0028] A restoration unit for restoring the orientation and projection matrix of the main camera.

[0029] Optionally, the texture mixing unit is specifically configured to:

[0030] Mix the reflection texture and the sky texture through the A channel to obtain a completed reflection texture. The mixing process is:

[0031] refl.rgb = lerp(reflSkybox.rgb, refl.rgb, refl.a);

[0032] In the formula, refl.rgb is the color of the reflection texture, refl.a is the value of the A channel of the reflection texture, reflSkybox.rgb is the color of the sky texture, and lerp() is a linear interpolation function.

[0033] A third aspect of the present application provides a reflection texture completion device, which includes a processor and a memory;

[0034] The memory is used to store program code and transmit the program code to the processor;

[0035] The processor is used to execute any one of the reflection texture completion methods described in the first aspect according to the instructions in the program code.

[0036] A fourth aspect of the present application provides a computer-readable storage medium, which is used to store program code, and when the program code is executed by a processor, it implements any one of the reflection texture completion methods described in the first aspect.

[0037] It can be seen from the above technical solutions that the present application has the following advantages:

[0038] The present application provides a method for completing a reflection texture, including: generating a reflection texture from an image obtained by a main camera based on screen space planar reflection technology; rotating the orientation of the main camera to the reflection orientation, where the original orientation and the reflection orientation of the main camera are mirror images based on the reflection plane; vertically flipping the image of the main camera, and having the main camera render a skybox once, and rendering the obtained image onto a texture to obtain a sky texture in the reflection direction; and mixing the reflection texture and the sky texture to obtain a completed reflection texture.

[0039] In the present application, after generating a reflection texture based on screen space planar reflection technology, the orientation of the main camera is rotated to the reflection orientation, and the image of the main camera is vertically flipped. The main camera renders a skybox once, and the obtained image is rendered onto a texture to obtain a sky texture in the reflection direction. By using the method of a reflection camera, the sky texture in the reflection direction is rendered, which is a real reflection result. The visual effect after mixing is very good, improving the inherent defect that the screen space planar reflection technology cannot obtain the reflection content outside the screen, resulting in incomplete reflection texture and poor visual effect, as well as the technical problem that using the skybox sampling result to repair the reflection texture has poor effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0041] Figure 1 It is a schematic flowchart of a method for completing a reflection texture provided by an embodiment of the present application;

[0042] Figure 2 It is a schematic diagram of the orientation of a main camera provided by an embodiment of the present application;

[0043] Figure 3 It is a schematic structural diagram of a device for completing a reflection texture provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] The present application provides a method, device, equipment, and storage medium for completing a reflection texture, which are used to improve the inherent defect that the screen space planar reflection technology cannot obtain the reflection content outside the screen, resulting in incomplete reflection texture and poor visual effect, as well as the technical problem that using the skybox sampling result to repair the reflection texture has poor effect.

[0045] To enable those skilled in the art to better understand the solution of this application, the following will clearly and completely describe the technical solution in the embodiments of this application in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0046] Due to the limited machine performance of the mobile platform, it is very suitable to select the screen space planar reflection technology to achieve the reflection visual effect of mobile games, and this technology has been adopted by many games. The essence of the screen space planar reflection technology is to generate a reflection texture based on the screen image, and then the shader obtains this texture and draws the reflection content, with balanced and stable performance. However, the screen space planar reflection technology has its own limitations. Since the content outside the screen cannot be known, the incompleteness of the reflection texture is inevitable, which results in poor reflection effects in some perspectives. In practical applications, the incomplete part of the reflection texture is often the sky part of the scene. To improve this problem, this application provides a method for complementing the reflection texture, aiming to complement the missing sky part content in the reflection texture generated by the screen space planar reflection technology, thereby optimizing the visual effect.

[0047] For ease of understanding, please refer to Figure 1 , the embodiments of this application provide a method for complementing the reflection texture. This method is implemented based on the Unity engine, and the method includes:

[0048] Step 101: Generate a reflection texture from the image obtained by the main camera based on the screen space planar reflection technology.

[0049] Obtain the screen image through the current main camera of the game in the current orientation, and then based on the screen space planar reflection technology, render the screen image obtained by the current main camera to the texture to generate the reflection texture. Among them, the screen space planar reflection technology belongs to the prior art, and the specific generation process of the reflection texture will not be elaborated here.

[0050] Step 102: Rotate the orientation of the main camera to the reflection orientation, and the original orientation and the reflection orientation of the main camera are mirror images based on the reflection plane.

[0051] Rotate the orientation of the current main camera of the game to the reflection orientation. Among them, the original orientation and the reflection orientation of the main camera are mirror images based on the reflection plane, and reference can be made to Figure 2 . This approach is similar to that of a reflection camera, but does not require adjusting the position of the main camera because only the content of the skybox needs to be rendered here, and the camera is always at the center of the skybox.

[0052] Step 103: Vertically flip the image of the main camera, and let the main camera render the skybox once. Render the obtained image to the texture to obtain the sky texture in the reflection direction.

[0053] After rotating the orientation of the main camera, vertically flip the image of the main camera. This can be achieved by multiplying the projection matrix of the main camera by a preset parameter matrix to vertically flip the image of the main camera.

[0054] After vertically flipping the image of the main camera, let the main camera render the skybox once, obtain the image currently captured by the main camera, and render this image to the texture to obtain the sky texture in the reflection direction. The sky texture obtained at this time is the real reflection result.

[0055] Furthermore, after obtaining the sky texture, restore the orientation and projection matrix of the main camera.

[0056] Step 104: Blend the reflection texture and the sky texture to obtain the completed reflection texture.

[0057] After obtaining the sky texture, it is necessary to determine how to blend the sky texture and the reflection texture generated by the screen space planar reflection technology. The embodiments of this application find that the A channel of the reflection texture indicates where the texture has actual reflection content. Therefore, the reflection texture and the sky texture can be blended through the A channel. In the A channel, the areas closer to 1 (i.e., whiter) tend to draw the content of the reflection texture, and the areas closer to 0 (i.e., blacker) tend to draw the content of the sky texture. The embodiments of this application use an interpolation method to blend the reflection texture and the sky texture. Specifically:

[0058] refl.rgb = lerp(reflSkybox.rgb, refl.rgb, refl.a);

[0059] In the formula, refl.rgb is the color of the reflection texture, refl.a is the value of the A channel of the reflection texture, reflSkybox.rgb is the color of the sky texture, and lerp() is a linear interpolation function.

[0060] After blending the sky texture and the reflection texture, draw the reflection content through the shaders of relevant scene objects. By comparison, it is found that the effect after completing the sky texture is much more beautiful than the original effect.

[0061] The performance cost and visual performance of screen-space planar reflection technology are relatively balanced. Reflection cameras have high performance costs but excellent visual performance. The embodiments of this application utilize the concept of reflection cameras to optimize and complement the distorted portions of the reflection texture generated by screen-space planar reflection technology within an acceptable performance range, thereby improving visual effects. Furthermore, the complementation method in this embodiment of the application offers developers greater controllability and adjustable rendering efficiency. Compared to sampling the skybox once more in the shader, this application allows developers to balance image quality accuracy with performance cost, adjusting the resolution and skybox effects when rendering the sky texture, thereby achieving a balance between visual effects and rendering efficiency. Since most game scenes have simple skybox tones and reflection textures do not require high resolution, the resolution of the sky texture can be reduced to reduce rendering time. Testing has shown that when the sky texture resolution is reduced to 0.04 times the original resolution, the visual effect of planar reflections remains very good. At this point, the GPU time required to render the sky texture is only 18% to 30% of the time required to render the scene skybox. This reasonable increase in time is exchanged for a significant improvement in the planar reflection effect.

[0062] In an embodiment of the present application, after generating a reflection texture based on screen space planar reflection technology, the orientation of the main camera is rotated to the reflection orientation, and the image of the main camera is flipped vertically. The main camera renders the sky box once, and the acquired image is rendered to the texture to obtain the sky texture in the reflection direction. By using the reflection camera, the sky texture in the reflection direction is rendered, which is a real reflection result. The mixed visual effect is very good, which improves the inherent defect of using screen space planar reflection technology to be unable to obtain the reflection content outside the screen, resulting in incomplete reflection texture and poor visual effect, as well as the technical problem of using sky box sampling results to repair the reflection texture with poor effect.

[0063] The above is an embodiment of a reflection texture completion method provided by the present application, and the following is an embodiment of a reflection texture completion device provided by the present application.

[0064] Please refer to Figure 3 , an embodiment of the present application provides a reflective texture completion device, comprising:

[0065] a reflection texture generation unit, configured to generate a reflection texture from an image acquired from a main camera based on a screen space planar reflection technique;

[0066] A rotation unit, configured to rotate the orientation of the main camera to a reflected orientation, wherein the original orientation of the main camera and the reflected orientation are mirror images based on a reflection plane;

[0067] A sky texture generation unit for vertically flipping the image of the main camera, rendering the skybox once with the main camera, and rendering the acquired image onto a texture to obtain the sky texture in the reflection direction;

[0068] A texture blending unit for blending the reflection texture and the sky texture to obtain the completed reflection texture.

[0069] In the embodiment of the present application, the reflection texture generation unit obtains the screen image through the current main camera of the game in the current orientation, and then renders the screen image acquired by the current main camera onto a texture based on the screen space planar reflection technology to generate the reflection texture. Among them, the screen space planar reflection technology belongs to the prior art, and the specific generation process of the reflection texture will not be elaborated here.

[0070] The rotation unit rotates the orientation of the main camera to the reflection orientation, where the original orientation and the reflection orientation of the main camera are mirror images based on the reflection plane. The sky texture generation unit multiplies the projection matrix of the main camera by a preset parameter matrix to vertically flip the image of the main camera. The preset parameter matrix is Render the skybox once with the main camera, and render the acquired image onto a texture to obtain the sky texture in the reflection direction. The sky texture acquired at this time is the real reflection result.

[0071] As a further improvement, the device further includes:

[0072] A restoration unit for restoring the orientation and projection matrix of the main camera. After the sky texture generation unit acquires the sky texture, the restoration unit restores the orientation and projection matrix of the main camera.

[0073] After acquiring the sky texture, it is necessary to determine how to blend the sky texture and the reflection texture generated by the screen space planar reflection technology. The embodiment of the present application finds that the A channel of the reflection texture identifies where the texture has actual reflection content. Therefore, the reflection texture and the sky texture can be blended through the A channel. The closer to 1 (i.e., the whiter) in the A channel, the more likely it is to draw the content of the reflection texture, and the closer to 0 (i.e., the blacker), the more likely it is to draw the content of the sky texture. Therefore, the texture blending unit blends the reflection texture and the sky texture through the A channel to obtain the completed reflection texture. The blending process is:

[0074] refl.rgb = lerp(reflSkybox.rgb, refl.rgb, refl.a);

[0075] In the formula, refl.rgb is the color of the reflection texture, refl.a is the value of the A channel of the reflection texture, reflSkybox.rgb is the color of the sky texture, and lerp() is the linear interpolation function.

[0076] In the embodiment of the present application, after generating the reflection texture based on the screen space planar reflection technology, the orientation of the main camera is rotated to the reflection orientation, and the image of the main camera is vertically flipped. Then, the main camera renders the skybox once, and the obtained image is rendered to the texture to obtain the sky texture in the reflection direction. By using the method of the reflection camera, the sky texture in the reflection direction is rendered, which is a real reflection result, and the mixed visual effect is very good. This improves the inherent defect that the screen space planar reflection technology cannot obtain the reflection content outside the screen, resulting in incomplete reflection texture and poor visual effect, as well as the technical problem that using the skybox sampling result to repair the reflection texture has poor effect.

[0077] The embodiment of the present application also provides a reflection texture completion device, which includes a processor and a memory;

[0078] The memory is used to store program codes and transmit the program codes to the processor;

[0079] The processor is used to execute the reflection texture completion method in the foregoing method embodiment according to the instructions in the program codes.

[0080] The embodiment of the present application also provides a computer-readable storage medium, which is used to store program codes. When the program codes are executed by a processor, the reflection texture completion method in the foregoing method embodiment is implemented.

[0081] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0082] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0083] It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Among them, A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (one)" or a similar expression below refers to any combination of these items, including any combination of single item (one) or plural items (ones). For example, at least one (one) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0084] In several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical or other forms.

[0085] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0086] In addition, the functional units in each embodiment of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0087] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (English full name: Read-Only Memory, English abbreviation: ROM), random access memories (English full name: Random Access Memory, English abbreviation: RAM), magnetic disks, or optical discs.

[0088] As described above, the above embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of various embodiments of this application.

Claims

1. A method for completing a reflection texture, characterized in that including: generating a reflection texture from an image obtained by a main camera based on screen space planar reflection technology; rotating the orientation of the main camera to a reflection orientation, where the original orientation and the reflection orientation of the main camera are mirror images based on a reflection plane; vertically flipping the image of the main camera, and having the main camera render a skybox once, and rendering the obtained image to a texture to obtain a sky texture in the reflection direction; mixing the reflection texture and the sky texture to obtain a completed reflection texture.

2. The reflection texture completion method according to claim 1, wherein The vertically flipping the image of the main camera includes: Multiply the projection matrix of the main camera by a preset parameter matrix to vertically flip the image of the main camera. The preset parameter matrix is 3. The method for complementing the reflection texture according to claim 2, wherein The method further includes: restoring the orientation and projection matrix of the main camera.

4. The reflection texture completion method according to claim 1, characterized in that The mixing the reflection texture and the sky texture to obtain a completed reflection texture includes: mixing the reflection texture and the sky texture through the A channel to obtain a completed reflection texture, and the mixing process is: refl.rgb = lerp(reflSkybox.rgb, refl.rgb, refl.a); In the formula, refl.rgb is the color of the reflection texture, refl.a is the value of the A channel of the reflection texture, reflSkybox.rgb is the color of the sky texture, and lerp() is a linear interpolation function.

5. A reflection texture completion device, characterized in that, including: a reflection texture generation unit for generating a reflection texture from an image obtained by a main camera based on screen space planar reflection technology; a rotation unit for rotating the orientation of the main camera to a reflection orientation, where the original orientation and the reflection orientation of the main camera are mirror images based on a reflection plane; a sky texture generation unit for vertically flipping the image of the main camera, and having the main camera render a skybox once, and rendering the obtained image to a texture to obtain a sky texture in the reflection direction; a texture mixing unit for mixing the reflection texture and the sky texture to obtain a completed reflection texture.

6. The reflection texture completion device according to claim 5, wherein The sky texture generation unit is specifically used for: Multiply the projection matrix of the main camera by a preset parameter matrix to vertically flip the image of the main camera. The preset parameter matrix is having the main camera render a skybox once, and rendering the obtained image to a texture to obtain a sky texture in the reflection direction.

7. The reflection texture completion device according to claim 6, wherein The apparatus further includes: a restoration unit for restoring the orientation and projection matrix of the main camera.

8. The reflection texture completion device according to claim 6, characterized in that, The texture mixing unit is specifically used for: mixing the reflection texture and the sky texture through the A channel to obtain a completed reflection texture, and the mixing process is: refl.rgb = lerp(reflSkybox.rgb, refl.rgb, refl.a); In the formula, refl.rgb is the color of the reflection texture, refl.a is the value of the A channel of the reflection texture, reflSkybox.rgb is the color of the sky texture, and lerp() is a linear interpolation function.

9. A reflection texture completion device, characterized in that, The device includes a processor and a memory; The memory is used for storing program code and transmitting the program code to the processor; The processor is used for executing the reflection texture completion method according to any one of claims 1-4 based on the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used for storing program code, and when the program code is executed by a processor, the reflection texture completion method according to any one of claims 1-4 is implemented.

Images