Image rendering method and device, storage medium and electronic equipment

Abstract

The invention discloses an image rendering method and device, a storage medium and electronic equipment. The method comprises the steps that baking operation is conducted on a virtual object model based on the light source direction of a virtual light source, a first color map, a first depth map and a first coordinate set are obtained, the virtual object model is a three-dimensional model, and the virtual object model is within the shooting range of a first virtual camera; coordinates of all points in the virtual object model under the target coordinate system are obtained, and a second coordinate set is obtained; according to the first depth map, the first coordinate set and the second coordinate set, shadow information corresponding to the virtual object model is determined, and the shadow information is used for representing shadow values of all points in the virtual object model under the target coordinate system; and according to the first color map, the first coordinate set and the shadow information, performing rendering processing on the virtual object model to obtain a first target rendering image. The technical problem of low efficiency in the image rendering process is solved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image rendering method and device, a storage medium, and an electronic device. Background Art

[0002] In the display technology of virtual scenes, the virtual object model is usually rendered at different angles by using an artist to make an insert model. The main process includes: using tools to bake pictures of the virtual object model at different angles, and then storing three sets of atlases for color, roughness and normal respectively; for a virtual object (for example, a tree), the corresponding virtual object model is rendered in the form of multiple inserts, and then the rendered image is displayed.

[0003] However, there are at least the following problems with the above-mentioned image rendering method: (1) During the baking operation of the virtual object model, the color information of the original image is directly stored without calculating the lighting, that is, the stored color information does not include the effect after being illuminated. Therefore, the lighting needs to be calculated in real time during the rendering process, which increases the calculation cost; (2) During the game production process, the baking operation is performed, and all the baked maps of multiple angles of the baking operation are stored, which occupies a large amount of storage resources and causes the virtual game package to become larger; (3) Multiple inserts are required to render the virtual object model, which consumes a lot of resources.

[0004] It can be seen that the above-mentioned image rendering method not only takes a lot of time to calculate the lighting in real time, but also takes up additional disk space to store the baking results generated by the baking operation performed during the game production process, which increases the computing cost and image processing time, thereby causing the technical problem of low effect in the image rendering process.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present application provide an image rendering method and device, a storage medium, and an electronic device to at least solve the technical problem of low efficiency that occurs during the image rendering process.

[0007] According to one aspect of an embodiment of the present application, an image rendering method is provided, comprising: performing a baking operation on a virtual object model based on a light source direction of a virtual light source to obtain a first color map, a first depth map, and a first coordinate set, wherein the virtual object model is a three-dimensional model, the virtual object model is within the shooting range of a first virtual camera, the first color map includes color values ​​of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model according to the first light source direction, the first depth map is used to represent the distance between each point in the virtual object model and the first virtual camera, and the first coordinate set is used to represent the position of each point in the virtual object model; obtaining the coordinates of each point in the virtual object model in a target coordinate system to obtain a second coordinate set; determining shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent the shadow value of each point in the virtual object model in the target coordinate system; rendering the virtual object model according to the first color map, the first coordinate set, and the shadow information to obtain a first target rendered image.

[0008] Optionally, the above-mentioned determining the shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set and the second coordinate set includes: determining the i-th shadow value in the shadow information by the following steps, wherein the i-th shadow value is the shadow value of the i-th point in the virtual object model in the target coordinate system, and i is a positive integer greater than or equal to 1: according to the i-th coordinate in the first coordinate set, obtaining the i-th depth value corresponding to the i-th point in the first depth map, wherein the i-th coordinate is used to indicate the position of the i-th point, and the i-th depth value is used to indicate the distance between the i-th point in the virtual object model and the first virtual The distance between the cameras; according to the parameters in the target projection matrix, the i-th depth value is restored to obtain the i-th restored depth value, wherein the target projection matrix is ​​the projection matrix used when the virtual object model is baked, and the target projection matrix is ​​used to project the three-dimensional virtual object model into a two-dimensional first color map; according to the i-th restored depth value and the i-th initial coordinate in the second coordinate set, the i-th shadow value is determined, wherein the i-th initial coordinate is used to represent the position of the i-th point in the virtual object model in the target coordinate system, and the coordinates in the second coordinate set are three-dimensional coordinates.

[0009] Optionally, the above-mentioned obtaining the i-th depth value corresponding to the i-th point in the first depth map according to the i-th coordinate in the first coordinate set includes: searching the first depth map for the point represented by the i-th coordinate; and determining the i-th depth value as the depth value of the found point.

[0010] Optionally, the above-mentioned restoring operation is performed on the i-th depth value according to the parameters in the target projection matrix to obtain the i-th restored depth value, including: when the parameters in the target projection matrix include the near clipping distance and the far clipping distance, the i-th restored depth value is determined to be equal to the sum of the target product value and the near clipping distance, the target product value is equal to the product of the i-th depth value and the distance difference value, the distance difference value is the difference obtained by subtracting the near clipping distance from the far clipping distance, the near clipping distance is used to represent the distance between the near clipping plane used when projecting the three-dimensional virtual object model into a two-dimensional first color map and the first virtual camera, and the far clipping distance is used to represent the distance between the far clipping plane used when projecting the three-dimensional virtual object model into the two-dimensional first color map and the first virtual camera.

[0011] Optionally, the above determining the i-th shadow value based on the i-th restored depth value and the i-th initial coordinate in the second coordinate set includes: determining the i-th restored coordinate based on the i-th restored depth value, the i-th sight direction vector, and the i-th initial coordinate, wherein the i-th sight direction vector is used to represent the direction vector between the second virtual camera to the i-th initial coordinate in the target coordinate system; determining the i-th shadow value based on the i-th restored coordinate.

[0012] Optionally, the above-mentioned determining the i-th restored coordinate based on the i-th restored depth value, the i-th sight direction vector, and the i-th initial coordinate includes: normalizing the i-th sight direction vector to obtain a normalized vector; determining the i-th restored coordinate to be equal to the sum of the i-th initial coordinate and a target vector, wherein the target vector is a vector obtained by multiplying the normalized vector by the i-th restored depth value.

[0013] Optionally, the determining the i-th shadow value according to the i-th restored coordinates includes: determining the shadow value of the i-th point having a mapping relationship with the i-th restored coordinates according to a preset mapping relationship.

[0014] Optionally, the above-mentioned rendering processing of the virtual object model according to the first color map, the first coordinate set and the shadow information to obtain the first target rendered image includes: determining the final color value of the i-th point in the virtual object model in the target coordinate system through the following steps, and determining the first target rendered image according to the virtual object model with the final color value determined in the target coordinate system, wherein i is a positive integer greater than or equal to 1: obtaining the i-th color value corresponding to the i-th point in the first color map according to the i-th coordinate in the first coordinate set, wherein the i-th coordinate is used to indicate the position of the i-th point, and the i-th color value is used to indicate the color value of the i-th point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model in the direction of the first light source; determining the final color value of the i-th point according to the i-th color value and the i-th shadow value in the shadow information, wherein the i-th shadow value is the shadow value of the i-th point in the virtual object model in the target coordinate system.

[0015] Optionally, the above-mentioned obtaining the i-th color value corresponding to the i-th point in the first color map according to the i-th coordinate in the first coordinate set includes: searching the point represented by the i-th coordinate in the first color map; and determining the i-th color value as the color value of the found point.

[0016] Optionally, determining the final color value of the ith point according to the ith color value and the ith shadow value in the shadow information includes: determining the final color value of the ith point to be equal to the product of the ith color value and the ith shadow value.

[0017] Optionally, determining the first target rendered image according to the virtual object model with a final color value determined in the target coordinate system includes: converting the virtual object model with a final color value determined into a two-dimensional first target rendered image.

[0018] Optionally, the method further includes: when the virtual light source illuminates the virtual object model in the direction of a second light source and the virtual object model is located in the shooting range of the first virtual camera, performing a baking operation on the virtual object model to obtain a second color map, a first depth map and a first coordinate set, wherein the second color map includes the color values ​​of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model in the direction of the second light source, and the angle between the second light source direction and the first light source direction is greater than or equal to a first preset threshold; determining shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set and the second coordinate set, wherein the shadow information is used to represent the shadow values ​​of each point in the virtual object model in the target coordinate system; rendering the virtual object model according to the second color map, the first coordinate set and the shadow information to obtain a second target rendered image.

[0019] Optionally, the method further includes: when the angle between the line between the first virtual camera and the center point of the virtual object model and the horizontal plane is converted from the first angle to the second angle, baking the virtual object model to obtain a third color map, a second depth map and a third coordinate set, wherein the angle difference between the first angle and the second angle is greater than or equal to a second preset threshold, and the horizontal plane is the plane where the virtual object model is located in the shooting range of the first virtual camera; determining shadow information corresponding to the virtual object model according to the third color map, the second coordinate set and the third coordinate set; and rendering the virtual object model according to the third color map, the third coordinate set and the shadow information to obtain a third target rendered image.

[0020] According to another aspect of an embodiment of the present application, an image rendering device is further provided, including: a first processing unit, configured to perform a baking operation on a virtual object model based on a light source direction of a virtual light source to obtain a first color map, a first depth map, and a first coordinate set, wherein the virtual object model is a three-dimensional model, the virtual object model is within the shooting range of a first virtual camera, the first color map includes color values ​​of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model in the first light source direction, the first depth map is used to represent the distance between each point in the virtual object model and the first virtual camera, and the first coordinate set is used to represent the position of each point in the virtual object model; a first acquisition unit, configured to acquire the coordinates of each point in the virtual object model in a target coordinate system to obtain a second coordinate set; a second processing unit, configured to determine shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent the shadow values ​​of each point in the virtual object model in the target coordinate system; a first rendering unit, configured to render the virtual object model according to the first color map, the first coordinate set, and the shadow information to obtain a first target rendered image.

[0021] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned image rendering method when running.

[0022] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0023] According to another aspect of the embodiments of the present application, there is further provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the image rendering method through the computer program.

[0024] Through the above-mentioned embodiments provided by the present application, by performing a baking operation on the virtual object model, a first depth map and a first color map containing the effect after being illuminated are obtained, and at the same time, the UV texture coordinates of each point in the virtual object model are obtained; the coordinates of each point in the virtual object model in the target coordinate system are obtained using the first depth map, and the shadow information corresponding to the virtual object model is calculated, and finally the virtual object model is rendered to obtain the first target rendering model. In other words, by calculating the illumination during the baking operation, a large amount of computing resources consumed by real-time calculation of illumination during the image rendering process is avoided, the overall performance of image rendering is improved, and the technical effect of improving the efficiency of image rendering is achieved.

[0025] In addition, in the related art, the virtual object model is pre-baked before real-time rendering, and the baked assets (for example, color maps) are stored; while the technical solution in the embodiments of the present application is to perform real-time baking operations during the real-time rendering process, thereby avoiding the problem of resource waste caused by storing a large amount of baked assets, reducing the game package size, increasing the game frame rate, and enhancing the gaming experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0027] Figure 1 is a schematic diagram of an application scenario of an optional image rendering method according to an embodiment of the present application;

[0028] Figure 2 is a flowchart of an optional image rendering method according to an embodiment of the present application;

[0029] Figure 3 is a schematic diagram of an optional depth map and billboard according to an embodiment of the present application;

[0030] Figure 4 is a schematic diagram of an optional image rendering method according to an embodiment of the present application;

[0031] Figure 5 is an example of an optional first depth map and a first color map according to an embodiment of the present application;

[0032] Figure 6 is a schematic diagram of an optional local coordinate system and a world coordinate system according to an embodiment of the present application;

[0033] Figure 7 is a schematic diagram of an optional baking result according to an embodiment of the present application;

[0034] Figure 8 is a schematic diagram of an optional rendering of a Billboard according to an embodiment of the present application;

[0035] Fig. 9 is a schematic diagram of a change in light source direction of an optional virtual light source according to an embodiment of the present application;

[0036] Fig.10 is a schematic diagram of an optional change in the angle between the direction of a virtual camera toward a tree and a horizontal plane according to an embodiment of the present application;

[0037] Fig.11 is a schematic diagram of an optional refresh atlas according to an embodiment of the present application;

[0038] Fig.12 is an overall flow chart of an optional image rendering method according to an embodiment of the present application;

[0039] Fig.13 This is an optional rendering of large-scale vegetation according to an embodiment of the present application;

[0040] Fig.14 It is a comparison diagram of the effects between the related technology and the technical solution of this application;

[0041] Fig.15 is a schematic structural diagram of an optional image rendering device according to an embodiment of the present application;

[0042] Fig.16 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0045] The technical solutions in the embodiments of the present application will comply with legal provisions during implementation. When performing operations according to the technical solutions in the embodiments, the data used will not involve user privacy. While ensuring that the operation process is compliant and legal, the security of the data is guaranteed.

[0046] Glossary:

[0047] 1. Billboard: Use a patch that always faces the camera (a rectangular patch consisting of two triangles) for rendering. It is usually used to achieve effects such as particles and UI. For details, please refer to Figure 3 The schematic diagram of Billboasrd shown;

[0048] 2. LOD (Level of Detail): In order to maintain the frame rate, 3D games optimize performance by reducing the number of faces of distant models;

[0049] 3. Render Target (RT): A render target is a surface on which the graphics API can "draw" things. In essence, a render target is a continuous memory area. Multiple such memory areas can exist at the same time, that is, multiple render targets.

[0050] 4. Depth Texture: It is used to record the distance from each pixel on the screen to the camera. It is also a RenderTarget. In the rasterization stage of the rendering pipeline, the GPU calculates the depth of the object's mesh at the corresponding pixel (a floating point number in the range of 0-1) and outputs it to RT to obtain the depth map. For details, please refer to Figure 3 The depth map shown.

[0051] According to one aspect of the embodiments of the present application, an image rendering method is provided. As an optional implementation, the image rendering method can be but is not limited to being applied to Figure 1 The application scenario shown in Figure 1 In the application scenario shown, the terminal device 102 may, but is not limited to, communicate with the server 106 via the network 104, and the server 106 may, but is not limited to, perform operations on the database 108, such as write data operations or read data operations. The above-mentioned terminal device 102 may, but is not limited to, include a human-computer interaction screen, a processor, and a memory. The above-mentioned human-computer interaction screen may, but is not limited to, be used to display a virtual light source, a first virtual camera, and a first target rendered image on the terminal device 102. The above-mentioned processor may, but is not limited to, be used to respond to the above-mentioned human-computer interaction operation, perform corresponding operations, or generate corresponding instructions, and send the generated instructions to the server 106. The above-mentioned memory is used to store relevant processing data, such as the first light source direction, the first color map, and the first depth map.

[0052] As an optional manner, the following steps in the image rendering method may be performed on the server 106: Step S102, performing a baking operation on the virtual object model based on the illumination direction of the virtual light source to obtain a first color map, a first depth map and a first coordinate set, wherein the virtual object model is a three-dimensional model, the virtual object model is within the shooting range of the first virtual camera, the first color map includes the color values ​​of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model in the direction of the first light source, the first depth map is used to represent the distance between each point in the virtual object model and the first virtual camera, and the first coordinate set is used to represent the position of each point in the virtual object model; Step S104, obtaining the coordinates of each point in the virtual object model in the target coordinate system to obtain a second coordinate set; Step S106, determining the shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set and the second coordinate set, wherein the shadow information is used to represent the shadow value of each point in the virtual object model in the target coordinate system; Step S108, rendering the virtual object model according to the first color map, the first coordinate set and the shadow information to obtain a first target rendered image.

[0053] It should be noted that the above-mentioned image rendering method can be applied to, but is not limited to, rendering of virtual objects such as large-scale vegetation, stacked rocks, and buildings with repeated styles in virtual scenes.

[0054] In the above manner, by performing a baking operation on the virtual object model, a first depth map and a first color map containing the effect after being illuminated are obtained, and at the same time, the UV texture coordinates of each point in the virtual object model are obtained; the coordinates of each point in the virtual object model in the target coordinate system are obtained using the first depth map, and the shadow information corresponding to the virtual object model is calculated, and finally the virtual object model is rendered to obtain the first target rendering model. In other words, by calculating the illumination during the baking operation, a large amount of computing resources consumed by real-time calculation of illumination during the image rendering process is avoided, the overall performance of the image rendering is improved, and the technical effect of improving the efficiency of image rendering is achieved.

[0055] In order to solve the problem of low efficiency in the process of determining the above relationship tags, an image rendering method is proposed in an embodiment of the present application. Figure 2 is a flow chart of an image rendering method according to an embodiment of the present application, and the flow chart includes the following steps S202 to S206.

[0056] Step S202, performing a baking operation on the virtual object model based on the illumination direction of the virtual light source to obtain a first color map, a first depth map, and a first coordinate set, wherein the virtual object model is a three-dimensional model, the virtual object model is within the shooting range of the first virtual camera, the first color map includes the color value of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model in the direction of the first light source, the first depth map is used to represent the distance between each point in the virtual object model and the first virtual camera, and the first coordinate set is used to represent the position of each point in the virtual object model;

[0057] Step S204, obtaining the coordinates of each point in the virtual object model in the target coordinate system to obtain a second coordinate set;

[0058] Step S206, determining shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent a shadow value of each point in the virtual object model in the target coordinate system;

[0059] Step S208: Rendering the virtual object model according to the first color map, the first coordinate set and the shadow information to obtain a first target rendered image.

[0060] like Figure 4 As shown in (a), it is assumed that the virtual object model is the original model of the tree. In the local coordinate system where the original model is located, the virtual light source illuminates the original model according to the direction of the first light source, and the original model is baked to obtain a first color map, a first depth map and a first coordinate set.

[0061] Among them, the first color map includes but is not limited to the color values ​​of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model according to the direction of the first light source. The effects after being illuminated include diffuse reflection and highlights, but do not include ambient light (GI).

[0062] In other words, the sum of the baking operations is the model that the human eye sees after being illuminated, which can also be understood as the effect that the human eye can finally see. For details, please refer to Figure 5 The first color map is shown.

[0063] Obviously, it is easy to understand that when the virtual light source illuminates the virtual object model in a direction different from the first light source direction, the color map obtained after the baking operation will change. In other words, the color map obtained after the baking operation changes as the light source direction of the virtual light illuminating the virtual object model changes.

[0064] At the same time, after the baking operation, a first depth map can also be obtained, wherein the depth map is used to record the distance from each pixel in the screen to the camera. In this embodiment, the distance between each pixel point in the baked map and the first virtual camera in the local coordinate system where the virtual object model is located is recorded. For details, please refer to Figure 5 The first depth map is shown.

[0065] It should be noted that each object in a scene or virtual scene has its own local coordinate system. For example, when building a cube, the origin of the local coordinate system is often placed at the center of the cube or a vertex. The local coordinate system is used to construct the positional relationship between objects. For details, please refer to Figure 6 The world coordinate system can be understood as the global coordinate system of the entire scene, which is used to describe the positional relationship between objects. For details, please refer to Figure 6 As shown in (b).

[0066] In addition, during the baking operation, in addition to obtaining the above-mentioned first color map and first depth map, the UV texture coordinates of each point on the model in the local coordinate system where the virtual object model is located are recorded, and the first coordinate set is obtained based on the UV texture coordinates of all points.

[0067] like Figure 3As shown, since Billboard is a technology that uses a camera that is always facing the camera (a rectangular patch consisting of two triangles) to render a 3D virtual object model into a 2D image, and the camera that the Billboard faces represents what the human eye sees and is different from the first virtual camera, and the uv texture coordinates in the first coordinate set are the coordinates of each point in the virtual object model in the above local coordinate system, which are different from the Billboard world coordinates used in the actual rendering process. Therefore, it is necessary to reconstruct the Billboard world coordinates based on the uv texture coordinates in the first coordinate set to obtain the second coordinate set.

[0068] Then, shadow information corresponding to the virtual object model is determined according to the first depth map, the first coordinate set and the second coordinate set, and then the virtual object model is rendered using the first color map, the first coordinate set and the shadow information to obtain a first target rendered image.

[0069] As an optional implementation manner, the above-mentioned determining the shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set and the second coordinate set includes:

[0070] The i-th shadow value in the shadow information is determined by the following steps, wherein the i-th shadow value is the shadow value of the i-th point in the virtual object model in the target coordinate system, and i is a positive integer greater than or equal to 1:

[0071] According to the i-th coordinate in the first coordinate set, obtaining an i-th depth value corresponding to the i-th point in the first depth map, wherein the i-th coordinate is used to represent the position of the i-th point, and the i-th depth value is used to represent the distance between the i-th point in the virtual object model and the first virtual camera;

[0072] Performing a restoration operation on the i-th depth value according to the parameters in the target projection matrix to obtain the i-th restored depth value, wherein the target projection matrix is ​​a projection matrix used when performing a baking operation on the virtual object model, and the target projection matrix is ​​used to project the three-dimensional virtual object model into a two-dimensional first color map;

[0073] The ith shadow value is determined according to the ith restored depth value and the ith initial coordinate in the second coordinate set, wherein the ith initial coordinate is used to represent the position of the ith point in the virtual object model in the target coordinate system, and the coordinates in the second coordinate set are three-dimensional coordinates.

[0074] Assuming that the virtual object model is the original model of the tree, the projection matrix is ​​used to convert the original 3D tree model into a 2D baked map. In this process, the parameters of the projection matrix are recorded, and the parameters of the projection matrix are used to restore the i-th depth value in the first depth map to obtain the i-th restored depth value.

[0075] Finally, the i-th shadow value is determined according to the i-th restored depth value and the i-th initial coordinate in the second coordinate set.

[0076] The specific implementation process is as follows:

[0077] Step (1): Record the near and far clipping distances of the projection matrix

[0078] float n=_IMPOSTOR_PROJECT_NEAR;

[0079] float f=_IMPOSTOR_PROJECT_FAR;

[0080] float2 ImpostorZBufferParam=new Vector2(n,fn).

[0081] Where n is parameter 1 of the projection matrix, and (fn) is parameter 2 of the projection matrix.

[0082] Step (2): Reconstruct the world coordinates by sampling the cached depth map:

[0083] / / Billboard world coordinates corresponding to the current pixel

[0084] float3 positionWS=input.positionWS.xyz;

[0085] / / Sample the cached depth map

[0086] float depth=tex2D(_ImpostorDepthTex,input.uv).r;

[0087] / / Restore the original value of Depth. ImpossorZBufferParam records the parameters of the projection matrix when baking to the atlas.

[0088] depth=(depth*_ImpostorZBufferParam.y+_ImpostorZBufferParam.x);

[0089] / / Offset along the line of sight to restore the real coordinates

[0090] positionWS+=normalize(input.viewDirWS.xyz)*depth.

[0091] Step (3): Sample the shadow by coordinates:

[0092] / / The MainLightShadow function compares the occlusion relationship between the current coordinates and the projection to determine whether the current pixel is in the shadow

[0093] float shadow=MainLightShadow(positionWS).

[0094] For the above step (2), the specific implementation process is as follows:

[0095] S11, using float3 positionWS=input.positionWS.xyz, to obtain the world coordinates of each pixel point in the virtual object model in the world coordinate system;

[0096] For example, the ith Billboard world coordinate of the ith pixel.

[0097] S12, obtaining an i-th depth value corresponding to the i-th point from the first depth map through the i-th coordinate in the first coordinate set;

[0098] As an optional implementation, it includes:

[0099] Find the point represented by the i-th coordinate in the first depth map;

[0100] The i-th depth value is determined as the depth value of the found point.

[0101] Specifically, it is implemented through float depth=tex2D(_ImpostorDepthTex,input.uv).r, where ImpostorDepthTex is the first depth map recorded during the baking operation. The tex2D function is used to generate the uv texture coordinates of each point in the original model recorded during the baking operation (that is, the coordinates in the first coordinate set), and the i-th depth value corresponding to the current i-th pixel is found from the first depth map.

[0102] The UV coordinates recorded in the first coordinate set are used to represent the position of each pixel in the baked map, and the position of each pixel is used to find the depth value corresponding to the position.

[0103] S13, using parameter 1 and parameter 2 in the projection matrix, performing a restoration operation on the i-th depth value to obtain the i-th restored depth value;

[0104] As an optional implementation method, the i-th restored depth value is obtained, including:

[0105] When the parameters in the target projection matrix include the near clipping distance and the far clipping distance, the i-th restored depth value is determined to be equal to the sum of the target product value and the near clipping distance, the target product value is equal to the product of the i-th depth value and the distance difference, the distance difference is the difference obtained by subtracting the near clipping distance from the far clipping distance, the near clipping distance is used to represent the distance between the near clipping plane used when projecting the three-dimensional virtual object model into a two-dimensional first color map and the first virtual camera, and the far clipping distance is used to represent the distance between the far clipping plane used when projecting the three-dimensional virtual object model into a two-dimensional first color map and the first virtual camera.

[0106] Specifically, it is achieved through depth=(depth*_ImpostorZBufferParam.y+_ImpostorZBufferParam.x) in the above embodiment, wherein x=n, y=fn.

[0107] Among them, parameter 1 is n, parameter 2 is (fn), when baking the original model of the tree, parameter 1 and parameter 2 are recorded in ImpossorZBufferParam, and then passed to the GPU (graphics card) together.

[0108] It should be noted that the reason for performing the above-mentioned restoration operation is that the depth value of the pixel point in the first depth map generated during the baking operation is the depth value in the local coordinate system where the virtual object model is located, which deviates from the depth value in the Billboard world coordinate system. Therefore, the depth value of the i-th pixel point in the first depth map is restored through parameters 1 and 2 of the above-mentioned projection matrix.

[0109] S14, restoring the real coordinates of the i-th pixel point in the world coordinate system according to the i-th restored depth value;

[0110] Since the virtual object (eg, a tree) itself has a certain width, the i-th restored depth value mentioned above needs to be used when converting the coordinates of each point in the baked map obtained during the baking operation into three-dimensional coordinates in the world coordinate system.

[0111] Specifically, it is achieved by positionWS+=normalize(input.viewDirWS.xyz)*depth, which will be described in detail below in conjunction with specific embodiments.

[0112] In the above manner, by performing a baking operation on the virtual object model, a first depth map and a first color map containing the effect after being illuminated are obtained, and at the same time, the UV texture coordinates of each point in the virtual object model are obtained; the coordinates of each point in the virtual object model in the target coordinate system are obtained using the first depth map, and the shadow information corresponding to the virtual object model is calculated, and finally the virtual object model is rendered to obtain the first target rendering model. In other words, by calculating the illumination during the baking operation, a large amount of computing resources consumed by real-time calculation of illumination during the image rendering process is avoided, the overall performance of the image rendering is improved, and the technical effect of improving the efficiency of image rendering is achieved.

[0113] As an optional implementation, the above determining the i-th shadow value according to the i-th restored depth value and the i-th initial coordinate in the second coordinate set includes:

[0114] Determine the i-th restored coordinate according to the i-th restored depth value, the i-th sight direction vector, and the i-th initial coordinate, wherein the i-th sight direction vector is used to represent the direction vector from the second virtual camera to the i-th initial coordinate in the target coordinate system;

[0115] Determine the i-th shadow value based on the i-th restored coordinate.

[0116] As an optional example, the above determining the i-th restored coordinate according to the i-th restored depth value, the i-th sight direction vector, and the i-th initial coordinate includes:

[0117] Normalize the i-th sight direction vector to obtain a normalized vector;

[0118] The i-th restored coordinate is determined to be equal to the sum of the i-th initial coordinate and a target vector, where the target vector is a vector obtained by multiplying the normalized vector by the i-th restored depth value.

[0119] Among them, the i-th restored coordinate is obtained by positionWS+=normalize(input.viewDirWS.xyz)*depth, positionWS is the i-th initial coordinate in the second coordinate set, and input.viewDirWS.xyz is the i-th sight direction vector.

[0120] Then the i-th restored coordinate is the sum of the i-th initial coordinate positionWS and the target vector, where the target vector is the vector obtained by multiplying the normalized vector normalize(input.viewDirWS.xyz) and the i-th restored depth value.

[0121] In the above positionWS+=normalize(input.viewDirWS.xyz)*depth, “+=” means that the i-th restored coordinate on the left side of the equation is equal to the sum of the i-th initial coordinate and the target vector on the right side of the equation.

[0122] It should be noted that the i-th sight direction vector may be, but is not limited to, the direction vector between the human eye and the i-th pixel point in the Billboard model. The i-th restored coordinates are the real coordinates of the i-th pixel point in the Billboard model in the world coordinate system.

[0123] As an optional implementation, the above method of determining the i-th shadow value according to the i-th restored coordinate includes:

[0124] According to a preset mapping relationship, a shadow value of the i-th point having a mapping relationship with the i-th restored coordinate is determined.

[0125] Specifically, the MainLightShadow function compares the occlusion relationship between the current coordinates and the projection to determine whether the current pixel is in the shadow. When the judgment result is that the current pixel is in the shadow, 0 is returned; otherwise, 1 is returned.

[0126] It should be noted that the shadow value in this embodiment may be, but is not limited to, the actual color of each pixel in the Billboard model after being illuminated, which is dark or dark.

[0127] In combination with the description of the above embodiments, it can be seen that the technical solution of the present application is to use the depth and color of the pixel points on the 2D baking map cached during the baking operation to represent the points on the Billboard model, sample the cached depth map to reconstruct the world coordinates when rendering the Billboard, and sample the shadows through the world coordinates to obtain a more realistic light and shadow effect, thereby improving the accuracy of shadow calculation.

[0128] The above shadow calculation method is suitable for large-scale forest vegetation image rendering, and the effect is more obvious when used from a bird's-eye view. For details, please refer to Fig.13 As shown in (a).

[0129] As an optional example, the above-mentioned rendering process of the virtual object model according to the first color map, the first coordinate set and the shadow information to obtain the first target rendered image includes:

[0130] The final color value of the i-th point in the virtual object model in the target coordinate system is determined by the following steps, and a first target rendered image is determined according to the virtual object model whose final color value is determined in the target coordinate system, where i is a positive integer greater than or equal to 1:

[0131] According to the i-th coordinate in the first coordinate set, obtaining the i-th color value corresponding to the i-th point in the first color map, wherein the i-th coordinate is used to represent the position of the i-th point, and the i-th color value is used to represent the color value of the i-th point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model in the direction of the first light source;

[0132] The final color value of the i-th point is determined according to the i-th color value and the i-th shadow value in the shadow information, wherein the i-th shadow value is the shadow value of the i-th point in the virtual object model in the target coordinate system.

[0133] As an optional implementation manner, the above acquiring the i-th color value corresponding to the i-th point in the first color map according to the i-th coordinate in the first coordinate set includes:

[0134] Find the point represented by the i-th coordinate in the first color map;

[0135] Determine the i-th color value as the color value of the found point.

[0136] Specifically, the i-th color value corresponding to the i-th point is obtained by combining the color information in the first color map cached during the baking operation and using the following method: half4 color = tex2D(_ImpostorColorTex, input.uv), where ImpostorColorTex is the first color map baked during the baking operation, input.uv represents the point represented by the i-th uv coordinate, and the tex2D function is used to find the point represented by the i-th coordinate in the first color map, and the i-th color value is determined as the color value of the found point.

[0137] Then, the final color value of the i-th point is determined according to the i-th color value and the i-th shadow value in the shadow information.

[0138] As an optional implementation, the above method of determining the final color value of the i-th point according to the i-th color value and the i-th shadow value in the shadow information includes:

[0139] The final color value of the i-th point is determined to be equal to the product between the i-th color value and the i-th shade value.

[0140] The final color value of the i-th point is realized by color.rgb*=shadow, where shadow represents the i-th shadow value, and the final color value of the i-th point is equal to the product of the i-th color value color.rgb found in the first color map and the i-th shadow value shadow.

[0141] Determining a first target rendering image according to the virtual object model whose final color value is determined in the target coordinate system includes:

[0142] The virtual object model with the final color value determined is converted into a two-dimensional first target rendering image.

[0143] It should be noted that before calculating the final color value of the above-mentioned i-th point, the GPU will pre-rasterize the Billboard model, so that the virtual object model with the final color value determined above is converted from model data to pixel data projected into the screen space.

[0144] Then, the above calculation process of color.rgb*=shadow is executed in the screen space, and the calculation result is directly output to the screen, so that the rendered 2D image can be displayed on the screen.

[0145] In the above manner, under the condition of accurate calculation of shadows, the virtual object model with a determined final color value is converted from a 3D model into a 2D image on the screen using the Billboard rendering technology, thereby avoiding deformation of the rendered image and improving rendering performance.

[0146] In order to understand the above image rendering method more clearly, Fig.12 The overall flow chart shown further describes it.

[0147] like Fig.12 As shown, the specific steps are as follows:

[0148] S1202, loading models and materials;

[0149] S1204, generating an atlas;

[0150] like Figure 7 As shown, in a specific embodiment, the original image is divided into 4×4 grids according to the region, each grid stores the baking result of a tree, and a new image is added whenever there are more than 16 types of trees.

[0151] S1206-1 and S1206-2, create images for caching color and depth, and bake the color and depth of the original model into corresponding images according to the current camera angle;

[0152] It should be noted that the baking results include but are not limited to two groups of pictures, one of which is a depth map and the other is a color map. When caching the color map, the lighting is calculated in advance. Therefore, the cached color map contains the effect after being illuminated, avoiding the waste of computing resources caused by real-time calculation of lighting and improving the efficiency of image rendering.

[0153] In step S1206-1, the illumination information of the main light source is calculated when the model is baked, and the effect after receiving the light is output and saved. Note that shadows and fog effects cannot be calculated here. Shadows and fog effects are calculated in real time when rendering the Billboard.

[0154] In step S1206-2, the depth value of each model is output to another set of atlases and cached for subsequent shadow calculation (since the calculation of shadows requires the coordinate information of each point of the model, if only the color is cached, the coordinate value cannot be obtained when the billboard is subsequently rendered, so the depth value is cached here for subsequent coordinate reconstruction).

[0155] In addition, since the baking operation in the embodiment of the present application is performed during the running of the game, there is no need to calculate the lighting in real time, and there is no need to pre-store the baked maps at different angles in the disk space, which reduces the pre-baked assets, reduces the game package size, improves the loading efficiency of the game screen, and further enhances the gaming experience.

[0156] S1208, generating a Billboard model;

[0157] Generating a Billboard model means generating its patch model (including vertices, UVs, normals, etc.), and at the same time, it is necessary to record the coordinate information of a large number of tree objects to mark the position of each Billboard model.

[0158] One tree is displayed using one billboard. If there are 1,000 trees, 1,000 billboards are generated, and the face is always rendered toward the camera. For details, please refer to Figure 8 Illustration of how to render large-scale vegetation using Billboard, with trees in the distance shown.

[0159] It should be noted that the Billboard is generated based on the position and size of the virtual object (eg, tree) itself, not based on the baked map.

[0160] S1210, reconstructing coordinates and calculating shadows;

[0161] In combination with the description of the shadow calculation part in the above embodiment, the present application caches the first depth map during pre-baking, samples the first depth map when rendering the Billboard to reconstruct the world coordinates of the virtual object model, and accurately samples the shadow through the world coordinates to obtain a realistic light and shadow effect.

[0162] S1212, recording the calculated shadow value;

[0163] S1214, synthesizing a final rendered image according to the first depth map and the shadow value cached during the baking operation;

[0164] As described in the above embodiment, the color value of the point represented by the ith coordinate is obtained from the first color map through half4 color=tex2D(_ImpostorColorTex,input.uv), and then the final color value of the ith point is determined using color.rgb*=shadow, and the final color value is output to the screen to obtain the first target rendered image. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0165] S1216, checking the change of camera height and light source direction;

[0166] When the change in camera height or light source direction meets the preset conditions, refresh the atlas, as shown in step S1218; otherwise, execute step S1220.

[0167] S1218, determining whether to refresh the atlas;

[0168] As an optional implementation method, the implementation process of determining whether to refresh the atlas includes:

[0169] When the virtual light source illuminates the virtual object model according to the second light source direction and the virtual object model is located in the shooting range of the first virtual camera, a baking operation is performed on the virtual object model to obtain a second color map, a first depth map and a first coordinate set, wherein the second color map includes color values ​​of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model according to the second light source direction, and an angle between the second light source direction and the first light source direction is greater than or equal to a first preset threshold;

[0170] Determine shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent a shadow value of each point in the virtual object model in the target coordinate system;

[0171] The virtual object model is rendered according to the second color map, the first coordinate set and the shadow information to obtain a second target rendered image.

[0172] like Fig. 9 As shown, assuming that the virtual light source illuminates the virtual object model from the direction of the first light source, the following is obtained: Fig.11In the atlas shown in (a), when the virtual light source switches from the first light source direction to the second light source direction, and the angle θ between the first light source direction and the second light source direction is greater than or equal to a first preset threshold (for example, greater than 5°), the virtual object is baked again, and the baked atlas is as follows: Fig.11 As shown in (b).

[0173] Since the light source direction changes, the cached color map changes during the baking operation, that is, it is converted from the first color map to the second color map, and the second target rendered image is obtained according to the above method.

[0174] It should be noted that, when the relative position relationship between the virtual camera and the virtual model in the local coordinate system does not change and only the light source changes, the depth map remains unchanged.

[0175] As another optional implementation, the implementation process of determining whether to refresh the atlas also includes:

[0176] When the angle between the line between the first virtual camera and the center point of the virtual object model and the horizontal plane is converted from the first angle to the second angle, a baking operation is performed on the virtual object model to obtain a third color map, a second depth map, and a third coordinate set, wherein the angle difference between the first angle and the second angle is greater than or equal to a second preset threshold, and the horizontal plane is a plane where the virtual object model is located in the shooting range of the first virtual camera;

[0177] Determine shadow information corresponding to the virtual object model according to the third color map, the second coordinate set, and the third coordinate set;

[0178] The virtual object model is rendered according to the third color map, the third coordinate set and the shadow information to obtain a third target rendered image.

[0179] like Fig.10 As shown, assuming that the angle difference between the direction of the first virtual camera toward the virtual object and the horizontal plane exceeds 5°, the virtual object is re-baked.

[0180] Specifically, assuming that the x-axis direction is the horizontal plane where the virtual object (cube) is located, the angle between the line between the first virtual camera and the center point of the cube and the x-axis is Fig.10 When θ1 shown in (a) changes to θ2 and the angle difference between θ1 and θ2 exceeds 5°, the atlas is refreshed. At the same time, the shadow is calculated according to the method in the above embodiment, and the final third target rendering image is obtained.

[0181] According to the image rendering method in the above embodiment, when large-scale vegetation is rendered, a better rendering effect can be obtained under the premise of saving computing time and reducing storage space. For details, please refer to Fig.13 As shown in (b).

[0182] The technical solution in the above embodiment is adopted to obtain a realistic effect by using pre-baked lighting and reconstructed coordinate sampling shadows, which solves the performance consumption problem of large-scale vegetation rendering in traditional solutions and has wide hardware compatibility and can run on low-end mobile phones.

[0183] In addition, when using the image rendering method in the relevant technology to render thousands of trees at the same time in a large-scale forest scene in the game, the number of triangles will exceed the limit, the game frame rate will drop, and the gaming experience will be affected. Generally, the number of triangles of distant trees will be reduced by making LOD models by artists, but the deformation of trees after the reduction of faces by artists will be more serious, thus affecting the display effect.

[0184] In the embodiment of the present application, in order to ensure the outline of the distant trees, the tree model is pre-baked into the picture to replace the distant display solution. This not only reduces the number of faces, but also ensures the display effect as much as possible.

[0185] Compared with the image rendering method in the related art, the image rendering method in the technical solution of the present application can avoid the process of real-time calculation of illumination while reducing the number of facets. Compared with the traditional insert solution, the technical solution in the embodiment of the present application can present better rendering effects at different angles, and the calculation of light and shadow effects is more accurate. For details, please refer to Fig.14 Comparison table shown.

[0186] In addition, the technical solution in the embodiment of the present application can also be directly used to replace the display of other large numbers of repeated models: such as stacked rocks, buildings with repeated styles, etc. At the same time, the functions of the technical solution in the embodiment of the present application can also be expanded to a certain extent (support for skeletal animation) to replace the rendering of thousands of soldiers in large-scale war scenes.

[0187] As an optional example, the technical solution in the embodiment of the present application can be applied to, but not limited to, rendering a large number of soldiers in a simulation game (SLG) of a virtual war type, and the specific implementation steps are as follows:

[0188] 1) Loading models and materials to virtual objects;

[0189] 2) Perform baking operations to generate atlases;

[0190] For example, soldiers with similar orientations and movements are cached as a virtual object and share a cache unit.

[0191] It should be noted that since thousands of soldiers are skeletal animation objects, and in the game scene, skeletal animation objects are dynamic objects, they need to be refreshed every frame during the baking process.

[0192] 3) During the baking operation, create images for caching color and depth, and bake the color and depth of the virtual object model into the corresponding images according to the current camera angle;

[0193] The baking results include but are not limited to two groups of images, one of which is a depth map and the other is a color map. When caching the color map, the illumination is pre-calculated, so the cached color map contains the effect after being illuminated.

[0194] While caching the color map, the depth value of each virtual object model is also output to another set of atlases and cached for subsequent shadow calculations.

[0195] 4) Generate Billboard model;

[0196] The generated Billboard model includes the vertices, UVs and normals of the model. At the same time, it is necessary to record the coordinate information of each virtual object model corresponding to a large number of soldiers to mark the position of each Billboard model.

[0197] The billboard is generated based on the position and size of the virtual object (for example, a soldier) itself, not based on the baked map; in addition, a soldier uses a billboard instead of display, and the billboard patch is always rendered towards the camera.

[0198] 5) Use the depth map cached during pre-baking to reconstruct the coordinates and calculate the shadows;

[0199] Since the present application caches the depth map during pre-baking, the first depth map is sampled when rendering the billboard to reconstruct the world coordinates of the virtual object model. Only through the world coordinates can the shadow be accurately sampled to obtain a realistic light and shadow effect. The specific implementation process can refer to the description of the shadow calculation part in the above embodiment, which will not be repeated here.

[0200] 6) Based on the depth map cached during the baking process and the shadow values ​​recorded in step (5), the final rendered image of thousands of soldiers is synthesized.

[0201] In combination with the description in the above embodiment, it can be seen that the color value of the point represented by the i-th coordinate in a Billboard model is obtained from the color map cached during the baking operation, and then color.rgb*=shadow is used to determine the final color value of the i-th point, and then output to the screen according to the final color value, so as to obtain the target rendering image of a soldier, and so on, to obtain the target rendering images of multiple soldiers.

[0202] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0203] According to another aspect of the embodiment of the present application, there is also provided Fig.15 An image rendering device as shown, the device comprising:

[0204] A first processing unit 1502 is configured to perform a baking operation on the virtual object model based on a light source direction of the virtual light source to obtain a first color map, a first depth map, and a first coordinate set, wherein the virtual object model is a three-dimensional model, the virtual object model is within a shooting range of a first virtual camera, the first color map includes color values ​​of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model in the direction of the first light source, the first depth map is used to represent the distance between each point in the virtual object model and the first virtual camera, and the first coordinate set is used to represent the position of each point in the virtual object model;

[0205] A first acquisition unit 1504 is used to acquire the coordinates of each point in the virtual object model in the target coordinate system to obtain a second coordinate set;

[0206] A second processing unit 1506 is used to determine shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent a shadow value of each point in the virtual object model in the target coordinate system;

[0207] The first rendering unit 1508 is used to render the virtual object model according to the first color map, the first coordinate set and the shadow information to obtain a first target rendered image.

[0208] Optionally, the second processing unit 1504 includes:

[0209] The first processing module is used to determine the i-th shadow value in the shadow information through the following steps, wherein the i-th shadow value is the shadow value of the i-th point in the virtual object model in the target coordinate system, and i is a positive integer greater than or equal to 1: according to the i-th coordinate in the first coordinate set, obtain the i-th depth value corresponding to the i-th point in the first depth map, wherein the i-th coordinate is used to represent the position of the i-th point, and the i-th depth value is used to represent the distance between the i-th point in the virtual object model and the first virtual camera; according to the parameters in the target projection matrix, restore the i-th depth value to obtain the i-th restored depth value, wherein the target projection matrix is ​​the projection matrix used when the virtual object model is baked, and the target projection matrix is ​​used to project the three-dimensional virtual object model into a two-dimensional first color map; according to the i-th restored depth value and the i-th initial coordinate in the second coordinate set, determine the i-th shadow value, wherein the i-th initial coordinate is used to represent the position of the i-th point in the virtual object model in the target coordinate system, and the coordinates in the second coordinate set are three-dimensional coordinates.

[0210] Optionally, the first processing module includes:

[0211] A first search submodule, used to search for a point represented by an i-th coordinate in the first depth map;

[0212] The first processing submodule is used to determine the i-th depth value as the depth value of the found point.

[0213] Optionally, the first processing module includes:

[0214] The second processing submodule is used to determine the i-th restored depth value to be equal to the sum of the target product value and the near clipping distance when the parameters in the target projection matrix include the near clipping distance and the far clipping distance, the target product value is equal to the product of the i-th depth value and the distance difference value, the distance difference value is the difference obtained by subtracting the near clipping distance from the far clipping distance, the near clipping distance is used to represent the distance between the near clipping plane used when projecting the three-dimensional virtual object model into a two-dimensional first color map and the first virtual camera, and the far clipping distance is used to represent the distance between the far clipping plane used when projecting the three-dimensional virtual object model into a two-dimensional first color map and the first virtual camera.

[0215] Optionally, the first processing module includes:

[0216] A third processing submodule is used to determine the i-th restored coordinate according to the i-th restored depth value, the i-th sight direction vector, and the i-th initial coordinate, wherein the i-th sight direction vector is used to represent the direction vector from the second virtual camera to the i-th initial coordinate in the target coordinate system;

[0217] The fourth processing submodule is used to determine the i-th shadow value according to the i-th restored coordinate.

[0218] Optionally, the first processing module includes:

[0219] The fifth processing submodule is used to perform normalization processing on the i-th sight direction vector to obtain a normalized vector;

[0220] The sixth processing submodule is used to determine the i-th restored coordinate to be equal to the sum of the i-th initial coordinate and a target vector, wherein the target vector is a vector obtained by multiplying the normalized vector by the i-th restored depth value.

[0221] Optionally, the first processing module includes:

[0222] The seventh processing submodule is used to determine the shadow value of the i-th point having a mapping relationship with the i-th restored coordinate according to a preset mapping relationship.

[0223] Optionally, the first rendering unit 1508 includes:

[0224] The second processing module is used to determine the final color value of the i-th point in the virtual object model in the target coordinate system through the following steps, and determine the first target rendered image according to the virtual object model whose final color value is determined in the target coordinate system, wherein i is a positive integer greater than or equal to 1:

[0225] According to the i-th coordinate in the first coordinate set, obtaining the i-th color value corresponding to the i-th point in the first color map, wherein the i-th coordinate is used to represent the position of the i-th point, and the i-th color value is used to represent the color value of the i-th point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model in the direction of the first light source;

[0226] The final color value of the i-th point is determined according to the i-th color value and the i-th shadow value in the shadow information, wherein the i-th shadow value is the shadow value of the i-th point in the virtual object model in the target coordinate system.

[0227] Optionally, the second processing module includes:

[0228] A second search submodule is used to search for a point represented by the i-th coordinate in the first color map;

[0229] The eighth processing submodule is used to determine the i-th color value as the color value of the found point.

[0230] Optionally, the second processing module includes:

[0231] The ninth processing submodule is used to determine the final color value of the i-th point to be equal to the product of the i-th color value and the i-th shadow value.

[0232] Optionally, the second processing module includes:

[0233] The conversion submodule is used to convert the virtual object model with a determined final color value into a two-dimensional first target rendering image.

[0234] Optionally, the above device further includes:

[0235] a first baking unit, configured to perform a baking operation on the virtual object model when the virtual light source illuminates the virtual object model in a second light source direction and the virtual object model is located in a shooting range of the first virtual camera, to obtain a second color map, a first depth map, and a first coordinate set, wherein the second color map includes color values ​​of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model in the second light source direction, and an angle between the second light source direction and the first light source direction is greater than or equal to a first preset threshold;

[0236] A third processing unit, configured to determine shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent a shadow value of each point in the virtual object model in the target coordinate system;

[0237] The second rendering unit is used to render the virtual object model according to the second color map, the first coordinate set and the shadow information to obtain a second target rendered image.

[0238] Optionally, the above device further includes:

[0239] a second baking unit, configured to perform a baking operation on the virtual object model when an angle between a line between the first virtual camera and a center point of the virtual object model and a horizontal plane is converted from a first angle to a second angle, so as to obtain a third color map, a second depth map, and a third coordinate set, wherein an angle difference between the first angle and the second angle is greater than or equal to a second preset threshold, and the horizontal plane is a plane where the virtual object model is located in a shooting range of the first virtual camera;

[0240] a fourth processing unit, configured to determine shadow information corresponding to the virtual object model according to the third color map, the second coordinate set, and the third coordinate set;

[0241] The third rendering unit is used to render the virtual object model according to the third color map, the third coordinate set and the shadow information to obtain a third target rendered image.

[0242] By applying the above device to the virtual object model for baking operation, a first depth map and a first color map containing the effect after being illuminated are obtained, and at the same time, the UV texture coordinates of each point in the virtual object model are obtained; the coordinates of each point in the virtual object model in the target coordinate system are obtained using the first depth map, and the shadow information corresponding to the virtual object model is calculated, and finally the virtual object model is rendered to obtain the first target rendering model. In other words, by calculating the illumination during the baking operation, a large amount of computing resources consumed by real-time calculation of illumination during the image rendering process is avoided, the overall performance of the image rendering is improved, and the technical effect of improving the efficiency of image rendering is achieved.

[0243] It should be noted that the embodiment of the image rendering device here can refer to the embodiment of the above-mentioned image rendering method, which will not be repeated here.

[0244] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned image rendering method is also provided. The electronic device may be Fig.16 The terminal device shown in FIG. This embodiment is described by taking the electronic device as a background device as an example. Fig.16 As shown, the electronic device includes a memory 1602 and a processor 1604. The memory 1602 stores a computer program, and the processor 1604 is configured to execute the steps in any of the above method embodiments through the computer program.

[0245] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0246] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0247] S1, performing a baking operation on the virtual object model based on the light source direction of the virtual light source to obtain a first color map, a first depth map and a first coordinate set, wherein the virtual object model is a three-dimensional model, the virtual object model is within the shooting range of the first virtual camera, the first color map includes the color value of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model according to the first light source direction, the first depth map is used to represent the distance between each point in the virtual object model and the first virtual camera, and the first coordinate set is used to represent the position of each point in the virtual object model;

[0248] S2, obtaining the coordinates of each point in the virtual object model in the target coordinate system to obtain a second coordinate set;

[0249] S3, determining shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent a shadow value of each point in the virtual object model in the target coordinate system;

[0250] S4, rendering the virtual object model according to the first color map, the first coordinate set and the shadow information to obtain a first target rendered image.

[0251] Alternatively, a person skilled in the art may understand that: Fig.16 The structure shown is for illustration only, and the electronic device may also be a target terminal such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, and a mobile Internet device (MID), a PAD, etc. Fig.16 The electronic device and the electronic equipment described above are not limited in structure. Fig.16 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig.16 Different configurations shown.

[0252] Among them, the memory 1602 can be used to store software programs and modules, such as program instructions / modules corresponding to the image rendering method and device in the embodiments of the present application. The processor 1604 executes various functional applications and data processing by running the software programs and modules stored in the memory 1602, that is, realizing the above-mentioned image rendering method. The memory 1602 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1602 may further include a memory remotely located relative to the processor 1604, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 1602 can be specifically, but not limited to, used to store a first depth map, a first color map, and a first coordinate set, etc. As an example, such as Fig.16 As shown, the memory 1602 may include, but is not limited to, the first processing unit 1502, the first acquisition unit 1504, the second processing unit 1506, and the first rendering unit 1508 in the image rendering device. In addition, other module units in the image rendering device may also be included but are not limited to, which will not be repeated in this example.

[0253] Optionally, the transmission device 1606 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired network and a wireless network. In one example, the transmission device 1606 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 1606 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0254] In addition, the electronic device further includes: a display 1608 for displaying the direction prompt information of the target sound; and a connection bus 1610 for connecting various module components in the electronic device.

[0255] In other embodiments, the target terminal or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. Among them, a peer-to-peer network may be formed between the nodes, and any form of computing device, such as a server, terminal or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0256] According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the image rendering method provided in various optional implementations of the above-mentioned server verification processing and other aspects, wherein the computer program is configured to execute the steps of any of the above-mentioned method embodiments when running.

[0257] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0258] S1, performing a baking operation on the virtual object model based on the light source direction of the virtual light source to obtain a first color map, a first depth map and a first coordinate set, wherein the virtual object model is a three-dimensional model, the virtual object model is within the shooting range of the first virtual camera, the first color map includes the color value of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model according to the first light source direction, the first depth map is used to represent the distance between each point in the virtual object model and the first virtual camera, and the first coordinate set is used to represent the position of each point in the virtual object model;

[0259] S2, obtaining the coordinates of each point in the virtual object model in the target coordinate system to obtain a second coordinate set;

[0260] S3, determining shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent a shadow value of each point in the virtual object model in the target coordinate system;

[0261] S4, rendering the virtual object model according to the first color map, the first coordinate set and the shadow information to obtain a first target rendered image.

[0262] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the target terminal through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0263] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0264] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods of each embodiment of the present application.

[0265] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0266] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0267] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0268] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0269] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image rendering method, characterized in that: include: The virtual object model is baked based on the light source direction of the virtual light source to obtain a first color map, a first depth map and a first coordinate set, wherein the virtual object model is a three-dimensional model, the virtual object model is within the shooting range of a first virtual camera, the first color map includes the color value of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model according to the first light source direction, the first depth map is used to represent the distance between each point in the virtual object model and the first virtual camera, and the first coordinate set is used to represent the position of each point in the virtual object model; Obtaining the coordinates of each point in the virtual object model in the target coordinate system to obtain a second coordinate set; Determine shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent a shadow value of each point in the virtual object model in the target coordinate system; The virtual object model is rendered according to the first color map, the first coordinate set and the shadow information to obtain a first target rendered image.

2. The method according to claim 1, characterized in that The determining, according to the first depth map, the first coordinate set, and the second coordinate set, shadow information corresponding to the virtual object model includes: The i-th shadow value in the shadow information is determined by the following steps, wherein the i-th shadow value is the shadow value of the i-th point in the virtual object model in the target coordinate system, and i is a positive integer greater than or equal to 1: According to the i-th coordinate in the first coordinate set, obtaining an i-th depth value corresponding to the i-th point in the first depth map, wherein the i-th coordinate is used to represent the position of the i-th point, and the i-th depth value is used to represent the distance between the i-th point in the virtual object model and the first virtual camera; Performing a restoration operation on the i-th depth value according to parameters in a target projection matrix to obtain an i-th restored depth value, wherein the target projection matrix is ​​a projection matrix used when performing the baking operation on the virtual object model, and the target projection matrix is ​​used to project the three-dimensional virtual object model into the two-dimensional first color map; The i-th shadow value is determined based on the i-th restored depth value and the i-th initial coordinate in the second coordinate set, wherein the i-th initial coordinate is used to represent the position of the i-th point in the virtual object model in the target coordinate system, and the coordinates in the second coordinate set are three-dimensional coordinates.

3. The method according to claim 2, characterized in that The acquiring, in the first depth map according to the i-th coordinate in the first coordinate set, the i-th depth value corresponding to the i-th point includes: Searching the first depth map for the point represented by the i-th coordinate; The i-th depth value is determined as the depth value of the found point.

4. The method according to claim 2, characterized in that: The performing a restoration operation on the i-th depth value according to the parameters in the target projection matrix to obtain the i-th restored depth value includes: In a case where the parameters in the target projection matrix include a near clipping distance and a far clipping distance, the i-th restored depth value is determined to be equal to the sum of a target product value and the near clipping distance, the target product value being equal to the product of the i-th depth value and a distance difference value, the distance difference value being a difference obtained by subtracting the near clipping distance from the far clipping distance, the near clipping distance being used to represent the distance between a near clipping plane used when projecting the three-dimensional virtual object model into the two-dimensional first color map and the first virtual camera, and the far clipping distance being used to represent the distance between a far clipping plane used when projecting the three-dimensional virtual object model into the two-dimensional first color map and the first virtual camera.

5. The method according to claim 2, characterized in that: The determining the i-th shadow value according to the i-th restored depth value and the i-th initial coordinate in the second coordinate set includes: Determine the i-th restored coordinate according to the i-th restored depth value, the i-th sight direction vector, and the i-th initial coordinate, wherein the i-th sight direction vector is used to represent the direction vector from the second virtual camera to the i-th initial coordinate in the target coordinate system; The i-th shadow value is determined according to the i-th restored coordinate.

6. The method according to claim 5, characterized in that The determining the i-th restored coordinate according to the i-th restored depth value, the i-th sight direction vector, and the i-th initial coordinate includes: Normalizing the i-th sight direction vector to obtain a normalized vector; The i-th restored coordinate is determined to be equal to the sum of the i-th initial coordinate and a target vector, wherein the target vector is a vector obtained by multiplying the normalized vector by the i-th restored depth value.

7. The method according to claim 5, characterized in that The determining the i-th shadow value according to the i-th restored coordinates includes: According to a preset mapping relationship, a shadow value of the i-th point having a mapping relationship with the i-th restored coordinate is determined.

8. The method according to any one of claims 1 to 7, characterized in that The step of rendering the virtual object model according to the first color map, the first coordinate set and the shadow information to obtain a first target rendered image includes: The final color value of the i-th point in the virtual object model in the target coordinate system is determined by the following steps, and the first target rendered image is determined according to the virtual object model whose final color value is determined in the target coordinate system, wherein i is a positive integer greater than or equal to 1: According to the i-th coordinate in the first coordinate set, obtaining, in the first color map, an i-th color value corresponding to the i-th point, wherein the i-th coordinate is used to represent the position of the i-th point, and the i-th color value is used to represent the color value of the i-th point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model in the direction of the first light source; Determine a final color value of the i-th point according to the i-th color value and the i-th shadow value in the shadow information, wherein the i-th shadow value is a shadow value of the i-th point in the virtual object model in the target coordinate system.

9. The method according to claim 8, characterized in that The acquiring, in the first color map according to the i-th coordinate in the first coordinate set, the i-th color value corresponding to the i-th point includes: Find the point represented by the i-th coordinate in the first color map; The i-th color value is determined as the color value of the found point.

10. The method according to claim 8, characterized in that Determining the final color value of the i-th point according to the i-th color value and the i-th shadow value in the shadow information includes: The final color value of the i-th point is determined to be equal to the product between the i-th color value and the i-th shadow value.

11. The method according to claim 8, characterized in that The step of determining the first target rendered image according to the virtual object model whose final color value is determined in the target coordinate system comprises: The virtual object model with the determined final color value is converted into the two-dimensional first target rendering image.

12. The method according to any one of claims 1 to 7, characterized in that The method further comprises: When the virtual light source illuminates the virtual object model according to a second light source direction and the virtual object model is located in the shooting range of the first virtual camera, a baking operation is performed on the virtual object model to obtain a second color map, the first depth map and the first coordinate set, wherein the second color map includes color values ​​of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model according to the second light source direction, and an angle between the second light source direction and the first light source direction is greater than or equal to a first preset threshold; Determine shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent a shadow value of each point in the virtual object model in the target coordinate system; The virtual object model is rendered according to the second color map, the first coordinate set and the shadow information to obtain a second target rendered image.

13. The method according to any one of claims 1 to 7, characterized in that The method further comprises: When the angle between the line between the first virtual camera and the center point of the virtual object model and the horizontal plane is converted from the first angle to the second angle, a baking operation is performed on the virtual object model to obtain a third color map, a second depth map, and a third coordinate set, wherein the angle difference between the first angle and the second angle is greater than or equal to a second preset threshold, and the horizontal plane is a plane where the virtual object model is located in the shooting range of the first virtual camera; Determining shadow information corresponding to the virtual object model according to the third color map, the second coordinate set, and the third coordinate set; The virtual object model is rendered according to the third color map, the third coordinate set and the shadow information to obtain the third target rendered image.

14. An image rendering device, characterized in that: include: a first processing unit, configured to perform a baking operation on the virtual object model based on a light source direction of a virtual light source to obtain a first color map, a first depth map, and a first coordinate set, wherein the virtual object model is a three-dimensional model, the virtual object model is within a shooting range of a first virtual camera, the first color map includes color values ​​of each point in the virtual object model after being illuminated when the virtual light source illuminates the virtual object model according to the first light source direction, the first depth map is used to represent the distance between each point in the virtual object model and the first virtual camera, and the first coordinate set is used to represent the position of each point in the virtual object model; A first acquisition unit, used to acquire the coordinates of each point in the virtual object model in the target coordinate system to obtain a second coordinate set; a second processing unit, configured to determine shadow information corresponding to the virtual object model according to the first depth map, the first coordinate set, and the second coordinate set, wherein the shadow information is used to represent a shadow value of each point in the virtual object model in the target coordinate system; The first rendering unit is used to render the virtual object model according to the first color map, the first coordinate set and the shadow information to obtain a first target rendered image.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program can be executed by a terminal device or a computer to execute the method described in any one of claims 1 to 13.

16. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 13 are implemented.

17. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 13 through the computer program.

Images