Image rendering method, apparatus and medium

Abstract

Embodiments of the application disclose an image rendering method and device and a medium, and relate to the technical field of computers. The image rendering method comprises: obtaining data of a surface point corresponding to a pixel fragment of a target gemstone model; wherein the data comprises: initial texture coordinates of the surface point and a line-of-sight direction vector of the surface point in a tangent space of the surface point; determining a refraction direction vector based on the line-of-sight direction vector, performing a parallax shift operation on the initial texture coordinates based on the refraction direction vector and a height value corresponding to the surface point, and generating parallax shift texture coordinates; determining intersection coordinates of an opposite direction of the line-of-sight direction vector and a virtual refraction surface corresponding to the surface point, generating internal refraction texture coordinates based on the intersection coordinates; mixing the parallax shift texture coordinates and the internal refraction texture coordinates to generate sampling texture coordinates, sampling a light effect texture map of the target gemstone using the sampling texture coordinates, and determining a color of the pixel fragment based on a texture sampling result.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image rendering method, apparatus and medium. Background Technology

[0002] In related technologies, applications such as mobile games require real-time image rendering of 3D models of faceted gemstones like diamonds. Currently, rendering faceted gemstones such as diamonds typically relies on complex physical optics calculation models, employing real-time ray tracing algorithms based on path tracing, photon mapping, spectral rendering, or fine refraction-reflection iterative solutions. These methods require high-precision simulation of multiple total internal reflections and refraction paths within the gemstone, combined with calculations of physical effects such as dispersion, Fresnel reflection, and volume absorption, making the optical calculations quite complex. Faceted gemstones like diamonds have numerous facets, each causing an exponential increase in the light path, making real-time rendering of these gemstone 3D models require extremely high computing resources and video memory bandwidth, resulting in high performance consumption that is difficult for mobile devices to support. Summary of the Invention

[0003] The purpose of this application is to provide an image rendering method, apparatus, and medium that balances performance consumption and rendering effect, providing a low-performance, realistic image rendering method for faceted gemstone models such as diamonds.

[0004] In a first aspect, an embodiment of this application provides an image rendering method, the method comprising: Obtain the surface point data corresponding to the pixel fragment of the target gem model; wherein, the data includes: the initial texture coordinates of the surface point and the viewing direction vector of the surface point in the tangent space of the surface point; Based on the line-of-sight vector, the refraction direction vector is determined. Based on the refraction direction vector and the height value corresponding to the surface point, a parallax offset operation is performed on the initial texture coordinates to generate parallax offset texture coordinates. Determine the coordinates of the intersection point between the opposite direction of the line of sight vector and the virtual refractive surface corresponding to the surface point, and generate internal refractive texture coordinates based on the intersection point coordinates; The parallax offset texture coordinates and the internal refraction texture coordinates are mixed to generate sampled texture coordinates. The sampled texture coordinates are then used to sample the light effect texture map of the target gemstone. The color of the pixel fragment is determined based on the texture sampling results.

[0005] Secondly, embodiments of this application provide an image rendering apparatus, the apparatus comprising: The data acquisition module is used to acquire data of surface points corresponding to pixel segments of the target gem model; wherein, the data includes: the initial texture coordinates of the surface points and the viewing direction vector of the surface points in the tangent space of the surface points; The first generation module is used to determine the refraction direction vector based on the viewing direction vector, and to perform a parallax offset operation on the initial texture coordinates based on the refraction direction vector and the height value corresponding to the surface point to generate parallax offset texture coordinates. The second generation module is used to determine the intersection coordinates of the opposite direction of the viewing direction vector and the virtual refraction surface corresponding to the surface point, and generate internal refraction texture coordinates based on the intersection coordinates; The color determination module is used to mix the parallax offset texture coordinates and the internal refraction texture coordinates to generate sampled texture coordinates, use the sampled texture coordinates to sample the light effect texture map of the target gemstone, and determine the color of the pixel segment based on the texture sampling result.

[0006] Thirdly, embodiments of this application provide a storage medium for storing computer-executable instructions, which can be executed by a processor to perform the steps of the above-described image rendering method.

[0007] The image rendering method, apparatus, and medium provided in this application embodiment acquire data of surface points corresponding to pixel fragments of a target gemstone model. This data includes the initial texture coordinates of the surface points and the viewing direction vector of the surface points in the tangent space. Then, based on the viewing direction vector, a refraction direction vector is determined. Based on the refraction direction vector and the height value corresponding to the surface point, a parallax offset operation is performed on the initial texture coordinates to generate parallax offset texture coordinates. The intersection coordinates of the opposite direction of the viewing direction vector and the virtual refraction surface corresponding to the surface point are determined. Internal refraction texture coordinates are generated based on the intersection coordinates. Finally, the parallax offset texture coordinates and the internal refraction texture coordinates are mixed to generate sampled texture coordinates. The sampled texture coordinates are used to sample the light effect texture map of the target gemstone. Based on the texture sampling results, the color of the pixel fragment is determined. This method can replace complex optical calculations with texture coordinate calculations and texture sampling, reducing performance consumption during gemstone image rendering, improving rendering effects, and facilitating efficient and realistic real-time rendering of faceted gemstone images with complex internal light interactions in scenarios such as mobile terminal games. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart illustrating an image rendering method according to an embodiment of this application; Figure 2 This is a schematic diagram of a diamond refraction texture map according to an embodiment of this application; Figure 3 This is a schematic diagram of a diamond caustic texture map according to an embodiment of this application; Figure 4 This is a schematic diagram illustrating a process for determining the color of a pixel segment according to an embodiment of this application; Figure 5 This is a schematic diagram of a diamond material capturing specular texture mapping according to an embodiment of this application; Figure 6 This is a schematic diagram of a module of an image rendering apparatus according to an embodiment of this application. Detailed Implementation

[0010] This application provides an image rendering method, apparatus, device, and medium.

[0011] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0012] This application provides an image rendering method, which can be a real-time dynamic rendering of gem images in a game program. The execution subject of this method can be a mobile terminal device, such as a mobile phone or tablet computer. Figure 1 This is a schematic flowchart illustrating an image rendering method according to an embodiment of this application. Figure 1 As shown, in each rendering frame, for each pixel fragment of the target gem model, an image rendering method according to an embodiment of this application may include the following steps: S102, Obtain the surface point data corresponding to the pixel segment of the target gem model.

[0013] The target gemstone can be a faceted gemstone made of transparent material, such as a diamond. The target gemstone model can be a 3D model of the target gemstone, such as a 3D model of a diamond. The pixel segments of the target gemstone model can be the pixel segments corresponding to the screen pixels occupied by the target gemstone model in the image rendering pipeline. The surface points corresponding to the pixel segments can be the position points on the surface of the target gemstone model corresponding to the pixel segments. The surface point data can be the attribute data of the surface points, which can include the initial texture coordinates of the surface points and the attribute data in the tangent space of the surface points, such as the view direction vector and normal direction vector of the surface points in the tangent space.

[0014] The tangent space of a surface point is a local tangent space with the surface point as the reference point, and it varies depending on the surface point. The tangent space of a surface point directly corresponds to the texture space. The X-axis and Y-axis of the tangent space correspond to the U-axis and V-axis of the texture space, respectively. The XY components of the viewing direction vector or refraction direction vector in the tangent space can be stably mapped to the displacement direction of the UV plane, driving the offset of the sampled texture coordinates. Specifically, the X-axis of the tangent space can be the Tangent axis (tangent axis), the Y-axis can be the Bitangent axis (bitangent axis), and the Z-axis can be the Normal axis (normal axis).

[0015] In implementation, Figure 1 The illustrated process can be executed by the fragment shader in the image rendering pipeline. Before step S102, when the game program on the mobile terminal needs to render the target gem image, it can load the model data of the target gem model and send the model data into the image rendering pipeline. The image rendering pipeline can perform vertex shading processing on the model data through the vertex shader, and then perform primitive assembly, connecting the vertices in the model into primitives such as triangles. After that, through rasterization processing, the primitives are converted into a series of pixel fragments covering the primitive area. The data of these pixel fragments can be input into the fragment shader, which then performs processing based on... Figure 1 The process shown performs fragment coloring on each pixel segment.

[0016] The model data of the target gemstone model may include vertex data and index data. Vertex data may include various attribute data of vertices, such as vertex position, normals, texture coordinates (UV coordinates), etc. Index data is used to define the connection relationship between vertices. In this embodiment, the target gemstone is a faceted gemstone such as a diamond. To make the final rendered gemstone image have a faceted effect, hard edge normals can be set for the target gemstone model when modeling it. The edges in the model can be marked or set as hard edges, so that two primitives sharing the same edge do not share the normal vector at their vertices during image rendering. Each primitive uses its own independent vertex normal, thereby making the final rendered model image present a faceted effect.

[0017] During the vertex shader stage, the view direction vector of each vertex can be obtained and transformed into the vertex's tangent space. During the rasterization stage, for each surface point corresponding to a pixel fragment, the view direction vector in the tangent space of the primitive to which the surface point belongs can be interpolated to obtain the view direction vector in the tangent space of the surface point. Through this process, the computationally expensive spatial transformation of the view direction can be moved from the frequently executed fragment shader to the less frequently executed vertex shader, reducing computational load and thus performance consumption.

[0018] It is worth mentioning that in this embodiment, the target gemstone model is rendered dynamically in real time. In the time dimension, the data value of the same surface point can change frame by frame; in the spatial dimension, the data values ​​of different surface points can be different. In each rendering frame, the data of the surface points corresponding to each pixel fragment of the target gemstone model in the current rendering frame can be obtained.

[0019] S104: Based on the viewing direction vector, determine the refraction direction vector. Based on the refraction direction vector and the height value corresponding to the surface point, perform a parallax offset operation on the initial texture coordinates to generate parallax offset texture coordinates.

[0020] The refraction direction vector represents the direction of light refraction after passing through the surface point and entering the target gem model; this vector can vary depending on the surface point. The height value corresponding to the surface point can be obtained by sampling the height map of the target gem model using the initial texture coordinates of the surface point; this value can also vary depending on the surface point. The parallax offset texture coordinates are the offset texture coordinates obtained through parallax calculation; these are the dynamic texture coordinates of the surface point, changing frame by frame and varying depending on the surface point.

[0021] In implementation, the refraction direction vector of the surface point can be calculated in the tangent space of the surface point using Snell's law, based on the viewing direction vector, the normal direction vector, and the refractive index of the target gemstone. Based on the initial texture coordinates of the surface point, the height map of the target gemstone model is sampled to obtain the height value corresponding to the surface point. Based on the refraction direction vector and the corresponding height value of the surface point, a parallax mapping operation is performed to obtain the parallax offset vector of the surface point. Based on the parallax offset vector, the initial texture coordinates of the surface point are offset to generate the parallax offset texture coordinates of the surface point. The height map of the target gemstone model can be a refraction texture map of the target gemstone model. Partial code examples are shown below: refractDirTS = refract(-i.vDirTS, i.nDirTS, _IOR); height = tex2D(_ParallaxMap, i.uv).r; paraOffsetVec= refractDirTS.xy / refractDirTS.z height _ParallaxStrength; paraUV = paraOffsetVec + i.uv (s_u, s_v).

[0022] Here, `refractDirTS` represents the refraction direction vector in the tangent space of the surface point. `i` represents the data of the surface point corresponding to the pixel segment. `i.vDirTS` represents the viewing direction vector in the tangent space of the surface point, i.e., the direction vector from the surface point to the game scene camera. `-i.vDirTS` represents the refraction incident direction vector, i.e., the direction vector from the game scene camera to the surface point. `i.nDirTS` represents the normal direction vector in the tangent space of the surface point. `_IOR` represents the refractive index of the target gemstone. `refract(i.vDirTS, i.nDirTS, _IOR)` is a function that uses Snell's law to calculate the refraction direction vector using `i.vDirTS`, `i.nDirTS`, and `_IOR`.

[0023] `height` represents the height value corresponding to the surface point. `_ParallaxMap` represents the height map of the target gem model, `i.uv` represents the initial texture coordinates of the surface point, and `tex2D(_ParallaxMap, i.uv).r` means sampling the height map using the initial texture coordinates of the surface point, taking the sampled value of the red channel.

[0024] `paraOffsetVec` represents the parallax offset vector. `refractDirTS.xy` represents the projection of the refraction direction vector in the tangent space of the surface point onto the XY plane of that surface point's tangent space. `refractDirTS.z` represents the projection of the refraction direction vector of the surface point onto the Z-axis of that surface point's tangent space. `_ParallaxStrength` represents the parallax offset intensity coefficient, which is an adjustable parameter. `refractDirTS.xy` / `refractDirTS.z` height _ParallaxStrength indicates a parallax mapping operation based on the refraction direction vector and height value.

[0025] paraUV represents the parallax offset texture coordinates. s_u represents the U-axis scaling factor, and s_v represents the V-axis scaling factor; s_u and s_v are adjustable parameters. i.uv (s_u, s_v) represents scaling the original texture coordinates using scaling factors, paraOffsetVec+ i.uv (s_u, s_v) indicates that the scaled original texture coordinates are offset using the disparity offset vector.

[0026] In this embodiment, the height value of the surface point is obtained by sampling the height map of the target gemstone model by calculating the refraction direction vector of the surface point. Based on the refraction direction vector and the height value of the surface point, the parallax offset operation is performed on the initial texture coordinates. Based on the surface unevenness and the viewing direction, the initial offset caused by microscopic unevenness when light enters the gemstone surface can be simulated, which increases the sense of detail and roughness at the entrance. The surface parallax phenomenon caused by the internal refraction of light after entering the target gemstone is simulated, making the rendered gemstone image more three-dimensional.

[0027] S106, determine the coordinates of the intersection point between the opposite direction of the viewing direction vector and the virtual refraction surface corresponding to the surface point, and generate the internal refraction texture coordinates based on the intersection point coordinates.

[0028] The virtual refractive surface corresponding to a surface point is a conceptual geometric abstraction introduced to simulate the refraction of light within an object. It is not a real geometric model, but rather a reference plane used to simplify calculations. The depth of the virtual plane corresponding to a surface point can be determined based on the height value of the surface point, and it can vary depending on the surface point.

[0029] In implementation, the virtual refractive surface corresponding to a surface point can be determined based on its height value. The incident ray is determined by using the surface point's position coordinates as the incident starting point and the opposite direction of the surface point's line-of-sight vector as the incident direction. The coordinates of the intersection point between the incident ray and the virtual refractive surface corresponding to the surface point are calculated. The intersection coordinates are then scaled to obtain the internal refractive texture coordinates of the surface point.

[0030] In this embodiment, the target gemstone is a faceted gemstone, with each facet approximately planar. The local tangent spaces within the same facet are essentially consistent, and due to the hard-edge normal settings during modeling, the tangent spaces of adjacent facets do not share normals. To simplify calculations, reduce performance consumption, and improve rendering realism, rotational differences between different tangent spaces can be ignored, and the above calculations can be performed in a local-global hybrid tangent space. In the local-global hybrid tangent space, the position of a surface point can be taken from its position in the global tangent space, which can be determined based on the texture coordinates of the surface point. The viewing direction vector can be taken from the viewing direction vector in the tangent space corresponding to the surface point, i.e., the local tangent space. Unlike the local tangent space, which serves as the local coordinate system of the surface point, the global tangent space can be based on the tangent space of the texture space, i.e., the global texture coordinate space. This space is a three-dimensional extension of the texture space, a unified tangent space with UV coordinates as XY coordinates. In this space, for any point on the model, its coordinates can be represented by its texture coordinates (u,v), and the z-coordinate is always 0.

[0031] The code examples for some of the steps are shown below: GetPosAnyPlaneCrossDir ( posPlane,posRay, nDirPlane,nDirRay ) { deltaPos = posPlane - posRay; temp = dot(nDirPlane, deltaPos) / dot(nDirPlane, nDirRay); return temp nDirRay + posRay; } o.UV = GetPosAnyPlaneCrossDir((0, 0, -_Thick - height _HeightScale),(i.uv.xy, 0), (0, 0, 1), -i.vDirTS).

[0032] refractUV = _Refract_ST.xy o.UV + _Refract_ST.zw.

[0033] GetPosAnyPlaneCrossDir is a function that calculates the intersection point of a ray and a plane.

[0034] `posPlane` represents the position of the virtual refractive surface corresponding to the surface point. This position can be determined by the height value of the surface point and changes as the height value changes. The position of the virtual refractive surface can be a planar position in global tangent space (0, 0, -_Thick - height). _HeightScale). Where -_Thick represents the base height, height represents the height value corresponding to the surface point, and _HeightScale represents the height scaling factor.

[0035] posRay represents the starting position of the incident ray. This starting position is the position of the surface point in the global tangent space. This position can be determined based on the initial texture coordinates of the surface point and can be equal to (i.uv.xy, 0).

[0036] nDirPlane represents the normal direction vector of the virtual refraction plane, with a value of (0, 0, 1).

[0037] nDirRay represents the direction of the incident ray, which can be the opposite direction of the line-of-sight vector of a surface point in the local tangent space, i.e., -i.vDirTS.

[0038] dot(nDirPlane, deltaPos) calculates the projection length of the vector deltaPos onto the plane normal, which represents the perpendicular distance from the ray origin to the virtual plane.

[0039] dot(nDirPlane, nDirRay) calculates the dot product of the ray direction and the plane normal, representing the degree to which the ray is oriented towards the plane normal, i.e., the degree to which the ray is nearly perpendicular to the plane.

[0040] temp represents the core scaling factor.

[0041] o.UV represents the coordinates of the intersection point. Refract_ST.xy and _Refract_ST.zw are the coordinate transformation coefficients.

[0042] refractUV represents the internal refractive texture coordinates of a surface point.

[0043] In this embodiment, the virtual refraction surface is a virtual plane defined inside the target gemstone model that is parallel to the surface tangent plane. The internal refraction texture coordinates are generated based on the intersection coordinates of the opposite direction of the viewing direction and the virtual refraction surface. This can simulate the internal texture effect that is visually seen after light is refracted inside the gemstone. With low computational overhead, it approximates the visual perception of the direction shift and optical path change that occurs when light propagates inside the target gemstone volume, making the refraction texture present obvious volume and internal structural differences.

[0044] Furthermore, the target gemstone is a faceted gemstone such as a diamond. Theoretically, when finding the intersection with a virtual refractive surface, the direction of refraction is usually used as the incident direction to simulate the refraction path. In practical applications, faceted gemstones such as diamonds typically have a large number of facets, the light path inside the gemstone is extremely complex, and the direction of the refracted light is highly variable. Attempting to accurately simulate a single refraction path is counterproductive. In this embodiment, it is assumed that regardless of how the light refracts inside the target gemstone, the apparent visual path of the light from a point in the internal virtual layer that the user's eye ultimately sees is roughly a straight line from the eye to that point. The opposite direction of the line of sight is used as the incident direction to find the intersection with the virtual refractive surface. This method can balance physical approximation and visual deception, providing a stable, efficient, and visually appealing simplified method.

[0045] S108, perform a mixing operation on the parallax offset texture coordinates and the internal refraction texture coordinates to generate the sampled texture coordinates of the surface points, use the sampled texture coordinates to perform texture sampling on the light effect texture map of the target gemstone, and determine the color of the pixel fragment based on the texture sampling results.

[0046] Specifically, the above-mentioned blending operation can employ weighted blending. The light effect texture map of the target gemstone is a visual effect map generated by simulating the interaction between light and the internal medium of the target gemstone. This effect map can be a light effect volumetric map, such as the refractive texture map or the caustic texture map of the target gemstone.

[0047] The refractive texture map of the target gemstone is a texture pattern that simulates the refractive effect of the target gemstone. Its texel values ​​encode the refractive effect information that is ultimately exhibited after light shines at a specific angle and passes through the volume of the target gemstone. This effect is the base effect of the target gemstone. The caustic texture map of the target gemstone is a texture pattern that simulates the caustic effect of the target gemstone. Its texel values ​​are encoded as the dispersive effect information that is ultimately exhibited after light shines at a specific angle and passes through the volume of the target gemstone.

[0048] In this embodiment, the refractive texture map of the target gemstone is used as the height map of the target gemstone for height sampling, and the refractive texture map of the target gemstone is also used for refractive texture sampling. The refractive texture map of the target gemstone is a texture image used to simulate the internal light refraction effect of the target gemstone, which simultaneously encodes surface height information and refractive texture information. In one embodiment, the refractive texture map can be four-dimensionally continuous, comprising multiple closely arranged triangles with random orientations. Each triangle represents a facet, and the edges of each triangle are outlined by thin white lines to strengthen the boundaries between facets and avoid structural blurring during refractive texture sampling or height sampling. The interior of each triangle is filled with black and white gradients in different directions. The directional changes of the black and white gradients are used to represent the spatial normal differences of the facets, and the gradient values ​​of the black and white gradients are used to represent the depth changes or transmission thickness changes of the refraction path presented by light passing through the facets.

[0049] For example, the target gemstone could be a diamond. A diamond refraction texture map could be like... Figure 2 As shown. Figure 2 As shown, diamond refraction texture sticker Figure 4 A tileable quadrilateral pattern is composed of closely arranged triangles of various shapes. Each triangle is filled with black and white gradients in different directions. The direction of these gradients expresses the spatial difference in the normals of the refractive surfaces, and the gradient magnitude can represent the depth or transmission thickness variations of the refraction path. The edges of the triangles are filled with thin white lines, which strengthens the boundaries between the refractive surfaces, preventing structural blurring during sampling or parallax perturbations, and helping to maintain a clear sense of separation between facets in dynamic refraction calculations. Diamond caustic texture maps can be used as follows: Figure 3 As shown. Figure 3 As shown, diamond caustic texture patch Figure 4 It is a continuous spectrum with iridescent dispersion, used to represent the dispersion effect inside a diamond.

[0050] In this embodiment, texture coordinates are generated using two methods: parallax offset operation on the initial texture coordinates and the intersection coordinates of the opposite direction of the viewing direction vector and the virtual refraction surface. The texture coordinates of these two methods are mixed, and the texture map of the target gemstone is sampled using the mixed texture coordinates. This reduces rendering performance consumption while ensuring that the rendered pixel colors not only contain surface parallax details but also show the internal texture effect "seen" after light is refracted inside the gemstone. Visually, it presents the brilliant light effect transmitted after light is refracted, reflected, and transmitted inside the three-dimensional gemstone, greatly enhancing the realism and visual impact of the gemstone rendering.

[0051] In one implementation, to achieve more complex lighting effect simulation and further improve the realism of gemstone rendering, the lighting effect texture map of the target gemstone may include both a refractive texture map and a caustic texture map of the target gemstone. The refractive texture map of the target gemstone can be sampled using sampled texture coordinates to obtain the refractive color, and the caustic texture map of the target gemstone can be sampled using the same sampled texture coordinates to obtain the caustic color. Multiplying the refractive color and the caustic color yields the color of the pixel segment.

[0052] In one implementation, to achieve more complex lighting effect simulation and further improve the realism of the rendering, dual-spectrum disparity offset texture coordinates can be used. There can be two disparity offset texture coordinates, namely a first disparity offset texture coordinate and a second disparity offset texture coordinate. Correspondingly, there can be two sampling texture coordinates, namely a first sampling texture coordinate and a second sampling texture coordinate. The first sampling texture coordinate can be obtained by weighted mixing of the first disparity offset texture coordinate and the internal refraction texture coordinate, and the second sampling texture coordinate can be obtained by weighted mixing of the second disparity offset texture coordinate and the internal refraction texture coordinate. A parallax mapping operation can be performed based on the refraction direction vector and the height value corresponding to the surface point to obtain a parallax offset vector. The initial texture coordinates are first scaled using a first scaling factor, and the first scaled initial texture coordinates are offset based on the parallax offset vector to generate the first parallax offset texture coordinates. The initial texture coordinates are second scaled using a second scaling factor, and the second scaled initial texture coordinates are offset based on the parallax offset vector to generate the second parallax offset texture coordinates. The first parallax offset texture coordinates and the internal refraction texture coordinates are mixed to generate the first sampled texture coordinates. The light effect texture map of the target gemstone is sampled using the first sampled texture coordinates, and the light effect texture map of the target gemstone is sampled using the second sampled texture coordinates. The color of the pixel segment is determined by combining the results of the two samplings.

[0053] The first scaling factor and the second scaling factor can be a set of factors with different and slightly asymmetrical main scaling directions. The main scaling direction of the first scaling factor can be the U direction, used to stretch the initial texture coordinates in the U direction. The main scaling direction of the second scaling factor can be the V direction, used to stretch the texture coordinates in the V direction. The stretching coefficients of the two directions are close but not exactly the same. For example, the first scaling factor can be (1.8, 1), and the second scaling factor can be (1, 1.79). Partial code examples are shown below: paraUV1 = paraOffsetVec + i.uv (s_u1, s_v1) paraUV2 = paraOffsetVec + i.uv (s_u2, s_v2) GemUV1 = paraUV1 _ParaUVInt + refractUV _RefractUVInt; GemUV2 = paraUV2 _ParaUVInt + refractUV _RefractUVInt; Wherein, paraOffsetVec represents the disparity offset vector, i.uv represents the initial texture coordinates, (s_u1, s_v1) represents the first scaling factor, paraUV1 represents the first disparity offset texture coordinates, (s_u2, s_v2) represents the second scaling factor, paraUV2 represents the second disparity offset texture coordinates, refractUV represents the internal refraction texture coordinates, _ParaUVInt represents the disparity offset intensity, _RefractUVInt represents the internal refraction intensity, GemUV1 represents the first sampled texture coordinates, and GemUV2 represents the second sampled texture coordinates.

[0054] Figure 4 This is a schematic diagram illustrating a process for determining the color of a pixel segment according to an embodiment of this application. For example... Figure 4 As shown, in one embodiment of this application, step S108 may specifically include the following steps: S402, the refractive texture map of the target gemstone is sampled using the first sampled texture coordinates and the second sampled texture coordinates respectively, and the refractive color is obtained based on the product of the sampling results.

[0055] In implementation, the refractive texture map of the target gemstone can be sampled using the first sampling texture coordinates. Based on the value of the specified channel in the sampling result and the first intensity coefficient, the first refractive color is obtained. The refractive texture map of the target gemstone can be sampled using the second sampling texture coordinates. Based on the value of the specified channel in the sampling result and the second intensity coefficient, the second refractive color is obtained. The first refractive color and the second refractive color are multiplied together to obtain the final refractive color.

[0056] S404, use the first or second sampled texture coordinates to sample the caustic texture map of the target gem to obtain the caustic color.

[0057] S406 uses caustic colors to modulate refracted colors, resulting in caustic-modulated colors.

[0058] One method is to use caustic color to modulate refracted color. Specifically, this can be achieved by multiplying the caustic color and the refracted color, and the result of the multiplication is the caustic-modulated color.

[0059] S408 determines the color of a pixel segment based on the caustic-modulated color.

[0060] Among these, the color obtained by caustic modulation can be determined as the color of a pixel segment.

[0061] The code examples for the above steps are shown below: gemPara1 = tex2D(GemTex, GemUV1).r _paraInt1; gemPara2 = tex2D(GemTex, GemUV2).r _paraInt2; gemColor = saturate(gemPara1 gemPara2).

[0062] causticCol=tex2D(_CausticTex, GemUV1); gemColor = causticCol.

[0063] Where GemTex represents the refractive texture map of the target gem, GemUV1 represents the first sampled texture coordinates, _paraInt1 represents the first intensity coefficient, gemPara1 represents the first refractive color, GemUV2 represents the second sampled texture coordinates, _paraInt2 represents the second intensity coefficient, gemPara2 represents the second refractive color, and saturate(gemPara1) represents the second refractive color. gemPara2) means gemPara1 `gemPara2` is clamped within the range [0,1], where `gemColor` represents the refraction color. `_CausticTex` represents the caustic texture map of the target gem, and `causticCol` represents the caustic color.

[0064] In the above embodiments, the caustic-modulated color is directly determined as the color of the pixel segment. In some embodiments, random highlights and / or ambient colors can be further superimposed on the caustic-modulated color, and the superimposed color can be determined as the color of the pixel segment to further increase the realism of the image rendering.

[0065] In one implementation, the specular sampling coordinates can be determined based on the normal direction vector in the view space of the surface points. The specular texture map of the target gemstone is sampled based on the specular sampling coordinates to obtain the specular color. The specular color is then superimposed with the caustic-modulated color to obtain the superimposed specular color. The color of the pixel segment is determined based on the superimposed specular color.

[0066] This includes the Material Capture (MatCap) specular texture map of the target gemstone. In the case of a diamond, the Material Capture specular texture map would look like this: Figure 5 As shown. Partial code examples are illustrated below: nDirVS = mul(UNITY_MATRIX_V, normalWorld); matcapUV = saturate(nDirVS.xy 0.5 + 0.5); matcapSpecular = tex2D(_Matcap, matcapUV).rgb; GemColor = 1 + matcapSpecular.xyz _MatcapIntensity.

[0067] Where `nDirVS` represents the normal direction vector in view space, and `normalWorld` represents the normal direction vector in world space. `mul(UNITY_MATRIX_IT_MV, normalWorld)` converts the world space normal direction vector `normalWorld` to the view space normal vector `nDirVS`. `matcapUV` is the texture coordinate used to sample the specular texture map of the material; `matcapUV = saturate(nDirVS.xy)`. 0.5 + 0.5) represents mapping the horizontal orientation (x, y) of the view space normal from the range [-1, 1] to the texture coordinate space [0, 1]. matcapSpecular represents the color value sampled from the material capture specular texture map, and _Matcap represents the material capture specular texture map. GemColor can be the caustic modulated color. = 1 + matcapSpecular.xyz _MatcapIntensity indicates that specular / ambient reflections simulated using Matcap textures are used to brighten the causticized color of the gemstone.

[0068] In the above embodiments, the Matcap specular texture map is sampled, and direct specular highlights are generated through the MatCap scheme. The specular values ​​are superimposed with the causticized colors to produce complex and realistic flickering and reflection effects, while the computational overhead is extremely low.

[0069] In one implementation, a reflection vector can be determined based on the normal direction vector and the viewing direction vector in the view space of the surface point. The environmental cube texture map of the target gemstone is sampled based on the reflection vector to obtain the environmental color. The environmental color is then superimposed on the color of the specular overlay to obtain the color of the pixel segment.

[0070] In the above implementation, a Cubemap environment reflection scheme is further introduced, which can supplement the ambient specular highlights generated by the target gem under the surrounding ambient lighting. By superimposing the direct specular highlights generated by MatCap with the ambient reflection of the Cubemap, we can simultaneously obtain the faceted specular highlights with strong direct light directionality and transient brightness, as well as the secondary reflective highlights with soft ambient light and omnidirectional brightness. This combination can maintain the "sparkling point specular highlights" unique to the target gem material such as diamonds, and also make the overall brightness have a reasonable global distribution in the environment, thereby improving the spatial realism and lighting consistency of target gems such as diamonds in mobile rendering.

[0071] The above describes the image rendering method provided in the embodiments of this application. Based on the same idea, the embodiments of this application also provide an image rendering apparatus. Figure 6 As shown, the image rendering apparatus 600 includes: The data acquisition module 610 is used to acquire data of surface points corresponding to pixel segments of the target gem model; wherein, the data includes: the initial texture coordinates of the surface points and the viewing direction vector of the surface points in the tangent space of the surface points; The first generation module 620 is used to determine the refraction direction vector based on the viewing direction vector, and to perform a parallax offset operation on the initial texture coordinates based on the refraction direction vector and the height value corresponding to the surface point to generate parallax offset texture coordinates. The second generation module 630 is used to determine the intersection coordinates of the opposite direction of the viewing direction vector and the virtual refraction surface corresponding to the surface point, and generate internal refraction texture coordinates based on the intersection coordinates; The color determination module 640 is used to mix the parallax offset texture coordinates and the internal refraction texture coordinates to generate sampled texture coordinates, use the sampled texture coordinates to sample the light effect texture map of the target gemstone, and determine the color of the pixel segment based on the texture sampling result.

[0072] In one embodiment, the height value corresponding to the surface point is obtained by sampling the height map of the target gemstone model based on the initial texture coordinates, the virtual refractive surface corresponding to the surface point is determined based on the height value corresponding to the surface point, and the height map of the target gemstone is generated based on the refractive texture map of the target gemstone.

[0073] In one embodiment, the disparity offset texture coordinates include: a first disparity offset texture coordinate and a second disparity offset texture coordinate, and the sampled texture coordinates include: a first sampled texture coordinate and a second sampled texture coordinate; the first sampled texture coordinate is obtained by weighted mixing of the first disparity offset texture coordinate and the internal refraction texture coordinate, and the second sampled texture coordinate is obtained by weighted mixing of the second disparity offset texture coordinate and the internal refraction texture coordinate.

[0074] In one embodiment, the first generation module 620 is specifically used for: A parallax mapping operation is performed based on the refraction direction vector and the height value corresponding to the surface point to obtain the parallax offset vector. The initial texture coordinates are scaled using a first scaling factor, and the first scaled initial texture coordinates are offset based on the disparity offset vector to generate the first disparity offset texture coordinates. The initial texture coordinates are scaled a second time using a second scaling factor, and the scaled initial texture coordinates are offset based on the disparity offset vector to generate the second disparity offset texture coordinates.

[0075] In one embodiment, the light effect texture map includes: a refractive texture map and a caustic texture map, and the color determination module 640 is specifically used for: The refractive texture map of the target gemstone is sampled using the first and second sampled texture coordinates, and the refractive color is obtained based on the product of the sampling results. The caustic texture map of the target gem is sampled using the first or the second sampled texture coordinates to obtain the caustic color; The refracted color is modulated using the caustic color to obtain a caustic-modulated color; The color of the pixel segment is determined based on the causticized color.

[0076] In one embodiment, the color determination module 640 is specifically used for: Based on the normal direction vector in the view space of the surface points, the specular sampling coordinates are determined, and the specular texture map of the target gemstone is sampled based on the specular sampling coordinates to obtain the specular color. The highlight color is superimposed with the causticized color to obtain the superimposed highlight color; The color of the pixel segment is determined based on the color of the superimposed specular highlights.

[0077] In one embodiment, the color determination module 640 is specifically used for: Based on the normal direction vector and the viewing direction vector in the view space of the surface points, the reflection vector is determined, and the environmental cube texture map of the target gem is sampled based on the reflection vector to obtain the environmental color; The color of the pixel segment is obtained by superimposing the ambient color onto the color of the specular overlay.

[0078] This application provides an image rendering apparatus that can serve as the execution subject of the above-described image rendering method and implement the steps of the above-described image rendering method. Since the principle is the same, it will not be described again here.

[0079] The above describes the image rendering apparatus provided in the embodiments of this application. Based on the same concept, the embodiments of this application also provide an image rendering device. The image rendering device is used to execute the above-described image rendering method, and is, for example, a mobile terminal device, such as a mobile phone or tablet computer. Image rendering devices can vary significantly due to differences in configuration or performance, and may include one or more processors and a memory. The memory may store computer-executable instructions, which, when executed, cause the processor to perform the steps of the above-described image rendering method. Since the principle is the same, it will not be described again here.

[0080] Furthermore, embodiments of this application also provide a non-transient readable storage medium for storing computer-executable instructions that can be executed by a processor to implement the steps of the above-described image rendering method. Since the principle is the same, it will not be described again here.

[0081] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0082] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0083] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0084] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0085] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing one or more embodiments of this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0086] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] Embodiments of this application are described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable parallel device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable parallel device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable fraud device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions can also be loaded onto a computer or other programmable device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0091] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0092] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0093] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0094] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, one or more embodiments of this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this application may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] One or more embodiments of this application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more embodiments of this application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0096] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0097] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An image rendering method, characterized in that, The method includes: Obtain the surface point data corresponding to the pixel fragment of the target gem model; wherein, the data includes: the initial texture coordinates of the surface point and the viewing direction vector of the surface point in the tangent space of the surface point; Based on the line-of-sight vector, the refraction direction vector is determined. Based on the refraction direction vector and the height value corresponding to the surface point, a parallax offset operation is performed on the initial texture coordinates to generate parallax offset texture coordinates. Determine the coordinates of the intersection point between the opposite direction of the line of sight vector and the virtual refractive surface corresponding to the surface point, and generate internal refractive texture coordinates based on the intersection point coordinates; The parallax offset texture coordinates and the internal refraction texture coordinates are mixed to generate sampled texture coordinates. The sampled texture coordinates are then used to sample the light effect texture map of the target gemstone. The color of the pixel fragment is determined based on the texture sampling results.

2. The method according to claim 1, characterized in that, The height value corresponding to the surface point is obtained by sampling the height map of the target gemstone model based on the initial texture coordinates. The virtual refractive surface corresponding to the surface point is determined based on the height value corresponding to the surface point. The height map of the target gemstone is generated based on the refractive texture map of the target gemstone.

3. The method according to claim 1, characterized in that, The disparity offset texture coordinates include: a first disparity offset texture coordinate and a second disparity offset texture coordinate; the sampled texture coordinates include: a first sampled texture coordinate and a second sampled texture coordinate; the first sampled texture coordinate is obtained by weighted mixing of the first disparity offset texture coordinate and the internal refraction texture coordinate; the second sampled texture coordinate is obtained by weighted mixing of the second disparity offset texture coordinate and the internal refraction texture coordinate.

4. The method according to claim 3, characterized in that, The step of performing a disparity offset operation on the initial texture coordinates based on the refraction direction vector and the height value corresponding to the surface point to generate disparity offset texture coordinates includes: A parallax mapping operation is performed based on the refraction direction vector and the height value corresponding to the surface point to obtain the parallax offset vector. The initial texture coordinates are scaled using a first scaling factor, and the first scaled initial texture coordinates are offset based on the disparity offset vector to generate the first disparity offset texture coordinates. The initial texture coordinates are scaled a second time using a second scaling factor, and the scaled initial texture coordinates are offset based on the disparity offset vector to generate the second disparity offset texture coordinates.

5. The method according to claim 3, characterized in that, The light effect texture map includes: a refraction texture map and a caustic texture map. The step of sampling the light effect texture map of the target gemstone using sampled texture coordinates, and determining the color of the pixel segment based on the texture sampling results, includes: The refractive texture map of the target gemstone is sampled using the first and second sampled texture coordinates, and the refractive color is obtained based on the product of the sampling results. The caustic texture map of the target gem is sampled using the first or the second sampled texture coordinates to obtain the caustic color; The refracted color is modulated using the caustic color to obtain a caustic-modulated color; The color of the pixel segment is determined based on the causticized color.

6. The method according to claim 5, characterized in that, The process of obtaining the color of the pixel segment based on the caustic-modulated color includes: Based on the normal direction vector in the view space of the surface points, the specular sampling coordinates are determined, and the specular texture map of the target gemstone is sampled based on the specular sampling coordinates to obtain the specular color. The highlight color is superimposed with the causticized color to obtain the superimposed highlight color; The color of the pixel segment is determined based on the color of the superimposed specular highlights.

7. The method according to claim 6, characterized in that, Determining the color of the pixel segment based on the color after the specular overlay includes: Based on the normal direction vector and the viewing direction vector in the view space of the surface points, the reflection vector is determined, and the environmental cube texture map of the target gem is sampled based on the reflection vector to obtain the environmental color; The color of the pixel segment is obtained by superimposing the ambient color onto the color of the specular overlay.

8. The method according to claim 2 or 5, characterized in that, The refractive texture map is continuous in all directions and includes multiple closely arranged triangles with random orientations. Each triangle represents a facet, and the edges of each triangle are outlined by thin white lines. The interior of each triangle is filled with black and white gradients in different directions. The direction of the black and white gradients is used to represent the spatial normal differences of the facets, and the gradient value of the black and white gradients is used to represent the depth changes of the refraction path when light passes through the facets.

9. An image rendering apparatus, characterized in that, The device includes: The data acquisition module is used to acquire data of surface points corresponding to pixel segments of the target gem model; wherein, the data includes: the initial texture coordinates of the surface points and the viewing direction vector of the surface points in the tangent space of the surface points; The first generation module is used to determine the refraction direction vector based on the viewing direction vector, and to perform a parallax offset operation on the initial texture coordinates based on the refraction direction vector and the height value corresponding to the surface point to generate parallax offset texture coordinates. The second generation module is used to determine the intersection coordinates of the opposite direction of the viewing direction vector and the virtual refraction surface corresponding to the surface point, and generate internal refraction texture coordinates based on the intersection coordinates; The color determination module is used to mix the parallax offset texture coordinates and the internal refraction texture coordinates to generate sampled texture coordinates, use the sampled texture coordinates to sample the light effect texture map of the target gemstone, and determine the color of the pixel segment based on the texture sampling result.

10. A storage medium, characterized in that, The storage medium is used to store computer-executable instructions that can be executed by a processor to implement the steps of the image rendering method according to any one of claims 1 to 8.

Images