A parametric modeling method for generating a normal map of a three-dimensional object model

Abstract

The application discloses a method for generating a normal map of a three-dimensional object model, comprising the following steps: obtaining pixel data of an original image / map image of the object model; calculating the gradient of each pixel in the original image and the map image respectively; determining the normal vector of each pixel in the original image / map image according to the gradient of the pixel; fusing the original image and the map image by the following method to obtain the normal map: calculating the average value of the normal vector of each pixel in the original image and the normal vector of the corresponding pixel in the map image; converting the average value of the calculated normal vector into a color value according to a preset mapping relationship; storing the converted color value of each pixel in the original image in a normal map data set; and putting the data in the normal map data set back into the original image to obtain the normal map. The application provides a normal map generation solution capable of improving the stereoscopic effect of an image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a parametric modeling method for normal map of a generative three-dimensional object model. BACKGROUND

[0002] A normal map is a texture that contains information about the surface normal of a 3D model. It is a way to add detail to a 3D model without having to create a high-poly model. A normal map is a texture that contains information about the surface normal of a 3D model. It is a way to add detail to a 3D model without having to create a high-poly model.

[0003] Normal maps are often used in animation rendering and game screen production. A model with high detail is baked into a normal map through mapping, and then applied to the normal map channel of a low-end model, so that the surface has a rendering effect of light and shadow distribution, which can greatly reduce the number of surfaces and calculation content required for object rendering, thereby optimizing the rendering effect of animation and game.

[0004] However, normal maps are typically created based on three-dimensional models, which has at least the following drawbacks:

[0005] Creating a three-dimensional model for different objects requires a lot of work, and the creation of the resulting normal map is inefficient;

[0006] The three-dimensional model is established by a model maker, while the normal map is developed by another programmer, and both parties need to collaborate to complete the work;

[0007] Once the three-dimensional model is modified, the normal map needs to be regenerated and imported into the three-dimensional engine.

[0008] The disclosure of the above background art is only used to assist in understanding the inventive concept and technical solutions of the present application, and it does not necessarily belong to the prior art of the present application, nor does it necessarily provide technical teaching; in the absence of explicit evidence that the above content has been disclosed before the filing date of the present application, the above background art should not be used to evaluate the novelty and inventiveness of the present application. SUMMARY

[0009] The purpose of the present application is to provide a normal map parametric modeling method that does not require a three-dimensional model to achieve a stereoscopic image effect.

[0010] To achieve the above purpose, the technical solution adopted by the present application is as follows:

[0011] A parametric modeling method for normal map of a generative three-dimensional object model, comprising the following steps:

[0012] Obtain the pixel data of the original image of the object model and the pixel data of the texture image of the object model, wherein the pixel matrix size of the original image is the same as the pixel matrix size of the texture image;

[0013] Calculate the gradient of each pixel in the original image and the textured image respectively;

[0014] Based on the gradient of pixels in the original image, determine the normal vector of each pixel in the original image; based on the gradient of pixels in the texture image, determine the normal vector of each pixel in the texture image.

[0015] The normal map is obtained by fusing the original image and the texture image in the following way:

[0016] Calculate the average of the normal vector of each pixel in the original image and the normal vector of its corresponding pixel in the texture image, wherein the orientation of the pixel in the original image is the same as the orientation of its corresponding pixel in the texture image.

[0017] Based on the preset mapping relationship, the average value of the calculated normal vector is converted into a color value;

[0018] The color values ​​of each pixel in the original image after conversion are stored in the normal map dataset;

[0019] The data from the normal map dataset is put back into the original image to obtain the normal map.

[0020] Furthermore, following any one or a combination of the aforementioned technical solutions, the generative 3D object model normal map parametric modeling method further includes adjusting the smoothness of the current normal map through the following steps:

[0021] Create a reference height image, which is related to the amount of change in the color value of adjacent pixels in the current normal map;

[0022] Calculate the gradient of each pixel in the reference height image to determine the normal vector of each pixel in the reference height image;

[0023] Based on the normal vectors of each pixel in the original image and the normal vectors of each pixel in the reference height image, the original image and the reference height image are fused to obtain a transition normal map;

[0024] Using an image addition algorithm, the current normal map and the transition normal map are combined into a new current normal map with improved smoothness.

[0025] Further, any one of the technical solutions or the combination of the technical solutions mentioned above, the method further comprises:

[0026] calculating the color value difference between each pixel and its adjacent pixels in the current normal map;

[0027] The greater the color value difference, the greater the gray value of the pixel in the corresponding position in the reference height image.

[0028] Further, any one of the technical solutions or the combination of the technical solutions mentioned above, before adjusting the smoothness of the current normal map, further comprises:

[0029] If not, adjust the smoothness of the current normal map one or more times until the smoothness of the new current normal map reaches the preset standard.

[0030] Further, any one of the technical solutions or the combination of the technical solutions mentioned above, the color value difference between each pixel and its adjacent pixels in the current normal map is calculated, and the average color value difference is calculated;

[0031] The smoothness of the current normal map is evaluated in any of the following ways:

[0032] Determine the maximum value of the average color value difference corresponding to each pixel in the current normal map, and compare it with the first threshold value, if it is greater than the first threshold value, it is determined that the smoothness of the current normal map does not reach the preset standard;

[0033] Or, calculate the comprehensive average value of the average color value difference corresponding to each pixel in the current normal map, and compare it with the second threshold value, if it is greater than the second threshold value, it is determined that the smoothness of the current normal map does not reach the preset standard.

[0034] Further, any one of the technical solutions or the combination of the technical solutions mentioned above, the map image of the object model is an image representing the concave-convex texture of the surface of the object model obtained by pre-baking;

[0035] According to the pixel matrix size of the original image, the 2D brush context operation is performed on the map image, so that the pixel matrix size of the map image is the same as the pixel matrix size of the original image.

[0036] Further, any one of the technical solutions or the combination of the technical solutions mentioned above, after obtaining the normal map, further comprises:

[0037] define a transformation matrix from the tangent space of the normal map to a predetermined eye space based on the surface tangent, normal vector of the normal map;

[0038] divide the normal map into a plurality of polygons;

[0039] perform the following rendering operations on each polygon:

[0040] convert the surface tangent at each point in the polygon region to a coordinate in the eye space using the transformation matrix;

[0041] select an interpolation position in the polygon, and interpolate the corresponding polygon region coordinate of the texture image corresponding to the object model at the interpolation position;

[0042] read the texture bump value at each point in the corresponding polygon region from the texture image after the interpolation operation is completed;

[0043] convert the texture bump value to a texture bump value in the eye space using the transformation matrix;

[0044] associate the surface tangent converted to a coordinate in the eye space with the texture bump value converted to the eye space;

[0045] perform lighting calculation in the eye coordinate based on the associated coordinate and texture bump value in the eye space.

[0046] Further, any of the technical solutions or combinations of the technical solutions described above, the lighting calculation includes implementing Gouraud shading and Phong shading, wherein the Gouraud shading is a color value generated by interpolation at each point of the polygon of the normal map;

[0047] The Phong shading is to estimate the pixel color value of the area outside the interpolation position in the polygon of the normal map based on interpolation method.

[0048] Further, any of the technical solutions or combinations of the technical solutions described above, the data in the normal map data set is put back into the original image by the following way:

[0049] put the converted color value back into the pixel of the original image, which has the same position in the original image as the pixel used to calculate the average value of the corresponding normal vector;

[0050] until the converted color value is put back into each pixel of the original image, the normal map is obtained.

[0051] Further, any of the technical solutions or the combination of the technical solutions described above, the object model is an object in a real image or a virtual object made by using image software.

[0052] The technical solutions provided by the present application have the following beneficial effects:

[0053] a. A normal map can be generated without the need to establish a three-dimensional model of the object, improving the stereoscopic effect and realistic effect of the image;

[0054] b. The smoothness factor can be ignored in the initial normal map generation stage, reducing the difficulty of generation; the smoothness of the initial normal map is adjusted after it is generated;

[0055] c. Based on the transformation between the tangent space and the eye space, the surface tangent in the polygon of the normal map and the texture concave-convex are converted to the eye space, so that the final imaging effect is more consistent with the 3D scene. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0057] Figure 1 The flow chart of the parametric modeling method for generating the normal map of the three-dimensional object model is provided for an exemplary embodiment of the present application;

[0058] Figure 2 The flow chart of the normal map with smoothness meeting the requirements is provided for an exemplary embodiment of the present application;

[0059] Figure 3 The flow chart of the further rendering of the normal map is provided for an exemplary embodiment of the present application;

[0060] Figure 4 The original image of the object model is provided for an exemplary embodiment of the present application;

[0061] Figure 5 The original image of the object model is provided for an exemplary embodiment of the present application; Figure 4 The effect diagram after the normal map is generated for the original image. DETAILED DESCRIPTION

[0062] In order to better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the scope of protection of the present application.

[0063] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product or apparatus including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or apparatus.

[0064] A normal map is a special texture that is itself a picture. The RGB color channels of this picture do not store color, but instead encode the normal direction (i.e. the direction perpendicular to the surface) of each pixel of the object's surface; when rendering, the way light is calculated is changed by the normal map, allowing for extremely realistic bump, scratch, and other detail shading effects on low-polygon models without actually increasing the geometric complexity of the model.

[0065] In an embodiment of the present application, a normal map parameterized modeling method for generating a three-dimensional object model is provided, that is, a method for generating a normal map through an algorithm or an AI model without relying on a high-precision three-dimensional model. As shown in the figure, Figure 1 The normal map parameterized modeling method includes the following steps:

[0066] Firstly, pixel data of an original image of an object model and pixel data of a map image of the object model are obtained, wherein the pixel matrix size of the original image is the same as the pixel matrix size of the map image.

[0067] Specifically, the object model is an object in a real image or a virtual object made by using image software. For the object in the real image, the original image is the real image thereof; for the virtual object, the original image is a planar image of the virtual object. In other words, the original image can be a two-dimensional image, and it is not necessary to create a three-dimensional model of the object, but to directly infer and generate the normal map from multiple two-dimensional images.

[0068] Specifically, the mapping image of the object model is an image representing the concave-convex texture of the surface of the object model obtained by pre-baking; and the 2D brush context operation is performed on the mapping image according to the pixel matrix size of the original image, so that the pixel matrix size of the mapping image is the same as the pixel matrix size of the original image.

[0069] In the second step, the gradient of each pixel in the original image and the mapping image is calculated respectively.

[0070] The pixel gradient is a concept in image processing, which represents the change rate of a pixel in the image in x and y directions compared with adjacent pixels. It is a two-dimensional vector composed of two components, which are the changes in X and Y axes respectively. The calculation results of the two components form a two-dimensional vector, and thus the image gradient of the pixel is obtained.

[0071] In the third step, the normal vector of each pixel in the original image is determined according to the gradient of the pixel in the original image, and the normal vector of each pixel in the mapping image is determined according to the gradient of the pixel in the mapping image.

[0072] The normal vector is a key factor for generating the normal map, and is a vector represented by a straight line perpendicular to the plane, which can be calculated by the gradient.

[0073] In the fourth step, the original image and the mapping image are fused by the steps as shown in Figure 1 to obtain the normal map.

[0074] In step 4.1, the average value of the normal vector of each pixel in the original image and the normal vector of the corresponding pixel in the mapping image is calculated, wherein the position of the pixel in the original image is the same as the position of the corresponding pixel in the mapping image.

[0075] In step 4.2, the average value of the calculated normal vector is converted into a color value according to a preset mapping relationship.

[0076] In step 4.3, the converted color value of each pixel in the original image is stored in the normal map data set.

[0077] In step 4.4, the data in the normal map data set is put back into the original image to obtain the normal map. Specifically, the data in the normal map data set is put back into the original image in the following way: the converted color value is put back into the pixel of the original image, and the position of the pixel in the original image is the same as the position of the pixel in the original image used for calculating the average value of the corresponding normal vector; until the converted color value of each pixel in the original image is put back, the normal map is obtained.

[0078] by Figure 4 For example, the object model is a road surface, which is paved with multiple stones. Figure 4 The displayed road surface has obvious planar image features, resulting in insufficient three-dimensionality of the road surface and an unrealistic image effect;

[0079] After image processing to generate the normal map in this embodiment, the image with the normal map is as follows: Figure 5 As shown, compared to Figure 4 , Figure 5 The stones on the road surface are more three-dimensional, highlighting the unevenness of a real road surface.

[0080] In one embodiment of the present invention, after generating the normal map, the smoothness of the normal map is evaluated. If it does not meet a preset smoothness standard, the smoothness of the current normal map is adjusted, specifically as follows: Figure 2 As shown. For example, the smoothness of the normal map can be evaluated in the following ways:

[0081] Calculate the color value difference between each pixel in the current normal map and its multiple neighboring pixels, and calculate the average color value difference; determine the maximum value among the average color value differences corresponding to each pixel in the current normal map, and compare it with a preset first threshold. If it is greater than the first threshold, it indicates that the color value difference between adjacent pixels is too large, and the smoothness of the current normal map is not evaluated to meet the preset standard.

[0082] The smoothness of the normal map can also be evaluated in the following way: calculate the color value difference between each pixel in the current normal map and its multiple neighboring pixels, and calculate the average color value difference; calculate the comprehensive average value of the average color value difference corresponding to each pixel in the current normal map, and compare it with a preset second threshold. If it is greater than the second threshold, it means that the color value difference between adjacent pixels is too large, and the smoothness of the current normal map does not meet the preset standard.

[0083] If the smoothness of the current normal map does not meet the preset standard, adjust the smoothness of the current normal map using the following steps:

[0084] A reference height image is created, which is related to the change in color values ​​of adjacent pixels in the current normal map. The creation criteria for the reference height image can refer to the following: the pixel matrix size of the reference height image is the same as the pixel matrix size of the original image, and the color value difference between each pixel in the current normal map and its adjacent pixels is calculated; the larger the color value difference, the larger the grayscale value of the pixel in the corresponding position in the reference height image.

[0085] calculating the gradient of each pixel in the reference height image to determine the normal vector of each pixel in the reference height image;

[0086] fusing the original image and the reference height image according to the normal vector of each pixel in the original image and the normal vector of each pixel in the reference height image to obtain a transition normal map; the method of fusing the original image and the reference height image is the same as the method of fusing the original image and the map image in the above embodiment, as shown in Figure 2

[0087] Step 5.1, calculating the average of the normal vector of each pixel in the original image and the normal vector of the corresponding pixel in the reference height image, wherein the position of the pixel in the original image is the same as the position of the corresponding pixel in the reference height image;

[0088] Step 5.2, converting the calculated average of the normal vector into a color value according to a preset mapping relationship;

[0089] Step 5.3, putting the converted color value back into the pixel in the original image, wherein the position of the pixel in the original image is the same as the position of the pixel in the original image used for calculating the average of the corresponding normal vector;

[0090] Step 5.4, until the converted color value is put back into each pixel in the original image to obtain a transition normal map.

[0091] combining the current normal map and the transition normal map by using an image addition algorithm to obtain a new current normal map with improved smoothness.

[0092] Optionally, the smoothness of the new current normal map can be evaluated again, and if the smoothness still does not reach the preset standard, the transition normal map is obtained again by creating a reference height image and fusing it with the original image, and then the transition normal map is combined with the current normal map to obtain a new current normal map with further improved smoothness, until the smoothness of the new current normal map reaches the preset standard.

[0093] In an optional embodiment, after obtaining the normal map, or after obtaining the normal map with smoothness reaching the preset standard, a rendering operation as shown in Figure 3 is further performed:

[0094] defining a transformation matrix from the tangent space of the normal map to a preset eye space based on the surface tangent and the normal vector of the normal map;

[0095] dividing the normal map into a plurality of polygons;

[0096] ​performing the following rendering operations on each polygon:

[0097] converting surface tangents at various locations within the polygonal region to coordinates in eye space using the transformation matrix;

[0098] selecting an interpolation position within the polygon and interpolating corresponding polygonal region coordinates of a texture image corresponding to the object model at the interpolation position;

[0099] reading texture bump values at various locations within the corresponding polygonal region from the texture image after the interpolation operation is completed;

[0100] converting the texture bump values to texture bump values in eye space using the transformation matrix;

[0101] associating the surface tangents switched to coordinates in eye space with the texture bump values converted to texture bump values in eye space;

[0102] performing illumination calculations in eye coordinates based on the associated coordinates and texture bump values in eye space. Specifically, the illumination calculations include implementing Gouraud shading, which is a color value generated by interpolating at each point of the polygon of the normal map, and Phong shading, which is an estimation of pixel color values for areas outside the interpolation position of the polygon of the normal map based on interpolation.

[0103] It is noted that the relative terms such as first and second are used herein solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any such actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0104] The above description is merely that of a specific implementation of the application, and it is to be understood that many changes and modifications can be made to the application without departing from the spirit and scope thereof. Accordingly, it is intended to cover in the appended claims all such changes and modifications that fall within the scope of the application.

Claims

1. A generative 3D object model normal mapping parametric modeling method, characterized in that, Includes the following steps: Obtain the pixel data of the original image of the object model and the pixel data of the texture image of the object model, wherein the pixel matrix size of the original image is the same as the pixel matrix size of the texture image; Calculate the gradient of each pixel in the original image and the textured image respectively; Based on the gradient of pixels in the original image, determine the normal vector of each pixel in the original image; based on the gradient of pixels in the texture image, determine the normal vector of each pixel in the texture image. The normal map is obtained by fusing the original image and the texture image in the following way: Calculate the average of the normal vector of each pixel in the original image and the normal vector of its corresponding pixel in the texture image, wherein the orientation of the pixel in the original image is the same as the orientation of its corresponding pixel in the texture image. Based on the preset mapping relationship, the average value of the calculated normal vector is converted into a color value; The color values ​​of each pixel in the original image after conversion are stored in the normal map dataset; The data in the normal map dataset is put back into the original image to obtain the normal map; The method also includes adjusting the smoothness of the current normal map through the following steps: creating a reference height image, which is related to the change in color values ​​of adjacent pixels in the current normal map; calculating the gradient of each pixel in the reference height image to determine the normal vector of each pixel in the reference height image; fusing the original image and the reference height image according to the normal vectors of each pixel in the original image and the normal vectors of each pixel in the reference height image to obtain a transition normal map; and using an image addition algorithm, combining the current normal map and the transition normal map into a new current normal map with improved smoothness.

2. The generative 3D object model normal mapping parametric modeling method according to claim 1, characterized in that, Also includes: Calculate the color value difference between each pixel in the current normal map and its neighboring pixels; The larger the color value difference, the larger the gray value of the pixel in the corresponding orientation in the reference height image.

3. The generative 3D object model normal mapping parametric modeling method according to claim 1, characterized in that, Before adjusting the smoothness of the current normal map, the following steps are also included: Evaluate whether the smoothness of the current normal map meets the preset standard. If not, adjust the smoothness of the current normal map once or multiple times until the smoothness of the new current normal map meets the preset standard.

4. The generative 3D object model normal mapping parametric modeling method according to claim 3, characterized in that, Calculate the color value difference between each pixel in the current normal map and its multiple neighboring pixels, and calculate the average color value difference. Evaluate the smoothness of the current normal map using any of the following methods: Determine the maximum value among the average color value differences of each pixel in the current normal map, and compare it with a preset first threshold. If it is greater than the first threshold, the smoothness of the current normal map is evaluated as not meeting the preset standard. Alternatively, calculate the average value of the difference in average color values ​​corresponding to each pixel in the current normal map, and compare it with a preset second threshold. If the average value is greater than the second threshold, the smoothness of the current normal map is evaluated as not meeting the preset standard.

5. The generative 3D object model normal mapping parametric modeling method according to claim 1, characterized in that, The texture image of the object model is an image representing the surface bump texture of the object model obtained from the previous baking process; Based on the pixel matrix size of the original image, a 2D brush context operation is performed on the texture image so that the pixel matrix size of the texture image is the same as the pixel matrix size of the original image.

6. The generative 3D object model normal mapping parametric modeling method according to claim 5, characterized in that, After obtaining the normal map, the following is also included: Based on the surface tangents and normal vectors of the normal map, a transformation matrix is ​​defined from the tangent space of the normal map to a preset eye space; The normal map is divided into multiple polygons; Perform the following rendering operation on each polygon: Using the transformation matrix, the surface tangents at various points within the polygonal region are converted into coordinates in eye space; Select an interpolation position within the polygon, and interpolate the coordinates of the corresponding polygon region of the texture image corresponding to the object model at the interpolation position; Read the texture bump values ​​at each point within the corresponding polygonal region from the texture image after the interpolation operation is completed; Using the transformation matrix, the texture bump values ​​are converted to texture bump values ​​in eye space; The surface tangent is switched to coordinates in eye space and associated with the texture bump values ​​transformed to eye space; Illumination calculations are performed in eye coordinates based on the associated coordinates and texture bump values ​​in the eye space.

7. The generative 3D object model normal mapping parametric modeling method according to claim 6, characterized in that, Lighting calculations include implementing Glaude shading and Phong shading, where Glaude shading is a color value interpolated for each point of the polygon in the normal map; Phong shading is based on interpolation to estimate pixel color values ​​for regions outside the polygon interpolation locations of the normal map.

8. The generative 3D object model normal mapping parametric modeling method according to any one of claims 1 to 7, characterized in that, The data from the normal map dataset is then put back into the original image in the following manner: The converted color value is placed back into a pixel of the original image, the pixel being in the same orientation as the pixel in the original image used to calculate the average value of the corresponding normal vector. The process continues until the converted color values ​​of each pixel in the original image have been replaced, resulting in a normal map.

9. The generative 3D object model normal mapping parametric modeling method according to claim 8, characterized in that, The object model is an object in a real image or a virtual object created using image software.

Images