Generative three-dimensional object model normal mapping parametric modeling method

Abstract

The invention discloses a parametric modeling method for a normal map of a generative three-dimensional object model. The parametric modeling method comprises the following steps: acquiring pixel data of an original image / map image of the object model; respectively calculating the gradient of each pixel in the original image and the chartlet image; determining a normal vector of each pixel in the original image / chartlet image according to the gradient of the pixels in the original image / chartlet image; fusing the original image and the chartlet image to obtain a normal chartlet: calculating the average value of the normal vector of each pixel in the original image and the normal vector of the corresponding pixel in the chartlet image; converting the calculated average value of the normal vectors into a color value according to a preset mapping relation; storing the converted color value of each pixel in the original image in a normal map data set; and putting the data in the normal map data set back to the original image to obtain a normal map. The invention provides a normal map generation solution capable of improving the stereoscopic effect of the image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a parametric modeling method for generating normal map of a three-dimensional object model. BACKGROUND

[0002] A normal map is a texture map that contains information about the surface normal of a 3D model. It is a way to add detail to a 3D model without having to create a high-poly model. Normal maps are used in computer graphics to add surface detail to a 3D model. They are a way to add detail to a 3D model without having to create a high-poly model. Normal maps are used in computer graphics to add surface detail to a 3D model.

[0003] Normal maps are used in computer graphics to add surface detail to a 3D model. They are a way to add detail to a 3D model without having to create a high-poly model. Normal maps are used in computer graphics to add surface detail to a 3D model.

[0004] However, creating a normal map typically requires a three-dimensional model, which has at least the following drawbacks: Creating a three-dimensional model for different objects requires a lot of work, and the resulting normal map is created inefficiently; A three-dimensional model is established by a model maker, while a normal map is developed by another programmer, and both parties need to collaborate to complete the work; Once the three-dimensional model is modified, the normal map needs to be regenerated and imported into the three-dimensional engine.

[0005] The disclosure of the above background art content is only used to assist in understanding the inventive concept and technical solutions of the present application, and it does not necessarily belong to the prior art of the present application, nor does it necessarily provide technical teaching; in the absence of explicit evidence that the above content has been disclosed before the filing date of the present application, the above background art should not be used to evaluate the novelty and inventiveness of the present application. SUMMARY

[0006] The present application aims to provide a normal map parametric modeling method that does not require a three-dimensional model to achieve a stereoscopic image effect.

[0007] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows: A parametric modeling method for generating normal map of a three-dimensional object model, comprising the following steps: acquire pixel data of an original image of an object model and pixel data of a texture image of the object model, wherein a pixel matrix size of the original image is the same as a pixel matrix size of the texture image; respectively calculate gradients of each pixel in the original image and the texture image; determine normal vectors of each pixel in the original image according to the gradients of the pixels in the original image, and determine normal vectors of each pixel in the texture image according to the gradients of the pixels in the texture image; fuse the original image and the texture image to obtain a normal map by the following way: calculate an average value of the normal vector of each pixel in the original image and the normal vector of the corresponding pixel in the texture image, wherein the position of the pixel in the original image is the same as the position of the corresponding pixel in the texture image; convert the average value of the calculated normal vector into a color value according to a preset mapping relationship; store the converted color value of each pixel in the original image in a normal map data set; put the data in the normal map data set back into the original image to obtain a normal map.

[0008] Further, any one of the technical solutions or a combination of the technical solutions, the method for generating a normal map of a three-dimensional object model further comprises adjusting the smoothness of the current normal map by the following steps: create a reference height image, wherein the reference height image is related to the variation of the color values of adjacent pixels in the current normal map; calculate the gradients of each pixel in the reference height image to determine the normal vectors of each pixel in the reference height image; fuse the original image and the reference height image to obtain a transition normal map according to the normal vectors of each pixel in the original image and the normal vectors of each pixel in the reference height image; use an image addition algorithm to combine the current normal map and the transition normal map to obtain a new current normal map with improved smoothness.

[0009] Further, any one of the technical solutions or a combination of the technical solutions, the method for generating a normal map of a three-dimensional object model further comprises: calculate the color value difference between each pixel and its adjacent pixels in the current normal map; the greater the color value difference, the greater the gray value of the pixel at the corresponding position in the reference height image.

[0010] Further, any one of the above technical solutions or combinations thereof, before adjusting the smoothness of the current normal map, further comprises: evaluating whether the smoothness of the current normal map reaches a preset standard, if not, performing one or more times of adjusting the smoothness of the current normal map until the smoothness of the new current normal map reaches the preset standard.

[0011] Further, any one of the above technical solutions or combinations thereof, calculating the color value difference between each pixel in the current normal map and its adjacent pixels, and calculating the average color value difference thereof; evaluating the smoothness of the current normal map by any one of the following ways: determining the maximum value in the average color value difference corresponding to each pixel in the current normal map, and comparing the maximum value with a preset first threshold value, if the maximum value is greater than the first threshold value, evaluating that the smoothness of the current normal map does not reach the preset standard; or, calculating a comprehensive average value of the average color value difference corresponding to each pixel in the current normal map, and comparing the comprehensive average value with a preset second threshold value, if the comprehensive average value is greater than the second threshold value, evaluating that the smoothness of the current normal map does not reach the preset standard.

[0012] Further, any one of the above technical solutions or combinations thereof, the map image of the object model is an image representing the concave-convex texture of the surface of the object model obtained by pre-baking; performing a 2D brush context operation on the map image according to the pixel matrix size of the original image, so that the pixel matrix size of the map image is the same as the pixel matrix size of the original image.

[0013] Further, any one of the above technical solutions or combinations thereof, after obtaining the normal map, further comprises: defining a transformation matrix from the tangent space of the normal map to a preset eye space based on the surface tangent and the normal vector of the normal map; dividing the normal map into a plurality of polygons; performing the following rendering operation on each polygon: converting the surface tangent at each position in the polygon region into coordinates in the eye space by using the transformation matrix; selecting an interpolation position in the polygon, and interpolating the corresponding polygon region coordinates corresponding to the map image of the object model at the interpolation position; reading the texture concave-convex values at each position in the corresponding polygon region from the map image after completing the interpolation operation; Converting the texture bump value to a texture bump value in the eye space by using the transformation matrix; Associating the surface tangent in the eye space with the texture bump value converted to the eye space; Performing illumination calculation in the eye coordinate based on the associated coordinate and texture bump value in the eye space.

[0014] Further, any one of the technical solutions or the combination of the technical solutions described above, the illumination calculation comprises implementing Gouraud shading and Phong shading, wherein the Gouraud shading is a color value generated by interpolation in each point of the polygon of the normal map; The Phong shading is to estimate the pixel color value of the area outside the interpolation position of the polygon of the normal map based on interpolation.

[0015] Further, any one of the technical solutions or the combination of the technical solutions described above, the data in the normal map data set is put back to the original image by the following way: The converted color value is put back to the pixel of the original image, and the position of the pixel in the original image is the same as the position of the pixel in the original image used for calculating the average value of the corresponding normal vector; Until the converted color value is put back to each pixel of the original image, the normal map is obtained.

[0016] Further, any one of the technical solutions or the combination of the technical solutions described above, the object model is an object in a real image or a virtual object made by using image software.

[0017] The technical solutions provided by the application have the following beneficial effects: a. The normal map can be generated without establishing a three-dimensional model of the object, and the stereoscopic degree and realistic effect of the image are improved; b. The smoothness factor can be not considered in the initial normal map generation stage, and the generation difficulty is reduced; the smoothness adjustment is performed after the initial normal map is generated; c. The surface tangent in the polygon of the normal map and the texture bump are converted to the eye space based on the transformation between the tangent space and the eye space, so that the final imaging effect is more consistent with the 3D scene. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments or the prior art, the drawings needed in the embodiment or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 The flow chart of the method for generating a normal map of a three-dimensional object model according to an exemplary embodiment of the present application; Figure 2 The flow chart of the method for generating a normal map with a required smoothness according to an exemplary embodiment of the present application; Figure 3 The flow chart of the method for further rendering a normal map according to an exemplary embodiment of the present application; Figure 4 The original image of an object model according to an exemplary embodiment of the present application; Figure 5 The effect diagram of the original image of Figure 4 after generating a normal map. DETAILED DESCRIPTION

[0020] In order to make the personnel in the technical field better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by the personnel in the technical field without creative labor should belong to the protection scope of the present application.

[0021] It should be noted that the terms “first”, “second”, and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be exchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, device, product, or equipment including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products, or equipment.

[0022] A normal map is a special texture, which is a picture itself. The RGB color channels of this picture are not used to store colors, but encode the normal direction (i.e. the direction perpendicular to the surface) information of each pixel point of the object surface; when rendering, the calculation method of light is changed through the normal map, so that extremely realistic concave-convex, scratch, and other detailed light and shadow effects can be created on a low-polygon model without actually increasing the geometric complexity of the model.

[0023] In an embodiment of the present application, a parametric modeling method of generating a normal map of a three-dimensional object model is provided, that is, a method of generating a normal map by an algorithm or an AI model instead of relying on a high-precision three-dimensional model. As shown in Figure 1 The normal map parametric modeling method comprises the following steps: Firstly, pixel data of an original image of an object model and pixel data of a map image of the object model are obtained, wherein the pixel matrix size of the original image is the same as the pixel matrix size of the map image.

[0024] Specifically, the object model is an object in a real image or a virtual object made by image software. For an object in a real image, the original image is the real image thereof; for a virtual object, the original image is a planar image of the virtual object. In other words, the original image is a two-dimensional image, and it is not necessary to create a three-dimensional model of the object, but to directly infer and generate a normal map from multiple two-dimensional images.

[0025] Specifically, the map image of the object model is an image representing the concave-convex texture of the surface of the object model obtained by pre-baking; according to the pixel matrix size of the original image, a 2D brush context operation is performed on the map image, so that the pixel matrix size of the map image is the same as the pixel matrix size of the original image.

[0026] Secondly, the gradient of each pixel in the original image and the map image is calculated respectively. The pixel gradient is a concept in image processing, which represents the rate of change of a pixel in an image in x and y directions compared with adjacent pixels. It is a two-dimensional vector composed of two components, namely the change in X-axis and the change in Y-axis, and the calculation results of the two components form a two-dimensional vector, thus obtaining the image gradient of the pixel.

[0027] Thirdly, the normal vector of each pixel in the original image is determined according to the gradient of the pixel in the original image, and the normal vector of each pixel in the map image is determined according to the gradient of the pixel in the map image. The normal vector is a key factor for generating a normal map, which is a vector represented by a straight line perpendicular to a plane, and can be calculated by a gradient.

[0028] Fourthly, the original image and the map image are fused by the steps as shown in Figure 1 to obtain a normal map: Step 4.1, the average value of the normal vector of each pixel in the original image and the normal vector of the corresponding pixel in the map image is calculated, wherein the position of the pixel in the original image is the same as the position of the corresponding pixel in the map image. Step 4.2: Based on the preset mapping relationship, convert the average value of the calculated normal vector into a color value; Step 4.3: Store the converted color values ​​of each pixel in the original image in the normal map dataset; Step 4.4: Replace the data from the normal map dataset back into the original image to obtain a normal map. Specifically, the data from the normal map dataset is replaced back into the original image in the following way: the converted color value is replaced back into the pixel of the original image, and the orientation of this pixel in the original image is the same as the orientation of the pixel in the original image used to calculate the average value of the corresponding normal vector; until the converted color value replacement operation is completed for each pixel in the original image, a normal map is obtained.

[0029] by Figure 4 For example, the object model is a road surface, which is paved with multiple stones. Figure 4 The displayed road surface has obvious planar image features, resulting in insufficient three-dimensionality of the road surface and an unrealistic image effect; After image processing to generate the normal map in this embodiment, the image with the normal map is as follows: Figure 5 As shown, compared to Figure 4 , Figure 5 The stones on the road surface are more three-dimensional, highlighting the unevenness of a real road surface.

[0030] In one embodiment of the present invention, after generating the normal map, the smoothness of the normal map is evaluated. If it does not meet a preset smoothness standard, the smoothness of the current normal map is adjusted, specifically as follows: Figure 2 As shown. For example, the smoothness of the normal map can be evaluated in the following ways: Calculate the color value difference between each pixel in the current normal map and its multiple neighboring pixels, and calculate the average color value difference; determine the maximum value among the average color value differences corresponding to each pixel in the current normal map, and compare it with a preset first threshold. If it is greater than the first threshold, it indicates that the color value difference between adjacent pixels is too large, and the smoothness of the current normal map is not evaluated to meet the preset standard.

[0031] The smoothness of the normal map can also be evaluated in the following way: calculate the color value difference between each pixel in the current normal map and its multiple neighboring pixels, and calculate the average color value difference; calculate the comprehensive average value of the average color value difference corresponding to each pixel in the current normal map, and compare it with a preset second threshold. If it is greater than the second threshold, it means that the color value difference between adjacent pixels is too large, and the smoothness of the current normal map does not meet the preset standard.

[0032] In the case that the smoothness of the current normal map does not reach the preset standard, the smoothness of the current normal map is adjusted by the following steps: A reference height image is created, which is related to the variation of color values of adjacent pixels in the current normal map; the creation standard of the reference height image can refer to the following: the pixel matrix size of the reference height image is the same as that of the original image, and the color value difference of each pixel in the current normal map and its adjacent pixels is calculated; the greater the color value difference is, the greater the gray value of the pixel in the corresponding position of the reference height image is determined to be.

[0033] The gradient of each pixel in the reference height image is calculated to determine the normal vector of each pixel in the reference height image. The original image and the reference height image are fused according to the normal vector of each pixel in the original image and the normal vector of each pixel in the reference height image to obtain a transition normal map; the method of fusing the original image and the reference height image is the same as the method of fusing the original image and the map image in the above embodiment, as shown in the following formula: Figure 2 Step 5.1, the average value of the normal vector of each pixel in the original image and the normal vector of the corresponding pixel in the reference height image is calculated, wherein the position of the pixel in the original image is the same as the position of the corresponding pixel in the reference height image. Step 5.2, the average value of the calculated normal vector is converted into a color value according to a preset mapping relationship. Step 5.3, the converted color value is put back into the pixel of the original image, and the position of the pixel in the original image is the same as the position of the pixel in the original image used to calculate the average value of the corresponding normal vector. Step 5.4, until the converted color value is put back into each pixel in the original image, a transition normal map is obtained.

[0034] The current normal map and the transition normal map are combined by using an image addition algorithm to obtain a new current normal map with improved smoothness.

[0035] Optionally, the smoothness of the new current normal map can be evaluated again, and if the smoothness still does not reach the preset standard, the transition normal map is obtained again by creating a reference height image and fusing the original image, and then the transition normal map is combined with the current normal map to obtain a new current normal map with further improved smoothness, until the smoothness of the new current normal map reaches the preset standard.

[0036] In an optional embodiment, after obtaining the normal map, or after obtaining the normal map with smoothness reaching the preset standard, the following is further included:​Figure 3 The rendering operation shown: Based on the surface tangents and normal vectors of the normal map, a transformation matrix is ​​defined from the tangent space of the normal map to a preset eye space; The normal map is divided into multiple polygons; Perform the following rendering operation on each polygon: Using the transformation matrix, the surface tangents at various points within the polygonal region are converted into coordinates in eye space; Select an interpolation position within the polygon, and interpolate the coordinates of the corresponding polygon region of the texture image corresponding to the object model at the interpolation position; Read the texture bump values ​​at each point within the corresponding polygonal region from the texture image after the interpolation operation is completed; Using the transformation matrix, the texture bump values ​​are converted to texture bump values ​​in eye space; The surface tangent is switched to coordinates in eye space and associated with the texture bump values ​​transformed to eye space; Lighting calculations are performed in eye coordinates based on the associated eye-space coordinates and texture bump values. Specifically, the lighting calculations include implementing Glaude shading and Phong shading, where Glaude shading is a color value interpolated at each point of the polygon in the normal map; and Phong shading is a pixel color value estimated based on interpolation for regions outside the polygon interpolation locations in the normal map.

[0037] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0038] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A generative 3D object model normal mapping parametric modeling method, characterized in that, Includes the following steps: Obtain the pixel data of the original image of the object model and the pixel data of the texture image of the object model, wherein the pixel matrix size of the original image is the same as the pixel matrix size of the texture image; Calculate the gradient of each pixel in the original image and the textured image respectively; Based on the gradient of pixels in the original image, determine the normal vector of each pixel in the original image; based on the gradient of pixels in the texture image, determine the normal vector of each pixel in the texture image. The normal map is obtained by fusing the original image and the texture image in the following way: Calculate the average of the normal vector of each pixel in the original image and the normal vector of its corresponding pixel in the texture image, wherein the orientation of the pixel in the original image is the same as the orientation of its corresponding pixel in the texture image. Based on the preset mapping relationship, the average value of the calculated normal vector is converted into a color value; The color values ​​of each pixel in the original image after conversion are stored in the normal map dataset; The data from the normal map dataset is put back into the original image to obtain the normal map.

2. The generative 3D object model normal mapping parametric modeling method according to claim 1, characterized in that, It also includes adjusting the smoothness of the current normal map through the following steps: Create a reference height image, which is related to the amount of change in the color value of adjacent pixels in the current normal map; Calculate the gradient of each pixel in the reference height image to determine the normal vector of each pixel in the reference height image; Based on the normal vectors of each pixel in the original image and the normal vectors of each pixel in the reference height image, the original image and the reference height image are fused to obtain a transition normal map; Using an image addition algorithm, the current normal map and the transition normal map are combined into a new current normal map with improved smoothness.

3. The generative 3D object model normal mapping parametric modeling method according to claim 2, characterized in that, Also includes: Calculate the color value difference between each pixel in the current normal map and its neighboring pixels; The larger the color value difference, the larger the gray value of the pixel in the corresponding orientation in the reference height image.

4. The generative 3D object model normal mapping parametric modeling method according to claim 2, characterized in that, Before adjusting the smoothness of the current normal map, the following steps are also included: Evaluate whether the smoothness of the current normal map meets the preset standard. If not, adjust the smoothness of the current normal map once or multiple times until the smoothness of the new current normal map meets the preset standard.

5. The generative 3D object model normal mapping parametric modeling method according to claim 4, characterized in that, Calculate the color value difference between each pixel in the current normal map and its multiple neighboring pixels, and calculate the average color value difference. Evaluate the smoothness of the current normal map using any of the following methods: Determine the maximum value among the average color value differences of each pixel in the current normal map, and compare it with a preset first threshold. If it is greater than the first threshold, the smoothness of the current normal map is evaluated as not meeting the preset standard. Alternatively, calculate the average value of the difference in average color values ​​corresponding to each pixel in the current normal map, and compare it with a preset second threshold. If the average value is greater than the second threshold, the smoothness of the current normal map is evaluated as not meeting the preset standard.

6. The generative 3D object model normal mapping parametric modeling method according to claim 1, characterized in that, The texture image of the object model is an image representing the surface bump texture of the object model obtained from the previous baking process; Based on the pixel matrix size of the original image, a 2D brush context operation is performed on the texture image so that the pixel matrix size of the texture image is the same as the pixel matrix size of the original image.

7. The generative 3D object model normal mapping parametric modeling method according to claim 6, characterized in that, After obtaining the normal map, the following is also included: Based on the surface tangents and normal vectors of the normal map, a transformation matrix is ​​defined from the tangent space of the normal map to a preset eye space; The normal map is divided into multiple polygons; Perform the following rendering operation on each polygon: Using the transformation matrix, the surface tangents at various points within the polygonal region are converted into coordinates in eye space; Select an interpolation position within the polygon, and interpolate the coordinates of the corresponding polygon region of the texture image corresponding to the object model at the interpolation position; Read the texture bump values ​​at each point within the corresponding polygonal region from the texture image after the interpolation operation is completed; Using the transformation matrix, the texture bump values ​​are converted to texture bump values ​​in eye space; The surface tangent is switched to coordinates in eye space and associated with the texture bump values ​​transformed to eye space; Illumination calculations are performed in eye coordinates based on the associated coordinates and texture bump values ​​in the eye space.

8. The generative 3D object model normal mapping parametric modeling method according to claim 7, characterized in that, Lighting calculations include implementing Glaude shading and Phong shading, where Glaude shading is a color value interpolated for each point of the polygon in the normal map; Phong shading is based on interpolation to estimate pixel color values ​​for regions outside the polygon interpolation locations of the normal map.

9. The generative 3D object model normal mapping parametric modeling method according to any one of claims 1 to 8, characterized in that, The data from the normal map dataset is then put back into the original image in the following manner: The converted color value is placed back into a pixel of the original image, the pixel being in the same orientation as the pixel in the original image used to calculate the average value of the corresponding normal vector. The process continues until the converted color values ​​of each pixel in the original image have been replaced, resulting in a normal map.

10. The generative 3D object model normal mapping parametric modeling method according to claim 9, characterized in that, The object model is an object in a real image or a virtual object created using image software.

Images