An Efficient Color Palette Method for Large-Scale 3D Models

Abstract

An efficient color adjustment method for large-scale 3D models, comprising: Step 1, implementing a visualization system for large-scale OSGB format models by using tools such as the OSG graphics engine; Step 2, implementing a color adjustment function on the visualization system, modifying color adjustment parameters through human-computer interaction, and realizing real-time preview of the color adjustment of the OSGB model; Step 3, exporting the color-adjusted OSGB model in the original format. The present invention enables human-computer interaction on the model scene, can perform overall or partial color adjustment on the model, and can preview the color adjustment result in real time, realizing the separation of preview and storage, and greatly improving the efficiency of model color adjustment.

Description

Technical Field

[0001] The present invention relates to a color adjustment method for large-scale three-dimensional models, which can realize real-time preview of texture color adjustment of OSGB format three-dimensional models during interactive operations and improve the efficiency of setting color adjustment parameters. Background Art

[0002] Oblique photography modeling is an emerging three-dimensional real-scene modeling technology. The common format of the generated model is OSGB. This technology uses unmanned aerial vehicles to collect image information from different perspectives and then uses professional software to perform efficient and automated real-scene three-dimensional model modeling.

[0003] However, this large-scale modeling method often causes varying degrees of deviation in the texture tones of the models due to problems such as shooting time and weather factors, such as the overall or local areas of the model being dull and the lighting being inconsistent. If the aerial images are pre-processed for color adjustment and then modeled, it will take several hours or days of modeling time. Moreover, when adjusting the color of the images, it is not possible to preview whether the color tone of the three-dimensional model is acceptable, and repeated modeling may be required. Another method is to manually adjust the color of the texture images of the three-dimensional model. However, due to the huge amount of data of the three-dimensional model, the processing is extremely time-consuming, and the color adjustment effect cannot be previewed. Similarly, repeated color adjustment may be required. Therefore, there is an urgent need for a method that can directly adjust the color of the built three-dimensional model, can realize real-time preview of the adjustment results according to the interaction, allows users to repeatedly try different color adjustment parameters, so as to quickly obtain appropriate color adjustment parameters and improve the efficiency of model color adjustment. Summary of the Invention

[0004] In order to solve the problems of deviation in the texture tones of the model after oblique photography modeling and low modification efficiency caused by the large amount of model data, the present invention provides an interactive color adjustment method for three-dimensional models in OSGB format. The present invention can perform human-computer interaction on the model scene, can perform overall or local color adjustment on the model, and can real-time preview the color adjustment results, realizing the separation of preview and storage, and greatly improving the efficiency of model color adjustment.

[0005] In order to achieve the above object, the technical solution steps adopted by the present invention are as follows:

[0006] Step 1: Use the OSG graphics engine to implement a visualization system for large-scale OSGB models;

[0007] Step 2: Implement a color adjustment function on the model visualization system, modify the color adjustment parameters through human-computer interaction, and apply a color adjustment algorithm to realize real-time preview of the color adjustment of the OSGB model;

[0008] Step 3: Export the color-adjusted OSGB model in the original format.

[0009] Preferably, the color adjustment function described in step 2 adopts a modification mode in which preview and storage are separated. The preview function uses the large-scale parallel mechanism of the GPU to quickly render the color adjustment result according to the color adjustment parameters, so that the user can get timely feedback during the interaction, and the result is stored after the user is satisfied with the preview result. The steps of implementing the color adjustment algorithm in the GPU are: S1) binding the texture image of the OSGB model to the texture unit No. 0, and passing the texture sampler No. 0 to the fragment shader; S2) obtaining the screen pixel color data, and sampling the passed-in texture data using the texture coordinates of the current vertex in the shader program; S3) determining the color adjustment area through the parameters of the user interaction, and calculating all the pixels to be adjusted through the mapping relationship between the two-dimensional pixels and the three-dimensional model; S4) applying the color adjustment algorithm to all the pixels to be adjusted for color adjustment;

[0010] The color adjustment algorithm described in step 2 is a lighting inconsistency adjustment algorithm, including:

[0011] T1) Obtain a color buffer image and a depth buffer image from a bird's-eye view of the scene.

[0012] T2) Use the depth buffer image to remove the background of the color buffer, and then calculate the lighting component of the current rendered scene. According to the Retinex theory, it can be inferred that the lighting component of the image mainly exists in the low-frequency part of the image, changes slowly, and is insensitive to convolution processing. Therefore, in order to calculate the lighting component, the color space must first be converted to the HSV color space, and then the Gaussian function is used to convolve the brightness value of the color buffer image (the V channel in the HSV color space) to extract the lighting component of the image. The Gaussian function used is in the form of:

[0013]

[0014] Where G(x,y) is the value at the position of the Gaussian kernel (x,y); x represents the horizontal position of the pixel in the image, and y represents the vertical position of the pixel in the image; c is the scale factor, and the larger c is, the better the global characteristics of the extracted illumination value;

[0015] T3) Two-dimensional gamma correction is performed on the pixel value to be adjusted. The correction method and the gamma function used are:

[0016]

[0017] Where O(x,y) is the calculated brightness value of the pixel (x,y), and its value range is [0,1]; F(x,y) is the brightness component of the pixel (x,y) of the input image, and its value range is [0,1]; I(x,y) is the illumination component; m is the mean illumination component, and m is set to the mean illumination component of the model color cache, and is exposed as a parameter for the user to adjust;

[0018] The real-time preview of color adjustment of the OSGB model described in step 2 includes two color adjustment modes: overall color adjustment and local color adjustment. Overall color adjustment means modifying the color of the entire model through the overall color distribution of the model, while local color adjustment is to adjust the color according to the range drawn by the user during interaction, specifically including:

[0019] U1) Interact with the model scene, and obtain the world coordinate points {p1, p2, … p n} of the area to be color-adjusted by clicking the mouse;

[0020] U2) Pass the coordinate points {p1, p2, … p n} into the fragment shader;

[0021] U3) Calculate the world coordinate w_p corresponding to the pixel on the screen in the fragment shader;

[0022] U4) Calculate the shortest distance d from the coordinate w_p to {p1, p2, … p n};

[0023] U5) Smooth the color adjustment parameters according to d to play a transitional role and make the color adjustment result more realistic. The smoothing formula applied in the present invention is:

[0024] S = (1 - SmoothStep(R × a, R × b, d)) × t (3)

[0025] Where S is the smoothed color adjustment parameter value; t is the initial color adjustment parameter value, and its value range is [0, 1]; SmoothStep is a common smoothing function in OpenGL, and returns a value in the range of [0, 1]; R is the color adjustment range radius, which refers to the radius of a circle with a certain coordinate point to be color-adjusted as the center point in the model; d is the shortest distance calculated in U4); a and b are smoothing parameters and can be interactively adjusted, usually a < b.

[0026] Furthermore, the scale factor c described in step 2 takes three different components, with values of 15, 80, and 250 respectively, to ensure the balance of the extracted values. The illumination component I can be obtained by performing three convolution operations with these three components and taking the average value.

[0027] Preferably, exporting the color - adjusted OSGB model in the original format in step 3 specifically includes: In order to generate a new model file, the internal structure of the original OSGB file is parsed to modify the corresponding texture data. The characteristics of each node type in the OSGB file are as follows: The Node node is usually used as the root node to index high - level nodes, with only one geometry body and at most one texture data inside; The Group node is used as a group node to store multiple geometry bodies, with multiple geometry bodies and at least one texture data inside; The Geode node is used as a leaf node to store the final rendering information of the model, usually with only one geometry body and one texture data inside. During the file generation process, the rest of the model data remains unchanged. The texture data is traversed according to the file node type, and the color - adjustment parameters interacted by the user and the corresponding color - adjustment algorithm are applied to each pixel of the original texture. Finally, the new image is compressed into a new OSGB file through the file stream method, so that the new file has the same organizational structure and data volume as the original file.

[0028] Furthermore, for the color - adjustment algorithm running in the shader, both the pixel color and the color - adjustment parameter values are in the range of [0, 1], while the texture image pixel values range from [0, 255]. The color - adjustment parameters are determined according to the color - adjustment attributes, with the brightness value range of [0, 100] and the hue range of [0, 360]. Normalization operations are performed on these numerical values with different ranges to avoid drastic color changes caused by range differences.

[0029] The working principle of the present invention is: Utilizing the large - scale parallel mechanism of the GPU, the color - adjustment algorithm of the model is implemented on the fragment shader in the programmable graphics rendering pipeline to achieve the preview effect. For example, when adjusting the lighting inconsistency of the model, the foreground color buffer is used to calculate the mean value of the overall lighting component, and this mean value is passed into the GPU to perform lighting correction on each pixel of the screen to quickly render the color - adjustment result. After the preview is completed, the model texture files are processed sequentially by the CPU and the model is exported again.

[0030] The advantages of the present invention are: For a relatively large - scale model in the OSGB format, the color - adjustment result after user interaction can be quickly rendered by the GPU, giving real - time feedback to the user, which can effectively improve the color - adjustment efficiency of models with a large amount of data; At the same time, when the model is exported, it follows the format type of the original OSGB model, and does not change the original internal organizational format and data volume of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is the overall flowchart of the OSGB model color - adjustment of the present invention

[0032] Figure 2 is the flowchart of the lighting - inconsistency adjustment of the model color - adjustment algorithm of the present invention

[0033] Figure 3It is the overall / local color adjustment flowchart of the present invention Specific embodiments

[0034] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be described in detail below in conjunction with specific implementation embodiments and with reference to the accompanying drawings.

[0035] Figure 1 The following shows the overall technical flowchart of the present invention. The implementation steps of the present invention are specifically as follows:

[0036] Step 1: Implement a visualization system for large-scale OSGB models using the OSG graphics engine.

[0037] Step 2: Implement a color adjustment function on the model visualization system. Modify the color adjustment parameters through human-computer interaction, and apply a color adjustment algorithm to achieve real-time preview of the color adjustment of the OSGB model. In order to improve the efficiency of model color adjustment, the present invention adopts a modification mode that separates preview and storage for its color adjustment function. Its preview function utilizes the large-scale parallel mechanism provided by the GPU to implement the color adjustment algorithm in the fragment shader of the programmable graphics rendering pipeline, and exposes the parameters to the user. After the user adjusts, the parameters are then passed from the CPU to the GPU to achieve dynamic update of the model rendering result, achieving the effect of real-time preview of color adjustment. The steps to implement the color adjustment algorithm in the GPU are as follows: 1) Bind the texture image of the OSGB model to texture unit 0, and pass the texture sampler of texture unit 0 into the fragment shader; 2) Obtain the screen pixel color data, and sample the passed texture data using the texture coordinates of the current vertex in the shader program; 3) Determine the color adjustment area through the parameters of user interaction, and calculate all the pixels to be color-adjusted through the mapping relationship from two-dimensional pixel points to the three-dimensional model; 4) Apply the color adjustment algorithm to all the pixels to be color-adjusted for color adjustment.

[0038] Step 3, export the OSGB model after color adjustment in the original format. This step is to apply the color adjustment parameters in the preview to the actual texture file of the model, so as to generate a new model file. In order to generate a new model file, the present invention modifies the corresponding texture data by parsing the internal structure of the original OSGB file. The characteristics of each node type of the OSGB file are as follows: the Node node is usually used as a root node to index high-level nodes, and there is only one geometry and at most one texture data inside; the Group node is used as a group node to store multiple geometries, and there can be multiple geometries and at least one texture data inside; the Geode node is used as a leaf node to store the final drawing information of the model, and there is usually only one geometry and one texture data inside. In the process of generating files, the present invention keeps the rest of the model data unchanged, traverses the texture data according to the file node type, and applies the user-interactive color adjustment parameters and the corresponding color adjustment algorithm to each pixel of the original texture. It should be noted that the color adjustment algorithm running in the shader has pixel colors and color adjustment parameter values ​​in the interval [0,1], while the range of texture image pixel values ​​is usually [0,255]. The color adjustment parameters are determined according to the color adjustment properties, such as the brightness value range is [0,100], and the hue range is [0,360]. The present invention performs normalization operations on these values ​​in different ranges to avoid drastic color changes caused by range differences. Finally, the new image is compressed into a new OSGB file through a file stream, so that the new file can have the same organizational structure and data size as the original file.

[0039] The color adjustment algorithm in step 2 is described in detail as an illumination inconsistency adjustment algorithm. The illumination inconsistency adjustment is to solve the problem of uneven illumination of the entire model. The flowchart is as follows: Figure 2 The main steps are:

[0040] 1) Obtain the color buffer image and depth buffer image from the top-down perspective of the scene.

[0041] 2) Use the depth buffer image to remove the background of the color buffer, and then calculate the illumination component of the current rendered scene. According to the Retinex theory, it can be inferred that the illumination component of the image mainly exists in the low-frequency part of the image, changes slowly, and is insensitive to convolution processing. Therefore, in order to calculate the illumination component, the color space must first be converted to the HSV color space, and then the brightness value of the color buffer image (V channel in the HSV color space) is convolved with the Gaussian function to extract the illumination component of the image. The Gaussian function used is in the form of:

[0042]

[0043] Where G(x, y) is the value of the Gaussian kernel at the position (x, y); x represents the horizontal position of the pixel in the image, and y represents the vertical position of the pixel in the image; c is the scale factor, and the larger c is, the better the global characteristics of the extracted illumination value. To ensure the balance of the extracted values in the present invention, c takes three different components, with their values being 15, 80, and 250 respectively. By performing three convolution operations with these three components and taking the average value, the illumination component I can be obtained.

[0044] 3) Perform two-dimensional gamma correction on the pixel value to be color-adjusted. The correction method and the gamma function used are:

[0045]

[0046] Where O(x, y) is the brightness value of the pixel (x, y) after calculation, and its value range is [0, 1]; F(x, y) is the brightness component of the pixel (x, y) in the input image, and its value range is [0, 1]; I(x, y) is the illumination component; m is the average value of the illumination component. In the present invention, m is set to the average value of the illumination component in the model color cache and is exposed as a parameter for the user to adjust.

[0047] The "real-time preview of color adjustment in the OSGB model" in step 2 includes two color adjustment modes: overall color adjustment and local color adjustment. Overall color adjustment means modifying the color of the entire model through the overall color distribution of the model, and local color adjustment is to adjust the color according to the range drawn by the user during the interaction. The color adjustment flowcharts of these two modes are as Figure 3 shown, and the specific steps are:

[0048] 1) Determine the color adjustment mode through the interaction. If it is the overall color adjustment of the model, directly apply the color adjustment algorithm to each pixel in the fragment shader.

[0049] 2) If it is local color adjustment, the specific scheme steps are:

[0050] [1] Interact with the model scene and obtain the world coordinate points {p1, p2,... p n} of the area to be color-adjusted by clicking the mouse.

[0051] [2] Pass the coordinate points {p1, p2,... p n} into the fragment shader.

[0052] [3] Calculate the world coordinate w_p corresponding to the pixel on the screen in the fragment shader.

[0053] [4] Calculate the shortest distance d from the coordinate w_p to {p1, p2,... p n}.

[0054] [5]Smoothing the color adjustment parameters according to d to play a transitional role and make the color adjustment result more realistic. The smoothing formula applied in the present invention is:

[0055] S = (1 - SmootStep(R × a, R × b, d)) × t (3)

[0056] Where S is the value of the color adjustment parameter after smoothing; t is the initial color adjustment parameter value, and its value range is [0, 1]; SmoothStep is a common smoothing function in OpenGL, which returns a value in the range of [0, 1]; R is the radius of the color adjustment range, which refers to the radius of a circle with a certain coordinate point to be color-adjusted in the model; d takes the shortest distance calculated in [4], and a and b are smoothing parameters, which can be interactively adjusted, and usually a < b.

[0057] Currently, there are few good color adjustment tools for large-scale 3D models. The scene models built with a large amount of manpower, material resources and time often have some defects due to factors such as unattractive color tones. The present invention designs an interactive and real-time previewable color adjustment method for 3D models in OSGB format, realizes the texture color adjustment function for large-scale scene models, and can achieve overall and local area color adjustment on the complex model structure of OSGB with cross-blocks and multiple layers. Its real-time preview function can greatly improve the efficiency of model color adjustment.

[0058] The content described in the embodiments of this specification is only a list of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms stated in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art according to the inventive concept.

Claims

1. An efficient color grading method for a large-scale three-dimensional model, comprising the following steps: Step 1: Use OSG graphics engine to realize the visualization system of large-scale OSGB model; Step 2: Implement the color adjustment function on the model visualization system, modify the color adjustment parameters through human-computer interaction, and apply the color adjustment algorithm to achieve real-time preview of the color adjustment of the OSGB model; Step 3: Export the adjusted OSGB model in the original format; The color adjustment function described in step 2 adopts a modification mode in which preview and storage are separated. The preview function uses the large-scale parallel mechanism of the GPU to quickly render the color adjustment result according to the color adjustment parameters, so that the user can get timely feedback during the interaction, and the result is stored after the user is satisfied with the preview result. The steps of implementing the color adjustment algorithm in the GPU are: S1) Bind the texture image of the OSGB model to the texture unit No. 0, and pass the texture sampler No. 0 to the fragment shader; S2) Obtain screen pixel color data, and use the texture coordinates of the current vertex in the shader program to sample the passed-in texture data; S3) Determine the color adjustment area through the parameters of the user interaction, and calculate all the pixels to be adjusted through the mapping relationship between the two-dimensional pixel points and the three-dimensional model; S4) applying a color adjustment algorithm to all pixels to be adjusted for color adjustment; The color adjustment algorithm described in step 2 is a lighting inconsistency adjustment algorithm, including: T1) obtaining a color buffer image and a depth buffer image from a bird's-eye view of the scene; T2) Use the depth buffer image to remove the background of the color buffer, and then calculate the lighting component of the current rendered scene; first convert the color space to the HSV color space, and then use the Gaussian function to convolve the brightness value of the color buffer image, that is, the V channel in the HSV color space, so as to extract the lighting component of the image; the Gaussian function used is in the form of: Where G(x,y) is the value at the pixel (x,y) position; x represents the horizontal position of the pixel in the image, and y represents the vertical position of the pixel in the image; c is the scale factor, and the larger c is, the better the global characteristics of the extracted illumination value; T3) Two-dimensional gamma correction is performed on the pixel value to be adjusted. The correction method and the gamma function used are: Where O(x,y) is the calculated brightness value of the pixel (x,y), and its value range is [0,1]; F(x,y) is the brightness component of the pixel (x,y) of the input image, and its value range is [0,1]; I(x,y) is the illumination component; m is the mean illumination component, and m is set to the mean illumination component of the model color cache, and is exposed as a parameter for the user to adjust; The real-time preview of the color adjustment of the large-scale 3D model described in step 2 includes two color adjustment modes: overall color adjustment and local color adjustment. Overall color adjustment is to modify the color of the entire model by the overall color distribution of the model, and local color adjustment is to adjust the color according to the range drawn by the user during the interaction, specifically including: U1) Interact with the model scene to obtain the world coordinate points {p1, p2, … p of the area to be color - adjusted by clicking the mouse n}; U2) Pass the coordinate points {p1, p2, … p n} into the fragment shader; U3) Calculate the world coordinate w_p corresponding to the pixel on the screen in the fragment shader; U4) Calculate the shortest distance d from the coordinate w_p to {p1, p2, … p n}; U5) Smoothing the color adjustment parameters according to d to achieve a transition effect and make the color adjustment result more realistic. The smoothing formula is: S = (1 - SmoothStep(R × a, R × b, d)) × t (3) Where S is the value of the color - adjustment parameter after smoothing; t is the initial color - adjustment parameter value, and its value range is [0, 1]; SmoothStep is a common smoothing function in OpenGL, which returns a value in the range of [0, 1]; R is the radius of the color - adjustment range, which refers to the radius of the circle centered at a certain coordinate point to be color - adjusted in the model; d is the shortest distance calculated in (U4); a and b are smoothing parameters, which can be interactively adjusted, and a < b.

2. The efficient color adjustment method for a large-scale 3D model according to claim 1, characterized in that The scale factor c described in step 2 is to ensure the balance of the extracted values. Three different components are adopted, and their values are 15, 80, and 250 respectively. By performing three - time convolution operations with these three components and taking the average value, the illumination component I can be obtained.

3. An efficient color adjustment method for a large-scale three-dimensional model according to claim 1, characterized in that Exporting the color - adjusted OSGB model in the original format described in step 3 specifically includes: To generate a new model file, the internal structure of the original OSGB file is parsed to modify the corresponding texture data; The characteristics of each node type in the OSGB file are as follows: The Node node is used as the root node to index high - level nodes, and there is only one geometry body and at most one texture data inside; The Group node is used as a group node to store multiple geometry bodies, and there can be multiple geometry bodies and at least one texture data inside; The Geode node is used as a leaf node to store the final rendering information of the model, and there is only one geometry body and one texture data inside; During the file - generation process, the remaining data of the model remains unchanged. According to the file node type, the texture data is traversed, and the color - adjustment parameter interacted by the user and the corresponding color - adjustment algorithm are applied to each pixel of the original texture; Finally, the new image is compressed into the new OSGB file through the file - stream method, and the new file can have the same organizational structure and data volume as the original file.

4. The efficient color adjustment method for a large-scale three-dimensional model according to claim 3, characterized in that The color - adjustment algorithm running in the shader, the pixel color and the color - adjustment parameter values are both in the range of [0, 1], while the value range of the texture image pixels is [0, 255]. The color - adjustment parameter is determined according to the color - adjustment attribute, the brightness value range is [0, 100], and the hue range is [0, 360]; Normalization operations are performed on these values in different ranges to avoid drastic color changes caused by range differences.

Images