A three-dimensional model texture enhancement method, device, electronic device and medium

Abstract

The present invention discloses a 3D model texture enhancement method, device, electronic device and medium, the method comprising: segmenting the 3D model to be enhanced by ground object category to obtain vegetation covered areas and non-vegetation covered areas; performing specific channel color enhancement on the vegetation covered areas, and performing color optimization on the non-vegetation covered areas to obtain a texture enhanced 3D model. The present invention improves the accuracy and efficiency of improving the visual effect of the 3D model, enhances the spatial perception ability of the 3D model, and reduces the workload of model color enhancement that relies on manual processing.

Description

Technical Field

[0001] The present invention relates to the technical field of real-scene three-dimensional model processing and optimization, and in particular to a three-dimensional model texture enhancement method, device, electronic equipment and medium. Background Art

[0002] With the rapid development of computer graphics, computer vision, and machine learning, the technology of generating real-world 3D models has been greatly improved. With modern data acquisition technologies such as laser scanning, photogrammetry, and drone aerial photography, a large amount of real-world 2D and 3D data can be quickly acquired. With the emergence of automated and semi-automated model generation tools, the geometry and texture of objects can be reconstructed from real-world data without much human intervention to generate high-quality real-world 3D models. However, due to environmental influences such as lighting conditions and material properties, as well as improper settings of photographic electronic equipment, inaccurate exposure, or limited performance of acquisition electronic equipment, low contrast and gray effects of texture colors may occur.

[0003] Texture and color enhancement of real-life 3D models plays an important role in the field of real-life 3D construction. It can improve the visual effect of the model, enhance spatial perception, and also help increase the interactivity and immersion of users during use. Image segmentation refers to the process of dividing an image into regions with unique attributes or features. Common image segmentation methods include threshold segmentation, edge detection, region growing, and segmentation based on graph methods. These methods are based on different principles and criteria, and divide images into different regions by measuring and analyzing the similarity between pixels or regions. Image color enhancement methods can generally be divided into spatial domain methods, frequency domain methods, and mixed domain methods. Spatial domain methods are methods that operate and process at the pixel level of the original image, including histogram equalization, linear filtering, etc.; frequency domain methods generally perform image processing by performing Fourier transform or other frequency domain transforms on the image; mixed domain methods combine spatial domain and frequency domain methods, and use their respective advantages to perform image enhancement, including bidirectional filtering, bidirectional random walk, etc. Frequency domain and mixed domain methods usually require the use of local or global statistical information of the image for enhancement processing. Since texture images are composed of scattered and irregularly shaped texture maps spliced ​​together to minimize storage space, they are usually disordered, that is, they have no continuity in general image content and cannot provide contextual information in the image. Therefore, texture image processing requires direct processing of pixel values. The color and texture of the model usually need to be manually edited to achieve the desired effect, which is not only time-consuming and laborious, but also difficult to guarantee the effect. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned technical deficiencies and provide a three-dimensional model texture enhancement method, device, electronic device and medium to solve the technical problem in the prior art of manually adjusting the color of the three-dimensional model in the model editing software, resulting in low efficiency and poor effect.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a three-dimensional model texture enhancement method, comprising:

[0007] Segment the object categories of the enhanced 3D model to obtain vegetation covered areas and non-vegetation covered areas;

[0008] The color of the vegetation-covered area is enhanced in a specific channel, and the color of the non-vegetation-covered area is optimized to obtain a texture-enhanced three-dimensional model.

[0009] In some embodiments, the segmentation of the object category of the to-be-enhanced three-dimensional model to obtain the vegetation-covered area and the non-vegetation-covered area includes:

[0010] Based on the overgreen index, determining an overgreen grayscale image of the three-dimensional model to be enhanced;

[0011] The overgreen grayscale image is segmented by grayscale using a preset maximum inter-class method difference to obtain vegetation covered areas and non-vegetation covered areas.

[0012] In some embodiments, the grayscale segmentation of the overgreen grayscale image using a preset maximum inter-class method difference to obtain vegetation covered areas and non-vegetation covered areas includes:

[0013] Traverse all preset thresholds and determine that the preset threshold corresponding to the minimum weighted intra-class variance is the optimal threshold;

[0014] The overgreen grayscale image is binarized and segmented using the optimal threshold value to determine that the area corresponding to the pixels in the overgreen grayscale image whose grayscale level is greater than the optimal threshold value is the vegetation covered area, and the area corresponding to the pixels in the overgreen grayscale image whose grayscale level is less than the optimal threshold value is the non-vegetation covered area.

[0015] In some embodiments, the step of performing color enhancement of a specific channel on the vegetation covered area includes:

[0016] The color of the vegetation coverage area is enhanced by using a preset gamma transform to obtain an enhanced vegetation area map.

[0017] In some embodiments, the step of using a preset gamma transform to perform color enhancement on the vegetation coverage area to obtain an enhanced vegetation area map includes:

[0018] The grayscale image of the vegetation coverage area is transformed based on a gamma transformation function to change the brightness distribution of the green channel of the vegetation coverage area of ​​the image, thereby obtaining an enhanced vegetation area map.

[0019] In some embodiments, the color optimization of the non-vegetation covered area includes:

[0020] The color of the non-vegetation covered area is optimized by using a preset improved gamma transform to obtain an optimized non-vegetation area map.

[0021] In some embodiments, the improved gamma transform may be expressed by the following formula:

[0022]

[0023] Among them, O is the gamma value, γ is the gamma parameter, and I is the output parameter grayscale value.

[0024] In a second aspect, the present invention further provides a three-dimensional model texture enhancement device, comprising:

[0025] A segmentation module is used to segment the object categories of the enhanced 3D model to obtain vegetation covered areas and non-vegetation covered areas;

[0026] The texture enhancement module is used to perform color enhancement of a specific channel on the vegetation-covered area and to perform color optimization on the non-vegetation-covered area to obtain a texture-enhanced three-dimensional model.

[0027] In a third aspect, the present invention further provides an electronic device, comprising: a processor and a memory;

[0028] The memory stores a computer-readable program executable by the processor;

[0029] When the processor executes the computer-readable program, the steps in the three-dimensional model texture enhancement method as described above are implemented.

[0030] In a fourth aspect, the present invention also provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the three-dimensional model texture enhancement method as described above.

[0031] Compared with the prior art, the present invention provides a three-dimensional model texture enhancement method, device, electronic device and medium, which first segment the three-dimensional model to be enhanced by ground object categories to obtain vegetation covered areas and non-vegetation covered areas, fully considering the large color difference between the background and the real scene in the ground object scene, and obtain a texture enhanced three-dimensional model by segmenting the image, performing specific channel color enhancement on the vegetation covered area, and performing color optimization on the non-vegetation covered area, thereby further improving the accuracy and efficiency of improving the visual effect of the three-dimensional model, enhancing the spatial perception ability of the three-dimensional model, and reducing the workload of model color enhancement relying on manual processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a flow chart of an embodiment of a three-dimensional model texture enhancement method provided by the present invention;

[0033] Figure 2 is a schematic diagram of an embodiment of step S101 in the three-dimensional model texture enhancement method provided by the present invention;

[0034] Figure 3 It is a schematic diagram of an embodiment of a result of extracting vegetation information from a texture image in the three-dimensional model texture enhancement method provided by the present invention;

[0035] Figure 4 It is a schematic diagram of an embodiment of gamma transformation of grayscale values ​​under different parameters in the three-dimensional model texture enhancement method provided by the present invention;

[0036] Figure 5 is a schematic diagram of an embodiment of a three-dimensional model texture enhancement device provided by the present invention;

[0037] Figure 6 It is a schematic diagram of the operating environment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0039] Image segmentation refers to the process of dividing an image into regions with unique properties or features. Common image segmentation methods include threshold segmentation, edge detection, region growing, and segmentation based on graph methods. These methods are based on different principles and criteria. They divide images into different regions by measuring and analyzing the similarity between pixels or regions. Image color enhancement methods can generally be divided into spatial domain methods, frequency domain methods, and mixed domain methods. Spatial domain methods are methods that operate and process at the pixel level of the original image, including histogram equalization, linear filtering, etc.; frequency domain methods generally perform image processing by performing Fourier transform or other frequency domain transforms on the image; mixed domain methods combine spatial domain and frequency domain methods and use their respective advantages to enhance images, including bidirectional filtering, bidirectional random walk, etc. Both frequency domain and mixed domain methods usually require the use of local or global statistical information of the image for enhancement processing. Since texture images are composed of scattered and irregularly shaped texture maps spliced ​​according to minimized storage space, they are usually disordered, that is, there is no continuity in general image content and cannot provide contextual information in the image. Therefore, the processing of texture images requires direct processing of pixel values. This patent focuses on the low contrast and gray effects of real-life 3D model texture images, and proposes a spatial processing method for different types of objects in the texture atlas. There are relatively few studies on color enhancement of texture images directly in the prior art, which has broad research space and application value.

[0040] The present invention provides a method for enhancing the texture of a three-dimensional model. Figure 1 ,include:

[0041] S101, segmenting the three-dimensional model to be enhanced into ground object categories to obtain vegetation covered areas and non-vegetation covered areas;

[0042] S102: Perform color enhancement of a specific channel on the vegetation-covered area and perform color optimization on the non-vegetation-covered area to obtain a texture-enhanced three-dimensional model.

[0043] In this embodiment, the three-dimensional model to be enhanced is first segmented by object categories to obtain vegetation-covered areas and non-vegetation-covered areas, taking into full consideration the large color difference between the background and the real scene in the object scene. By segmenting the image, performing specific channel color enhancement on the vegetation-covered areas, and performing color optimization on the non-vegetation-covered areas, a texture-enhanced three-dimensional model is obtained, which further improves the accuracy and efficiency of improving the visual effect of the three-dimensional model, enhances the spatial perception ability of the three-dimensional model, and reduces the workload of model color enhancement that relies on manual processing.

[0044] It should be noted that the feature category refers to different types of features or landforms in the geographic space, which are used to describe different objects or scenes on the surface. Feature categories are usually used in the fields of geographic information systems (GIS), remote sensing image analysis, and map making. Feature categories can be natural geographical elements, such as water bodies, forests, grasslands, mountains, rivers, etc.; they can also be man-made constructions or artificial facilities, such as buildings, roads, bridges, farmlands, cities, etc. Different feature categories have different characteristics and attributes, and can be classified through a specific classification system. In this embodiment, in order to better repair the problem of low contrast and grayness of the image, the image is divided into green plant coverage areas and non-green plant coverage areas according to the characteristics of the feature image, so as to better enhance the color of the image and improve the efficiency and effect of image color enhancement.

[0045] Furthermore, image segmentation refers to the process of dividing an image into different areas with unique attributes. The embodiment of the present invention performs threshold segmentation on the image based on the overgreen index of the texture space, and divides the vegetation coverage area and the non-vegetation coverage area by the binary method, so as to facilitate color enhancement processing for different categories of objects. In the image color enhancement processing, the gamma transform method processed at the pixel level is a commonly used color enhancement technology, which improves the appearance and visual effect of the image by adjusting the image brightness and contrast. Specifically, the brightness distribution of the image is changed through a nonlinear grayscale transformation function to make the texture details more distinct. The embodiment of the present invention improves the gamma transform on the basis of image segmentation to achieve color enhancement processing of texture images that takes into account the category of objects.

[0046] In a specific embodiment, the object category segmentation is performed on the to-be-enhanced 3D model to obtain vegetation-covered areas and non-vegetation-covered areas. Figure 2 ,include:

[0047] S201, determining an over-green grayscale image of the three-dimensional model to be enhanced based on an over-green index;

[0048] S202 , performing grayscale segmentation on the overgreen grayscale image using a preset maximum inter-class method difference to obtain vegetation covered areas and non-vegetation covered areas.

[0049] In this embodiment, unlike the object category segmentation of hyperspectral or panchromatic images, the visible light camera used in oblique photogrammetry can only obtain images of the three channels of red, green, and blue, and cannot obtain the common indicators of remote sensing image classification such as the traditional normalized vegetation index NDVI. Therefore, indicators are constructed based on the three band channels of visible light, and the model texture image is segmented into vegetation-covered areas and non-vegetation-covered areas. In the oblique photography model, the texture space of the image is composed of color images of the three color channels of red, green, and blue, and the vegetation-covered areas and non-vegetation-covered areas have different characteristics in the values ​​of these three color components. By combining different color components, each pixel in the image can be converted to enhance the contrast between vegetation and non-vegetation-covered areas, so that vegetation and non-vegetation coverage categories can be better identified. In this embodiment, the visible light band index-excess green index (ExG) is used:

[0050] ExG=2g ’ -r ′ -b ′

[0051] In the formula, r ′ , g ′ , b ′ are the normalized values ​​of the grayscale values ​​of the red, green, and blue channels in the RGB color space of the image, that is, r ′ =R / (R+G+B), g ′ =G / (R+G+B), b ′ =B(R+G+B), where R, G, and B are the pixel values ​​of the red, green, and blue bands respectively.

[0052] After obtaining the grayscale image with the green index calculated, the maximum inter-class variance method is used for grayscale segmentation. The maximum inter-class variance algorithm is a global binarization algorithm that automatically determines the threshold based on maximizing the inter-class variance between the target and the background. The algorithm attempts to find a threshold t that minimizes the weighted intra-class variance given by formula (2).

[0053]

[0054] in is the weighted intra-class variance of the grayscale image, q1(t) and are the proportion of foreground target pixels to the total number of pixels and the grayscale variance, q2(t) and are the ratio of background pixels to total pixels and the grayscale variance respectively. The algorithm traverses all possible thresholds (0 to 255) and calculates the corresponding intra-class variance to find the threshold that makes The minimum optimal threshold t is used to binarize the image. Pixels with grayscale levels less than or equal to the threshold are set as background, and pixels greater than the threshold are set as foreground. The segmentation result binary image is as follows: Figure 3shown.

[0055] In some embodiments, the step of performing color enhancement of a specific channel on the vegetation covered area includes:

[0056] The color of the vegetation coverage area is enhanced by using a preset gamma transform to obtain an enhanced vegetation area map.

[0057] The color of vegetation is enhanced by adjusting the pixel grayscale value of the green channel of the vegetation coverage area. Generally, the green color of the vegetation coverage area can be made more prominent by increasing the gain of the green channel, for example, by adjusting the color gain parameter or directly multiplying by a gain coefficient. In order to prevent grayscale value overflow or image distortion during color enhancement, in this embodiment, a gamma transform based on grayscale value is used for enhancement processing. The function of gamma transform is as follows: Figure 4 As shown in the figure, by changing the brightness distribution of the green channel in the vegetation coverage area of ​​the image through a nonlinear grayscale transformation function, the lower gamma value can be adjusted to reduce the overall contrast, increase the brightness of green, and make the vegetation texture more vivid and soft.

[0058] In some embodiments, the color optimization of the non-vegetation covered area includes:

[0059] The color of the non-vegetation covered area is optimized by using a preset improved gamma transform to obtain an optimized non-vegetation area map.

[0060] It should be noted that the color enhancement method of the non-vegetation covered area is to process the pixels of the area in the texture space, including the enhancement of contrast, saturation and brightness.

[0061] Contrast stretching increases the difference between pixels through linear or nonlinear transformation to improve the visual effect of the image. The embodiment of the present invention improves the gamma transform based on the low contrast and gray characteristics of the original texture image. For the input gray value I, the mapping output value O of the nonlinear transformation is calculated by formula (3), which is as follows:

[0062]

[0063] Among them, γ is the gamma parameter and I is the output parameter grayscale value.

[0064] The enhancement of saturation and brightness can make the color of the building area more vivid. Using color space conversion, the image is converted from RGB color space to HSV (hue, saturation, brightness) color space, and then the interval function (Formula 4) is used to increase the pixel value of the saturation channel to achieve saturation enhancement, and the pixel value of the brightness channel is modified by gamma transformation to achieve brightness enhancement. After the enhancement is completed, the color space is transformed back to RGB space.

[0065] It should be noted that formula (4) is as follows:

[0066]

[0067] Based on the above three-dimensional model texture enhancement method, the embodiment of the present invention also provides a three-dimensional model texture enhancement device 500. Figure 5 The three-dimensional model texture enhancement device 500 includes a segmentation module 510 and a texture enhancement module 520 .

[0068] The segmentation module 510 is used to segment the three-dimensional model to be enhanced into ground object categories to obtain vegetation covered areas and non-vegetation covered areas;

[0069] The texture enhancement module 520 is used to perform color enhancement of a specific channel on the vegetation-covered area and to perform color optimization on the non-vegetation-covered area to obtain a texture-enhanced three-dimensional model.

[0070] like Figure 6 As shown, based on the above three-dimensional model texture enhancement method, the present invention also provides an electronic device, which can be a computing device such as a mobile terminal, a desktop computer, a notebook, a palm computer, and a server. The electronic device includes a processor 610, a memory 620, and a display 630. Figure 6 Only some components of the electronic device are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0071] In some embodiments, the memory 620 may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. In other embodiments, the memory 620 may also be an external storage electronic device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 620 may also include both an internal storage unit of the electronic device and an external storage electronic device. The memory 620 is used to store application software and various types of data installed in the electronic device, such as program codes for installing the electronic device. The memory 620 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a three-dimensional model texture enhancement program 640 is stored on the memory 620, and the three-dimensional model texture enhancement program 640 can be executed by the processor 610, thereby realizing the three-dimensional model texture enhancement method of each embodiment of the present application.

[0072] In some embodiments, the processor 610 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 620, such as executing a three-dimensional model texture enhancement method.

[0073] In some embodiments, the display 630 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 630 is used to display information on the three-dimensional model texture enhancement electronic device and to display a visual user interface. The components 610-630 of the electronic device communicate with each other via a system bus.

[0074] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, an optical disk, etc.

[0075] The specific implementation of the present invention described above does not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A three-dimensional model texture enhancement method, characterized in that: include: Performing object category segmentation on the three-dimensional model to be enhanced, and determining an over-green grayscale image of the three-dimensional model to be enhanced based on an over-green index; Traverse all preset thresholds and determine that the preset threshold corresponding to the minimum weighted intra-class variance is the optimal threshold; The overgreen grayscale image is binarized and segmented using the optimal threshold, and the area corresponding to the pixels in the overgreen grayscale image whose grayscale level is greater than the optimal threshold is determined to be the vegetation-covered area, and the area corresponding to the pixels in the overgreen grayscale image whose grayscale level is less than the optimal threshold is determined to be the non-vegetation-covered area; wherein the texture space of the image is composed of color images of three color channels of red, green and blue, and the values ​​of the three color components of red, green and blue between the vegetation-covered area and the non-vegetation-covered area have different characteristics; The color of vegetation is enhanced by adjusting the pixel grayscale value of the green channel of the vegetation covered area, and the contrast, saturation and brightness of the pixels in the non-vegetation covered area in the texture space are enhanced to obtain a texture enhanced three-dimensional model.

2. The three-dimensional model texture enhancement method according to claim 1, characterized in that: The step of enhancing the color of vegetation by adjusting the pixel grayscale value of the green channel of the vegetation covered area includes: The preset gamma transform is used to adjust the pixel grayscale value of the green channel of the vegetation coverage area to enhance the color of the vegetation.

3. The three-dimensional model texture enhancement method according to claim 2, characterized in that: The method of using a preset gamma transform to adjust the pixel grayscale value of the green channel of the vegetation coverage area to enhance the color of the vegetation includes: The grayscale image of the vegetation coverage area is transformed based on a gamma transformation function to change the brightness distribution of the green channel of the vegetation coverage area of ​​the image, thereby obtaining an enhanced vegetation area map.

4. The three-dimensional model texture enhancement method according to claim 1, characterized in that: The processing of enhancing the contrast, saturation and brightness of the pixels in the non-vegetation covered area includes: The preset gamma transform is used to enhance the contrast, saturation and brightness of the pixels in the non-vegetation covered area.

5. The three-dimensional model texture enhancement method according to claim 4, characterized in that: The default gamma transform is expressed by the following formula: in, is the gamma parameter, and I is the output parameter grayscale value.

6. A 3D model texture enhancement device, characterized in that: include: A segmentation module is used to segment the three-dimensional model to be enhanced into ground object categories, determine the over-green grayscale image of the three-dimensional model to be enhanced based on the over-green index; traverse all preset thresholds, and determine that the preset threshold corresponding to the minimum weighted intra-class variance is the optimal threshold; The overgreen grayscale image is binarized and segmented using the optimal threshold, and the area corresponding to the pixels in the overgreen grayscale image whose grayscale level is greater than the optimal threshold is determined to be the vegetation-covered area, and the area corresponding to the pixels in the overgreen grayscale image whose grayscale level is less than the optimal threshold is determined to be the non-vegetation-covered area; wherein the texture space of the image is composed of color images of three color channels of red, green and blue, and the values ​​of the three color components of red, green and blue between the vegetation-covered area and the non-vegetation-covered area have different characteristics; The texture enhancement module is used to enhance the color of vegetation in the vegetation-covered area by adjusting the pixel grayscale value of the green channel of the vegetation-covered area, and to enhance the contrast, saturation and brightness of the pixels in the non-vegetation-covered area in the texture space to obtain a texture-enhanced three-dimensional model.

7. An electronic device, characterized in that: include: Processor and memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the three-dimensional model texture enhancement method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the three-dimensional model texture enhancement method as described in any one of claims 1-5.

Images