Intelligent building texture repairing and beautifying method fusing deep learning model

Abstract

The invention is suitable for the technical field of building model data processing, and provides an intelligent building texture repairing and beautifying method fusing a deep learning model, and the method comprises the steps: firstly, carrying out the recombination of the texture of a three-dimensional building model, and generating a to-be-processed wall surface and roof texture picture; a to-be-repaired area, needing to be repaired, of the building texture is extracted by combining low-rank matrix decomposition and an SAM image segmentation algorithm, and then whether an LAMA image repair model or a PatchMatch image repair algorithm is used for texture processing is determined according to the proportion of the to-be-repaired area in a whole texture image so as to achieve the maximum repair effect. And finally, carrying out texture distortion repair and material beautification by using an image diffusion model, a ControlNet control chart and cue words, and outputting beautified wall surface and roof textures. The problems of texture shielding, garland, noise, window distortion and the like can be solved at the same time, automatic detection and automatic texture repairing and beautifying of the texture shielding garland area are achieved, and the time consumption of texture editing processing in the building monotonized modeling process is remarkably shortened.

Description

Technical Field

[0001] The present invention belongs to the technical field of building model data processing, and in particular relates to a method for intelligently repairing and beautifying building textures by integrating a deep learning model. Background Art

[0002] With the rapid development of real-world 3D, digital twin cities, and the low-altitude economy, single-unit structured modeling technology for 3D buildings plays a crucial role in the rapid construction of 3D urban scenes. This technology generally involves two steps: structural reconstruction of building geometry and reconstruction of building texture mapping. Structural reconstruction of building geometry has become increasingly mature, enabling automated construction of standard, regular buildings.

[0003] However, due to the non-structural nature of architectural texture images, even after automatic mapping of architectural textures, occlusion, streaks, noise, and distortion are still common on the model. To address these architectural model texture quality issues, manual editing and repair using image processing software is often required before remapping them onto the architectural model. This solution requires considerable manpower to repair and beautify architectural texture images, impacting the cycle and cost of the entire building modeling process. According to statistics from relevant 3D modeling departments, in the process of individualizing urban building 3D models, the time spent on mapping and manually editing and repairing building textures accounts for over 70% of the total time spent on individual modeling.

[0004] 1km of technical solutions based on manual modeling 2 The time consumption analysis of geometric editing and texture editing for single-unit modeling of urban scenes is shown in Table 1 below:

[0005] Table 1 Statistical analysis of time consumption for manual structural modeling of building units

[0006]

[0007] The statistical analysis above shows that the majority of time spent manually modeling individual urban buildings is spent on editing and repairing building textures. The traditional approach of manually editing and repairing building textures using image processing software is no longer sufficient for the rapid updating of urban 3D scene models and building models.

[0008] Therefore, the realization of automated intelligent repair and beautification technology for distorted textures such as occlusion, noise, streaks, and distortion in the wall and roof textures of building models is of great significance to the reconstruction of urban building three-dimensional models, and can greatly improve the efficiency of urban three-dimensional building model reconstruction and three-dimensional scene construction. Summary of the Invention

[0009] In view of the above problems, the purpose of the present invention is to provide an intelligent restoration and beautification method for architectural textures that integrates deep learning models, aiming to solve the problem that the restoration and beautification of architectural textures at this stage mainly rely on manual processing by editing software. By integrating multiple deep learning models, the restoration and beautification of textures can be achieved, thereby reducing the cycle and cost of building monomer modeling, improving the effect of monomer structured reconstruction of urban building models, increasing the realism of the wall and roof textures of the building model, and reducing the distortion of the building model texture caused by occlusion of the original building texture, quality problems and loss of accuracy in building geometry reconstruction.

[0010] The present invention adopts the following technical solutions:

[0011] The method for intelligent restoration and beautification of architectural textures by integrating a deep learning model comprises the following steps:

[0012] Step S1: Generate texture images of the wall and roof to be processed based on the texture reconstruction of the three-dimensional building monomer model, and use the image low-rank matrix segmentation algorithm and the SAM image segmentation algorithm to extract the area to be repaired where the building texture needs to be repaired;

[0013] Step S2: Automatically repair the area to be repaired using the LAMA image restoration model or the PatchMach image restoration algorithm according to the proportion of the area to be repaired to the entire building texture area;

[0014] Step S3: Use the YOLO deep learning model to identify the billboard and text areas in the restored wall texture image, and crop and output sub-images of the text and billboard areas;

[0015] Step S4: combining the image diffusion model, the ControlNet control diagram, and the image prompt words to guide the establishment of a building texture image beautification model, beautifying the restored texture image, and outputting the beautified wall and roof texture images;

[0016] Step S5: Re-integrate the cropped text and billboard area sub-image to their original positions on the wall.

[0017] Furthermore, the specific process of step S1 is as follows:

[0018] S11, loading the original three-dimensional building model, reading the model's geometric information and texture information, sampling, reorganizing, and merging the model's building texture in horizontal and vertical directions, and obtaining texture images of the building's walls and roof respectively;

[0019] S12, using an image low-rank matrix segmentation algorithm to obtain distorted texture areas in the wall and roof texture images to obtain a first texture restoration area;

[0020] S13, segmenting the wall and roof texture images according to the SAM image segmentation algorithm, extracting the wall occlusion and roof coverage areas using prompt words, and obtaining a second texture restoration area;

[0021] S14: Merge the first and second texture repair areas to the final repair area of the building wall and roof.

[0022] Furthermore, the specific process of step S12 is as follows:

[0023] (1) Convert the input texture image into a single-channel matrix D by grayscale processing, and then normalize it to obtain the normalized matrix Dnormal;

[0024] (2) Decompose the normalized matrix Dnormal into a low-rank matrix A and a sparse matrix E, where the low-rank matrix A represents the main structural information of the building texture and the sparse matrix E represents the distorted texture outliers;

[0025] (3) Solve the model using the alternating direction multiplier algorithm to obtain the low-rank matrix A and the sparse matrix E;

[0026] (4) The obtained sparse matrix E is processed and the positions and values corresponding to the non-zero elements are extracted to obtain the positions of the distorted building texture areas and generate the first texture repair area in the form of a binary image.

[0027] Furthermore, the specific process of step S13 is as follows:

[0028] The SAM image segmentation algorithm is used to segment the wall and roof texture images. Masks are automatically extracted for the repair areas of wall occlusion and roof coverage according to the prompt words. The extracted masks are then binarized and connected domain analysis is performed to remove noise and small isolated areas, retaining larger areas as the second texture repair areas for the wall and roof textures.

[0029] Furthermore, the specific process of step S2 is as follows:

[0030] S21. Calculate the area ratio of the wall surface repair area and the roof texture repair area to the corresponding building texture area. For the area ratio of the repair area that is less than or equal to 10%, automatically repair the area based on the LAMA image repair model.

[0031] S22. For an area to be repaired that accounts for more than 10% and less than 30% of the area, the current area to be repaired is divided into multiple sub-texture repair areas with an area of less than 10% according to a grid, and the sub-texture repair areas are automatically repaired using the LAMA image repair model;

[0032] S23. For areas to be repaired that account for more than or equal to 30% of the area, the PatchMach image repair algorithm is used to supplement and repair the wall and roof textures.

[0033] Among them, the LAMA image restoration model directly repairs texture images with a resolution larger than 256x256 without downsampling the texture image first.

[0034] Furthermore, in step S4, the specific process of establishing the building texture image beautification model is as follows:

[0035] S41. Select the Stable Diffusion diffusion model as the basic model;

[0036] S42. Add ControlNet control diagrams to the Stable Diffusion model

[0037] S43. Add positive and negative prompt words based on the architectural texture type, style, and architectural element layout requirements of the architectural texture image to guide the stable diffusion model when generating textures.

[0038] Furthermore, the specific process of step S42 is as follows: the structural information of the building texture is extracted through the preprocessor of Canny edge detection and Depth depth estimation, and the control tensor is generated and fused with the U-Net feature map of the Stable Diffusion diffusion model.

[0039] The beneficial effects of the present invention are:

[0040] First of all, the present invention is an intelligent repair solution for building textures that integrates deep learning models. The entire process, from the identification and generation of the building texture area to be repaired to the repair of the texture image, is automated. The building wall and roof textures can be intelligently repaired without human intervention, greatly reducing the time and labor cost of texture editing in the process of building individual modeling.

[0041] Secondly, the present invention integrates multiple image processing and deep learning algorithms to identify the areas to be repaired on building walls and texture maps and to repair building texture images. Combined with the streaks and occlusions in the texture images, the present invention uses image low-rank matrix decomposition and SAM (Segment Anything Model) image segmentation algorithms to obtain the repair area, thereby improving the accuracy of extracting the areas to be repaired on building textures.

[0042] Third, through in-depth analysis and comparison of the advantages and disadvantages of the LAMA (Large Mask Inpainting) image restoration model and the PatchMatch image restoration algorithm, corresponding restoration algorithms and algorithm sub-graph input strategies were used according to the area ratio of the two types of texture restoration areas to improve the model texture restoration quality.

[0043] Finally, to address the problems of distortion and noise that are prone to occur in building wall textures, the Stable Diffusion diffusion model plus ControlNet and prompt word guidance are used to beautify and repair the texture image. At the same time, during the beautification and repair process, the problem of blurry and distorted text caused by excessive divergence in the wall text and billboard areas is taken into consideration. The method of separate identification and then pasting back to the original image is adopted. This not only ensures the beautification effect of the building texture, solves the texture distortion and noise problems, but also ensures the authenticity of the text and billboard areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of a method for intelligent restoration and beautification of architectural textures integrating a deep learning model provided by an embodiment of the present invention;

[0045] Figure 2 It is the original texture map of the building wall;

[0046] Figure 3 It is the texture map of the building wall after occlusion and repair;

[0047] Figure 4 It is the texture map of the building wall after beautification and restoration;

[0048] Figure 5 This is the original texture map of the building roof;

[0049] Figure 6 This is the texture map of the building roof after repair;

[0050] Figure 7 This is the texture image of the building roof after beautification and restoration. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0052] During the structural reconstruction of a single building model, the complex three-dimensional spatial relationships between buildings and trees, roof attachments, and debris in urban scenes, as well as the loss of geometric accuracy after structural reconstruction of buildings, lead to distortion problems such as occlusion, blank spaces, streaks, and window distortion in the mapped textures. For example, the wall textures of buildings acquired using oblique photography often have occlusion by trees in low-rise areas, windows on building walls are distorted to a certain extent, and the roof textures of buildings may have snow accumulation in winter. These texture quality issues affect the realism of the building model. As a result, after texture mapping, building textures often have distortion problems, and a large amount of manpower is still required to repair and beautify the occlusion of building wall and roof textures. The method of using image processing software to manually model and repair texture occlusion, streaks, and other problems is costly and time-consuming, which limits the further application of automated modeling technology in the fields of real-life 3D and digital twins.

[0053] To this end, this embodiment provides a method for intelligent restoration and beautification of architectural textures that integrates a deep learning model to solve this problem. In order to illustrate the technical solution described in the present invention, a specific embodiment is provided below for illustration.

[0054] like Figure 1 As shown, the method for intelligent restoration and beautification of architectural textures integrating a deep learning model provided in this embodiment includes the following steps:

[0055] Step S1: Generate texture images of the wall and roof to be processed based on the texture reconstruction of the three-dimensional building monomer model, and use the image low-rank matrix segmentation algorithm and the SAM image segmentation algorithm to extract the area to be repaired where the building texture needs to be repaired.

[0056] This step mainly aims to identify the areas of building texture that need to be repaired. The specific process is as follows:

[0057] S11. Load the original three-dimensional building model, read the model's geometric information and texture information, sample, reorganize, and merge the model's building texture in horizontal and vertical directions, and obtain texture images of the building's walls and roof, respectively.

[0058] By loading the building model and reading relevant information, and then performing the corresponding building texture analysis, we can obtain the texture images of the building walls and roofs. These wall and roof texture images will serve as input data for subsequent intelligent texture restoration and beautification.

[0059] S12. Use an image low-rank matrix segmentation algorithm to obtain distorted texture areas in the wall and roof texture images to obtain a first texture repair area.

[0060] This step uses a low-rank image matrix segmentation algorithm to identify noise, streaks, and blank areas within the building wall and roof texture images, generating the first texture restoration area. For these areas, the image matrix can be decomposed into a low-rank matrix representing the primary structural features of the building texture and a sparse matrix representing outliers such as streaks and noise. This step leverages the global repetitive nature of building textures to isolate these areas through solution optimization.

[0061] The specific process of step S12 is as follows:

[0062] (1) Texture image preprocessing: The input texture image is grayscaled and converted into a single-channel matrix D, and then normalized to obtain the normalized matrix Dnormal.

[0063] Input building texture RGB image I∈R m×n×3 , convert it into a single-channel matrix D by grayscale processing, and normalize it to a normalized matrix Dnormal in the range of [0,1].

[0064] (2) Constructing an optimization model: Decompose the normalized matrix Dnormal into a low-rank matrix A and a sparse matrix E, where the low-rank matrix A represents the main structural information of the building texture and the sparse matrix E represents the distorted texture outliers.

[0065] Assume that the building texture image normalized matrix Dnormal is decomposed into a low-rank matrix A and a sparse matrix E, that is, Dnormal=A+E.

[0066] (3) Solve the optimization problem: Solve the model using the alternating direction multiplier algorithm (ADMM) to obtain the low-rank matrix A and the sparse matrix E.

[0067] The iterative update formula is as follows:

[0068] a. Update the low-rank matrix A, where P u represents the soft threshold processing operation, U (k) is the Lagrange multiplier matrix:

[0069] ;

[0070] b. Update the sparse matrix E, where S λ It is a soft threshold processing operation, the core of which is to shrink the matrix elements to zero, thereby extracting and retaining the non-zero elements of the sparse matrix:

[0071] ;

[0072] c. Update the Lagrange multiplier matrix U:

[0073] ;

[0074] (4) Post-processing: Process the obtained sparse matrix E and extract the positions and values corresponding to the non-zero elements therein to obtain the positions of the distorted building texture areas and generate the first texture repair area in the form of a binary image.

[0075] S13. Segment the wall and roof texture images according to the SAM image segmentation algorithm, extract the wall occlusion and roof coverage areas using prompt words, and obtain a second texture restoration area.

[0076] The wall and roof texture maps are segmented using the SAM (Segment Anything Model) segmentation algorithm. For example, the tree and snow cues are used to extract tree-occluded areas on the wall and snow-covered areas on the roof, resulting in a second texture restoration area. SAM is an advanced image segmentation model designed to quickly and accurately segment any object in an image by providing cues (such as points, boxes, and text).

[0077] This step uses the SAM image segmentation algorithm to segment the wall and roof texture images. Masks are automatically extracted for the repair areas blocked by the wall and covered by the roof based on prompt words (such as trees on the wall and snow on the roof). The extracted masks are then binarized and connected domain analysis is performed to remove noise and small isolated areas, retaining larger areas as the second texture repair area for the wall and roof textures.

[0078] S14: Merge the first and second texture repair areas to the final repair area of the building wall and roof.

[0079] Step S2: According to the proportion of the area to be repaired to the entire building texture area, the LAMA image repair model or the PatchMach image repair algorithm is used to automatically repair the area to be repaired.

[0080] LAMA (Large Mask Inpainting) is a deep learning model for image inpainting. Combining advanced frameworks such as Fast Fourier Convolution (FFC), this model expands the receptive field and can better capture the global information of an image. It also performs well in completing images with periodic structures, effectively handling various occlusion scenarios, such as building walls and roofs. Although the LAMA model demonstrates good generalization when processing images with resolutions higher than the training 256x256, to further ensure inpainting results when processing wall and roof textures with resolutions exceeding 256x256, the LAMA image inpainting model input has been fine-tuned to support direct inpainting of texture images with a resolution of 2048x2048, without first downsampling the texture image. This also avoids the color difference caused by splitting a large image into multiple sub-images and then merging them.

[0081] In this step, the LAMA image restoration model is selected as the deep learning model for texture restoration. The input parameters of the LAMA image restoration model are fine-tuned to support direct input of 2048x2048 images to ensure the restoration effect of high-resolution architectural texture images while avoiding color differences caused by sub-image segmentation.

[0082] The specific process of this step is as follows:

[0083] S21. Calculate the area ratio of the wall surface repair area and the roof texture repair area to the corresponding building texture area respectively. For the area to be repaired with an area ratio less than or equal to 10%, automatically repair the area to be repaired based on the LAMA image repair model.

[0084] The area ratio here refers to the ratio of the wall repair area to the total wall texture area, and the ratio of the roof texture repair area to the total roof texture area. The area ratios in subsequent steps have the same meaning.

[0085] When the area to be repaired accounts for a small proportion of the entire texture area, the LAMA image restoration model can usually repair the missing area well, ensuring the naturalness and coherence of the texture restoration area.

[0086] S22. For the area to be repaired that accounts for more than 10% and less than 30% of the area, the current area to be repaired is divided into multiple sub-texture repair areas with an area of less than 10% according to the grid, and then the sub-repair areas are automatically repaired using the LAMA image repair model.

[0087] Although the LAMA image inpainting model uses fast Fourier convolution to increase the receptive field, when the area to be inpainted accounts for a large proportion of the entire texture area, the limited receptive field makes it difficult for the model to capture the global information of the image, which in turn affects the restoration effect of the building texture. Therefore, when the area of the texture to be inpainted exceeds 10% but less than 30%, this step divides the area to be inpainted into multiple sub-areas with an area of less than 10% and then inpaints them one by one. This avoids the problem of poor texture restoration results of the LAMA image inpainting model caused by a single inpainting area being too large.

[0088] S23. For areas to be repaired that account for more than or equal to 30% of the area, the PatchMach image repair algorithm is used to supplement and repair the wall and roof textures.

[0089] When the area to be repaired accounts for more than 30% of the entire texture area, in order to avoid the problem of poor texture repair effect of the LAMA model due to the large area of the area to be repaired, PatchMatch image repair is used to supplement the wall and roof textures and output the repaired building texture map.

[0090] Compared with the LAMA image restoration model, the PatchMatch image restoration algorithm uses the method of filling missing areas with known area information. Although it is not as good as the LAMA image restoration model in terms of the restoration effect of complex images, it can effectively fill the missing areas when the occluded restoration area is large, making up for the problem of excessive divergence and uncontrollable quality of the restoration area caused by the large area of the LAMA image restoration model.

[0091] In step S2 above, different image restoration algorithms and parameters are selected for restoration based on the proportion of the area to be restored for different building textures. This paper conducts in-depth analysis and experimental comparison of the characteristics and advantages of the LAMA image restoration model and the PatchMatch restoration algorithm in restoring images with different degrees of occlusion. Furthermore, to address the issue of uneven borders in the restored sub-images when splitting large building texture images into multiple sub-images, the input of the LAMA network model is fine-tuned to support direct restoration of texture images with a resolution of 2048x2048, without the need to first downsample or sub-image-split the texture image.

[0092] To address the diverse sizes of architectural textures requiring restoration, a hybrid approach was employed, depending on the proportion of the restoration area to the entire texture map. These included directly applying the LAMA image restoration model to the texture restoration area, splitting the restoration area into sub-images and then performing LAMA restoration, and employing the PatchMatch algorithm for restoration. This hybrid approach not only enables high-quality restoration of higher-resolution architectural texture images, but also effectively adapts to architectural textures with varying degrees of occlusion, avoiding the problem of suboptimal restoration of some architectural textures when the restoration area is too large, resulting from the direct use of a single-scale LAMA image restoration deep learning model.

[0093] Step S3: Use the YOLO deep learning model to identify the billboard and text areas in the restored wall texture image, and crop and output sub-images of the text and billboard areas.

[0094] Text and billboard areas are crucial information in building texture images and are essential for ensuring the authenticity of building models. The YOLO (You Only Look Once) model is an advanced deep learning object detection algorithm that can identify text and billboard objects in building textures. It can quickly, efficiently, and accurately identify text and billboard areas in building wall texture images. For example, the YOLOv8 deep learning model is used to identify text and billboard areas on building walls and crop and output sub-images of these areas. This paves the way for addressing the distortion caused by over-repairing billboard and text areas during beautification of large model images.

[0095] Step S4: Combine the image diffusion model, the ControlNet control diagram, and the image prompt words to guide the establishment of a building texture image beautification model, beautify the repaired texture image, and output the beautified wall and roof texture images.

[0096] In addition to occlusion from trees, snow, and texture smears, architectural model textures often suffer from issues like window distortion and noise in the building textures. The LAMA image restoration algorithm is unable to address these issues. This step, based on the StableDiffusion diffusion model and supplemented with ControlNet (a Stable Diffusion extension plug-in) control charts and prompts, implements a model for beautifying the raw architectural texture images. The restored textures output from the previous steps are subjected to beautification processing, including noise removal and distortion repair, to produce beautified architectural wall and roof textures. The StableDiffusion diffusion model is an open-source diffusion model that supports image-to-image generation. Its core concept is to first add a large amount of Gaussian noise to the original image, resulting in an almost entirely noisy image. Then, a U-Net neural network is used to gradually remove the noise and restore the original image.

[0097] Here, the specific process of establishing the architectural texture map beautification model is as follows:

[0098] S41. Select the Stable Diffusion diffusion model as the basic model.

[0099] The Stable Diffusion XL model was specifically selected as the basic model for image restoration. The Stable Diffusion XL model supports the generation of high-resolution images of 1024x1024. Compared with models such as 1.5, it has significantly improved the details and clarity, and its ability to understand prompt words has also been significantly improved.

[0100] S42. Add ControlNet control diagram to the Stable Diffusion diffusion model.

[0101] To enable the Stable Diffusion model to incorporate the structural features of building roof and wall textures for beautification and restoration, this step supplements the ControlNet control map with more precise control over the generation of building texture maps. Specifically, a preprocessor using Canny edge detection and Depth estimation extracts structural information from building textures. This generates a control tensor that is then fused with the Stable Diffusion model's U-Net feature map. This achieves both beautification and geometric consistency between the generated texture and the original image, ensuring greater fidelity in beautification and restoration of 3D building model textures.

[0102] S43. Add positive and negative prompt words based on the architectural texture type, style, and architectural element layout requirements of the architectural texture image to guide the stable diffusion model when generating textures.

[0103] Positive prompts clarify the semantic information, details, and quality requirements for building texture generation. Negative prompts eliminate undesirable features and content, enhancing the rationality of the generated results. For example, the positive prompt for this step in building image beautification is "Input image is facade or roof of a building. Generate a clear, noiseless, and high-quality image based on input image. Regenerate any blurry or distorted area. Do not create new doors, windows, or other components that do not exist in input image," while the negative prompt is "low quality, blurry, noisy, unsharp, distortion." By adding these prompts, the Stable Diffusion model can be effectively guided to perform building texture distortion and noise beautification repair while maintaining maximum fidelity.

[0104] Step S5: Re-integrate the cropped text and billboard area sub-image to their original positions on the wall.

[0105] To avoid over-repairing the text and billboard areas on the building wall using the Stable Diffusion diffusion model, this step replaces and updates the beautified building wall texture image with the text and billboard sub-images cropped in step S3. A Poisson blending image processing algorithm is then used to fuse the color differences in the adjacent regions, making the text and billboard images appear more natural when integrated into the beautified texture. The resulting beautified texture image, which retains the text and billboard areas, is then output, completing the fully automated restoration and beautification of the original building model texture. This technical process improves the quality of the building model texture and significantly reduces the time and cost of manual texture restoration.

[0106] The effect of using the method of the present invention to repair the texture of building walls and roofs is as follows: Figure 2-Figure 7 As shown, Figure 2-Figure 4 They are the original texture map of the building wall, the texture map of the building wall after occlusion and repair, and the texture map of the building wall after beautification and repair. Figure 5-Figure 7 They are the original texture map of the building roof, the texture map of the building roof after covering and repair, and the texture map of the building roof after beautification and repair.

[0107] In summary, the present invention first uses the image low-rank matrix segmentation algorithm and the SAM (Segment Anything Model) deep learning image segmentation algorithm to automatically detect the occlusion, graffiti, and noise areas of the walls and roofs to obtain the building texture area to be repaired.

[0108] Second, according to the proportion of the area to be repaired to the entire building texture area, the LAMA (Large Mask Inpainting) image repair model and the PatchMatch image repair algorithm are used for automatic texture repair respectively to ensure the repair effect of texture images with different occlusion degrees.

[0109] Third, the Stable Diffusion diffusion model, ControlNet control chart, and prompt word guidance are used to beautify the texture image after occlusion repair. Window distortion repair and noise removal are further performed to output the beautified building wall and roof texture images.

[0110] Finally, to address the problem that the diffusion model easily causes smearing and excessive repair of billboards and text areas on building walls, the YOLO (You Only Look Once) model is used to identify billboards and text areas on building walls. After using the diffusion model to perform texture repair, they are reintegrated into the original position, solving the text distortion problem caused by texture beautification using the image diffusion model.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for intelligent restoration and beautification of architectural textures integrating a deep learning model, characterized in that: The method comprises the following steps: Step S1: Generate texture images of the wall and roof to be processed based on the texture reconstruction of the three-dimensional building monomer model, and use the image low-rank matrix segmentation algorithm and the SAM image segmentation algorithm to extract the area to be repaired where the building texture needs to be repaired; Step S2: Automatically repair the area to be repaired using the LAMA image restoration model or the PatchMach image restoration algorithm according to the proportion of the area to be repaired to the entire building texture area; Step S3: Use the YOLO deep learning model to identify the billboard and text areas in the restored wall texture image, and crop and output sub-images of the text and billboard areas; Step S4: combining the image diffusion model, the ControlNet control diagram, and the image prompt words to guide the establishment of a building texture image beautification model, beautifying the restored texture image, and outputting the beautified wall and roof texture images; Step S5: Re-integrate the cropped text and billboard area sub-image to their original positions on the wall.

2. The method for intelligent restoration and beautification of architectural textures integrating a deep learning model as claimed in claim 1, characterized in that: The specific process of step S1 is as follows: S11, loading the original three-dimensional building model, reading the model's geometric information and texture information, sampling, reorganizing, and merging the model's building texture in horizontal and vertical directions, and obtaining texture images of the building's walls and roof respectively; S12, using an image low-rank matrix segmentation algorithm to obtain distorted texture areas in the wall and roof texture images to obtain a first texture restoration area; S13, segmenting the wall and roof texture images according to the SAM image segmentation algorithm, extracting the wall occlusion and roof coverage areas using prompt words, and obtaining a second texture restoration area; S14: Merge the first and second texture repair areas to the final repair area of the building wall and roof.

3. The method for intelligent restoration and beautification of architectural textures integrating a deep learning model as claimed in claim 2, characterized in that: The specific process of step S12 is as follows: (1) Convert the input texture image into a single-channel matrix D by grayscale processing, and then normalize it to obtain the normalized matrix Dnormal; (2) Decompose the normalized matrix Dnormal into a low-rank matrix A and a sparse matrix E, where the low-rank matrix A represents the main structural information of the building texture and the sparse matrix E represents the distorted texture outliers; (3) Solve the model using the alternating direction multiplier algorithm to obtain the low-rank matrix A and the sparse matrix E; (4) The obtained sparse matrix E is processed and the positions and values corresponding to the non-zero elements are extracted to obtain the positions of the distorted building texture areas and generate the first texture repair area in the form of a binary image.

4. The method for intelligent restoration and beautification of architectural textures integrating a deep learning model as claimed in claim 3, characterized in that: The specific process of step S13 is as follows: The SAM image segmentation algorithm is used to segment the wall and roof texture images. Masks are automatically extracted for the repair areas of wall occlusion and roof coverage according to the prompt words. The extracted masks are then binarized and connected domain analysis is performed to remove noise and small isolated areas, retaining larger areas as the second texture repair areas for the wall and roof textures.

5. The method for intelligent restoration and beautification of architectural textures integrating a deep learning model as claimed in claim 4, characterized in that: The specific process of step S2 is as follows: S21. Calculate the area ratio of the wall surface repair area and the roof texture repair area to the corresponding building texture area. For the area ratio of the repair area that is less than or equal to 10%, automatically repair the area based on the LAMA image repair model. S22. For an area to be repaired that accounts for more than 10% and less than 30% of the area, the current area to be repaired is divided into multiple sub-texture repair areas with an area of less than 10% according to a grid, and the sub-texture repair areas are automatically repaired using the LAMA image repair model; S23. For the area to be repaired that accounts for more than or equal to 30%, the PatchMach image restoration algorithm is used to perform additional restoration on the wall and roof textures. Among them, the LAMA image restoration model directly repairs texture images with a resolution larger than 256x256 without downsampling the texture image first.

6. The method for intelligent restoration and beautification of architectural textures integrating a deep learning model as claimed in claim 5, characterized in that: In step S4, the specific process of establishing the building texture image beautification model is as follows: S41. Select the Stable Diffusion diffusion model as the basic model; S42. Add ControlNet control diagrams to the Stable Diffusion diffusion model; S43. Add positive and negative prompt words based on the architectural texture type, style, and architectural element layout requirements of the architectural texture image to guide the stable diffusion model when generating textures.

7. The method for intelligent restoration and beautification of architectural textures integrating a deep learning model as claimed in claim 6, characterized in that: The specific process of step S42 is as follows: the structural information of the building texture is extracted through the preprocessor of Canny edge detection and Depth depth estimation, and the control tensor is generated and fused with the U-Net feature map of the Stable Diffusion diffusion model.

Images