Repairing method and device for multi-style damaged building components and storage medium

Abstract

The invention discloses a repairing method and device for multi-style damaged building components and a storage medium, and belongs to the technical field of building gallery construction. The method comprises the following steps: constructing a reference image library composed of a plurality of single-style building component reference images; acquiring a plurality of images of the same building component to be repaired of the same multi-style fusion building; preprocessing the to-be-restored building component image; matching the reference images with the similarity higher than a similarity threshold value; obtaining a defect position of the to-be-repaired building component image; fusing and reconstructing the plurality of to-be-repaired building component images with different defect positions to form a fused image of the building component; the fused image of the building component is artificially approved, and the fused image meeting the requirement is added into the reference image library. According to the method, the reference image library is established by fusing and reconstructing the defective building component images of the multi-style fused building, and reference is provided for subsequent repairing of the multi-style fused building component and the digital image of the multi-style fused building component.

Description

[0001] Divisional application This application is a divisional application of the Chinese invention patent application [Application Number: 2024108223548] [Invention Name: A method and device for repairing damaged architectural components of multiple styles, and storage medium] filed on June 24, 2024. Technical Field

[0002] The present invention belongs to the technical field of building restoration, and in particular relates to a method and device for repairing damaged building components of multiple styles, and a storage medium. Background Art

[0003] Fusion architecture is a form of architecture that combines elements of many different styles. This architectural style usually combines multiple architectural design elements to create a unique and individual architectural form. Fusion architecture can integrate the characteristics of different cultures, eras and styles in design, showing diversity and innovation.

[0004] Shanghai-style architecture is a typical multi-style fusion architecture, which usually refers to the Shikumen and lilong buildings with local characteristics that emerged in Shanghai in the first half of the 20th century. These buildings represent part of Shanghai's history and culture and have important historical, cultural and artistic values.

[0005] Many multi-style fusion buildings, including Shanghai-style buildings, are currently facing certain difficulties in repair and protection, such as demolition and reconstruction, lack of maintenance and restoration, planning conflicts, disaster damage and other issues. In order to better protect and inherit the cultural heritage of style-fusion buildings and protect the style and characteristics of style-fusion buildings, innovative use of database technology is used to digitally preserve the decorative features of Shanghai-style buildings to facilitate the subsequent restoration and repair process.

[0006] In the actual use of the architectural digital image database, it was found that due to the particularity of multi-style fusion buildings, the short period of popularity and the small region, the ones that can survive are even rarer. In addition, most of the surviving ancient buildings are poorly repaired and are rather dilapidated, making it difficult to find a complete one for reference.

[0007] Chinese patent application CN202010352962.9 discloses a multi-directional repair system and method for damaged ancient building columns. The multi-directional repair system for damaged ancient building columns includes an image acquisition module, an image analysis module, a central control module, a repair site determination module, a damage degree measurement module, a three-dimensional modeling module, a repair simulation module, a repair plan generation module, a feasibility analysis module, and a display module; the multi-directional repair method for damaged ancient building columns includes collecting and analyzing images of ancient building columns; determining the repair site and its area and depth; establishing a three-dimensional model of the column to simulate the repair operation; generating a repair plan and analyzing the feasibility.

[0008] In the above-mentioned prior art, a method of repairing by collecting architectural components of the ancient building to be repaired and determining the repair site and the degree of damage is mentioned. However, for the existing multi-style fusion buildings, which are extremely rare, it is difficult to find reference objects or / and reference images for repair. The repair can only be carried out based on the restorer's own understanding of the building. The final repair result is related to the restoration personnel's literacy and lacks objectivity.

[0009] Most of the existing ancient buildings that integrate multiple styles are severely damaged, and it is almost impossible to have complete building components preserved, making them impossible to be directly used as a reference for restoration. Summary of the invention

[0010] The purpose of the present invention is to provide a method and device for repairing damaged building components of multiple styles, and a storage medium, which partially solves or alleviates the above-mentioned deficiencies in the prior art, and establishes a reference library by fusing and reconstructing images of defective building components of multi-style fusion buildings, thereby providing a reference for the repair of multi-style fusion buildings and their digital images.

[0011] In order to solve the above-mentioned technical problems, the present invention specifically adopts the following technical solutions: A method for repairing damaged building components of multiple styles, comprising: Construct a benchmark image library consisting of several benchmark images of architectural components of a single style; Acquire several images of the same building component to be restored in the same multi-style fusion building; Preprocessing the image of the building component to be repaired, and extracting the contour information of the building component in the image of the building component to be repaired; Calculating the similarity between the image of the building component to be repaired and each building component reference image in the reference image library, and matching the building component reference images whose similarity is higher than a similarity threshold; Comparing the contour information of the image of the building component to be repaired with the matched reference image of the building component to obtain the defect position of the image of the building component to be repaired; fusing and reconstructing a plurality of images of building components to be repaired having different defect positions to form a fused image of the building components; The fused images of the building components are manually reviewed, and the fused images that meet the requirements are added to the reference library.

[0012] As an improvement, the repair method further comprises the steps of: Preprocessing the input image of the building component to be repaired to extract contour information of the building component in the image of the building component to be repaired; Comparing the extracted contour information with the corresponding main style reference image of the building component to be repaired in the reference image library to obtain the defect position and degree of the image of the building component to be repaired; According to the defect position and degree of the image of the building component to be repaired, the fused image of the multi-style building component in the reference image library or the reference image of the building component in the reference image library is matched; The building component image is repaired by referring to the fused image of multi-style building components in the matched reference library or the single-style building component reference image in the reference image library to obtain the building component repaired image.

[0013] As an improvement, the step of matching the fused image of multi-style building components in the reference image library or the building component reference image in the reference image library according to the defect position and degree of the defect of the building component image specifically includes: The defect state of the building component image is obtained by using the defect position and defect degree of the building component image; Determining whether the defect state is within a preset first threshold interval, a second threshold interval, or a third threshold interval; When the defect state is within the first threshold interval, the style of the building component image is judged to obtain the style of the building component image, and the fused image in the reference image library or the reference image in the reference image library is matched according to the style of the building component image; When the defect state is within the second threshold interval, matching the corresponding fused image in the reference image library; When the defect state is within the third threshold interval, a reference image of the building component having the same main style as the damaged building component in the reference image library is matched.

[0014] As an improvement, the image of the building component to be repaired is a 2D image or / and a 3D image, and the building component reference image includes a 2D reference image and a 3D point cloud reference image; the 2D reference image of the building component reference image includes 2D reference images at several angles.

[0015] As an improvement, when obtaining images of building components to be restored of a multi-style fusion building through on-site acquisition, 2D images and 3D point cloud images of the building components from several angles are collected.

[0016] As an improvement, the main style of the building in the image of the building component to be repaired is manually annotated in advance, and then a building component reference image that matches the main style of the image of the building component to be repaired is matched among reference images whose similarity is higher than a similarity threshold.

[0017] As an improvement, the degree of defect of the image of the building component to be repaired is obtained, and when a plurality of images of the building component to be repaired with different defect positions are fused and reconstructed, the image of the building component to be repaired with a degree of defect less than a preset degree threshold is selected.

[0018] As an improvement, the age and region of the building components in the image of the building component to be repaired are manually annotated; when the defect of the fused image exceeds the defect threshold, the reference image of the building component of the same age, region and main style is matched for fusion and reconstruction, and the fused and reconstructed fused image is submitted to manual review, and the fused image that meets the requirements is added to the reference library.

[0019] The present invention also provides a repair device for damaged building components of multiple styles, comprising: A reference image library construction module is used to construct a reference image library consisting of several reference images of building components of a single style; The contour information extraction module obtains several images of the same building component to be restored of the same multi-style fusion building; pre-processes the image of the building component to be restored, and extracts the contour information of the building component in the image of the building component to be restored; A similarity calculation module is used to calculate the similarity between the image of the building component to be repaired and each building component reference image in the reference image library, and match the reference images whose similarity is higher than the similarity threshold; A defect position comparison module is used to compare the contour information of the image of the building component to be repaired with the matched reference image of the building component to obtain the defect position of the image of the building component to be repaired; A fusion and reconstruction module is used to fuse and reconstruct multiple images of building components to be repaired with different defect positions to form a fused image of the building component; The reference library construction module manually reviews the fused images of building components and adds the fused images that meet the requirements into the reference library.

[0020] As an improvement, the method further includes: a secondary reconstruction module, which is used to manually mark the age and region of the building components in the image of the building component to be repaired; when the defect of the fused image exceeds the defect threshold, a reference image of the same age, region and main style is matched for fusion and reconstruction, and the fused and reconstructed fused image is submitted to manual review, and the fused image that meets the requirements is added to the reference library.

[0021] The present invention also provides a storage medium, in which a computer program is stored. When the computer program is executed, the above-mentioned method for constructing a digital pattern reference library for multi-style fusion architectural restoration can be implemented.

[0022] A computer system includes a memory and a processor; the memory stores a computer program, and when the computer program is executed by the processor, the method for constructing a digital pattern reference library for multi-style fusion architectural restoration can be implemented.

[0023] The present invention is beneficial in that: The method for repairing damaged building components of multiple styles of the present invention constructs a reference image library of building components of a single style (for example, a main style or an external style) to obtain the defect position and defect of the image of the building component to be repaired of the multi-style fusion building. Then, multiple images of the building components to be repaired with different defect positions are stitched together to form a fused multi-style building component image, and multiple fused images are obtained in this way to construct a reference library for reference in the repair of multi-style damaged building components or multi-style damaged buildings and their images.

[0024] The fused image in the present invention is composed of images of different defective positions of the same multi-style fused building. Compared with restoration based on manual experience or images of other styles (including single style or multi-style fusion), the style unity can be maintained to the greatest extent. When the restored fused image is used as a reference to further restore the building itself or the digital image of the building, it can be integrated with the style at that time as much as possible to avoid abruptness.

[0025] In addition, the present invention also uses manual (expert) review to check the spliced ​​fusion image, and reviews the fusion image through the expert's professional knowledge and artistic accumulation to prevent the spliced ​​fusion image from deviating from the original style. Once the fusion images that have passed the expert review are accumulated to a sufficient number to form a library, they can be used to repair the building itself and the digital image of the building, thereby greatly reducing the workload of the experts and making the restoration work more efficient and guided.

[0026] Finally, for the fused image whose defect is still larger than the threshold, the present invention further matches the reference image of the same era, region and main style for fusion reconstruction, and submits the fused and reconstructed fused image to manual review, and adds the fused image that meets the requirements into the reference library. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale. Obviously, the drawings described below are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without paying creative labor.

[0028] Figure 1 This is a flow chart of Embodiment 1 of the present invention; Figure 2 This is a functional module diagram of Embodiment 2 of the present invention; Figure 3 and Figure 4 Images of damaged buildings of various styles collected at the field site; Figure 5 This is a flow chart of Embodiment 3 of the present invention; Figure 6 This is a functional module diagram of embodiment 4 of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] Herein, suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings by themselves. Therefore, "module", "component" or "unit" can be used mixedly.

[0031] In this document, the terms "upper", "lower", "inner", "outer", "front", "back", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0032] In this document, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0033] Herein "and / or" includes any and all combinations of one or more of the associated listed items.

[0034] Herein, "plurality" means two or more than two, ie, it includes two, three, four, five, etc.

[0035] In this article, "multi-style fusion architecture" refers to buildings that integrate multiple architectural styles. Generally speaking, multi-style fusion architecture is formed by an existing main style and some foreign styles. In a multi-style building, it is possible that some building components are the main style, while other building components are foreign styles. It is also possible that a certain building component is the main style combined with some foreign styles. And the fusion is relatively random, and the location and degree of fusion are based on the personal preferences of the craftsmen, and there is no universal regularity. Accordingly, once any of the building components is damaged, the building is also damaged, which is also called a damaged multi-style building.

[0036] In this article, "reference image" refers to an image of a single style and complete standard building component. For example, an image of only a main style (e.g., a style of local architectural characteristics) and complete standard building components, or an image of only an exotic style (e.g., a style of foreign or foreign architecture) and complete standard building components. Usually, for building components of a single style, there is only one reference image of each style. Specifically, it can be obtained from various architecture-related teaching materials or existing materials.

[0037] In this article, "complete image" refers to a complete image obtained by stitching together multiple images of the same building component with different defect locations or the same building component. The building component is a fusion of multiple styles, for example, a main style is combined with at least one foreign style. The main style refers to the style that most of the structures or outlines of the building component that combines multiple styles adopt, while a small part of the structure or outline adopts another style.

[0038] Example 1: Chinese patent application CN202010352962.9 discloses a multi-directional repair system and method for damaged ancient building columns. The multi-directional repair system for damaged ancient building columns includes an image acquisition module, an image analysis module, a central control module, a repair site determination module, a damage degree measurement module, a three-dimensional modeling module, a repair simulation module, a repair plan generation module, a feasibility analysis module, and a display module; the multi-directional repair method for damaged ancient building columns includes collecting and analyzing images of ancient building columns; determining the repair site and its area and depth; establishing a three-dimensional model of the column to simulate the repair operation; generating a repair plan and analyzing the feasibility.

[0039] In the above-mentioned prior art, a method of repairing by collecting architectural components of the ancient building to be repaired and determining the repair site and the degree of damage is mentioned. However, for the existing multi-style fusion buildings, which are extremely rare, it is difficult to find reference objects or / and reference images for repair. The repair can only be carried out based on the restorer's own understanding of the building. The final repair result is related to the restoration personnel's literacy and lacks objectivity.

[0040] Most of the existing ancient buildings that integrate multiple styles are severely damaged, and it is almost impossible to have complete building components preserved, making them impossible to be directly used as a reference for restoration.

[0041] like Figure 1 As shown, in order to solve the above technical problems, the present invention provides a method for repairing damaged building components of multiple styles, which provides a reference for repair by constructing a digital pattern reference library of multi-style fusion building components, and the specific steps include: S11 builds a benchmark image library consisting of several benchmark images of building components of a single style.

[0042] The building components described in the present invention include basic elements of buildings such as doors, beams, walls, and columns. Single-style buildings, such as Hui-style buildings, have distinct characteristics and have been popular for a long time, so there are many existing buildings and images. Accordingly, a corresponding benchmark image library can be established according to each style.

[0043] In this implementation, the purpose of constructing a benchmark image library is to serve as a basic structural benchmark for the multi-style building component drawings that are expected to be restored later. This will be introduced in detail later and will not be repeated here.

[0044] Of course, in order to be closer to the image of the damaged building to be repaired and integrated with multiple styles (or called multi-style fusion building) and / or the multi-style building components therein, a single-style building similar to the multi-style fusion building can be selected. For example, the main style of some multi-style fusion buildings is Huizhou architecture, and some other elements are integrated. Therefore, in this step, the architectural component baseline image of Huizhou architecture can be selected.

[0045] In addition, for the convenience of subsequent use, the reference image in this embodiment includes a 2D reference image and a 3D point cloud reference image; the 2D reference image of the reference image includes 2D reference images at several angles. Since there are many existing physical objects and images of single-style buildings, it is also convenient to collect 3D point clouds and multi-angle 2D images.

[0046] S12 obtains a plurality of images of the same building component to be repaired in the same multi-style fusion building; pre-processes the images of the building component to be repaired, and extracts contour information of the building component in the images of the building component to be repaired.

[0047] There are several ways to obtain images of architectural components to be restored in multi-style fusion buildings: First, the building itself is preserved, but a certain architectural component is missing. In this case, on-site collection can be carried out. Figure 3 and Figure 4 When the images of the building components to be restored in the multi-style fusion building are obtained by on-site acquisition, 2D images and 3D point cloud images of the building components at several angles are collected.

[0048] The second is that the building has been completely destroyed and disappeared, with only some pictures or videos remaining. In this case, it is necessary to decompose the various building components as much as possible based on the remaining information to provide assistance for subsequent integration and reconstruction.

[0049] In this embodiment, a building component, such as a column, is used as an example. In this step, multiple images of columns in a certain multi-style fusion building can be obtained from the above channels (of course, the building components in these images are all defective). Of course, the more the number, the more information, and the greater the probability of successfully fusion and reconstruction into a complete image.

[0050] After obtaining the images of the building components to be repaired of the multi-style fusion buildings, these images may contain various background information and other information unrelated to the building components. Therefore, it is necessary to preprocess these images of the building components to be repaired, mainly to extract their contour information to obtain the building component images of the damaged building components themselves for subsequent comparison.

[0051] In addition, the age and region of the building components in the image of the building component to be restored can also be manually annotated.

[0052] S13 calculates the similarity between the image of the building component to be repaired and a reference image of the building component in the reference image library, and matches the reference image whose similarity is higher than a similarity threshold.

[0053] Since the image of the building component to be repaired does not have a complete physical object or image for reference, it is difficult to know the specific location and extent of its damage. The benchmark image library actually provides a rough reference. For example, as a column, even if the style is different, the general structure is still similar, only some decorative patterns, sculptures, etc. are different. Therefore, compared with the benchmark image, the approximate defect situation can still be obtained. Preferably, a similarity comparison is performed based on the above-mentioned contour information. If there are multiple images above the similarity threshold, the one with the highest similarity is used as the benchmark image for comparison. Preferably, the one with the most complete contour information is selected from several images of the building component to be repaired to calculate the similarity with each building component benchmark image.

[0054] In this implementation, since the reference image library includes reference images of various single-style building components, for example, reference images of columns in the Hui style, reference images of other single-style columns that are significantly different from the Hui style, and reference images of columns in other styles similar to the Hui style, the main style of the building component image to be repaired or the main style of the building in the building component image can be manually annotated in advance, and the reference image that matches the main style of the building component image to be repaired can be matched in the reference images whose similarity is higher than the similarity threshold. Of course, in some embodiments, AI can also be selected for annotation, but its accuracy is not as good as the manual standard.

[0055] For example, the building component to be repaired is a fusion of the Hui style and other architectural elements, so during preprocessing, its main style can be manually annotated. Then, in this implementation, under the premise that the similarity meets the requirements, a building component reference image with the same main style can be selected, so that the building component reference image is closer to the appearance of the building component to be repaired.

[0056] In order to make the similarity calculation more accurate and thus find a reference image that is closer to the image of the building component to be repaired, in this implementation, the reference image should collect 3D point cloud images and multi-angle 2D images as much as possible.

[0057] In addition, calculating the similarity between images is a prior art and will not be described in detail in the present invention. For example: When calculating the similarity between 2D images, commonly used methods include: Perceptual Hashing: Maps images to hash values ​​of fixed length and evaluates the similarity of two images by comparing their hash values.

[0058] Structural Similarity Index (SSIM): It measures the similarity between two images by comparing the brightness, contrast, and structural information of the images. It is usually used to evaluate the quality of images.

[0059] Histogram Similarity: Compares the color histograms of images and evaluates the similarity of images by calculating the distance between histograms.

[0060] For similarity calculation between 3D images, commonly used methods include: 3D shape matching: Extract features from 3D shapes and then compare the similarity between two 3D shapes through feature matching.

[0061] 3D mesh comparison: 3D mesh models are registered or aligned, and then the similarity between 3D models is evaluated by comparing the differences between mesh coordinates or vertices.

[0062] 3D point cloud registration: Two 3D point clouds are registered and then their similarity is evaluated by measuring the difference between the point cloud transformations.

[0063] S14 compares the contour information of the image of the building component to be repaired with the matched reference image of the building component to obtain the defect position of the building component to be repaired.

[0064] After selecting a suitable reference image, the comparison can be performed.

[0065] When the image of the building component to be repaired is a 3D point cloud, the defect of the building component to be repaired can be obtained by directly subtracting the 3D point cloud image of the building component to be repaired from the 3D point cloud reference image.

[0066] When the image of the building component to be repaired is a 2D image, the image of the building component to be repaired and the reference image are first segmented, and the segmented building components are compared, and the missing part is the defect of the building component.

[0067] S15 fuses and reconstructs a plurality of images of the building components to be repaired having different defect positions to form a fused image of the building components.

[0068] The difficulty in the prior art is actually that there is no complete image of the building component for reference, and the problem to be solved by the present invention is actually to combine the existing defective images of a certain building component to try to construct a complete image of the building component (i.e., a fused image).

[0069] Since the defect positions of each image have been found in the previous step, theoretically, a complete image of the building component can be obtained by stitching together multiple images with different defect positions. For example, the base of the column is missing in image A, and part of the relief of the column is missing in image B, so a complete image of the column can be obtained by stitching the two together. Image A and image B can be images of the same building component on different buildings, or images of the same building component on the same building. They can also be images of the same building component with different defect positions and / or defect degrees collected at different times.

[0070] Of course, since there are multiple images, it is possible that the defect position of two images is the same. Therefore, when fusing and reconstructing, the image with the smaller degree of defect is selected, that is, the degree of defect of the image of the building component to be repaired is obtained. When fusing and reconstructing multiple images of the building component to be repaired with different defect positions, the image of the building component to be repaired with a degree of defect less than the degree threshold is selected.

[0071] S16 manually reviews the fused images of building components and adds the fused images that meet the requirements into the reference library.

[0072] In the previous step, a fused image was obtained by fusing and reconstructing multiple images of the building components to be restored. Although the images of the same building components of the same multi-style fusion building are used, the multi-style fusion building elements are relatively complex. Even in the same style, there will be differences between different buildings. Therefore, in this step, the fused image needs to be reviewed to determine whether it conforms to the style.

[0073] Identifying a style is actually a subjective judgment that is difficult to be completed by AI. Therefore, in this step, it is recommended to use manual review to determine whether the image meets the requirements based on the expert's deep understanding of the architectural style.

[0074] Any images that meet the requirements can be added to the library and used as a reference for subsequent restoration of the electronic image of the multi-style fusion building or the building itself.

[0075] When the defect of the fused image exceeds the defect threshold, S17 matches the reference image of the same era, region and main style for fusion reconstruction, and submits the fused and reconstructed fused image for manual review, and adds the fused image that meets the requirements into the reference library.

[0076] It is possible that even if multiple images to be restored are fused, they cannot be pieced together into a complete image of the building component, so further fusion and reconstruction are required.

[0077] In step S12, the age and region of the building components to be repaired are marked. When the defect of the fused image exceeds the defect threshold, for example, 5%, a reference image of the same age, region and main style is found from the reference image library for fusion and reconstruction. Of course, the fused image reconstructed by this method is more likely to be unsatisfactory, so it also needs to be manually reviewed and added to the reference library.

[0078] When the reference library is built, it can be used to repair building components. For example, an image of the building component to be repaired is input into the system, and then its contour information is extracted and matched to a corresponding building component reference image based on the contour information for repair. Specifically, the method also includes: S18 pre-processes the input image of the building component to be repaired, and extracts the contour information of the damaged building component in the image of the building component to be repaired.

[0079] As mentioned above, the purpose of obtaining the contour information of the building component in the building component image is to obtain the defect status of the building component.

[0080] S19 compares the contour information extracted in step S18 with a reference image in a reference image library having the same main style as the damaged building component to obtain the defect position and degree of the image of the building component to be repaired.

[0081] It is understandable that the style of the reference image of the building component is consistent with the main style of the building component to be repaired, so that the defect of the building component image can be restored as accurately as possible. Therefore, a reference image with the same main style is first matched in the reference image library, and then the defect position and degree are compared with the matched reference image. Since there is only one reference image for each style of building component in the reference library, once the name and main style of the damaged building component are known (for example, manually marked or entered), the system can match it with a reference image with the same main style in the reference library.

[0082] When the image of the building component is a 3D point cloud, the defect of the building component to be repaired can be obtained by directly subtracting the 3D point cloud image of the building component from the 3D point cloud reference image.

[0083] When the building component image is a 2D image, the building component image and the reference image are first segmented, and the segmented building components are compared. The missing part is the defect of the building component.

[0084] For a multi-style fusion building, its building components may be complete or incomplete. According to this step, it is possible to determine whether the building component image is defective and what the defect is. Of course, for a complete building component image, the next part of the building component image restoration work can be omitted. Of course, the complete building component image can also be added to the gallery as a reference image for restoration.

[0085] S20 matches the fused image of the multi-style building component in the reference image library or the reference image of the building component in the reference image library according to the defect position and degree of the defect of the image of the building component to be repaired.

[0086] Since the situation of style fusion in multi-style fusion buildings is relatively complicated, when it is possible to determine whether the building component is the main style, the main style fused with foreign styles, or purely foreign styles, the style of the building component is determined first. Then, according to the style of the building component, it is determined whether to use the complete image (multi-style fusion) for restoration or the reference image (single style, including the main style and foreign styles) for restoration.

[0087] In addition, when the damage of a building component exceeds a certain degree, it is difficult to determine which of the above styles it belongs to. Therefore, more specifically, the present embodiment also includes the following steps: First, the defect state of the building component image is obtained based on the defect position and defect degree of the image of the building component to be repaired. Specifically, defect state = a*defect position score + b*defect degree score, where a and b are defect coefficients. For example, taking the 100-point system as an example, if the defect position is at the corner, the defect position score is 5; if the defect position is at the edge, the defect position score is 10; if the defect position is in the middle, the defect position score is 15, and then the number of defects at each position is multiplied by the corresponding score, and so on. If the defect area is less than 5%, the defect degree score is 50; if the defect area is between 5% and 10%, the defect degree score is 70; if the defect area is above 10%, the defect degree score is 100, and so on. The specific value of the defect coefficient can be adjusted according to the actual situation, for example, a is 35% and b is 75%. Finally, the lower the score of the defect state, the higher the integrity of the building component.

[0088] Then, it is determined whether the calculated defect state is within a preset first threshold interval, a second threshold interval, or a third threshold interval; if the defect state is within the first threshold interval, the style of the building component image is determined to obtain the style of the building component image, and the fused image of multi-style building components in the reference library or the benchmark image of the building component in the benchmark image library is matched according to the style of the building component image; if the defect state is within the second threshold interval, the fused image of multi-style building components in the reference library is matched; if the defect state is within the third threshold interval, the benchmark image in the benchmark image library with the same main style as the building component is matched.

[0089] When the defect state is within the first threshold interval, it means that the defect of the building component can be judged by style. When it is judged to be a single style, it is sufficient to match the single style reference image; when it is judged to be a fusion of multiple styles, it is sufficient to match the complete image of the fusion of multiple styles. When the defect state is within the second threshold interval, it means that the style of the building component can no longer be judged. In this case, the probability of meeting the requirements after repairing using the complete image of the fusion of multiple styles is higher, so the complete image of the fusion of multiple styles is matched. Specifically, since there are many ways to fuse multiple styles, even if multiple styles are manually annotated in advance, it is possible to match multiple complete images. Therefore, it is sufficient to match the complete image with the highest similarity from the matched multiple complete images. For example, the similarity between the contour information and the contour information of the corresponding area in the complete image is calculated, and a complete image with the highest similarity is matched. When the defect state is within the third threshold interval, it means that the style of the building component can no longer be judged and the degree of defect is very serious. In this case, the probability of success is highest when the reference image of its main style is used for repair. It can be understood that in this embodiment, the first threshold interval < the second threshold interval < the third threshold interval.

[0090] S21 refers to the matched fusion image of the multi-style building components in the reference library or the benchmark image of the single-style building components in the benchmark image library, and repairs the image of the building component to be repaired to obtain a repaired image of the building component.

[0091] When a reference image is obtained, the building components can be repaired according to the reference image. When repairing, the position can be filled according to the reference image.

[0092] Example 2: Figure 2 As shown, the present invention also provides a repair device for damaged building components of multiple styles, which provides repair reference sample images for building component repair by constructing a digital pattern reference library for multi-style fusion building repair. Specifically, it includes: A reference image library construction module is used to construct a reference image library consisting of several reference images of building components of a single style; The contour information extraction module obtains several images of the same building component to be restored of the same multi-style fusion building; pre-processes the image of the building component to be restored, and extracts the contour information of the building component in the image of the building component to be restored; A similarity calculation module is used to calculate the similarity between the image of the building component to be repaired and the reference image of the building component in the reference image library, and match the reference image whose similarity is higher than the similarity threshold; A defect position comparison module is used to compare the contour information of the image of the building component to be repaired with the matched reference image of the building component to obtain the defect position of the image of the building component to be repaired; A fusion and reconstruction module is used to fuse and reconstruct multiple images of building components to be repaired with different defect positions to form a fused image of the building component; The reference library construction module manually reviews the fused images of building components and adds the fused images that meet the requirements into the reference library.

[0093] In some embodiments, the restoration device also includes: a secondary reconstruction module, which is used to manually mark the age and region of the building components in the image of the building component to be restored; when the defect of the fused image exceeds the defect threshold, match the reference image of the same age, region and main style for fusion reconstruction, and submit the fused and reconstructed fused image for manual review, and add the fused image that meets the requirements to the reference library.

[0094] In some embodiments, the repair device also includes: a preprocessing module, which preprocesses the input image of the building component to be repaired and extracts the contour information of the building component in the image of the building component to be repaired; a defect acquisition module, which is used to compare the contour information of the building component to be repaired with the reference image of the building component to be repaired in the reference image library to obtain the defect position and degree of defect of the image of the building component to be repaired; a matching module, which is used to match the complete image of the building component of the multi-style fusion building in the reference library or the reference image of the building component in the reference image library according to the defect position and degree of defect of the image of the building component to be repaired; a building component image repair module, which is used to repair the image of the building component to be repaired with reference to the complete image of the building component of the multi-style fusion building in the matched reference library or the reference image of the building component to be repaired in the reference image library to obtain a repaired image of the building component to be repaired.

[0095] Embodiment 3: The so-called multi-style fusion building refers to a building that integrates multiple architectural styles. Generally speaking, a multi-style fusion building is formed by an existing main style and some foreign styles. In a multi-style building, it is possible that some building components are the main style, while other building components are foreign styles. It is also possible that a certain building component is the main style and some foreign styles are integrated. In addition, the fusion is relatively random, and the location and degree of fusion are based on the personal preferences of the craftsmen. There is no universal regularity, which makes it difficult to repair the building after it is damaged.

[0096] In response to the above technical issues, such as Figure 5 As shown, the present invention provides a method for repairing damaged buildings of multiple styles (i.e., damaged buildings with multiple styles integrated), which repairs the digital images of damaged buildings of multiple styles based on the digital pattern reference library constructed in Example 1, and also uses the reference library constructed in Example 1. The specific steps of the present invention include: S1 decomposes the building image to be restored to obtain a separate building component image of each building component in the building image to be restored.

[0097] In some embodiments, after obtaining an image of a multi-style fusion building to be restored, it needs to be decomposed into images of building components.

[0098] There are several ways to obtain images of multi-style fusion buildings to be restored: First, the building itself is preserved, but some of its building components are missing. In this case, on-site collection can be performed. When obtaining images of building components of multi-style fusion buildings to be restored through on-site collection, 2D images of building components and 3D point cloud images from several angles are collected.

[0099] Second, the building has been completely destroyed and disappeared, leaving only some pictures or images. In this case, it is necessary to decompose each building component as much as possible based on the remaining data. The decomposition method can use an artificial intelligence model to perform instance segmentation, decomposing each building component to form a separate building component image. Instance segmentation is a prior art and will not be described in detail in the present invention.

[0100] In addition, the era and region of the multi-style fusion buildings in the building images to be restored can also be manually marked.

[0101] S2 preprocesses each building component image to extract contour information of the building component in the building component image.

[0102] Similar to Example 1, the purpose of acquiring the contour information of the building component in the building component image is to obtain the defect status of the building component. The specific method is shown in Example 1 and will not be described in detail in this example.

[0103] S3 compares the outline information of the building component with the main style reference image of the building component in the reference image library to obtain the defect position and degree of the building component image.

[0104] It is understandable that the style of the reference image of the building component is consistent with the main style of the multi-style fusion building to be repaired, so that the defect of the building component image can be restored as accurately as possible. Therefore, when the main style of the building is known, a reference image with the same main style as the building (that is, a reference image of the main style of the building component) can be matched from the reference image library, and then compared to determine the location and degree of the defect. Since there is only one reference image of each style of building component in the reference library, once the name of the damaged building component and the main style of the corresponding building are known (for example, manually marked or entered), the system can match it with a reference image with the same main style of its building in the reference library.

[0105] When the image of the building component is a 3D point cloud, the defect of the building component to be repaired can be obtained by directly subtracting the 3D point cloud image of the building component from the 3D point cloud reference image.

[0106] When the building component image is a 2D image, the building component image and the reference image are first segmented, and the segmented building components are compared. The missing part is the defect of the building component.

[0107] For a multi-style fusion building, its building components may be complete or incomplete. According to this step, it is possible to determine whether the building component image is defective and what the defect is. Of course, for a complete building component image, the next part of the building component image restoration work can be omitted. Of course, the complete building component image can also be added to the gallery as a reference image for restoration.

[0108] S4 matches the complete image of the building component of the multi-style fusion building in the reference library or the reference image of the building component in the reference image library according to the defect position and degree of the defect of the building component image.

[0109] Since the situation of style fusion in multi-style fusion buildings is relatively complicated, when it is possible to determine whether the building component is the main style, the main style fused with foreign styles, or purely foreign styles, the style of the building component is determined first. Then, according to the style of the building component, it is determined whether to use the complete image (multi-style fusion) for restoration or the reference image (single style, including the main style and foreign styles) for restoration.

[0110] In addition, when the damage of a building component exceeds a certain degree, it is difficult to determine which of the above styles it belongs to. Therefore, more specifically, the present embodiment also includes the following steps: S41 obtains the defect state of the building component image by using the defect position and defect degree of the building component image.

[0111] Specifically, the step of obtaining the defect state of the building component image includes: Defect state = a*defect position score + b*defect degree score, where a and b are defect coefficients. For example, taking the 100-point system as an example, if the defect position is at the corner, the defect position score is 5; if the defect position is at the edge, the defect position score is 10; if the defect position is in the middle, the defect position score is 15, and then the number of defects at each position is multiplied by the corresponding score, and so on. If the defect area is less than 5%, the defect degree score is 50; if the defect area is between 5% and 10%, the defect degree score is 70; if the defect area is above 10%, the defect degree score is 100, and so on. The specific value of the defect coefficient can be adjusted according to the actual situation, for example, a is 35% and b is 75%.

[0112] The lower the final defect score, the higher the integrity of the building component.

[0113] S42, when the defect state is within the first threshold interval, determines the style of the building component image to obtain the style of the building component image, and matches the complete image of the building component of the multi-style fusion building in the reference library or the benchmark image of the building component in the benchmark image library according to the style of the building component image.

[0114] When the defect state is within the first threshold interval, it means that the defect of the building component can be judged by style. Specifically, the judgment is made by the style type that is manually annotated in advance (for example, if it is a single-style image, the corresponding style can be annotated; if it is a fusion of multiple styles, the multiple styles that are fused can be annotated). If it is judged to be a single style, it is enough to match the reference image of the single style (that is, the same style); if it is judged to be a fusion of multiple styles, it is enough to match the corresponding complete image of the fusion of multiple styles.

[0115] S43 matches the complete image of the building component of the multi-style fusion building in the reference library when the defect state is within the second threshold interval.

[0116] When the defect state is within the second threshold interval, it means that the style of the building component can no longer be determined. In this case, the probability of meeting the requirements after repair using a complete image with multiple styles fused is higher, so the corresponding complete image with multiple styles fused is matched. Specifically, since there are many ways to fuse multiple styles, even if multiple styles are manually annotated in advance, it is possible to match multiple complete images. Therefore, the complete image with the highest similarity can be matched from the multiple matched complete images. For example, the similarity between the contour information and the contour information of the corresponding area in the complete image is calculated, and a complete image with the highest similarity is matched.

[0117] S44: when the defect state is within the third threshold range, matching a reference image of the main style of the building component in the reference image library. When the defect state is within the third threshold range, it means that the style of the building component can no longer be determined and the degree of defect is very serious. In this case, the probability of success is highest when the main style benchmark image is used for repair.

[0118] It can be understood that, in this embodiment, the first threshold interval < the second threshold interval < the third threshold interval.

[0119] S5 refers to the complete image of the building component of the multi-style fusion building in the matched reference library or the benchmark image of the building component in the benchmark image library, and repairs the building component image to obtain the building component repair image.

[0120] When a reference image is obtained, the building components can be repaired according to the reference image. When repairing, the position can be filled according to the reference image.

[0121] S6 uses the architectural component restoration image to restore the architectural image to be restored to obtain a restored architectural image.

[0122] After obtaining the restored images of all building components (of course, the original complete building component images can be used directly), all the building component images are assembled to obtain the final restored building image.

[0123] In fact, it is unknown whether the restored architectural image meets the requirements. It is possible that the style of the restored architectural image will be very abrupt and not meet the requirements. Therefore, after obtaining the restored architectural image, further processing is required.

[0124] S7 calculates the manual review degree according to the defect state of each building component image in the building image to be restored. If the manual review degree exceeds the manual review threshold, the restored building image is submitted to manual review.

[0125] The defect state represents the degree of defect of building components. If the degree of defect of the entire building is low, there is more confidence in the repair of the above steps. Therefore, it can be directly considered that the restoration is successful and no subsequent steps are required.

[0126] However, when the degree of damage to the entire building exceeds a certain threshold, manual review will be required, and experts will analyze whether the repair work is qualified.

[0127] In this step, the method of calculating the manual appraisal degree according to the defect state of each building component image in the building image to be repaired includes: S71 divides architectural components into several categories according to their influence on the overall style of the building, and sets a manual review coefficient for each category of architectural component images.

[0128] For example, the external, more conspicuous building components are divided into one category, the less conspicuous ones into another category, and the internal, less conspicuous ones into another category. The first category is given a higher weight; the second category has the second highest weight, and the third category has the lowest weight. The manual review coefficient is adaptively set according to the weight of the type.

[0129] S72 calculates the average number of defect states of each type of building component image, and adjusts the average number using the manual review coefficient of the building component image of that type to obtain the manual review degree of the single type of building component image.

[0130] For example, there are 5 types of the first category of building components. After obtaining the average number of defect states of this type of building components and multiplying it by the manual evaluation coefficient, the manual evaluation degree of this type of building components can be obtained.

[0131] S73 takes the sum of the manual review degrees of all single-type building component images as the manual review degree of the building image to be restored.

[0132] Continuing with the previous example, by summing up the manual review degrees of the three types of building components, we can get the manual review degree of the entire building.

[0133] Finally, the manual review degree is compared with the manual review threshold to determine whether the restored building image still needs manual review.

[0134] S8, in the case where the manual review has not been passed, replace the reference of the image of the building component in the defective state within the third threshold interval to perform secondary repair to obtain a secondary repaired image of the building component; use the secondary repaired image of the building component to replace the repaired image of the building component, and then perform secondary restoration of the building image to obtain a secondary restored building image; and perform secondary manual review on the secondary restored building image.

[0135] After manual review by experts, it was found that the restored architectural image style was abrupt and did not meet the requirements, indicating that there was a problem with the restored architectural component image. At this time, the restoration strategy needs to be adjusted.

[0136] The most likely problem is the image of the building component whose defective state is within the third threshold interval. In the original strategy, the image of the building component whose defective state is within the third threshold interval is referenced by the reference image of the main style, but it does not meet the requirements. Therefore, in this step, the complete image of multi-style fusion is used for restoration to form a secondary restoration image of the building component.

[0137] The images of building components whose defects are within the first and second threshold ranges are repaired according to the original strategy without changing.

[0138] Then the image is stitched and restored again to obtain the second restored image of the building. The second restored image of the building needs to be manually reviewed to see if it is qualified.

[0139] S9, in the case where the second manual review fails, replaces the reference of the building component image in the defective state within the second threshold interval for secondary repair; uses the secondary repaired image of the building component to replace the repaired image of the building component, and then performs three restorations on the building image to be repaired to obtain the three restored building images; and performs three manual reviews on the three restored building images.

[0140] After manual review by experts, it was considered that the style of the architectural image after the second restoration was still abrupt and did not meet the requirements. It was possible that there was a problem with the repair of the defective architectural components within the second threshold range.

[0141] In the original strategy, the image of the building component with a defect state within the second threshold interval is referenced by the complete image of multi-style fusion, but it does not meet the requirements. Therefore, in this step, the reference image of the main style is used for restoration to form a secondary restoration image of the building component.

[0142] The images of building components whose defective states are within the first and third threshold intervals are repaired according to the original strategy without changing.

[0143] Then the image is stitched together and restored again to obtain the three restored images of the building. The three restored images of the building need to be manually reviewed to see if they are qualified.

[0144] S10, in the case that the three manual reviews have not been passed, the reference of the building component image in the defective state within the second threshold interval and the third threshold interval is replaced simultaneously for secondary restoration; the repaired image of the building component is replaced with the secondary repaired image of the building component, and then the building image to be repaired is restored four times to obtain the four restored building images; the four restored building images are subjected to four manual reviews.

[0145] After manual review by experts, it was considered that the architectural images after three restorations still had an abrupt style and did not meet the requirements. It was possible that there were problems with the restoration of the building components in the second and third threshold ranges.

[0146] In the original strategy, the image of the building component whose defective state is within the third threshold interval is referenced by the reference image of the main style; the image of the building component whose defective state is within the second threshold interval is referenced by the complete image of multi-style fusion, but it does not meet the requirements. Therefore, in this step, the image of the building component whose defective state is within the second threshold interval is repaired by using the reference image of the main style to form a secondary repaired image of the building component. The image of the building component whose defective state is within the third threshold interval is repaired by using the complete image of multi-style fusion to form a secondary repaired image of the building component.

[0147] The images of building components whose defective states are within the first threshold range are repaired according to the original strategy without changing.

[0148] Then the image is stitched together and restored again to obtain the fourth restored image of the building. And the fourth restored image of the building needs to be manually reviewed to see if it is qualified.

[0149] In the case that S11 fails to pass four manual reviews, it obtains architectural images of the same era, the same region and the same main style as reference for restoration according to the main style of the architectural image to be restored.

[0150] If all four restorations fail to pass manual review, the next best option is to obtain architectural images of the same era, same region and with the same main style as the main style of the building image to be restored as a reference for restoration.

[0151] It is foreseeable that the results of this repair will still need to be submitted to manual review. If the manual review fails again, it means that automatic repair is hopeless and the repair work can only be taken over by manual personnel.

[0152] Example 4: Figure 6As shown, the present invention also provides a repair device for damaged buildings of multiple styles, which comprises: A decomposition module, used for decomposing the building image to be restored, and obtaining a separate building component image of each building component in the building image to be restored; A preprocessing module preprocesses each building component image and extracts contour information of the building component in the building component image; A defect acquisition module is used to compare the contour information of the building component with the reference image of the building component in the reference image library to obtain the defect position and degree of the building component image; A matching module is used to match the complete image of the building component of the multi-style fusion building in the reference library or the reference image of the building component in the reference image library according to the defect position and degree of the defect of the building component image; A building component image restoration module is used to repair the building component image by referring to the complete image of the building component of the multi-style fusion building in the matched reference library or the reference image of the building component in the reference image library to obtain a restored image of the building component; The building image restoration module uses the building component restoration image to restore the building image to be restored to obtain the restored building image.

[0153] In some embodiments, the matching module specifically includes: A defect state acquisition unit, used to acquire the defect state of the building component image by using the defect position and defect degree of the building component image; A judging unit, used for judging whether the defect state of the building construction image is within a first threshold interval, a second threshold interval, or a third threshold area; a first reference image acquisition unit, configured to, when the judgment unit judges that the defect state of the building construction image is within a first threshold interval, judge the main style of the building component image to acquire the style of the building component image, and match the complete image of the building component of the multi-style fusion building in the reference library or the reference image of the building component in the reference image library according to the style of the building component image; A second reference image acquisition unit is configured to match a complete image of a building component of a multi-style fusion building in a reference image library when the judgment unit judges that the defect state of the building construction image is within a second threshold interval; The third reference image acquisition unit is used to match the reference image of the main style of the building component in the reference image library when the judgment unit judges that the defect state of the building construction image is within a third threshold range.

[0154] The present invention also provides a computer-readable storage medium, in which a computer program is stored; when the computer program is executed, the above-mentioned method for repairing damaged buildings or damaged building components of multiple styles can be implemented.

[0155] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0156] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a computer terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0157] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for repairing damaged building components of multiple styles, characterized in that include: Construct a benchmark image library consisting of several benchmark images of architectural components of a single style; Acquire several images of the same building component to be restored in the same multi-style fusion building; Preprocessing the image of the building component to be repaired to extract contour information of the building component in the image of the building component to be repaired; Calculating the similarity between the building component image to be repaired and the building component reference images in the reference image library, and matching the building component reference images whose similarity is higher than a similarity threshold; Comparing the contour information of the image of the building component to be repaired with the matched reference image of the building component to obtain the defect position of the image of the building component to be repaired; fusing and reconstructing a plurality of images of building components to be repaired with different defect positions to form a fused image of the building components; wherein the plurality of images of building components to be repaired with different defect positions are images of the same building component on different buildings, and when there are two images with the same defect position, obtaining the degree of defect of the image of the building component to be repaired, and when fusing and reconstructing the plurality of images of building components to be repaired with different defect positions, selecting the image of the building component to be repaired with a degree of defect less than a degree threshold; The fused images of the building components are manually reviewed, and the fused images that meet the requirements are added to the reference library to be used as a repair reference for repair.

2. A method for repairing damaged building components of multiple styles according to claim 1, characterized in that Also includes the steps: Preprocessing the input image of the building component to be repaired to extract contour information of the damaged building component in the image of the building component to be repaired; Comparing the extracted contour information with a reference image in the reference image library having the same main style as the damaged building component to obtain a defect position and degree of the image of the building component to be repaired; Matching the fused image of the multi-style building components in the reference library or the building component reference image in the reference image library according to the defect position and degree of the defect of the image of the building component to be repaired; Referring to the matched fusion image of multi-style building components in the reference library or the single-style building component reference image in the reference image library, the image of the building component to be repaired is repaired to obtain a building component repair image; The step of matching the fused image of the multi-style building components in the reference image library or the building component reference image in the reference image library according to the defect position and degree of the defect of the image of the building component to be repaired specifically includes: Acquiring a defect state of the image of the building component to be repaired by using the defect position and degree of the image of the building component to be repaired; Determining whether the defect state is within a preset first threshold interval, a second threshold interval, or a third threshold interval; When the defect state is within the first threshold interval, the style of the image of the building component to be repaired is judged to obtain the style of the image of the building component to be repaired, and the fused image in the reference image library or the reference image in the reference image library is matched according to the style of the image of the building component to be repaired; When the defect state is within the second threshold interval, matching the corresponding fused image in the reference image library; When the defect state is within the third threshold interval, a building component reference image in the reference image library having the same main style as the damaged building component is matched.

3. A method for repairing damaged building components of multiple styles according to claim 2, characterized in that: When the defect state is within the second threshold range, the step of matching the corresponding fused image in the reference library specifically includes: when multiple fused images are matched, calculating the similarity between the contour information of the building component to be repaired and the contour information of the corresponding area in the matched fused image, and selecting the fused image with the highest similarity.

4. The method for repairing damaged building components of multiple styles according to claim 1, characterized in that: The image of the building component to be repaired is a 2D image or / and a 3D image, and the building component reference image includes a 2D reference image and a 3D point cloud reference image; the 2D reference image of the building component reference image includes 2D reference images at several angles.

5. A method for repairing damaged building components of multiple styles according to claim 4, characterized in that: When the images of the building components to be restored of the multi-style fusion building are obtained by on-site acquisition, 2D images and 3D point cloud images of the building components at several angles are collected.

6. The method for repairing damaged building components of multiple styles according to claim 1, characterized in that: By manually marking the main style of the image of the building component to be repaired, a building component reference image that matches the main style of the image of the building component to be repaired is matched among reference images whose similarity is higher than a similarity threshold.

7. The method for repairing damaged building components of multiple styles according to claim 1, characterized in that: The age and region of the building components in the image of the building component to be repaired are manually annotated; when the defect of the fused image exceeds the defect threshold, the reference image of the building components of the same age, region and main style is matched for fusion and reconstruction, and the fused and reconstructed fused image is submitted to manual review, and the fused image that meets the requirements is added to the reference library.

8. A repair device for damaged building components of various styles, characterized in that include: A reference image library construction module is used to construct a reference image library consisting of several reference images of building components of a single style; The contour information extraction module is used to obtain a plurality of images of the same building component to be restored of the same multi-style fusion building; pre-process the image of the building component to be restored, and extract the contour information of the building component in the image of the building component to be restored; A similarity calculation module is used to calculate the similarity between the image of the building component to be repaired and the reference image of the building component in the reference image library, and match the reference image whose similarity is higher than the similarity threshold; A defect position comparison module is used to compare the contour information of the image of the building component to be repaired with the matched reference image of the building component to obtain the defect position of the image of the building component to be repaired; a fusion and reconstruction module, fusing and reconstructing a plurality of images of building components to be repaired with different defect positions to form a fused image of the building components; wherein the plurality of images of building components to be repaired with different defect positions are images of the same building components on different buildings, and when there are two images with the same defect position, obtaining the degree of defect of the image of the building component to be repaired, and when fusing and reconstructing the plurality of images of building components to be repaired with different defect positions, selecting the image of the building component to be repaired with a degree of defect less than a degree threshold; The reference library construction module manually verifies the fused images of the building components and adds the fused images that meet the requirements into the reference library.

9. The device for repairing damaged building components of various styles according to claim 8, characterized in that Also includes: The secondary reconstruction module is used to manually mark the age and region of the building components in the image of the building component to be repaired; when the defect of the fused image exceeds the defect threshold, the reference image of the same age, region and main style is matched for fusion and reconstruction, and the fused and reconstructed fused image is submitted for manual review, and the fused image that meets the requirements is added to the reference library.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed, the method for repairing damaged building components of multiple styles as described in any one of claims 1 to 7 can be implemented.

Images