Method for constructing digital drawing reference library for multi-style integrated architectural restoration

Abstract

This invention discloses a method for constructing a digital image reference library for multi-style architectural restoration. The method includes: constructing a reference image library composed of several reference images of architectural components in a single style; acquiring several images of the same architectural component, preprocessing them to extract contour information, selecting the image with the most complete contour information, calculating its similarity to the corresponding reference image in the reference image library, and matching reference images with similarity thresholds; comparing the contour information of the image of the architectural component to be restored with the reference images to obtain the defect location; fusing and reconstructing multiple images of architectural components with different defect locations to obtain a fused image; adding images that meet the requirements to the reference library after manual review; if the defect exceeds a defect threshold, matching and reconstructing reference images of the same era, region, and main style, and manually reviewing them again, adding fused images that meet the requirements to the reference library.

Description

[0001] Divisional application

[0002] This application is a divisional application of Chinese invention patent application filed on June 24, 2024 [Application No.: 2024108241777] [Invention Title: A Method and Apparatus for Repairing Damaged Buildings of Multiple Styles, and Storage Medium]. Technical Field

[0003] This invention belongs to the field of architectural digital image restoration technology, and in particular relates to a method for constructing a digital pattern reference library for multi-style fusion architectural restoration. Background Technology

[0004] Multi-style fusion architecture is an architectural form that combines elements of multiple different styles. This style typically integrates various architectural design elements to create a unique and distinctive building form. Multi-style fusion architecture can blend characteristics of different cultures, eras, and styles in its design, showcasing diversity and innovation.

[0005] For example, Shanghai-style architecture is a typical example of multi-style fusion architecture, generally referring to the Shikumen and Lilong architectural forms with local characteristics that emerged in Shanghai during the first half of the 20th century. These buildings represent a part of Shanghai's history and culture and have significant historical, cultural, and artistic value.

[0006] Many multi-style buildings, including Shanghai-style architecture, are currently facing certain challenges in their restoration and preservation, such as demolition and reconstruction, lack of maintenance and repair, planning conflicts, and damage from disasters. In order to better protect and inherit the cultural heritage of multi-style buildings and preserve their architectural features, database technology is being innovatively used to digitally preserve the decorative features of Shanghai-style buildings, facilitating subsequent restoration and repair processes.

[0007] In the actual use of the architectural digital image database, it was found that multi-style integrated buildings, due to their special characteristics, had a short period of popularity and a small geographical area, and very few have survived. Moreover, most of the surviving ancient buildings are in poor condition and lack repair, making it difficult to find complete references. This means that the current restoration work relies on the restorers' own knowledge of the architecture and their own cultural background, resulting in low restoration efficiency and inconsistent restoration quality.

[0008] Chinese patent application CN202010352962.9 discloses a multi-directional repair system and method for damaged ancient building columns. The multi-directional repair system for damaged ancient building columns includes an image acquisition module, an image analysis module, a central control module, a repair location determination module, a damage degree measurement module, a three-dimensional modeling module, a repair simulation module, a repair scheme generation module, a feasibility analysis module, and a display module. The multi-directional repair method for damaged ancient building columns includes acquiring and analyzing images of the ancient building columns; determining the repair location and its area and depth; establishing a three-dimensional model of the column and simulating the repair operation; generating a repair scheme and analyzing its feasibility.

[0009] The aforementioned existing technologies mention methods for repairing ancient buildings by collecting architectural components and determining the repair locations and extent of damage. However, for multi-style buildings that are already extremely rare, it is difficult to find reference objects or / and reference images for repair. Repairs must rely on the restorers' own understanding of the building, and the final result is highly dependent on the restorers' expertise, lacking objectivity.

[0010] Most of the existing ancient buildings that blend various styles are severely damaged, and it is almost impossible to find complete architectural components that can be used directly as a reference for restoration. Summary of the Invention

[0011] The purpose of this invention is to provide a method and apparatus for repairing multi-style damaged buildings, as well as a computer-readable storage medium, which partially solves or alleviates the above-mentioned deficiencies in the prior art. By using complete images of building components in multi-style buildings or reference images of building components in a single style as a reference, the invention repairs digital images of multi-style damaged buildings, providing a reference for building repair and thereby improving the efficiency and quality of repair.

[0012] To address the aforementioned technical problems, the present invention specifically adopts the following technical solution: a method for repairing damaged buildings of various styles, comprising:

[0013] The building image to be repaired is decomposed to obtain individual building component images of each building component in the building image to be repaired.

[0014] The images of each building component are preprocessed to extract the contour information of the building components from the images;

[0015] The outline information of the building component is compared with the reference image of the building component in a pre-built reference image library to obtain the location and degree of the defect in the building component image;

[0016] Based on the location and degree of damage of the building component image, match the complete image of the building component of the multi-style fusion building in the pre-built reference image library or the reference image of the building component in the reference image library;

[0017] By referencing the complete image of the building component of a multi-style fusion building in the reference image library or the reference image of the building component in the reference image library, the building component image is repaired to obtain the repaired image of the building component;

[0018] The method involves using building component repair images to restore the building image to obtain the restored building image.

[0019] As an improvement, the steps of matching complete images of architectural components from a multi-style fusion building reference library or reference images of the architectural component from a reference image library based on the location and extent of the missing parts in the architectural component image include:

[0020] The defect state of a building component image is obtained by using the location and degree of defect in the image.

[0021] Determine whether the defective state is within a preset first threshold interval, or a second threshold interval, or a third threshold interval;

[0022] When the defective state is within the first threshold range, the style of the building component image is determined to obtain the main style of the building component image, and the complete image of the building component of the multi-style fused building in the reference image library or the reference image of the building component in the reference image library is matched according to the style of the building component image.

[0023] When the missing state is within the second threshold range, match the complete image of the architectural components of multi-style fusion buildings in the reference image library;

[0024] If the defective state is within the third threshold range, match the reference image of the building component in the reference image library.

[0025] As an improvement, the steps for obtaining the defect state of building component images include:

[0026] Defect status = a * defect location score + b * defect severity score, where a and b are defect coefficients.

[0027] As an improvement, the degree of manual review is calculated based on the defect status of each building component image in the building image to be repaired. If the degree of manual review exceeds the manual review threshold, the restored building image is submitted to a human reviewer.

[0028] Without manual review, a reference image of a building component with a defective state within the third threshold range is replaced to perform secondary repair, resulting in a secondary repaired image of the building component. The repaired image of the building component is then replaced with the secondary repaired image of the building component, and the building image to be repaired is then restored to obtain a secondary restored building image. The secondary restored building image is then subject to a second manual review.

[0029] Without passing the second manual review, a reference image of the building component in the second threshold range is replaced for secondary repair; the repaired image of the building component is replaced with the secondary repaired image of the building component, and the building image to be repaired is restored three times to obtain the three-restored building image; the three-restored building image is then subject to three manual reviews.

[0030] Without passing three manual reviews, a secondary repair is performed by replacing the reference images of the building components in the second and third threshold ranges with images of the damaged components. The repaired images of the building components are then replaced with the repaired images of the building components, and the building images to be repaired are restored four times to obtain the restored building images. The restored building images are then subject to four manual reviews.

[0031] As an improvement, the steps for replacing the reference include:

[0032] When the original reference is a complete image of a building component of a multi-style integrated building in the reference image library, the reference image of that building component in the reference image library is used as a reference.

[0033] With the reference image of the building component in the original reference image library as the baseline image, the complete image of the building component of the multi-style integrated building in the reference image library is used as a reference.

[0034] As an improvement, in cases where the building image to be restored has not passed four rounds of manual review, images of buildings from the same era and region with the same main style are obtained as references for restoration.

[0035] The steps for calculating the degree of manual review based on the defect status of each building component image in the building image to be repaired include:

[0036] Architectural components are categorized according to their impact on the overall architectural style, and a manual review coefficient is assigned to the images of each category of architectural components.

[0037] Calculate the average number of defect states in the images of each type of building component, and adjust the average number by the manual review coefficient of the images of that type of building component to obtain the manual review degree of the images of a single type of building component.

[0038] The sum of the manual review scores of all single-category building component images is taken as the manual review score of the building image to be repaired.

[0039] The present invention also provides a repair device for damaged buildings of various styles, comprising:

[0040] The decomposition module is used to decompose the building image to be repaired and obtain individual building component images of each building component in the building image to be repaired.

[0041] The preprocessing module preprocesses the images of each building component and extracts the contour information of the building components from the images.

[0042] The defect acquisition module is used to compare the outline information of the building component with the reference image of the building component in the reference image library to obtain the defect location and degree of defect in the building component image;

[0043] The matching module is used to match complete images of architectural components from a multi-style fusion building in a reference image library or a reference image of the architectural component in a reference image library based on the location and degree of the missing parts in the architectural component image.

[0044] The building component image restoration module is used to restore the building component image by referring to the complete image of the building component of a multi-style fusion building in the reference image library or the reference image of the building component in the reference image library to obtain the restored image of the building component.

[0045] The building image restoration module uses building component restoration images to restore the building image to be restored, obtaining the restored building image.

[0046] As an improvement, the matching module specifically includes:

[0047] The defect state acquisition unit is used to acquire the defect state of the building component image by utilizing the defect location and degree of defect in the building component image;

[0048] The judgment unit is used to determine whether the defect state of the building component image is within a first threshold interval, or a second threshold interval, or a third threshold region.

[0049] The first reference image acquisition unit is used to determine the main style of the building component image when the judgment unit determines that the defect state of the building component image is within the first threshold range, obtain the style of the building component image, and match the complete image of the building component of the multi-style fused building in the reference image library or the reference image of the building component in the reference image library according to the style of the building component image.

[0050] The second reference image acquisition unit is used to match the complete image of the building component of a multi-style fusion building in the reference image library when the judgment unit determines that the defect state of the building component image is within the second threshold range.

[0051] The third reference image acquisition unit is used to match the reference image of the main style of the building component in the reference image library when the judgment unit determines that the defect state of the building component image is within the third threshold range.

[0052] The present invention also provides a storage medium storing a computer program; when the computer program is executed, the above-mentioned method for multi-style fusion architectural digital image restoration can be implemented.

[0053] The present invention also provides a computer system, including a memory and a processor; the memory stores a computer program; when the computer program is executed by the processor, the above-mentioned method for multi-style fusion architectural digital image restoration can be implemented.

[0054] The advantages of this invention are:

[0055] This invention pre-constructs complete images of architectural components incorporating multiple styles, as well as a single-style reference image library including the main style and external styles. After decomposing the digital image of the building to be repaired, each architectural component image is compared with the main style reference image to determine its extent of damage. Then, based on the extent of damage, a suitable image (complete image or reference image) is matched as a reference for repair.

[0056] After repairing all the images of the missing building components, the images are stitched together to form a restored building image. The degree of damage to the original building image is then used to determine whether manual review is necessary. Since each building contains hundreds or thousands of components, and the repaired images for some components may be based on complete images in multiple styles, while others may be based on a single-style reference image, manual review by experts is required to avoid stylistically incongruous or unreasonable final images. For example, if the building's damage is minor, even if the style of the damaged portion in the final image differs slightly from the original, the impact is minimal, and manual review may not be necessary. However, if the building's damage is significant, and the style of the damaged portion in the automatically repaired image differs from the original, it will severely affect the quality of the repair, thus requiring expert intervention for manual review.

[0057] For architectural images that do not meet the requirements after manual review, secondary, tertiary, quaternary, and even quinary repairs are carried out by adjusting the repair strategy of architectural components. This makes full use of existing reference resources to make the restored architectural images conform to the original style as much as possible, thereby improving the efficiency and quality of the repair.

[0058] In this invention, the repair of building components is carried out using complete images or reference images of the building components as a reference, eliminating the dependence on the repairer's own skills in manual repair. This greatly improves the consistency of repair quality.

[0059] When automatic repair fails to meet the requirements, manual review is then conducted, which ensures both the quality and efficiency of the repair while reducing the labor intensity of manual workers. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale. Obviously, the drawings described below are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0061] Figure 1 This is a flowchart of Embodiment 1 of the present invention;

[0062] Figure 2 This is a flowchart of Embodiment 2 of the present invention;

[0063] Figure 3A This is a functional block diagram of Embodiment 3 of the present invention;

[0064] Figure 3B This is a functional block diagram of Embodiment 4 of the present invention;

[0065] Figure 4 and Figure 5 Images of damaged buildings of various styles collected on-site. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0067] In this document, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" may be used interchangeably.

[0068] In this document, the terms "upper," "lower," "inner," "outer," "front," "rear," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the present invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0069] In this document, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0070] In this document, "and / or" includes any and all combinations of one or more of the listed related items.

[0071] In this article, "multiple" means two or more, that is, it includes two, three, four, five, etc.

[0072] In this article, "multi-style fusion architecture" refers to buildings that integrate multiple architectural styles. Generally, multi-style fusion architecture is formed by incorporating an existing main style with some external styles. In multi-style buildings, some architectural components may represent the main style, while others may represent external styles. It's also possible that a single architectural component represents the main style while incorporating some external styles. Furthermore, the fusion is highly random; the location and extent of the fusion depend on the individual preferences of the craftsmen, and there is no universally applicable rule. Consequently, if any architectural component is damaged, the entire building is damaged, hence the term "damaged multi-style building."

[0073] In this article, "reference image" refers to an image of a complete standard architectural component of a single style. For example, an image of a complete standard architectural component with only the dominant style (e.g., a style characteristic of local architecture), or an image of a complete standard architectural component with only an imported style (e.g., a foreign or expatriate architectural style). Typically, for architectural components of a single style, there is only one reference image for each style. Specifically, this image can be obtained from various architecture-related textbooks or existing materials.

[0074] In this article, "complete image" refers to a single image obtained by stitching (or fusing and reconstructing) images of the same architectural component with different missing locations, or multiple images of the same architectural component. This architectural component incorporates multiple styles; for example, it may be based on a primary style while incorporating at least one other style. The primary style refers to the architectural component in which most of its structure or outline adopts this style, while a small portion of its structure or outline adopts a different style.

[0075] Example 1: As Figure 1 As shown, this invention provides a method for constructing a digital drawing reference library for the restoration of buildings with multi-style integration, the specific steps of which include:

[0076] S11 constructs a reference image library consisting of reference images of architectural components in several single styles.

[0077] The architectural components described in this invention include basic elements that make up a building, such as doors, beams, walls, and columns. Buildings of a single style, such as the Huizhou style, have distinctive characteristics and a long period of popularity, resulting in a large number of existing buildings and images. Accordingly, a corresponding reference image library can be established for each style.

[0078] In this implementation, the purpose of constructing the benchmark image library is to serve as a basic structural reference for the later restoration of architectural component images with multiple styles. This will be explained in detail later and will not be repeated here.

[0079] Of course, to more closely resemble the image of the damaged building to be restored that incorporates multiple styles (or is called a multi-style fusion building), a single-style building similar to the multi-style fusion building can be selected. For example, some multi-style fusion buildings are primarily Hui-style architecture, incorporating other elements. Therefore, this step can select a reference image of the architectural components of a Hui-style building.

[0080] Furthermore, for ease of subsequent use, the reference image in this embodiment includes a 2D reference image and a 3D point cloud reference image; the 2D reference image includes 2D reference images from several angles. Since there are many existing physical objects and images of single-style buildings, it is also convenient to acquire 3D point clouds and multi-angle 2D images.

[0081] S12 acquires several images of the same building component to be repaired in a multi-style fusion building; preprocesses the images of the building component to be repaired, and extracts the outline information of the building component in the images of the building component to be repaired.

[0082] There are several ways to obtain architectural drawings of multi-style buildings awaiting restoration: First, the building itself may exist, but a certain architectural component may be missing. In this case, on-site data collection can be conducted. (See [link to relevant documentation]). Figure 4 and Figure 5 In acquiring images of architectural components to be repaired from multiple architectural styles through on-site data collection, 2D images and 3D point cloud images of the architectural components from several angles were also collected.

[0083] Secondly, if the building has been completely destroyed and disappeared, leaving only some pictures or videos, it is necessary to disassemble the various building components as much as possible based on the remaining data to help with subsequent integration and reconstruction.

[0084] This embodiment uses a building component, such as a column, as an example. In this step, multiple images of a multi-style architectural column can be obtained from the aforementioned channels (of course, the architectural components in these images are all incomplete). Naturally, the more images there are, the more information there is, and the greater the probability of successfully fusing and reconstructing a complete image.

[0085] After obtaining images of the building components to be repaired in a multi-style integrated building, these images may contain various background information and other information unrelated to the building components. Therefore, it is necessary to preprocess these images of the building components to be repaired, mainly by extracting their contour information to obtain the building component image of the damaged building component itself, for subsequent comparison.

[0086] In addition, the age and region of the building components in the images of the building components to be repaired can be manually marked.

[0087] S13 calculates the similarity between the image of the building component to be repaired and the corresponding reference image of the building component in the reference image library, and matches the reference image with a similarity higher than the similarity threshold.

[0088] Since there are no complete physical objects or images for reference when comparing images of the building components to be repaired, it is difficult to determine the specific location and extent of the damage. A reference image library actually provides a rough reference. For example, even if columns differ in style, their basic structure is similar, with only decorative patterns and sculptures differing. Therefore, comparing with a reference image can still provide a general understanding of the damage. Preferably, a similarity comparison is performed based on the aforementioned contour information. If multiple images have similarity values ​​exceeding a preset threshold, the one with the highest similarity is used as the reference image for comparison. Preferably, the image with the most complete contour information from several images of the building components to be repaired is selected and its similarity is calculated with the reference images of each building component.

[0089] In this implementation, since the reference image library includes reference images of various single-style architectural components, such as reference images of Hui-style columns, reference images of other single-style columns that differ significantly from the Hui-style, and reference images of columns of other styles similar to the Hui-style, the main style of the multi-style architectural component image to be repaired can be pre-annotated manually. Then, a reference image matching the main style of the architectural component image to be repaired is found among the reference images with a similarity higher than a similarity threshold; this is the main style reference image of the architectural component. Of course, in some embodiments, AI annotation can also be used, but its accuracy is lower than that of manual annotation.

[0090] For example, if the building component to be repaired is a fusion of Huizhou architectural style and other architectural elements, its main style can be manually labeled during preprocessing. Then, in this implementation, provided the similarity requirement is met, a reference image with the same main style can be selected, making the reference image more closely resemble the shape of the building component to be repaired.

[0091] In order to make the similarity calculation more accurate and find a reference image that is closer to the image of the building component to be repaired, the reference image in this implementation should be acquired as much as possible as 3D point cloud images and multi-angle 2D images.

[0092] Furthermore, calculating the similarity between images is a prior art technique and will not be elaborated upon in this invention. For example:

[0093] Commonly used methods for calculating similarity between 2D images include:

[0094] Perceptual Hashing: Maps images to fixed-length hash values ​​and evaluates the similarity between two images by comparing their hash values.

[0095] Structural Similarity Index (SSIM): This measure of similarity between two images is achieved by comparing their brightness, contrast, and structural information. It is typically used to evaluate image quality.

[0096] Histogram Similarity: Compares the color histograms of images and evaluates the similarity between them by calculating the distance between the histograms.

[0097] Common methods for calculating similarity between 3D images include:

[0098] 3D shape matching: Features are extracted from 3D shapes, and then the similarity between two 3D shapes is compared through feature matching.

[0099] 3D mesh alignment: Register or align 3D mesh models, and then evaluate the similarity between 3D models by comparing the differences between mesh coordinates or vertices.

[0100] 3D point cloud registration: Register two 3D point clouds and then evaluate their similarity by measuring the differences between the point cloud transformations.

[0101] S14 compares the contour information of the building component image to be repaired with the matched reference image of the building component to obtain the location of the defect in the building component to be repaired.

[0102] After selecting a suitable reference image, the comparison can be performed.

[0103] When the image of the building component to be repaired is a 3D point cloud, the defect of the building component to be repaired can be obtained by subtracting the 3D point cloud image of the building component to be repaired from the 3D point cloud reference image.

[0104] When the image of the building component to be repaired is a 2D image, the image of the building component to be repaired and the reference image are first segmented into instances. The segmented building components are then compared, and the missing parts are the defects of the building components.

[0105] S15 fuses and reconstructs multiple images of building components with different missing locations to form a fused image of the building component.

[0106] The difficulty in the existing technology is that there are no complete images of building components available for reference. The problem that this invention aims to solve is to combine existing images of a building component with defects in order to construct a complete image of that building component (i.e., a fused image).

[0107] Since the missing locations of each image were identified in the previous step, theoretically, a complete image of the architectural component can be obtained by stitching together multiple images with different missing locations. For example, image A is missing the base of a column, and image B is missing part of the relief on the column; therefore, simply stitching the two together will yield a complete image of the column. Images A and B can be images of the same architectural component on different buildings, or images of the same architectural component on the same building. They can also be images of the same architectural component taken at different times, showing different missing locations and / or degrees of damage.

[0108] Of course, since there are multiple images, it is possible that two images have the same missing location. Therefore, when fusing and reconstructing, the one with the smaller degree of missing is selected. That is, the missing degree of the building component image to be repaired is obtained. When fusing and reconstructing multiple images of building components to be repaired with different missing locations, the one with the missing degree less than the degree threshold is selected.

[0109] S16 manually reviews the fused images of building components and adds fused images that meet the requirements to the reference library.

[0110] In the previous step, multiple images of the building components to be repaired were merged and reconstructed to obtain a fused image. Although images of the same building components from the same multi-style fusion architecture were used, the elements of multi-style fusion architecture are quite complex, and even within the same style, there will be differences between different buildings. Therefore, in this step, it is also necessary to review the fused image to determine whether it conforms to the style.

[0111] Determining the style is actually a rather subjective judgment, which is difficult to accomplish by AI. Therefore, in this step, it is recommended to use human review, so that experts can judge whether the image meets the requirements based on their in-depth understanding of architectural styles.

[0112] Images that meet the requirements can be added to the library for future reference when restoring electronic images of the multi-style fusion building or the building itself.

[0113] S17 If the defect in the fused image exceeds the defect threshold, it matches a reference image of the same era, region and main style for fusion reconstruction, and submits the fused image after reconstruction to manual review. The fused image that meets the requirements is added to the reference image library.

[0114] It is possible that even if multiple images to be repaired are merged, they cannot be pieced together into a complete architectural component image, thus requiring further fusion and reconstruction.

[0115] Step S12 marks the era and region of the building components to be repaired. If the defect in the fused image exceeds a defect threshold, such as 5%, a reference image of the same era, region, and main style is found from the reference image library for fusion reconstruction. Of course, the fused image reconstructed by this method is more likely to be unacceptable, so it also needs to be manually reviewed and approved before being added to the reference image library.

[0116] Example 2: So-called multi-style fusion architecture refers to buildings that integrate multiple architectural styles. Generally, multi-style fusion architecture is formed by combining an existing main style with some external styles. In multi-style buildings, some architectural components may be of the main style, while others may be of a different style, i.e., single-style components. Alternatively, a single architectural component may be a combination of the main style and some external styles, forming a multi-style component. Due to the high degree of randomness in multi-style fusion, the location and extent of the fusion are determined by the individual preferences of the craftsmen, and there is no universal rule, making the restoration of multi-style fusion buildings after damage is quite difficult.

[0117] In response to the above technical problems, such as Figure 2 As shown, this invention provides a method for repairing damaged buildings of multiple styles (i.e., damaged buildings with a blend of multiple styles). This method repairs digital images of damaged buildings of multiple styles based on the digital image reference library constructed in Example 1, and also uses the baseline image library constructed using the method provided in Example 1. The specific steps of this invention include:

[0118] S1 decomposes the building image to be repaired and obtains individual building component images of each building component in the building image to be repaired.

[0119] In some embodiments, after obtaining an image of a multi-style fusion building to be repaired, it is necessary to decompose it into images of building components.

[0120] There are several ways to obtain images of multi-style architectural structures awaiting restoration: First, the building itself may remain, but some structural components may be missing. In this case, on-site data collection is possible. When obtaining images of the structural components of a multi-style architectural structure awaiting restoration through on-site data collection, 2D images and 3D point cloud images of the structural components from several angles should be acquired.

[0121] Secondly, if the building has been completely destroyed and only some pictures or videos remain, it is necessary to decompose the individual building components as much as possible based on the remaining data. The decomposition method can use an artificial intelligence model for instance segmentation, separating each building component into individual building component images. Instance segmentation is existing technology and will not be elaborated upon in this invention.

[0122] In addition, the era and region of buildings with multiple styles in the architectural images to be restored can be manually marked.

[0123] S2 preprocesses the images of each building component to extract the contour information of the building components from the images.

[0124] Similar to Example 1, the purpose of obtaining the outline information of the building components in the building component image is to obtain the damage status of the building components. For the specific method, please refer to Example 1, and it will not be repeated in this example.

[0125] S3 compares the outline information of the building component with the main style reference image of the building component in the reference image library to obtain the location and degree of damage of the building component image.

[0126] Understandably, the style of the reference image of the building component should match the main style of the multi-style fusion building to be repaired. This allows for the most accurate reconstruction of the missing parts in the building component image. Therefore, given the main style of the building, a reference image with the same main style as the building (i.e., the reference image of the building component's main style) can be matched from the reference image library, and then compared to determine the location and extent of the damage. Since there is only one reference image for each style of building component in the reference image library, once the name of the damaged building component and the corresponding main style of the building are known (e.g., manually labeled or entered), the system can match a reference image with the same main style as the building in the reference image library.

[0127] When the building component image is a 3D point cloud, the defect of the building component to be repaired can be obtained by directly subtracting the 3D point cloud image of the building component from the 3D point cloud reference image.

[0128] When the building component image is a 2D image, the building component image and the reference image are first segmented into instances. The segmented building components are then compared, and the missing parts are the defects of the building components.

[0129] For a building that blends multiple styles, its architectural components may be complete or incomplete and damaged. This step can determine whether the architectural component image is damaged and the extent of the damage. Of course, for complete architectural component images, the next step of architectural component image restoration work is unnecessary. Alternatively, the complete architectural component image can be added to the image library as a reference image during restoration.

[0130] S4 matches the complete image of the architectural component from the reference image library or the reference image of the architectural component from the reference image library based on the location and degree of the missing part.

[0131] Because the fusion of styles in multi-style architecture is quite complex, when it is possible to determine whether a building component is in the primary style, a primary style fused with other styles, or purely an other style, the style of the building component should be determined first. Then, based on the style of the building component, the appropriate method for repair is determined: whether to use the complete image (multi-style fusion) or the baseline image (single style, including both primary and other styles) for repair.

[0132] Furthermore, when the damage to a building component exceeds a certain level, it becomes difficult to determine which of the aforementioned styles it belongs to. Therefore, this embodiment also includes the following steps:

[0133] S41 uses the location and degree of damage of the building component image to obtain the damage state of the building component image.

[0134] Specifically, the steps for obtaining the defect state of building component images include:

[0135] The defect status is calculated as follows: a * defect location score + b * defect severity score, where a and b are defect coefficients. For example, using a 100-point scale, a defect at a corner receives a score of 5; a defect at an edge receives 10; a defect in the center receives 15. The score is then multiplied by the number of defects at each location, and so on. A defect severity score is 50 for a defect area less than 5%; 70 for a defect area between 5% and 10%; and 100 for a defect area greater than 10%, and so on. The specific values ​​of the defect coefficients can be adjusted according to the actual situation; for example, a could be 35% and b could be 75%.

[0136] The lower the score of the missing state, the higher the integrity of the building component.

[0137] S42, when the defective state is within the first threshold range, determines the style of the building component image, obtains the style of the building component image, and matches the complete image of the building component of the multi-style fused building in the reference image library or the corresponding reference image of the building component in the reference image library according to the style of the building component image.

[0138] When the defective state falls within the first threshold range, it indicates that the style of the architectural component's defect can be determined. Specifically, this is done by manually labeling the style type beforehand (e.g., labeling the image with the corresponding style for a single-style image, and labeling the multiple styles for a multi-style fusion). If the image is determined to be a single style, a base image of that single style (i.e., the same style) is matched; if the image is determined to be a multi-style fusion, a complete image of the corresponding multi-style fusion is matched.

[0139] S43 matches complete images of architectural components of multi-style fusion buildings in the reference image library when the defective state is within the second threshold range.

[0140] When the missing state falls within the second threshold range, it indicates that the style of the architectural component cannot be determined. In this case, the probability of meeting the requirements after repair using a complete image fused from multiple styles is higher. Therefore, the corresponding complete image fused from multiple styles is matched. Specifically, since there are various ways to fuse multiple styles, even if multiple styles are manually labeled beforehand, multiple complete images may still be matched. Therefore, the complete image with the highest similarity is matched from the multiple matched complete images. For example, the similarity between the contour information and the contour information of the corresponding region in the complete image is calculated, and the complete image with the highest similarity is matched.

[0141] S44 matches the main style reference image of the building component in the reference image library when the defective state is within the third threshold range.

[0142] When the defect is within the third threshold range, it means that the architectural component is no longer able to determine the degree of style defect and is already very serious. In this case, using the baseline image of the main style for repair has the highest probability of success.

[0143] It is understood that in this embodiment, the first threshold interval < the second threshold interval < the third threshold interval.

[0144] The S5 reference matching uses a reference library containing complete images of architectural components from multiple styles of buildings or a reference image of the architectural component from a reference image library to repair the architectural component image and obtain a repaired architectural component image.

[0145] With a reference image available, the building components can be repaired according to the reference image. During repair, the positions will be filled in based on the reference image.

[0146] S6 uses building component repair images to restore the building image to be repaired and obtain the restored building image.

[0147] After obtaining the repair images of all the building components (of course, the original complete building component images can be used directly), all the building component images are assembled to obtain the final restored building image.

[0148] In reality, it's impossible to know whether the restored architectural image will meet the requirements; the restored image might even have a jarring style that doesn't meet the specifications. Therefore, after obtaining the restored architectural image, further processing is necessary.

[0149] S7 calculates the degree of manual review based on the defect status of each building component image in the building image to be repaired. If the degree of manual review exceeds the manual review threshold, the restored building image will be submitted to a human reviewer.

[0150] The "damaged state" represents the condition of missing building components. If the degree of damage to the entire building is relatively low, there is confidence in the repair process described above. Therefore, it can be considered that the restoration has been successful and no further steps need to be taken.

[0151] However, when the damage to the entire building exceeds a certain threshold, it is necessary to conduct a manual review, with experts analyzing whether the repair work is up to standard.

[0152] In this step, the method for calculating the degree of manual review based on the defect status of each building component image in the building image to be repaired includes:

[0153] S71 categorizes architectural components into several classes based on their impact on the overall architectural style, and sets a manual review coefficient for the images of each class of architectural components.

[0154] For example, external, more conspicuous building components are grouped into one category, less conspicuous ones into another, and internal, less conspicuous components into yet another. The first category is assigned a higher weight; the second category has a lower weight, and the third category has the lowest weight. The manual review coefficient is then adaptively set based on the weight of each category.

[0155] S72 calculates the average number of defect states in the images of each type of building component, and adjusts the average number by the manual review coefficient of the images of that type of building component to obtain the manual review degree of the images of a single type of building component.

[0156] For example, there are 5 types of building components in the first category. After obtaining the average number of defective building components of this type, multiplying it by the artificial approval coefficient will give the artificial approval degree of this type of building component.

[0157] S73 takes the sum of the manual review scores of all single-class building component images as the manual review score of the building image to be repaired.

[0158] Continuing with the previous example, by summing the human review rates of the three types of building components, we can obtain the human review rate of the entire building.

[0159] Finally, by comparing the degree of manual review with the manual review threshold, it can be determined whether the restored architectural image still needs to be manually reviewed.

[0160] S8 replaces the reference image of the building component with the one in the third threshold range for secondary repair without passing manual review, and obtains a secondary repair image of the building component; replaces the repair image of the building component with the secondary repair image of the building component, and then performs secondary restoration on the building image to be repaired to obtain a secondary restored building image; and performs secondary manual review on the secondary restored building image.

[0161] After manual review by experts, it was determined that the restored architectural image had an jarring style and did not meet the requirements, indicating a problem with the restored architectural component images. At this point, an adjustment to the restoration strategy was necessary.

[0162] The most likely source of problems is the image of the building component whose defect state falls within the third threshold range. In the original strategy, this image was referenced using a baseline image of the main style, but this did not meet the requirements. Therefore, in this step, a complete image fused from multiple styles is used for repair, resulting in a secondary repaired image of the building component.

[0163] Images of building components with defects within the first and second threshold ranges are repaired unchanged according to the original strategy.

[0164] The images are then pieced together again to obtain a second, reconstructed architectural image. This second, reconstructed image then needs to be manually reviewed to determine its suitability.

[0165] S9 If the secondary manual review is not passed, the reference image of the building component in the second threshold range is replaced for secondary repair; the repair image of the building component is replaced by the secondary repair image of the building component, and the building image to be repaired is restored three times to obtain the three-restored building image; the three-restored building image is subject to three manual reviews.

[0166] After manual review by experts, it was determined that the architectural image after secondary restoration still had an abrupt style and did not meet the requirements. This may indicate that there was a problem with the repair of the architectural components in the second threshold range that were in a defective state.

[0167] In the original strategy, the images of the building components with defects within the second threshold range were referenced using a complete image fused from multiple styles, but this did not meet the requirements. Therefore, in this step, a reference image of the main style is used for repair, forming a secondary repaired image of the building component.

[0168] Images of building components with defects within the first and third threshold intervals are repaired unchanged according to the original strategy.

[0169] Then, the images are pieced together and restored three times to obtain the final restored architectural image. These three restored images then require further manual review to determine their suitability.

[0170] S10 If the three manual reviews are not passed, the reference images of the building components in the second and third threshold ranges of the defect state are replaced for secondary repair. The repaired image of the building component is replaced by the secondary repaired image of the building component. The building image to be repaired is restored four times to obtain the four-restored building image. The four-restored building image is then subject to four manual reviews.

[0171] After manual review by experts, it was determined that the architectural images after three restorations still had an abrupt style and did not meet the requirements. This may be because the repair of architectural components in the second and third threshold ranges of the defective state had problems.

[0172] In the original strategy, the images of building components with defects within the third threshold range were referenced using a base image of the main style; the images of building components with defects within the second threshold range were referenced using a complete image fused from multiple styles, but this did not meet the requirements. Therefore, in this step, the images of building components with defects within the second threshold range are repaired using the base image of the main style, forming a secondary repaired image of the building component. The images of building components with defects within the third threshold range are repaired using a complete image fused from multiple styles, forming a secondary repaired image of the building component.

[0173] Images of building components with defects within the first threshold range are repaired unchanged according to the original strategy.

[0174] Then, the images are pieced together and restored four times to obtain the final architectural image. These four restored images then undergo manual review to determine their suitability.

[0175] S11, without passing four rounds of manual review, uses architectural images of the same era, region, and main style as references for restoration based on the main style of the architectural image to be restored.

[0176] If all four restoration attempts fail to pass manual review, the only recourse is to obtain architectural images from the same era and region with the same main style as the building to be restored, and use them as a reference for restoration.

[0177] It is foreseeable that the results of this restoration will still need to be reviewed manually. If the manual review fails again, it means that automatic restoration is hopeless, and the restoration work will have to be taken over entirely by humans.

[0178] Example 3: As Figure 3A As shown, the present invention also provides a repair device for damaged buildings of various styles, comprising:

[0179] The decomposition module is used to decompose the building image to be repaired and obtain individual building component images of each building component in the building image to be repaired.

[0180] The preprocessing module preprocesses the images of each building component and extracts the contour information of the building components from the images.

[0181] The defect acquisition module is used to compare the outline information of the building component with the reference image of the building component in the reference image library to obtain the defect location and degree of defect in the building component image;

[0182] The matching module is used to match complete images of architectural components from a multi-style fusion building in a reference image library or a reference image of the architectural component in a reference image library based on the location and degree of the missing parts in the architectural component image.

[0183] The building component image restoration module is used to restore the building component image by referring to the complete image of the building component in the multi-style fusion building in the matched reference image library or the reference image of the building component in the reference image library to obtain the restored image of the building component.

[0184] The building image restoration module uses building component restoration images to restore the building image to be restored, obtaining the restored building image.

[0185] In some embodiments, the matching module specifically includes:

[0186] The defect state acquisition unit is used to acquire the defect state of the building component image by utilizing the defect location and degree of defect in the building component image;

[0187] The judgment unit is used to determine whether the defect state of the building component image is within a first threshold interval, or a second threshold interval, or a third threshold region.

[0188] The first reference image acquisition unit is used to determine the main style of the building component image when the judgment unit determines that the defect state of the building component image is within the first threshold range, obtain the style of the building component image, and match the complete image of the building component of the multi-style fused building in the reference image library or the reference image of the building component in the reference image library according to the style of the building component image.

[0189] The second reference image acquisition unit is used to match the complete image of the building component of a multi-style fusion building in the reference image library when the judgment unit determines that the defect state of the building component image is within the second threshold range.

[0190] The third reference image acquisition unit is used to match the reference image of the main style of the building component in the reference image library when the judgment unit determines that the defect state of the building component image is within the third threshold range.

[0191] Example 4: Figure 3B As shown, the present invention also provides a repair device for damaged building components of multiple styles, which provides repair reference sample images for building component repair by constructing a digital pattern reference library for building repair that integrates multiple styles. Specifically, it includes:

[0192] The benchmark image library construction module is used to build a benchmark image library consisting of benchmark images of architectural components in several single styles.

[0193] The contour information extraction module acquires several images of the same building component to be repaired in a multi-style fusion building; it preprocesses the images of the building component to be repaired and extracts the contour information of the building component from the images of the building component to be repaired.

[0194] The similarity calculation module is used to calculate the similarity between the image of the building component to be repaired and the reference image of the building component in the reference image library, and to match the reference image with a similarity higher than the similarity threshold.

[0195] The defect location comparison module is used to compare the contour information of the building component image to be repaired with the reference image of the building component to obtain the defect location of the building component image to be repaired.

[0196] The fusion and reconstruction module fuses and reconstructs multiple images of building components with different missing locations to form a fused image of the building component.

[0197] The reference image library construction module allows for manual review of the fused images of building components, adding fused images that meet the requirements to the reference image library.

[0198] In some embodiments, the device further includes: a secondary reconstruction module, used to manually annotate the age and region of the building components in the building component image to be repaired; when the defect of the fused image exceeds the defect threshold, it matches a reference image of the same age, region and main style for fusion reconstruction, and submits the fused image after fusion reconstruction to manual review, and adds the fused image that meets the requirements to the reference image library.

[0199] Example 5: The present invention also provides a computer-readable storage medium storing a computer program; when the computer program is executed, the above-mentioned method for repairing damaged buildings or building components of various styles can be implemented.

[0200] Example 6: The present invention also provides a computer system, including a memory and a processor; the memory stores a computer program; when the computer program is executed by the processor, the above-mentioned method for repairing damaged buildings or building components of various styles can be implemented.

[0201] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0202] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a computer terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0203] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for constructing a digital drawing reference library for the restoration of buildings with multi-style integration, characterized in that, Including the following steps: S11 constructs a reference image library consisting of reference images of architectural components in several single styles; S12 acquires several images of the same architectural component to be repaired in a multi-style fusion building; The images of the building components to be repaired are preprocessed to extract the contour information of the building components in the images of the building components to be repaired; S13 selects the image with the most complete outline information from several images of the building component, calculates its similarity with the corresponding reference image of the building component in the reference image library, and matches the reference image with a similarity higher than the similarity threshold. S14 compares the contour information of the building component image to be repaired with the matched reference image of the building component to obtain the location of the defect in the building component to be repaired. S15 fuses and reconstructs multiple images of building components with different missing locations to form a fused image of the building component; S16 manually reviews the fused images of building components and adds fused images that meet the requirements to the reference image library; S17 If the defect in the fused image exceeds the defect threshold, it matches a reference image of the same era, region and main style for fusion reconstruction, and submits the reconstructed fused image to manual review. The fused image that meets the requirements is added to the reference image library.

2. The method for constructing a digital drawing reference library for multi-style architectural restoration according to claim 1, characterized in that, The image of the building component to be repaired is a 2D image and / or a 3D image. The reference image of the building component includes a 2D reference image and a 3D point cloud reference image. The 2D reference image of the building component includes 2D reference images at several angles.

3. The method for constructing a digital drawing reference library for multi-style architectural restoration according to claim 2, characterized in that, When acquiring images of architectural components to be repaired from multiple styles of buildings through on-site collection, 2D images and 3D point cloud images of the architectural components from several angles are collected.

4. The method for constructing a digital drawing reference library for multi-style architectural restoration according to claim 2, characterized in that, The degree of damage to the building component image to be repaired is obtained. When multiple images of building components with different damage locations are fused and reconstructed, the images of building components with a degree of damage less than the degree threshold are selected.

5. The method for constructing a digital drawing reference library for multi-style architectural restoration according to claim 2, characterized in that, In step S14, when the image of the building component to be repaired is a 3D point cloud, the defect of the building component to be repaired can be obtained by subtracting the 3D point cloud image of the building component to be repaired from the 3D point cloud reference image.

6. The method for constructing a digital drawing reference library for multi-style architectural restoration according to claim 2, characterized in that, In step S14, when the image of the building component to be repaired is a 2D image, the image of the building component to be repaired and the reference image are first segmented into instances, and the segmented building components are compared. The missing part is the defect of the building component.

7. A method for constructing a digital drawing reference library for multi-style architectural restoration according to claim 2, characterized in that, In step S15, if two images have the same missing location, select the image of the building component to be repaired whose missing degree is less than the degree threshold.

8. A method for constructing a digital drawing reference library for multi-style architectural restoration according to any one of claims 1 to 7, characterized in that, The building components include doors, beams, walls, and columns.

9. A method for constructing a digital drawing reference library for multi-style integrated architectural restoration according to any one of claims 1 to 7, characterized in that, The single style mentioned includes the Huizhou style.

10. A method for constructing a digital drawing reference library for multi-style architectural restoration according to any one of claims 1 to 7, characterized in that, It also includes the step of manually marking the age and region of the building components in the image of the building components to be restored.

Images