A method and system for detecting surface defects of buildings

Abstract

The present invention belongs to the field of wall surface monitoring, and provides a method and system for detecting surface defects of buildings, including: using 3D modeling software to create a virtual model of a building to be detected; generating diversified synthetic image data; extracting, matching and aligning features of the synthetic image data and real-shot image data; normalizing the aligned features; multimodally fusing the normalized synthetic image features with the real-shot image features to form a unified feature representation; training a building surface defect detection model based on the unified feature representation; using the building surface defect detection model to perform real-time detection on newly collected images, identifying defects on the building surface and marking the defect location and severity. The above scheme can more accurately detect surface defects of buildings.

Description

Technical Field

[0001] The present invention belongs to the field of wall surface monitoring, and in particular relates to a method and system for detecting surface defects of a building. Background Art

[0002] Building surface defect inspection refers to the inspection of the external surface of a building to identify and evaluate defects such as cracks, corrosion, peeling, discoloration, etc. that may affect the structural safety and appearance of the building. This type of inspection is of great significance to ensure the long-term safety and service life of the building, especially after the building has been used for many years or is in harsh environmental conditions, and regular inspection and maintenance are required.

[0003] Traditional building surface defect detection usually relies on manual inspection, which is not only time-consuming and labor-intensive, but also easily affected by human subjective factors. With the development of computer vision and artificial intelligence technology, more and more research and applications have begun to try to use image processing technology for automated building surface defect detection. By taking images of the building surface and using image analysis technology, various defects on the building surface can be automatically identified and located.

[0004] However, using images to detect surface defects in buildings faces some technical challenges, especially the impact of different lighting and weather conditions. In practical applications, the lighting conditions outside the building will change with time, season and weather, such as direct sunlight, shadow coverage, cloudy or rainy days. Changes in light intensity and direction may cause significant changes in the color, brightness and contrast of the building surface in the image, thus affecting the accuracy and reliability of defect detection. In addition, rainy and snowy weather may cause water stains or snow accumulation in the image, further interfering with the defect detection results. Summary of the invention

[0005] In order to solve the problems in the prior art, the present invention provides a method for detecting surface defects of buildings, the method comprising the following steps: using 3D modeling software to create a virtual model of a building to be detected, the virtual model reflecting the structure, material and surface features of the building; based on the virtual model, generating diversified synthetic image data under different lighting conditions, weather conditions and viewing angles through a 3D rendering engine; performing feature extraction on the synthetic image data and real-shot image data to obtain high-level features and local features of the image; matching and aligning the extracted synthetic image features with the real-shot image features; normalizing the aligned features, the normalization processing including color normalization, scale normalization, and contrast and brightness adjustment to reduce the impact of lighting, weather and viewing angle changes on the features; performing multimodal fusion of the normalized synthetic image features and the real-shot image features to form a unified feature representation; based on the unified feature representation, training a building surface defect detection model; using the building surface defect detection model to perform real-time detection on newly acquired images, identify defects on the surface of the building and mark the defect locations and severity.

[0006] Another aspect of the present invention provides a building surface defect detection system, the system comprising the following modules:

[0007] A modeling module, used to create a virtual model of the building to be inspected using 3D modeling software, wherein the virtual model reflects the structure, material and surface characteristics of the building;

[0008] A simulation module, for generating diversified synthetic image data under different lighting conditions, weather conditions and viewing angles through a 3D rendering engine based on the virtual model;

[0009] A matching module is used to extract features from the synthetic image data and the real-shot image data to obtain high-level features and local features of the image; and to match and align the extracted synthetic image features with the real-shot image features;

[0010] A normalization module is used to normalize the aligned features, wherein the normalization includes color normalization, scale normalization, and contrast and brightness adjustment to reduce the impact of illumination, weather, and viewing angle changes on the features;

[0011] The fusion module is used to perform multimodal fusion of the normalized synthetic image features and the real-shot image features to form a unified feature representation;

[0012] A training module, used for training a building surface defect detection model based on the unified feature representation;

[0013] The detection module is used to use the building surface defect detection model to perform real-time detection on newly collected images, identify defects on the building surface and mark the defect location and severity.

[0014] The present invention can produce the following beneficial effects through the above technical solution:

[0015] By using 3D modeling and multimodal feature fusion technology, the data advantages of synthetic images and real-life images are combined to generate a unified feature representation, which greatly improves the accuracy of building surface defect detection. The system can accurately identify and classify various types of defects, such as cracks, peeling, corrosion, etc., and quantify their severity to ensure the reliability of the detection results. The application of normalization processing and multimodal fusion enables the system to effectively cope with image data under different lighting, weather and viewing angle conditions, thereby maintaining consistent detection performance in various complex environments. Whether in adverse conditions such as strong light, shadows, rain and snow, the system can operate stably and provide high-quality detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0018] Below, the invention is preferably described in conjunction with the accompanying drawings and specific implementation methods.

[0019] This embodiment solves the above problem through the following steps:

[0020] In one embodiment, reference Figure 1 The present invention provides a method for detecting surface defects of buildings, which aims to identify and evaluate various problems on the surface of buildings that may affect their structural integrity or appearance quality, such as cracks, peeling, corrosion, discoloration, etc. Through such detection, potential safety hazards or aesthetic problems can be discovered in time, so that necessary maintenance or repair measures can be taken to extend the service life of the building and ensure its safety and aesthetics during use.

[0021] Step 1: Use 3D modeling software to create a virtual model of the building to be inspected, wherein the virtual model reflects the structure, material and surface features of the building.

[0022] In this step, a three-dimensional digital model corresponding to the actual building is constructed by using computer software. This process involves creating a virtual entity in the computer that accurately reflects the shape, size, and structural features of the building. 3D modeling software is a professional computer application used to generate and manipulate objects and scenes in three-dimensional space. Through this software, various parts of the building, such as walls, roofs, windows, doors, etc., can be modeled according to actual proportions and displayed in virtual space. Common 3D modeling software includes AutoCAD, Revit, Blender, SketchUp, etc. These software provide a variety of tools and functions that allow users to build various parts of the building through a graphical interface.

[0023] A virtual model is not just a simple copy of the building's appearance, but a detailed three-dimensional structure that can present the building's overall layout and spatial structure. A virtual model usually contains the building's geometric shape, size, and proportions, and is saved in digital form, so that it can be rotated, enlarged, reduced, cut, etc. on the computer, facilitating detailed analysis and testing.

[0024] The overall structure of a building expresses its load-bearing system, spatial layout, and the connections between its various parts. This includes the configuration of key structural components such as walls, beams, columns, floors, ceilings, and roofs. The virtual model presents these structures in three-dimensional space through accurate geometric modeling, reflecting the internal mechanical structure and spatial design of the building.

[0025] The virtual model not only shows the structure of the building, but also includes the properties of its surface materials. Materials usually refer to the physical materials covering the surface of the building, such as concrete, bricks, metal, glass, wood, etc. The texture, color, reflectivity, transparency and other physical properties of these materials can be simulated and rendered through 3D modeling software.

[0026] The surface features of the virtual model include specific details of the building's exterior surface, such as wall textures, surface unevenness, cracks, corrosion marks, stains, etc. These features can be reproduced through high-resolution texture mapping or direct modeling, making the model highly realistic and detailed.

[0027] Step S2, based on the virtual model, generating diversified synthetic image data under different lighting conditions, weather conditions and viewing angles through a 3D rendering engine.

[0028] In this step, the virtual model is used as a basis to simulate the appearance of the building under various environmental conditions through advanced 3D rendering technology, generating a series of image data with diverse characteristics. These image data can not only accurately reflect the appearance changes of the building at different times, weather conditions and observation angles, but can also be used to train and test the building surface defect detection algorithm, so that it can remain efficient and reliable in various complex practical application scenarios.

[0029] A 3D rendering engine is a computer software used to generate high-quality images. It can convert a 3D model into a two-dimensional image, simulate the interaction between light and the surface of an object, and the visual effects of the object under different environmental conditions. Common 3D rendering engines include Blender Cycles, Autodesk Arnold, Unreal Engine, etc. These engines use complex physical calculations and image processing technology to generate realistic images and animations.

[0030] During the rendering process, you can set different parameters to simulate the performance of buildings under various environmental conditions. These parameters include the type, position, and intensity of light sources, weather conditions such as sunny, cloudy, rainy, snowy, etc., and changes in viewing angles.

[0031] By adjusting the position and angle of the light source, the lighting conditions at different times of the day can be simulated, for example, oblique light in the morning, direct light at noon, side backlight at dusk, etc. Each lighting condition will affect the brightness, shadow and reflection effects on the surface of the building, and these changes have an important impact on the detection of surface defects.

[0032] In addition to natural light, different types of artificial light sources can also be simulated, such as street lights, lighting equipment in buildings, etc. These light sources are particularly important at night or in dim environments, and can reveal different apparent characteristics.

[0033] The 3D rendering engine is used to simulate the appearance of buildings under various weather conditions. For example, bright light on sunny days, soft light on cloudy days, wet effects and water droplet reflections on rainy days, and snow cover and light scattering on snowy days. These scenes can realistically reproduce the appearance of buildings under different climatic conditions.

[0034] In addition to static weather conditions, the rendering engine can also simulate dynamic weather effects, such as moving clouds, falling raindrops, wind and snow, etc. These dynamic effects make the generated image data closer to real scenes, providing richer training materials for the detection model.

[0035] By adjusting the position and angle of the virtual camera, images of the building taken from different perspectives are generated. This includes looking straight, looking up, looking down, and observing the surface of the building from different sides and angles. Multi-perspective data can help the detection algorithm understand the appearance characteristics of the building more comprehensively, especially for corners and hidden parts that are difficult to observe.

[0036] The rendering engine can also simulate the process of moving the camera to generate a series of images or videos taken continuously from different angles. These dynamic perspective data help improve the stability and accuracy of defect detection, especially in complex three-dimensional structures.

[0037] By combining different lighting conditions, weather conditions, and perspective changes, we can generate diverse synthetic image data. These images not only cover the appearance of buildings under various environmental conditions, but also provide rich visual features to help detection algorithms better adapt to the changing situations in practical applications.

[0038] Step S3, extracting features from the synthetic image data and the real-shot image data to obtain high-level features and local features of the image; matching and aligning the extracted synthetic image features with the real-shot image features.

[0039] In this step, advanced image processing and computer vision techniques are used to extract key information that helps detect building surface defects from virtually generated synthetic images and actual images. Next, the extracted features are matched and aligned to ensure that the data can be compared and processed consistently and accurately in subsequent analysis.

[0040] High-level features refer to abstract features extracted from images through deep learning algorithms or other advanced computer vision techniques. These features can describe global information such as the overall shape, structure, and texture of objects in the image. Usually, the extraction of high-level features relies on deep learning models such as convolutional neural networks (CNNs), which gradually extract more abstract and semantically meaningful representations from the original image through multi-layer convolution and pooling operations.

[0041] In the feature extraction process, pre-trained convolutional neural networks are used to process synthetic and real images. These networks can capture macroscopic features of the building surface, such as the overall shape of the wall, the layout of the windows, the structure of the roof, etc. Through multiple layers of convolution operations, high-level feature vectors reflecting the image content are gradually extracted. These features can be used to describe and classify the apparent characteristics of the building.

[0042] Local features refer to information in an image that reflects local details of an object, such as edges, corners, textures, cracks, corrosion, etc. These features are usually used to capture specific and subtle changes in an image, especially when detecting surface defects of a building, where local features often directly point to the location and shape of the defect.

[0043] In addition to using deep learning methods, classic computer vision algorithms such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Features), and ORB (Rapid Feature Point Detection) can also be used to extract stable local feature points with strong recognition capabilities from images. These algorithms can maintain feature consistency at different scales and rotation angles, thereby ensuring that detailed information on the building surface can still be accurately extracted under different lighting and viewing angle conditions.

[0044] Once high-level features and local features are extracted from the synthetic and real images, these feature vectors need to be matched. This is usually achieved by calculating the similarity between feature vectors, and common methods include Euclidean distance, cosine similarity, etc. The goal of matching is to find corresponding feature pairs that reflect the same building part in the synthetic and real images.

[0045] Algorithms such as FLANN (fast nearest neighbor search) and BFMatcher (brute force matching) can be used to match feature points. These algorithms establish the correspondence between feature points in the synthetic image and the real image by finding the nearest neighbors of the feature vector. The accuracy and speed of matching are crucial in this process, which directly affects the effect of the subsequent alignment process.

[0046] After feature matching is completed, the synthetic image and the real image need to be geometrically aligned to ensure the spatial consistency of the two. This step usually includes transformations such as rotation, scaling, and translation. By applying these geometric operations, the matched feature points are aligned in the two images, thereby eliminating differences caused by factors such as shooting angle, distance, and deformation.

[0047] During feature alignment, homogeneous transformations can be applied or homography matrices can be calculated to achieve accurate image registration. These mathematical tools can help adjust the geometric relationship of the images so that the corresponding feature points in the two images are aligned in the same coordinate system, so that the features of the synthetic image and the real image can be compared at the same scale and angle.

[0048] The alignment process often requires iterative optimization to improve the accuracy of alignment by minimizing the distance error between feature points. An algorithm based on RANSAC (Random Sample Consensus) can be used to filter out incorrect matching points, thereby improving the robustness and accuracy of alignment.

[0049] Through this process of feature extraction, matching and alignment, the advantages of synthetic images and real-life images can be effectively combined to achieve higher accuracy and consistency in building surface defect detection. This not only helps to reduce the differences caused by different data sources, but also provides a unified basis for subsequent defect identification and analysis, making the detection results more reliable and trustworthy.

[0050] Step S4, normalizing the aligned features, wherein the normalization includes color normalization, scale normalization, and contrast and brightness adjustment to reduce the impact of illumination, weather, and viewing angle changes on the features.

[0051] In this step, the matched and aligned image features are normalized to eliminate interference caused by different external environmental conditions. These interferences include changes in light intensity, variability in weather conditions, and differences in viewing angles. By normalizing the features, the accuracy and consistency of building surface defect detection can be ensured in various complex scenarios. The specific processing steps include the following aspects:

[0052] Color Normalization:

[0053] Images captured at different times and lighting conditions may have significant differences in color representation. For example, an image under strong sunlight may appear too bright and saturated, while an image under cloudy days or at night may appear dim or even color-biased. This inconsistency in color can affect feature accuracy and the performance of the detection model.

[0054] In addition, different camera devices and sensors may also cause color deviations. Even when shooting the same scene, the image colors may be different.

[0055] To achieve color normalization, first, convert the image from the RGB color space to other color spaces that are more suitable for normalization, such as the LAB color space. In the LAB space, the brightness information (L) is separated from the color information (A and B), which helps to adjust the color and brightness independently.

[0056] Through histogram equalization or tone mapping, the color distribution of the image is adjusted to a predefined standard range. This process can balance the color differences between different images so that images under different conditions have consistent color performance.

[0057] Use a white balance algorithm to correct for differences in image color temperature so that white areas in the image appear consistent under different lighting conditions, thereby reducing the impact of light source type and intensity on color.

[0058] Scale Normalization:

[0059] Due to different shooting angles and distances, different images of the same building may have different scale representations, that is, the size and proportion of the object in the image will be different. This will affect the comparison and matching of image features, especially in detail detection, where features of different scales may lead to misjudgment. Different camera equipment, lens focal length, shooting height and other factors can also lead to inconsistency in image scale.

[0060] Scale normalization is a method of scaling the features in an image to the same reference scale. This can be achieved through geometric transformations, such as scaling the image based on the average distance between feature points, the actual physical size of the target object, etc.

[0061] During the processing, a multi-scale feature extraction method can be used to ensure that no matter how the size of the objects in the image changes, the features can be analyzed and compared within a uniform scale range. This method is particularly suitable for processing images taken from different perspectives.

[0062] After extracting features, the features are normalized so that all feature vectors are compared in the same scale space. This can be done by dividing the feature value by the maximum or minimum value of the feature, or by normalizing it using a standard score (Z-score).

[0063] Contrast and brightness adjustment:

[0064] Different lighting conditions directly affect the brightness and contrast of the image. For example, in strong light, the image may appear overexposed and details may be lost; while in low light, the image may appear dim and noise may increase. These changes will directly affect the accuracy of feature extraction.

[0065] Different weather conditions, such as sunny, cloudy, rainy, etc., will also lead to differences in image brightness and contrast, affecting the visual effect and feature consistency of the image.

[0066] By adjusting the image brightness, the average brightness of all images is normalized to a preset value. This can be achieved through linear adjustment or gamma correction to ensure that the image brightness is consistent under different lighting conditions.

[0067] Contrast adjustment aims to enhance the details and edges in the image, making the features in the image more obvious. Techniques such as adaptive histogram equalization (CLAHE) can be used to enhance the local contrast of the image, thereby maintaining the clarity of details in different environments.

[0068] For images that are too bright or too dark, dynamic range compression technology is used to compress the brightness range of the image into a reasonable range, ensuring that all areas of the image have appropriate contrast and brightness.

[0069] By performing the above normalization processing on the aligned image features, the image inconsistency caused by external factors such as lighting, weather, and viewing angle can be effectively eliminated. The normalized image features have consistent performance in the same space, whether it is color, scale, brightness, and contrast, they are all at the same standard. This not only improves the accuracy of feature matching and alignment, but also ensures the consistency and reliability of building surface defect detection under different conditions.

[0070] Ultimately, through these normalization processes, the building surface defect detection system can maintain efficient and accurate detection capabilities in a variety of complex environments and provide more reliable detection results.

[0071] Step S5, multi-modally fuse the normalized synthetic image features with the real-shot image features to form a unified feature representation.

[0072] In this step, the normalized image features from different sources, namely the synthetic image features and the real-shot image features, are integrated through multimodal fusion technology to generate a unified and comprehensive feature representation. This unified feature representation can more comprehensively and accurately reflect the actual state of the building surface, thereby improving the accuracy and reliability of surface defect detection.

[0073] Synthetic image features are generated through 3D modeling and rendering technology, reflecting the simulated performance of buildings under different lighting, weather and viewing conditions. These features usually cover the ideal state of the building, are highly controllable and consistent, and help to supplement and simulate the complex situations in the actual detection environment.

[0074] Before multimodal fusion, these features have been normalized in terms of color, scale, contrast, and brightness to ensure that they can be directly compared with the real image features.

[0075] Real-life image features are extracted from actual building images and reflect the appearance and state of buildings in the real world. These features may be affected by a variety of environmental factors, such as lighting changes, weather conditions, shooting angles, etc., so they need to be normalized to eliminate the differences caused by these external factors.

[0076] Similarly, the real-shot image features have been normalized before multimodal fusion to make them consistent with the synthetic image features in terms of color, scale, brightness, etc.

[0077] The main purpose of multimodal fusion is to integrate feature data from different sources to form a unified feature representation. This fusion can comprehensively utilize the advantages of different image data, overcome the limitations of a single data source, and more accurately capture the details and features of the building surface.

[0078] By fusing features from different modes, the system can combat environmental changes, noise, and uncertainty, and improve the robustness of surface defect detection. For example, synthetic image features can make up for the poor performance of real images under certain extreme conditions (such as strong light and shadows).

[0079] In practical applications, the surface of a building may be disturbed by various external conditions, and the features of a single data source may not be sufficient to provide comprehensive information. Through multimodal fusion, the actual state of the building can be captured more accurately and the accuracy of defect detection can be improved.

[0080] The fused unified feature representation can contain more dimensional information, such as simultaneously reflecting the performance of the building under different lighting conditions and its morphology at different viewing angles, which enables the detection model to understand and analyze the building surface more comprehensively.

[0081] Furthermore, in this embodiment, a deep learning framework is used to input the synthetic image features and the real-shot image features into different network layers, and they are processed and extracted separately, and finally fused at a higher layer. This method can retain and utilize the performance of each feature at different abstract levels, and automatically learn the complex relationship between features through a deep network.

[0082] Specifically, two independent convolutional neural networks (CNNs) are used to process synthetic image features and real image features respectively.

[0083] Synthetic image feature extraction network (CNN-S):

[0084] Input: Synthetic image data

[0085] Structure: convolution layer + pooling layer + convolution layer + pooling layer + fully connected layer

[0086] Output: High-level synthetic image feature vector Fs

[0087] Real-shot image feature extraction network (CNN-R):

[0088] Input: real-time image data

[0089] Structure: convolution layer + pooling layer + convolution layer + pooling layer + fully connected layer

[0090] Output: High-level real-shot image feature vector Fr

[0091] Feature extraction at the middle layer

[0092] In their respective networks, features of synthetic and real images are extracted at different levels of abstraction. For example, convolutional layers extract low-level features such as edges and textures, while deeper convolutional layers extract high-level features such as shape and structure.

[0093] Choose to perform feature fusion in the middle layer of the two networks in order to preserve the features of each abstraction level and combine them. For example, feature fusion can be performed after the second convolutional layer:

[0094] Synthetic image intermediate feature vector Ms: The intermediate features of the synthetic image extracted from the output of the second convolutional layer of CNN-S.

[0095] Real-shot image intermediate feature vector Mr: The intermediate features of the real-shot image extracted from the output of the second convolutional layer of CNN-R.

[0096] Layer fusion process

[0097] The synthetic image features and the real-shot image features are fused in the middle layer. Specifically:

[0098] Concatenate the Ms and Mr feature vectors in the channel dimension to form a comprehensive feature representation

[0099]

[0100] Depth processing after fusion:

[0101] The concatenated comprehensive feature representation is input into a higher-level convolutional network for further feature extraction. This network can be a subsequent convolutional layer, pooling layer, and fully connected layer.

[0102] In higher-level networks, deep learning models automatically learn to integrate complex relationships between features through multiple convolution operations and nonlinear activation functions (such as ReLU) and extract more abstract and useful high-level features.

[0103] Final output:

[0104] In a higher fully connected layer, a final feature vector is generated, which combines the features of the synthetic image and the real image for the classification of building surface defects.

[0105] The unified feature representation generated by multimodal fusion is usually a high-dimensional vector that contains all the key features of the synthetic image and the real image. This high-dimensional feature vector has rich information dimensions and can fully reflect the apparent characteristics of the building.

[0106] Furthermore, in order to improve computational efficiency and the generalization ability of the model, the generated high-dimensional feature vector can be reduced in dimensionality, for example, by using principal component analysis (PCA), linear discriminant analysis (LDA) or autoencoder to compress the feature vector to a smaller dimension while retaining its main information.

[0107] The generated unified feature representation is used as input to the building surface defect detection model. By analyzing and processing these comprehensive features, the model can more accurately identify and locate defects on the building surface.

[0108] The feature representation after multimodal fusion can better capture the characteristics of buildings in complex environments, reduce the occurrence of false detection and missed detection, and significantly improve the performance and reliability of defect detection.

[0109] Step S6: training a building surface defect detection model based on the unified feature representation.

[0110] In this step, a deep learning model is built and optimized to achieve automatic detection of surface defects of buildings. This process not only includes model design and architecture selection, but also covers multiple steps such as model training, optimization and evaluation. The ultimate goal is to build an intelligent system that can accurately identify and classify surface defects of buildings.

[0111] Exemplary detection model

[0112] Input layer: accepts high-dimensional feature vectors from the unified feature representation as input, usually by flattening or adjusting the dimensions to fit the network structure.

[0113] Hidden layer: The hidden layer of the network can include multiple convolutional layers and fully connected layers to gradually extract higher-level features. The design of the hidden layer needs to consider the depth and width of the network and the choice of activation function, such as ReLU, Leaky ReLU, etc.

[0114] Output layer: The output layer is designed according to the requirements of the detection task. If it is a classification task, the output layer can be a Softmax layer that outputs the probability of each defect category.

[0115] Through multiple iterations of training, the model continuously updates parameters on the training set to gradually optimize its performance. Each iteration includes a forward propagation (calculating output and loss) and a backpropagation (updating parameters through gradients) process.

[0116] During the training process, in order to prevent the model from overfitting on the training set, regularization techniques (such as L2 regularization) are usually introduced, or the Dropout layer is used to randomly ignore the outputs of some neurons to enhance the generalization ability of the model.

[0117] Step S7: Use the building surface defect detection model to perform real-time detection on the newly acquired image, identify defects on the building surface, and mark the defect locations and severity.

[0118] In this step, the trained defect detection model is used to instantly process and analyze the latest acquired building images to quickly identify surface defects and accurately mark the specific location and severity of these defects.

[0119] Real-time image data is usually acquired through various image acquisition devices, such as drones, fixed cameras, handheld devices, etc. The acquisition devices shoot at different parts of the building according to the predetermined detection plan or external triggers (such as regular detection or event triggers) to generate high-resolution image data.

[0120] The collected images may be noisy or of poor quality. Before being input into the model, they are first preprocessed. De-noising algorithms (such as Gaussian filtering and median filtering) are used to eliminate random noise in the image, and image enhancement techniques (such as contrast enhancement and sharpening) are used to improve the clarity and detail of the image.

[0121] According to the requirements of the detection model, the resolution of the image is adjusted to an appropriate size to balance computational efficiency and detection accuracy. The image is usually scaled to the standard size of the model input layer while ensuring that important details are not lost.

[0122] The preprocessed image is input into the trained building surface defect detection model. The model quickly analyzes the features in the image and identifies potential defect areas through the forward propagation process of a multi-layer neural network.

[0123] Through the above steps, the present invention utilizes 3D modeling and multimodal feature fusion technology to combine the data advantages of synthetic images and real-shot images, generate a unified feature representation, and greatly improve the accuracy of building surface defect detection. The system can accurately identify and classify various types of defects, such as cracks, peeling, corrosion, etc., and quantitatively evaluate their severity to ensure the reliability of the detection results. The application of normalization processing and multimodal fusion enables the system to effectively respond to image data under different lighting, weather and viewing angle conditions, thereby maintaining consistent detection performance in various complex environments. Whether under adverse conditions such as strong light, shadows, rain and snow, the system can operate stably and provide high-quality detection results.

[0124] On the other hand, the present invention also provides a building surface defect detection system, comprising:

[0125] A modeling module, used to create a virtual model of the building to be inspected using 3D modeling software, wherein the virtual model reflects the structure, material and surface characteristics of the building;

[0126] A simulation module, for generating diversified synthetic image data under different lighting conditions, weather conditions and viewing angles through a 3D rendering engine based on the virtual model;

[0127] A matching module is used to extract features from the synthetic image data and the real-shot image data to obtain high-level features and local features of the image; and to match and align the extracted synthetic image features with the real-shot image features;

[0128] A normalization module is used to normalize the aligned features, wherein the normalization includes color normalization, scale normalization, and contrast and brightness adjustment to reduce the impact of illumination, weather, and viewing angle changes on the features;

[0129] The fusion module is used to perform multimodal fusion of the normalized synthetic image features and the real-shot image features to form a unified feature representation;

[0130] A training module, used for training a building surface defect detection model based on the unified feature representation;

[0131] The detection module is used to use the building surface defect detection model to perform real-time detection on newly collected images, identify defects on the building surface and mark the defect location and severity.

[0132] Furthermore, the specific implementation methods of the above-mentioned system for detecting apparent defects of a building are the same as a method for detecting apparent defects of a building, and all further technical solutions in a method for detecting apparent defects of a building are fully introduced into a system for detecting apparent defects of a building.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

[0134] The present invention does not particularly specify the structure of some modules, which shall be subject to the contents recorded in the prior art. The prior art mentioned in the above background technology section and the specific embodiment section of the present invention can be used as part of the present invention to understand the meaning of some technical features or parameters. The scope of protection of the present invention shall be subject to the contents actually recorded in the claims.

Claims

1. A method for detecting surface defects of a building, characterized in that: The method comprises the following steps: Using 3D modeling software to create a virtual model of the building to be inspected, the virtual model reflects the structure, material and surface features of the building; Based on the virtual model, a 3D rendering engine is used to generate diversified synthetic image data under different lighting conditions, weather conditions and viewing angles; Performing feature extraction on the synthetic image data and the real-shot image data to obtain high-level features and local features of the image; Match and align the extracted synthetic image features with the real-shot image features; Normalizing the aligned features, including color normalization, scale normalization, and contrast and brightness adjustment to reduce the impact of lighting, weather, and viewing angle changes on the features; Perform multimodal fusion of normalized synthetic image features and real-shot image features to form a unified feature representation; Based on the unified feature representation, a building surface defect detection model is trained; Using the building surface defect detection model to perform real-time detection on newly acquired images, identify defects on the building surface and mark the defect locations and severity; The 3D rendering engine generates diverse synthetic image data under different lighting conditions, weather conditions and viewing angles, including: By adjusting the position and angle of the light source, simulate the lighting conditions at different times of the day; Use 3D rendering engine to simulate the appearance of buildings under various weather conditions; By adjusting the position and angle of the virtual camera, images of the building taken from different perspectives are generated; Performing feature extraction on the synthetic image data and the real-shot image data to obtain high-level features and local features of the image includes: Use pre-trained convolutional neural networks to process synthetic and real images to capture macro features of building surfaces; Use computer vision algorithms to extract local feature points from images; The color normalization includes: converting the image from the RGB color space to the LAB color space; adjusting the color distribution of the image to a predefined standard range through histogram equalization or tone mapping; and correcting the color temperature difference of the image using a white balance algorithm so that the white area in the image appears consistent under different lighting conditions.

2. A method for detecting surface defects of a building according to claim 1, characterized in that: The multimodal fusion of the normalized synthetic image features and the real-shot image features includes: Use a deep learning framework to input synthetic image features and real-shot image features into different network layers, process and extract them separately, and finally fuse them at a higher layer.

3. A building surface defect detection system, characterized in that: The system includes the following modules: A modeling module, used to create a virtual model of the building to be inspected using 3D modeling software, wherein the virtual model reflects the structure, material and surface characteristics of the building; A simulation module, for generating diversified synthetic image data under different lighting conditions, weather conditions and viewing angles through a 3D rendering engine based on the virtual model; A matching module, used for extracting features from the synthetic image data and the real-shot image data to obtain high-level features and local features of the image; Match and align the extracted synthetic image features with the real-shot image features; A normalization module is used to normalize the aligned features, wherein the normalization includes color normalization, scale normalization, and contrast and brightness adjustment to reduce the impact of illumination, weather, and viewing angle changes on the features; The fusion module is used to perform multimodal fusion of the normalized synthetic image features and the real-shot image features to form a unified feature representation; A training module, used for training a building surface defect detection model based on the unified feature representation; A detection module, used to use the building surface defect detection model to perform real-time detection on newly acquired images, identify defects on the building surface and mark the defect locations and severity; The 3D rendering engine generates diverse synthetic image data under different lighting conditions, weather conditions and viewing angles, including: By adjusting the position and angle of the light source, simulate the lighting conditions at different times of the day; Use 3D rendering engine to simulate the appearance of buildings under various weather conditions; By adjusting the position and angle of the virtual camera, images of the building taken from different perspectives are generated; Performing feature extraction on the synthetic image data and the real-shot image data to obtain high-level features and local features of the image includes: Use pre-trained convolutional neural networks to process synthetic and real images to capture macro features of building surfaces; Use computer vision algorithms to extract local feature points from images; The color normalization includes: converting the image from the RGB color space to the LAB color space; adjusting the color distribution of the image to a predefined standard range through histogram equalization or tone mapping; and correcting the color temperature difference of the image using a white balance algorithm so that the white area in the image appears consistent under different lighting conditions.

4. A building surface defect detection system according to claim 3, characterized in that: The multimodal fusion of the normalized synthetic image features and the real-shot image features includes: Use a deep learning framework to input synthetic image features and real-shot image features into different network layers, process and extract them separately, and finally fuse them at a higher layer.

Images