Building facade material identification method based on computer vision, computing equipment and storage medium

Abstract

The invention provides a building facade material identification method based on computer vision, and relates to the technical field of computer vision, and the method comprises the steps: obtaining the street view image data of a building facade; marking and preprocessing the image data to obtain training data; a YOLO target detection model is improved, specifically, a feature extraction backbone network of YOLO is replaced with a Swinin-Transform network, an efficient ViT network and a convnextv2 network for evaluation, an optimal scheme is selected, and a CIoU loss function is replaced with an InerMDPIoU loss function; performing model training by using the training data; and processing the building facade image by adopting model identification. According to the method, the feature extraction network and the loss function are improved, so that the recognition accuracy is improved, the building facade material information can be quickly obtained, and the data acquisition cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a computer vision-based building facade material recognition method, computing equipment and storage medium. Background Art

[0002] With the development of the construction industry, the demand for accurate identification of building facade materials is increasing. Accurate identification of building facade materials not only helps to improve the accuracy of building quality assessment, but also provides important support for management work such as cost estimation, maintenance planning and material procurement. At present, building material identification mainly relies on technical means such as building information modeling (BIM), Internet of Things (IoT), laser scanning and imaging systems.

[0003] Existing BIM and IoT systems can provide relatively detailed information on building materials, but this information is mainly based on digital data of buildings. This method is difficult to apply to existing buildings, especially historical buildings, that lack complete digital models or real-time data support. For old buildings built before 2000, the use of BIM and IoT for data collection is not only costly, but the data obtained is often incomplete, which seriously restricts its practical application in the identification of building facade materials.

[0004] Laser scanning technology can capture detailed structural and material information of building facades, but it also has significant limitations. Not only is the equipment expensive, but the operation process and data processing require high professional skills. In addition, due to the labor-intensive data acquisition and processing process, the application efficiency of this technology is low when faced with a large number of existing urban buildings, and it is difficult to meet the identification needs of large-scale construction projects.

[0005] Although the imaging system can detect building facade materials through image data, its recognition accuracy is easily affected by external factors such as ambient light and weather. At the same time, when dealing with complex facade details, especially when the material surface is aged or polluted, the recognition accuracy of traditional imaging systems will drop significantly, and it is easy to make misidentification or incomplete recognition.

[0006] In summary, the existing technology has the following problems in the identification of building facade materials: first, it is not applicable to historical buildings and old buildings, and it is difficult to obtain complete and accurate material information; second, the equipment and data processing costs are high, and professional technical support is required; third, the identification results are easily affected by environmental factors, and the accuracy is unstable; fourth, it is difficult to meet the needs of rapid identification of large-scale building complexes. Summary of the invention

[0007] The purpose of the present invention is to provide a computer vision-based building facade material recognition method, computing device and storage medium, which can realize convenient, efficient and low-cost building facade material recognition to better support the management and analysis of existing building materials.

[0008] To achieve the above object, the present invention is implemented through the following technical solutions:

[0009] A computer vision-based building facade material recognition method comprises the following steps:

[0010] S1: Obtain street view image data containing building facades;

[0011] S2: labeling and preprocessing the street view image data to obtain training data;

[0012] S3: Improve the feature extraction network of the YOLO target detection model and use the improved YOLO target detection model to build a building material recognition model;

[0013] S4: training the building material recognition model using the training data;

[0014] S5: using the building material recognition model to perform recognition processing on the building facade image to obtain a recognition result of the building facade material.

[0015] Furthermore: the step S1 specifically includes: obtaining a street view image of the target area through a GIS system and a map API.

[0016] Further: the step S2 comprises:

[0017] S21: Use image segmentation tools to segment and annotate street view images;

[0018] S22: Data augmentation of the annotated images using random cropping, rotation, tilting, and noise blurring.

[0019] S23: Divide the dataset into training set and test set.

[0020] Further: the specific steps of improving the feature extraction network of the YOLO target detection model in step S3 include:

[0021] S31: Replace the feature extraction network of the YOLO target detection model with the Swin-Transformer network, efficientViT network and convnextv2 network for training and evaluation, and select the network replacement solution with the best performance;

[0022] S32: Replace the CIoU loss function in the selected network solution with the Inner_MDPIoU loss function. The Inner_MDPIoU loss function in step S32 is used to optimize the position prediction accuracy of the target frame.

[0023] S32: Replace the CIoU loss function in the selected network scheme with the Inner_MDPIoU loss function.

[0024] Further: the step S4 comprises:

[0025] S41: Use precision, recall, mean average precision and F1 score as evaluation indicators to evaluate the performance of the model;

[0026] S42: Optimize and adjust the model's hyperparameters based on the evaluation results.

[0027] Further: after step S5, the method further includes:

[0028] S6: Calculate the area ratio of different materials in the building facade.

[0029] Further: the step S6 comprises:

[0030] S61: Segment the recognition result into material regions;

[0031] S62: Calculate the number of pixels in each material area;

[0032] S63: Convert the number of pixels into actual area;

[0033] S64: Calculate the area proportion of different materials.

[0034] Furthermore: the loss function of the building material recognition model includes: position loss, confidence loss and category loss.

[0035] The present invention also provides a computer device, comprising a processor and a memory, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the above method are implemented.

[0036] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. This method reduces the cost of data collection and improves the efficiency of data acquisition by combining GIS system with map API to obtain street view images. This method does not require professional equipment and technicians, overcomes the difficulties of traditional technology in collecting data on historical buildings and old buildings, and makes it more convenient to obtain building material information.

[0039] Second, this method uses a variety of data enhancement methods such as random cropping, rotation, tilting and noise blurring to significantly improve the generalization ability of the model. This enables the model to maintain a high recognition accuracy rate when facing complex environments such as different lighting conditions, perspective changes and image noise, effectively solving the problem that traditional imaging systems are easily disturbed by environmental factors.

[0040] Third, this method improves the model's ability to capture complex features of building materials by replacing the feature extraction network of the YOLO target detection model with the Swin-Transformer network, efficientViT network, and convnextv2 network. This improvement fully utilizes the advantages of the two advanced network structures in feature extraction, while improving the recognition accuracy and maintaining a high processing efficiency, so that this method can meet the needs of rapid recognition of large-scale building complexes. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of a flow chart of a computer vision-based building facade material recognition method in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0044] like Figure 1 As shown: A building facade material recognition method based on computer vision, comprising the following steps:

[0045] S1: Obtain street view image data containing building facades;

[0046] Specifically, the street view images of the target area are obtained by combining a GIS system (such as ArcGIS or QGIS) with a map API (such as Baidu Map API). This method can quickly obtain a large number of building facade images, reducing the cost of data collection. The images obtained include real photos and street view photos collected through online maps.

[0047] S2: labeling and preprocessing the street view image data to obtain training data;

[0048] In some embodiments, step S2 includes:

[0049] S21: Use image segmentation tools to segment and annotate street view images;

[0050] Specifically, image segmentation tools such as roboflow are used to segment and annotate street view images, and the annotation content includes building material categories (such as steel, aluminum, glass curtain walls, etc.) and door and window structures.

[0051] S22: Perform data augmentation on the annotated images using random cropping, rotation, tilting, and noise blurring to improve the generalization ability of the model.

[0052] S23: Divide the dataset into training and test sets

[0053] S3: Improve the feature extraction network of the YOLO target detection model and use the improved YOLO target detection model to build a building material recognition model;

[0054] The YOLO model is known for its efficient detection capabilities and low computing requirements. It achieves target detection by dividing the image into grids and predicting the object category and position in each grid. The present invention selects YOLOv11 as the basic model and innovatively replaces its feature extraction network with Swin-Transformer network, efficientViT network and convnextv2 network. The specific steps include:

[0055] S31: Replace the feature extraction backbone network of YOLOv11 with the Swin-Transformer network, efficientViT network and convnextv2 network for training and evaluation, and select the network replacement solution with the best performance;

[0056] S32: Replace the CIoU loss function in the selected network solution with the Inner_MDPIoU loss function. The Inner_MDPIoU loss function in step S32 is used to optimize the position prediction accuracy of the target frame.

[0057] The loss function of the YOLOv11 basic model includes three parts: position loss, confidence loss, and category loss. Position loss is used to evaluate the difference between the predicted bounding box and the true bounding box, confidence loss indicates the confidence level of whether the predicted box contains the object, and category loss reflects the accuracy of the object category in the predicted box.

[0058] Swin Transformer V2 uses a multi-head self-attention mechanism to divide the image into non-overlapping windows and perform self-attention calculation in each window. The attention calculation formula is:

[0059]

[0060] Where Q (query), K (key) and V (value) are matrices obtained by transforming the image block, d k is the scaling factor. This structure enables the model to capture a wider range of features.

[0061] The ConvNext V2 network is based on the improvement of the convolutional network, using deeper convolutional layers and improved convolution kernel sizes. Its convolution calculation formula is:

[0062]

[0063] Where O(i,j) is the pixel value of the output image, I(i+m,j+n) is the pixel value of the input image, and K(m,n) is the convolution kernel. This structure enhances the ability to learn features.

[0064] S4: training the building material recognition model using the training data;

[0065] In some other embodiments, step S4 includes:

[0066] S41: Use precision, recall, mean average precision and F1 score as evaluation indicators to evaluate the performance of the model;

[0067] The calculation formula of precision is:

[0068]

[0069] Where TP is the number of correctly predicted positive samples, and FP is the number of incorrectly predicted positive samples.

[0070] The calculation formula for recall is:

[0071]

[0072] Where FN is the number of samples that are actually positive but predicted to be negative.

[0073] The mean average precision (mAP) is the average of the average precision of all categories, and the calculation formula is:

[0074]

[0075] Where N is the number of categories, AP c is the average precision for each category c, obtained by calculating the area under the precision and recall curves at different thresholds.

[0076] The F1 score is the harmonic mean of precision and recall, and is calculated as:

[0077]

[0078] The above indicators can comprehensively measure the classification performance of the model. By comparing the precision, recall, mAP and F1 value, the recognition accuracy and generalization ability of the model for building materials are evaluated. It is recommended to set the initial learning rate to 0.001 and the batch size to 32.

[0079] S42: Optimize and adjust the model's hyperparameters based on the evaluation results.

[0080] Specifically, the hyper parameters of the model, such as learning rate, batch size, etc., are optimized and adjusted. In this embodiment, the initial learning rate is set to 0.001 and the batch size is 32.

[0081] S5: using the building material recognition model to perform recognition processing on the building facade image to obtain a recognition result of the building facade material.

[0082] In some other embodiments, after step S5, the method further includes:

[0083] S6: Calculate the area ratio of different materials in the building facade. These data can be used for statistical analysis and management decisions of building materials.

[0084] Wherein, the step S6 comprises:

[0085] S61: Segment the recognition result into material regions;

[0086] S62: Calculate the number of pixels in each material area;

[0087] S63: Convert the number of pixels into actual area;

[0088] Specifically, the number of pixels is converted to actual area through known dimensions of the facade or perspective correction methods.

[0089] S64: Calculate the area proportion of different materials.

[0090] In another embodiment, the present invention further provides a computer device, including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any of the above methods are implemented.

[0091] In another embodiment, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0092] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A computer vision-based building facade material recognition method, characterized in that: The following steps are involved: S1: Obtain street view image data containing building facades; S2: labeling and preprocessing the street view image data to obtain training data; S3: Improve the feature extraction network of the YOLO target detection model and use the improved YOLO target detection model to build a building material recognition model; S4: training the building material recognition model using the training data; S5: using the building material recognition model to perform recognition processing on the building facade image to obtain a recognition result of the building facade material.

2. The computer vision-based building facade material recognition method according to claim 1, characterized in that: The step S1 specifically includes: obtaining a street view image of the target area through a GIS system and a map API.

3. The computer vision-based building facade material recognition method according to claim 1, characterized in that: The step S2 comprises: S21: Use image segmentation tools to segment and annotate street view images; S22: Data augmentation of the annotated images using random cropping, rotation, tilting, and noise blurring. S23: Divide the dataset into training set and test set.

4. The computer vision-based building facade material recognition method according to claim 1, characterized in that: The specific steps of improving the feature extraction network of the YOLO target detection model in step S3 include: S31: Replace the feature extraction network of the YOLO target detection model with the Swin-Transformer network, efficientViT network and convnextv2 network for training and evaluation, and select the network replacement solution with the best performance; S32: Replace the CIoU loss function in the selected network scheme with the Inner_MDPIoU loss function.

5. The computer vision-based building facade material recognition method according to claim 1, characterized in that: The step S4 comprises: S41: Use precision, recall, mean average precision and F1 score as evaluation indicators to evaluate the performance of the model; S42: Optimize and adjust the model's hyperparameters based on the evaluation results.

6. The computer vision-based building facade material recognition method according to claim 1, characterized in that: After step S5, the method further includes: S6: Calculate the area ratio of different materials in the building facade.

7. The computer vision-based building facade material recognition method according to claim 6, characterized in that: The step S6 comprises: S61: Segmenting the recognition result into material regions; S62: Calculate the number of pixels in each material area; S63: Convert the number of pixels into actual area; S64: Calculate the area proportion of different materials.

8. A computer vision-based building facade material recognition method according to any one of claims 1 to 6, characterized in that: The loss function of the building material recognition model includes: position loss, confidence loss and category loss.

9. A computer device comprising a processor and a memory, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images