A building defect detection and classification method based on contrast samples

Abstract

The application belongs to the technical field of image classification, and particularly relates to a building defect detection and classification method based on contrast samples. The method comprises the following steps: manually labeling features of original data and performing data enhancement; fine-tuning a deep matting model U2-Net to generate a contrast sample of a background image without features; building a model to perform image classification; calculating model loss; and adjusting the loss weight to make the model achieve the best classification effect. The method of the application can constrain the model to recognize label categories with complex data features through a designed contrast loss function, and can classify single category data through a cross-entropy loss function, so that the combination of the two can effectively improve the recognition ability of the model for complex labels.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image classification, and particularly relates to a building defect detection classification method based on contrast samples. BACKGROUND

[0002] The old buildings in communities have caused various safety problems such as facility aging and residents' unauthorized modification due to long service life, which has caused the trouble of urban management personnel and residents. The defect detection of old community buildings has important practical significance. The public space occupation problem is common and the sample features are complex and diverse, such as flowerpots, cabinets and bicycles in corridors. The traditional classification algorithm can only identify a single item, but the features of items in the public space occupation problem are complex and diverse. Identifying and classifying each item will cause a waste of a large amount of human and material resources, and will confuse the model in identifying other scene problems, which is not realistic for the realization of engineering tasks. SUMMARY

[0003] In view of the deficiencies in the prior art, the application provides a building defect detection classification method based on contrast samples, which comprises the following steps:

[0004] Step one: manually labeling features and data enhancement on original data;

[0005] Step two: fine-tuning the deep matting model U2-Net to generate a featureless background image as a contrast sample;

[0006] Step three: building a model for image classification;

[0007] Step four: calculating the model loss;

[0008] Step five: adjusting the loss weight to make the model achieve the best classification effect.

[0009] The specific steps of manually labeling features and data enhancement on original data in step one include:

[0010] Frame the position of the label features in the image of the obtained original data; adopt random cropping, horizontal flipping and brightness adjustment to expand the data samples.

[0011] The specific steps of fine-tuning the deep matting model U2-Net to generate a featureless background image as a contrast sample in step two include:

[0012] S1: randomly select several images from each type of sample, use the matting function of the matting software, and manually remove the features to generate a featureless background image corresponding to the original image;

[0013] S2: input the generated featureless background image into the U2-Net network with a pre-trained model, train the pre-trained model for a preset number of rounds to fine-tune the model, and generate a new pre-trained model to adapt to the building defect detection data set;

[0014] S3: using the new pre-trained model, performing image subtraction prediction on the original data without contrast samples;

[0015] S4: selecting a visual feature that meets the background image data only as an opposite sample to the original image from the prediction effect;

[0016] S5: feeding the selected new opposite sample data into the pre-trained model for fine-tuning;

[0017] S6: repeating steps S3-S5 until all samples meet the background image without data features.

[0018] The step three builds a model to classify images, and the specific steps include:

[0019] The model feature extraction channel 1 takes resnet50 as the backbone network to obtain the original data features .

[0020] The model feature extraction channel 2 takes resnet50 as the backbone network to obtain the background data features .

[0021] The original data features and the background data features are fused to generate fusion data features .

[0022] The fusion data features are finally predicted by a softmax function to obtain data classification.

[0023] The step four includes model loss calculation:

[0024] The output of the original data features is obtained by a softmax function to obtain the probability of each original data category , and the cross-entropy loss calculation is performed on to obtain , as shown in equation (1):

[0025] (1);

[0026] Wherein, is the number of samples, represents the i-th sample, is the data category label; represents the i-th sample, is the data category label; one class, representing the probability that the network channel 1 predicts the i-th sample to be a class ;

[0027] The output of the fusion data feature is obtained by a softmax function to obtain the probability of each fusion data class , loss calculation is performed on to obtain the fusion data loss value , as shown in formula (2):

[0028] (2);

[0029] The model loss is fused by the loss weight coefficient, as shown in formula (3):

[0030] (3);

[0031] wherein, and are weight coefficients.

[0032] The beneficial effects of the present application are: a building defect detection classification method based on contrast samples is provided, which constrains the model to recognize the label class with complex data features through the designed contrast loss function, and at the same time uses the cross-entropy loss function to classify single class data, and the combination of the two can effectively improve the recognition ability of the model for complex labels (public space occupancy). The use of this method has a strong guiding effect on building planners to formulate building rectification or construction schemes in the old community building defect detection scene. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a flowchart of a building defect detection classification method based on contrast samples according to an embodiment of the present application;

[0034] Figure 2 is a flowchart of the process of fine-tuning an existing deep matting model (U2-Net) to generate a featureless background image (opposite sample) according to an embodiment of the present application;

[0035] Figure 3 is a framework diagram of the model network according to an embodiment of the present application;

[0036] Figure 4 is a flowchart of the loss calculation process according to an embodiment of the present application;

[0037] Figure 5 is a statistical chart of the original data without data balancing and enhancement according to an embodiment of the present application;

[0038] Figure 6 ​is the data statistical graph of the data balancing and enhanced by the embodiment of the application;

[0039] Figure 7 is the model classification accuracy statistical graph of the ResNet50 backbone network and cross-entropy loss as the loss function of the model adopted by the embodiment of the application;

[0040] Figure 8 is the model classification accuracy statistical graph trained by the network architecture and loss algorithm proposed by the embodiment of the application;

[0041] Figure 9 is the attention comparison graph of the last convolution layer of the network of the traditional ResNet50 model and the network architecture and loss algorithm proposed by the embodiment of the application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the application more clear and understandable, the application will be further described in detail below in combination with the drawings and embodiments. It should be noted that the specific embodiments described herein are only used to explain the application and not to limit the application. The embodiments are implemented on the premise of the technical scheme of the application, and give detailed implementation modes and specific operation processes, but the protection scope of the application is not limited to the following embodiments.

[0043] The embodiment of the application provides a building defect detection classification method based on contrast samples, as shown in Figure 1 , comprising the following steps:

[0044] Step 1: Artificial feature labeling and data enhancement are performed on the original data.

[0045] The application collects data from old communities that have been permitted to collect building defect data. The collected data is labeled, and the labeled data is counted, as shown in Figure 5 . According to the statistics, the number of additional bay windows is increased, and the number of additional elevators is relatively small. In order to make full use of the data and reduce the impact of the long-tail distribution of the data on the performance of the model, the application randomly selects the data with more categories, and performs data enhancement on the data with fewer categories. The data enhancement steps include random cropping, random horizontal inversion (here, only random horizontal inversion is used without vertical inversion, because combined with the semantics of the image, the ground cannot be at the top of the image), brightness adjustment (part of the scene may appear too bright or too dark due to different shooting times and light intensities). The data statistics after data adjustment are shown in Figure 6 .

[0046] Step 2: Fine-tune the existing deep matting model (U2-Net) to generate featureless background data (opposite samples)

[0047] In this embodiment, the fine-tuning of the existing deep matting model (U2-Net) to generate featureless background data (opposite samples) includes the following steps:

[0048] First, 300 data are selected from each type of data, and the object features in the image are removed using a matting software to make it a background-only image (opposite sample). The opposite sample and the original image form a contrast image pair. The contrast image data set formed by the pair is input into the deep matting model (U2-Net) with a pre-trained model. Through training, the new model generated can adapt to the building defect detection data. The fine-tuned model generated is defined as M0. The fine-tuned model M0 is used to predict data without contrast sample pairs. From the prediction results, background-only images (opposite samples) that meet the visual features are selected, and the contrast image data formed by the selected data and its original image are merged with the existing contrast image data to form a new contrast image data set. The fine-tuned model M0 is fine-tuned again using the new contrast image data set to generate a new fine-tuned model defined as M1. The fine-tuned model M1 is used to continue to predict the remaining data without contrast sample pairs. The steps of selecting contrast samples that meet the visual features and fine-tuning the pre-trained model are repeated until all data have contrast samples. The implementation flowchart is shown in Figure 2 .

[0049] Step three: building a model to classify images;

[0050] In this embodiment, the step of building a model to classify images includes:

[0051] The overall structure of the model is shown in Figure 3 Among them, the feature extraction path 1 and the feature extraction path 2 are both resnet50 network architectures, which realize data feature extraction and obtain original sample features and contrast sample features . In order to fully utilize the contrast data information, the fusion data features , are obtained by adding and . The fusion data features are calculated by a softmax function to obtain the model classification probability. The model classification probability is used to obtain the final classification result of the model using the designed loss function. The network framework diagram is shown in Figure 3 .

[0052] Step four: model loss calculation design

[0053] In this embodiment, the model loss calculation process is shown in 4, and the specific steps include:

[0054] Original data features The output is subjected to a softmax function to obtain the probability of each original data category The cross-entropy loss calculation is performed on to obtain As shown in formula (1):

[0055] (1);

[0056] wherein, is the number of samples, represents the i-th sample, is the data category label; represents the i-th category, represents the probability that the i-th sample predicted by the network channel 1 is of the category . The loss calculation method is consistent with the effect of a traditional classification model. The output of the fused data features is subjected to a softmax function to obtain the probability of each fused data category The loss calculation is performed on

[0057] to obtain the fused data loss value As shown in formula (2): (2);

[0058] (3);

[0059] The model loss is fused by loss weight coefficients of formula (1) and formula (2), as shown in formula (3):

[0060] (3);

[0061] wherein, and are loss weight coefficients.

[0062] Step five: adjust the loss weight to make the model achieve the best classification effect, and the specific steps include:

[0063] The designed loss (formula (3)) and the collected and created data are used to train the model, and the loss weight coefficients and are adjusted to make the training effect reach the expectation or achieve the best model classification effect.

[0064] In order to verify the performance of the method proposed in the application, the comparative example uses ResNet50 as the backbone network, uses cross-entropy loss as the classification loss, and does not use the method proposed in the application to classify the created data set, and the classification accuracy category statistics are shown in Figure 7 . Figure 8 ​​A classification precision category statistical chart of the created data set classified by the loss algorithm of the application, wherein the loss weight coefficients and are 0.45 and 0.6 respectively. Figure 7 and Figure 8 It can be seen that the classification precision of the model using the method of the application has a higher improvement in most data categories, especially for the public space occupancy category (with complex semantic labels) data, the classification precision has a 21% point improvement. Figure 9 It can be seen that compared with the traditional ResNet50 classification model and the attention of the last layer of the convolution layer of the model architecture and algorithm of the application, the model architecture and algorithm of the application have obvious focusing effect on the model attention, and can more accurately identify the object and its position. In addition, the model also has good focusing effect on the attention of other categories of objects.

Claims

1. A method for classifying and detecting building defects based on comparative samples, characterized in that, Includes the following steps: Step 1: Perform manual feature annotation and data augmentation on the raw data; Step 2: Fine-tune the deep matting model U2-Net to generate featureless background image counterparts; specifically, this includes: S1, randomly select several images from each class of samples, use the matting function of matting software to manually remove features from them, and generate featureless background images corresponding to the original images. S2. Input the generated featureless background image into the U2-Net network with a pre-trained model, train it for a preset number of rounds to fine-tune the model, and generate a new pre-trained model to adapt to the building defect detection dataset. S3. A new pre-trained model is used to predict image removal from raw data without comparison samples. S4. Select visual features from the predicted results that match only the background image data as the opposite sample to the original image; S5. Input the newly selected data that matches the opposing samples into the pre-trained model and fine-tune the model again. S6. Repeat steps S3-S5 until all samples satisfy the background image without data features; Step 3: Build a model for image classification; specific steps include: Model feature extraction channel 1 uses ResNet50 as the backbone network to obtain the features of the raw data. ; Model feature extraction channel 2 uses ResNet50 as the backbone network to obtain background data features. ; Raw data characteristics and background data features Fusion generates fused data features ; The data features are fused and the data classification is finally predicted using the softmax function. Step 4: Calculate the model loss: Raw data characteristics The output is used to obtain the probability of each original data category through the softmax function. ,right Cross-entropy loss was calculated to obtain As shown in equation (1): （1）； in, It is the sample size. Indicates the first One sample, Indicates the first Categories This indicates the predicted first channel of network channel 1. Each sample is a category The probability of; Fusion data features The output is used to obtain the probability of each fused data category through the softmax function. ,right Perform loss calculation to obtain the fused data loss value. As shown in equation (2): （2）； The model loss is achieved by fusing equations (1) and (2) through loss weight coefficients, as shown in equation (3): （3）； in, and These are the loss weighting coefficients; Step 5: Adjust the loss weights to achieve the best classification results for the model.

2. The building defect detection and classification method based on comparative samples according to claim 1, characterized in that, In step one, the original data is manually labeled and augmented. The specific steps include: outlining the positions of the label features in the image of the acquired original data; and expanding the data samples by random cropping, horizontal flipping, and brightness adjustment.

Images