Building surface defect data set construction method, building surface defect detection method and building surface defect data set construction system

Abstract

The invention relates to the technical field of computer vision processing, image analysis and image processing, in particular to a building surface defect data set construction method, a building surface defect detection method and a building surface defect detection system, and the data set construction method comprises the following steps: S1, obtaining an image data set of building surface defects; s2, screening the obtained image data set, and then performing data simulation algorithm processing to obtain a simulation data set; s3, performing integration processing on the simulation data set and the acquired image data set to form a complete data set; s4, training a defect detection model by using the complete data set; and S5, detecting the data of the test set by using the trained defect detection model, and then evaluating the complexity and the detection result of the defect detection model. In conclusion, the precision and efficiency of defect detection are improved, and the defect types with complex backgrounds and diversification can be effectively processed. The method can quickly and accurately identify the defects on the surface of the building, and has a wide application prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision processing, image analysis and image processing, and in particular to a data set construction method, a detection method and a system for building surface defects based on deep learning. Background Art

[0002] With the widespread use of special equipment and buildings, the detection of surface defects has become increasingly important. Traditional defect detection methods mostly rely on manual visual inspection, which is not only time-consuming and labor-intensive, but also easily affected by subjective factors and has low detection accuracy. Therefore, it is particularly necessary to develop an efficient and automated defect detection system. In recent years, the rapid development of deep learning technology has provided new solutions for defect detection. The YOLO series of algorithms have been widely used in the field of target detection due to their fast and efficient characteristics. However, existing technologies still have certain challenges in dealing with complex backgrounds, defect diversity, and defect feature extraction.

[0003] For example, a Chinese patent document with publication number CN118570163A discloses a building surface crack detection model, method and system based on YOLOv8. The method for constructing a building surface crack detection model based on YOLOv8 proposed in the present invention improves the YOLOv8 model and then performs machine learning on the learning sample to obtain a crack detection model. In the present invention, the YOLOv8 model framework is adopted to maintain the lightweight and real-time advantages of the model, and an EC2f module is set to improve the accuracy of the model. In the EC2f module, the input data is convolved and then dimensionally segmented, and then the unit feature map is extracted and dot multiplication is performed to obtain a rich feature representation set.

[0004] The data set in the above technical solution is mainly composed of concrete crack images, with insufficient crack type and background diversity, including only longitudinal cracks, transverse cracks and fatigue cracks, and failing to cover complex backgrounds or cracks on the surface of buildings of different materials. Although enhanced data is generated by adjusting brightness, contrast and noise, it does not use a variety of complex simulation technologies, and the data richness is limited.

[0005] Moreover, the feature extraction depth is insufficient. Although the introduced EC2f module enhances the feature dimension segmentation and integration, it is limited to operations within the dimension and lacks sensitivity to crack edge features and complex background textures. For scenes where the crack and background characteristics are close or the crack features are weak, the above model is slightly limited.

[0006] Therefore, it is necessary to construct a dataset construction method, detection method and system for building surface defects, combined with image enhancement technology, to improve detection accuracy and generalization ability. Summary of the invention

[0007] In order to overcome the above-mentioned shortcomings, the present invention aims to provide a technical solution that can solve the above-mentioned problems.

[0008] The present invention provides a method for constructing a data set of building surface defects, and the data set construction method specifically comprises the following steps: S1, obtain an image dataset of building surface defects; S2, screening the acquired image data set, and processing the screened image data set with a data simulation algorithm to obtain a simulation data set; S3, integrating the obtained simulation data set with the acquired image data set to form a complete data set; S4. Train the defect detection model using the complete data set; S5. Use the trained defect detection model to detect the test set data, and then evaluate the complexity of the defect detection model and the detection results.

[0009] As a further solution of the present invention: in step S1, the acquisition sources of the image dataset include: an image dataset of building surface defects taken manually and a public defect image dataset, and the image dataset taken manually uses manual pre-annotation of building surface defects.

[0010] As a further solution of the present invention: in step S2, the acquired image data set is screened, and the screening criteria are as follows: the image is clearly visible, and blurry or too dark images are avoided.

[0011] As a further solution of the present invention: in step S2, the data simulation algorithm processing specifically includes the following steps: S2.1, preprocessing the labeled image data, then extracting the crack image from the preprocessed image data, and collecting the mask image corresponding to the crack image to obtain an initial crack image and an initial mask image; S2.2, performing a synchronous image enhancement operation on the initial crack image and the initial mask image, and generating an enhanced crack image after the image enhancement operation and a corresponding enhanced mask image; S2.3, embedding the enhanced crack image after the image enhancement operation into the new building background image to form a composite defect image; S2.4, the composite defect image and the corresponding enhanced mask image are named and saved according to the specification to form a simulation data set.

[0012] As a further solution of the present invention: in step S2.2, the image enhancement operation includes one or more image operation processing of rotating, scaling, filling, splicing, translating, mirroring, cropping or dilating the image.

[0013] As a further solution of the present invention: in step S2.3, the embedding operation includes using an optimization algorithm to operate and process the image to form a composite defect image, and the optimization algorithm includes at least one operation processing algorithm selected from feathering operation, brightness adjustment, contrast adjustment, saturation adjustment, edge blurring processing, and texture fusion operation.

[0014] As a further solution of the present invention: in step S4, the defect detection model is trained using the complete data set, specifically comprising the following steps: S4.1, in the data preparation stage, the complete data set is preprocessed and divided into training set, validation set and test set; S4.2, in the model architecture configuration stage, select the YOLOv8m-seg architecture configuration to form a defect detection model; S4.3, during the training process, the defect detection model is trained and learned using the image data in the training set, including loading the preset weights for transfer learning; and the loss function is used to calculate the loss at each forward propagation, and the optimizer is used to update the model weights in the back propagation algorithm to reduce the loss; S4.4, at the end of each epoch, the defect recognition performance of the defect detection model is evaluated using the validation set; S4.5, during the training process, save the two most recent and best model weight files and training configuration files, and set an early stopping mechanism to avoid over-training.

[0015] As a further solution of the present invention: in step S5, the test set data is tested using the trained defect detection model, which specifically includes the following steps: S5.1, defect detection is completed through forward reasoning. The preprocessed test set image data is input into the trained defect detection model, and the model generates and outputs the location, category and confidence of the defect; S5.2, post-process the model output results to ensure the reliability of the output results; S5.3, compare the model output after post-processing with the real annotation to quantitatively evaluate the output effect of the model; S5.4, the final test results are displayed visually, so that users can understand the test situation intuitively.

[0016] The present invention also provides a detection method, which constructs a data set according to the above-mentioned method for constructing a data set of building surface defects, and uses the constructed data set to train a defect detection model, and then uses the trained defect detection model to detect the image data set of the building surface defects to be processed.

[0017] As a further solution of the present invention: it includes a processor and a memory electrically connected to each other, the memory stores instructions that can be executed by the processor, and when the instructions are executed by the processor, the above-mentioned method for constructing a data set of building surface defects can be implemented.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. Accurately identify multiple defects: Through richer and more diverse data sets, more powerful YOLOv8m-seg models, and optimized data enhancement and model training strategies, the detection accuracy of various defects such as cracks and rust has been significantly improved, especially in the detection of small-scale cracks, low-contrast cracks, and cracks in complex backgrounds. It performs better, reduces missed detections and false detections, and shows stronger robustness than existing technologies.

[0019] 2. Adaptation to multiple materials and multiple scenarios: The data set covers a variety of materials and complex backgrounds. The model can adapt to the needs of building surface defect detection in different environments and materials, from only targeting a single material or scenario (such as steel surface, bridge cracks, etc.) to multiple materials and a wide range of scenarios, such as concrete, brick walls and other building surfaces, as well as special buildings, industrial equipment, pavement, etc. The versatility is significantly improved, and the generalization ability far exceeds the existing technology.

[0020] 3. Efficient and accurate detection process: The improved non-maximum suppression method and dynamic confidence threshold adjustment make the detection process more efficient and accurate. It can effectively handle problems such as crack target overlap in complex backgrounds, improve detection efficiency and accuracy, and the model's layered optimization of crack edge and background texture features further improves the overall detection performance.

[0021] Therefore, the present invention not only improves the accuracy and efficiency of defect detection by adopting the YOLOv8m-seg model and combining image enhancement technology, but also can effectively handle complex backgrounds and diverse defect types. By finely screening the data set and enhancing the image, the training effect of the model is optimized, the generalization ability and robustness of the model are enhanced, and the defects on the surface of special buildings can be quickly and accurately identified, which has broad application prospects.

[0022] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0024] Figure 1 It is a schematic diagram of the overall process of the present invention; Figure 2 is a schematic flow chart of step S2 of the present invention; Figure 3 is a schematic flow chart of step S4 of the present invention; Figure 4 is a schematic flow chart of step S5 of the present invention; Figure 5 It is a structural diagram of the data set and construction steps of the present invention; Figure 6 It is a structural block diagram of the data set and simulation processing steps of the present invention; Figure 7 It is a schematic diagram of the data simulation effect of the present invention. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] See also Figures 1 to 7 In an embodiment of the present invention, a method for constructing a data set of building surface defects includes the following steps: S1, obtain an image dataset of building surface defects; S2, screening the acquired image data set, and processing the screened image data set with a data simulation algorithm to obtain a simulation data set; S3, integrating the obtained simulation data set with the acquired image data set to form a complete data set; S4. Train the defect detection model using the complete data set; S5. Use the trained defect detection model to detect the test set data, and then evaluate the complexity of the defect detection model and the detection results.

[0027] Specifically, the present invention realizes intelligent recognition and detection of cracks, rust and other defects on the surface of special buildings that endanger the structural safety by constructing a data simulation algorithm and combining the data simulation algorithm with the YOLOv8 algorithm. The detection accuracy and generalization ability are significantly improved, and it can be applied to the automatic detection of surface defects of various special equipment. This method uses the YOLO (You Only Look Once) series of algorithms, especially the YOLOv8m-seg model, to achieve efficient detection and recognition of building surface defects such as cracks, rust and honeycomb surface, and has broad application prospects.

[0028] Secondly, by combining the on-site shooting data and public data sets and applying image enhancement algorithms, rich combinations of defect shapes and backgrounds are generated, ensuring the diversity and comprehensiveness of the data set, which helps to optimize the training effect of the YOLOv8m-seg model.

[0029] Thirdly, by using a variety of image enhancement operations, it is possible to simulate defect images in real scenes under different conditions, improve the diversity of training data, and effectively enhance the significance of crack areas. By embedding the enhanced crack image into a new concrete background and applying image fusion techniques (such as edge feathering and brightness contrast adjustment), the generated defect image is more natural, realistic, and in line with the actual scene, which improves the reliability of defect detection.

[0030] Therefore, the present invention not only improves the accuracy and efficiency of defect detection by adopting the YOLOv8m-seg model and combining image enhancement technology, but also can effectively handle complex backgrounds and diverse defect types. By finely screening the data set and enhancing the image, the training effect of the model is optimized, the generalization ability and robustness of the model are enhanced, and the defects on the surface of special buildings can be quickly and accurately identified, which has broad application prospects.

[0031] like Figure 1 and Figure 5 As shown, preferably, in step S1, the image dataset is obtained from: an artificially shot image dataset of building surface defects and a publicly available existing defect image dataset, and the artificially shot image dataset uses manual fine annotation of building surface defects.

[0032] Specifically, the defect datasets obtained come from two sources: proprietary datasets, which are image datasets of building surface defects taken on-site manually; and public datasets, which can select a portion of image data from existing public defect image datasets, and add the proprietary dataset and the public dataset to form a summed dataset of image datasets.

[0033] Secondly, the image dataset of surface defects of special buildings taken on site can be obtained by climbing on the surface of special buildings with cameras and wall-climbing robots. For example, a total of 2,323 defect images of various types were taken, including 2,085 cracks, 389 rusts, and 235 cracks. The defect image dataset taken on site also uses manual annotation to finely annotate the surface defects of buildings. Some of the existing publicly available defect image datasets include one or more existing defect drawing datasets such as CFD, Crack500, DeepCrack, forest+, or Noncrack. A part of one or more existing defect image datasets can be selected as the existing defect image dataset.

[0034] like Figure 1 and Figure 5 As shown, preferably, in step S2, the acquired image data set is screened, and the screening criteria are as follows: the image should be clearly visible, and blurry or too dark images should be avoided as much as possible.

[0035] Specifically, the acquired image data set can be manually screened to select the data with clear and visible images, and blurry or too dark images can be avoided as much as possible to ensure the accuracy of subsequent processing. The screened images have high quality and representativeness, which facilitates the subsequent extraction of crack images from the screened data set.

[0036] like Figure 1 , Figure 2 and Figure 6 As shown, preferably, in step S2, the data simulation algorithm processing includes the following steps: S2.1, preprocessing the labeled image data, then extracting the crack image from the preprocessed image data, and collecting the mask image corresponding to the crack image to obtain an initial crack image and an initial mask image; S2.2, performing a synchronous image enhancement operation on the initial crack image and the initial mask image, and generating an enhanced crack image after the image enhancement operation and a corresponding enhanced mask image; S2.3, embedding the enhanced crack image after the image enhancement operation into the new building background image to form a composite defect image; S2.4, the composite defect image and the corresponding enhanced mask image are named and saved according to the specification to form a simulation data set.

[0037] Specifically, crack images are extracted from annotated datasets, and after preprocessing the crack images and background images, multiple enhancement transformations are applied to them and then embedded into new concrete background images.

[0038] Secondly, the control function is used to turn on and off various enhancement transformation functions. When a function is turned on, the crack image and its corresponding mask image are operated synchronously to ensure the synchronization of their transformation. After the crack image has undergone various enhancement transformations, various optimization algorithms are applied to make the cracks appear more real and natural after being embedded in the background image. At the same time, the number of transformations is set to determine the number of repetitions of the transformation operation that needs to be performed each time the transformation algorithm is applied.

[0039] In addition, it should be noted that the image display function is also used to monitor the image transformation process before and after the transformation algorithm. By specifying the title and image, the crack image and mask image before and after the transformation are displayed to ensure that the transformation effect can be viewed in time during the image processing process, helping users to find and locate possible errors. During the display process, the image will be enlarged to an appropriate size and the coordinate axis will be hidden so that the image content can be clearly observed.

[0040] like Figure 2 and Figure 6 As shown, preferably, in step S2.1, the labeled image data is preprocessed, and then the crack image is extracted from the preprocessed image data, and the mask image corresponding to the crack image is collected to obtain an initial crack image and an initial mask image; Specifically, by manually screening the crack image and embedding the background image, the screened image is preprocessed, specifically including: Convert it to the format required by the algorithm (RGB and Grayscale) for later use, check whether the background image size is sufficient, if not, enlarge it to ensure sufficient processing space.

[0041] Read the required mask image and background image, making sure they have been converted to a format suitable for processing (RGB and Gray).

[0042] Load the background image and check if it is large enough. If not, scale it up multiple times to ensure enough space for processing.

[0043] Secondly, the crack image extraction process includes loading image files of different formats (such as PNG, JPG, BMP) from the specified folder and collecting the corresponding mask images. The mask image marks the crack area in black and white, where the white part represents the crack and the black part represents the background. The specific method of collecting the mask image includes extracting the crack area by setting a threshold and using Boolean indexing to generate an image containing only the crack part. Next, create a completely black image of the same size as the original image and fill the crack part of the mask area into it. During the processing, the mask is transformed synchronously with the original image to ensure size and position consistency.

[0044] like Figure 2 and Figure 6 As shown, preferably, in step S2.2, the image enhancement operation includes image operation processing such as rotating, scaling, filling, splicing, translating, mirroring, cropping or dilating the image.

[0045] Specifically, rotation ensures image integrity through random angles and expansion modes; scaling maintains image proportions and conforms to target size through random factors; dilation enhances the significance of crack areas; stitching combines images according to a predetermined layout to expand the range of defects; translation simulates different perspectives to increase variability; mirroring increases data diversity by flipping; cropping randomly selects areas for cropping to ensure that the image contains enough information. All enhancement operations are randomly combined through functions and act on the crack image and mask at the same time.

[0046] Secondly, rotation and scaling are used to maintain image integrity and proportion; dilation is used to enhance the prominence of crack areas; splicing is used to generate large-scale defect images; translation is used to simulate different observation angles; mirroring and cropping effectively improve data diversity. The above operations are randomly combined through control functions and act on crack images and their masks simultaneously to achieve diversity and richness of the data set.

[0047] Furthermore, it should be noted that the rotation operation ensures that the image will not be cropped after rotation by defining a function that receives the image and the rotation angle as parameters. By setting the parameters, it is ensured that the image boundary after rotation can fully display the rotated content.

[0048] Furthermore, it should be noted that both the scaling and filling operations adjust the size of the crack image and mask to fit the target size. The specific steps are as follows: First, read the size of the image and mask, as well as the target size. If the scaling operation is selected, the scaling ratio of the image is calculated, and the image does not exceed the target size while maintaining the aspect ratio. The image and mask are scaled to the new size using the LANCZOS interpolation method. When the scaling factor in the scaling operation is not 1 or 0, a floating factor is randomly generated to resize the image and mask, and a high-quality scaling method is used for processing. If the filling operation is selected, a black background image of the same size as the target is created, and the original image is embedded in the background in the center; the same method is applied to the mask. And ensure that the image and mask remain centered at the target size, and finally, return the processed image and mask objects.

[0049] Furthermore, it should be noted that the stitching operation randomly selects a certain number of crack images according to the set parameters and embeds them into the background image one by one. The whole process will be repeated multiple times in the loop, and a new stitched image will be generated each time. For each crack image, its size is first obtained and checked whether it exceeds the size of the background image. If the crack image is large, its size will be adjusted through a scaling operation to fit the background image. After the adjustment, the new size of the crack image is re-obtained and its insertable position in the background image is calculated. Subsequently, the insertion coordinates are randomly generated within the allowed range, and the crack image is embedded into the background image to complete the image stitching and enhancement. Further, it should be noted that the translation operation first checks whether the size of the specified area exceeds the range of the background image. If it exceeds, the size of the crack image and its mask is adjusted through a scaling operation to ensure that it fits the background image. After the adjustment is completed, the size of the area is updated, and the maximum insertion coordinates are recalculated to ensure that the crack image can be randomly inserted in the valid area of ​​the background image and will not exceed the boundary, and the translation operation is completed. Furthermore, it should be noted that the mirroring operation is implemented through the built-in method of the image processing library. When mirroring is enabled, the image and its mask will be flipped according to the specified direction to ensure the consistency of the image data.

[0050] Furthermore, it should be noted that the cropping operation randomly generates the starting coordinates of the cropping and the width and height of the cropping area to ensure that the cropping area is inside the image. Subsequently, the crack image and its mask are cropped and the size of the crack image is updated. If the size of the cropped crack image exceeds the background image, it is cropped to fit the background. After that, the maximum insertion coordinates are calculated, and the cropped crack image is embedded into the background image by randomly generating the insertion coordinates to achieve the splicing and enhancement effects.

[0051] Furthermore, it should be noted that the dilation operation expands the foreground area by processing the mask, and is automatically triggered when the scaling ratio is below a preset threshold. The core of the operation is to apply a dilation process to the mask, using a specified number of dilation iterations and a preset threshold. During the processing, the mask is first converted to an array and then dilated using a convolution kernel to increase the size of the masked area, thereby enhancing the visibility of the features and preparing for subsequent processing.

[0052] like Figure 2 and Figure 6 As shown, preferably, in step S2.3, the embedding operation includes using an optimization algorithm to operate and process the image to form a composite defect image, and the optimization algorithm includes at least one operation processing algorithm selected from feathering operation, brightness adjustment, contrast adjustment, saturation adjustment, edge blurring, and texture fusion operation.

[0053] Specifically, the enhanced crack image after the final transformation is embedded into the new concrete background image, and the edge feathering, brightness, contrast, saturation adjustment and texture fusion algorithm are applied to make the image more natural and realistic. Finally, the generated composite crack image and the corresponding enhanced mask image are named and saved according to the specification for subsequent model training. After the image is processed using the optimization algorithm, the generated defect image data can be made more realistic and natural.

[0054] Secondly, the composite defect images can also be optimized, such as performing post-processing such as Poisson fusion, feathering operation, and brightness adjustment, so as to generate image data that meets the model requirements, thereby expanding the acquired defect data set and ensuring that the enhanced image set has rich crack morphology and diverse background combinations to optimize the detection training effect of the YOLOv8m-seg model.

[0055] Furthermore, it should be noted that the feathering operation specifically includes two main operations: calculating the image edge sharpness and automatically feathering the mask edge. First, the input image is converted into a grayscale image, and the edge of the image is extracted using an edge detection algorithm. Then the density of the edge is calculated, and the edge sharpness of the image is obtained by averaging the pixel values ​​of the edge image and standardizing the result. Then, the feathering radius is automatically selected based on the edge sharpness of the crack image and the background image. At the same time, the mask image is converted into grayscale, and the mask image is Gaussian blurred using the selected feathering radius to obtain the feathered mask image.

[0056] Furthermore, it should be noted that the brightness adjustment and contrast adjustment operations specifically include the following steps: first, the image is converted into a grayscale image, and the average brightness and standard deviation of the image are obtained by statistical calculation as indicators of contrast. Then, according to the brightness and contrast of the crack image and the background image, it is determined whether adjustment is required. The adjustment factor is randomly generated within a certain range. During the adjustment process, the brightness of the image is first adjusted using a brightness enhancer, and then the contrast of the image is adjusted using a contrast enhancer, and finally the adjusted image is obtained.

[0057] Furthermore, it should be noted that the saturation adjustment operation specifically includes the following steps: first, the input image is converted into the HSV color mode, the saturation channel is extracted, and its average value is calculated to obtain the saturation of the image. Then, according to the saturation of the crack image and the background image, it is determined whether the saturation of the crack image needs to be adjusted. The adjustment factor is determined by random generation. During the adjustment process, the saturation of the crack image is enhanced or weakened using an enhancer, and finally the adjusted image is obtained.

[0058] Furthermore, it should be noted that the edge blurring process specifically includes the following steps: first, the input image is converted into a grayscale image, and the edge of the image is extracted using the Canny operator. Then, the density of the edge is calculated, and the edge sharpness of the image is obtained by calculating the average value of the edge pixels. The larger the sharpness value, the clearer the edge of the image. Then, based on the edge sharpness of the crack image and the background image, it is determined whether the blurring effect needs to be enhanced. Next, the edge of the mask image is Gaussian blurred using the selected blur kernel. Finally, the blurred mask is applied to the original image through a synthesis operation to generate an image with a blurred edge effect. This processing operation implements an operation of automatically adjusting the blurring of the image edge based on the edge sharpness of the crack image and the background image.

[0059] Furthermore, it should be noted that the texture fusion operation specifically includes the following steps: first, convert the crack image into a grayscale image, and use the Laplace operator to extract the texture information of the image. The texture roughness of the image is measured by calculating the variance of the Laplace operator. The larger the variance value, the rougher the texture of the image. Then, according to the texture roughness of the crack image and the preset threshold, a suitable fusion coefficient is selected. Finally, ensure that the size of the crack image is consistent with that of the background image, and use the weighted average method to fuse the crack image and the background image to generate a fused image.

[0060] In addition, preferably, the parameter values ​​used to determine the adjacent range in the above-mentioned optimization algorithm are all obtained by statistical calculation. The optimization algorithm determines the parameter values ​​of the adjacent range by statistical calculation. First, multiple attributes of the crack image and background image in the real data set are calculated, including sharpness, texture roughness, brightness, contrast and saturation. Then, based on the 75% middle range of each type of attribute, the quality and characteristics of the image are analyzed and compared. This range is used to accurately select parameters, thereby optimizing the process of embedding the crack image into the background image.

[0061] like Figure 2 and Figure 6 As shown, preferably, in step S2.4, the composite defect image and the corresponding enhanced mask image are saved after being named according to specifications to form a simulation data set.

[0062] Specifically, the crack image data and the corresponding mask image finally generated are saved after being named according to specifications, so as to facilitate automatic reading in the model training stage. The naming format includes information such as the original image name, the background image name and the enhancement parameters.

[0063] like Figure 1 and Figure 3 As shown, preferably, in step S4, the yolov8m-seg defect detection model is trained using the complete data set, and the specific steps include: S4.1, in the data preparation stage, the complete data set is preprocessed to divide it into training set, validation set and test set; S4.2, in the model architecture configuration stage, select the YOLOv8m-seg architecture configuration to form a defect detection model; S4.3, during the training process, the defect detection model is trained and learned using the image data in the training set, including loading the preset weights for transfer learning; and the loss function is used to calculate the loss at each forward propagation, and the optimizer is used to update the model weights in the back propagation algorithm to reduce the loss; S4.4, at the end of each epoch, the defect recognition performance of the defect detection model is evaluated using the validation set; S4.5, during the training process, save the two most recent and best model weight files and training configuration files, and set an early stopping mechanism to avoid overtraining; S4.6, during the training result analysis phase, visually analyze the loss curve and indicator graph to identify problems in training, and summarize and propose improvement suggestions based on the model performance.

[0064] Specifically, it includes data preparation, model configuration, training process, performance evaluation, model preservation and result analysis to optimize defect identification effects and adjust training strategies.

[0065] Preferably, in step S4.1, in the data preparation stage, data preprocessing is performed on the complete data set so as to divide it into a training set, a validation set and a test set.

[0066] Specifically, the complete dataset is divided into training set, validation set and test set according to the dataset division ratio (7:2:1), and then the data is formatted and all images are uniformly processed into the input format required by the model, including adjusting the image size and normalizing the pixel values ​​to meet the input requirements of the YOLOv8m-seg model. All images are uniformly adjusted to the input format required by the model, that is, all images are converted to a standard size of 640x640 pixels to ensure consistent size. Image pixel values ​​are normalized to the range of [0,1] to meet the model's requirements for input images.

[0067] Preferably, in step S4.2, in the model architecture configuration stage, the YOLOv8m-seg architecture is selected and the appropriate configuration is determined, the model structure parameters are optimized, and the training hyperparameters such as learning rate, batch size and number of iterations are set.

[0068] Specifically, configure the model architecture, select the yolov8m-seg model architecture, and set the model input to a three-channel (RGB) image. Initially, 16 filters are used for multi-level feature extraction, and they are gradually doubled. The pooling layer is used for downsampling to reduce the amount of calculation and improve the calculation efficiency. The activation function uses ReLU activation to enhance the expression of nonlinear features. At the same time, set the initial learning rate to 0.001, the batch size to 16× the number of graphics cards used for training, and the number of iterations to 3000 epochs to ensure sufficient training rounds to learn data features.

[0069] Preferably, in step S4.3, during the training process, the defect detection model is trained and learned using the image data in the training set, including loading preset weights for transfer learning; and using a loss function to calculate the loss during each forward propagation, and using an optimizer to update the model weights in the back propagation algorithm to reduce the loss.

[0070] Specifically, during the training process, transfer learning is performed by loading the weights of the pre-trained model, selecting appropriate pre-trained weights to improve initial performance, calculating losses each time forward propagation, using three types of loss functions unique to YOLO: positioning loss, confidence loss, and classification loss, and using one or more of the backpropagation algorithm, SGD optimizer, and AdamW optimizer to update model weights to reduce losses. The training process includes model weight initialization, using the COCO dataset pre-trained model for transfer learning to reduce training time. In each forward propagation, the positioning loss, confidence loss, and classification loss are calculated, and the weights are updated using backpropagation and SGD optimizer to gradually optimize the model.

[0071] Preferably, in step S4.4, at the end of each epoch, the model performance is evaluated using the validation set, and indicators such as accuracy, recall, and F1 score are recorded to evaluate the defect recognition ability of the model. During the training process, the best and two most recent model weight files and the training configuration file are saved.

[0072] Specifically, in the performance evaluation stage, after each epoch, the validation set is used to evaluate the accuracy, recall, and F1 score of the model, and indicators such as IOU, MSE, E-MEASURE, and S-MEASURE are introduced to comprehensively evaluate the model's recognition ability for different defect types and select the best model.

[0073] Preferably, in step S4.5, during the training process, the most recent best model is saved, and an early stopping mechanism is set to avoid overtraining. Specifically, when the detection result of the model does not improve significantly, the training is stopped. At the same time, an early stopping mechanism is set to stop training when the model has been trained for a long time but has not learned new defect features, so as to avoid wasting computing resources.

[0074] Preferably, in step S4.6, during the training result analysis phase, the loss curve and the indicator graph are visually analyzed to determine the problems in the training, and a summary is made based on the model performance and suggestions for improvement are proposed.

[0075] Specifically, by analyzing the loss curve and various indicator curves, we can identify problems in training, such as oscillation or overfitting, and ensure the stability of the model. Finally, we can develop a plan for further improvement based on the performance of the model on different defect types.

[0076] like Figure 1 and Figure 4 As shown, preferably, in step S5, the test set data is detected using the trained defect detection model, and the specific steps include: S5.1. During the test set data detection process, data preparation is first performed, and the test set images are preprocessed in the same way as the training set, including image scaling and normalization, to ensure that the test data is consistent with the model input format; specifically, the test set images are first preprocessed in the same way as the training set, including image scaling and normalization operations.

[0077] S5.2, defect detection is completed through forward reasoning. The preprocessed test image is input into the trained YOLOv8m-seg model, and the model generates the location, category and confidence of the defect. Specifically, the preprocessed image is input into the model, and forward reasoning is performed to obtain the defect location, category and confidence score in each image.

[0078] S5.3, post-process the model output results, including non-maximum suppression (NMS) to remove overlapping boxes and retain the boxes with the highest confidence, and set a confidence threshold to filter to ensure the reliability of the output results; specifically, post-process the detection results to improve accuracy and reliability. Through non-maximum suppression, remove overlapping boxes and low-confidence boxes, and only retain the most confident detection results. At the same time, set a confidence threshold to ensure that the output detection box has a high reliability.

[0079] S5.4, result evaluation is carried out by comparing with the true annotations, and the following performance indicators are calculated respectively: accuracy, recall, F1 score, mAP, MSE, E-Measure, S-Measure and IoU, so as to quantitatively evaluate the model effect; specifically, the detection results are quantitatively evaluated using the true annotation information, and the evaluation indicators include accuracy, recall, F1 score, MSE, E-Measure, S-Measure, IoU and mAP (mean average precision).

[0080] S5.5, the detection results are displayed visually, and the detection box, defect type, location and confidence information are drawn on the original image, so that users can intuitively understand the detection situation. Specifically, the detection results are visualized, and the detection box is drawn on the original image, and the defect type, location and confidence score are marked to help evaluate the detection ability of the model and provide a reference for subsequent optimization.

[0081] In general, the trained model is used to test the test set data, and then the model complexity and test results are evaluated. The test set images are preprocessed and input into the trained YOLOv8m-seg model for forward reasoning, and the test results are optimized by post-processing and non-maximum suppression. The model performance is evaluated by indicators such as accuracy and recall, and finally the test results are visualized.

[0082] This patent has made many improvements to the existing technology in terms of defect detection technology for steel and building surfaces. The main technical improvements include data set construction and enhancement, model optimization, and adaptability expansion, which has brought about technical improvements in many aspects such as detection accuracy and generalization ability. The following is a summary of the improvements: 1. Dataset Construction and Enhancement 1. Diversified data sources and material coverage: Build a multi-source dataset that covers defect images of various building materials such as steel, concrete, brick walls, etc., such as cracks, rust, etc., to change the situation in existing technical solutions where the dataset is single or limited to specific materials and scenes (such as only steel surface, only bridge cracks, etc.), enhance the diversity of the dataset, and provide a more comprehensive data foundation for model training.

[0083] 2. Refined labeling and data processing: Emphasis on manual labeling to ensure accurate positioning of crack positions and features, avoid possible deviations caused by automated labeling, and improve the quality of training data. At the same time, a series of complex image enhancement technologies are used, such as conventional operations such as rotation, scaling, expansion, splicing, translation, mirroring, and cropping, as well as advanced processing methods such as edge feathering, brightness, contrast, and saturation adjustment, to make the generated defect images more realistic and natural, increase the complexity and diversity of the data set, and optimize the model training effect.

[0084] 2. Model Optimization 1. Adopting YOLOv8m-seg model: Different from the YOLOv5, improved YOLOv5, deep residual network, YOLO algorithm and other models used in the prior art solutions, this patent uses the YOLOv8m-seg model, which supports semantic segmentation and can perform target detection and precise region segmentation at the same time, effectively improving the positioning accuracy of crack edges and details, and is more suitable for crack detection tasks under complex backgrounds, especially when the crack edges are blurred or have a large interference with the background.

[0085] 2. Optimize model training and detection performance: introduce an improved non-maximum suppression method, dynamically adjust the confidence threshold, reduce redundant detection frames, avoid false detection and missed detection, and improve the accuracy of detection results, especially for overlapping crack targets in complex backgrounds. At the same time, perform hierarchical optimization on the feature extraction of the model to improve the recognition ability of tiny cracks, crack edges and background texture features.

[0086] 3. Adaptability 1. Cross-scenario universal design: This patented technical solution aims to enhance the cross-scenario application capability of the model, making it applicable to a variety of building surfaces, including different materials such as concrete, metal, brick walls, coated walls, as well as special buildings, industrial equipment, pavement and other scenarios, breaking through the limitations of existing technical solutions that are only applicable to specific scenarios (such as only steel surfaces, only building cracks, etc.), and meeting the needs of different detection tasks.

[0087] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

Claims

1. A method for constructing a data set of building surface defects, characterized in that: The data set construction method specifically comprises the following steps: S1, obtain an image dataset of building surface defects; S2, screening the acquired image data set, and processing the screened image data set with a data simulation algorithm to obtain a simulation data set; S3, integrating the obtained simulation data set with the acquired image data set to form a complete data set; S4. Train the defect detection model using the complete data set; S5. Use the trained defect detection model to detect the test set data, and then evaluate the complexity of the defect detection model and the detection results.

2. The method for constructing a data set of building surface defects according to claim 1, characterized in that: In step S1, the acquisition sources of the image dataset include: an image dataset of building surface defects taken manually and a public defect image dataset, and the image dataset taken manually is pre-labeled with building surface defects manually.

3. The method for constructing a data set of building surface defects according to claim 1, characterized in that: In step S2, the acquired image data set is screened, and the screening criteria are as follows: the image is clear and visible, and blurry or too dark images are avoided.

4. The method for constructing a data set of building surface defects according to claim 1, characterized in that: In step S2, the data simulation algorithm processing specifically includes the following steps: S2.1, preprocessing the labeled image data, then extracting the crack image from the preprocessed image data, and collecting the mask image corresponding to the crack image to obtain an initial crack image and an initial mask image; S2.2, performing a synchronous image enhancement operation on the initial crack image and the initial mask image, and generating an enhanced crack image after the image enhancement operation and a corresponding enhanced mask image; S2.3, embedding the enhanced crack image after the image enhancement operation into the new building background image to form a composite defect image; S2.4, the composite defect image and the corresponding enhanced mask image are named and saved according to the specification to form a simulation data set.

5. The method for constructing a data set of building surface defects according to claim 4, characterized in that: In step S2.2, the image enhancement operation includes one or more of the following image operations: rotating, scaling, filling, splicing, translating, mirroring, cropping or dilating the image.

6. The method for constructing a data set of building surface defects according to claim 4, characterized in that: In step S2.3, the embedding operation includes using an optimization algorithm to operate and process the image to form a composite defect image, and the optimization algorithm includes at least one operation processing algorithm of feathering operation, brightness adjustment, contrast adjustment, saturation adjustment, edge blurring, and texture fusion operation.

7. The method for constructing a data set of building surface defects according to claim 1, characterized in that: In step S4, the defect detection model is trained using the complete data set, which specifically includes the following steps: S4.1, in the data preparation stage, the complete data set is preprocessed and divided into training set, validation set and test set; S4.2, in the model architecture configuration stage, select the YOLOv8m-seg architecture configuration to form a defect detection model; S4.3, during the training process, the defect detection model is trained and learned using the image data in the training set, including loading the preset weights for transfer learning; And use the loss function to calculate the loss at each forward propagation, and use the optimizer to update the model weights in the back-propagation algorithm to reduce the loss; S4.4, at the end of each epoch, the defect recognition performance of the defect detection model is evaluated using the validation set; S4.5, during the training process, save the two most recent and best model weight files and training configuration files, and set an early stopping mechanism to avoid over-training.

8. The method for constructing a data set of building surface defects according to claim 7, characterized in that: In step S5, the trained defect detection model is used to detect the test set data, which specifically includes the following steps: S5.1, input the preprocessed test set image data into the trained defect detection model, and the model generates and outputs the location, category and confidence of the defect; S5.2, post-process the model output results to ensure the reliability of the output results; S5.3, compare the model output results after post-processing with the actual annotations to quantitatively evaluate the output effect of the model.

9. A detection method, characterized in that: According to a method for constructing a dataset of building surface defects according to any one of claims 1 to 8, a dataset is constructed, and a defect detection model is trained using the constructed dataset, and then the trained defect detection model is used to detect the image dataset of building surface defects to be processed.

10. A system, characterized in that: It comprises a processor and a memory electrically connected to each other, wherein the memory stores instructions executable by the processor, and when the instructions are executed by the processor, a method for constructing a data set of building surface defects according to any one of claims 1 to 8 can be implemented.

Images