A robot intelligent detection method for surface defects of high-altitude buildings

Abstract

The present invention relates to a method for intelligent robot detection of surface defects in high-altitude buildings, comprising the following steps: obtaining an image of a surface defect to be detected, inputting the image of the surface defect to be detected into a high-altitude building surface defect detection model carried by the robot, and obtaining a surface defect detection result; the high-altitude building surface defect detection model is trained using a training set; the training set includes: surface defect images and defect labels; the high-altitude building surface defect detection model uses lightweight convolution for feature extraction, introduces a spatial attention mechanism to calculate the attention weight of the spliced ​​feature maps, and assigns the attention weight to the original feature maps. The present invention utilizes a neural network to learn the characteristics of different types of defects, constructs a deep learning network for defect detection, and carries it onto a robot operating system. Through the robot's movement, the network uses a video stream from a visual sensor to identify the presence and type of defects on the building surface.

Description

Technical Field

[0001] The present invention relates to the technical field of high-altitude building quality inspection, and in particular to a high-altitude building surface defect intelligent inspection method using a robot. Background Art

[0002] Large-scale infrastructure, bridges and other construction facilities have a specified service life. During this period, in order to ensure the normal and safe operation of large-scale infrastructure, bridges, etc., they must be subject to regular safety inspections. The surfaces of such large-scale facilities generally have no special protection measures and are easily affected by extreme weather, irregular changes in fatigue loads and unexpected situations during operation. If an accident occurs or fatigue stress is generated internally, cracks will form on the surface, which will also be reflected in the stability of the connection device. Traditional building surface inspection methods mainly focus on manual high-altitude inspections, through visual inspection or measurement instruments. However, this method is time-consuming, the area that can be inspected per unit time is small, and there are certain safety hazards. The time and economic costs are very high. Summary of the Invention

[0003] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a robot intelligent detection method for surface defects of high-altitude buildings, which uses a deep learning network to extract features. Unlike traditional manual and professional equipment detection, it requires filter denoising and grayscale calculation of the building surface.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A method for intelligent detection of surface defects of high-altitude buildings by a robot, comprising:

[0006] Acquire a surface defect image to be detected, input the surface defect image to be detected into a high-altitude building surface defect detection model carried by the robot, and obtain a surface defect detection result; the high-altitude building surface defect detection model is trained using a training set; the training set includes: surface defect images and defect labels;

[0007] The high-altitude building surface defect detection model uses lightweight convolution for feature extraction, introduces a spatial attention mechanism to calculate the attention weights after feature map splicing, and assigns the attention weights to the original feature maps.

[0008] Optionally, the high-altitude building surface defect detection model includes:

[0009] A Backbone module is used to extract several feature layers from the surface defect image to be detected;

[0010] Neck module, used to perform feature fusion on the several feature layers to obtain multiple prediction values;

[0011] The Head module is used to decode the multiple prediction values ​​to obtain the surface defect detection result.

[0012] Optionally, the Backbone module includes:

[0013] A first Conv submodule and a first residual connection submodule, wherein the first Conv submodule and the first residual connection submodule are alternately arranged, and the SPPF submodule is connected after the first residual connection submodule at the end;

[0014] The first Conv submodule is used to extract features using lightweight convolution, and perform convolution, batch normalization and activation function operations on the feature map obtained by convolution;

[0015] The first residual connection submodule is used to maintain the uniformity of feature extraction results;

[0016] The SPPF submodule is used to aggregate the features on the last feature layer output by the Backbone module;

[0017] The first residual connection submodule is sequentially provided with: a first residual connection unit, a second residual connection unit and a third residual connection unit.

[0018] Optionally, maintaining the uniformity of the feature extraction result includes:

[0019] X'=W*X+b

[0020]

[0021] Among them, W represents the weight of the convolution layer, X represents the input data, and b represents the bias of the convolution layer. represents the result of batch normalization output, γ and β are adjustable parameters, which are used to scale and translate the normalized data, μ represents the mean of the current batch data, σ represents the standard deviation of the current batch data, ε is a small constant to prevent division by 0, and Y represents the final output result of 0 and The larger value.

[0022] Optionally, the Neck module includes:

[0023] A plurality of second Conv submodules, upsampling submodules, first merging channel submodules and second residual connection submodules are connected in sequence according to the order of the second Conv submodule, the upsampling submodule, the first merging channel submodule and the second residual connection submodule; the second Conv submodule is sequentially provided with: a first Conv unit and a second Conv unit; the first Conv unit is connected to the SPPF submodule of the Backbone module; the first merging channel submodule is sequentially provided with: a first merging channel unit and a second merging channel unit; the first merging channel unit is connected to the second residual connection unit, and the second merging channel unit is connected to the first residual connection unit;

[0024] The second Conv submodule is used to extract features using lightweight convolution, and perform convolution, batch normalization and activation function operations on the feature map obtained by convolution;

[0025] The upsampling submodule is configured to perform an upsampling operation on the output result of the second Conv submodule;

[0026] The first merging channel submodule is configured to fuse the output result of the upsampling submodule with the output result of the residual connection submodule along the channel dimension;

[0027] The second residual connection submodule is used to accelerate the training and reasoning speed of the network while preventing gradient explosion or gradient zero.

[0028] Optionally, the Neck module further includes:

[0029] A plurality of downsampling submodules, a second merging channel submodule and a third residual connection submodule are connected in sequence according to the downsampling submodule, the second merging channel submodule and the third residual connection submodule, and the first downsampling submodule is connected to the last second residual connection submodule; the second merging channel submodule includes: a third merging channel unit and a fourth merging channel unit; the third merging channel unit is connected to the second Conv unit, and the fourth merging channel unit is connected to the first Conv unit;

[0030] The downsampling submodule is used to perform a downsampling operation on the output result of the residual connection submodule;

[0031] The second merging channel submodule is used to fuse the output result of the downsampling submodule with the output result of the second Conv submodule;

[0032] The third residual connection submodule is used to accelerate the training and reasoning speed of the network while preventing gradient explosion or gradient zero.

[0033] Optionally, the Neck module further includes:

[0034] Multiple CA submodules, each of which is sequentially provided with: a first CA unit, a second CA unit, and a third CA unit; the first CA unit is connected to the second residual connection submodule at the end, and the second CA unit and the third CA unit are respectively connected to the third residual connection submodule;

[0035] The CA submodule is used to calculate the attention weights after the feature maps are spliced, assign the attention weights to the original feature maps, and output three feature layers.

[0036] Optionally, calculating the attention weight after concatenating the feature maps and assigning the attention weight to the original feature maps includes:

[0037] Perform global average pooling on the feature maps in the horizontal and vertical directions:

[0038]

[0039] Among them, X h 、X w Represent the results of average pooling in the horizontal and vertical directions respectively. Pool(X,(H,1)) and Pool(X,(1,W)) represent the average pooling in the height and width directions respectively, that is, calculating the average value of all values ​​in the pooling window. X represents the input data, H represents the size of the pooling window in the height direction, and W represents the size of the pooling window in the width direction.

[0040] The horizontal and vertical feature maps are concatenated and transformed and the attention weights in both directions are calculated:

[0041] F=act(BN(conv(contact(X h ,X w ))))

[0042] g h =split(F h )

[0043] g w =split(F w )

[0044] Among them, F represents the feature map after average pooling, g h 、g w Represents the attention weights in height and width directions, BN represents batch normalization, conv represents convolution, contact(X h ,X w ) represents the splicing operation, F h ,F represents the division of F in the height and width directions;

[0045] Apply the attention weights in two directions to the original feature map:

[0046]

[0047] Among them, ⊙ represents the hadamard product of the matrix, that is, the multiplication of the elements in corresponding positions.

[0048] Optionally, the Head module includes:

[0049] a plurality of detection heads, the plurality of detection heads being connected to the first CA unit, the second CA unit, and the third CA unit, respectively;

[0050] The detection head is used to obtain the surface defect detection result according to the characteristic layer.

[0051] Optionally, the Head module further includes a C-IoU measurement to measure the positioning loss of the predicted box;

[0052] The expression of C-IoU is:

[0053]

[0054] Among them, ρ represents the Euclidean distance between the center point of the predicted box and the real box, l represents the diagonal length of the minimum rectangle surrounding the predicted box and the real box, v represents the similarity of the aspect ratio of the predicted box and the real box, α represents the influencing factor of the aspect ratio of the predicted box and the real box, and IoU represents the intersection-over-union ratio, which is the area of ​​the intersection divided by the maximum effective area occupied by the two boxes.

[0055] The beneficial effects of the present invention are:

[0056] Aiming at the problem of surface defect detection on large-scale infrastructure, bridges and other buildings, the present invention uses neural networks to learn the characteristics of different types of defects, constructs a deep learning network for defect detection, and installs it on a robot operating system. Through the movement of the robot, the video stream of the visual sensor is used to identify whether there are defects on the surface of the building and their types.

[0057] In response to the current problems of heavy workload, low efficiency, and serious false positives in manual inspections of high-altitude building surfaces, the present invention proposes a method for intelligent robot-based inspection of surface defects in high-altitude buildings. This method incorporates a spatial attention mechanism to improve feature extraction capabilities, uses lightweight convolutional computation to reduce the number of parameters and increase inference speed, enables the deployment of lightweight equipment, and ensures recognition accuracy and efficiency. A model is pre-trained using images containing various types of defects, which is then loaded onto the computing device of a lightweight inspection robot. The robot then carries a visual sensor to various areas of the building surface to complete defect identification and calibration. This improves inspection efficiency, provides more reliable data support for high-altitude building inspections, and ensures the safety of building inspection workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 This is a network structure diagram of a high-altitude building surface defect detection model according to an embodiment of the present invention;

[0060] Figure 2 This is a curve diagram of the learning rate change of the model training according to an embodiment of the present invention;

[0061] Figure 3 A curve diagram showing the change of the loss function during the training process of an embodiment of the present invention;

[0062] Figure 4 This is a graph showing changes in average accuracy during the model training process according to an embodiment of the present invention;

[0063] Figure 5 This is a diagram showing the detection effect of a high-altitude building according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] like Figure 1 As shown, this embodiment discloses a robot intelligent detection method for surface defects of high-altitude buildings, including: obtaining a surface defect image to be detected, inputting the surface defect image to be detected into a surface defect detection model of a high-altitude building carried by the robot, and obtaining a surface defect detection result; the surface defect detection model of the high-altitude building is trained using a training set; the training set includes: surface defect images and defect labels; the surface defect detection model of the high-altitude building uses lightweight convolution to extract features, and introduces a spatial attention mechanism to calculate the attention weight after the feature map is spliced, and the attention weight is assigned to the original feature map.

[0067] Constructing a dataset for surface defect detection of high-altitude buildings includes:

[0068] A well-characterized, sufficient, and representative building defect dataset has a profound impact on the algorithm's identification results. Consider the operating conditions of large-scale infrastructure, such as bridges: they are subject to pressure from their own weight, horizontal and vertical forces from wind, the impact of environmental factors such as earthquakes and wind on the building's base, thermal expansion and contraction due to temperature fluctuations, fatigue loads during operation, and preload from sediment and the building itself. These forces can exert bending moments, shear forces, compressive forces, and tensile forces on the building's surface and internal beams, causing surface deformation, shearing deformation, cracking, and loosening of bolts and other fasteners. Physical and chemical reactions during moisture absorption, migration, and evaporation can lead to material damage, manifesting as peeling, mold, and rust. Therefore, these potential destructive factors should be fully considered during inspection to ensure safety and durability under various loads. The present invention aims to conduct comprehensive and efficient inspection of the surface of high-altitude buildings. Therefore, network and field pattern collection of surface defects such as cracks, damaged connections and rust of high-altitude buildings is carried out to construct a specified high-altitude building surface defect detection dataset to detect these defects.

[0069] The present invention has the following requirements for the surface defect detection dataset:

[0070] (1) Diversity: This includes all types of defects that need to be detected, including cracks, dents, rust, loose connections, etc. Each defect should have different sizes and shapes, as well as postures at different angles and distances;

[0071] (2) Balance: The number of samples of various defects should be relatively balanced. If the number of samples of a certain category is too large, overfitting problems will occur;

[0072] (3) Representativeness: The various defect features in the image should be representative and obvious;

[0073] (4) Data volume: The number of images has a certain scale. Generally speaking, the larger the dataset, the stronger the generalization ability of the model.

[0074] At the same time, the resolution of the image should be clear. Clear images can provide more effective information. The dataset should minimize incorrect annotations and incorrect labels. Before training, strict label data cleaning is required to ensure the accuracy of learning.

[0075] Defect samples obtained through various collection methods include images with unclear images, poor image quality, erroneous image content, and repeated content. Each original image contains different information and the location of defects is also different. To ensure that the normalized data contains at least one defect type, a screening method must be used first to purposefully select images containing defects from all images and filter out samples with obvious defects.

[0076] Although the model's input supports training data of varying sizes, adaptive scaling and adding black borders can address the issue of varying input image sizes. These processes increase image redundancy, requiring increased computational effort. Varying sizes can also affect model training performance. Considering that this model is deployed on automated inspection equipment, where the onboard camera transmits video, all images are normalized and resized to the same resolution as the input device for training. Images are resized to 640*480 to ensure consistency between training and prediction data, improving model detection accuracy to a certain extent. Images are then randomly cropped or resized to intentionally retain target areas, remove irrelevant background information, and highlight defective content. After obtaining the normalized and processed images, labels are applied to the images, with rectangular boxes drawn to mark and classify defects. Finally, the dataset is partitioned into training, validation, and test sets in an 8:1:1 ratio.

[0077] Furthermore, the surface defect detection model for high-altitude buildings includes: a Backbone module, which is used to extract several feature layers from the surface defect image to be detected; a Neck module, which is used to fuse the features of several feature layers to obtain multiple prediction values; and a Head module, which is used to decode the multiple prediction values ​​to obtain the surface defect detection results.

[0078] Furthermore, the Backbone module includes: a first Conv submodule and a first residual connection submodule, the first Conv submodule and the first residual connection submodule are alternately arranged, and the SPPF submodule is connected after the first residual connection submodule at the end; the first Conv submodule is used to extract features using lightweight convolution, and the feature map obtained by convolution is subjected to one convolution, batch normalization and activation function operation; the first residual connection submodule is used to maintain the uniformity of the feature extraction results; the SPPF submodule is used to aggregate the features on the last feature layer output in the Backbone module; wherein, the first residual connection submodule is sequentially provided with: a first residual connection unit, a second residual connection unit and a third residual connection unit.

[0079] Furthermore, the Neck module includes: multiple second Conv sub-modules, upsampling sub-modules, first merging channel sub-modules and second residual connection sub-modules, which are connected in sequence according to the second Conv sub-module, upsampling sub-module, first merging channel sub-module and second residual connection sub-module; the second Conv sub-module is sequentially provided with: a first Conv unit and a second Conv unit; the first Conv unit is connected to the SPPF sub-module of the Backbone module; the first merging channel sub-module is sequentially provided with: a first merging channel unit and a second merging channel unit; the first merging channel unit is connected to the second residual connection unit, and the second merging channel unit is connected to the first residual connection unit; the second Conv sub-module is used to extract features using lightweight convolution, and perform a convolution, batch normalization and activation function operation on the feature map obtained by convolution; the upsampling sub-module is used to upsample the output result of the second Conv sub-module; the first merging channel sub-module is used to fuse the output result of the upsampling sub-module with the output result of the residual connection sub-module; the second residual connection sub-module is used to maintain the uniformity of the fusion result.

[0080] Furthermore, the Neck module also includes: multiple downsampling submodules, a second merging channel submodule and a third residual connection submodule, which are connected in sequence according to the downsampling submodule, the second merging channel submodule and the third residual connection submodule, and the first downsampling submodule is connected to the second residual connection submodule at the end; the second merging channel submodule includes: a third merging channel unit and a fourth merging channel unit; the third merging channel unit is connected to the second Conv unit, and the fourth merging channel unit is connected to the first Conv unit; the downsampling submodule is used to perform a downsampling operation on the output result of the residual connection submodule; the second merging channel submodule is used to fuse the output result of the downsampling submodule with the output result of the second Conv submodule; the third residual connection submodule is used to maintain the unity of the fusion result.

[0081] Furthermore, the Neck module also includes: multiple CA sub-modules, and the CA sub-modules are sequentially provided with: a first CA unit, a second CA unit and a third CA unit; the first CA unit is connected to the second residual connection sub-module at the end, and the second CA unit and the third CA unit are respectively connected to the third residual connection sub-module; the CA sub-module is used to calculate the attention weight after the feature map is spliced, assign the attention weight to the original feature map, and output three feature layers.

[0082] Furthermore, the Head module includes: multiple detection heads, which are respectively connected to the first CA unit, the second CA unit and the third CA unit; the detection heads are used to obtain surface defect detection results based on the feature layer.

[0083] Specifically:

[0084] Build a lightweight model for surface defect detection of high-altitude buildings.

[0085] The network structure of the detection model proposed in this paper can be divided into the following components: Backbone, Neck, and Head. The Backbone is the model's main feature extraction network. The network extracts three feature layers from the image from the front to the back for target detection. The obtained feature layers are then fed into the Neck enhanced feature fusion network to generate several predicted values. The final prediction result is obtained through subsequent decoding of the predicted values. The optimizer updates the model's weight parameters based on the loss between the predicted results and the actual labels, achieving a learning effect.

[0086] The model of the present invention uses a 6*6 convolution layer to slice the input layer to obtain a feature map with 32 channels. Because the 6*6 convolution kernel has a relatively large receptive field, it is conducive to capturing richer information, improving the generalization ability of the model on unknown samples, and making the network structure more concise, easy to understand and implement.

[0087] For an input feature map x, its dimension is (b, c, w, h), and the specific calculation method is:

[0088]

[0089] The convolution-derived feature map is subjected to a convolution + batch normalization + activation function operation, making the output nonlinear. This prevents the gradient from reaching zero or exploding, stabilizes the update of the convolution kernel weights, and mitigates the rapid decrease in gradients as the network depth increases, improving the stability of the model. This combination is implemented before the residual network in this model to maintain the uniformity of the input features. The formula is:

[0090] X'=W*X+b

[0091]

[0092] Among them, W represents the weight of the convolution layer, X represents the input data, and b represents the bias of the convolution layer. represents the result of batch normalization output, γ and β are adjustable parameters, which are used to scale and translate the normalized data, μ represents the mean of the current batch data, σ represents the standard deviation of the current batch data, ε is a small constant to prevent division by 0, and Y represents the final output result of 0 and The larger value.

[0093] The final step of the feature extraction network incorporates the SPPF structure, using a fixed-size convolutional kernel to aggregate features from different feature layers within the network. All feature maps are then concatenated at the final layer. First, a convolutional layer reduces the number of input channels from c1 by half to reduce computational effort. Three max pooling layers are then used to aggregate features. Three 5x5 pooling kernels are used with a stride of 1 and padding to half the kernel size. The outputs of the pooling layers are then concatenated, and a convolutional layer is used to convert the number of channels to c2. This achieves more than double the efficiency of the traditional SPP.

[0094] In the defect detection model, the main means of feature fusion is achieved through Neck. The Neck part is used to divide the large, medium and large-scale information of building surface defects. For example, if there are areas where defects are more concentrated, such as some large defects and small defects overlapping each other, and the scale span of various defects is large, a picture may contain large-scale and small-scale defect information at the same time. If these different-scale information are predicted using only one scale of feature information, it is obvious that the effect is not good, because the feature map of a certain level is only suitable for the inspection of a certain part of the scale information. The bottom-level feature map is suitable for capturing small defects, and the high-level feature map is used to capture large targets. If the receptive field is large, the model may lose the ability to detect small targets. If the receptive field is small, the model will miss or misdetect larger defects. In order to enable the model to simultaneously inspect targets of different scales, the present invention designs a feature fusion network of FPN+PAN+CA to fuse feature maps of different levels. The mathematical model of the spatial attention mechanism is expressed as follows:

[0095] (1) First perform global average pooling in the horizontal and vertical directions:

[0096]

[0097] Among them, X h 、X wRepresent the results of average pooling in the horizontal and vertical directions respectively. Pool(X,(H,1)) and Pool(X,(1,W)) represent the average pooling in the height and width directions respectively, that is, calculating the average value of all values ​​in the pooling window. X represents the input data, H represents the size of the pooling window in the height direction, and W represents the size of the pooling window in the width direction.

[0098] (2) Concatenate the horizontal and vertical feature maps, transform them, and calculate the attention weights in the two directions:

[0099] F=act(BN(conv(contact(X h ,X w ))))

[0100] g h =split(F h )

[0101] g w =split(F w )

[0102] Among them, F represents the feature map after average pooling, g h 、g w Represents the attention weights in height and width directions, BN represents batch normalization, conv represents convolution, contact(X h ,X w ) represents the splicing operation, F h 、F w represents the division of F in height and width directions;

[0103] (3) Apply the attention weights to the original feature map X:

[0104]

[0105] Neck outputs three feature layers with shapes of (256, 80, 60, 30), (512, 40, 30, 30), and (1024, 20, 15, 30). In the model, each feature point on each feature layer has three prior bounding boxes, responsible for predicting small, medium, and large objects, respectively. The last parameter of each feature layer can be split into 3*(5+5). The first 5 represents the model's predicted parameters: the relative offset of the prior bounding box from the initial values ​​in the x and y directions, the relative change in the predicted box's width and height from the preset values, and whether the predicted box contains an object. The bounding box regressor uses two 1*1 convolutions to obtain the center coordinate offset and size of the predicted bounding box, respectively. The classifier uses a 1*1 convolution layer to convert certain channels in the feature map into predicted probabilities for each class. The second 5 represents the probability of each of the five defect types predicted by the model, indicating the probability of each defect within the box.

[0106] After the model gives a prediction result, an indicator is needed to measure the quality of the model's prediction. The Head also contains a loss function to express a quantitative indicator of the degree to which the model's prediction result is inconsistent with the true result. The size of the loss value represents the quality of the model's prediction result. The larger the loss value, the worse the prediction result, and vice versa. In the training process of the model, the specific meaning of the loss function is to characterize the quality of the model's learning state. Specifically, it is necessary to use this indicator as a clue to update the model's weight parameters and find the weight of the fitting result closest to the true value, that is, the weight that makes the model's loss value as small as possible. The present invention uses C-IoU to measure the positioning loss of the prediction box. The expression formula of the C-IoU loss function is as follows:

[0107]

[0108] Where: ρ is the Euclidean distance between the center of the predicted box and the real box, l is the diagonal length of the minimum rectangle surrounding the predicted box and the real box, v is the similarity of the aspect ratio between the predicted box and the real box, and α is the influencing factor of the aspect ratio between the predicted box and the real box. The expressions are:

[0109]

[0110] where x i1 、x i2 、y i1 、y i2 represents the coordinates of the upper left corner and lower right corner of the i-th bounding box, i=l,p.

[0111] The present invention also uses binary cross entropy loss to penalize the difference between the defect category probability predicted by each prediction box and the true label. The expression formula is as follows:

[0112]

[0113] Train a model for detecting surface defects in high-altitude buildings.

[0114] The present invention adopts a method of public dataset training + self-built building surface damage dataset secondary training. The model is first pre-trained using an open source dataset, and the pre-trained weights are used as initialization parameters for training this model. The dataset used in the present invention is the source code and dataset of the building structure damage detection system, which contains 1,800 images, covering three main types of damage: corrosion, cracks, and spalling. According to the characteristics of this model and the high-altitude building surface defect dataset, the entire network of the model is first trained using a public dataset to obtain a pre-trained model, and then the weights of the model trained from the public dataset are transferred to the environment of the present invention for secondary training.

[0115] like Figure 2 As shown, the learning rate change strategy of the model is an important part of the training process, and appropriate learning rate changes can effectively improve the training effect of the model. The algorithm of the present invention uses a cosine annealing learning rate to dynamically adjust the learning rate of the training process. At the beginning of training, the model will adopt a smaller learning rate to allow the weight to change smoothly from the initial irregular state, preventing a larger learning rate from causing instability and oscillation in the weight change. In this stage, the learning rate will change linearly to the preset value. After the warm-up phase, the adjustment of the learning rate is similar to the cosine curve, slowly increasing from a smaller value, then increasing rapidly, and finally increasing slowly. At the beginning of training, a smaller learning rate is needed to ensure the stability of the network, then accelerate learning, reach near the optimal value, and then slowly reduce the learning rate until it slowly approaches the optimal solution. This learning rate adjustment strategy conforms to the law of model learning and can effectively train this model.

[0116] The present invention will be further described below with reference to the accompanying drawings and examples.

[0117] To develop an intelligent robotic detection algorithm for high-altitude building surface defects, the present invention collects building defect samples through a combined online and field collection method. The collected samples are normalized and image enhancement and data augmentation methods are combined to obtain labeled samples of building defect datasets. These samples are then categorized and labeled according to the characteristics of different defects, creating a dedicated dataset to improve detection capabilities for specific buildings. This dataset contains surface damage for a specific building and similar structures. In the second step, the proposed neural network model is trained and validated using both publicly available and self-constructed datasets.

[0118] The working environment configuration composition of this embodiment is shown in Table 1.

[0119] Table 1

[0120]

[0121]

[0122] Figure 3 The classification loss curve for the training process of this invention is shown. The model converges quickly in the first 20 or so epochs, corresponding to a period of rapid learning rate change. After 50 epochs, the loss changes more slowly, and the model gradually converges. At the end of training, the validation set loss is higher than the training set loss. This is because the validation set is only used to calculate model evaluation metrics and does not participate in training. In fact, the accuracy of all neural network validation sets is lower than that of the training set, which is normal.

[0123] Finally, mAP@.5 is used to represent the comprehensive performance of the model. Figure 4 The figure shows the changes in mAP values ​​during the training of the defect detection model of the present invention. The dashed line in the figure represents the fitting curve of the true value. During the first 20 epochs, the mAP@0.5 value rises rapidly, indicating that the model learns effectively in the early stages of training. Between epochs 20 and 40, the mAP@0.5 value fluctuates significantly, which is due to the influence of the model's weight adjustment and data augmentation training strategy. After 40 epochs, the fluctuation of the mAP@0.5 value begins to decrease, the model performance tends to stabilize, and the model's detection performance continues to improve, reaching a high value. Figure 5 The results show that the robot intelligent detection algorithm for surface defects of high-altitude buildings proposed in this invention can identify various defects on buildings with high accuracy.

[0124] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A robot intelligent detection method for surface defects of high-altitude buildings, characterized in that: include: Acquire a surface defect image to be detected, input the surface defect image to be detected into a high-altitude building surface defect detection model carried by the robot, and obtain a surface defect detection result; The high-altitude building surface defect detection model is obtained by training using a training set; The training set includes: surface defect images and defect labels; The high-altitude building surface defect detection model includes: A Backbone module is used to extract several feature layers from the surface defect image to be detected; Neck module, used to perform feature fusion on the several feature layers to obtain multiple prediction values; A Head module, configured to decode the multiple predicted values ​​to obtain the surface defect detection result; The Neck module also includes: Multiple CA submodules, each of which is sequentially provided with: a first CA unit, a second CA unit, and a third CA unit; the first CA unit is connected to the second residual connection submodule at the end, and the second CA unit and the third CA unit are respectively connected to the third residual connection submodule; The CA submodule is used to calculate the attention weights after the feature maps are spliced, assign the attention weights to the original feature maps, and output three feature layers; Calculating the attention weight after the feature map is spliced, and assigning the attention weight to the original feature map includes: Perform global average pooling on the feature maps in the horizontal and vertical directions: Among them, X h 、X w Represent the results of average pooling in the horizontal and vertical directions respectively. Pool(X,(H,1)) and Pool(X,(1,W)) represent the average pooling in the height and width directions respectively, that is, calculating the average value of all values ​​in the pooling window. X represents the input data, H represents the size of the pooling window in the height direction, and W represents the size of the pooling window in the width direction. The horizontal and vertical feature maps are concatenated and transformed and the attention weights in both directions are calculated: F=act(BN(conv(contact(X h ,X w )))) g h =split(F h ) g w =split(F w ) Among them, F represents the feature map after average pooling, g h 、g w Represents the attention weights in height and width directions, BN represents batch normalization, conv represents convolution, contact(X h ,X w ) represents the splicing operation, F h 、F w represents the division of F in height and width directions; Apply the attention weights in two directions to the original feature map: Among them, ⊙ represents the hadamard product of the matrix, that is, the multiplication of the elements at corresponding positions; The high-altitude building surface defect detection model uses lightweight convolution for feature extraction, introduces a spatial attention mechanism to calculate the attention weights after feature map splicing, and assigns the attention weights to the original feature maps.

2. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 1 is characterized in that: The Backbone module includes: A first Conv submodule and a first residual connection submodule, wherein the first Conv submodule and the first residual connection submodule are alternately arranged, and the SPPF submodule is connected after the first residual connection submodule at the end; The first Conv submodule is used to extract features using lightweight convolution, and perform convolution, batch normalization and activation function operations on the feature map obtained by convolution; The first residual connection submodule is used to maintain the uniformity of feature extraction results; The SPPF submodule is used to aggregate the features on the last feature layer output by the Backbone module; The first residual connection submodule is sequentially provided with: a first residual connection unit, a second residual connection unit and a third residual connection unit.

3. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 2 is characterized in that: Maintaining the uniformity of the feature extraction results includes: X'=w*X+b Among them, w represents the weight of the convolution layer, X represents the input data, and b represents the bias of the convolution layer. represents the result of batch normalization output, γ and β are adjustable parameters, which are used to scale and translate the normalized data, μ represents the mean of the current batch data, σ represents the standard deviation of the current batch data, ε is a small constant to prevent division by 0, and Y represents the final output result of 0 and The larger value.

4. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 2 is characterized in that: The Neck module includes: A plurality of second Conv submodules, upsampling submodules, first merging channel submodules and second residual connection submodules are connected in sequence according to the order of the second Conv submodule, the upsampling submodule, the first merging channel submodule and the second residual connection submodule; the second Conv submodule is sequentially provided with: a first Conv unit and a second Conv unit; the first Conv unit is connected to the SPPF submodule of the Backbone module; the first merging channel submodule is sequentially provided with: a first merging channel unit and a second merging channel unit; the first merging channel unit is connected to the second residual connection unit, and the second merging channel unit is connected to the first residual connection unit; The second Conv submodule is used to extract features using lightweight convolution, and perform convolution, batch normalization and activation function operations on the feature map obtained by convolution; The upsampling submodule is configured to perform an upsampling operation on the output result of the second Conv submodule; The first merging channel submodule is configured to fuse the output result of the upsampling submodule with the output result of the residual connection submodule along the channel dimension; The second residual connection submodule is used to accelerate the training and reasoning speed of the network while preventing gradient explosion or gradient zero.

5. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 4 is characterized in that: The Neck module also includes: A plurality of downsampling submodules, a second merging channel submodule and a third residual connection submodule are connected in sequence according to the downsampling submodule, the second merging channel submodule and the third residual connection submodule, and the first downsampling submodule is connected to the last second residual connection submodule; the second merging channel submodule includes: a third merging channel unit and a fourth merging channel unit; the third merging channel unit is connected to the second Conv unit, and the fourth merging channel unit is connected to the first Conv unit; The downsampling submodule is used to perform a downsampling operation on the output result of the residual connection submodule; The second merging channel submodule is used to fuse the output result of the downsampling submodule with the output result of the second Conv submodule; The third residual connection submodule is used to accelerate the training and reasoning speed of the network while preventing gradient explosion or gradient zero.

6. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 1 is characterized in that: The Head module includes: a plurality of detection heads, the plurality of detection heads being connected to the first CA unit, the second CA unit, and the third CA unit, respectively; The detection head is used to obtain the surface defect detection result according to the characteristic layer.

7. The method for intelligent detection of surface defects of high-altitude buildings by a robot according to claim 6, characterized in that: The Head module also includes C-IoU to measure the positioning loss of the prediction box; The expression of C-IoU is: Among them, ρ represents the Euclidean distance between the center point of the predicted box and the real box, l represents the diagonal length of the minimum rectangle surrounding the predicted box and the real box, v represents the similarity of the aspect ratio of the predicted box and the real box, α represents the influencing factor of the aspect ratio of the predicted box and the real box, and IoU represents the intersection-over-union ratio, which is the area of ​​the intersection divided by the maximum effective area occupied by the two boxes.

Images