High-altitude building surface defect robot intelligent detection method

Abstract

The invention relates to a high-altitude building surface defect robot intelligent detection method, which comprises the following steps: acquiring a to-be-detected surface defect image, inputting the to-be-detected surface defect image into a high-altitude building surface defect detection model carried by a robot, and acquiring a surface defect detection result; the high-altitude building surface defect detection model is obtained through training of a training set; the training set comprises a surface defect image and a defect label; the high-altitude building surface defect detection model uses lightweight convolution to perform feature extraction, introduces a space attention mechanism to calculate attention weight after feature map splicing, and endows the attention weight to an original feature map. The method utilizes the neural network to learn the characteristics of different types of defects, constructs a deep learning network for defect detection, carries the deep learning network on a robot operation system, and identifies whether the building surface has defects and the types of the defects by means of video streams of a visual sensor through the movement of a robot.

Description

Technical Field

[0001] The invention relates to the technical field of high-altitude building quality detection, and in particular to a high-altitude building surface defect robot intelligent detection method. Background Art

[0002] Large infrastructure equipment, bridges and other construction facilities have a specified service life. During this period, in order to ensure the normal and safe operation of large infrastructure equipment, bridges, etc., they must be regularly inspected for safety. There are generally no special protection measures for the surface of such large facilities. They are easily affected by extreme weather, irregular fatigue loads and unexpected situations during operation. If an accident occurs or fatigue stress is generated internally, cracks will form on the surface, which is also reflected in the stability of the connection device. Traditional building surface inspection methods mainly focus on artificial high-altitude inspection, which is detected by visual inspection or measuring instruments. However, this method takes a long time to work, the area that can be inspected per unit time is small, and there are certain safety hazards. It consumes a lot of time and economic costs. Summary of the invention

[0003] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a robot intelligent detection method for surface defects of high-altitude buildings, which uses a deep learning network to extract features. Different from traditional manual and professional equipment detection, it requires filter denoising and grayscale calculation of the building surface.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A robot intelligent detection method for surface defects of high-altitude buildings, comprising:

[0006] Acquire a surface defect image to be detected, input the surface defect image to be detected into a high-altitude building surface defect detection model carried by the robot, and obtain a surface defect detection result; the high-altitude building surface defect detection model is trained using a training set; the training set includes: a surface defect image and a defect label;

[0007] The high-altitude building surface defect detection model uses lightweight convolution for feature extraction, introduces a spatial attention mechanism to calculate the attention weights after the feature maps are spliced, and assigns the attention weights to the original feature maps.

[0008] Optionally, the high-altitude building surface defect detection model includes:

[0009] A Backbone module, used for extracting a plurality of feature layers from the surface defect image to be detected;

[0010] Neck module, used for fusing features of the several feature layers to obtain multiple prediction values;

[0011] The Head module is used to decode the multiple prediction values ​​to obtain the surface defect detection result.

[0012] Optionally, the Backbone module includes:

[0013] A first Conv submodule and a first residual connection submodule, wherein the first Conv submodule and the first residual connection submodule are alternately arranged, and the SPPF submodule is connected after the last first residual connection submodule;

[0014] The first Conv submodule is used to extract features using lightweight convolution, and perform one convolution, batch normalization and activation function operations on the feature map obtained by convolution;

[0015] The first residual connection submodule is used to maintain the uniformity of feature extraction results;

[0016] The SPPF submodule is used to aggregate the features on the last feature layer output by the Backbone module;

[0017] The first residual connection submodule is sequentially provided with: a first residual connection unit, a second residual connection unit and a third residual connection unit.

[0018] Optionally, maintaining the uniformity of the feature extraction result includes:

[0019] X'=W*X+b

[0020]

[0021] Among them, W represents the weight of the convolutional layer, X represents the input data, and b represents the bias of the convolutional layer. represents the result of batch standardization output, γ and β are adjustable parameters, which are used to scale and translate the standardized data, μ represents the mean of the current batch data, σ represents the standard deviation of the current batch data, ε is a very small constant to prevent division by 0, and Y represents the final output result of 0 and The larger value.

[0022] Optionally, the Neck module includes:

[0023] A plurality of second Conv submodules, upsampling submodules, first merged channel submodules and second residual connection submodules are sequentially connected in the order of the second Conv submodule, the upsampling submodule, the first merged channel submodule and the second residual connection submodule; the second Conv submodule is sequentially provided with: a first Conv unit and a second Conv unit; the first Conv unit is connected to the SPPF submodule of the Backbone module; the first merged channel submodule is sequentially provided with: a first merged channel unit and a second merged channel unit; the first merged channel unit is connected to the second residual connection unit, and the second merged channel unit is connected to the first residual connection unit;

[0024] The second Conv submodule is used to extract features by using light convolution, and perform one convolution, batch normalization and activation function operations on the feature map obtained by convolution;

[0025] The upsampling submodule is used to perform an upsampling operation on the output result of the second Conv submodule;

[0026] The first merging channel submodule is used to fuse the output result of the upsampling submodule with the output result of the residual connection submodule along the channel dimension;

[0027] The second residual connection submodule is used to accelerate the training and reasoning speed of the network while preventing the gradient from exploding or being zero.

[0028] Optionally, the Neck module further includes:

[0029] A plurality of downsampling submodules, a second merging channel submodule and a third residual connection submodule are connected in sequence according to the downsampling submodule, the second merging channel submodule and the third residual connection submodule, and the first downsampling submodule is connected to the second residual connection submodule at the end; the second merging channel submodule includes: a third merging channel unit and a fourth merging channel unit; the third merging channel unit is connected to the second Conv unit, and the fourth merging channel unit is connected to the first Conv unit;

[0030] The downsampling submodule is used to perform a downsampling operation on the output result of the residual connection submodule;

[0031] The second merging channel submodule is used to fuse the output result of the downsampling submodule with the output result of the second Conv submodule;

[0032] The third residual connection submodule is used to accelerate the training and reasoning speed of the network while preventing the gradient from exploding or being zero.

[0033] Optionally, the Neck module further includes:

[0034] A plurality of CA submodules, wherein the CA submodules are sequentially provided with: a first CA unit, a second CA unit and a third CA unit; the first CA unit is connected to the second residual connection submodule at the end, and the second CA unit and the third CA unit are respectively connected to the third residual connection submodule;

[0035] The CA submodule is used to calculate the attention weights after the feature maps are concatenated, assign the attention weights to the original feature maps, and output three feature layers.

[0036] Optionally, calculating the attention weights after the feature map concatenation and assigning the attention weights to the original feature maps includes:

[0037] Perform global average pooling on the feature maps in the horizontal and vertical directions:

[0038]

[0039] Among them, X h , X w Represent the results of average pooling in the horizontal and vertical directions respectively. Pool(X,(H,1)) and Pool(X,(1,W)) represent average pooling in the height and width directions respectively, that is, calculating the average value of all values ​​in the pooling window. X represents the input data, H represents the size of the pooling window in the height direction, and W represents the size of the pooling window in the width direction.

[0040] The horizontal and vertical feature maps are concatenated and transformed and the attention weights in both directions are calculated:

[0041] F=act(BN(conv(contact(X h ,X w ))))

[0042] g h =split(F h )

[0043] g w =split(F w )

[0044] Among them, F represents the feature map after average pooling, g h , g w represents the attention weights in the height and width directions, BN represents batch normalization, conv represents convolution, contact(X h ,X w ) represents the splicing operation, F h , F represents the division of F in the height and width directions;

[0045] Apply the attention weights in two directions to the original feature map:

[0046]

[0047] Among them, ⊙ represents the hadamard product of the matrix, that is, the multiplication of elements in corresponding positions.

[0048] Optionally, the Head module includes:

[0049] a plurality of detection heads, wherein the plurality of detection heads are respectively connected to the first CA unit, the second CA unit and the third CA unit;

[0050] The detection head is used to obtain the surface defect detection result according to the characteristic layer.

[0051] Optionally, the Head module further includes a C-IoU to measure the positioning loss of the prediction box;

[0052] The expression of C-IoU is:

[0053]

[0054] Among them, ρ represents the Euclidean distance between the center point of the predicted box and the real box, l represents the diagonal length of the minimum rectangle surrounding the predicted box and the real box, v represents the similarity of the aspect ratio of the predicted box and the real box, α represents the influencing factor of the aspect ratio of the predicted box and the real box, and IoU represents the intersection over union ratio, which is the area of ​​the intersection divided by the maximum effective area occupied by the two boxes.

[0055] The beneficial effects of the present invention are:

[0056] Aiming at the problem of surface defect detection of large-scale infrastructure, bridges and other buildings, the present invention uses neural networks to learn the characteristics of different types of defects, constructs a deep learning network for defect detection, and mounts it on a robot operating system. Through the movement of the robot and with the help of the video stream of the visual sensor, it can identify whether there are defects on the surface of the building and their types.

[0057] In view of the current problems of large workload, low efficiency, and serious false detection in manual inspection of the surface of high-altitude buildings, the present invention proposes a robot intelligent inspection method for surface defects of high-altitude buildings, which integrates the spatial attention mechanism to improve the feature extraction capability, uses lightweight convolution calculation to reduce the number of parameters and improve the reasoning speed, realizes the deployment of lightweight equipment, and can also ensure recognition accuracy and efficiency. A model is pre-trained using images containing various types of defects, and then the model is mounted on the computing device of a lightweight inspection robot, which carries a visual sensor to various areas on the surface of the building to complete the identification and calibration of defects, which can improve the inspection efficiency, provide more reliable data support for high-altitude building inspection, and ensure the safety of building inspection staff. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0059] Figure 1 A network structure diagram of a high-altitude building surface defect detection model according to an embodiment of the present invention;

[0060] Figure 2 This is a curve diagram of the learning rate change of the model training of an embodiment of the present invention;

[0061] Figure 3 A loss function variation curve diagram of the training process of an embodiment of the present invention;

[0062] Figure 4 A graph showing changes in average accuracy during the model training process of an embodiment of the present invention;

[0063] Figure 5 This is a diagram showing the detection effect of a high-altitude building according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] like Figure 1 As shown, this embodiment discloses a robot intelligent detection method for surface defects of high-altitude buildings, including: obtaining a surface defect image to be detected, inputting the surface defect image to be detected into a surface defect detection model of a high-altitude building carried by the robot, and obtaining a surface defect detection result; the surface defect detection model of the high-altitude building is trained using a training set; the training set includes: surface defect images and defect labels; the surface defect detection model of the high-altitude building uses lightweight convolution to extract features, and introduces a spatial attention mechanism to calculate the attention weight after the feature map is spliced, and the attention weight is assigned to the original feature map.

[0067] Constructing a high-altitude building surface defect detection dataset includes:

[0068] A building defect dataset with obvious characteristics, sufficient quantity and full representation has a profound impact on the recognition results of the algorithm. Consider the working conditions of large infrastructure equipment, bridges, etc.: they will be subject to the pressure of their own weight on the structure, the horizontal and vertical forces generated by the wind on the structure, the impact of environmental factors such as earthquakes and wind on the bottom of the building, thermal expansion and contraction caused by temperature changes, fatigue loads during work, and the preload of sediments and the building itself, etc., which will produce bending moments, shear forces, extrusion forces and tension forces on the building surface itself, internal beams, etc., causing surface deformation, shear deformation, surface cracking and loosening of bolts and other connecting devices; physical and chemical reactions occur during the process of absorbing moisture, migrating moisture, and evaporating moisture, leading to the destruction of building materials, which manifests as peeling, mildew, rust and other phenomena. Therefore, these potential destructive factors should be fully considered during detection to ensure its safety and durability under various loads. The present invention aims to conduct comprehensive and efficient inspection of the surface of high-altitude buildings. Therefore, network and field pattern collection is performed on surface defects such as cracks, connection damage and rust of high-altitude buildings, and a specified high-altitude building surface defect detection data set is constructed to detect these defects.

[0069] The present invention has the following requirements for the surface defect detection data set:

[0070] (1) Diversity: It includes all kinds of defects that need to be detected, including cracks, dents, rust, loose connections, etc. Each defect should have different sizes and shapes, and the postures of the defect at different angles and distances;

[0071] (2) Balance: The number of samples of various defects should be relatively balanced. If the number of samples of a certain category is too large, overfitting problems will occur;

[0072] (3) Representativeness: The various defect features in the image should be representative and obvious;

[0073] (4) Data volume: The number of images has a certain scale. Generally speaking, the larger the dataset, the stronger the generalization ability of the model.

[0074] At the same time, the resolution of the image should be clear. Clear images can provide more effective information. The data set should minimize incorrect annotations and wrong labels. Before training, strict label data cleaning is required to ensure the accuracy of learning.

[0075] The defect samples obtained through various collection methods include images with unclear images, poor image quality, erroneous image content, repeated content, etc. Each original image contains different information and the location of the defects is also different. In order to ensure that the normalized data contains at least one defect type, we must first use a screening method to purposefully select images containing defects from all images and filter out samples with obvious defects.

[0076] Although the input of this model can support the input of training data of different sizes, the problem of different input image sizes can be solved by adaptive scaling and adding black edges. These processes increase the redundant information of the image, which requires an increase in the amount of calculation. Different sizes will also affect the effect of model training. Considering that this model is deployed on an automated inspection device, since the camera device on it transmits video detection, the size of all images is normalized and adjusted to the same resolution as the input device for training. The size of all images is processed into 640*480, which can ensure the uniformity of training data and prediction data and improve the accuracy of model detection to a certain extent. Then the image is randomly cropped or scaled, the area where the target exists is purposefully retained, irrelevant background information is removed, and the defective content is highlighted. After obtaining the image after size normalization and data processing, the label information of the image is annotated, and a rectangular box is drawn to annotate the defects in the image and classify them. Finally, the data set is divided into training set, validation set and test set in a ratio of 8:1:1.

[0077] Furthermore, the high-altitude building surface defect detection model includes: a Backbone module, which is used to extract several feature layers from the surface defect image to be detected; a Neck module, which is used to fuse the features of several feature layers to obtain multiple prediction values; and a Head module, which is used to decode the multiple prediction values ​​to obtain the surface defect detection results.

[0078] Furthermore, the Backbone module includes: a first Conv submodule and a first residual connection submodule, the first Conv submodule and the first residual connection submodule are alternately arranged, and the SPPF submodule is connected to the first residual connection submodule at the end; the first Conv submodule is used to extract features using lightweight convolution, and the feature map obtained by convolution is subjected to one convolution, batch normalization and activation function operation; the first residual connection submodule is used to maintain the uniformity of the feature extraction results; the SPPF submodule is used to aggregate the features on the last feature layer output in the Backbone module; wherein the first residual connection submodule is sequentially arranged with: a first residual connection unit, a second residual connection unit and a third residual connection unit.

[0079] Furthermore, the Neck module includes: a plurality of second Conv submodules, an upsampling submodule, a first merging channel submodule and a second residual connection submodule, which are connected in sequence according to the order of the second Conv submodule, the upsampling submodule, the first merging channel submodule and the second residual connection submodule; the second Conv submodule is sequentially provided with: a first Conv unit and a second Conv unit; the first Conv unit is connected to the SPPF submodule of the Backbone module; the first merging channel submodule is sequentially provided with: a first merging channel unit and a second merging channel unit; the first merging channel unit is connected to the second residual connection unit, and the second merging channel unit is connected to the first residual connection unit; the second Conv submodule is used to extract features using lightweight convolution, and perform one convolution, batch normalization and activation function operations on the feature map obtained by convolution; the upsampling submodule is used to perform an upsampling operation on the output result of the second Conv submodule; the first merging channel submodule is used to fuse the output result of the upsampling submodule with the output result of the residual connection submodule; the second residual connection submodule is used to maintain the uniformity of the fusion result.

[0080] Furthermore, the Neck module also includes: multiple downsampling submodules, a second merging channel submodule and a third residual connection submodule, which are connected in sequence according to the downsampling submodule, the second merging channel submodule and the third residual connection submodule, and the first downsampling submodule is connected to the second residual connection submodule at the end; the second merging channel submodule includes: a third merging channel unit and a fourth merging channel unit; the third merging channel unit is connected to the second Conv unit, and the fourth merging channel unit is connected to the first Conv unit; the downsampling submodule is used to perform a downsampling operation on the output result of the residual connection submodule; the second merging channel submodule is used to fuse the output result of the downsampling submodule with the output result of the second Conv submodule; the third residual connection submodule is used to maintain the uniformity of the fusion result.

[0081] Furthermore, the Neck module also includes: multiple CA sub-modules, and the CA sub-modules are sequentially arranged with: a first CA unit, a second CA unit and a third CA unit; the first CA unit is connected to the second residual connection sub-module at the end, and the second CA unit and the third CA unit are respectively connected to the third residual connection sub-module; the CA sub-module is used to calculate the attention weight after the feature map is spliced, assign the attention weight to the original feature map, and output three feature layers.

[0082] Furthermore, the Head module includes: a plurality of detection heads, which are respectively connected to the first CA unit, the second CA unit and the third CA unit; and the detection heads are used to obtain surface defect detection results according to the feature layer.

[0083] Specifically:

[0084] Build a lightweight model for surface defect detection of high-altitude buildings.

[0085] The network structure of the detection model proposed in the present invention can be divided into the following parts: Backbone, Neck and Head. Backbone is the main feature extraction network of the model. The network extracts three feature layers from the image from front to back for target detection. After the obtained feature layers are strengthened by the Neck feature fusion network, several prediction values ​​are obtained. The final prediction result is obtained by subsequent decoding of the prediction value. The optimizer updates the weight parameters of the model according to the loss value of the prediction result and the actual label to achieve the learning effect.

[0086] The model of the present invention uses a 6*6 convolution layer to slice the input layer to obtain a feature map with 32 channels. Because the 6*6 convolution kernel has a relatively large receptive field, it is conducive to capturing richer information, improving the generalization ability of the model on unknown samples, and making the network structure more concise, easy to understand and implement.

[0087] For an input feature map x, its dimension is (b, c, w, h), and the specific calculation method is:

[0088]

[0089] The convolutional feature map is subjected to a convolution + batch normalization + activation function operation, and the output becomes nonlinear, preventing the gradient from being zero or exploding, making the update of the convolution kernel weight more stable, slowing down the phenomenon that the gradient decreases rapidly with the increase of the network depth, and improving the stability of the model. This model performs this combination before the residual network to maintain the uniformity of the input features. The formula is:

[0090] X'=W*X+b

[0091]

[0092] Among them, W represents the weight of the convolutional layer, X represents the input data, and b represents the bias of the convolutional layer. represents the result of batch standardization output, γ and β are adjustable parameters, which are used to scale and translate the standardized data, μ represents the mean of the current batch data, σ represents the standard deviation of the current batch data, ε is a very small constant to prevent division by 0, and Y represents the final output result of 0 and The larger value.

[0093] The last step of the feature extraction network is to add the SPPF structure, use a fixed-size convolution kernel to aggregate the features on different feature layers output in the network structure, and connect all feature maps at the last layer of the network. First, a convolution layer is used to reduce the input channels from c1 by half to reduce the amount of calculation. Then, three maximum pooling layers are used to aggregate the features. Three 5*5 pooling kernels are used with a step size of 1 and padded to half the kernel size. The outputs of the pooling layers are then connected, and the number of channels is converted to c2 through a convolution layer. Its efficiency is more than doubled compared to the traditional SPP.

[0094] In the defect detection model, the main means of feature fusion is achieved through Neck. The Neck part is used to divide the large, medium and large-scale information of the surface defects of the building. For example, there are areas where defects are more concentrated, such as some large defects and small defects overlap each other, and the scale span of various defects is large. A picture may contain large-scale and small-scale defect information at the same time. If these different-scale information only use one scale of feature information for prediction, it is obvious that the effect is not good, because the feature map of a certain level is only suitable for the inspection of a certain scale information. The bottom-level feature map is suitable for capturing small defects, and the high-level feature map is used to capture large targets. If the receptive field is large, the model may lose the ability to inspect small targets. If the receptive field is small, the model will miss or misdetect larger defects. In order to enable the model to simultaneously inspect targets of different scales, the present invention designs a feature fusion network of FPN+PAN+CA to fuse feature maps of different levels. The mathematical model of the spatial attention mechanism is expressed as follows:

[0095] (1) First, perform global average pooling in the horizontal and vertical directions:

[0096]

[0097] Among them, X h , X wRepresent the results of average pooling in the horizontal and vertical directions respectively. Pool(X,(H,1)) and Pool(X,(1,W)) represent average pooling in the height and width directions respectively, that is, calculating the average value of all values ​​in the pooling window. X represents the input data, H represents the size of the pooling window in the height direction, and W represents the size of the pooling window in the width direction.

[0098] (2) Concatenate the horizontal and vertical feature maps together, transform them, and calculate the attention weights in the two directions:

[0099] F=act(BN(conv(contact(X h ,X w ))))

[0100] g h =split(F h )

[0101] g w =split(F w )

[0102] Among them, F represents the feature map after average pooling, g h , g w represents the attention weights in the height and width directions, BN represents batch normalization, conv represents convolution, contact(X h ,X w ) represents the splicing operation, F h 、F w represents the division of F in height and width directions;

[0103] (3) Apply the attention weights to the original feature map X:

[0104]

[0105] Three feature layers are output from Neck, and their shapes are: (256, 80, 60, 30), (512, 40, 30, 30) and (1024, 20, 15, 30). In the model, there are 3 prior boxes for each feature point on each feature layer, and the 3 prior boxes are responsible for predicting small, medium and large targets respectively. The last parameter of each feature layer can be split into 3*(5+5). The first 5 means the parameters predicted by the model: the relative offset of the prior box to the initial value in the x and y directions, the relative change of the width and height of the predicted box to the preset value, and whether the predicted box contains an object. The bounding box regressor obtains the center coordinate offset and size of the predicted bounding box through two 1*1 convolutions. The classifier converts certain channels in the feature map into the predicted probability of each category through a 1*1 convolution layer. The second 5 means that this model is used to predict five types of defects, and the probability of which defect is in the box.

[0106] After the model gives a prediction result, an indicator is needed to measure the quality of the model's prediction. The Head also contains a loss function to represent the quantitative indicator of the degree to which the model's prediction result is inconsistent with the actual result. The size of the loss value represents the quality of the model's prediction result. The larger the loss value, the worse the prediction result, and vice versa. In the training process of the model, the specific meaning of the loss function is to characterize the quality of the model's learning state. Specifically, it is necessary to use this indicator as a clue to update the weight parameters of the model and find the weight of the fitting result that is closest to the true value, that is, the weight that makes the model's loss value as small as possible. The present invention uses C-IoU to measure the positioning loss of the prediction box. The expression formula of the C-IoU loss function is as follows:

[0107]

[0108] Where: ρ is the Euclidean distance between the center of the predicted box and the real box, l is the diagonal length of the minimum rectangle surrounding the predicted box and the real box, v is the similarity of the aspect ratio between the predicted box and the real box, α is the influencing factor of the aspect ratio between the predicted box and the real box, and the expressions are:

[0109]

[0110] where x i1 、x i2 ,y i1 ,y i2 Represents the coordinates of the upper left corner and lower right corner of the i-th bounding box, i=l,p.

[0111] The present invention also uses binary cross entropy loss to penalize the difference between the defect category probability predicted by each prediction box and the true label. The expression formula is as follows:

[0112]

[0113] Train a model for detecting surface defects on tall buildings.

[0114] The present invention adopts the method of public data set training + self-built building surface damage data set secondary training. First, the model is pre-trained using an open source data set, and the pre-trained weights are used as initialization parameters for training this model. The data set used in the present invention is the source code and data set of the building structure damage detection system, which contains 1,800 images, covering three main types of damage: corrosion, cracks and spalling. According to the characteristics of this model and the surface defect data set of high-altitude buildings, the entire network of the model is first trained using a public data set to obtain a pre-trained model, and then the model is migrated from the weights trained with the public data set to the environment of the present invention for secondary training.

[0115] like Figure 2 As shown, the learning rate change strategy of the model is an important part of the training process, and the appropriate learning rate change can effectively improve the training effect of the model. The algorithm of the present invention uses the cosine annealing learning rate to dynamically adjust the learning rate of the training process. At the beginning of the training, the model will adopt a smaller learning rate to allow the weight to change smoothly from the initial irregular state, preventing the instability and oscillation of the weight change caused by the larger learning rate. The learning rate will change linearly to the preset value at this stage. After the warm-up stage, the adjustment of the learning rate is similar to the cosine curve, which slowly increases from a smaller value, then increases rapidly, and finally increases slowly. At the beginning of the training, it is necessary to adopt a smaller learning rate to ensure the stability of the network, then accelerate the learning, reach near the optimal value, and then slowly reduce the learning rate until it slowly approaches the optimal solution. This learning rate adjustment strategy conforms to the law of model learning and can effectively train this model.

[0116] The present invention will be further described below in conjunction with the accompanying drawings and examples.

[0117] In order to build a robot intelligent detection algorithm for surface defects of high-altitude buildings, the present invention collects building defect samples through a combination of network-field collection, normalizes the collected samples, combines image enhancement and data augmentation methods, obtains labeled samples of building defect data sets, and classifies and labels them according to the characteristics of different defects. A dedicated data set is established to improve the detection capability of specific buildings. This data set contains surface damage of a specific building and similar buildings. In the second step, the neural network model proposed by the present invention is trained and verified using public data sets and self-built data sets.

[0118] The working environment configuration composition of this embodiment is shown in Table 1.

[0119] Table 1

[0120]

[0121]

[0122] Figure 3 The classification loss change curve of the training process of the present invention is shown. It can be seen from the image that the model converges quickly in the first 20 rounds or so, corresponding to the stage of rapid change in learning rate. After 50 rounds, the loss changes slowly and the model gradually converges. At the end of training, the validation set loss is higher than the training set loss, because the validation set is only used to calculate the evaluation index of the model and does not participate in training. In fact, the accuracy of all neural network validation sets is lower than that of the training set, which is a normal phenomenon.

[0123] Finally, mAP@.5 is used to represent the overall performance of the model. Figure 4 The figure shows the change of mAP value during the training process of the defect detection model of the present invention. The dotted line in the figure means the fitting curve of the true value. In the first 20 epochs, the mAP@0.5 value rises rapidly, showing that the model has a significant learning effect in the early training. Between 20 and 40 epochs, the mAP@0.5 value fluctuates greatly, which is due to the influence of the model's adjustment of weights and data enhancement training strategies. After 40 epochs, the fluctuation of the mAP@0.5 value begins to decrease, the model performance tends to be stable, and the detection performance of the model is also continuously improved to a higher value. Figure 5 The results show that the robot intelligent detection algorithm for surface defects of high-altitude buildings proposed in this invention can identify various defects on buildings with high accuracy.

[0124] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A robot intelligent detection method for surface defects of high-altitude buildings, characterized in that: include: Acquire a surface defect image to be detected, input the surface defect image to be detected into a high-altitude building surface defect detection model carried by the robot, and obtain a surface defect detection result; The high-altitude building surface defect detection model is obtained by training with a training set; the training set includes: surface defect images and defect labels; The high-altitude building surface defect detection model uses lightweight convolution for feature extraction, introduces a spatial attention mechanism to calculate the attention weights after the feature maps are spliced, and assigns the attention weights to the original feature maps.

2. The high-altitude building surface defect robot intelligent detection method according to claim 1 is characterized in that: The high-altitude building surface defect detection model includes: A Backbone module, used for extracting a plurality of feature layers from the surface defect image to be detected; Neck module, used for fusing features of the several feature layers to obtain multiple prediction values; The Head module is used to decode the multiple prediction values ​​to obtain the surface defect detection result.

3. The high-altitude building surface defect robot intelligent detection method according to claim 2 is characterized in that: The Backbone module includes: A first Conv submodule and a first residual connection submodule, wherein the first Conv submodule and the first residual connection submodule are alternately arranged, and the SPPF submodule is connected after the last first residual connection submodule; The first Conv submodule is used to extract features using lightweight convolution, and perform one convolution, batch normalization and activation function operations on the feature map obtained by convolution; The first residual connection submodule is used to maintain the uniformity of feature extraction results; The SPPF submodule is used to aggregate the features on the last feature layer output by the Backbone module; The first residual connection submodule is sequentially provided with: a first residual connection unit, a second residual connection unit and a third residual connection unit.

4. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 3 is characterized in that: Maintaining the uniformity of the feature extraction results includes: X'=W*X+b Among them, W represents the weight of the convolutional layer, X represents the input data, and b represents the bias of the convolutional layer. represents the result of batch standardization output, γ and β are adjustable parameters, which are used to scale and translate the standardized data, μ represents the mean of the current batch data, σ represents the standard deviation of the current batch data, ε is a very small constant to prevent division by 0, and Y represents the final output result of 0 and The larger value.

5. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 3 is characterized in that: The Neck module includes: A plurality of second Conv submodules, upsampling submodules, first merged channel submodules and second residual connection submodules are sequentially connected in the order of the second Conv submodule, the upsampling submodule, the first merged channel submodule and the second residual connection submodule; the second Conv submodule is sequentially provided with: a first Conv unit and a second Conv unit; the first Conv unit is connected to the SPPF submodule of the Backbone module; the first merged channel submodule is sequentially provided with: a first merged channel unit and a second merged channel unit; the first merged channel unit is connected to the second residual connection unit, and the second merged channel unit is connected to the first residual connection unit; The second Conv submodule is used to extract features by using light convolution, and perform one convolution, batch normalization and activation function operations on the feature map obtained by convolution; The upsampling submodule is used to perform an upsampling operation on the output result of the second Conv submodule; The first merging channel submodule is used to fuse the output result of the upsampling submodule with the output result of the residual connection submodule along the channel dimension; The second residual connection submodule is used to accelerate the training and reasoning speed of the network while preventing the gradient from exploding or being zero.

6. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 5 is characterized in that: The Neck module also includes: A plurality of downsampling submodules, a second merging channel submodule and a third residual connection submodule are connected in sequence according to the downsampling submodule, the second merging channel submodule and the third residual connection submodule, and the first downsampling submodule is connected to the second residual connection submodule at the end; the second merging channel submodule includes: a third merging channel unit and a fourth merging channel unit; the third merging channel unit is connected to the second Conv unit, and the fourth merging channel unit is connected to the first Conv unit; The downsampling submodule is used to perform a downsampling operation on the output result of the residual connection submodule; The second merging channel submodule is used to fuse the output result of the downsampling submodule with the output result of the second Conv submodule; The third residual connection submodule is used to accelerate the training and reasoning speed of the network while preventing the gradient from exploding or being zero.

7. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 2 is characterized in that: The Neck module also includes: A plurality of CA submodules, wherein the CA submodules are sequentially provided with: a first CA unit, a second CA unit and a third CA unit; the first CA unit is connected to the second residual connection submodule at the end, and the second CA unit and the third CA unit are respectively connected to the third residual connection submodule; The CA submodule is used to calculate the attention weights after the feature maps are concatenated, assign the attention weights to the original feature maps, and output three feature layers.

8. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 7 is characterized in that: Calculate the attention weight after the feature map is concatenated, and assign the attention weight to the original feature map including: Perform global average pooling on the feature maps in the horizontal and vertical directions: Among them, X h , X w Represent the results of average pooling in the horizontal and vertical directions respectively. Pool(X,(H,1)) and Pool(X,(1,W)) represent average pooling in the height and width directions respectively, that is, calculating the average value of all values ​​in the pooling window. X represents the input data, H represents the size of the pooling window in the height direction, and W represents the size of the pooling window in the width direction. The horizontal and vertical feature maps are concatenated and transformed and the attention weights in both directions are calculated: F=act(BN(conv(contact(X h ,X w )))) g h =split(F h ) g w =split(F w ) Among them, F represents the feature map after average pooling, g h , g w represents the attention weights in the height and width directions, BN represents batch normalization, conv represents convolution, contact(X h ,X w ) represents the splicing operation, F h 、F w represents the division of F in height and width directions; Apply the attention weights in two directions to the original feature map: Among them, ⊙ represents the hadamard product of the matrix, that is, the multiplication of elements in corresponding positions.

9. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 7 is characterized in that: The Head module includes: a plurality of detection heads, wherein the plurality of detection heads are respectively connected to the first CA unit, the second CA unit and the third CA unit; The detection head is used to obtain the surface defect detection result according to the characteristic layer.

10. The robot intelligent detection method for surface defects of high-altitude buildings according to claim 9 is characterized in that: The Head module also includes C-IoU to measure the positioning loss of the prediction box; The expression of C-IoU is: Among them, ρ represents the Euclidean distance between the center point of the predicted box and the real box, l represents the diagonal length of the minimum rectangle surrounding the predicted box and the real box, v represents the similarity of the aspect ratio of the predicted box and the real box, α represents the influencing factor of the aspect ratio of the predicted box and the real box, and IoU represents the intersection over union ratio, which is the area of ​​the intersection divided by the maximum effective area occupied by the two boxes.

Images