Rural house outer wall crack detection method

Abstract

The invention discloses a rural house outer wall crack detection method, and belongs to the technical field of artificial intelligence and crack detection. The problem that an existing method is low in detection accuracy is solved. The MSFA unit and the sparse context feature enhancement unit are introduced into the trunk module of the crack detection model to enhance the understanding of the model on global image information, so that attention is effectively guided to concentrate on key areas in an image, the context perception capability and the multi-scale feature extraction capability of the crack detection model are effectively improved, and the detection accuracy of the crack detection model is improved. And the sensing capability on micro cracks and complex crack forms is enhanced. In addition, by designing the Shape-NWD loss function, the model can adapt to the characteristics of changeable fracture shapes, the stable detection capability is provided, and more attention is paid to the shape matching property of the detection frame, so that the requirements of fracture detection are better met, and the model is helped to accurately position the fracture. The method can be applied to house outer wall crack detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and crack detection, and in particular relates to a method for detecting cracks in the exterior walls of rural houses. Background Art

[0002] According to online research and field investigations, many rural houses currently suffer from problems such as the use of low-quality building materials, non-standard construction, complex geological conditions, and harsh natural environments. These problems have led to varying degrees of damage to the exterior wall structures, with wall cracks being the most common. Some cracks may even gradually spread to the interior walls, affecting interior surfaces and causing more extensive damage, seriously threatening residents' lives. With the rapid development of deep learning and object detection technologies, traditional manual crack detection methods have gradually exposed some shortcomings. Manual inspection often relies on manual experience, is labor-intensive, and is susceptible to human factors such as omissions and fatigue, resulting in unstable detection results and low efficiency. In contrast, automated detection methods based on computer vision, with their advantages of rapid detection speed, wide coverage, and strong repeatability, have become a more efficient and reliable crack detection solution, significantly improving crack identification accuracy and detection efficiency. However, cracks on the exterior walls of houses usually have various shapes, irregular distribution, and complex development trends. They can appear in various forms such as straight lines, curves, and serrated shapes, and are easy to change over time. Therefore, the accuracy of existing automated detection methods is still relatively limited. Therefore, it is an urgent need to solve the problem of low detection accuracy of existing methods to achieve efficient and accurate detection of cracks on the exterior walls of rural houses. Summary of the Invention

[0003] The purpose of the present invention is to solve the problem of low detection accuracy of existing methods and to propose a method for detecting cracks in the exterior walls of rural houses.

[0004] The technical solution adopted by the present invention to solve the above technical problems is: a method for detecting cracks in the exterior walls of rural houses, the method specifically comprising the following steps:

[0005] Step 1: Obtain a dataset of real rural house images taken by drones and add labels to each acquired image.

[0006] Step 2: Construct a crack detection model, which includes a backbone module, an encoding module, and a decoding module, wherein:

[0007] In the crack detection model, the house image is used as the input of the backbone module, the output of the backbone module is used as the input of the encoding module, and the output of the encoding module is used as the input of the decoding module, which outputs the crack detection results.

[0008] Step 3: Use the images and labels obtained in step 1 to train the crack detection model. During the training process, the loss function value is calculated based on the crack detection results and labels output by the decoding module;

[0009] Adjust the parameters of the crack detection model according to the back propagation of the loss function value until the loss function value converges and stop training to obtain a trained crack detection model;

[0010] Step 4: Input the image of the rural house to be detected into the trained crack detection model, and output the crack detection results through the trained crack detection model.

[0011] Furthermore, each acquired image is labeled, wherein the label includes a crack type and a crack position;

[0012] Crack types include no cracks and various crack grades.

[0013] Furthermore, the working process of the backbone module is as follows:

[0014] The house image is used as the input of the backbone module. In the backbone module, the input house image first passes through the HGStem unit, and then the output of the HGStem unit is used as the input of the first MSFA unit;

[0015] The output of the first MSFA unit is used as the input of the first depth-wise separable convolutional layer, and the output of the first depth-wise separable convolutional layer is used as the input of the second MSFA unit.

[0016] The output of the second MSFA unit is used as the input of the second depth-wise separable convolutional layer, and the output of the second depth-wise separable convolutional layer is used as the input of the third MSFA unit.

[0017] The output of the third MSFA unit is used as the input of the fourth MSFA unit, and the output of the fourth MSFA unit is used as the input of the fifth MSFA unit;

[0018] The output of the fifth MSFA unit is used as the input of the third depth-wise separable convolutional layer, and the output of the third depth-wise separable convolutional layer is used as the input of the sixth MSFA unit;

[0019] The output of the first MSFA unit is passed through the sparse context feature enhancement unit to obtain output S3;

[0020] The output of the second MSFA unit is passed through the sparse context feature enhancement unit to obtain output S4;

[0021] The output of the sixth MSFA unit is passed through the sparse context feature enhancement unit to obtain output S5.

[0022] Furthermore, the working process of the first MSFA unit is:

[0023] The input of the first MSFA unit After the dilation rate is 1, the output F1 is obtained; after upsampling F1, the result of upsampling F1 is compared with the input The output of the dilation rate 3 dilation convolution layer is added to get the sum. :

[0024]

[0025] in, represents a dilated convolutional layer with a dilation rate of 3;

[0026] Indicates upsampling processing;

[0027] The result of upsampling F2 is compared with the input The output of the dilation rate 5 dilation convolution layer is added to get the sum :

[0028]

[0029] in, Indicates the dilation rate of the dilation layer is 5;

[0030] Pass F1 through the first convolution layer with a convolution kernel size of 1×1 to obtain the output L1 of the first convolution layer. Then upsample the output L1 and add the upsampled result of L1 to the output of F2 after passing the second convolution layer with a convolution kernel size of 1×1 to obtain the added result L2;

[0031] Add the up-sampled result of L2 to the output of the third convolutional layer of F3 with a convolution kernel size of 1×1 to obtain the added result L3;

[0032] Then L1, L2 and L3 are added together, and the addition result is used as the output of the first MSFA unit.

[0033] Furthermore, the working process of the sparse context feature enhancement unit is as follows:

[0034] The feature map output by the first MSFA unit is denoted as P3, and P3 is used as the input of the threshold segmentation subunit. The output of the threshold segmentation subunit is then used as the input of the mask generation subunit, and the output of the mask generation subunit and P3 are used as the input of the sparse convolution layer. The output of the sparse convolution layer is added to the feature map P3, and the addition result is denoted as S3.

[0035] Furthermore, the working process of the encoding module is as follows:

[0036] Step 1: Flatten the feature map S5 to obtain a sequence, then perform position encoding on the obtained sequence, and add the position encoding result to the sequence to obtain a sequence with position encoding added;

[0037] Step 2: Use the sequence with positional encoding added as the input of the AIFI unit to obtain the output of the AIFI unit;

[0038] The working process of the AIFI unit is:

[0039] The sequence with positional encoding is passed through the multi-head self-attention layer, and then the sequence with positional encoding is residually connected with the output of the multi-head self-attention layer to obtain the residual connection result a; the residual connection result a is layer-normalized and then input into the FFN layer;

[0040] Perform a residual connection between the output of the FFN layer and the input of the FFN layer to obtain the residual connection result b; perform layer normalization on the residual connection b, and restore the layer normalization result to the same dimension as the feature map S5. The restored result is recorded as F5;

[0041] Step 3: Take the feature map S3, feature map S4, and the output F5 of the AIFI unit as the input of the CCFF unit, and output the fused features through the CCFF unit to obtain the output of the encoding module;

[0042] The CCFF unit includes the first feature fusion subunit to the fourth feature fusion subunit. The working process of the CCFF unit is as follows:

[0043] The sequence obtained by flattening the feature map S3 is recorded as F6, and the sequence obtained by flattening the feature map S4 is recorded as F7;

[0044] Use F5 as the input of the fourth convolutional layer with a convolution kernel size of 1×1, then use the output of the fourth convolutional layer as the input of the first BN layer, and the output of the first BN layer as the input of the first SiLU activation function layer;

[0045] The output of the first SiLU activation function layer and feature map S4 are used as the input of the first feature fusion subunit, the output of the first feature fusion subunit is used as the input of the fifth convolutional layer with a convolution kernel size of 1×1, the output of the fifth convolutional layer is used as the input of the second BN layer, and the output of the second BN layer is used as the input of the second SiLU activation function layer;

[0046] The output of the second SiLU activation function layer and the feature map S3 are used as the input of the second feature fusion subunit. The output of the second feature fusion subunit is used as the input of the sixth convolutional layer with a convolution kernel size of 3×3. The output of the sixth convolutional layer is used as the input of the third BN layer, and the output of the third BN layer is used as the input of the third SiLU activation function layer.

[0047] The output of the second SiLU activation function layer and the output of the third SiLU activation function layer are used as the input of the third feature fusion subunit;

[0048] The output of the third feature fusion subunit is used as the input of the seventh convolutional layer with a convolution kernel size of 3×3, the output of the seventh convolutional layer is used as the input of the fourth BN layer, and the output of the fourth BN layer is used as the input of the fourth SiLU activation function layer;

[0049] The output of the first SiLU activation function layer and the output of the fourth SiLU activation function layer are used as the input of the fourth feature fusion subunit;

[0050] Finally, the outputs of the second feature fusion subunit, the third feature fusion subunit, and the fourth feature fusion subunit are spliced, and the splicing result is used as the output of the CCFF unit.

[0051] Furthermore, the working process of the first feature fusion subunit is as follows:

[0052] After concatenating the two inputs of the first feature fusion subunit, the concatenated result is passed through two parallel branches, where:

[0053] In the first branch, the concatenated result passes through the eighth convolution layer with a convolution kernel size of 1×1;

[0054] In the second branch, the concatenated result passes through the ninth convolutional layer with a convolution kernel size of 1×1, and then the output of the ninth convolutional layer passes through N series-connected RepBlocks in sequence;

[0055] The output of the eighth convolutional layer and the output of the last RepBlock are added together, and the result of the addition is flattened. The flattened result is the output of the first feature fusion subunit.

[0056] Furthermore, the decoding module includes N' serial decoding units, each of which includes a multi-head self-attention layer, an FFN layer, a multi-scale deformable attention layer and an FFN layer in sequence;

[0057] The working process of the decoding module is:

[0058] Step 1: Randomly initialize H learnable query vectors, and then use the uncertainty-minimum query selection strategy to select the query vectors with the top h importance from the H query vectors;

[0059] For any selected query vector, after position encoding the query vector, the position code is added to the query vector to obtain the query vector with position code added; and each selected query vector is traversed;

[0060] Step 2: Initialize n=1;

[0061] Step 3: For any query vector obtained in step 1 , for the query vector Perform linear transformation to generate query ,key Sum :

[0062]

[0063]

[0064]

[0065] in, 、 and is the weight matrix;

[0066] Then according to the query ,key Sum Computing attention output :

[0067]

[0068] in, Is a query Dimensions;

[0069] After traversing all query vectors obtained in step 1, the attention outputs corresponding to each query vector are spliced ​​to obtain the splicing result :

[0070]

[0071] The stitching results Perform linear transformation to obtain linear transformation result :

[0072]

[0073] in, The weight matrix representing the linear transformation;

[0074] Step 4: Linear transformation results Input the first FFN layer and get the output of the first FFN layer :

[0075]

[0076] in, and is the weight matrix;

[0077] and is the bias term;

[0078] is the ReLU activation function;

[0079] Step 5: Perform linear transformation to obtain the query of multi-head deformable attention layer ;

[0080] When n=1, the output of the encoding module is linearly transformed to obtain the key of the multi-head deformable attention layer Sum ;

[0081] When n>1, the output of the n-1th decoding unit is linearly transformed to obtain the key of the multi-head deformable attention layer Sum ;

[0082] Step 6: Use the output of the multi-head deformable attention layer as the input of the second FFN layer, and use the output of the second FFN layer as the output of the n-th decoding unit;

[0083] Step 7: Let n=n+1, return to step 3, and continue until the output of the N'th decoding unit is obtained. The output of the N'th decoding unit is passed through the detection head, and the detection head is used to output the crack detection result of the image.

[0084] Furthermore, the working process of the multi-head deformable attention layer is:

[0085]

[0086] in, represents the feature representation of the query Q without linear transformation, is the sampling offset, is the attention weight of the mth attention head at the kth sampling point of the lth layer feature map, , is the total number of layers of feature maps, , is the number of sampling points. Due to the different resolutions of feature maps, represents the coordinates of the query point, Represents the operation of converting the coordinates of the query point to the l-th layer feature map of the m-th attention head, represents the final sampling position, It represents the feature representation obtained by sampling the final sampling position on the l-th layer feature map of the m-th attention head. represents the weight matrix, represents the weight matrix of the linear transformation of the m-th attention head, represents the number of attention heads in the multi-head deformable attention layer, Represents the output of the multi-head deformable attention layer.

[0087] Furthermore, the loss function is:

[0088]

[0089] in, and is the weighting coefficient;

[0090]

[0091]

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098] Where: B is the area of ​​the predicted bounding box;

[0099] is the area of ​​the ground-truth bounding box;

[0100] and are the width and height of the true bounding box respectively;

[0101] h and w are the width and height of the predicted bounding box respectively;

[0102] r and scale are hyperparameters;

[0103] and Represents the center point coordinates of the true bounding box;

[0104] and Represents the coordinates of the center point of the predicted bounding box;

[0105] c represents the scaling factor;

[0106] Indicates calculation of absolute value;

[0107]

[0108] in, is the base of natural logarithms;

[0109] is a constant;

[0110]

[0111] Among them, weight is the mean of the last dimension in the feature map output by the N'th decoding unit.

[0112] The beneficial effects of the present invention are:

[0113] This invention introduces the MSFA unit and the sparse contextual feature enhancement unit into the core module of the crack detection model to strengthen the model's understanding of global image information, effectively guiding attention to key areas in the image. This effectively improves the crack detection model's contextual perception and multi-scale feature extraction capabilities, and enhances its perception of small cracks and complex crack morphologies. Furthermore, by designing a Shape-NWD loss function, the invention adapts the model to the changing characteristics of crack morphology, provides stable detection capabilities, and pays more attention to the shape matching of the detection frame, thereby better meeting the needs of crack detection, helping the model accurately locate cracks, significantly improving the accuracy of target detection, and eliminating the shortcomings of manual detection to achieve real-time automated detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0114] Figure 1 is a structural diagram of a crack detection model of the present invention;

[0115] Figure 2 It is the structural diagram of the MSFA unit;

[0116] Figure 3 It is the structural diagram of the backbone module;

[0117] Figure 4 It is a schematic diagram of the sparse context feature enhancement unit;

[0118] Figure 5This is a schematic diagram without cracks;

[0119] Figure 6 It is a schematic diagram of a mild crack;

[0120] Figure 7 is a schematic diagram of a moderate crack;

[0121] Figure 8 This is a schematic diagram of severe cracks;

[0122] Figure 9 It is a structural diagram of the fusion subunit. DETAILED DESCRIPTION

[0123] Specific implementation method 1: Combination Figure 1 This embodiment describes a method for detecting cracks in the exterior walls of rural houses, the method specifically comprising the following steps:

[0124] Step 1: Obtain a dataset of real rural house images taken by drones and add labels to each acquired image.

[0125] Step 2: Construct a crack detection model, which includes a backbone module, an encoding module, and a decoding module, wherein:

[0126] In the crack detection model, the house image is used as the input of the backbone module, the output of the backbone module is used as the input of the encoding module, and the output of the encoding module is used as the input of the decoding module, which outputs the crack detection results.

[0127] The backbone module incorporates MSFA units and sparse context feature enhancement units to better learn context. The encoding module utilizes AIFI units and CCFF units to achieve multi-scale feature interaction and fusion, maintaining a balance between high-level semantic information and low-level detail features. The decoding module adopts an uncertainty-minimum query selection strategy to select the top h query vectors with both high classification scores and high IoU scores, reducing uncertainty and improving efficiency.

[0128] Step 3: Use the images and labels obtained in step 1 to train the crack detection model. During the training process, the loss function value is calculated based on the crack detection results and labels output by the decoding module;

[0129] Adjust the parameters of the crack detection model according to the back propagation of the loss function value until the loss function value converges and stop training to obtain a trained crack detection model;

[0130] Step 4: Input the image of the rural house to be detected into the trained crack detection model, and output the crack detection results through the trained crack detection model.

[0131] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that a label is added to each acquired image, and the label includes the crack type and crack location;

[0132] Crack types include no cracks and various crack grades.

[0133] Other steps and parameters are the same as those in the first embodiment.

[0134] This method captured over 20,000 drone-captured images of rural houses in a county under the jurisdiction of City A. Experts assigned labels to each image, specifically categorizing the cracks on the exterior walls of the houses into four categories: 1) no cracks; 2) mild cracks; 3) moderate cracks; and 4) severe cracks.

[0135] Among them, no cracks indicate that the wall is intact; mild cracks are shallow, usually only cracks on the surface of the wall, and generally do not affect the structural stability of the wall; moderate cracks are of average depth, with a large number or wide distribution, which may have a certain impact on the load-bearing structure of the wall; severe cracks are usually deeper, or there is obvious crack expansion, which may be accompanied by wall deformation, penetration, tilting or sinking, etc., seriously affecting the safety of the house. Cracks of different severity have different focuses in repair. No cracks do not need to be treated; mild cracks focus on filling and restoring the appearance, mainly dealing with surface problems; moderate cracks are filled if they do not affect the load-bearing structure, otherwise materials must be used to reinforce the wall to enhance the stability; severe cracks should fully reinforce the foundation and walls to ensure the structural safety of the house, and the house needs to be thoroughly structurally reinforced and monitored for a long time. The schematic diagrams of the four types of cracks are as follows Figures 5 to 8 shown.

[0136] Specific implementation method three: Combination Figure 3 This embodiment differs from the first or second embodiment in that the working process of the backbone module is as follows:

[0137] The house image is used as the input of the backbone module. In the backbone module, the input house image first passes through the HGStem unit, and then the output of the HGStem unit is used as the input of the first MSFA unit;

[0138] The output of the first MSFA unit is used as the input of the first depth-wise separable convolutional layer (DWConv), and the output of the first depth-wise separable convolutional layer is used as the input of the second MSFA unit.

[0139] The output of the second MSFA unit is used as the input of the second depth-wise separable convolutional layer, and the output of the second depth-wise separable convolutional layer is used as the input of the third MSFA unit.

[0140] The output of the third MSFA unit is used as the input of the fourth MSFA unit, and the output of the fourth MSFA unit is used as the input of the fifth MSFA unit;

[0141] The output of the fifth MSFA unit is used as the input of the third depth-wise separable convolutional layer, and the output of the third depth-wise separable convolutional layer is used as the input of the sixth MSFA unit;

[0142] The output of the first MSFA unit is passed through the sparse context feature enhancement unit to obtain output S3;

[0143] The output of the second MSFA unit is passed through the sparse context feature enhancement unit to obtain output S4;

[0144] The output of the sixth MSFA unit is passed through the sparse context feature enhancement unit to obtain output S5.

[0145] Other steps and parameters are the same as those in the first or second embodiment.

[0146] The HGStem unit extracts preliminary features from the input image, while the MSFA unit combines contextual information enhancement and feature refinement techniques, using multi-scale dilated convolutions to achieve feature fusion. These features are then injected layer by layer from top to bottom into the feature pyramid network, enabling the model to learn features at multiple scales, effectively improving the feature extraction capability of small objects and bridging the semantic gaps between layers, thereby providing richer contextual information. This strategy helps enhance the model's adaptability to various object sizes and shapes, thereby strengthening multi-scale feature learning.

[0147] Specific implementation method four: Combination Figure 2 This embodiment is different from any one of the first to third embodiments in that the working process of the first MSFA unit is as follows:

[0148] The input of the first MSFA unit After the dilation rate is 1, the output F1 is obtained; after upsampling F1, the result of upsampling F1 is compared with the input The output of the dilation rate 3 dilation convolution layer is added to get the sum. :

[0149]

[0150] in, represents a dilated convolutional layer with a dilation rate of 3. Indicates upsampling processing;

[0151] The result of upsampling F2 is compared with the input The output of the dilation rate 5 dilation convolution layer is added to get the sum :

[0152]

[0153] in, Indicates the dilation rate of the dilation layer is 5;

[0154] Pass F1 through the first convolution layer with a convolution kernel size of 1×1 to obtain the output L1 of the first convolution layer. Then upsample the output L1 and add the upsampled result of L1 to the output of F2 after passing the second convolution layer with a convolution kernel size of 1×1 to obtain the added result L2;

[0155] Add the up-sampled result of L2 to the output of the third convolutional layer of F3 with a convolution kernel size of 1×1 to obtain the added result L3;

[0156] Then L1, L2 and L3 are added together, and the addition result is used as the output of the first MSFA unit.

[0157] The other steps and parameters are the same as those in the first to third embodiments.

[0158] Specific implementation method five: Combination Figure 4 This embodiment is different from any one of the specific embodiments 1 to 4 in that the working process of the sparse context feature enhancement unit is as follows:

[0159] The feature map output by the first MSFA unit is denoted as P3, and P3 is used as the input of the threshold segmentation subunit (OtsuThresholding). The output of the threshold segmentation subunit is then used as the input of the mask generation subunit (MaskGeneration), and the output of the mask generation subunit and P3 are used as the input of the sparse convolution layer (SparseConvolution). The output of the sparse convolution layer is added to the feature map P3, and the addition result is denoted as S3.

[0160] The other steps and parameters are the same as those in the first to fourth embodiments.

[0161] The output P4 of the second MSFA unit and the output P5 of the sixth MSFA unit are processed in the same manner as P3. The threshold segmentation subunit automatically calculates a global optimal threshold using the Otsu algorithm and distinguishes the foreground and background areas of the image based on the threshold.

[0162] The steps of Otsu algorithm are as follows:

[0163] In the first step, for each possible threshold T, the image is divided into two parts: pixels with grayscale values ​​less than or equal to T (background) and pixels with grayscale values ​​greater than T (foreground).

[0164] The second step is to calculate the inter-class variance. The larger the inter-class variance, the greater the difference between the foreground and background, and the better the image segmentation effect. The calculation formula is as follows:

[0165]

[0166] in and are the weights of foreground and background respectively (pixel ratio), and are the average grayscale values ​​of the foreground and background respectively, assuming that the image has L grayscale levels (usually 0 to 255):

[0167]

[0168]

[0169]

[0170]

[0171] The third step is to select the The maximized threshold is the optimal segmentation threshold obtained by the Otsu algorithm:

[0172]

[0173] The mask generation subunit is used to generate a mask matrix based on the differentiation results. That is, the element value corresponding to the foreground pixel in the mask matrix is ​​recorded as 1, and the element value corresponding to the background pixel in the mask matrix is ​​recorded as 0 to generate a mask matrix. The new feature map is obtained by performing sparse convolution calculation on this mask matrix, and then fused with the input feature map to enhance the model's context perception ability, thereby accelerating the inference process of the deep learning model and improving the effect of feature learning. The schematic diagram is shown as follows: Figure 4 After processing, the feature maps S3, S4, and S5 are output in order from low to high according to the feature level. The feature maps S3, S4, and S5 are flattened into sequences and then sent to the encoding module respectively.

[0174] Specific embodiment 6: This embodiment differs from any one of specific embodiments 1 to 5 in that the working process of the encoding module is as follows:

[0175] Step 1: Flatten the feature map S5 to obtain a sequence, then perform position encoding on the obtained sequence, and add the position encoding result to the sequence to obtain a sequence with position encoding added;

[0176] The specific process of the position encoding is:

[0177]

[0178]

[0179] Step 2: Use the sequence with positional encoding added as the input of the AIFI unit to obtain the output of the AIFI unit;

[0180] The working process of the AIFI unit is:

[0181] The sequence with positional encoding is passed through the multi-head self-attention layer, and then the sequence with positional encoding is residually connected with the output of the multi-head self-attention layer to obtain the residual connection result a. After layer normalization processing is performed on the residual connection result a, the layer normalization result is input into the FFN layer (Feed Forward). FFN is used to perform effective feature transformation at each position.

[0182] Perform a residual connection between the output of the FFN layer and the input of the FFN layer to obtain the residual connection result b; perform layer normalization on the residual connection b, and restore the layer normalization result to the same dimension as the feature map S5. The restored result is recorded as F5;

[0183] Step 3: Take the feature map S3, feature map S4, and the output F5 of the AIFI unit as the input of the CCFF unit, and output the fused features through the CCFF unit to obtain the output of the encoding module;

[0184] The CCFF unit includes the first feature fusion subunit to the fourth feature fusion subunit. The working process of the CCFF unit is as follows:

[0185] The sequence obtained by flattening the feature map S3 is recorded as F6, and the sequence obtained by flattening the feature map S4 is recorded as F7;

[0186] Use F5 as the input of the fourth convolutional layer with a convolution kernel size of 1×1, then use the output of the fourth convolutional layer as the input of the first BN layer, and the output of the first BN layer as the input of the first SiLU activation function layer;

[0187] The output of the first SiLU activation function layer and feature map S4 are used as the input of the first feature fusion subunit, the output of the first feature fusion subunit is used as the input of the fifth convolutional layer with a convolution kernel size of 1×1, the output of the fifth convolutional layer is used as the input of the second BN layer, and the output of the second BN layer is used as the input of the second SiLU activation function layer;

[0188] The output of the second SiLU activation function layer and the feature map S3 are used as the input of the second feature fusion subunit. The output of the second feature fusion subunit is used as the input of the sixth convolutional layer with a convolution kernel size of 3×3. The output of the sixth convolutional layer is used as the input of the third BN layer, and the output of the third BN layer is used as the input of the third SiLU activation function layer.

[0189] The output of the second SiLU activation function layer and the output of the third SiLU activation function layer are used as the input of the third feature fusion subunit;

[0190] The output of the third feature fusion subunit is used as the input of the seventh convolutional layer with a convolution kernel size of 3×3, the output of the seventh convolutional layer is used as the input of the fourth BN layer, and the output of the fourth BN layer is used as the input of the fourth SiLU activation function layer;

[0191] The output of the first SiLU activation function layer and the output of the fourth SiLU activation function layer are used as the input of the fourth feature fusion subunit;

[0192] Finally, the outputs of the second feature fusion subunit, the third feature fusion subunit, and the fourth feature fusion subunit are spliced, and the splicing result is used as the output of the CCFF unit.

[0193] The other steps and parameters are the same as those in the first to fifth embodiments.

[0194] Specific implementation method seven: combination Figure 9 The difference between this embodiment and any one of the specific embodiments 1 to 6 is that the working process of the first feature fusion subunit is as follows:

[0195] After concatenating the two inputs of the first feature fusion subunit, the concatenated result is passed through two parallel branches, where:

[0196] In the first branch, the concatenated result passes through the eighth convolution layer with a convolution kernel size of 1×1;

[0197] In the second branch, the concatenated result passes through the ninth convolutional layer with a convolution kernel size of 1×1, and then the output of the ninth convolutional layer passes through N series-connected RepBlocks in sequence;

[0198] The output of the eighth convolutional layer and the output of the last RepBlock are added together, and the result of the addition is flattened. The flattened result is the output of the first feature fusion subunit.

[0199] The other steps and parameters are the same as those in the first to sixth embodiments.

[0200] The CCFF unit is optimized based on the cross-scale fusion sub-unit. By inserting several fusion sub-units composed of convolutional layers into the fusion path, the fusion sub-unit is used to fuse two adjacent scale features into a new feature. Its structure is as follows: Figure 9 As shown in Figure 1, the fusion subunit contains two convolutional layers with a convolution kernel size of 1×1 to adjust the number of channels, uses N RepBlocks composed of RepConv for feature fusion, and fuses the outputs of the two branches by element-wise addition.

[0201] Specific embodiment eight: This embodiment differs from any one of specific embodiments one to seven in that the decoding module includes N' serial decoding units, each of which includes a multi-head self-attention layer, an FFN layer, a multi-scale deformable attention layer, and an FFN layer in sequence;

[0202] The working process of the decoding module is:

[0203] Step 1: Randomly initialize H learnable query vectors, and then use the uncertainty-minimum query selection strategy to select the query vectors with the top h importance from the H query vectors;

[0204] For any selected query vector, after position encoding the query vector, the position code is added to the query vector to obtain the query vector with position code added; and each selected query vector is traversed;

[0205] Step 2: Initialize n=1;

[0206] Step 3: For any query vector obtained in step 1 , for the query vector Perform linear transformation to generate query ,key Sum :

[0207]

[0208]

[0209]

[0210] in, 、 and is the weight matrix;

[0211] Then according to the query ,key Sum Computing attention output :

[0212]

[0213] in, Is a query Dimensions;

[0214] After traversing all query vectors obtained in step 1, the attention outputs corresponding to each query vector are spliced ​​to obtain the splicing result :

[0215]

[0216] The stitching results Perform linear transformation to obtain linear transformation result :

[0217]

[0218] in, The weight matrix representing the linear transformation;

[0219] Step 4: Linear transformation results Input the first FFN layer to enhance the expressiveness of the model and obtain the output of the first FFN layer :

[0220]

[0221] in, and is the weight matrix, and is the bias term, is the ReLU activation function;

[0222] Step 5: Perform linear transformation to obtain the query of multi-head deformable attention layer ;

[0223] When n=1, the output of the encoding module is linearly transformed to obtain the key of the multi-head deformable attention layer Sum ;

[0224] When n>1, the output of the n-1th decoding unit is linearly transformed to obtain the key of the multi-head deformable attention layer Sum ;

[0225] Step 6: Use the output of the multi-head deformable attention layer as the input of the second FFN layer, and use the output of the second FFN layer as the output of the n-th decoding unit;

[0226] Step 7: Let n=n+1, return to step 3, and continue until the output of the N'th decoding unit is obtained. The output of the N'th decoding unit is passed through the detection head, and the detection head is used to output the crack detection result of the image.

[0227] The other steps and parameters are the same as those in the first to seventh embodiments.

[0228] The crack detection results include the bounding box coordinates of the crack location, the probability distribution of the crack category, and the probability of the crack existing.

[0229] Specific embodiment 9: This embodiment differs from any one of specific embodiments 1 to 8 in that the working process of the multi-head deformable attention layer is as follows:

[0230]

[0231] in, represents the feature representation of the query Q without linear transformation ( After linear transformation, a 3MK-dimensional vector is obtained, and the first 2MK dimensions of the vector are used to represent the sampling offset , the posterior MK dimension of the vector is used to represent the attention weight ), is the sampling offset, is the attention weight of the mth attention head at the kth sampling point of the lth layer feature map, , is the total number of layers of feature maps, , is the number of sampling points. Due to the different resolutions of feature maps, represents the coordinates of the query point, represents the operation of converting the normalized coordinates of the query point to the l-th layer feature map of the m-th attention head, represents the final sampling position, It represents the feature representation obtained by sampling the final sampling position on the l-th layer feature map of the m-th attention head. Represents the learnable weight matrix of the transformed features, used to transform the feature map Perform linear transformation and each attention head has an independent weight matrix, and Multiply to get the value of the corresponding position, and then Multiply it by the obtained Value to get the attention size; Represents the weight matrix of the linear transformation of the mth attention head, ensuring that the dimension of the output is consistent with the dimension of the input feature, represents the number of attention heads in the multi-head deformable attention layer, Represents the output of the multi-head deformable attention layer.

[0232] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0233] The mechanism of the multi-head deformable attention layer is similar to that of the multi-head self-attention layer. However, it changes the sampling method and spatial location of attention calculation in the multi-head self-attention layer, focusing only on a small set of key sampling points around the reference point, regardless of the spatial size of the feature map. By assigning a small fixed number of keys to each query, it alleviates convergence and feature space resolution issues. In addition to introducing an offset to dynamically adjust the sampling position, it can also process multi-scale feature maps, allowing the model to focus on important areas of the image. Each query vector extracts content related to the image target from the feature sequence output by the encoding unit through the multi-head deformable attention layer.

[0234] Specific embodiment ten: This embodiment differs from any one of specific embodiments one to nine in that the loss function is:

[0235]

[0236] in, and is the weighting coefficient;

[0237]

[0238]

[0239]

[0240]

[0241]

[0242]

[0243]

[0244]

[0245] Where: B is the area of ​​the predicted bounding box, is the area of ​​the ground-truth bounding box, and are the width and height of the true bounding box, h and w are the width and height of the predicted bounding box, r and scale are hyperparameters, and represents the center point coordinates of the true bounding box, and represents the coordinates of the center point of the predicted bounding box, c represents the scaling factor (the present invention uses the area of ​​the predicted bounding box as the scaling factor, which is beneficial for providing a stable scaling reference when the target size varies greatly, and is particularly suitable for detection tasks with large target sizes such as cracks). Indicates calculation of absolute value;

[0246]

[0247] in, is the base of natural logarithms, is a constant (specifically, the value in the present invention is 12.8);

[0248]

[0249] Among them, weight is the mean of the last dimension in the feature map output by the N'th decoding unit.

[0250] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0251] To better capture the geometric characteristics of the target, this implementation proposes the Shape-NWD loss function for the first time. The Shape-NWD loss function consists of two parts: Shape_IoU and NWD. Shape_IoU focuses on shape similarity and emphasizes shape alignment; while NWD focuses on the weighted distance of position and size and emphasizes position and scale alignment. The complementary use of the two can capture different aspects of bounding box matching, provide more reasonable and detailed error feedback when detecting complex shapes or small targets, and provide more detailed shape alignment than IOU.

[0252] The above examples are merely illustrative of the calculation model and process of the present invention and are not intended to limit the embodiments of the present invention. Persons skilled in the art will readily appreciate that other variations or modifications based on the above description are possible. This list of embodiments is not exhaustive; however, any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.

Claims

1. A method for detecting cracks in the exterior walls of rural houses, characterized in that: The method specifically comprises the following steps: Step 1: Obtain a dataset of real rural house images taken by drones and add labels to each acquired image. Step 2: Construct a crack detection model, which includes a backbone module, an encoding module, and a decoding module, wherein: In the crack detection model, the house image is used as the input of the backbone module, the output of the backbone module is used as the input of the encoding module, and the output of the encoding module is used as the input of the decoding module, which outputs the crack detection results. Step 3: Use the images and labels obtained in step 1 to train the crack detection model. During the training process, the loss function value is calculated based on the crack detection results and labels output by the decoding module; Adjust the parameters of the crack detection model according to the back propagation of the loss function value until the loss function value converges and stop training to obtain a trained crack detection model; Step 4: Input the image of the rural house to be detected into the trained crack detection model, and output the crack detection results through the trained crack detection model.

2. A method for detecting cracks in the exterior walls of rural houses according to claim 1, characterized in that: adding a label to each acquired image, wherein the label includes a crack type and a crack position; Crack types include no cracks and various crack grades.

3. The method for detecting cracks in the exterior wall of a rural house according to claim 1, characterized in that: The working process of the backbone module is as follows: The house image is used as the input of the backbone module. In the backbone module, the input house image first passes through the HGStem unit, and then the output of the HGStem unit is used as the input of the first MSFA unit; The output of the first MSFA unit is used as the input of the first depth-wise separable convolutional layer, and the output of the first depth-wise separable convolutional layer is used as the input of the second MSFA unit. The output of the second MSFA unit is used as the input of the second depth-wise separable convolutional layer, and the output of the second depth-wise separable convolutional layer is used as the input of the third MSFA unit. The output of the third MSFA unit is used as the input of the fourth MSFA unit, and the output of the fourth MSFA unit is used as the input of the fifth MSFA unit; The output of the fifth MSFA unit is used as the input of the third depth-wise separable convolutional layer, and the output of the third depth-wise separable convolutional layer is used as the input of the sixth MSFA unit; The output of the first MSFA unit is passed through the sparse context feature enhancement unit to obtain output S3; The output of the second MSFA unit is passed through the sparse context feature enhancement unit to obtain output S4; The output of the sixth MSFA unit is passed through the sparse context feature enhancement unit to obtain output S5.

4. A method for detecting cracks in the exterior walls of rural houses according to claim 3, characterized in that: The working process of the first MSFA unit is: The input of the first MSFA unit After the dilation rate of 1, the output F1 is obtained; After upsampling F1, the upsampling result of F1 is compared with the input The output of the dilation rate 3 dilation convolution layer is added to get the sum. : in, represents a dilated convolutional layer with a dilation rate of 3; Indicates upsampling processing; The result of upsampling F2 is compared with the input The output of the dilation rate 5 dilation convolution layer is added to get the sum : in, Indicates the dilation rate of the dilation layer is 5; Pass F1 through the first convolution layer with a convolution kernel size of 1×1 to obtain the output L1 of the first convolution layer. Then upsample the output L1 and add the upsampled result of L1 to the output of F2 after passing the second convolution layer with a convolution kernel size of 1×1 to obtain the added result L2; Add the up-sampled result of L2 to the output of the third convolutional layer of F3 with a convolution kernel size of 1×1 to obtain the added result L3; Then L1, L2 and L3 are added together, and the addition result is used as the output of the first MSFA unit.

5. A method for detecting cracks in the exterior walls of rural houses according to claim 3, characterized in that: The working process of the sparse context feature enhancement unit is as follows: The feature map output by the first MSFA unit is denoted as P3, and P3 is used as the input of the threshold segmentation subunit. The output of the threshold segmentation subunit is then used as the input of the mask generation subunit, and the output of the mask generation subunit and P3 are used as the input of the sparse convolution layer. The output of the sparse convolution layer is added to the feature map P3, and the addition result is denoted as S3.

6. A method for detecting cracks in the exterior walls of rural houses according to claim 3, characterized in that: The working process of the encoding module is as follows: Step 1: Flatten the feature map S5 to obtain a sequence, then perform position encoding on the obtained sequence, and add the position encoding result to the sequence to obtain a sequence with position encoding added; Step 2: Use the sequence with positional encoding added as the input of the AIFI unit to obtain the output of the AIFI unit; The working process of the AIFI unit is: The sequence with positional encoding added is passed through the multi-head self-attention layer, and then the sequence with positional encoding added is residually connected with the output of the multi-head self-attention layer to obtain the residual connection result a; After performing layer normalization on the residual connection result a, the layer normalization result is input into the FFN layer; Perform a residual connection between the output of the FFN layer and the input of the FFN layer to obtain the residual connection result b; perform layer normalization on the residual connection b, and restore the layer normalization result to the same dimension as the feature map S5. The restored result is recorded as F5; Step 3: Take the feature map S3, feature map S4, and the output F5 of the AIFI unit as the input of the CCFF unit, and output the fused features through the CCFF unit to obtain the output of the encoding module; The CCFF unit includes the first feature fusion subunit to the fourth feature fusion subunit. The working process of the CCFF unit is as follows: The sequence obtained by flattening the feature map S3 is recorded as F6, and the sequence obtained by flattening the feature map S4 is recorded as F7; Use F5 as the input of the fourth convolutional layer with a convolution kernel size of 1×1, then use the output of the fourth convolutional layer as the input of the first BN layer, and the output of the first BN layer as the input of the first SiLU activation function layer; The output of the first SiLU activation function layer and feature map S4 are used as the input of the first feature fusion subunit, the output of the first feature fusion subunit is used as the input of the fifth convolutional layer with a convolution kernel size of 1×1, the output of the fifth convolutional layer is used as the input of the second BN layer, and the output of the second BN layer is used as the input of the second SiLU activation function layer; The output of the second SiLU activation function layer and the feature map S3 are used as the input of the second feature fusion subunit. The output of the second feature fusion subunit is used as the input of the sixth convolutional layer with a convolution kernel size of 3×3. The output of the sixth convolutional layer is used as the input of the third BN layer, and the output of the third BN layer is used as the input of the third SiLU activation function layer. The output of the second SiLU activation function layer and the output of the third SiLU activation function layer are used as the input of the third feature fusion subunit; The output of the third feature fusion subunit is used as the input of the seventh convolutional layer with a convolution kernel size of 3×3, the output of the seventh convolutional layer is used as the input of the fourth BN layer, and the output of the fourth BN layer is used as the input of the fourth SiLU activation function layer; The output of the first SiLU activation function layer and the output of the fourth SiLU activation function layer are used as the input of the fourth feature fusion subunit; Finally, the outputs of the second feature fusion subunit, the third feature fusion subunit, and the fourth feature fusion subunit are spliced, and the splicing result is used as the output of the CCFF unit.

7. A method for detecting cracks in the exterior walls of rural houses according to claim 6, characterized in that: The working process of the first feature fusion subunit is as follows: After concatenating the two inputs of the first feature fusion subunit, the concatenated result is passed through two parallel branches, where: In the first branch, the concatenated result passes through the eighth convolution layer with a convolution kernel size of 1×1; In the second branch, the concatenated result passes through the ninth convolutional layer with a convolution kernel size of 1×1, and then the output of the ninth convolutional layer passes through N series-connected RepBlocks in sequence; The output of the eighth convolutional layer and the output of the last RepBlock are added together, and the result of the addition is flattened. The flattened result is the output of the first feature fusion subunit.

8. The method for detecting cracks in the exterior wall of a rural house according to claim 1, characterized in that: The decoding module includes N' serial decoding units, each of which includes a multi-head self-attention layer, an FFN layer, a multi-scale deformable attention layer and an FFN layer in sequence; The working process of the decoding module is: Step 1: Randomly initialize H learnable query vectors, and then use the uncertainty-minimum query selection strategy to select the query vectors with the top h importance from the H query vectors; For any selected query vector, after position encoding the query vector, the position code is added to the query vector to obtain the query vector with position code added; and each selected query vector is traversed; Step 2: Initialize n=1; Step 3: For any query vector obtained in step 1 , for the query vector Perform linear transformation to generate query ,key Sum : in, 、 and is the weight matrix; Then according to the query ,key Sum Computing attention output : in, Is a query Dimensions; After traversing all query vectors obtained in step 1, the attention outputs corresponding to each query vector are spliced ​​to obtain the splicing result : The stitching results Perform linear transformation to obtain linear transformation result : in, The weight matrix representing the linear transformation; Step 4: Linear transformation results Input the first FFN layer and get the output of the first FFN layer : in, and is the weight matrix; and is the bias term; is the ReLU activation function; Step 5: Perform linear transformation to obtain the query of multi-head deformable attention layer ; When n=1, the output of the encoding module is linearly transformed to obtain the key of the multi-head deformable attention layer Sum ; When n>1, the output of the n-1th decoding unit is linearly transformed to obtain the key of the multi-head deformable attention layer Sum ; Step 6: Use the output of the multi-head deformable attention layer as the input of the second FFN layer, and use the output of the second FFN layer as the output of the n-th decoding unit; Step 7: Let n=n+1, return to step 3, and continue until the output of the N'th decoding unit is obtained. The output of the N'th decoding unit is passed through the detection head, and the detection head is used to output the crack detection result of the image.

9. A method for detecting cracks in the exterior walls of rural houses according to claim 8, characterized in that: The working process of the multi-head deformable attention layer is: in, represents the feature representation of the query Q without linear transformation, is the sampling offset, is the attention weight of the mth attention head at the kth sampling point of the lth layer feature map, , is the total number of layers of feature maps, , is the number of sampling points. Due to the different resolutions of feature maps, represents the coordinates of the query point, Represents the operation of converting the coordinates of the query point to the l-th layer feature map of the m-th attention head, represents the final sampling position, It represents the feature representation obtained by sampling the final sampling position on the l-th layer feature map of the m-th attention head. represents the weight matrix, represents the weight matrix of the linear transformation of the m-th attention head, represents the number of attention heads in the multi-head deformable attention layer, Represents the output of the multi-head deformable attention layer.

10. A method for detecting cracks in the exterior walls of rural houses according to claim 9, characterized in that: The loss function is: in, and is the weighting coefficient; Where: B is the area of ​​the predicted bounding box; is the area of ​​the ground-truth bounding box; and are the width and height of the true bounding box respectively; h and w are the width and height of the predicted bounding box respectively; r and scale are hyperparameters; and Represents the center point coordinates of the true bounding box; and Represents the coordinates of the center point of the predicted bounding box; c represents the scaling factor; Indicates calculation of absolute value; in, is the base of natural logarithms; is a constant; Among them, weight is the mean of the last dimension in the feature map output by the N'th decoding unit.

Images