Deep learning based unmanned aerial vehicle building outer wall crack self-adaptive segmentation method

Abstract

The application belongs to the technical field of computer vision, and discloses a method for self-adaptive segmentation of building outer wall cracks of an unmanned aerial vehicle based on deep learning, which comprises collecting outer wall images in the process that the unmanned aerial vehicle flies along the outer facade of a building, and synchronously acquiring spatial reference information corresponding to the outer wall images; based on the spatial reference information, a spatial reference relationship corresponding to the outer facade of the building is established for the outer wall images; basic image correction processing is performed on the outer wall images, and meanwhile, according to the differences in the surface material and texture features of the outer wall, the contrast and noise suppression parameters of the outer wall images are adaptively adjusted; deep feature coding for crack analysis is performed according to the adjusted outer wall images, and in the coding process, through adaptive modeling of the crack slender morphology, direction consistency and cross-region continuous characteristics, crack structure perception features containing crack structure prior information are formed; high-precision, good-continuity and spatially-locatable automatic segmentation and detection of outer wall cracks are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, more particularly, the present application relates to a deep learning-based adaptive segmentation method for cracks in the outer wall of a building by a UAV. BACKGROUND

[0002] The existing deep learning-based adaptive segmentation method for cracks in the outer wall of a building by a UAV has the following defects:

[0003] In the prior art, the method for segmenting cracks in the outer wall of a building by a UAV usually analyzes the pixel-level semantic features or local texture responses of the outer wall image to realize automatic crack discrimination. The existing method mainly relies on brightness difference, edge strength or convolution feature activation as the basis for crack determination. However, when the crack width is small, the contrast is low or highly similar to the texture of the outer wall, the crack features are easily overwhelmed by background noise, leading to missed detection or broken segmentation, affecting the accuracy of crack detection.

[0004] To solve the problem of small crack size or low contrast, some existing methods introduce multi-scale feature fusion technology, but usually use simple feature splicing or weighted summation, without modeling the differences in local detail sensitivity and overall structural continuity of cracks at different scales, resulting in ineffective coordination of structural semantics between high-resolution features and low-resolution features, affecting the integrity and continuity of the segmentation results.

[0005] In addition, the existing deep learning crack segmentation method usually uses a fixed threshold for threshold selection, which cannot adapt to the lighting conditions, material characteristics and texture differences of different regions of the outer wall, and is prone to crack missed detection or false detection. Traditional methods lack analysis of crack structure response and non-crack response distribution characteristics, and cannot adaptively adjust the segmentation determination, so that local details and crack continuity features cannot be fully utilized. At the same time, the existing method does not fully consider the pixel position-related outer wall scene information, and the segmentation result is difficult to optimize locally for different regions, affecting the robustness and applicability of crack detection.

[0006] In view of the above problems, the present application proposes a deep learning-based adaptive segmentation method for cracks in the outer wall of a building by a UAV. SUMMARY

[0007] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned purpose, the present application provides the following technical solution: a deep learning-based adaptive segmentation method for cracks in the outer wall of a building by a UAV, comprising:

[0008] S1, collecting an outer wall image during the flight of a UAV along the outer facade of a building, and synchronously acquiring spatial reference information corresponding to the outer wall image; based on the spatial reference information, establishing a spatial reference relationship corresponding to the building outer facade for the outer wall image;

[0009] S2, performing a basic image correction process on the outer wall image, and simultaneously performing adaptive adjustment on the outer wall image contrast and noise suppression parameters according to the outer wall surface material and texture feature differences;

[0010] S3, performing a depth feature coding of the crack analysis according to the adjusted outer wall image, and in the coding process, forming a crack structure perception feature containing crack structure prior information by adaptive modeling of the crack elongated shape, direction consistency and cross-region continuous characteristics;

[0011] S4, based on the crack structure perception feature, analyzing the distribution characteristics of the crack structure response and the non-crack structure response in the outer wall image, and dynamically generating a segmentation decision threshold parameter matched with the outer wall image according to the distribution characteristics;

[0012] S5, performing segmentation processing on the crack structure perception feature under the constraint of the segmentation decision threshold parameter, obtaining a segmentation map of the outer wall crack, realizing the positioning and scale expression of the crack region on the building facade in combination with the spatial reference relationship, and generating a corresponding crack detection report.

[0013] Preferably, the method for obtaining the spatial reference information comprises:

[0014] In the process of the unmanned aerial vehicle flying along the building facade, the outer wall images of the building are continuously collected by the carried airborne imaging unit, the outer wall images are recorded in time sequence, and each frame of the outer wall image is assigned with a corresponding collection time identifier;

[0015] At the same time, the unmanned aerial vehicle synchronously obtains the spatial reference information corresponding to the current flight state through the airborne positioning unit and the attitude perception unit; the spatial reference information comprises the position information and the attitude information of the unmanned aerial vehicle at the moment of collecting the outer wall image.

[0016] Preferably, the method for establishing the spatial reference relationship comprises:

[0017] Taking the collection time of the outer wall image as a time alignment reference, each frame of the outer wall image is associated with the synchronously obtained position information and attitude information, forming the outer wall image data with spatial attribute identifiers; the observation position of the outer wall image relative to the building facade is determined according to the position information, and the imaging direction and the viewing angle range of the outer wall image are determined according to the attitude information;

[0018] A preset reference plane is provided, the spatial corresponding relationship between the outer wall image and the building facade is constructed by projecting the observation position and the imaging direction of the outer wall image onto the reference plane of the building facade; the outer wall image is divided into different field of view sub-regions along the imaging direction and the viewing angle range, each field of view sub-region corresponds to a different projection section on the reference plane, and the projection section is assigned with a corresponding spatial index identifier.

[0019] Preferably, the method for adaptively adjusting the contrast and noise suppression parameters of the external wall image comprises:

[0020] Performing a basic image correction process on the external wall image, the basic image correction process comprising a geometric distortion correction and a brightness consistency correction on the external wall image, the geometric distortion correction being performed by mapping and correcting the spatial positions of each pixel point in the external wall image by using the imaging parameters of the onboard imaging unit;

[0021] The brightness consistency correction is performed by statistically analyzing the brightness of different regions in the external wall image, estimating the brightness difference caused by changes in lighting conditions and shooting angles, and compensating the brightness component of the external wall image;

[0022] After the basic image correction process is completed, the gray scale statistical parameters are obtained by statistically analyzing the gray scale mean and gray scale variance of the local region in the external wall image, the texture variation characteristic parameters are obtained by statistically analyzing the frequency and amplitude of the pixel intensity variation, and the edge feature parameters are obtained by detecting the edge distribution density of the local region in the external wall image through an edge detection algorithm;

[0023] The gray scale statistical parameters, the texture variation characteristic parameters and the edge feature parameters are combined to form image feature parameters representing the material difference of the external wall surface, and the strength of the contrast enhancement and the noise suppression parameters of the external wall image are adaptively adjusted based on the image feature parameters to obtain the adjusted external wall image.

[0024] Preferably, the method for performing deep feature coding of the crack analysis comprises:

[0025] The adjusted external wall image is input into a pre-trained deep feature coding network to perform deep feature coding processing of the crack analysis on the external wall image; the training process of the deep feature coding network is based on a pre-constructed external wall crack image training sample set, the training sample set comprising an external wall image sample set and corresponding crack annotation information of the external wall image sample set; the deep feature coding network is any one of a convolutional neural network model, a graph neural network model or a self-encoding model;

[0026] In the training process, the external wall image sample set in the training sample set is input into the deep feature coding network, the input external wall image sample set is extracted layer by layer through convolution operation, nonlinear activation and downsampling operation on the external wall image to output corresponding crack information features;

[0027] The crack information features are compared with the crack labeling information, a loss function for measuring the consistency of the prediction results and the true labels is constructed, the network parameters are optimized based on the loss function, the gradient of the loss function with respect to the network parameters is calculated through a back propagation algorithm, and the network parameters are iteratively updated by using a gradient descent algorithm;

[0028] When the change amplitude of the loss function is less than a preset change amplitude threshold during the training iteration process, it is determined that the deep feature coding network reaches a convergence state, and the updating of the network parameters is stopped; the deep feature coding network after the training is performed on the adjusted external wall image to obtain crack information features for representing the crack structure of the external wall.

[0029] Preferably, the method for forming the crack structure perception features comprises:

[0030] Based on the multi-scale feature representations of the crack information features corresponding to different network levels in the deep feature coding network, the feature responses of each pixel position in the external wall image are analyzed to obtain multi-scale crack features reflecting the structural change characteristics of the cracks at different spatial scales;

[0031] A crack structure consistency response function is introduced to depict the gradient accumulation characteristics related to the elongated morphology of the cracks in the multi-scale crack features, and a structure consistency response strength representing the elongated morphology characteristics of the cracks is obtained; based on the structure consistency response strength, the main direction information of the crack information features at the pixel position is extracted;

[0032] A direction continuity constraint function is constructed to compare the main direction of the crack information features at the pixel position with the main direction of the crack information features in the pixel neighborhood region, and a direction consistency constraint representing the direction continuity characteristics of the cracks is constructed.

[0033] The structure consistency response strength and the direction consistency constraint are fused to generate a crack structure perception strength representing the matching degree of the crack structure at the pixel position, thereby forming the crack structure perception features containing the crack structure prior information.

[0034] Preferably, the method for analyzing the distribution characteristics of the crack structure responses and the non-crack structure responses in the external wall image comprises:

[0035] In the range of the external wall image, the crack structure perception strength corresponding to each pixel position is taken as a crack structure response representation quantity, and the spatial distribution of the crack structure perception strength in the range of the external wall image is statistically analyzed; by statistically analyzing the concentration degree and the dispersion degree of the crack structure perception strength in the external wall image, distribution description parameters reflecting the distinguishing characteristics of the crack structure responses and the non-crack structure responses are obtained.

[0036] Preferably, the method for generating the segmentation determination threshold domain parameters comprises:

[0037] Based on the distribution description parameters, an adaptive threshold function is constructed, and a segmentation decision threshold is dynamically generated according to the statistical distribution characteristics of the crack structure response; in the threshold generation process, a scene modulation function related to the pixel position is introduced to locally modulate the segmentation decision threshold; the segmentation decision threshold is calculated pixel by pixel for the outer wall image to obtain the corresponding segmentation decision threshold parameters generated by different pixel positions in the outer wall image.

[0038] Preferably, the method for obtaining the segmentation map of the outer wall crack comprises:

[0039] Under the constraint of the segmentation decision threshold parameter, pixel-level decision processing is performed on the crack structure perception features of each pixel position in the outer wall image; for any pixel position in the outer wall image, if the crack structure perception feature value of the pixel is greater than or equal to the corresponding segmentation decision threshold, it is determined that the pixel belongs to the crack region, otherwise it is determined that the pixel belongs to the non-crack region; the determination results of all pixels are combined to obtain the segmentation map of the outer wall crack.

[0040] Preferably, the method for generating the corresponding crack detection report comprises:

[0041] According to the spatial reference relationship of each frame of outer wall image, the pixels determined as cracks in the segmentation map are mapped to the reference plane of the building facade, the mapped crack pixels are subjected to connectivity analysis, the adjacent or continuous crack pixels are aggregated into crack regions, and the spatial scale information of each crack region on the building facade is obtained, the spatial scale information including crack length, width and starting position; based on the spatial scale information and position relationship of the crack region, a crack detection report is generated.

[0042] The technical effects and advantages of the unmanned aerial vehicle building outer wall crack adaptive segmentation method based on deep learning are as follows:

[0043] By obtaining multi-scale crack information features at different levels of the deep feature encoding network, and spatially aligning and combining the feature vectors of the same pixel at different scales, the structural response of the crack at the local detail scale and the overall extension scale is uniformly expressed; by weighting and accumulating the spatial gradients of the multi-scale crack features, a structural consistency response function is constructed, the gradient continuity of the crack along the main direction is converted into a quantifiable response strength, thereby suppressing the interference of non-slim structure and highlighting the linear geometric properties of the crack. By performing local direction analysis on the structural consistency response features, the main direction information of the pixel-level crack is extracted, and a direction continuity constraint function is constructed to constrain the continuous consistency of the crack direction within the neighborhood range in the form of cosine similarity, effectively reducing the crack breaking and direction deviation problems.

[0044] The distribution characteristics of the crack structure perception features are counted to dynamically generate a segmentation decision threshold, avoiding the missing detection or false detection problems caused by the fixed threshold. A pixel position related scene modulation function is introduced to integrate the information such as light, material and texture complexity into the threshold generation, realizing the local self-adaptation of the segmentation decision. The segmentation decision threshold is calculated for the outer wall image pixel by pixel, so that the crack decision of each pixel can combine the local distribution characteristics and scene information, improving the accuracy and reliability of the crack detection. It can be applied to different building outer wall materials and complex environmental conditions, ensuring more accurate and continuous crack segmentation results, which is beneficial to crack positioning and size expression. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 FIG. 1 is a flowchart of a deep learning based adaptive segmentation method for cracks in the outer wall of a building by a UAV.

[0046] Figure 2 FIG. 2 is a schematic diagram of a deep learning based adaptive segmentation system for cracks in the outer wall of a building by a UAV. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0048] Embodiment 1.

[0049] Please refer to Figure 1 The embodiment provides a deep learning based adaptive segmentation method for cracks in the outer wall of a building by a UAV, which specifically includes the following steps:

[0050] S1, collecting an outer wall image during the flight of a UAV along the outer facade of a building, and synchronously acquiring spatial reference information corresponding to the outer wall image; based on the spatial reference information, establishing a spatial reference relationship corresponding to the building outer facade for the outer wall image;

[0051] S2, performing a basic image correction process on the outer wall image, and simultaneously adaptively adjusting the contrast and noise suppression parameters of the outer wall image according to the differences in the surface material and texture characteristics of the outer wall;

[0052] S3, performing deep feature coding for crack analysis according to the adjusted outer wall image, and in the coding process, forming crack structure perception features containing crack structure prior information by adaptively modeling the crack elongated shape, direction consistency and cross-region continuity characteristics;

[0053] S4. Based on the crack structure perception features, analyze the distribution characteristics of crack structure response and non-crack structure response in the external wall image, and dynamically generate segmentation judgment threshold parameters that match the external wall image according to the distribution characteristics.

[0054] S5. Under the constraint of the segmentation judgment threshold parameter, the segmentation processing of the crack structure perception feature is performed to obtain the segmentation map of the external wall crack; combined with the spatial reference relationship, the location and scale expression of the crack area on the building facade are realized, and the corresponding crack detection report is generated.

[0055] Methods for obtaining spatial reference information include:

[0056] As the drone flies along the building facade, it continuously acquires images of the building's exterior walls through its onboard imaging unit, records the exterior wall images in chronological order, and assigns a corresponding acquisition time marker to each frame of the exterior wall image.

[0057] It should be noted that the airborne imaging unit can be a high-definition visible light imaging device installed on the drone body, such as an industrial-grade CMOS camera or a multi-lens visible light camera, used to continuously image and acquire data on the exterior wall surface as the drone flies along the building facade.

[0058] At the same time, the UAV acquires spatial reference information corresponding to its current flight status through its onboard positioning unit and attitude perception unit; the spatial reference information includes the UAV's position and attitude information at the moment of image acquisition on the exterior wall.

[0059] Location information includes the longitude, latitude, and altitude of the UAV at the time of image acquisition; attitude information includes the UAV's pitch angle, roll angle, and yaw angle; it should be noted that the airborne positioning unit can be a global satellite navigation and positioning module integrated on the UAV, such as a GPS or BeiDou positioning module, used to obtain the UAV's location information in geospatial space in real time during the external wall image acquisition process.

[0060] The attitude sensing unit can be an inertial measurement unit installed on a drone, used to synchronously acquire the drone's attitude state information during the acquisition of exterior wall images. This attitude sensing unit can output the drone's attitude angle parameters at the moment of image acquisition, such as roll, pitch, and yaw angles.

[0061] Methods for establishing spatial reference relationships include:

[0062] Using the acquisition time of the exterior wall image as the time alignment reference, each frame of the exterior wall image is associated with the synchronously acquired position and attitude information to form exterior wall image data with spatial attribute identifiers; the observation position of the exterior wall image relative to the building facade is determined according to the position information, which is used to characterize the distance, orientation and height relationship between the UAV and the building facade; and the imaging direction and viewing angle range of the exterior wall image are determined according to the attitude information.

[0063] A preset reference plane is a plane pre-defined based on the overall orientation of the building facade, used to characterize the spatial distribution of the building facade. By projecting the observation position and the imaging direction of the exterior wall image onto the reference plane of the building facade, a spatial correspondence between the exterior wall image and the building facade is constructed. The exterior wall image is divided into different field-of-view sub-regions along the imaging direction and viewing angle range. Each field-of-view sub-region corresponds to a different projection segment on the reference plane, and a corresponding spatial index label is assigned to the projection segment.

[0064] Methods for adaptively adjusting the contrast and noise suppression parameters of exterior wall images include:

[0065] Basic image correction processing is performed on the external wall image. The basic image correction processing includes geometric distortion correction and brightness consistency correction of the external wall image. Geometric distortion correction is performed by mapping and correcting the spatial position of each pixel in the external wall image using the imaging parameters of the airborne imaging unit.

[0066] Brightness consistency correction estimates the brightness differences caused by changes in lighting conditions and shooting angle by statistically comparing the brightness of different areas in the exterior wall image, and then compensates for the brightness components of the exterior wall image. The compensation process includes applying brightness gain to areas with low brightness and performing brightness compression on areas with high brightness.

[0067] Geometric distortion correction is used to correct radial and tangential distortion caused by the characteristics of the airborne imaging unit lens. Brightness consistency correction is used to reduce brightness differences caused by changes in shooting angle, uneven lighting conditions, or shadow occlusion, so as to obtain an exterior wall image with more consistent overall brightness and geometry; thereby reducing image distortion caused by lens characteristics, changes in shooting angle, and uneven lighting during the imaging process.

[0068] After completing the basic image correction process, gray-level statistical parameters are obtained by statistically analyzing the mean and variance of gray levels in local areas of the exterior wall image; texture change feature parameters are obtained by statistically analyzing the frequency and amplitude of pixel intensity changes; and edge feature parameters are obtained by detecting the edge distribution density in local areas of the exterior wall image using an edge detection algorithm.

[0069] By combining grayscale statistical parameters, texture variation feature parameters, and edge feature parameters, image feature parameters characterizing the differences in material properties of the exterior wall surface are constructed. Based on these image feature parameters, the intensity of contrast enhancement and noise suppression parameters of the exterior wall image are adaptively adjusted to obtain the adjusted exterior wall image. This approach aims to enhance the contrast between the crack area and the background area while suppressing random noise interference and maintaining the slender structure and edge continuity of the cracks.

[0070] It should be noted that the process of adaptively adjusting the intensity of contrast enhancement and noise suppression parameters of the exterior wall image based on image feature parameters includes determining the overall brightness and contrast level of the exterior wall image according to grayscale statistical parameters, and adjusting the intensity of contrast enhancement accordingly.

[0071] The complexity of the exterior wall surface texture is determined based on the texture change feature parameters. When the frequency and amplitude of pixel intensity changes are high, the contrast is reduced and the intensity is increased to avoid excessive texture magnification. The clarity of the local structure is determined based on the edge feature parameters, and the noise suppression parameters are adjusted accordingly. When the edge distribution density is high, the noise suppression intensity is reduced to preserve fine structural features such as cracks.

[0072] Methods for performing depth feature encoding in crack analysis include:

[0073] The adjusted exterior wall image is input into a pre-trained deep feature encoding network to perform deep feature encoding processing for crack analysis on the exterior wall image. The training process of the deep feature encoding network is based on a pre-constructed training sample set of exterior wall crack images, which includes an exterior wall image sample set and crack annotation information corresponding to the exterior wall image sample set. The deep feature encoding network can be any one of a convolutional neural network model, a graph neural network model, or an autoencoder model.

[0074] During training, the external wall image sample set in the training sample set is input into the deep feature encoding network. By performing convolution operation, nonlinear activation and downsampling operation on the external wall images, the input external wall image sample set is subjected to layer-by-layer feature extraction, and the corresponding crack information features are output.

[0075] The crack information features are compared with the crack annotation information to construct a loss function to measure the consistency between the prediction results and the actual annotations. The network parameters are optimized based on the loss function. The gradient of the loss function with respect to the network parameters is calculated by the backpropagation algorithm, and the network parameters are iteratively updated by the gradient descent algorithm.

[0076] When the change in the loss function is less than the preset change threshold during the training iteration, the deep feature encoding network is determined to have reached convergence, and the update of the network parameters is stopped. The adjusted exterior wall image is then encoded using the trained deep feature encoding network to obtain crack information features that characterize the exterior wall crack structure.

[0077] It should be noted that the preset change range threshold is set by staff. By collecting different change ranges, the average of multiple change ranges is taken as the preset change range threshold.

[0078] Methods for forming crack structure sensing features include:

[0079] Based on the multi-scale feature representation of crack information features at different network levels in a deep feature coding network, the feature response of each pixel position in the external wall image is analyzed to obtain multi-scale crack features that reflect the structural change characteristics of cracks at different spatial scales.

[0080] It should be noted that the deep feature encoding network outputs feature maps corresponding to crack information features at different network layers. These feature maps at different network layers have different spatial resolutions and receptive fields. For the same pixel location, the corresponding feature vector is obtained from the feature maps of each network layer, and these feature vectors are aligned so that features at different scales correspond to the same pixel location in a unified spatial coordinate system. Subsequently, the feature vectors corresponding to the pixel location at each network layer are combined to form a multi-scale crack feature that characterizes the structural response of the pixel at different spatial scales. High-resolution features reflect local detail changes in the crack, while low-resolution features reflect the overall extension structure of the crack.

[0081] A crack structure consistency response function is introduced to characterize the gradient accumulation characteristics related to the slender morphology of cracks in multi-scale crack features, and obtain the structure consistency response intensity that characterizes the slender morphology of cracks; based on the structure consistency response intensity, the main direction information of crack information features at pixel locations is extracted.

[0082] The structural uniformity response function for cracks is: ;in, Indicates pixel position The structural uniformity of the crack at the location; This represents a pixel location in the image of the exterior wall; This indicates that the deep feature encoding network is in the first... Layer, pixel position The crack information features output at the location; Indicates the first Layer crack information features at pixel location Spatial gradient at a given location; Represents the magnitude of the spatial gradient; Indicates the first Weighting coefficients corresponding to the characteristics of layer crack information; This indicates the number of deep feature coding network layers involved in multi-scale feature analysis; Indicates the index of the deep feature encoding network layer;

[0083] It should be noted that after obtaining the structural consistency response intensity characterizing the slender morphology of the crack, local directional analysis is performed on the feature map corresponding to the structural consistency response intensity at the pixel location. By calculating the gradient components of the feature response at the pixel location in the horizontal and vertical directions, the directional response vector at that pixel location is obtained. This directional response vector is used to characterize the crack response intensity of the pixel in different directions and is a discrete set of directions. In the discrete set of directions, the direction with the largest crack response intensity is selected as the principal directional response vector, and the principal directional response vector is used as the principal directional information of the crack information feature at that pixel location.

[0084] A directional continuity constraint function is constructed, which compares the main direction of the crack information feature at the pixel location with the main direction of the crack information feature within the pixel's neighborhood region to construct a directional consistency constraint that characterizes the directional continuity of the crack.

[0085] The directional continuity constraint function is: ;in, Indicates pixel position The directional continuity constraint value is used to characterize the consistency between the main direction of the crack information features at that pixel location and the main direction of the crack information features within its neighborhood. Cosine similarity is used to measure the directional difference; the more consistent the directions, the better. The closer to 1; Indicates pixel position The principal direction response vector of the crack information characteristics; Indicates pixel position The main direction response vector of crack information features within the neighborhood region; Indicates pixel position The neighborhood region is used to calculate the consistency of the crack direction of neighboring pixels. The scope can be specified, for example, by pixel position. Centered on, with radius The circular neighborhood; Indicates pixel position The direction similarity between the direction of a crack and the direction of cracks in its neighborhood is such that the smaller the direction difference, the closer the corresponding cosine value is to 1.

[0086] By fusing structural consistency response intensity with directional consistency constraints, a crack structure sensing intensity is generated that characterizes the degree of crack structure matching at pixel locations. This leads to the formation of crack structure perception features that contain prior information about the crack structure.

[0087] It should be noted that after obtaining the crack structure perception intensity at the pixel location, this intensity is used as a structural weighting factor to modulate the original crack information features. This modulates the crack information features, enhancing the feature responses at pixel locations with high crack structure perception intensity and suppressing those at low intensity. The structure-weighted modulated feature responses are then used as the crack structure perception features. This allows the crack structure perception features to retain the semantic expressive power of the original crack information features while further incorporating prior structural information such as the crack's elongated shape, directional consistency, and cross-regional continuity.

[0088] This paper addresses the following technical problems in existing technologies: Existing methods for segmenting cracks in building exterior walls largely rely on pixel-level semantic features or local texture responses for discrimination, typically using brightness differences, edge intensity, or convolutional feature activation as the primary basis. When the crack width is extremely small, the contrast is low, or it is highly similar to the exterior wall texture, the crack features are easily submerged by background noise, leading to missed detections or fragmentation. Although existing methods introduce multi-scale feature fusion, they often employ simple stitching or weighted summation, failing to differentiate and model the differences in local detail sensitivity and overall structural continuity exhibited by cracks at different scales. This results in a failure to effectively synergize the structural semantics between high-resolution and low-resolution features.

[0089] Compared to existing technologies, the advantages are as follows: By acquiring multi-scale crack information features at different levels of a deep feature encoding network and spatially aligning and combining feature vectors of the same pixel at different scales, the structural response of cracks at both the local detail scale and the overall extension scale can be uniformly expressed; by weighted accumulation of the spatial gradients of multi-scale crack features, a structural consistency response function is constructed, transforming the gradient continuity of cracks along the main direction into quantifiable response intensity, thereby suppressing interference from non-slender structures and highlighting the unique linear geometric properties of cracks. By performing local direction analysis on the structural consistency response features, pixel-level crack main direction information is extracted, and a direction continuity constraint function is constructed to constrain the continuity and consistency of crack direction within the neighborhood in the form of cosine similarity, effectively reducing crack breakage and direction shift problems.

[0090] Methods for analyzing the distribution characteristics of cracked and non-cracked structural responses in exterior wall images include:

[0091] Within the scope of the exterior wall image, the perceived intensity of the crack structure at each pixel location is used as the characterization quantity of the crack structure response. The spatial distribution of the perceived intensity of the crack structure within the scope of the exterior wall image is statistically analyzed. By statistically analyzing the concentration and dispersion of the perceived intensity of the crack structure in the exterior wall image, distribution description parameters reflecting the distinguishing characteristics between the crack structure response and the non-crack structure response are obtained.

[0092] The degree of concentration is: ;in, This represents the average intensity of crack structure perception at all pixel locations in the exterior wall image, reflecting the concentration of crack structure perception intensity in the current exterior wall scene. This represents the mathematical expectation operation, used for overall statistical analysis of the perceived intensity of crack structures within the range of an external wall image;

[0093] The degree of dispersion is: ;in, The variance representing the perceived intensity of the cracked structure reflects the degree of distinction between cracked and non-cracked regions in structural response. This represents variance calculation, used to measure the fluctuation of the perceived intensity of crack structures in external wall images;

[0094] Methods for generating segmentation threshold parameters include:

[0095] Based on the distribution description parameters, an adaptive threshold function is constructed to dynamically generate the segmentation judgment threshold according to the statistical distribution characteristics of the crack structure response. During the threshold generation process, a scene modulation function related to the pixel position is introduced to locally modulate the segmentation judgment threshold. The segmentation judgment threshold is calculated pixel by pixel in the external wall image to obtain the corresponding segmentation judgment threshold parameters generated at different pixel positions in the external wall image.

[0096] The adaptive threshold function is: ;in, Indicates pixel position The corresponding segmentation threshold is used as the criterion for determining whether a pixel belongs to the crack region during crack segmentation. This represents the threshold adjustment factor, which is used to control the influence weight of the intensity distribution characteristics of the crack structure in the threshold generation process. The threshold adjustment factor can be set according to the system design requirements. This represents the scene modulation function, used to introduce a modulation scheme related to pixel position. The relevant exterior wall scene information is used to locally modulate the segmentation judgment threshold. The exterior wall scene information includes the lighting conditions, surface material characteristics or texture complexity of the exterior wall image, thereby reflecting the differences in the sensitivity of different exterior wall areas to crack judgment.

[0097] The scene modulation function is: ;in, This indicates the preset reference segmentation threshold, while this indicates the default segmentation threshold in the absence of scene modulation. This represents the adjustment coefficient, used to control the strength of the effect of scene modulation on the threshold. The larger the value, the greater the influence of scene information on the threshold; This measure indicates the indistinguishability between the local crack response distribution and the background response distribution. It is obtained by statistically analyzing the structural sensing intensity distribution characteristics of crack response regions and non-crack response regions, and calculating the overlap (intersection area) of the two response distributions, reflecting the pixel... The difficulty in distinguishing the structural response between cracked and non-cracked structures in the same area;

[0098] This solution addresses the following technical problems of existing technologies: Existing deep learning crack segmentation methods typically use fixed thresholds, failing to account for differences in lighting, material, and texture across different areas of the exterior wall, easily leading to missed or false crack detections. Traditional methods lack analysis of the distribution characteristics of crack and non-crack structural responses, failing to adaptively adjust segmentation decisions, resulting in underutilization of local details and continuity features. Existing methods do not adequately consider pixel-position-related exterior wall scene information, making it difficult to locally optimize segmentation results for different regions.

[0099] Compared to existing technologies, the advantages are as follows: By dynamically generating segmentation thresholds based on the distribution characteristics of crack structure perception features, the problems of missed or false detections caused by fixed thresholds are avoided. A scene modulation function related to pixel position is introduced, incorporating information such as illumination, material, and texture complexity into the threshold generation, achieving local adaptation in segmentation judgment. Segmentation thresholds are calculated pixel-by-pixel for the exterior wall image, ensuring that crack judgment for each pixel combines local distribution features and scene information, improving the accuracy and reliability of crack detection. It is applicable to different building exterior wall materials and complex environmental conditions, ensuring more accurate and continuous crack segmentation results, which is beneficial for crack localization and size representation.

[0100] Methods for obtaining segmentation diagrams of external wall cracks include:

[0101] Under the constraint of the segmentation judgment threshold parameter, pixel-level judgment processing is performed on the crack structure perception features of each pixel position in the external wall image. For any pixel position in the external wall image, if the crack structure perception feature value of the pixel is greater than or equal to the corresponding segmentation judgment threshold, the pixel is judged to belong to the crack region; otherwise, the pixel is judged to belong to the non-crack region. The judgment results of all pixels are combined to obtain the segmentation map of the external wall crack.

[0102] Methods for generating corresponding crack detection reports include:

[0103] Based on the spatial reference relationship of each frame of exterior wall image, the pixels identified as cracks in the segmentation image are mapped to the reference plane of the building facade. Connectivity analysis is performed on the mapped crack pixels, and adjacent or continuous crack pixels are aggregated into crack regions. The spatial scale information of each crack region on the building facade is obtained, including crack length, width and starting position. Based on the spatial scale information and positional relationship of the crack regions, a crack detection report is generated.

[0104] It should be noted that the crack detection report includes the number of cracks, the length, width, direction of each crack, and its spatial coordinates on the building facade. It can also be classified and statistically presented or visualized according to different floors or facade areas.

[0105] This embodiment acquires multi-scale crack information features at different levels of a deep feature encoding network and spatially aligns and combines feature vectors of the same pixel at different scales, enabling a unified expression of the structural response of cracks at both local detail and overall extension scales. By weighted accumulation of the spatial gradients of the multi-scale crack features, a structural consistency response function is constructed, transforming the gradient continuity of the crack along the principal direction into a quantifiable response intensity. This suppresses interference from non-slender structures and highlights the unique linear geometric properties of cracks. Local directional analysis of the structural consistency response features extracts pixel-level crack principal direction information, and a direction continuity constraint function is constructed. This function uses cosine similarity to constrain the continuity and consistency of the crack direction within its neighborhood, effectively reducing crack breakage and direction shift problems.

[0106] By dynamically generating segmentation thresholds based on the distribution characteristics of crack structure perception features, the problem of missed or false detections caused by fixed thresholds is avoided. A scene modulation function related to pixel position is introduced, incorporating information such as illumination, material, and texture complexity into the threshold generation, achieving local adaptation in segmentation judgment. The segmentation threshold is calculated pixel-by-pixel in the exterior wall image, ensuring that crack judgment for each pixel combines local distribution features and scene information, improving the accuracy and reliability of crack detection. It is applicable to different building exterior wall materials and complex environmental conditions, ensuring more accurate and continuous crack segmentation results, which is beneficial for crack localization and size representation.

[0107] Example 2.

[0108] Please see Figure 2 As shown, parts not described in detail in this embodiment are described in Embodiment 1. A deep learning-based UAV-based adaptive segmentation system for building exterior wall cracks is provided, including:

[0109] The UAV data acquisition module is used to acquire images of the exterior walls while the UAV flies along the building facade, and simultaneously obtain spatial reference information corresponding to the exterior wall images; based on the spatial reference information, a spatial reference relationship is established between the exterior wall images and the building facade.

[0110] The image feature enhancement module is used to perform basic image correction processing on the exterior wall image, and at the same time, adaptively adjust the contrast and noise suppression parameters of the exterior wall image according to the differences in the material and texture features of the exterior wall surface.

[0111] The crack structure prior module is used to perform deep feature encoding for crack analysis based on the adjusted external wall image. During the encoding process, the crack structure perception feature containing crack structure prior information is formed by adaptively modeling the slender shape, directional consistency and cross-regional continuity of the crack.

[0112] The segmentation threshold adjustment module analyzes the distribution characteristics of crack structure response and non-crack structure response in the external wall image based on crack structure perception features, and dynamically generates segmentation judgment threshold parameters that match the external wall image according to the distribution characteristics.

[0113] The segmentation result generation module is used to perform segmentation processing on the perceived features of crack structures under the constraints of segmentation judgment threshold parameters to obtain a segmentation map of external wall cracks; combined with spatial reference relationships, it realizes the positioning and scale expression of crack areas on the building facade and generates corresponding crack detection reports.

[0114] Since the electronic device described in this embodiment is the one used to implement the deep learning-based UAV adaptive segmentation method for building exterior wall cracks in this application embodiment, those skilled in the art can understand the specific implementation and various variations of the electronic device in this embodiment based on the deep learning-based UAV adaptive segmentation method for building exterior wall cracks described in this application embodiment. Therefore, how the electronic device implements the method in this application embodiment will not be described in detail here. Any electronic device used by those skilled in the art to implement the deep learning-based UAV adaptive segmentation method for building exterior wall cracks in this application embodiment falls within the scope of protection of this application.

[0115] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0116] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for users of ordinary technical skills, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A deep learning-based adaptive segmentation method for building exterior wall cracks using unmanned aerial vehicles (UAVs), characterized in that, include: S1. Collect images of the exterior walls while the drone flies along the building facade, and simultaneously acquire spatial reference information corresponding to the exterior wall images; Based on spatial reference information, establish a spatial reference relationship between the exterior wall image and the building facade; S2. Perform basic image correction processing on the exterior wall image, and adaptively adjust the contrast and noise suppression parameters of the exterior wall image according to the differences in the material and texture characteristics of the exterior wall surface. S3. Perform depth feature encoding for crack analysis based on the adjusted exterior wall image. During the encoding process, adaptive modeling of the crack's elongated shape, directional consistency, and cross-regional continuity characteristics is used to form crack structure perception features containing prior information about the crack structure. The method for performing depth feature encoding for crack analysis includes: The adjusted exterior wall image is input into a pre-trained deep feature encoding network to perform deep feature encoding processing for crack analysis on the exterior wall image. The training process of the deep feature encoding network is based on a pre-constructed training sample set of exterior wall crack images, which includes an exterior wall image sample set and crack annotation information corresponding to the exterior wall image sample set. The deep feature encoding network can be any one of a convolutional neural network model, a graph neural network model, or an autoencoder model. During training, the external wall image sample set in the training sample set is input into the deep feature encoding network. By performing convolution operation, nonlinear activation and downsampling operation on the external wall images, the input external wall image sample set is subjected to layer-by-layer feature extraction, and the corresponding crack information features are output. The crack information features are compared with the crack annotation information to construct a loss function to measure the consistency between the prediction results and the actual annotations. The network parameters are optimized based on the loss function. The gradient of the loss function with respect to the network parameters is calculated by the backpropagation algorithm, and the network parameters are iteratively updated by the gradient descent algorithm. When the change in the loss function is less than the preset change threshold during the training iteration, the deep feature encoding network is determined to have reached convergence, and the update of the network parameters is stopped. The adjusted exterior wall image is then subjected to deep feature encoding by the trained deep feature encoding network to obtain crack information features that characterize the exterior wall crack structure. The method for forming the crack structure sensing features includes: Based on the multi-scale feature representation of crack information features at different network levels in a deep feature coding network, the feature response of each pixel position in the external wall image is analyzed to obtain multi-scale crack features that reflect the structural change characteristics of cracks at different spatial scales. A crack structure consistency response function is introduced to characterize the gradient accumulation characteristics related to the slender morphology of cracks in multi-scale crack features, and obtain the structure consistency response intensity that characterizes the slender morphology of cracks; based on the structure consistency response intensity, the main direction information of crack information features at pixel locations is extracted. A directional continuity constraint function is constructed, which compares the main direction of the crack information feature at the pixel location with the main direction of the crack information feature within the pixel's neighborhood region to construct a directional consistency constraint that characterizes the directional continuity of the crack. By fusing structural consistency response intensity with directional consistency constraint, a crack structure perception intensity is generated that characterizes the degree of crack structure matching at the pixel location, thereby forming a crack structure perception feature containing prior information about the crack structure. S4. Based on the crack structure perception features, analyze the distribution characteristics of crack structure response and non-crack structure response in the external wall image, and dynamically generate segmentation judgment threshold parameters that match the external wall image according to the distribution characteristics. S5. Under the constraint of the segmentation judgment threshold parameter, the segmentation processing of the crack structure perception feature is performed to obtain the segmentation map of the external wall crack; combined with the spatial reference relationship, the location and scale expression of the crack area on the building facade are realized, and the corresponding crack detection report is generated.

2. The deep learning-based UAV-based adaptive segmentation method for building exterior wall cracks according to claim 1, characterized in that, The method for obtaining the spatial reference information includes: As the drone flies along the building facade, it continuously acquires images of the building's exterior walls through its onboard imaging unit, records the exterior wall images in chronological order, and assigns a corresponding acquisition time marker to each frame of the exterior wall image. At the same time, the UAV acquires spatial reference information corresponding to its current flight status through its onboard positioning unit and attitude perception unit; the spatial reference information includes the UAV's position and attitude information at the moment of image acquisition on the exterior wall.

3. The deep learning-based UAV-based adaptive segmentation method for building exterior wall cracks according to claim 2, characterized in that, The method for establishing the spatial reference relationship includes: Using the acquisition time of the exterior wall image as the time alignment reference, each frame of the exterior wall image is associated with the synchronously acquired position and attitude information to form exterior wall image data with spatial attribute identifiers; the observation position of the exterior wall image relative to the building facade is determined based on the position information, and the imaging direction and viewing angle range of the exterior wall image are determined based on the attitude information. A reference plane is preset, and the spatial correspondence between the exterior wall image and the building facade is established by projecting the observation position and the imaging direction of the exterior wall image onto the reference plane of the building facade. The exterior wall image is divided into different field-of-view sub-regions along the imaging direction and viewing angle range. Each field-of-view sub-region corresponds to a different projection segment on the reference plane, and a corresponding spatial index label is assigned to the projection segment.

4. The deep learning-based UAV-based adaptive segmentation method for building exterior wall cracks according to claim 3, characterized in that, The method for adaptively adjusting the contrast and noise suppression parameters of the external wall image includes: Basic image correction processing is performed on the external wall image. The basic image correction processing includes geometric distortion correction and brightness consistency correction of the external wall image. Geometric distortion correction is performed by mapping and correcting the spatial position of each pixel in the external wall image using the imaging parameters of the airborne imaging unit. Brightness consistency correction estimates the brightness differences caused by changes in lighting conditions and shooting angle by statistically comparing the brightness of different areas in the exterior wall image, and then compensates for the brightness components of the exterior wall image. After completing the basic image correction process, gray-level statistical parameters are obtained by statistically analyzing the mean and variance of gray levels in local areas of the exterior wall image; texture change feature parameters are obtained by statistically analyzing the frequency and amplitude of pixel intensity changes; and edge feature parameters are obtained by detecting the edge distribution density in local areas of the exterior wall image using an edge detection algorithm. By combining grayscale statistical parameters, texture variation feature parameters, and edge feature parameters, image feature parameters that characterize the differences in material properties of the exterior wall surface are constructed. Based on these image feature parameters, the intensity of contrast enhancement and noise suppression parameters of the exterior wall image are adaptively adjusted to obtain the adjusted exterior wall image.

5. The deep learning-based UAV-based adaptive segmentation method for building exterior wall cracks according to claim 4, characterized in that, The method for analyzing the distribution characteristics of cracked and non-cracked structural responses in exterior wall images includes: Within the scope of the exterior wall image, the perceived intensity of the crack structure at each pixel location is used as the characterization quantity of the crack structure response. The spatial distribution of the perceived intensity of the crack structure within the scope of the exterior wall image is statistically analyzed. By statistically analyzing the concentration and dispersion of the perceived intensity of the crack structure in the exterior wall image, distribution description parameters reflecting the distinguishing characteristics between the crack structure response and the non-crack structure response are obtained.

6. The deep learning-based UAV-based adaptive segmentation method for building exterior wall cracks according to claim 5, characterized in that, The method for generating the segmentation determination threshold parameter includes: Based on the distribution description parameters, an adaptive threshold function is constructed to dynamically generate the segmentation judgment threshold according to the statistical distribution characteristics of the crack structure response. During the threshold generation process, a scene modulation function related to the pixel position is introduced to locally modulate the segmentation judgment threshold. The segmentation judgment threshold is calculated pixel by pixel in the external wall image to obtain the corresponding segmentation judgment threshold parameters generated at different pixel positions in the external wall image.

7. The deep learning-based UAV-based adaptive segmentation method for building exterior wall cracks according to claim 6, characterized in that, The method for obtaining the segmentation diagram of external wall cracks includes: Under the constraint of the segmentation judgment threshold parameter, pixel-level judgment processing is performed on the crack structure perception features of each pixel position in the external wall image. For any pixel position in the external wall image, if the crack structure perception feature value of the pixel is greater than or equal to the corresponding segmentation judgment threshold, the pixel is judged to belong to the crack region; otherwise, the pixel is judged to belong to the non-crack region. The judgment results of all pixels are combined to obtain the segmentation map of the external wall crack.

8. The deep learning-based UAV-based adaptive segmentation method for building exterior wall cracks according to claim 7, characterized in that, The method for generating the corresponding crack detection report includes: Based on the spatial reference relationship of each frame of exterior wall image, the pixels identified as cracks in the segmentation image are mapped to the reference plane of the building facade. Connectivity analysis is performed on the mapped crack pixels, and adjacent or continuous crack pixels are aggregated into crack regions. The spatial scale information of each crack region on the building facade is obtained, including crack length, width and starting position. Based on the spatial scale information and positional relationship of the crack regions, a crack detection report is generated.

Images