Channel concrete crack detection method based on unmanned aerial vehicle photogrammetry data

Abstract

The invention discloses a channel concrete crack detection method based on unmanned aerial vehicle photogrammetry data. The method comprises the steps of unmanned aerial vehicle photogrammetry data acquisition, channel concrete image optimization, channel concrete crack region extraction, channel concrete crack detection model acquisition and channel concrete crack detection. The invention relates to the technical field of image data processing, in particular to a channel concrete crack detection method based on unmanned aerial vehicle photogrammetry data, which innovatively proposes to combine unmanned aerial vehicle IMU data and image GPS positioning data to carry out image geometric correction and GPS positioning registration so as to effectively eliminate the image quality problem. A channel concrete crack area is accurately extracted through a multi-stage feature extraction and fusion combined high-resolution image generation method; the optimization algorithm for obtaining the hyper-parameters is improved by mixing the local search strategy and introducing the dynamic gain item, the hyper-parameters of the optimal parameter value adjustment model are obtained, and the accuracy of the output result of the crack detection model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and specifically refers to a method for detecting concrete cracks in channels based on unmanned aerial vehicle (UAV) photogrammetry data. Background Art

[0002] If the crack problem of the channel concrete structure fails to be detected and repaired in time, it may lead to structural damage, instability, and even seriously affect the normal use and safe operation of the channel. Traditional crack detection methods usually rely on manual inspections and visual checks, which are not only time-consuming and laborious, but also limited by the vision, work intensity, and environmental conditions of the manual inspectors, resulting in the detection accuracy and coverage often being difficult to meet the requirements. Therefore, a method for detecting concrete cracks in channels based on UAV photogrammetry data has emerged. It is a method that uses a UAV equipped with photogrammetry equipment to obtain image data of the surface of the channel concrete, and through image processing and image analysis techniques, detects, classifies, and locates the cracks on the surface of the channel concrete, effectively improving the detection efficiency, accuracy, and coverage.

[0003] However, there are technical problems of poor image data quality and inaccurate crack positions in traditional channel concrete crack detection methods, resulting in poor crack detection effects and affecting the crack detection efficiency; existing methods for extracting crack regions in channel concrete have limitations in dealing with complex channel concrete environments and micro-cracks, and it is difficult to effectively extract crack features; existing channel concrete crack detection models have inappropriate parameter settings, and the ability to obtain the global optimal solution of the model parameter optimization algorithm is weak, resulting in inaccurate crack detection results. Summary of the Invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides a channel concrete crack detection method based on UAV photogrammetry data. In view of the technical problems of poor image data quality and inaccurate crack position in the traditional channel concrete crack detection method, which leads to poor crack detection effect and affects the efficiency of crack detection, this solution innovatively proposes to combine UAV IMU data with image GPS positioning data to perform image geometric correction and GPS positioning alignment, which can effectively eliminate image quality problems, ensure that the image truly and accurately reflects the actual situation of the surface of the channel concrete, and significantly improve the spatial accuracy of the image, improve the high-precision alignment effect of the image and the actual geographical environment, and can also accurately locate the position of the crack, thereby improving the efficiency, accuracy and reliability of crack detection, and providing accurate data support for subsequent channel concrete crack related work; in view of the existing method for extracting crack areas in channel concrete in dealing with complex channel concrete environments and micro Small cracks have limitations and it is difficult to effectively extract crack features. This solution innovatively uses multi-stage feature extraction and fusion, combined with high-resolution image generation methods, to deeply explore the key features of the image, and by focusing on important areas, accurately extract channel concrete crack areas, thereby significantly improving the recognition ability of crack areas, effectively improving the accuracy of crack detection, reducing false alarms and missed alarms, and ensuring the high reliability of detection results; in view of the technical problems that the existing model for channel concrete crack detection has inappropriate parameter settings and weak ability to obtain the global optimal solution of the model parameter optimization algorithm, resulting in inaccurate crack detection results, this solution effectively avoids the optimization algorithm from falling into the local optimal solution by mixing local search strategies and introducing dynamic gain terms, accurately obtains the optimal parameter values, and adjusts the model's hyperparameters, significantly improving the accuracy of the output results of the channel concrete crack detection model, thereby improving the accuracy and generalization ability of channel concrete crack detection.

[0005] The technical solution adopted by the present invention is as follows: The present invention provides a channel concrete crack detection method based on drone photogrammetry data, the method comprising the following steps:

[0006] Step S1: Acquire UAV photogrammetry data;

[0007] Step S2: channel concrete image optimization;

[0008] Step S3: Extracting the crack area of ​​channel concrete;

[0009] Step S4: obtaining a channel concrete crack detection model;

[0010] Step S5: channel concrete crack detection.

[0011] Further, in step S1, the acquisition of the UAV photogrammetry data is specifically to obtain the UAV photogrammetry data by collecting from the UAV aerial photography equipment; the UAV photogrammetry data includes historical channel concrete crack detection data and real-time channel concrete crack detection data; both the historical channel concrete crack detection data and the real-time channel concrete crack detection data include channel concrete image data, UAV IMU data, and image GPS positioning data; the historical channel concrete crack detection data further includes channel concrete crack annotation data.

[0012] Further, in step S2, the optimization of the channel concrete image is used to optimize the channel concrete image data in the UAV photogrammetry data, specifically including image geometric correction, image GPS positioning registration, image stitching enhancement, and image quality optimization to obtain an optimized image for channel concrete crack detection; it includes the following steps:

[0013] Step S21: Image geometric correction is used to ensure the geometric accuracy of the image, correct the image distortion caused by the movement of the UAV during the shooting process, and ensure that the image can accurately reflect the actual morphology of the channel concrete surface. It specifically includes the following steps:

[0014] Step S211: Pose correction is used to eliminate the deformation generated by the UAV during the shooting process and ensure the geometric consistency of the image. Specifically, use the pose data in the UAV IMU data to generate a rotation matrix, and use the rotation matrix to correct the pose of the image;

[0015] Step S212: Distortion removal processing is used to eliminate the image distortion caused by the tilt and vibration of the UAV during flight and restore the true geometric shape of the image; specifically, use the acceleration data and angular velocity data in the UAV IMU data, combined with the perspective distortion in the image, and perform distortion removal processing on the image through the perspective transformation matrix;

[0016] Step S22: Image GPS positioning registration is used to provide accurate geographical coordinates for each image through the GPS positioning data, ensure the precise registration between the image and the actual geographical environment, and improve the spatial accuracy of the image data; specifically, use the image GPS positioning data through the RANSAC algorithm to correspond the pixel coordinates of the image with the GPS coordinates;

[0017] Step S23: Image stitching enhancement is used to improve the stitching accuracy of the images taken by the UAV and ensure that the images taken from different perspectives can be seamlessly docked; specifically, use the SURF algorithm to accurately align multiple images to ensure that there is no misalignment during image stitching; adopt the multi-resolution fusion image fusion technology to smooth the stitching seam area, eliminate the illumination difference, and ensure the natural transition after image stitching;

[0018] Step S24: Image quality optimization, which is used to improve the image resolution and enhance the visibility of cracks. Specifically, the image resolution is improved through an enhanced super-resolution generative adversarial network.

[0019] Furthermore, in step S3, the extraction of the channel concrete crack area is used to extract the image crack area from the optimized image of the channel concrete crack detection, and specifically includes the following steps:

[0020] Step S31: Preliminary feature analysis stage, which is used to accurately capture the local features of the image and enhance the detailed information of the image; specifically includes the following steps:

[0021] Step S311: Image division, specifically dividing the optimized image of the channel concrete crack detection into 4 smaller regions on average to obtain sub-images ;

[0022] Step S312: Preliminary feature processing, specifically first using a 3×3 convolutional kernel to perform a convolution operation on each sub-image to expand the number of feature channels, and then performing weighted processing on the convolved feature map through a channel attention mechanism to highlight important feature channels. Finally, dilated convolution and pyramid pooling are used to fuse multi-scale features with different dilation rates to obtain the final output of fine features ; The formula used is as follows:

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] In the formula, represents the weight parameter of the 3×3 convolutional kernel, and represent the weight matrices of the fully connected layers, which are used to generate the channel attention weights, represents the Sigmoid function, represents element-wise multiplication, represents 3×3 dilated convolution, which is used to capture multi-scale features, d represents the dilation rate, which is used to control the sampling interval of the convolutional kernel, represents the output feature map of the channel attention module, represents the result of multi-scale feature splicing, represents the pooling operation, which reduces the dimensionality of the feature map to a single channel, represents the convolution operation, which is used to adjust the number of channels, represents the upsampling operation, which is used to restore the size of the feature map, Indicates a splicing operation, Indicates an operation to prevent overfitting, and k represents an index variable;

[0028] Step S32: Deep feature analysis stage, which is used to mine the deep context information of the image and further improve the understanding of the crack area; it specifically includes the following steps:

[0029] Step S321: Image division, specifically dividing the optimized image of the channel concrete crack detection into 2 regions on average to obtain sub-images ;

[0030] Step S322: Deep feature processing, specifically using a 3×3 convolutional kernel to perform a convolution operation on the divided to expand the number of feature channels, performing weighted processing on the convolved feature map through a channel attention mechanism to highlight important feature channels, and finally using dilated convolution and pyramid pooling to fuse multi-scale features with different dilation rates to obtain the final output of the deep features ;

[0031] Step S323: Multi-scale feature fusion stage, specifically using an encoder-decoder structure to fuse feature information of different scales, and through downsampling and upsampling operations, and are fused; the formula used is as follows:

[0032] ;

[0033] In the formula, represents 6 consecutive channel attention modules, represents the downsampling operation, represents the low-dimensional deep features output by the encoder, which retains the deep semantic information of the image, represents the high-resolution features output by the decoder, which combines deep semantic information and shallow detail information, represents 8 consecutive channel attention modules, represents the skip connection features, which come from the outputs of different stages of the encoder, represents the feature fusion operation, represents the multi-scale features;

[0034] Step S33: Generate a high-resolution feature map, specifically receiving the multi-scale features through the original resolution sub-network , performing fusion and processing, and finally generating a high-resolution feature map;

[0035] ;

[0036] In the formula, Represents the features after initial processing, Represents the weight matrix of the initial convolution, Represents the bias term parameter of the initial convolution, Represents the operation composed of n residual blocks, Represents the upsampling factor, Represents the final high-resolution feature map, Represents the weight matrix after the final convolution, Represents the bias term parameter after the final convolution;

[0037] Step S34: Crack area extraction, specifically, inputting the high-resolution feature map into the Sigmoid activation function for processing to generate a probability map of each pixel belonging to the crack area , and performing binary processing on the probability map by setting a threshold. If the probability value of a certain pixel is greater than the set threshold, the pixel is considered to belong to the crack area and is assigned a value of 1; otherwise, it is assigned a value of 0 to obtain a binary image .

[0038] Furthermore, in step S4, the acquisition of the channel concrete crack detection model is used to obtain the model required for channel concrete crack detection; specifically, it includes the following steps:

[0039] Step S41: Extract crack classification features, specifically, first using a convolutional neural network to extract crack classification features from the binary image to learn the local features of the image, capture the edge, texture, and morphological information of the crack, reduce the computational complexity through max pooling operation, retain the key feature information, and finally flatten the extracted feature map into a one-dimensional vector;

[0040] Step S42: Channel concrete crack classification, specifically, inputting the flattened one-dimensional vector into the fully connected layer, and after processing by the non-linear activation function , using the activation function to map the features into the probability distributions of various categories. Finally, by selecting the crack category corresponding to the maximum probability, the classification result of the crack is output;

[0041] Step S43: Model training, specifically, constructing a channel concrete crack detection model through the extraction of crack classification features and the channel concrete crack classification, using the historical channel concrete crack detection data as the model training data. During the training process, the cross-entropy loss function is used to calculate the loss, the optimizer Adam is selected to update the parameters of the model, and by continuously iteratively updating the parameters of the model, the value of the loss function is gradually reduced to obtain the channel concrete crack detection model;

[0042] Step S44: Improve the classification effect of the model to obtain the optimal hyperparameter combination and enhance the model's classification effect. Specifically, globally optimize the hyperparameters of the channel concrete crack detection model by improving the optimization algorithm, obtain the optimal hyperparameter combination of the channel concrete crack detection model, and adjust the hyperparameters of the model according to the hyperparameter combination to obtain the improved channel concrete crack detection model, including the following steps:

[0043] Step S441: Initialize the positions of the population individuals, specifically, randomly initialize the positions of each search individual in the population;

[0044] Step S442: Calculate the individual fitness value, specifically, calculate the individual fitness value f in the population i ; Take the performance of the concrete crack detection model as the fitness value of the individual;

[0045] Step S443: Update the individual positions, specifically, perform local search by combining the local search strategy and introducing a dynamic gain term; the formula used is as follows:

[0046] ;

[0047] In the formula, represents the position of the i-th individual in the d-th dimension of the (t + 1)-th generation population, represents the position of the i-th individual in the d-th dimension of the t-th generation population, represents that the search area remains unchanged, represents that the search area changes, Q represents a random number following a normal distribution, represents the smoothing term, which is a constant, t represents the number of iterations, and represent random numbers within the range of [0, 1], and represent the lower and upper limits of the search range respectively, represents the maximum number of iterations, represents the adjustment factor for controlling the search enhancement, represents the warning value, represents the safety threshold, represents the dynamic gain term;

[0048] Step S444: Update the current optimal solution, specifically, recalculate the fitness values of the individuals after position upgrade and the current individuals, and compare their fitness with the current optimal solution. If the fitness of the current solution is better than the global optimal solution, update the optimal solution;

[0049] Step S445: Obtain the position of the optimal individual, specifically, by when the individual fitness value f iWhen the fitness threshold is higher than the set fitness threshold and the maximum number of iterations is reached, the search is terminated and the individual global optimal position is obtained. The individual global optimal position specifically refers to the optimal hyperparameter combination of the concrete crack detection model;

[0050] Step S446: obtaining a high-performance model, specifically adjusting the hyperparameters of the concrete crack detection model according to the optimal hyperparameter combination of the concrete crack detection model to obtain an improved channel concrete crack detection model.

[0051] Furthermore, in step S5, the channel concrete crack detection is specifically to use the real-time channel concrete crack detection data as input data of the improved channel concrete crack detection model to obtain channel concrete crack detection results, and the channel concrete crack detection results include crack classification and crack location, providing accurate data support for subsequent crack repair and maintenance.

[0052] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0053] (1) In view of the technical problems of poor image data quality and inaccurate crack location in traditional channel concrete crack detection methods, which lead to poor crack detection results and affect the efficiency of crack detection, this solution innovatively proposes to combine UAV IMU data with image GPS positioning data to perform image geometric correction and GPS positioning registration. This can effectively eliminate image quality problems, ensure that the image truly and accurately reflects the actual surface conditions of the channel concrete, and significantly improve the spatial accuracy of the image. It also improves the high-precision registration effect of the image and the actual geographical environment, and can accurately locate the position of the cracks, thereby improving the efficiency, accuracy and reliability of crack detection, and providing accurate data support for subsequent channel concrete crack-related work.

[0054] (2) The existing methods for extracting crack areas in channel concrete have limitations in dealing with complex channel concrete environments and tiny cracks, making it difficult to effectively extract crack features. This scheme innovatively extracts and fuses features in multiple stages, and combines it with a high-resolution image generation method to deeply explore the key features of the image. By focusing on important areas, the crack areas in channel concrete can be accurately extracted, thereby significantly improving the recognition ability of crack areas, effectively improving the accuracy of crack detection, reducing false alarms and missed alarms, and ensuring the high reliability of detection results.

[0055] (3) Aiming at the technical problems that the existing channel concrete crack detection models have inappropriate parameter settings and weak ability to obtain the global optimal solution of the model parameter optimization algorithm, resulting in inaccurate crack detection results, this solution effectively avoids the optimization algorithm from falling into the local optimal solution by mixing local search strategies and introducing dynamic gain terms, accurately obtains the optimal parameter values, and adjusts the hyperparameters of the model, significantly improving the accuracy of the output results of the channel concrete crack detection model, thereby improving the accuracy and generalization ability of the channel concrete crack detection. Description of the Drawings

[0056] Figure 1 It is a schematic flow chart of the channel concrete crack detection method based on unmanned aerial vehicle photogrammetry data provided by the present invention;

[0057] Figure 2 It is a schematic flow chart of step S2;

[0058] Figure 3 It is a schematic flow chart of step S3;

[0059] Figure 4 It is a schematic flow chart of step S4;

[0060] Figure 5 It is a schematic flow chart of step S44;

[0061] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. Specific Embodiments

[0062] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0063] In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the system or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation to the present invention.

[0064] Example 1, refer to Figure 1, the technical solution adopted by the present invention is as follows: The present invention provides a method for detecting concrete cracks in channels based on UAV photogrammetry data, and the method includes the following steps:

[0065] Step S1: Obtain UAV photogrammetry data, specifically, from a UAV aerial photography device, and obtain UAV photogrammetry data through acquisition.

[0066] Step S2: Optimize the channel concrete image, which is used to optimize the channel concrete image data in the UAV photogrammetry data. Specifically, it includes image geometric correction, image GPS positioning registration, image stitching enhancement, and image quality optimization to obtain an optimized image for detecting channel concrete cracks.

[0067] Step S3: Extract the channel concrete crack area, which is used to extract the image crack area in the optimized image for detecting channel concrete cracks. Specifically, it includes a preliminary feature analysis stage, a deep feature analysis stage, a multi-scale feature fusion stage, generating a high-resolution feature map, and crack area extraction to obtain a binary image.

[0068] Step S4: Obtain a model for detecting channel concrete cracks to obtain the model required for detecting channel concrete cracks. Specifically, extract crack classification features through a convolutional neural network, use a fully connected layer to generate model outputs, and train and optimize the model to obtain an improved model for detecting channel concrete cracks.

[0069] Step S5: Detect channel concrete cracks. Specifically, input the data into the improved model for detecting channel concrete cracks to obtain the detection results of channel concrete cracks, providing accurate data support for subsequent work on channel concrete cracks.

[0070] Example 2, refer to Figure 1, This embodiment is based on the above embodiment. In step S1, the acquisition of UAV photogrammetry data is specifically to obtain UAV photogrammetry data from a UAV aerial photography device through acquisition; the UAV photogrammetry data includes historical channel concrete crack detection data and real-time channel concrete crack detection data; both the historical channel concrete crack detection data and the real-time channel concrete crack detection data include channel concrete image data, UAV IMU data, and image GPS positioning data; the historical channel concrete crack detection data also includes channel concrete crack annotation data; the channel concrete crack annotation data is specifically no crack, slight crack, moderate crack, and severe crack; the channel concrete image data is used to provide visual information on the surface cracks of the channel concrete; the UAV IMU data is used to ensure the accurate attitude and spatial correction of image shooting, avoiding distortion and positioning errors, including attitude data, acceleration data, and angular velocity data. The attitude data includes pitch angle, roll angle, and yaw angle. The image GPS positioning data is used to provide precise geographical coordinates for each image, enabling the image data to be accurately paired with the actual geographical information, including longitude and latitude information data and altitude data.

[0071] Embodiment 3, refer to Figure 1 and Figure 2 , This embodiment is based on the above embodiment. In step S2, the optimization of the channel concrete image is used to optimize the channel concrete image data in the UAV photogrammetry data, specifically including image geometric correction, image GPS positioning registration, image stitching enhancement, and image quality optimization to obtain an optimized image for channel concrete crack detection; it includes the following steps:

[0072] Step S21: Image geometric correction is used to ensure the geometric accuracy of the image, correct the image distortion caused by the movement of the UAV during the shooting process, and ensure that the image can accurately reflect the actual shape of the channel concrete surface. It specifically includes the following steps:

[0073] Step S211: Attitude correction is used to eliminate the deformation generated by the UAV during the shooting process and ensure the geometric consistency of the image. Specifically, the attitude data in the UAV IMU data is used to generate a rotation matrix, and the rotation matrix is used to correct the attitude of the image; the formula used is as follows:

[0074] ;

[0075] In the formula, represents the overall rotation matrix, represents the rotation matrix around the X-axis, the rotation matrix around the Y-axis, the rotation matrix around the Z-axis, represents the pitch angle of the attitude data, Represents the roll angle of the attitude data, represents the yaw angle of the attitude data;

[0076] Step S212: Distortion removal processing, which is used to eliminate the image distortion caused by the tilt and vibration of the drone during flight and restore the true geometric shape of the image; specifically, by using the acceleration data and angular velocity data in the drone IMU data, combined with the perspective distortion in the image, the image is processed for distortion removal through a perspective transformation matrix;

[0077] Step S22: Image GPS positioning and registration, which is used to provide accurate geographical coordinates for each image through GPS positioning data, ensure the precise registration between the image and the actual geographical environment, and improve the spatial accuracy of the image data; specifically, by using the image GPS positioning data through the RANSAC algorithm, the pixel coordinates of the image are corresponding to the GPS coordinates;

[0078] Step S23: Image stitching and enhancement, which is used to improve the stitching accuracy of the images taken by the drone and ensure that the images taken from different perspectives can be seamlessly docked; specifically, the SURF algorithm is used to accurately align multiple images to ensure that there is no misalignment during image stitching; the multi-resolution fusion image fusion technology is adopted to smooth the stitching seam area, eliminate the illumination difference, and ensure the natural transition after image stitching;

[0079] Step S24: Image quality optimization, which is used to improve the image resolution and the visibility of cracks. Specifically, the image resolution is improved through an enhanced super-resolution generative adversarial network, and the image is increased from 512×512 pixels to 960×960 pixels.

[0080] By performing the above operations, aiming at the technical problems of poor image data quality and inaccurate crack positions in the traditional channel concrete crack detection method, which lead to poor crack detection effect and affect the efficiency of crack detection, this solution innovatively proposes to combine the drone IMU data with the image GPS positioning data for image geometric correction and GPS positioning registration, which can effectively eliminate image quality problems, ensure that the image truly and accurately reflects the actual situation of the surface of the channel concrete, and significantly improve the spatial accuracy of the image, improve the high-precision registration effect between the image and the actual geographical environment, and can also accurately locate the position of the crack, thereby improving the efficiency, accuracy and reliability of crack detection, and providing accurate data support for subsequent work related to channel concrete cracks.

[0081] Example 4, refer to Figure 1 and Figure 3 , based on the above example, in step S3, the extraction of the channel concrete crack area is used to extract the image crack area in the optimized image of the channel concrete crack detection, and specifically includes the following steps:

[0082] Step S31: The preliminary feature analysis stage is used to accurately capture the local features of the image and enhance the detailed information of the image; specifically, it includes the following steps:

[0083] Step S311: Image division, specifically, the optimized image for detecting concrete cracks in the channel is evenly divided into 4 smaller regions to obtain sub-images ;

[0084] Step S312: Preliminary feature processing, specifically, first use a 3×3 convolutional kernel to perform convolution operations on each sub-image to expand the number of feature channels, perform weighted processing on the convolutional feature map through the channel attention mechanism to highlight important feature channels, and finally use dilated convolution and pyramid pooling to fuse multi-scale features with different dilation rates to obtain the final output of fine features ; The formula used is as follows:

[0085] ;

[0086] ;

[0087] ;

[0088] ;

[0089] ;

[0090] In the formula, represents the weight parameter of the 3×3 convolutional kernel, represents the number of channels of the input feature map, represents the number of channels of the output feature map, and represent the weight matrices of the fully connected layers, used to generate channel attention weights, represents the Sigmoid function, represents per-channel multiplication, represents a 3×3 dilated convolution, used to capture multi-scale features, d represents the dilation rate, used to control the sampling interval of the convolutional kernel, represents the output feature map of the channel attention module, represents the result of multi-scale feature concatenation, represents the pooling operation, which reduces the dimensionality of the feature map to a single channel, represents the convolution operation, used to adjust the number of channels, represents the upsampling operation, used to restore the size of the feature map, represents the concatenation operation, represents the operation to prevent overfitting, k represents an index variable;

[0091] Step S32: Deep feature analysis stage, which is used to mine the deep context information of the image and further improve the understanding of the crack area; specifically including the following steps:

[0092] Step S321: Image division, specifically dividing the optimized image of the channel concrete crack detection into 2 regions on average to obtain sub-images ;

[0093] Step S322: Deep feature processing, specifically using a 3×3 convolutional kernel to perform convolution operations on the divided , expanding the number of feature channels, weighting the convolved feature map through a channel attention mechanism to highlight important feature channels, and finally using dilated convolution and pyramid pooling to fuse multi-scale features with different dilation rates to obtain the final output of the deep features ;

[0094] Step S323: Multi-scale feature fusion stage, specifically using an encoder-decoder structure to fuse feature information of different scales, and through downsampling and upsampling operations, and are fused; the formula used is as follows:

[0095] ;

[0096] In the formula, represents 6 consecutive channel attention modules, represents the downsampling operation, represents the low-dimensional deep features output by the encoder, which retains the deep semantic information of the image, represents the high-resolution features output by the decoder, which combines deep semantic information and shallow detail information, represents 8 consecutive channel attention modules, represents the skip connection features, which come from the outputs of different stages of the encoder, represents the feature fusion operation, represents the multi-scale features;

[0097] Step S33: Generate a high-resolution feature map, specifically receiving the multi-scale features through the original resolution sub-network , performing fusion and processing, and finally generating a high-resolution feature map;

[0098] ;

[0099] In the formula, represents the initially processed features, represents the weight matrix of the initial convolution, represents the bias parameter of the initial convolution, represents an operation consisting of n residual blocks, represents the upsampling factor, represents the final high-resolution feature map, represents the weight matrix after the final convolution, Represents the bias parameter after the final convolution;

[0100] Step S34: extracting crack regions, specifically converting the high-resolution feature map Input to the Sigmoid activation function for processing to generate a probability map of each pixel belonging to the crack area , and the probability map is binarized by setting a threshold. If the probability value of a pixel is greater than the set threshold, the pixel is considered to belong to the crack area and is assigned a value of 1; otherwise, it is assigned a value of 0, and a binary image is obtained. .

[0101] By performing the above operations, the existing methods for extracting crack areas in channel concrete have limitations in dealing with complex channel concrete environments and tiny cracks, making it difficult to effectively extract crack features. This solution innovatively extracts and fuses multi-stage features, and combines high-resolution image generation methods to deeply explore the key features of the image. By focusing on important areas, the channel concrete crack areas are accurately extracted, thereby significantly improving the recognition ability of crack areas, effectively improving the accuracy of crack detection, reducing false alarms and missed alarms, and ensuring the high reliability of detection results.

[0102] Example 5, see Figure 1 , Figure 4 and Figure 5 This embodiment is based on the above embodiment. In step S4, the channel concrete crack detection model is obtained to obtain the model required for channel concrete crack detection. Specifically, the following steps are included:

[0103] Step S41: Extracting crack classification features, specifically, first using a convolutional neural network to extract crack classification features from the binary image The crack classification features are extracted from the image to learn the local features of the image, capture the edge, texture and morphological information of the cracks, reduce the computational complexity through the maximum pooling operation, retain the key feature information, and finally flatten the extracted feature map into a one-dimensional vector;

[0104] Step S42: Channel concrete crack classification, specifically, flattening the one-dimensional input to the fully connected layer, and passing through a nonlinear activation function After processing, use The activation function maps the features into probability distributions of various categories, and outputs the classification results of cracks by selecting the crack category corresponding to the maximum probability;

[0105] Step S43: Conduct model training. Specifically, construct a channel concrete crack detection model by using the extracted crack classification features and the classification of channel concrete cracks. Use the historical channel concrete crack detection data as model training data. During the training process, calculate the loss using the cross-entropy loss function, select the Adam optimizer to update the model parameters. By continuously iterating and updating the model parameters, gradually reduce the value of the loss function to obtain the channel concrete crack detection model;

[0106] Step S44: Improve the model classification effect to obtain the optimal hyperparameter combination and improve the model classification effect. Specifically, globally optimize the hyperparameters of the channel concrete crack detection model by improving the optimization algorithm, obtain the optimal hyperparameter combination of the channel concrete crack detection model, and adjust the hyperparameters of the model according to the hyperparameter combination to obtain the improved channel concrete crack detection model, including the following steps:

[0107] Step S441: Initialize the individual positions of the population. Specifically, randomly initialize the positions of each search individual in the population;

[0108] Step S442: Calculate the individual fitness value. Specifically, calculate the individual fitness value f i ; Use the performance of the concrete crack detection model as the fitness value of the individual;

[0109] Step S443: Update the individual positions. Specifically, conduct local search by mixing the local search strategy and introducing a dynamic gain term. The formula used is as follows:

[0110] ;

[0111] In the formula, represents the position of the i-th individual in the d-th dimension of the (t + 1)-th generation population, represents the position of the i-th individual in the d-th dimension of the t-th generation population, represents that the search area does not change, represents that the search area changes, Q represents a random number following a normal distribution, represents the smoothing term, which is a constant, t represents the number of iterations, and represent random numbers within the range of [0, 1], and represent the lower and upper limits of the search range respectively, represents the maximum number of iterations, represents the adjustment factor for controlling the search enhancement, represents the warning value, represents the safety threshold, represents the dynamic gain term;

[0112] Step S444: Update the current optimal solution. Specifically, recalculate the fitness values of the individual after position upgrade and the current individual, and compare their fitness with the current optimal solution. If the fitness of the current solution is better than the global optimal solution, update the optimal solution.

[0113] Step S445: Obtain the optimal individual position. Specifically, when the fitness value f of the individual i is higher than the set fitness threshold and when the maximum number of iterations is reached, terminate the search and obtain the global optimal position of the individual. The global optimal position of the individual specifically refers to the optimal hyperparameter combination of the concrete crack detection model.

[0114] Step S446: Obtain a high-performance model. Specifically, adjust the hyperparameters of the concrete crack detection model according to the optimal hyperparameter combination of the concrete crack detection model to obtain an improved channel concrete crack detection model.

[0115] By performing the above operations, aiming at the technical problems that the existing channel concrete crack detection model has inappropriate parameter settings and the global optimal solution acquisition ability of the model parameter optimization algorithm is weak, resulting in inaccurate crack detection results, this solution effectively avoids the optimization algorithm falling into the local optimal solution, accurately obtains the optimal parameter values, and adjusts the hyperparameters of the model by mixing the local search strategy and introducing the dynamic gain term, significantly improving the accuracy of the output results of the channel concrete crack detection model, thereby improving the accuracy and generalization ability of the channel concrete crack detection.

[0116] Example Six, refer to Figure 1 , based on the above example, in step S5, the channel concrete crack detection specifically uses the real-time channel concrete crack detection data as the input data of the improved channel concrete crack detection model to obtain the channel concrete crack detection results. The channel concrete crack detection results include crack classification and crack location, providing accurate data support for subsequent crack repair and maintenance.

[0117] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0118] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art will appreciate that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the invention.

[0119] The present invention and its embodiments have been described above. Such description is not restrictive. What is shown in the drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto. Generally speaking, if those of ordinary skill in the art are inspired by it and, without departing from the purpose of the present invention, creatively design a structural mode and embodiments similar to the technical solution, they shall fall within the protection scope of the present invention.

Claims

1. A method for detecting concrete cracks in channels based on UAV photogrammetry data, characterized in that: The method includes the following steps: Step S1: Obtain unmanned aerial vehicle (UAV) photogrammetry data, specifically, collect UAV photogrammetry data from a UAV aerial photography device; Step S2: Optimize the channel concrete image, which is used to optimize the channel concrete image data in the UAV photogrammetry data. Specifically, perform image geometric correction, image GPS positioning registration, image stitching enhancement, and image quality optimization to obtain an optimized image for channel concrete crack detection; Step S3: Extract the channel concrete crack area, which is used to extract the image crack area in the optimized image for channel concrete crack detection. Specifically, complete the extraction of the crack area through multi-stage feature processing and multi-scale feature fusion, and combine it with a high-resolution image generation method to finally obtain a binary image. It includes the following steps: Step S31: Preliminary feature analysis stage, Step S32: Deep feature analysis stage, Step S33: Multi-scale feature fusion stage, Step S34: Generate a high-resolution feature map, Step S35: Set crack area extraction; Step S4: Obtain a channel concrete crack detection model. Specifically, extract crack classification features through a convolutional neural network, use a fully connected layer to generate model outputs, train the model, optimize and improve the optimization algorithm for obtaining model hyperparameters through a hybrid local search strategy and introduce a dynamic gain term, obtain the optimal hyperparameter combination, and adjust the hyperparameters of the model to obtain an improved channel concrete crack detection model; Step S5: Detect channel concrete cracks. Specifically, input the data into the improved channel concrete crack detection model to obtain the channel concrete crack detection result.

2. The method for detecting concrete cracks in channels based on UAV photogrammetry data according to claim 1, wherein: In Step S2, the optimization of the channel concrete image specifically includes the following steps: Step S21: Image geometric correction, which is used to ensure the geometric accuracy of the image, correct the image distortion caused by the movement of the UAV during the shooting process, and ensure that the image can accurately reflect the actual shape of the channel concrete surface. It specifically includes the following steps: Step S211: Attitude correction, which is used to eliminate the deformation generated by the UAV during the shooting process and ensure the geometric consistency of the image. Specifically, use the attitude data in the UAV IMU data to generate a rotation matrix and use the rotation matrix to correct the attitude of the image; Step S212: Distortion removal processing, which is used to eliminate the image distortion caused by the tilt and vibration of the UAV during flight and restore the true geometric shape of the image. Specifically, use the acceleration data and angular velocity data in the UAV IMU data, combine with the perspective distortion in the image, and perform distortion removal processing on the image through a perspective transformation matrix; Step S22: Image GPS positioning registration, which is used to provide accurate geographical coordinates for each image through GPS positioning data, ensure the precise registration between the image and the actual geographical environment, and improve the spatial accuracy of the image data. Specifically, use the image GPS positioning data through the RANSAC algorithm to correspond the pixel coordinates of the image with the GPS coordinates; Step S23: Image stitching enhancement is used to improve the stitching accuracy of the images taken by the drone to ensure that the images taken from different perspectives can be seamlessly connected; specifically, the SURF algorithm is used to accurately align multiple images to ensure that there is no misalignment when the images are stitched; multi-resolution fusion image fusion technology is used to smooth the stitching seam area, eliminate lighting differences, and ensure a natural transition after image stitching; Step S24: Image quality optimization is used to improve image resolution and enhance the visibility of cracks, specifically by improving image resolution through an enhanced super-resolution generative adversarial network.

3. The method for detecting concrete cracks in channels based on UAV photogrammetry data according to claim 1, wherein: In step S3, the channel concrete crack region extraction is used to extract the image crack region in the channel concrete crack detection optimization image, which specifically includes the following steps: Step S31: The preliminary feature analysis stage is used to accurately capture the local features of the image and enhance the detailed information of the image; specifically, it includes the following steps: Step S311: Image division, specifically, the optimized image for detecting concrete cracks in the channel is evenly divided into 4 smaller regions to obtain sub-images ; Step S312: Preliminary feature processing, specifically, first use a 3×3 convolutional kernel to perform a convolution operation on each sub-image to expand the number of feature channels, perform weighted processing on the convolved feature map through a channel attention mechanism to highlight important feature channels, and finally use dilated convolution and pyramid pooling to fuse multi-scale features with different dilation rates to obtain the final output of fine features ; The formula used is as follows: ； ； ； ； Wherein, represents the weight parameters of a 3×3 convolutional kernel, and represent the weight matrix of the fully connected layer, which is used to generate the channel attention weights, represents the Sigmoid function, represents per-channel multiplication, represents a 3×3 dilated convolution, which is used to capture multi-scale features, and d represents the dilation rate, which is used to control the sampling interval of the convolutional kernel, represents the output feature map of the channel attention module, represents the result of multi-scale feature concatenation, represents a pooling operation, which reduces the dimensionality of the feature map to a single channel, represents a convolution operation, which is used to adjust the number of channels, represents an upsampling operation, which is used to restore the size of the feature map, represents a concatenation operation, represents an operation to prevent overfitting, and k represents an index variable; Step S32: deep feature analysis phase, used to mine deep context information of the image to further improve the understanding of the crack area; Step S33: Generate a high-resolution feature map, specifically by receiving multi-scale features through the original resolution sub-network , perform fusion and processing, and finally generate a high-resolution feature map; ； In the formula, represents the feature after initial processing, represents the weight matrix of the initial convolution, represents the bias term parameter of the initial convolution, represents the operation composed of n residual blocks, represents the upsampling factor, represents the final high-resolution feature map, represents the weight matrix after the final convolution, represents the bias term parameter after the final convolution; Step S34: Crack area extraction, specifically, input the high-resolution feature map into the Sigmoid activation function for processing to generate a probability map of each pixel belonging to the crack area , and perform binarization processing on the probability map by setting a threshold to obtain a binarized image .

4. The method for detecting concrete cracks in channels based on UAV photogrammetry data according to claim 1, characterized in that: In step S32: the deep feature analysis stage, the following steps are specifically included: Step S321: Image division, specifically, the optimized image for detecting concrete cracks in the channel is evenly divided into 2 regions to obtain sub-images ; Step S322: Deep feature processing, specifically using a 3×3 convolutional kernel to perform convolution operations on the divided to expand the number of feature channels, performing weighted processing on the convolved feature map through a channel attention mechanism to highlight important feature channels, and finally using dilated convolution and pyramid pooling to fuse multi-scale features with different dilation rates to obtain the final output of the deep features ; Step S323: Multi-scale feature fusion stage. Specifically, use the encoder-decoder structure to fuse feature information of different scales. Through downsampling and upsampling operations, fuse and ; The formula used is as follows: ； Wherein, represents 6 consecutive channel attention modules, represents a downsampling operation, represents the low-dimensional deep features output by the encoder, retaining the deep semantic information of the image, represents the high-resolution features output by the decoder, combining deep semantic information and shallow detail information, represents 8 consecutive channel attention modules, represents the skip connection features, coming from the outputs of different stages of the encoder, represents a feature fusion operation, represents multi-scale features.

5. The method for detecting concrete cracks in channels based on UAV photogrammetry data according to claim 1, wherein: In step S4, the acquisition of the channel concrete crack detection model is used to obtain the model required for channel concrete crack detection; specifically, the following steps are included: Step S41: Extract crack classification features. Specifically, first use a convolutional neural network to extract crack classification features from the binary image and reduce the computational complexity through max pooling operation. Finally, flatten the extracted feature map into a one-dimensional vector; Step S42: Classify the cracks in the channel concrete. Specifically, flatten it into one dimension and input it into the fully connected layer. After passing through the non-linear activation function processing, use the activation function to map the features into the probability distributions of various categories. Finally, by selecting the crack category corresponding to the maximum probability, output the classification result of the cracks; Step S43: performing model training, specifically, constructing a channel concrete crack detection model by extracting crack classification features and classifying channel concrete cracks, using the historical channel concrete crack detection data as model training data, and during the training process, using a cross entropy loss function to calculate the loss, selecting an optimizer Adam to update the parameters of the model, and updating the parameters of the model by continuous iteration so that the value of the loss function is gradually reduced, thereby obtaining a channel concrete crack detection model; Step S44: Improve the model classification effect to obtain the optimal hyperparameter combination, specifically by globally optimizing the hyperparameters of the channel concrete crack detection model through improving the optimization algorithm, obtaining the optimal hyperparameter combination of the channel concrete crack detection model, adjusting the hyperparameters of the model according to the hyperparameter combination, and obtaining the improved channel concrete crack detection model.

6. The method for detecting concrete cracks in channels based on UAV photogrammetry data according to claim 1, wherein: In step S44, the improvement of the model classification effect specifically includes the following steps: Step S441: Initialize the position of the individual in the population, specifically, randomly initialize the position of each search individual in the population; Step S442: Calculate the individual fitness value, specifically, calculate the individual fitness value f in the population i ; Take the performance of the concrete crack detection model as the fitness value of the individual; Step S443: Individual position update, specifically, local search is performed by mixing local search strategies and introducing dynamic gain terms; the formula used is as follows: ； In the formula, represents the position of the $i$-th individual in the $d$-th dimension of the $(t + 1)$-th generation population, represents the position of the $i$-th individual in the $d$-th dimension of the $t$-th generation population, represents that the search area remains unchanged, represents that the search area changes, and $Q$ represents a random number following a normal distribution, represents the smoothing term, which is a constant, and $t$ represents the number of iterations, and represent random numbers within the range of $[0, 1]$, and represent the lower and upper limits of the search range respectively, represents the maximum number of iterations, represents the adjustment factor for controlling the search intensification, represents the warning value, represents the safety threshold, represents the dynamic gain term; Step S444: updating the current optimal solution, specifically recalculating the fitness values ​​of the individuals after the position upgrade and the current individuals, and comparing them with the fitness of the current optimal solution. If the fitness of the current solution is better than the global optimal solution, then updating the optimal solution; Step S445: Obtain the optimal individual position, specifically by terminating the search and obtaining the global optimal position of the individual when the fitness value f of the individual i is higher than the set fitness threshold and when the maximum number of iterations is reached. The global optimal position of the individual specifically refers to the optimal hyperparameter combination of the concrete crack detection model; Step S446: obtaining a high-performance model, specifically adjusting the hyperparameters of the concrete crack detection model according to the optimal hyperparameter combination of the concrete crack detection model to obtain an improved channel concrete crack detection model.

7. The method for detecting concrete cracks in channels based on UAV photogrammetry data according to claim 1, characterized in that: In step S5, the detection of the concrete cracks in the channel specifically involves using the real-time detection data of the concrete cracks in the channel as the input data of the improved detection model for the concrete cracks in the channel to obtain the detection results of the concrete cracks in the channel. The detection results of the concrete cracks in the channel include crack classification and crack location, providing accurate data support for subsequent crack repair and maintenance.

8. The method for detecting concrete cracks in channels based on UAV photogrammetry data according to claim 1, wherein: In step S1, the acquisition of the UAV photogrammetry data includes historical detection data of the concrete cracks in the channel and real-time detection data of the concrete cracks in the channel. Both the historical detection data of the concrete cracks in the channel and the real-time detection data of the concrete cracks in the channel include channel concrete image data, UAV IMU data, and image GPS positioning data. The historical detection data of the concrete cracks in the channel also includes channel concrete crack annotation data.

Images