Intelligent identification method of dike crack danger based on edge enhancement

Abstract

The present invention discloses an intelligent identification method for dike crack danger based on edge enhancement, including improving and training the edge enhanced YOLOv8 model, and the improvement and training of the YOLOv8 model has the following contents: data collection, sample making, sample generation and data set division, improving the edge enhanced YOLOv8 model, training the edge enhanced YOLOv8 model: dike crack danger detection using the edge enhanced YOLOv8 model, and determination of dike crack danger. The present invention will use the edge feature extraction module to extract the high-frequency edge information of the image, which is beneficial to the characteristics of the dike crack danger; design two backbone network feature extraction modules to improve the accuracy of dike crack danger detection; design a new feature enhancement fusion module for fusing various features, thereby effectively improving the accuracy of dike crack danger detection.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an intelligent identification method for dangerous conditions of dike cracks based on edge enhancement. Background Art

[0002] Levees are an important part of water conservancy projects. They are not only key facilities for preventing floods and protecting the safety of coastal residents, but also an important guarantee for maintaining ecological balance and promoting sustainable development. However, in the long-term operation of levees, due to the combined effects of natural factors (such as water scouring, soil erosion) and human factors (such as design defects, construction quality problems, etc.), cracks and other safety hazards may occur. These cracks will not only affect the structural integrity of the levee, but may also become channels for water infiltration, further weakening the flood control capacity of the levee. Therefore, how to quickly detect crack hazards, improve levee patrol efficiency, and reduce personnel input has an important role and practical significance in flood prevention and disaster reduction.

[0003] Since deep learning models can automatically learn image features and achieve superior performance in many tasks, they have also been applied to the field of dike crack hazard detection and have achieved good results. However, these dike crack hazard detection methods based on deep learning do not fully utilize the edge feature information related to cracks in dike crack hazards to improve the accuracy of dike crack hazard detection, and there is still great potential for improvement in accuracy and detection speed.

[0004] As an efficient one-stage target detection algorithm, the YOLO series has been widely used in the field of target detection. Among them, the YOLOv8 version of the YOLO series has high detection accuracy and lightweight network structure while ensuring real-time performance, which is suitable for improvement research on embankment crack detection tasks.

[0005] Therefore, the present invention selects the YOLOv8 model as the benchmark model. A new backbone network (EFB) is introduced to extract edge feature information from the image to enhance it, and a novel feature enhancement fusion module (FEFM) is used to fuse the original image features with the edge feature information features, thereby improving the accuracy of embankment crack detection. The present invention can be used for embankment crack detection based on drone visible light images, and has the advantages of fast detection speed and low cost. Summary of the invention

[0006] The purpose of the present invention is to provide an intelligent identification method for levee crack hazards based on edge enhancement in response to the shortcomings of the prior art. The edge feature extraction module is used to extract high-frequency edge information of the image, which is beneficial to the characteristics of levee crack hazards; two backbone network feature extraction modules are designed to improve the accuracy of levee crack hazard detection; and a new feature enhancement fusion module is designed to fuse various features.

[0007] To achieve the above-mentioned purpose, the present invention adopts the following technical scheme: an intelligent identification method for dike crack danger based on edge enhancement, including two processes of improving and training the YOLOv8 model and dike crack danger detection;

[0008] The specific steps to improve and train the YOLOv8 model are as follows:

[0009] Step S11, data collection: using a drone carrying a visible light sensor to inspect the levee, and obtaining visible light image data of the levee, including visible light images with dangerous conditions of levee cracks and visible light images without dangerous conditions of levee cracks;

[0010] Step S12, making samples: annotating and arranging the visible light image data collected in step S11 to make samples of dike crack danger;

[0011] Step S13, improving and training the YOLOv8 model: the improved YOLOv8 model consists of an original backbone network, an edge backbone network and a detection module, wherein the original backbone network and the edge backbone network are composed of a CBS convolution module, a C2f module, a spatial pyramid pooling module, an edge feature extraction module and a feature enhancement fusion module;

[0012] The specific steps for detecting embankment crack danger are as follows:

[0013] Step S21, acquiring an image: acquiring visible light image data of the levee to be detected as an input image for the levee crack hazard detection process;

[0014] Step S22, dividing the window: dividing the visible light image data of the levee into windows with 20% overlap, using a sliding window of 640*640 pixels with a sliding step of 480 pixels, and traversing the entire input image; moving the window with a height and width less than 640 pixels upward or leftward by the coordinate of the upper left corner so that the window size with a height and width less than 640 pixels is maintained at 640*640 pixels;

[0015] Record the corresponding number and upper left corner coordinates of each window in the input image, recorded as (R, C, X, Y), where R is the number of rows, C is the number of columns, and X, Y are the corresponding upper left corner coordinates;

[0016] Step S23, inputting the visible light image data of each window into the improved YOLOv8 model that has been trained in step S13; setting a confidence threshold, obtaining the center point position, size, category and confidence of the detection box that exceeds the confidence threshold; and calculating the actual position of the detection box in the input image according to formula (1);

[0017] (1);

[0018] in,( , ) is the position of the center point of the k-th detection box in the input image; ( , ) is the coordinate of the upper left corner of the current window where the detection box is located, ( , ) is the position of the center point of the kth detection box in the current window image;

[0019] Step S24, non-maximum suppression of detection results: after all windows are detected, multiple detection frames are obtained on the input image, and then weighted non-maximum suppression is used to remove overlapping detection frames;

[0020] Step S25, outputting the final detection result: outputting the result after weighted non-maximum suppression to obtain the final detection result.

[0021] Furthermore, in step S12, the visible light image data collected in step S11 is annotated and sorted to produce dike crack hazard samples; wherein the dike crack hazard samples are composed of positive samples and negative samples; specifically:

[0022] Step S121, annotating the collected visible light image data, selecting visible light image data with obvious features, and removing invalid visible light image data that is difficult to identify or has been checked;

[0023] Step S122, segmenting the selected visible light image data using a pixel size of 640*640, annotating the segmented visible light image data with the dangerous features of the dike cracks using the software Labelme, annotating with a rectangular frame, using "crack" as the dangerous mark of the dike cracks, and selecting the visible light image data with the marked rectangular frame as a positive sample, and selecting the visible light image data without the dangerous features of the dike cracks as a negative sample;

[0024] Step S123, randomly cropping, flipping, blurring, changing brightness, contrast, and randomly arranging to amplify the dike crack hazard samples, and all dike crack hazard samples are uniformly scaled to a size of 640*640 pixels;

[0025] Step S124, all the embankment crack hazard samples after uniform scaling are randomly divided into a training set, a validation set, and a test set according to a ratio of 8:2:1, and the K-means algorithm is used to cluster the rectangular boxes marked as "cracks" in the training set to optimize the size of the generated rectangular boxes.

[0026] Further, improve and train the YOLOv8 model. The specific steps are:

[0027] Step S131, the input of the YOLOv8 model is the visible light image data of 640*640 pixels after linear normalization; the input end is connected to the original backbone network and the edge backbone network respectively;

[0028] Step S132, in the original backbone network, the input end is connected to the first CBS convolution module, the second CBS convolution module and the first C2f module, the first C2f module is connected to the third CBS convolution module and the second C2f module to obtain a feature map A1, the second C2f module is connected to the fourth CBS convolution module and the third C2f module to obtain a feature map A2, and the third C2f module is connected to the fifth CBS convolution module, the fourth C2f module and the spatial pyramid pooling module to obtain a feature map A3;

[0029] Step S133, in the edge backbone network, the input end is connected to the edge feature extraction module to obtain an edge information image, the edge feature extraction module is connected to the sixth CBS convolution module, the seventh CBS convolution module and the fifth C2f module, the fifth C2f module is connected to the eighth CBS convolution module and the sixth C2f module to obtain an edge feature map B1, and the sixth C2f module is connected to the ninth CBS convolution module and the seventh C2f module to obtain a feature map B2;

[0030] Step S134, the feature map A1 and the feature map B1 are connected to the feature enhancement fusion module for fusion to obtain a first feature map AB1, and the feature map A2 and the feature map B2 are connected to the feature enhancement fusion module for fusion to obtain a second feature map AB2;

[0031] Step S135, the spatial pyramid pooling module is connected to the first upsampling module, the output of the first upsampling module is connected to the second feature map AB2, and the feature enhancement fusion module is used to fuse them to form a feature map A4;

[0032] Feature map A4 is connected to the eighth C2f module to obtain feature map A5;

[0033] Feature map A5 is connected to the second upsampling module, and the output of the second upsampling module and the first feature map AB1 are fused using the feature enhancement fusion module to form feature map A6;

[0034] Feature map A6 is connected to the ninth C2f module to obtain feature map A7;

[0035] Feature map A7 is connected to the tenth CBS convolution module, and the output of the tenth CBS convolution module is connected to feature map A5 to form feature map A8;

[0036] Feature map A8 is connected to the tenth C2f module to obtain feature map A9;

[0037] Feature map A9 is connected to the eleventh CBS convolution module, and the output of the eleventh CBS convolution module is connected to feature map A3 to form feature map A10;

[0038] Feature map A10 is connected to the eleventh C2f module to obtain feature map A11;

[0039] Feature maps A7, A9, and A11 are connected to the detection module respectively to obtain the output of the YOLOv8 model.

[0040] Furthermore, the C2f module consists of a CBS convolution module, a split module and a bottleneck block. The input of the C2f module is connected to the CBS convolution module, which is then connected to the split module to obtain two outputs. The first output is connected to n bottleneck blocks to obtain n outputs, which are concatenated with the second output obtained from the split module, and the concatenated result is connected to the second CBS convolution module to obtain the output of the C2f module.

[0041] Furthermore, the spatial pyramid pooling module includes two CBS convolution modules and three maximum pooling layers; the input of the spatial pyramid pooling module passes through the first CBS convolution module, and then connects to the three maximum pooling layers. Finally, the output of the first CBS convolution module is bridged with the output of the three maximum pooling layers, and then connected to the second CBS convolution module after bridged to obtain the output of the spatial pyramid pooling module.

[0042] Furthermore, the edge feature extraction module preprocesses the training samples using Gaussian blur to obtain a Gaussian blurred image I, and then connects three edge detection operators respectively to calculate the first-order discrete difference value of the image grayscale function to extract high-frequency edge information; the convolution kernels of the three edge detection operators are the Sobel operator, the Prewitt operator and the Roberts operator, and the calculation formulas of the three edge detection operators are shown in formula (2), formula (3), formula (4), formula (5) and formula (6);

[0043] (2);

[0044] (3);

[0045] (4);

[0046] (5);

[0047] (6);

[0048] Among them, K Sx and K Sy are the convolution kernels of the Sobel operator in the horizontal direction x and vertical direction y respectively; K Pxand K Py is the convolution kernel of the Prewitt operator in the horizontal direction x and vertical direction y; K Rx and K Ry G is the convolution kernel of the Roberts operator in the horizontal direction x and vertical direction y; x represents the horizontal gradient, G y represents the vertical gradient; I represents the Gaussian blurred image, represents the convolution operation, K x , K y Represent the convolution kernels in the horizontal direction x and vertical direction y respectively;

[0049] Using formula (7) to calculate the horizontal gradient G x and the vertical gradient G y Combine to get high-frequency edge information G i ;

[0050] (7);

[0051] Among them, G i Represents a high-frequency edge information graph;

[0052] Through the Sobel operator, Prewitt operator and Roberts operator, three different single-channel high-frequency edge information maps are obtained, namely, the single-channel high-frequency edge information map G of the Sobel operator S , the high-frequency edge information graph G of a single channel of the Prewitt operator P , the high-frequency edge information graph G of a single channel of the Roberts operator R ; The three sets of high-frequency edge information graphs are spliced ​​to obtain the input G of the edge backbone network, as shown in formula (8);

[0053] (8);

[0054] G represents three single-channel high-frequency edge information graphs concatenated into the input of the edge backbone network.

[0055] Furthermore, the feature enhancement fusion module consists of a coordinate attention module and a context modeling module; the feature enhancement fusion module uses the coordinate attention module to Optimize and use relatively advanced feature maps The weights of different channels are obtained through the context modeling module; the relatively low-level feature maps of different channels are obtained by Optimize; specifically:

[0056] The relatively low-level feature maps of the original backbone network After the coordinate attention module, the position information is embedded into the coordinate attention module to obtain the optimized low-level feature map ;

[0057] The relatively high-level feature map of the edge backbone network After the context modeling module, a relatively high-level feature map is obtained The weight of each channel in , and then the optimized low-level feature map With the weight of each channel Perform element-by-element multiplication to obtain the final optimized low-level feature map ;

[0058] The final optimized low-level feature map Compared with the relatively high-level feature map Splice to get the output of the feature enhancement fusion module ; See formula (9), formula (10), formula (11) and formula (12);

[0059] (9);

[0060] (10);

[0061] (11);

[0062] (12);

[0063] Among them, relatively low-level feature maps , relatively high-level feature map ; C, H and W are the number of channels, height and width respectively, is the real number space; is the optimized low-level feature map, is a relatively low-level feature map, Relatively high-level feature map The weight of each channel in is a relatively high-level feature map, is the optimized low-level feature map, is the output of the feature enhancement fusion module; CA is the coordinate attention module, CM is the context modeling module, and concat is the concatenation operation.

[0064] The beneficial effects of the technical solution of the present invention are as follows:

[0065] (1) The present invention designs an edge feature extraction module, which uses three high-frequency filtering operators to filter the low-frequency information of the image and extract the high-frequency edge information of the image, which is beneficial to the characteristics of the dangerous situation of the embankment cracks;

[0066] (2) The present invention designs two backbone network feature extraction modules to improve the accuracy of dike crack hazard detection. The original backbone network captures global information features from the original image, while the edge backbone network processes high-frequency edge information and extracts edge features; and by using multiple feature enhancement fusion modules to fuse the extracted semantic features and edge features of different layers, the accuracy of dike crack hazard detection is improved;

[0067] (3) The present invention designs a new feature enhancement fusion module for fusing various features. The module integrates the attention mechanism and the global context module, which enhances the model's ability to focus on high-frequency edge feature information and enhances context information, thereby effectively improving the accuracy of embankment crack hazard detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 The flowchart of the present invention for improving and training the YOLOv8 model. DETAILED DESCRIPTION

[0069] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application are described clearly, completely and in detail below, and the described embodiments are part of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0070] The present invention works and is implemented as follows: Figure 1 As shown, an intelligent identification method for dike crack danger based on edge enhancement includes two processes: improving and training the YOLOv8 model and detecting dike crack danger;

[0071] The specific steps to improve and train the YOLOv8 model are as follows:

[0072] Step S11, data collection: using a drone carrying a visible light sensor to inspect the levee, and obtaining visible light image data of the levee, including visible light images with dangerous conditions of levee cracks and visible light images without dangerous conditions of levee cracks;

[0073] Step S12, making samples: annotating and arranging the visible light image data collected in step S11 to make samples of dike crack danger;

[0074] Step S13, improving and training the YOLOv8 model: the improved YOLOv8 model consists of an original backbone network, an edge backbone network and a detection module, wherein the original backbone network and the edge backbone network are composed of a CBS convolution module, a C2f module, a spatial pyramid pooling module, an edge feature extraction module and a feature enhancement fusion module;

[0075] The specific steps for detecting embankment crack danger are as follows:

[0076] Step S21, acquiring an image: acquiring visible light image data of the levee to be detected as an input image for the levee crack hazard detection process;

[0077] Step S22, dividing the window: dividing the visible light image data of the levee into windows with 20% overlap, using a sliding window of 640*640 pixels with a sliding step of 480 pixels, and traversing the entire input image; moving the window with a height and width less than 640 pixels upward or leftward by the coordinate of the upper left corner so that the window size with a height and width less than 640 pixels is maintained at 640*640 pixels;

[0078] Record the corresponding number and upper left corner coordinates of each window in the input image, recorded as (R, C, X, Y), where R is the number of rows, C is the number of columns, and X, Y are the corresponding upper left corner coordinates;

[0079] Step S23, inputting the visible light image data of each window into the improved YOLOv8 model that has been trained in step S13; setting a confidence threshold, obtaining the center point position, size, category and confidence of the detection box that exceeds the confidence threshold; and calculating the actual position of the detection box in the input image according to formula (1);

[0080] (1);

[0081] in,( , ) is the position of the center point of the k-th detection box in the input image; ( , ) is the coordinate of the upper left corner of the current window where the detection box is located, ( , ) is the position of the center point of the kth detection box in the current window image;

[0082] Step S24, non-maximum suppression of detection results: after all windows are detected, multiple detection frames are obtained on the input image, and then weighted non-maximum suppression is used to remove overlapping detection frames;

[0083] Step S25, outputting the final detection result: outputting the result after weighted non-maximum suppression to obtain the final detection result.

[0084] Furthermore, in step S12, the visible light image data collected in step S11 is annotated and sorted to produce dike crack hazard samples; wherein the dike crack hazard samples are composed of positive samples and negative samples; specifically:

[0085] Step S121, annotating the collected visible light image data, selecting visible light image data with obvious features, and removing invalid visible light image data that is difficult to identify or has been checked;

[0086] Step S122, segmenting the selected visible light image data using a pixel size of 640*640, annotating the segmented visible light image data with the dangerous features of the dike cracks using the software Labelme, annotating with a rectangular frame, using "crack" as the dangerous mark of the dike cracks, and selecting the visible light image data with the marked rectangular frame as a positive sample, and selecting the visible light image data without the dangerous features of the dike cracks as a negative sample;

[0087] Step S123, randomly cropping, flipping, blurring, changing brightness, contrast, and randomly arranging to amplify the dike crack hazard samples, and all dike crack hazard samples are uniformly scaled to a size of 640*640 pixels;

[0088] Step S124, all the embankment crack hazard samples after uniform scaling are randomly divided into a training set, a validation set, and a test set according to a ratio of 8:2:1, and the K-means algorithm is used to cluster the rectangular boxes marked as "cracks" in the training set to optimize the size of the generated rectangular boxes.

[0089] Further, improve and train the YOLOv8 model. The specific steps are:

[0090] Step S131, the input of the YOLOv8 model is the visible light image data of 640*640 pixels after linear normalization; the input end is connected to the original backbone network and the edge backbone network respectively;

[0091] Step S132, in the original backbone network, the input end is connected to the first CBS convolution module, the second CBS convolution module and the first C2f module, the first C2f module is connected to the third CBS convolution module and the second C2f module to obtain a feature map A1, the second C2f module is connected to the fourth CBS convolution module and the third C2f module to obtain a feature map A2, and the third C2f module is connected to the fifth CBS convolution module, the fourth C2f module and the spatial pyramid pooling module to obtain a feature map A3;

[0092] Step S133, in the edge backbone network, the input end is connected to the edge feature extraction module to obtain an edge information image, the edge feature extraction module is connected to the sixth CBS convolution module, the seventh CBS convolution module and the fifth C2f module, the fifth C2f module is connected to the eighth CBS convolution module and the sixth C2f module to obtain an edge feature map B1, and the sixth C2f module is connected to the ninth CBS convolution module and the seventh C2f module to obtain a feature map B2;

[0093] Step S134, the feature map A1 and the feature map B1 are connected to the feature enhancement fusion module for fusion to obtain a first feature map AB1, and the feature map A2 and the feature map B2 are connected to the feature enhancement fusion module for fusion to obtain a second feature map AB2;

[0094] Step S135, the spatial pyramid pooling module is connected to the first upsampling module, the output of the first upsampling module is connected to the second feature map AB2, and the feature enhancement fusion module is used to fuse them to form a feature map A4;

[0095] Feature map A4 is connected to the eighth C2f module to obtain feature map A5;

[0096] Feature map A5 is connected to the second upsampling module, and the output of the second upsampling module and the first feature map AB1 are fused using the feature enhancement fusion module to form feature map A6;

[0097] Feature map A6 is connected to the ninth C2f module to obtain feature map A7;

[0098] Feature map A7 is connected to the tenth CBS convolution module, and the output of the tenth CBS convolution module is connected to feature map A5 to form feature map A8;

[0099] Feature map A8 is connected to the tenth C2f module to obtain feature map A9;

[0100] Feature map A9 is connected to the eleventh CBS convolution module, and the output of the eleventh CBS convolution module is connected to feature map A3 to form feature map A10;

[0101] Feature map A10 is connected to the eleventh C2f module to obtain feature map A11;

[0102] Feature maps A7, A9, and A11 are connected to the detection module respectively to obtain the output of the YOLOv8 model.

[0103] Furthermore, the C2f module consists of a CBS convolution module, a split module and a bottleneck block. The input of the C2f module is connected to the CBS convolution module, which is then connected to the split module to obtain two outputs. The first output is connected to n bottleneck blocks to obtain n outputs, which are concatenated with the second output obtained from the split module, and the concatenated result is connected to the second CBS convolution module to obtain the output of the C2f module.

[0104] Furthermore, the spatial pyramid pooling module includes two CBS convolution modules and three maximum pooling layers; the input of the spatial pyramid pooling module passes through the first CBS convolution module, and then connects to the three maximum pooling layers. Finally, the output of the first CBS convolution module is bridged with the output of the three maximum pooling layers, and then connected to the second CBS convolution module after bridged to obtain the output of the spatial pyramid pooling module.

[0105] Furthermore, the edge feature extraction module preprocesses the training samples using Gaussian blur to obtain a Gaussian blurred image I, and then connects three edge detection operators respectively to calculate the first-order discrete difference value of the image grayscale function to extract high-frequency edge information; the convolution kernels of the three edge detection operators are the Sobel operator, the Prewitt operator and the Roberts operator, and the calculation formulas of the three edge detection operators are shown in formula (2), formula (3), formula (4), formula (5) and formula (6);

[0106] (2);

[0107] (3);

[0108] (4);

[0109] (5);

[0110] (6);

[0111] Among them, K Sx and K Sy are the convolution kernels of the Sobel operator in the horizontal direction x and vertical direction y respectively; K Px and K Py is the convolution kernel of the Prewitt operator in the horizontal direction x and vertical direction y; K Rx and K Ry G is the convolution kernel of the Roberts operator in the horizontal direction x and vertical direction y; x represents the horizontal gradient, G y represents the vertical gradient; I represents the Gaussian blurred image, represents the convolution operation, K x , K y Represent the convolution kernels in the horizontal direction x and vertical direction y respectively;

[0112] Using formula (7) to calculate the horizontal gradient G x and the vertical gradient G y Combine to get high-frequency edge information G i ;

[0113] (7);

[0114] Among them, G i Represents a high-frequency edge information graph;

[0115] Through the Sobel operator, Prewitt operator and Roberts operator, three different single-channel high-frequency edge information maps are obtained, namely, the single-channel high-frequency edge information map G of the Sobel operator S , the high-frequency edge information graph G of a single channel of the Prewitt operator P , the high-frequency edge information graph G of a single channel of the Roberts operator R ; The three sets of high-frequency edge information graphs are spliced ​​to obtain the input G of the edge backbone network, as shown in formula (8);

[0116] (8);

[0117] G represents three single-channel high-frequency edge information graphs concatenated into the input of the edge backbone network.

[0118] Furthermore, the feature enhancement fusion module consists of a coordinate attention module and a context modeling module; the feature enhancement fusion module uses the coordinate attention module to Optimize and use relatively advanced feature maps The weights of different channels are obtained through the context modeling module; the relatively low-level feature maps of different channels are obtained by Optimize; specifically:

[0119] The relatively low-level feature maps of the original backbone network After the coordinate attention module, the position information is embedded into the coordinate attention module to obtain the optimized low-level feature map ;

[0120] The relatively high-level feature map of the edge backbone network After the context modeling module, a relatively high-level feature map is obtained The weight of each channel in , and then the optimized low-level feature map With the weight of each channel Perform element-by-element multiplication to obtain the final optimized low-level feature map ;

[0121] The final optimized low-level feature map Compared with the relatively high-level feature map Splice to get the output of the feature enhancement fusion module ; See formula (9), formula (10), formula (11) and formula (12);

[0122] (9);

[0123] (10);

[0124] (11);

[0125] (12);

[0126] Among them, relatively low-level feature maps , relatively high-level feature map ; C, H and W are the number of channels, height and width respectively, is the real number space; is the optimized low-level feature map, is a relatively low-level feature map, Relatively high-level feature map The weight of each channel in is a relatively high-level feature map, is the optimized low-level feature map, is the output of the feature enhancement fusion module; CA is the coordinate attention module, CM is the context modeling module, and concat is the concatenation operation.

[0127] The above is a detailed introduction to the intelligent identification method of embankment crack hazards based on edge enhancement provided by the present invention. Specific isolation is applied in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An intelligent identification method for dike crack danger based on edge enhancement, characterized by: It includes two processes: improving and training the YOLOv8 model and detecting embankment crack hazards; The specific steps to improve and train the YOLOv8 model are as follows: Step S11, data collection: using a drone carrying a visible light sensor to inspect the levee, and obtaining visible light image data of the levee, including visible light images with dangerous conditions of levee cracks and visible light images without dangerous conditions of levee cracks; Step S12, making samples: annotating and arranging the visible light image data collected in step S11 to make samples of dike crack danger; Step S13, improving and training the YOLOv8 model: the improved YOLOv8 model consists of an original backbone network, an edge backbone network and a detection module, wherein the original backbone network and the edge backbone network are composed of a CBS convolution module, a C2f module, a spatial pyramid pooling module, an edge feature extraction module and a feature enhancement fusion module; The specific steps for detecting embankment crack danger are as follows: Step S21, acquiring an image: acquiring visible light image data of the levee to be detected as an input image for the levee crack hazard detection process; Step S22, dividing the window: dividing the visible light image data of the levee into windows with 20% overlap, using a sliding window of 640*640 pixels with a sliding step of 480 pixels, and traversing the entire input image; moving the window with a height and width less than 640 pixels upward or leftward by the coordinate of the upper left corner so that the window size with a height and width less than 640 pixels is maintained at 640*640 pixels; Record the corresponding number and upper left corner coordinates of each window in the input image, recorded as (R, C, X, Y), where R is the number of rows, C is the number of columns, and X, Y are the corresponding upper left corner coordinates; Step S23, inputting the visible light image data of each window into the improved YOLOv8 model that has been trained in step S13; setting a confidence threshold, obtaining the center point position, size, category and confidence of the detection box that exceeds the confidence threshold; and calculating the actual position of the detection box in the input image according to formula (1); （1）； in,( , ) is the position of the center point of the k-th detection box in the input image; ( , ) is the coordinate of the upper left corner of the current window where the detection box is located, ( , ) is the position of the center point of the kth detection box in the current window image; Step S24, non-maximum suppression of detection results: after all windows are detected, multiple detection frames are obtained on the input image, and then weighted non-maximum suppression is used to remove overlapping detection frames; Step S25, outputting the final detection result: outputting the result after weighted non-maximum suppression to obtain the final detection result.

2. According to claim 1, a method for intelligently identifying dangerous conditions of dike cracks based on edge enhancement is characterized by: In step S12, the visible light image data collected in step S11 is annotated and sorted to produce dike crack hazard samples; wherein the dike crack hazard samples are composed of positive samples and negative samples; specifically: Step S121, annotating the collected visible light image data, selecting visible light image data with obvious features, and removing invalid visible light image data that is difficult to identify or has been checked; Step S122, segmenting the selected visible light image data using a pixel size of 640*640, annotating the segmented visible light image data with the dangerous features of the dike cracks using the software Labelme, annotating with a rectangular frame, using "crack" as the dangerous mark of the dike cracks, and selecting the visible light image data with the marked rectangular frame as a positive sample, and selecting the visible light image data without the dangerous features of the dike cracks as a negative sample; Step S123, randomly cropping, flipping, blurring, changing brightness, contrast, and randomly arranging to amplify the dike crack hazard samples, and all dike crack hazard samples are uniformly scaled to a size of 640*640 pixels; Step S124, all the embankment crack danger samples after uniform scaling are randomly divided into a training set, a validation set, and a test set according to 8:2:1, and the K-means algorithm is used to cluster the rectangular boxes marked as "cracks" in the training set to optimize the size of the generated rectangular boxes.

3. The method for intelligently identifying dangerous conditions of dike cracks based on edge enhancement according to claim 2 is characterized by: Improve and train the YOLOv8 model. The specific steps are: Step S131, the input of the YOLOv8 model is the visible light image data of 640*640 pixels after linear normalization; the input end is connected to the original backbone network and the edge backbone network respectively; Step S132, in the original backbone network, the input end is connected to the first CBS convolution module, the second CBS convolution module and the first C2f module, the first C2f module is connected to the third CBS convolution module and the second C2f module to obtain a feature map A1, the second C2f module is connected to the fourth CBS convolution module and the third C2f module to obtain a feature map A2, and the third C2f module is connected to the fifth CBS convolution module, the fourth C2f module and the spatial pyramid pooling module to obtain a feature map A3; Step S133, in the edge backbone network, the input end is connected to the edge feature extraction module to obtain an edge information image, the edge feature extraction module is connected to the sixth CBS convolution module, the seventh CBS convolution module and the fifth C2f module, the fifth C2f module is connected to the eighth CBS convolution module and the sixth C2f module to obtain an edge feature map B1, and the sixth C2f module is connected to the ninth CBS convolution module and the seventh C2f module to obtain a feature map B2; Step S134, the feature map A1 and the feature map B1 are connected to the feature enhancement fusion module for fusion to obtain a first feature map AB1, and the feature map A2 and the feature map B2 are connected to the feature enhancement fusion module for fusion to obtain a second feature map AB2; Step S135, the spatial pyramid pooling module is connected to the first upsampling module, the output of the first upsampling module is connected to the second feature map AB2, and the feature enhancement fusion module is used to fuse them to form a feature map A4; Feature map A4 is connected to the eighth C2f module to obtain feature map A5; Feature map A5 is connected to the second upsampling module, and the output of the second upsampling module and the first feature map AB1 are fused using the feature enhancement fusion module to form feature map A6; Feature map A6 is connected to the ninth C2f module to obtain feature map A7; Feature map A7 is connected to the tenth CBS convolution module, and the output of the tenth CBS convolution module is connected to feature map A5 to form feature map A8; Feature map A8 is connected to the tenth C2f module to obtain feature map A9; Feature map A9 is connected to the eleventh CBS convolution module, and the output of the eleventh CBS convolution module is connected to feature map A3 to form feature map A10; Feature map A10 is connected to the eleventh C2f module to obtain feature map A11; Feature maps A7, A9, and A11 are connected to the detection module respectively to obtain the output of the YOLOv8 model.

4. The method for intelligently identifying dangerous conditions of dike cracks based on edge enhancement according to claim 3 is characterized by: The C2f module consists of a CBS convolution module, a split module and a bottleneck block. The input of the C2f module is connected to the CBS convolution module, and then connected to the split module to obtain two outputs. The first output is connected to n bottleneck blocks to obtain n outputs, and the n outputs are spliced ​​with the second output obtained from the split module. The spliced ​​result is connected to the second CBS convolution module to obtain the output of the C2f module.

5. The method for intelligently identifying dangerous conditions of dike cracks based on edge enhancement according to claim 4 is characterized in that: The spatial pyramid pooling module consists of two CBS convolution modules and three maximum pooling layers. The input of the spatial pyramid pooling module passes through the first CBS convolution module and then connects to three maximum pooling layers. Finally, the output of the first CBS convolution module is bridged with the output of the three maximum pooling layers. After the bridge, it is connected to the second CBS convolution module to obtain the output of the spatial pyramid pooling module.

6. The method for intelligently identifying dangerous conditions of dike cracks based on edge enhancement according to claim 5 is characterized by: The edge feature extraction module preprocesses the training samples using Gaussian blur to obtain Gaussian blurred image I, and then connects three edge detection operators to calculate the first-order discrete difference value of the image grayscale function to extract high-frequency edge information; The convolution kernels of the three edge detection operators are Sobel operator, Prewitt operator and Roberts operator. The calculation formulas of the three edge detection operators are shown in formula (2), formula (3), formula (4), formula (5) and formula (6). (2)； (3)； (4)； (5)； (6)； Among them, K Sx and K Sy are the convolution kernels of the Sobel operator in the horizontal direction x and vertical direction y respectively; K Px and K Py is the convolution kernel of the Prewitt operator in the horizontal direction x and vertical direction y; K Rx and K Ry G is the convolution kernel of the Roberts operator in the horizontal direction x and vertical direction y; x represents the horizontal gradient, G y represents the vertical gradient; I represents the Gaussian blurred image, represents the convolution operation, K x , K y Represent the convolution kernels in the horizontal direction x and vertical direction y respectively; Using formula (7) to calculate the horizontal gradient G x and the vertical gradient G y Combine to get high-frequency edge information G i ; (7)； Among them, G i Represents a high-frequency edge information graph; Through the Sobel operator, Prewitt operator and Roberts operator, three different single-channel high-frequency edge information maps are obtained, namely, the single-channel high-frequency edge information map G of the Sobel operator S , the high-frequency edge information graph G of a single channel of the Prewitt operator P , the high-frequency edge information graph G of a single channel of the Roberts operator R ; The three sets of high-frequency edge information graphs are spliced ​​to obtain the input G of the edge backbone network, as shown in formula (8); (8)； G represents three single-channel high-frequency edge information graphs concatenated into the input of the edge backbone network.

7. The method for intelligently identifying dangerous conditions of dike cracks based on edge enhancement according to claim 6 is characterized by: The feature enhancement fusion module consists of a coordinate attention module and a context modeling module. The feature enhancement fusion module uses the coordinate attention module to Optimize and use relatively advanced feature maps The weights of different channels are obtained through the context modeling module; the relatively low-level feature maps of different channels are obtained by Optimize; specifically: The relatively low-level feature maps of the original backbone network After the coordinate attention module, the position information is embedded into the coordinate attention module to obtain the optimized low-level feature map ; The relatively high-level feature map of the edge backbone network After the context modeling module, a relatively high-level feature map is obtained The weight of each channel in , and then the optimized low-level feature map With the weight of each channel Perform element-by-element multiplication to obtain the final optimized low-level feature map ; The final optimized low-level feature map Compared with the relatively high-level feature map Splice to get the output of the feature enhancement fusion module ; See formula (9), formula (10), formula (11) and formula (12); (9)； (10)； (11)； (12)； Among them, relatively low-level feature maps , relatively high-level feature map ; C, H and W are the number of channels, height and width respectively, is the real number space; is the optimized low-level feature map, is a relatively low-level feature map, Relatively high-level feature map The weight of each channel in is a relatively high-level feature map, is the optimized low-level feature map, is the output of the feature enhancement fusion module; CA is the coordinate attention module, CM is the context modeling module, and concat is the concatenation operation.

Images