A Rapid Detection Method and System for Multiple Hazards in Concrete Dams Based on HDIOCNet Model

Abstract

This invention discloses a rapid detection method and system for multiple types of hidden dangers in concrete dams based on the HDIOCNet model. First, infrared-visible dual-light images of the concrete structure surface are acquired, preprocessed, and semantically annotated to construct a training dataset integrating the dual-light images for concrete dam hidden danger detection. Second, a deep learning model, HDIOCNet, for rapid detection of multiple types of hidden dangers in concrete dams is constructed. This model employs an efficient encoder-decoder hourglass structure. The encoder part includes a lightweight edge and texture feature extraction backbone network and a multi-scale receptive field extended dilated convolutional pooling pyramid to encode the features of the input dual-light images and output feature maps at different semantic levels. Finally, the HDIOCNet model is trained based on the dual-light dataset. The trained model is used to predict the types of hidden dangers and calculate and compare the areas of hidden dangers. This invention has the advantages of high detection accuracy, fast detection speed, and intelligence, providing an efficient solution for the safety monitoring of concrete structures.

Description

Technical Field

[0001] This invention relates to the fields of defect detection and semantic segmentation of hydraulic concrete structures, specifically to a rapid detection method and system for multiple hidden dangers in concrete dams based on the deep learning model HDIOCNet (Hidden Danger Image Of Concrete Dam Detection Network). Background Technology

[0002] Currently, the detection of hidden dangers in concrete dams still relies primarily on manual inspections. However, manual inspections suffer from limitations such as incomplete coverage, high risk, and limited frequency, often resulting in time-consuming and ineffective methods. Furthermore, when encountering seepage hazards, the difference between what the naked eye sees and the actual seepage situation is significant. In such cases, additional instruments, such as infrared thermal imagers, are needed to capture the true extent of the seepage. Therefore, manual inspections require auxiliary equipment, making them very labor-intensive. With the emergence of the concept of intelligent drones, many scholars have begun researching intelligent drone inspection models. Drone inspection models primarily utilize machine vision technology to input non-destructively acquired images into a computer for detection and recognition. There are two main methods for input image detection and recognition: methods based on digital image processing and methods based on deep learning. The digital image processing method first enhances the image and then segments the hazard area for quantitative analysis. This method is typically tailored to a specific type of hazard, determining appropriate segmentation methods and parameters. However, it lacks adaptability when the image contains noise. Moreover, it can only identify specific hazards and lacks research on classifying multiple hazards. Research on deep learning-based methods for engineering hazard detection mainly focuses on two directions: bounding box detection and pixel-by-pixel semantic segmentation. Bounding box detection methods can classify and locate various hazards, but they struggle to pinpoint hazards lacking clear boundaries. However, safety inspections of concrete dams require precise location identification to pinpoint the hazard and implement corrective measures; therefore, this method is unsuitable for hazards lacking clear boundaries. Pixel-by-pixel semantic segmentation groups pixels belonging to the same hazard in an image, effectively isolating the hazard. Since hazard detection often requires quantitative and point-based analysis, segmenting the hazard is crucial, leading to its widespread application. However, deep learning-based methods typically involve numerous parameters and large models, resulting in slower speeds in practical engineering applications.

[0003] In summary, manual inspection is time-consuming, labor-intensive, and has a high error rate. UAV inspection models based on digital image processing only address one type of hazard and lack adaptability. Among deep learning-based UAV inspection models, bounding box detection methods are ineffective for detecting hazards with missing boundaries, while pixel-by-pixel semantic segmentation models are generally large and slow. Therefore, there is a need to research an efficient, intelligent, fast, and highly accurate method for detecting hazard in concrete dams, realizing a complete process from dataset construction to hazard detection. Summary of the Invention

[0004] Purpose of the invention: In order to overcome the shortcomings of existing technologies in the detection of hidden dangers in concrete dams, this invention provides a rapid detection method and system for multiple hidden dangers in concrete dams based on the HDIOCNet model, providing advanced technology for smart water conservancy and river and lake health management.

[0005] Technical Solution: To achieve the above-mentioned objectives, the technical solution adopted by this invention is: a rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model, comprising the following steps:

[0006] (1) Collect visible light images of cracks and spalling hazards, collect infrared images of leakage hazards, perform pixel-by-pixel semantic annotation on the preprocessed visible light-infrared images, and construct a training dataset for concrete dam hazard detection integrating dual-light images.

[0007] (2) Construct a deep learning model HDIOCNet for rapid detection of multiple hidden dangers in concrete dams. The model adopts an encoder-decoder hourglass structure. The encoder part includes a lightweight edge and texture feature extraction backbone network and a multi-scale receptive field extended dilated convolutional pooling pyramid to complete the encoding of the input dual-light image features and output feature maps at different semantic levels. The decoder part includes a channel adjustment module, an upsampling module and a fusion module. The channel adjustment module adjusts the number of channels for low-level semantic features of edges and textures. The upsampling module adjusts the resolution of global high-level semantic features. The fusion module fuses the low-level semantic features of edges and textures obtained by the feature extraction backbone network with the multi-level high-level semantic feature maps obtained by the multi-scale receptive field extended dilated convolutional pooling pyramid. The fused features generate a prediction feature map with the same size as the original input image through channel number adjustment and upsampling operations, which serves as the discriminant feature.

[0008] (3) The HDIOCNet model was trained using a training dataset for detecting hidden dangers in concrete dams that integrates dual-light images.

[0009] Preferably, the preprocessing of the visible light image in step (2) includes histogram equalization and spatial filtering smoothing; the preprocessing of the infrared image includes denoising based on morphology and connected component methods.

[0010] Preferably, the lightweight edge and texture feature extraction backbone of the encoder part in step (2) includes an up-dimensional convolutional layer and multiple multi-granularity feature extraction sub-blocks. Each multi-granularity feature extraction sub-block is composed of one or more unit block structures connected in series and / or in parallel. The unit block structure is composed of two convolutional layers and one depthwise separable convolutional layer. The multiple multi-granularity feature extraction sub-blocks include at least the following types of feature extraction sub-blocks: feature extraction sub-blocks composed of three or fewer unit block structures connected in series; feature extraction sub-blocks composed of three unit block structures connected in series and in parallel; feature extraction sub-blocks composed of four unit block structures connected in series and in parallel. The multiple multi-granularity feature extraction sub-blocks output at least four extracted features at different scales. The unit block structures include two types: the first type of unit block has a convolution stride of one, and the output is added to the input in terms of structure; the second type of unit block has a convolution stride of two.

[0011] Preferably, the plurality of multi-granularity feature extraction sub-blocks consists of seven multi-granularity feature extraction sub-blocks. The first granularity feature extraction sub-block is composed of a first-class unit block. The second granularity feature extraction sub-block is composed of a second-class unit block and a first-class unit block connected in series. The extracted features include low-level features of edges and textures, which serve as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The third granularity feature extraction sub-block is composed of a second-class unit block and a first-class unit block connected in parallel, followed by a first-class unit block connected in series. The two parallel unit blocks enrich the feature hierarchy. The fusion result is connected in series with the third unit block, further fusing multi-scale information, enabling the network to extract diverse mid-level features, which serve as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The fourth granularity feature extraction sub-block consists of four first-class unit blocks, with the first and second unit blocks connected in parallel, and the third... The third and fourth unit blocks are connected in parallel, while the two are connected in series. The two parallel unit blocks improve the network's adaptability to different types of features, and the series connection between the two parallel unit blocks fuses information from different branches. The extracted features are used as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The fifth granularity feature extraction sub-block is designed in reverse with the third sub-block. This structure can acquire information at different scales simultaneously, improving the network's sensitivity to multi-scale features. It can acquire high-level semantic information while retaining more detailed features, and is used as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The sixth granularity feature extraction sub-block consists of three first-type unit blocks connected in series, improving the network's adaptability to features at different scales and orientations. The extracted features are used as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The seventh granularity feature extraction sub-block is designed the same as the first sub-block.

[0012] Preferably, the multi-scale receptive field extended dilated convolutional pooling pyramid of the encoder part in step (2) contains at least four branches, which are respectively used to input the features output by the multi-granularity feature extraction sub-block; each branch enhances and perceives features of different scales through convolution, normalization and activation operations to obtain feature maps of different semantic levels of the same size.

[0013] Preferably, the multi-scale receptive field extended dilated convolutional pooling pyramid of the encoder part in step (2) consists of five branches. The first branch is used to capture edge and texture information and improve the perception of local structure; the second branch is used to extract and enhance mid-level features of the image; the third branch is used to acquire structural and semantic information with a larger receptive field than the second branch; the fourth branch is used to acquire global high-level semantic information; and the fifth branch includes global average pooling, which is used to aggregate global information and compress features.

[0014] Preferably, the fusion process of the decoder part in step (2) is as follows: First, the feature map X obtained from the multi-scale receptive field expansion dilated convolution pooling pyramid is... concat After a convolutional layer and upsampling of a new feature layer X concat′ Secondly, the low-level semantic feature map X l A new feature layer X is obtained after passing through a convolutional layer. l′ Then X concat′ and X l′ The new feature map X is obtained by splicing. concat″ Finally, X concat″ After a convolutional layer and upsampling, a prediction image X is obtained with the same number of channels, height, and width as the input image. out .

[0015] Preferably, the loss function for training the HDIOCNet model in step (3) is expressed as:

[0016] L total =L CE +L Dice

[0017] Among them, L CE It is the cross-entropy loss, L Dice This is the Dice loss, expressed as:

[0018]

[0019] Where N represents the total number of pixels, C represents the number of categories, and Y... i,c P represents the true label of the i-th pixel belonging to the c-th category. i,c Y is the probability that the i-th pixel in the model output belongs to the c-th class.i P is the true label of the i-th pixel. i It is the model output of the i-th pixel.

[0020] A rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model includes the following steps:

[0021] Semantic segmentation of images of potential hazards in dual-light concrete dams is achieved based on the aforementioned method;

[0022] Record the type, location, and area of ​​any identified hazards. Determine the severity of the hazard based on the area. If the hazard is deemed to require repair, issue a warning to personnel to carry out the repair. If the hazard is not deemed to require repair, save the data.

[0023] Images of previously detected hazards are captured at fixed points at intervals. The HDIOCNet model is used to output the type, location, and area of ​​the same hazard. The hazard area in the two periods is compared. If there is an increase, the early warning staff will take corresponding measures.

[0024] A computer system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the steps of a rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model.

[0025] Beneficial Effects: The HDIOCNet model constructed in this invention effectively solves the problems of time-consuming and labor-intensive manual inspections, high misjudgment rates, and difficulty in detecting hidden dangers due to missing boundaries. The time-consuming and labor-intensive nature of manual inspections, along with high misjudgment rates, mainly stems from the presence of leaks in concrete structures that are invisible to the naked eye, as well as blind spots for hidden dangers that are difficult to observe. This invention utilizes a drone equipped with an infrared-visible dual-light camera. The infrared camera captures images of leaks, while the visible light camera captures images of cracks, spalling, and other hidden dangers, thus effectively solving the problems of time-consuming and labor-intensive manual inspections and high misjudgment rates. For the problem of difficulty in detecting hidden dangers due to missing boundaries, a multi-scale receptive field extended dilated convolutional pooling pyramid module is constructed to enhance the learning of object features and improve the discriminative ability of the network model. Addressing the issue of large parameter counts and slow detection speed in deep learning models, this invention constructs a lightweight edge and texture feature extraction backbone network. This network consists of multiple multi-granularity feature extraction sub-blocks, each composed of one or more unit block structures connected in series and / or parallel. By expanding the extraction scale through different serial-parallel connections in multi-granularity feature extraction sub-blocks, high feature extraction capability is achieved with a smaller number of unit blocks. This addresses the issues of large parameter count and slow detection speed in deep learning models while maintaining detection accuracy. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the rapid detection method for various hidden dangers in concrete dams according to an embodiment of the present invention.

[0027] Figure 2 This is an example image of a training dataset for detecting hidden dangers in concrete dams that integrates dual-light images, as described in this embodiment of the invention.

[0028] (a) is the original hazard image, and (b) is the label corresponding to the hazard image.

[0029] Figure 3 This is a network structure diagram of HDIOCNet, a deep learning model for rapid detection of hidden dangers in concrete dams, as described in this embodiment of the invention.

[0030] Figure 4 This is an example diagram of the unit block structure in an embodiment of the present invention.

[0031] Figure 5 This is an experimental diagram of the rapid detection of various hidden dangers in concrete dams according to an embodiment of the present invention. Detailed Implementation

[0032] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0033] Combination Figure 1 The technical details of this invention are described below. An embodiment of this invention discloses a rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model, which mainly includes the following steps:

[0034] Step 1: Construct a training dataset for detecting hidden dangers in concrete dams by integrating dual-light images.

[0035] Multiple dual-light images of various potential hazards in different concrete dams were collected, mainly images of seepage, cracks, and spalling. For the visible light images of cracks and spalling hazards, histogram equalization and spatial filtering smoothing were performed. For the infrared images of seepage, thermal signal enhancement and denoising based on morphological and connected component methods were performed. Pixel-by-pixel semantic annotation of the processed visible light-infrared images was then performed. An integrated dual-light image training dataset for concrete dam hazard detection was constructed. An example of the dataset can be found in [link to dataset]. Figure 2 ;

[0036] The denoising operation on infrared images based on morphology and connected component methods includes: performing connected component filtering on the binarized image to remove smaller connected components as non-hazardous regions; performing dilation on the filtered image to connect isolated hazardous regions separated by noise or with indistinct boundaries; for each connected component, determining whether it is a hazardous region: first, whether the area of ​​the connected component is higher than a threshold; if so, it is considered a hazardous region and the process moves to the next connected component; if not, proceeding to the second step: second, whether the distance from the boundary point of the connected component to the nearest point within the connected component is lower than a threshold; if so, it is considered a hazardous region and the process moves to the next connected component; if not, it is considered a non-hazardous region, its grayscale value is changed to 255, and it is included in the background area; finally, performing erosion processing on the obtained image to further remove noise that may lead to misidentification.

[0037] The dataset construction process includes: First, preprocessing such as denoising of various dual-light images of potential hazards is completed to obtain the processed integrated visible-infrared images; second, the Labelme tool is used to annotate the processed visible-infrared objects with semantic labels, and data augmentation techniques such as random rotation, flipping, and contrast adjustment are used to expand the data and avoid overfitting; finally, a training dataset of 2,600 integrated dual-light images for the detection of potential hazards in concrete dams is constructed.

[0038] Step 2: Construct HDIOCNet, a deep learning model for rapid detection of hidden dangers in concrete dams.

[0039] HDIOCNet network architecture, a deep learning model for rapid detection of hidden dangers in concrete dams. Figure 3 As shown, the model employs an efficient encoder-decoder hourglass structure. The encoder consists of a lightweight edge and texture feature extraction backbone and a multi-scale receptive field extended dilated convolutional pooling pyramid, which encodes the features of the input dual-light image and outputs feature maps at different semantic levels. The decoder consists of a channel adjustment module, an upsampling module, and a fusion module. The channel adjustment module adjusts the number of channels for low-level semantic features such as edges and textures, the upsampling module adjusts the resolution of global high-level semantic features, and the fusion module fuses the low-level semantic features such as edges and textures obtained from the feature extraction backbone with the multi-level high-level semantic feature maps obtained from the multi-scale receptive field extended dilated convolutional pooling pyramid. The fused features are then used to generate a predicted feature map with the same size as the original input image through channel number adjustment and upsampling operations, and this feature map is used as the discriminant feature.

[0040] To reduce the number of model parameters, a lightweight edge and texture feature extraction backbone network structure was designed in the encoder part. This structure mainly includes an up-dimensional convolutional layer and multiple multi-granularity feature extraction sub-blocks. Each multi-granularity feature extraction sub-block is composed of one or more unit block structures connected in series and / or parallel. Each unit block structure consists of two convolutional layers and one depthwise separable convolutional layer. The multiple multi-granularity feature extraction sub-blocks include at least the following forms: feature extraction sub-blocks composed of three or fewer unit block structures connected in series; feature extraction sub-blocks composed of three unit block structures connected in series and parallel; feature extraction sub-blocks composed of four unit block structures connected in series and parallel. The multiple multi-granularity feature extraction sub-blocks output at least four extracted features at different scales. The unit block structures include two types: the first type has a convolutional stride of one, and structurally, the output is added to the input; the second type has a convolutional stride of two.

[0041] In this embodiment, Figure 3The feature extraction backbone shown includes seven multi-granularity feature extraction sub-blocks. The first granularity feature extraction sub-block consists of one cascaded first-class unit block, with a channel dimensionality increase factor t = 1 and a convolution stride s = 1. The second granularity feature extraction sub-block consists of two cascaded unit blocks, where the first unit block is a second-class unit block with a channel dimensionality increase factor t = 6 and a convolution stride s = 2, and the second unit block is a first-class unit block with a channel dimensionality increase factor t = 6 and a convolution stride s = 1. The extracted features include low-level features such as edges and textures, which serve as one of the inputs to the multi-scale receptive field extended dilated convolution pooling pyramid. The third granularity feature extraction sub-block... The feature extraction sub-block consists of a series-parallel structure. The first and second unit blocks are in parallel, and the fused result is concatenated with the third unit block. The first unit block is a second-type unit block with a channel dimensionality increase factor t = 6 and a convolutional stride s = 2. The second unit block is a first-type unit block with a channel dimensionality increase factor t = 6 and a convolutional stride s = 1. The third unit block is also a first-type unit block with a channel dimensionality increase factor t = 6 and a convolutional stride s = 1. The two parallel unit blocks enrich the feature hierarchy, and the fused result, concatenated with the third unit block, further integrates multi-scale information, enabling the network to extract diverse mid-level features as multi-scale... One of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid; the fourth granularity feature extraction sub-block consists of two parallel and one serial unit block. The first and second unit blocks are in parallel, the third and fourth unit blocks are in parallel, and the two are serially connected. All use the first type of unit block with a channel dimensionality increase factor t=6 and a convolution stride s=1. The two parallel unit blocks improve the network's adaptability to different types of features, and the serial connection between the two parallel unit blocks fuses information from different branches. The extracted features serve as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid; the fifth granularity feature extraction sub-block is designed in reverse to the third sub-block. The structure can acquire information at different scales simultaneously, improving the network's sensitivity to multi-scale features. It can acquire high-level semantic information while retaining more detailed features, serving as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The sixth granularity feature extraction sub-block consists of three first-class unit blocks concatenated, improving the network's adaptability to features at different scales and orientations. The extracted features serve as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The seventh granularity feature extraction sub-block has the same design as the first sub-block. There are a total of 35 convolutional layers, including 17 separable convolutional layers, with a network parameter count of only 8.81M.

[0042] The unit block structure is shown below. Figure 4 The image manipulation process is as follows: First, the input feature map X is processed... in = C×H×W, where C, H, and W represent the number of channels, height, and width of the feature map, respectively. After passing through the first convolutional layer (t×C, 1×1), the feature map X is increased in dimension to obtain the feature map. up=t×C×H×W; Next, the feature map X up The feature map is obtained after passing through a depthwise separable convolutional layer (t×C, 3×3). Then feature map X s After the second convolutional layer (C′, 1×1), the dimensionality is reduced to obtain the feature map.

[0043] To capture multi-scale information, a multi-scale receptive field extended dilated convolutional pooling pyramid module is constructed in the encoder section. This multi-scale receptive field extended dilated convolutional pooling pyramid contains at least four branches, each used to input the features output from the multi-granularity feature extraction sub-blocks. Each branch enhances and perceives features at different scales through convolution, normalization, and activation operations, resulting in feature maps of the same size but different semantic levels. In this embodiment, Figure 3 The multi-scale receptive field extended dilated convolutional pooling pyramid module shown consists of five branches. The first branch consists of a 1×1 convolutional kernel, a BN normalization layer, and a ReLU activation function, which captures edge and texture information and improves the perception of local structures. The input to this branch is the low-level features such as edges and textures extracted by the second-granularity feature extraction sub-block of the feature extraction backbone, and the output is a feature map X. h1 =C″×H′×W′; The second branch consists of a 3×3 convolutional kernel with a dilation rate of 6, a BN normalization layer, and a ReLU activation function, achieving efficient extraction and enhancement of mid-level features in the image. The input to this branch is the diverse mid-level features extracted by the third-granularity feature extraction sub-block of the feature extraction backbone, and the output is a feature map X. h2 =C″×H′×W′; The third branch consists of a 3×3 convolutional kernel with a dilation rate of 12, a BN normalization layer, and a ReLU activation function. It acquires structural and semantic information with a larger receptive field than the second branch. The input of this branch is the feature extracted by the fourth-granularity feature extraction sub-block of the feature extraction backbone, and the output is the feature map X. h3 =C″×H′×W′; The fourth branch consists of a 3×3 convolutional kernel with a dilation rate of 18, a BN normalization layer, and a ReLU activation function, which realizes the acquisition of global high-level semantic information. The input of this branch is the high-level features extracted by the fifth-granularity feature extraction sub-block of the feature extraction backbone, and the output is the feature map X. h4 =C″×H′×W′; The fifth branch consists of global average pooling, a 1×1 convolutional kernel, a BN normalization layer, and a ReLU activation function, achieving aggregation of global information and compression of features. The input of this branch is the features extracted by the sixth-granularity feature extraction sub-block of the feature extraction backbone, and the output is the feature map X. h5= C″×H′×W′. Where C″, H′, and W′ are the number of channels, height, and width of the high-level semantic feature map obtained from each layer of the multi-scale receptive field expanded dilated convolutional pooling pyramid, respectively. Finally, the five feature maps X... h1 X h2 X h3 X h4 X h5 The feature map X is obtained by splicing. concat =5C″×H′×W′.

[0044] The fusion module of the decoder section fuses low-level semantic features such as edges and textures obtained from the fourth layer of the feature extraction backbone network with multi-level high-level semantic feature maps obtained from the multi-scale receptive field extended dilated convolutional pooling pyramid. The fused features are then used to generate a predicted feature map with the same size as the original input image through channel number adjustment and upsampling operations, and this feature map is used as the discriminative feature. The fusion process of this module is as follows: First, the feature map X obtained from the multi-scale receptive field extended dilated convolutional pooling pyramid is... concat =5C″×H′×W′ passes through a 1×1 convolutional layer and is upsampled by 4 times to obtain a new feature layer X. concat′ = C″×4H′×4W′, where C″, H′, and W′ are the number of channels, height, and width of the high-level semantic feature map obtained from each layer of the multi-scale receptive field expanded dilated convolutional pooling pyramid, respectively; then, the low-level semantic feature map X... l =C l ×H l ×W l A new feature layer X is obtained after passing through a 1×1 convolutional layer. l′ =2C l ×4H′×4W′, where C l H l W l These are the number of channels, height, and width of the low-level semantic feature maps such as edges and textures obtained from the fourth layer of the feature extraction backbone; then X concat′ and X l′ The new feature map X is obtained by splicing. concat″ =(C″+2C l )×4H′×4W′; Finally, X concar″ After passing through a 3×3 convolutional layer and upsampling by 4, a prediction image X with the same number of channels, height, and width as the input image is obtained. out = class_num × 16H′ × 16W′, where class_num is the number of predicted hazard categories, including background classes.

[0045] Step 3: Train the HDIOCNet model using the integrated dual-light image training dataset for detecting hidden dangers in concrete dams.

[0046] First, the HDIOCNet model is trained. The maximum learning rate is set to 0.007, and the minimum learning rate is set to 0.01 times the maximum learning rate, i.e., 0.00007. The Adam optimizer is used to update parameters to accelerate model convergence. The momentum factor used internally by the optimizer is set to 0.9. To prevent overfitting, the weight decay factor is set to 0.0005. The learning rate decrease method uses cosine. The HDIOCNet model is then iteratively trained using the above parameters.

[0047] The loss function for training the HDIOCNet model is expressed as:

[0048] L total =L CE +L Dice

[0049] Among them, L CE It is the cross-entropy loss, L Dice This is Dice loss, or cross-entropy loss, used to measure the difference between the model's output probability distribution and the distribution of the true labels. Dice loss measures the similarity between the regions predicted by the model and the true labels. It is expressed as:

[0050]

[0051] Where N represents the total number of pixels, C represents the number of categories, and Y... i,c P represents the true label of the i-th pixel belonging to the c-th category. i,c Y is the probability that the i-th pixel in the model output belongs to the c-th class. i P is the true label of the i-th pixel. i This is the model output for the i-th pixel. In this formula, the 2x intersection of the numerator represents the intersection of the model output and the true label, and the denominator represents the union of the two. The smaller the Dice loss, the closer the model's prediction is to the true label.

[0052] The rapid detection method for multiple hidden dangers in concrete dams disclosed in this invention, based on the above-trained HDIOCNet model, is used for the rapid detection of multiple hidden dangers in concrete dams, including:

[0053] First, the acquired dual-light images of potential hazards are sequentially input into the model. Semantic segmentation of the dual-light concrete dam hazard images is then performed based on the HDIOCNet model. The semantic segmentation results are shown below. Figure 5 .

[0054] Secondly, the type, location, and area of ​​the identified hazards are recorded. The severity of the hazard is determined based on the area. If the hazard reaches the level requiring repair, staff are alerted to proceed with the repair. If the hazard does not reach the level requiring repair, the data is saved, and the drone is re-flyed periodically to take photos of the previously detected hazard locations. The HDIOCNet model is used to output the type, location, and area of ​​the same hazard. The hazard area in the two periods is compared, and if there is an increase, the staff are alerted to take corresponding measures.

[0055] Based on the same inventive concept, the present invention discloses a computer system including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the steps of the above-described method for rapid detection of multiple hidden dangers in concrete dams based on the HDIOCNet model.

[0056] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model, characterized in that, Includes the following steps: (1) Collect visible light images of cracks and spalling hazards, collect infrared images of leakage hazards, perform pixel-by-pixel semantic annotation on the preprocessed visible light-infrared images, and construct a training dataset for concrete dam hazard detection integrating dual-light images. (2) A deep learning model HDIOCNet for rapid detection of multiple hidden dangers in concrete dams was constructed. This model adopts an encoder-decoder hourglass structure. The encoder part includes a lightweight edge and texture feature extraction backbone network and a multi-scale receptive field extended dilated convolutional pooling pyramid to encode the features of the input dual-light image and output feature maps at different semantic levels. The decoder part includes a channel adjustment module, an upsampling module, and a fusion module. The channel adjustment module adjusts the number of channels for low-level semantic features of edges and textures. The upsampling module adjusts the resolution of global high-level semantic features. The fusion module fuses the low-level semantic features of edges and textures obtained from the feature extraction backbone network with the multi-level high-level semantic feature maps obtained from the multi-scale receptive field extended dilated convolutional pooling pyramid. The fused features are then used to generate predicted features with the same size as the original input image through channel number adjustment and upsampling operations. The feature map is used as a discriminative feature; the lightweight edge and texture feature extraction backbone network includes an up-dimensional convolutional layer and multiple multi-granularity feature extraction sub-blocks. Each multi-granularity feature extraction sub-block is composed of one or more unit block structures connected in series and / or parallel, wherein the unit block structure is composed of two convolutional layers and one depthwise separable convolutional layer; the multiple multi-granularity feature extraction sub-blocks include at least the following types of feature extraction sub-blocks: feature extraction sub-blocks composed of three or fewer unit block structures connected in series; feature extraction sub-blocks composed of three unit block structures connected in series and parallel; feature extraction sub-blocks composed of four unit block structures connected in series and parallel; the multiple multi-granularity feature extraction sub-blocks output at least four extracted features at different scales; the unit block structures include two types: the first type of unit block has a convolution stride of one, and structurally the output is added to the input; the second type of unit block has a convolution stride of two. (3) The HDIOCNet model was trained using the training dataset for detecting hidden dangers in concrete dams with integrated dual-light images.

2. The rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model according to claim 1, characterized in that, The preprocessing of the visible light image in step (2) includes histogram equalization and spatial filtering smoothing; the preprocessing of the infrared image includes denoising based on morphology and connected component methods.

3. The rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model according to claim 1, characterized in that, The multiple multi-granularity feature extraction sub-blocks consist of seven sub-blocks. The first granularity feature extraction sub-block is composed of a first-class unit block. The second granularity feature extraction sub-block is composed of a second-class unit block and a first-class unit block concatenated together. The extracted features include low-level features of edges and textures, serving as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The third granularity feature extraction sub-block is composed of a second-class unit block and a first-class unit block concatenated in parallel, followed by concatenation with another first-class unit block. The two parallel unit blocks enrich the feature hierarchy, and the fusion result is concatenated with the third unit block to further integrate multi-scale information, enabling the network to extract diverse mid-level features, which serve as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The fourth granularity feature extraction sub-block consists of four first-class unit blocks, with the first and second unit blocks concatenated in parallel, and the third and fourth unit blocks concatenated together. The fourth unit block is connected in parallel, and the two parallel unit blocks are connected in series. The two parallel unit blocks improve the network's adaptability to different types of features, and the series connection between the two parallel unit blocks fuses information from different branches. The extracted features are used as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The fifth granularity feature extraction sub-block is designed in the opposite way to the third sub-block. This structure can acquire information at different scales simultaneously, improving the network's sensitivity to multi-scale features. It can acquire high-level semantic information and retain more detailed features, and is used as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The sixth granularity feature extraction sub-block consists of three first-type unit blocks connected in series, improving the network's adaptability to features at different scales and directions. The extracted features are used as one of the inputs to the multi-scale receptive field extended dilated convolutional pooling pyramid. The seventh granularity feature extraction sub-block is designed the same as the first sub-block.

4. The rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model according to claim 1, characterized in that, The multi-scale receptive field extended dilated convolutional pooling pyramid of the encoder part in step (2) contains at least four branches, which are respectively used to input the features output by the multi-granularity feature extraction sub-block; each branch enhances and perceives features of different scales through convolution, normalization and activation operations to obtain feature maps of different semantic levels of the same size.

5. The rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model according to claim 3, characterized in that, The multi-scale receptive field extended dilated convolutional pooling pyramid of the encoder part in step (2) consists of five branches. The first branch is used to capture edge and texture information and improve the perception of local structure. The second branch is used to extract and enhance mid-level features of the image. The third branch is used to acquire structural and semantic information with a larger receptive field than the second branch. The fourth branch is used to acquire global high-level semantic information. The fifth branch includes global average pooling, which is used to aggregate global information and compress features.

6. The rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model according to claim 1, characterized in that, The fusion process of the decoder part in step (2) is as follows: First, the feature map obtained from the multi-scale receptive field expansion dilated convolution pooling pyramid is used as the fusion process. After a convolutional layer and an upsampling layer of new features Secondly, the low-level semantic feature map A new feature layer is obtained after passing through a convolutional layer. Then and A new feature map is obtained by splicing. ;at last After a convolutional layer and upsampling, a prediction image with the same number of channels, height, and width as the input image is obtained. .

7. The rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model according to claim 1, characterized in that, The loss function for training the HDIOCNet model in step (3) is expressed as: ;in, It is cross-entropy loss. This is the Dice loss, expressed as: ; = ;in, This represents the total number of pixels. Indicates the number of categories, Indicates the first The pixel belongs to the first The true label of the category, It is the first output of the model The pixel belongs to the first The probability of the category, It is the first The real label of each pixel It is the first The model output is for each pixel.

8. A rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model, characterized in that, Includes the following steps: A rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model according to any one of claims 1-7 is used to achieve semantic segmentation of hidden danger images of dual-light concrete dams. Record the type, location, and area of ​​any identified hazards. Determine the severity of the hazard based on the area. If the hazard is deemed to require repair, issue a warning to personnel to carry out the repair. If the hazard is not deemed to require repair, save the data. Images of previously detected hazards are captured at fixed points at intervals. The HDIOCNet model is used to output the type, location, and area of ​​the same hazard. The hazard area in the two periods is compared. If there is an increase, the early warning staff will take corresponding measures.

9. A computer system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements the steps of the rapid detection method for multiple hidden dangers in concrete dams based on the HDIOCNet model according to any one of claims 1-8.

Images