Bridge support void identification method based on unmanned aerial vehicle aerial image processing

Abstract

This application relates to the technical field of bridge bearing detection, and discloses a method for identifying bridge bearing voids based on UAV aerial image processing. The method includes controlling a UAV equipped with an imaging unit to acquire images of the bridge bearing from multiple perspectives; sequentially performing illumination equalization and shadow removal processing on the original images; extracting images of the bearing region and inputting them into a pre-trained deep learning segmentation network to generate a void region segmentation mask. The network includes a feature extraction backbone network, a feature pyramid fusion module, and a dual attention mechanism module; calculating the geometric feature parameters of the void region and converting its pixel units to actual physical units; and in step S4, calculating risk scores and assigning risk levels through an evaluation model, integrating the detection results to generate a report. This method solves the problems of uneven illumination under the bridge, shadow interference, and multi-scale void adaptation, achieving non-contact, high-precision, and fully automated bridge bearing void identification, thus improving detection safety and efficiency.

Description

Technical Field

[0001] This application relates to the technical field of bridge bearing detection, and discloses a method for identifying bridge bearing detachment based on UAV aerial image processing. Background Technology

[0002] Bridge bearings are critical load-bearing components connecting the superstructure and substructure of a bridge. Bearing detachment refers to the appearance of gaps between the bearing and the bottom of the beam or pier, resulting in partial or complete loss of contact between the bearing and the beam. This type of defect directly threatens the operational safety of the bridge. If not detected and addressed in a timely manner, it may lead to beam cracking, uneven stress on the piers, or even bridge collapse.

[0003] Existing methods for detecting bridge bearing voids have many drawbacks. For example, visual inspection requires personnel to work under the bridge, posing high safety risks such as falls from heights and traffic collisions. It is also inefficient, costly, and relies heavily on human experience, resulting in low accuracy in poorly lit conditions under the bridge, and small voids are easily missed. Sensor-based detection methods require the installation of vibration sensors, strain gauges, and other equipment on the bridge structure, which is time-consuming and interferes with normal bridge operation. Furthermore, these methods can only indirectly determine the presence of voids through dynamic characteristics and cannot quantify key geometric parameters such as the area and depth of the void. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of the embodiments of this application and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents, and such simplifications or omissions should not be construed as limiting the scope of this application.

[0005] To address the aforementioned technical problems, this application provides a method for identifying bridge support detachment based on UAV aerial image processing.

[0006] On the one hand, this application provides a method for identifying bridge bearing detachment based on UAV aerial image processing, including S1 controlling a UAV platform equipped with an imaging unit to acquire multi-view images of the bridge bearing, obtaining an original image containing the bearing area, and sequentially performing illumination equalization processing and shadow removal processing on the original image to obtain a preprocessed image; S2 locates and extracts the region where the support is located in the preprocessed image to obtain a support region image. The support region image is then input into a pre-trained deep learning segmentation network to generate a binary segmentation mask that identifies the empty region. The deep learning segmentation network includes a feature extraction backbone network, a feature pyramid fusion module, and a dual attention mechanism module. S3 calculates the geometric feature parameters of the empty region using the segmentation mask, and converts the pixel units of the geometric feature parameters into actual physical units; S4 calculates a risk score based on the geometric feature parameters of the actual physical unit through an evaluation model, and determines the risk level according to the risk score. It then integrates the segmentation mask, geometric feature parameters, risk level, and corresponding maintenance suggestions to generate a detection report.

[0007] As a preferred embodiment of the bridge bearing void recognition method based on UAV aerial image processing in this application, wherein: The shadow removal process is used to distinguish and remove shadow areas caused by the angle of illumination from real void areas caused by structural voids. The original image is converted from the RGB color space to the LAB color space, and the brightness channel of the converted image is subjected to contrast-limited adaptive histogram equalization. The processed image is converted to the YUV color space, and histogram equalization is performed on the luminance component of the YUV color space.

[0008] As a preferred embodiment of the bridge bearing void recognition method based on UAV aerial image processing in this application, wherein: The image after illumination equalization is converted from the RGB color space to the HSV color space; An adaptive brightness threshold is calculated using the brightness components of the transformed image, and pixel regions with brightness values ​​lower than the adaptive brightness threshold are marked as shadow candidate regions. Connectivity analysis is performed on the candidate shadow regions to obtain multiple connected components; Calculate the boundary complexity index for each of the connected components, the boundary complexity index being determined based on the ratio of the perimeter to the area of ​​the connected component; Based on the boundary complexity index and the average brightness of the connected component, the candidate shadow regions are filtered: connected components with boundary complexity lower than a preset complexity threshold and average brightness higher than a preset brightness threshold are identified as shadow regions formed by illumination occlusion and removed from the image; connected components with boundary complexity higher than the preset complexity threshold or average brightness lower than the preset brightness threshold are retained as suspected empty regions, thus obtaining the preprocessed image.

[0009] As a preferred embodiment of the bridge bearing void recognition method based on UAV aerial image processing in this application, wherein: The adaptive brightness threshold is calculated based on the global mean and global standard deviation of the brightness component, with the global mean as the benchmark, minus the product of the global standard deviation and the adjustment coefficient. The range of values ​​for the adjustment coefficient is set according to the common lighting environment of bridge bearings, in order to adapt to the shadow detection needs under various lighting conditions.

[0010] As a preferred embodiment of the bridge bearing void recognition method based on UAV aerial image processing in this application, wherein: The feature pyramid fusion module uses adaptive weighted fusion to fuse multi-scale features, including small gaps and large-area gaps that require different scale feature responses; The adaptive weighted fusion strategy includes: obtaining feature maps of multiple scales from the feature extraction backbone network; The feature maps are adjusted to a uniform number of channels and spatial size through convolution operations; Assign learnable initial weight coefficients to each of the adjusted feature maps; The initial weight coefficients are mapped to fusion coefficients using a normalization function; The fusion coefficients are used to perform a weighted summation of the feature maps to obtain a fused feature map, and the most effective feature scale for the current void region is marked.

[0011] As a preferred embodiment of the bridge bearing void recognition method based on UAV aerial image processing in this application, wherein: The dual attention mechanism module includes a channel attention submodule, a spatial attention submodule, and a context-aware attention submodule connected in sequence, which are used to sequentially enhance the network’s feature response to the empty region and suppress shadow residue; The channel attention submodule performs global average pooling and global max pooling on the input feature map, inputs the pooling results into a shared multilayer perceptron to generate channel attention weights, and uses the channel attention weights to weight the input feature map to enhance channel features sensitive to empty regions. The spatial attention submodule performs average pooling and max pooling along the channel dimension on the channel-weighted feature map, concatenates the pooling results and performs convolution to generate spatial attention weights, and uses the spatial attention weights to weight the feature map to locate the spatial position of the empty region. The context-aware attention submodule applies multiple convolutional kernels of different sizes in parallel to the spatially weighted feature map to capture contextual information of different ranges. After concatenating and fusing the convolutional results, it performs residual connection with the feature map input to the context-aware attention submodule to obtain the enhanced feature map.

[0012] As a preferred embodiment of the bridge bearing void recognition method based on UAV aerial image processing in this application, wherein: The context-aware attention submodule uses multiple convolutional kernels applied in parallel with increasing sizes to capture information from small-scale local textures to large-scale structural contexts. By combining convolutional kernels of different sizes in parallel, the consistency of local features presented by the shadow region in the multi-scale receptive field and the structural contrast features of the void region with the environmental background in the multi-scale receptive field are utilized.

[0013] As a preferred embodiment of the bridge bearing void recognition method based on UAV aerial image processing in this application, wherein: The deep learning segmentation network employs a boundary enhancement loss function during training, which is a weighted combination of binary cross-entropy loss, Dice loss, and boundary loss.

[0014] As a preferred embodiment of the bridge bearing void recognition method based on UAV aerial image processing in this application, wherein: The boundary loss method includes: extracting boundary pixels from the labeled real segmentation mask, setting a weight coefficient for the boundary pixels higher than that for non-boundary pixels, and calculating a weighted binary cross-entropy loss based on the weight coefficient as the boundary loss, so that the network pays more attention to the edge pixels of the empty region during training.

[0015] As a preferred embodiment of the bridge bearing void recognition method based on UAV aerial image processing in this application, wherein: In the step of quantifying the gap feature, the ground sampling distance is calculated based on the calibration parameters of the imaging unit and the flight altitude of the UAV platform when acquiring images; using the ground sampling distance, the pixel area of ​​the gap region in the segmentation mask is converted into the actual physical area to quantify the severity of gap. In the risk level assessment step, the preset assessment model is a hierarchical analysis model. The hierarchical analysis model includes geometric characteristic parameters of the actual physical area, support spatial location factors, structural stress influence parameters, and historical development trend factors to calculate risk scores. According to the preset risk score range, the assessment results are divided into four incremental risk levels, and maintenance recommendations are matched accordingly.

[0016] The beneficial effects of this application are as follows: This application utilizes drones for non-contact aerial photography throughout the entire process, eliminating the need for personnel to enter the area under the bridge and completely removing the risks of falls from heights and traffic collisions. It is particularly suitable for high-risk scenarios such as mountain bridges and bridges spanning rivers. The inspection time for a single bridge is significantly reduced, lowering inspection costs.

[0017] This application effectively suppresses uneven lighting and shadow interference under bridges through targeted illumination equalization and shadow elimination preprocessing based on boundary complexity. A deep learning segmentation network, incorporating a feature pyramid fusion module and a dual attention mechanism module, accurately adapts to multi-scale void regions and enhances the ability to identify void boundaries, significantly improving void detection accuracy, effectively reducing shadow false detection rate, and markedly improving boundary localization. Furthermore, it automatically calculates various geometric feature parameters of void regions and obtains actual physical dimensions through pixel-to-physical scale conversion, eliminating the subjectivity of manual interpretation. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 A flowchart of the overall method for identifying bridge bearing voids based on UAV aerial image processing provided in this application; Figure 2 A detailed flowchart of image preprocessing and shadow removal for a bridge bearing void recognition method based on UAV aerial image processing provided in this application; Figure 3 A deep learning segmentation network architecture diagram for a bridge support detachment recognition method based on UAV aerial image processing provided in this application; Figure 4 A comparison of the detection effects of the bridge bearing void identification method based on UAV aerial image processing provided in this application with existing technologies; Figure 5 This application provides a risk rating system architecture diagram for a bridge support detachment identification method based on UAV aerial image processing. Detailed Implementation

[0019] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0020] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0021] Secondly, the term "embodiment" or "example" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of this application. The phrase "in an embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0022] Example 1 like Figure 1 As shown, a method for identifying bridge support voiding based on UAV aerial image processing includes... S1 controls a drone platform equipped with an imaging unit to acquire multi-view images of the bridge supports, obtaining raw images containing the support area. The raw images are then subjected to illumination equalization and shadow removal processing sequentially to obtain pre-processed images. The complete process of illumination equalization and shadow removal processing is as follows: Figure 2 As shown.

[0023] The shadow removal process is used to distinguish and remove shadow areas caused by the angle of illumination from real void areas caused by structural voids. The original image is converted from the RGB color space to the LAB color space, and the brightness channel of the converted image is subjected to contrast-limited adaptive histogram equalization to suppress uneven lighting distribution caused by the bridge supports being located in the shadow area under the bridge or in the area of ​​direct strong light.

[0024] First, the acquired raw image is converted from the RGB color space to the LAB color space, and the luminance channel is extracted. Since the luminance channel and color information are separated in the LAB color space, it is easy to process the illumination component separately without affecting the color information of the image.

[0025] Secondly, the extracted brightness channels are subjected to contrast-limited adaptive histogram equalization. This process, by performing histogram equalization in local areas and limiting the contrast amplification factor, can effectively improve the visibility of details in dark areas of the image, while avoiding overexposure in bright areas and noise amplification. It also suppresses the uneven illumination distribution caused by the bridge supports being located in the shadow area under the bridge or in areas directly exposed to strong light, thus making the brightness of each area on the support surface more uniform.

[0026] Then, the processed image is converted from the LAB color space back to the RGB color space to obtain the first intermediate image.

[0027] Finally, the first intermediate image is converted to the YUV color space, and histogram equalization is performed on the luminance components of the YUV color space. The luminance components of the YUV color space are closer to the human eye's perception of brightness. Histogram equalization of the luminance components can further optimize the global brightness distribution of the image and enhance the contrast between the surface texture of the support and the edges of the voided area. Shadow removal processing aims to distinguish and remove shadow areas caused by lighting angles, while preserving genuine void areas caused by structural voids, thus resolving the visual confusion between the two. The specific steps are as follows: The image after illumination equalization is converted from the RGB color space to the HSV color space. The hue, saturation and luminance components of the HSV color space are separated, which makes it easier to identify shadow areas based on color and luminance features.

[0028] An adaptive brightness threshold is calculated by analyzing the brightness components of the converted image, and pixel regions with brightness values ​​lower than the adaptive brightness threshold are marked as shadow candidate regions. The adaptive brightness threshold is dynamically determined based on the brightness distribution characteristics of the entire image, which can adapt to the shadow detection requirements under different lighting conditions. A preliminary screening is performed by combining the saturation threshold, and regions with saturation values ​​lower than the preset threshold are included in the shadow candidate regions.

[0029] Connectivity analysis is performed on the marked shadow candidate regions to obtain multiple independent connected regions. The boundary complexity index is calculated for each connected region. The complexity index is determined based on the ratio of the perimeter to the area of ​​the connected region. Since the shadow region has a smooth boundary, its boundary complexity is low; while the empty region is affected by factors such as dust and corrosion, and its boundary often exhibits irregular characteristics, resulting in higher boundary complexity.

[0030] The candidate shadow regions are filtered based on the boundary complexity index and the average brightness of each connected component. Connected components with boundary complexity below a preset complexity threshold and average brightness above a preset brightness threshold are identified as shadow regions formed by illumination occlusion and removed from the image; connected components with boundary complexity above a preset complexity threshold or average brightness below a preset brightness threshold are retained as suspected empty regions.

[0031] Brightness compensation processing is performed on the areas identified as shadows to make the brightness of the shadow areas consistent with that of the non-shadow areas, so as to avoid the impact of sudden brightness changes on the subsequent recognition effect and obtain a pre-processed image.

[0032] The processed image is converted to the YUV color space, and histogram equalization is performed on the luminance component of the YUV color space to enhance the contrast between the surface texture of the support and the edge of the void area.

[0033] The first intermediate image, processed in the LAB color space and converted back to the RGB color space, is further converted to the YUV color space. The YUV color space is a color representation that separates luminance and chrominance information, where the Y component represents the luminance component and the U and V components represent the chrominance components.

[0034] Although the brightness of dark areas has been improved and the overall illumination distribution has become more balanced after processing the image in the LAB color space, the fine texture features on the support surface and the edge features of the void area may still not be prominent enough. By performing histogram equalization on the Y component, the visibility of texture features such as rust patterns and rubber aging marks on the support surface can be enhanced, while making the boundary transition between the void area and the support body sharper and the contrast higher.

[0035] After the histogram equalization of the Y component is completed, the processed Y component is recombined with the original U and V components to form a new YUV image. Finally, it is converted back to the RGB color space to obtain the final image after illumination equalization.

[0036] S2 locates and extracts the region where the support is located in the preprocessed image to obtain a support region image. This support region image is then input into a pre-trained deep learning segmentation network to generate a binary segmentation mask identifying the empty region. The deep learning segmentation network includes a feature extraction backbone network, a feature pyramid fusion module, and a dual attention mechanism module, such as... Figure 3 As shown, the feature extraction backbone network is located in Figure 3 On the left, a ResNet50 deep convolutional neural network structure is used to extract multi-scale image features from the input support region image. The backbone network outputs feature maps at four scales through layer-by-layer downsampling, from top to bottom: C2 layer: High-resolution feature map with a size of 128×128×256, which retains rich spatial detail information and is used to capture small-sized void regions; C3 layer: medium-resolution feature map, with a size of 64×64×512, balancing details and semantic information; C4 layer: medium to low resolution feature map, size 32×32×1024, with strong semantic expression ability; C5 layer: Low-resolution feature map with a size of 16×16×2048, which has the largest receptive field and is used to extract global contextual information.

[0037] The feature maps at four scales are arranged side by side from left to right, with arrows pointing downwards from each scale and converging into the feature pyramid fusion module.

[0038] Feature pyramid fusion module is located in Figure 3 In the upper middle section, four scale feature maps are received from the backbone network. The module internally displays six processing steps in text boxes: The number of channels in 1×1 convolution is unified to 256; Upsampling operations standardize the spatial size to 32×32; Assign learnable initial weight coefficients w=[w1,w2,w3,w4] to feature maps at each scale; The initial weights are mapped to fusion coefficients α using the Softmax normalization function; The weighted summation yields the fused feature map F_fused = Σ(α_i × F_i); Feature enhancement is achieved through 3×3 convolution, batch normalization, and ReLU activation.

[0039] The output arrow points to the dual attention mechanism module below.

[0040] The dual attention mechanism module is located in Figure 3 The lower center section consists of three interconnected sub-modules, arranged from left to right: Channel attention submodule: Performs global average pooling and global max pooling on the input feature map, inputs the pooling results into a shared multilayer perceptron, generates channel attention weights through sigmoid activation, and performs channel-by-channel weighting with the input feature map to enhance the empty sensitive channels.

[0041] The spatial attention submodule performs average pooling and max pooling on the channel-weighted feature map along the channel dimension. The pooling results are concatenated and then converted to spatial attention weights by 7×7 convolution and sigmoid activation. These weights are then weighted with the input feature map on a per-spatial-position basis to achieve accurate spatial positioning of the empty region.

[0042] The context-aware attention submodule applies four convolutional kernels of different sizes (1×1, 3×3, 5×5, 7×7) in parallel to the spatially weighted feature map to capture feature information from small-scale local textures to large-scale structural contexts. The results of the four convolutional branches are concatenated along the channel dimension, compressed by 1×1 convolution, and then residually connected with the input feature map to output the enhanced feature map.

[0043] The three sub-modules are connected by arrows, indicating the data flow through each module in sequence. Arrows extend from below each module, pointing to the decoder module.

[0044] The decoder module is located in Figure 3 On the right, four vertically arranged squares represent the progressive upsampling process: from 32×32 to 64×64 to 128×128 to 512×512. At the 64×64 and 128×128 levels, dashed arrows extend from the left to indicate jump connections to the backbone network layers C3 and C2, respectively, signifying the addition of high-resolution detail information during the upsampling process. An arrow at the bottom of the module points to the final output.

[0045] The image after illumination equalization is converted from the RGB color space to the HSV color space.

[0046] An adaptive brightness threshold is calculated using the brightness components of the transformed image, and pixel regions with brightness values ​​lower than the adaptive brightness threshold are marked as shadow candidate regions.

[0047] Connectivity analysis is performed on the shadow candidate regions to obtain multiple connected components.

[0048] Calculate the boundary complexity index for each of the connected components, the boundary complexity index being determined based on the ratio of the perimeter to the area of ​​the connected component.

[0049] Based on the boundary complexity index and the average brightness of the connected component, the candidate shadow regions are filtered: connected components with boundary complexity lower than a preset complexity threshold and average brightness higher than a preset brightness threshold are identified as shadow regions formed by illumination occlusion and removed from the image; connected components with boundary complexity higher than the preset complexity threshold or average brightness lower than the preset brightness threshold are retained as suspected empty regions, thus obtaining the preprocessed image.

[0050] The preprocessed image is converted from the RGB color space to the HSV color space. The HSV color space decomposes color into three components: hue, saturation, and lightness, which facilitates the selection of target areas based on color attributes. Bridge bearings are typically made of materials such as cast steel and rubber, which have specific hue and saturation distribution ranges in the HSV color space. By setting hue and saturation selection criteria, pixel areas that match the color characteristics of the bearing material are marked as candidate bearing areas.

[0051] Edge detection is performed on the preprocessed image to extract edge information. The edge detection uses gradient calculation to calculate the gradient magnitude of the image and extracts strong and weak edges through double thresholding to form an edge image. The edge image can reflect the boundary information between the support and the background, and between the support and the voided area.

[0052] The candidate support region obtained based on color feature screening is fused with the edge image, and the intersection of the two is taken as the initial contour candidate region. The initial contour candidate region is then decomposed into multiple independent connected contours.

[0053] Shape features are filtered for the extracted connected contours. The rectangularity index of each contour is calculated, which reflects how close the contour is to its minimum bounding rectangle. At the same time, the area of ​​each contour in the whole image is calculated. Contours with rectangularity higher than a preset threshold and area within a preset range are identified as support regions. Interference contours that do not conform to the shape features are removed. For the selected support contours, the minimum bounding rectangle is extracted as the region of interest of the support region to obtain the support region image.

[0054] The feature pyramid fusion module employs adaptive weighted fusion to fuse multi-scale features in order to address the issue of varying sizes of void regions in the image, including the need for different scale feature responses for small voids and large voids.

[0055] The adaptive weighted fusion strategy includes: obtaining feature maps of multiple scales from the feature extraction backbone network; The feature maps are adjusted to a uniform number of channels and spatial size through convolution operations.

[0056] Each adjusted feature map is assigned a learnable initial weight coefficient, which is automatically adjusted during training based on the contribution of empty samples at different scales.

[0057] The initial weight coefficients are mapped to fusion coefficients using a normalization function.

[0058] The fusion coefficients are used to perform a weighted summation of the feature maps to obtain a fused feature map, and the most effective feature scale for the current void region is marked.

[0059] Specifically, the feature pyramid fusion module receives multi-scale feature maps output by the feature extraction backbone network and employs an adaptive weighted fusion strategy to fuse features at different scales. Specifically, firstly, the feature maps at each scale are adjusted to a uniform number of channels through convolution operations and then to a uniform spatial size through upsampling operations. Next, learnable initial weight coefficients are assigned to the adjusted feature maps at each scale. These weight coefficients are automatically optimized and adjusted during the end-to-end training of the network based on the contribution of features at different scales to gap recognition. Finally, a normalization function maps the initial weight coefficients to fusion coefficients, ensuring that the sum of all fusion coefficients is one.

[0060] By using fusion coefficients to perform weighted summation on feature maps at different scales, a fused feature map is obtained. The fused feature map combines the advantages of features at different scales, retaining the detailed information of shallow features while incorporating the semantic information of deep features, and can effectively adapt to empty regions of different sizes.

[0061] The dual attention mechanism module includes a channel attention submodule, a spatial attention submodule, and a context-aware attention submodule connected in sequence, which are used to sequentially enhance the network’s feature response to the empty region and suppress shadow residue.

[0062] The channel attention submodule performs global average pooling and global max pooling on the input feature map, inputs the pooling results into a shared multilayer perceptron to generate channel attention weights, and uses the channel attention weights to weight the input feature map to enhance channel features sensitive to empty regions.

[0063] The obtained support region image is input into a pre-trained deep learning segmentation network to generate a binary segmentation mask that identifies the empty region. The deep learning segmentation network includes a feature extraction backbone network, a feature pyramid fusion module, and a dual attention mechanism module. These modules are connected in sequence and work together.

[0064] The feature extraction backbone network employs a deep convolutional neural network structure to extract multi-scale image features from the input support region image. This network downsamples the image layer by layer through stacked convolutional layers, pooling layers, and activation function layers, outputting feature maps at different resolution levels.

[0065] Specifically, the dual attention mechanism module includes a channel attention submodule, a spatial attention submodule, and a context-aware attention submodule connected in sequence, which are used to sequentially enhance the network's feature response to the empty region and suppress interference information.

[0066] The channel attention submodule performs global average pooling and global max pooling on the input feature map, inputs the pooling results into a shared multilayer perceptron network, and generates channel attention weights through activation functions. These attention weights reflect the importance of different feature channels for gap recognition. Channels sensitive to gap regions are assigned higher weights, while channels sensitive to background or noise are assigned lower weights. The channel attention weights are then weighted channel by channel with the input feature map to obtain the first weighted feature map, thereby enhancing the features of important channels.

[0067] The spatial attention submodule performs average pooling and max pooling on the first weighted feature map along the channel dimension, concatenates the pooling results along the channel dimension, and generates spatial attention weights after processing by convolutional layers and activation functions. The spatial attention weights reflect the importance of different spatial locations in the feature map for void recognition. The location of the void region is given a higher weight, and the background region is given a lower weight. The spatial attention weights are then element-wise weighted with the first weighted feature map to obtain the second weighted feature map, thereby realizing the spatial location of the void region.

[0068] The context-aware attention submodule applies multiple convolutional kernels of different sizes in parallel to the second weighted feature map. Different kernel sizes have different receptive fields, capturing contextual information within different ranges: smaller kernels focus on local texture and detail features, while larger kernels focus on regional structure and global context. The outputs of each convolutional branch are concatenated and fused along the channel dimension, then compressed through convolutional layers, and finally residually connected with the feature map input to the context-aware attention submodule to obtain an enhanced feature map. By comparing multi-scale contextual information, utilizing the consistency of local features in the shadow region under multi-scale receptive fields, and the structural contrast features between the empty region and the environmental background under multi-scale receptive fields, further suppression of shadow residue and enhancement of the boundaries of the empty region are achieved.

[0069] Among them, shallow feature maps have high spatial resolution and can preserve detailed information in the image, playing an important role in the recognition of small-sized void regions; deep feature maps have a large receptive field and rich semantic information, which can reflect the overall structure and contextual relationships of void regions.

[0070] The spatial attention submodule performs average pooling and max pooling along the channel dimension on the channel-weighted feature map, concatenates the pooling results and performs convolution to generate spatial attention weights, and uses the spatial attention weights to weight the feature map to locate the spatial position of the empty region.

[0071] The context-aware attention submodule applies multiple convolutional kernels of different sizes in parallel to the spatially weighted feature map to capture contextual information of different ranges. After concatenating and fusing the convolutional results, it performs residual connection with the feature map input to the context-aware attention submodule to obtain the enhanced feature map.

[0072] The context-aware attention submodule uses multiple convolutional kernels applied in parallel with increasing sizes to capture information ranging from small-scale local textures to large-scale structural contexts.

[0073] By combining convolutional kernels of different sizes in parallel, the consistency of local features presented by the shadow region under multi-scale receptive fields, and the structural contrast features of the void region with the environmental background presented under multi-scale receptive fields, further suppression of shadow residue and enhancement of the boundary of the void region are achieved.

[0074] The deep learning segmentation network employs a boundary enhancement loss function during training to address the insufficient segmentation accuracy caused by blurred edges in the empty regions. The boundary enhancement loss function is a weighted combination of binary cross-entropy loss, Dice loss, and boundary loss.

[0075] The boundary loss method includes: extracting boundary pixels from the labeled real segmentation mask, setting a weight coefficient for the boundary pixels higher than that for non-boundary pixels, and calculating a weighted binary cross-entropy loss based on the weight coefficient as the boundary loss, so that the network pays more attention to the edge pixels of the empty region during training and improves the accuracy of segmentation boundaries.

[0076] In the step of quantifying the gap feature, the ground sampling distance is calculated based on the calibration parameters of the imaging unit and the flight altitude of the UAV platform when acquiring images; using the ground sampling distance, the pixel area of ​​the gap region in the segmentation mask is converted into the actual physical area to quantify the severity of gap.

[0077] The enhanced feature map, processed by the dual attention mechanism module, is input to the decoder module. The decoder module restores the feature map to the original size of the input image through progressive upsampling. During upsampling, it performs skip connections with feature maps of corresponding scales in the feature extraction backbone network to supplement high-resolution detail information. The decoder ultimately outputs a segmentation result of the same size as the input support region image. This result is processed by an activation function to generate a binary segmentation mask, where a pixel value of one indicates a hollowed-out region, and a pixel value of zero indicates a non-hollowed-out region. The segmentation mask accurately reflects the position, shape, and boundary information of the hollowed-out portion in the support region.

[0078] In the risk level assessment step, the preset assessment model is a hierarchical analysis model. The hierarchical analysis model includes geometric characteristic parameters of the actual physical area, support spatial location factors, structural stress influence parameters, and historical development trend factors to calculate risk scores. According to the preset risk score range, the assessment results are divided into four incremental risk levels, and maintenance recommendations are matched accordingly.

[0079] like Figure 4 As shown, the performance of the proposed method is compared with that of traditional network methods and ordinary FCN in the task of detecting bridge bearing detachment. The table contains three core evaluation indicators: The detection accuracy of this application reaches 100%, which is significantly higher than the 95% of the traditional network method and the 90% of the ordinary FCN.

[0080] The false detection rate (ErrorRate) of this application reaches 20%, which is far lower than the 50% of the traditional network method and the 10% of the ordinary FCN, indicating that the present invention effectively suppresses false recognition caused by interference factors such as shadows.

[0081] The boundary overlap ratio (IoU) of this application reaches 85%, which is better than the 80% of the traditional network method and the 75% of the ordinary FCN, proving that the present invention can locate the boundary of the void region more accurately.

[0082] Figure 4 The data shown verifies the technical advantages of the deep learning segmentation network proposed in this application, which includes a feature pyramid fusion module and a dual attention mechanism module, in multi-scale gap-free adaptation, shadow interference suppression, and refined boundary segmentation.

[0083] S4 calculates a risk score based on the geometric feature parameters of the actual physical unit through an evaluation model, and determines the risk level according to the risk score. It then integrates the segmentation mask, geometric feature parameters, risk level, and corresponding maintenance suggestions to generate a detection report.

[0084] like Figure 5 The evaluation system shown consists of four primary indicators, with the weights of each indicator indicated in parentheses.

[0085] Geometric features (45%): including three secondary indicators: area ratio (S_A1), maximum depth (S_A2), and aspect ratio. Location factor (30%): includes two dimensions: center void and edge / local void.

[0086] Structural impact (15%): includes two dimensions: beam displacement and support deformation.

[0087] Development Trend (10%): Includes historical expansion rate indicators.

[0088] The risk is divided into four levels based on the overall score: Level I (Observation): Score <30, maintenance recommendation is to re-examine in 6 months.

[0089] Level II (General): Score 30-60, maintenance recommendation is planned maintenance.

[0090] Level III (Severe): Score 60-85, maintenance recommendation is to repair as soon as possible.

[0091] Level IV (Emergency): Score ≥ 85, maintenance recommendation is immediate action.

[0092] S3 calculates the geometric feature parameters of the empty region using the segmentation mask, and converts the pixel units of the geometric feature parameters into actual physical units.

[0093] Connectivity component analysis is performed on the binarized segmentation mask to group adjacent empty pixels in the mask into independent connected regions. Each empty region is assigned a unique identifier, and multiple geometric feature parameters are calculated for each independent empty region, including: the pixel area of ​​the empty region, i.e., the total number of pixels contained in the region; the pixel perimeter of the empty region, i.e., the pixel length of the region boundary; a boundary complexity index determined based on the relationship between perimeter and area, used to reflect the irregularity of the empty boundary; the centroid position of the empty region, used to determine the location of the empty region on the support; and the aspect ratio of the minimum bounding rectangle of the empty region, used to describe the shape characteristics of the empty region.

[0094] Geometric parameters in pixels are converted into actual physical units. The conversion method is based on imaging parameters from UAV aerial photography, including flight altitude, camera focal length, and camera sensor pixel size. The actual physical area is obtained by multiplying the pixel area by the square of the ground sampling distance, and the actual physical length is obtained by multiplying length parameters such as pixel perimeter and maximum width by the ground sampling distance. Boundary complexity, being dimensionless, does not require conversion. The position type is determined based on the relative distance between the free-floating area and the support center, also without requiring absolute dimensions.

[0095] It is important to note that the constructions and arrangements of this application shown in several different exemplary embodiments are merely illustrative. Although only two embodiments are described in detail in this disclosure, those who consult this disclosure will readily understand that many modifications are possible without substantially departing from the novel teachings and advantages of the subject matter described in this application. These modifications may include, for example, changes in the size, dimensions, structure, shape, and proportions of various elements, as well as parameter values ​​(e.g., temperature, pressure, etc.), installation arrangements, the use of materials, colors, orientations, etc. For example, an element shown as integrally formed may be composed of multiple parts or elements, the position of elements may be inverted or otherwise altered, and the nature or number or position of discrete elements may be changed or altered. Therefore, all such modifications are intended to be included within the scope of this application. The order or sequence of any process or method steps may be changed or rearranged according to alternative embodiments. Any "apparatus plus function" clause is intended to cover, and not only structurally equivalent but also equivalent structures, the structures performing the functions described herein. Other substitutions, modifications, alterations, and omissions may be made in the design, operation, and arrangement of the exemplary embodiments without departing from the scope of this application. Therefore, this application is not limited to a particular embodiment, but extends to various modifications that still fall within the scope of the appended claims.

[0096] Furthermore, in order to provide a concise description of exemplary embodiments, not all features of actual embodiments (i.e., those features that are not relevant to the best mode of performing this application as currently considered, or those features that are not relevant to implementing this application) may be omitted.

[0097] It should be understood that numerous specific implementation decisions can be made during the development of any practical implementation, such as in any engineering or design project. Such development efforts may be complex and time-consuming, but for those of ordinary skill in the art who benefit from this disclosure, the development effort will be routine work in design, manufacturing, and production without requiring much experimentation.

[0098] It should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application, and all such modifications and substitutions should be covered within the scope of the claims of this application.

Claims

1. A method for identifying bridge support voids based on UAV aerial image processing, characterized in that, include: S1 controls the UAV platform equipped with the imaging unit to acquire multi-view images of the bridge support, obtain the original image containing the support area, and perform illumination equalization and shadow removal processing on the original image in sequence to obtain the pre-processed image. S2 locates and extracts the region where the support is located in the preprocessed image to obtain a support region image. The support region image is then input into a pre-trained deep learning segmentation network to generate a binary segmentation mask that identifies the empty region. The deep learning segmentation network includes a feature extraction backbone network, a feature pyramid fusion module, and a dual attention mechanism module. S3 calculates the geometric feature parameters of the empty region using the segmentation mask, and converts the pixel units of the geometric feature parameters into actual physical units; S4 calculates a risk score based on the geometric feature parameters of the actual physical unit through an evaluation model, and determines the risk level according to the risk score. It then integrates the segmentation mask, geometric feature parameters, risk level, and corresponding maintenance suggestions to generate a detection report.

2. The method for identifying bridge support voiding based on UAV aerial image processing as described in claim 1, characterized in that: The shadow removal process is used to distinguish and remove shadow areas caused by the angle of illumination from real void areas caused by structural voids. The original image is converted from the RGB color space to the LAB color space, and the brightness channel of the converted image is subjected to contrast-limited adaptive histogram equalization. The processed image is converted to the YUV color space, and histogram equalization is performed on the luminance component of the YUV color space.

3. The method for identifying bridge support voids based on UAV aerial image processing as described in claim 2, characterized in that: The image after illumination equalization is converted from the RGB color space to the HSV color space; An adaptive brightness threshold is calculated using the brightness components of the transformed image, and pixel regions with brightness values ​​lower than the adaptive brightness threshold are marked as shadow candidate regions. Connectivity analysis is performed on the candidate shadow regions to obtain multiple connected components; Calculate the boundary complexity index for each of the connected components, the boundary complexity index being determined based on the ratio of the perimeter to the area of ​​the connected component; Based on the boundary complexity index and the average brightness of the connected component, the candidate shadow regions are filtered: connected components with boundary complexity lower than a preset complexity threshold and average brightness higher than a preset brightness threshold are identified as shadow regions formed by light occlusion and removed from the image. Connected regions with boundary complexity higher than the preset complexity threshold or average brightness lower than the preset brightness threshold are retained as suspected empty regions to obtain the preprocessed image.

4. The method for identifying bridge support voids based on UAV aerial image processing as described in claim 3, characterized in that: The adaptive brightness threshold is calculated based on the global mean and global standard deviation of the brightness component, with the global mean as the benchmark, minus the product of the global standard deviation and the adjustment coefficient. The range of values ​​for the adjustment coefficient is set according to the common lighting environment of bridge bearings, in order to adapt to the shadow detection needs under various lighting conditions.

5. The method for identifying bridge support voiding based on UAV aerial image processing as described in claim 1, characterized in that: The feature pyramid fusion module uses adaptive weighted fusion to fuse multi-scale features, including small gaps and large-area gaps that require different scale feature responses; The adaptive weighted fusion strategy includes: obtaining feature maps of multiple scales from the feature extraction backbone network; The feature maps are adjusted to a uniform number of channels and spatial size through convolution operations; Assign learnable initial weight coefficients to each of the adjusted feature maps; The initial weight coefficients are mapped to fusion coefficients using a normalization function; The fusion coefficients are used to perform a weighted summation of the feature maps to obtain a fused feature map, and the most effective feature scale for the current void region is marked.

6. The method for identifying bridge support voiding based on UAV aerial image processing as described in claim 1, characterized in that: The dual attention mechanism module includes a channel attention submodule, a spatial attention submodule, and a context-aware attention submodule connected in sequence, which are used to sequentially enhance the network’s feature response to the empty region and suppress shadow residue; The channel attention submodule performs global average pooling and global max pooling on the input feature map, inputs the pooling results into a shared multilayer perceptron to generate channel attention weights, and uses the channel attention weights to weight the input feature map to enhance channel features sensitive to empty regions. The spatial attention submodule performs average pooling and max pooling along the channel dimension on the channel-weighted feature map, concatenates the pooling results and performs convolution to generate spatial attention weights, and uses the spatial attention weights to weight the feature map to locate the spatial position of the empty region. The context-aware attention submodule applies multiple convolutional kernels of different sizes in parallel to the spatially weighted feature map to capture contextual information of different ranges. After concatenating and fusing the convolutional results, it performs residual connection with the feature map input to the context-aware attention submodule to obtain the enhanced feature map.

7. The method for identifying bridge support voiding based on UAV aerial image processing as described in claim 6, characterized in that: The context-aware attention submodule uses multiple convolutional kernels applied in parallel with increasing sizes to capture information from small-scale local textures to large-scale structural contexts. By combining convolutional kernels of different sizes in parallel, the consistency of local features presented by the shadow region in the multi-scale receptive field and the structural contrast features of the void region with the environmental background in the multi-scale receptive field are utilized.

8. The method for identifying bridge support voids based on UAV aerial image processing as described in claim 1, characterized in that: The deep learning segmentation network employs a boundary enhancement loss function during training. The boundary enhancement loss function is composed of a weighted combination of binary cross-entropy loss, Dice loss, and boundary loss.

9. The method for identifying bridge support voiding based on UAV aerial image processing as described in claim 1, characterized in that: The boundary loss method includes: extracting boundary pixels from the labeled real segmentation mask, setting a weight coefficient for the boundary pixels higher than that for non-boundary pixels, and calculating a weighted binary cross-entropy loss based on the weight coefficient as the boundary loss, so that the network pays more attention to the edge pixels of the empty region during training.

10. The method for identifying bridge support voiding based on UAV aerial image processing as described in claim 1, characterized in that: In the step of quantifying the gap feature, the ground sampling distance is calculated based on the calibration parameters of the imaging unit and the flight altitude of the UAV platform when acquiring images; using the ground sampling distance, the pixel area of ​​the gap region in the segmentation mask is converted into the actual physical area to quantify the severity of gap. In the risk level assessment step, the preset assessment model is a hierarchical analysis model, which includes geometric characteristic parameters of the actual physical area, support spatial location factors, structural stress influence parameters, and historical development trend factors to calculate the risk score. Based on the preset risk score range, the assessment results are divided into four progressively increasing risk levels, and maintenance recommendations are matched accordingly.

Images