Concrete bridge apparent disease instance segmentation method based on disease label co-occurrence

Abstract

The application discloses a concrete bridge apparent disease instance segmentation method based on disease label co-occurrence, and belongs to the technical field of computer vision and intelligent detection of civil engineering. Bridge surface images are collected to generate standardized input images; the images are sent into a backbone network, disease structure feature expression is strengthened through a CG-Former structure perception feature enhancement module, and multi-scale initial feature maps are output; an adaptive multi-scale feature fusion module AFFM is adopted for lightweight reconstruction and adaptive weighted fusion to obtain a fused feature map, a label co-occurrence matrix is constructed, a graph convolution network is modeled to represent semantic features of disease co-occurrence relationships, and a relationship enhancement channel attention module RECA is used to map semantic information to a channel space to complete feature enhancement; the enhanced features are input into a three-branch prediction head to synchronously output disease categories, position information and pixel-level segmentation masks. The application improves the multi-disease segmentation precision in a complex scene, is lightweight and easy to deploy, and is suitable for bridge automatic inspection and intelligent disease evaluation.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent inspection technology in civil engineering, specifically to a method for segmenting apparent defects in concrete bridges based on the co-occurrence of defect labels. Background Technology

[0002] Bridges are critical infrastructure in transportation systems, and their structural safety directly impacts public travel safety and the sustainable development of regional economies. Concrete bridges, in particular, are highly susceptible to surface defects such as cracks, spalling, exposed rebar, and water erosion during long-term service due to the combined effects of vehicle loads, temperature variations, freeze-thaw cycles, rainwater erosion, carbonation, and chloride ion penetration. These surface damages are often early signs of internal structural deterioration; failure to detect and repair them promptly can lead to reduced load-bearing capacity, decreased durability, and in severe cases, even structural instability or collapse. Therefore, establishing an efficient and reliable technology system for detecting and identifying surface defects in bridges is of significant engineering and social value in ensuring the safe operation of bridges.

[0003] Commonly used methods for bridge defect detection at engineering sites include manual inspection, traditional image processing methods, and intelligent detection methods based on deep learning. Manual inspection relies on inspectors visually observing or using tools such as magnifying glasses, steel rulers, and crack width meters to identify and record defects. While intuitive, this method suffers from high labor intensity, low efficiency, and strong subjectivity. When there are a large number of bridges or their structures are complex, manual inspection not only struggles to cover all areas but is also susceptible to factors such as ambient lighting and surface contamination, leading to missed or incorrect detections. Furthermore, some bridges are located in dangerous environments such as high altitudes, water surfaces, or mountainous areas, posing significant safety risks to manual inspection.

[0004] To improve the automation level of inspection, researchers have proposed semi-automatic detection methods based on traditional image processing, such as edge detection, threshold segmentation, grayscale projection, and morphological operations. However, these algorithms are extremely sensitive to noise, highly dependent on parameters, and difficult to adapt to various disease types in complex backgrounds. With the rapid development of computer vision and deep learning technologies, disease detection methods based on convolutional neural networks (CNNs) have gradually become mainstream. Through large-scale sample learning, these methods can automatically extract multi-layer features and achieve high-precision identification of diseases such as cracks and spalling. Typical examples include Mask R-CNN, the YOLO series, the Transformer structure, and its improved versions. They have achieved certain results in bridge surface disease detection tasks, but still have the following shortcomings:

[0005] The ability to extract multi-scale features is limited. Bridge surface defects vary significantly in morphology and size. Cracks are mostly thin and elongated lines, while spalling and exposed rebar appear as large, irregular blocks. Existing models struggle to simultaneously represent the features of both micro-cracks and large-scale defects within a unified framework, leading to blurred boundaries or missing regions in the segmentation results.

[0006] The co-occurrence relationships of structural defects are not fully utilized. In actual structures, different defects often exhibit strong correlations. For example, the propagation of cracks can cause concrete spalling, and the reinforcing steel in the spalled area is prone to corrosion. These semantic co-occurrence relationships contain valuable prior knowledge and can provide contextual constraints for the model. However, most existing methods rely solely on visual features for classification or segmentation, neglecting the semantic dependencies between different defects, making it difficult for the model to maintain recognition stability in complex co-occurrence scenarios.

[0007] Feature fusion methods are relatively simple. While deep features have strong semantic expressive power, their spatial resolution is low; while shallow features contain rich edge information, they lack contextual semantics. Traditional top-down pyramid structures (such as FPN and PANet) use static fusion strategies, which cannot adaptively adjust the weights of shallow and deep information according to disease features, resulting in loss of details or semantic deviation.

[0008] The real-time performance and deployability of the models still need improvement. Bridge inspections largely rely on drones or mobile robot platforms, which have limited computing resources. Some high-precision models have a large number of parameters and slow inference speeds, making it difficult to meet the needs of real-time on-site inspections.

[0009] Therefore, existing bridge surface defect detection technologies still have significant shortcomings in terms of accuracy, robustness, and real-time performance. There is an urgent need for a novel instance segmentation method that combines structural perception features with semantic co-occurrence modeling, which can achieve accurate identification of multiple defects and lightweight deployment in complex backgrounds. Summary of the Invention

[0010] Based on the aforementioned technical problems, this application discloses a method for segmenting apparent defects in concrete bridges based on the co-occurrence of defect tags, specifically including:

[0011] Acquire images of the concrete bridge surface, perform preprocessing and data augmentation, and generate standardized input images;

[0012] Standardized images are input into the backbone feature extraction network. During the feature extraction process, the CG-Former structure-aware feature enhancement module is embedded. The disease structure features are modeled through depthwise separable convolution and convolution-gated linear units, and multi-scale initial feature maps are output.

[0013] The adaptive multi-scale feature fusion module AFFM is used to perform lightweight reconstruction and learnable weighted fusion of shallow spatial features and deep semantic features to obtain a fused feature map.

[0014] Based on the statistical analysis of disease category co-occurrence frequency in the training dataset, a co-occurrence matrix based on disease label vectors is constructed and modeled using a graph convolutional network to obtain semantic features representing disease co-occurrence relationships.

[0015] The Relationship Enhancement Channel Attention Module (RECA) maps semantic features representing disease co-occurrence relationships to the channel space, and performs channel attention weighted enhancement on the fused feature map to obtain semantically enhanced features.

[0016] The semantic enhancement features are input into the three-branch instance segmentation prediction head, which outputs the disease category, location information, and pixel-level instance segmentation mask.

[0017] Preferably, the image preprocessing and data enhancement specifically include: performing random horizontal flipping, random cropping, scale transformation and brightness perturbation operations on the original image, while performing size normalization and pixel value standardization processing to eliminate interference caused by lighting and shooting angle, and generating a standardized image adapted to neural network input.

[0018] Preferably, the CG-Former structure-aware feature enhancement module first extracts spatial structure features through depthwise separable convolution, specifically calculated as follows:

[0019]

[0020] in, and These represent pointwise convolution operations, This indicates a depthwise separable convolution operation. It is a non-linear activation function that reduces computational complexity while maintaining spatial structure perception capabilities.

[0021] Preferably, the CG-Former module uses convolutional gated linear units for adaptive adjustment of channel features after spatial feature extraction, and the calculation formula is as follows:

[0022]

[0023] in, and The input features are divided into two sets of features along the channel dimension. For activation function, This is an element-wise multiplication operation. To achieve the output linear mapping operation, the disease-related channels are enhanced and background noise is suppressed.

[0024] Preferably, the adaptive multi-scale feature fusion module AFFM first performs channel reconstruction and supplementary feature generation on the shallow backbone features, and the calculation process satisfies:

[0025]

[0026]

[0027]

[0028] Then, the adaptive weighted fusion of shallow and deep features is achieved through learnable weight parameters.

[0029] Preferably, the adaptive multi-scale feature fusion module achieves dynamic weighted fusion through learnable weights, which are obtained by normalization using the Sigmoid function. The fusion formula is as follows:

[0030]

[0031] in , These are learnable parameters that enable the model to automatically adjust the fusion ratio of shallow details and deep semantics according to the disease scale.

[0032] Preferably, the disease label co-occurrence matrix is ​​constructed based on training set statistics, and the matrix elements are used to characterize the conditional co-occurrence probability among diseases, calculated as follows:

[0033]

[0034] in, Disease category With category The number of times they appear simultaneously in the same image. Disease category The total number of occurrences is used to uncover the true symbiotic relationships between diseases.

[0035] Preferably, the graph convolutional network propagates and encodes disease semantic relationships based on the co-occurrence matrix, and the feature propagation formula for the graph convolutional layer is:

[0036]

[0037] in, This is the normalized co-occurrence matrix. It is the identity matrix. For degree matrix, For learnable weight matrix, This is the activation function.

[0038] Preferably, the Relationship Enhancement Channel Attention Module (RECA) maps the co-occurrence semantic features of diseases to the image feature channel space. It first performs feature aggregation and weight mapping through global max pooling, calculated as follows:

[0039]

[0040]

[0041] in, This is a global max pooling operation. and For linear mapping operations, This is the Sigmoid activation function.

[0042] Preferably, the RECA module further models channel correlation through global average pooling and one-dimensional convolution, and concatenates the original features with semantically enhanced features for output. The final output features are:

[0043]

[0044] in, , This is a global average pooling operation, which allows the output features to retain both spatial details and semantic information related to disease coexistence.

[0045] Compared with the prior art, the technical solution of this application has the following technical effects:

[0046] This invention embeds the CG-Former structure-aware feature enhancement module into the backbone network, and uses depthwise separable convolution and convolution-gated linear units to perform structural modeling of image features. This significantly enhances the structural feature representation of typical defects such as cracks, spalling, and exposed rebar, improves the model's perception accuracy for slender and irregular defects, and maintains the efficiency and lightweight nature of the feature extraction process, providing stable and reliable basic features for subsequent multi-scale fusion and semantic enhancement.

[0047] This invention employs an adaptive multi-scale feature fusion module (AFFM), which achieves adaptive weighted fusion of shallow spatial features and deep semantic features through lightweight feature reconstruction and learnable weights. It can automatically adjust the feature ratio according to the scale characteristics of different diseases, so that the fused feature map retains rich spatial details and high-level semantic information at the same time, effectively improving the model's ability to uniformly express multi-scale and multi-morphological diseases, and enhancing the integrity and discriminability of features.

[0048] This invention constructs a co-occurrence matrix based on the co-occurrence relationship of disease labels and combines it with graph convolutional networks for semantic modeling. It can accurately discover the inherent dependencies and co-occurrence rules among diseases, transform domain prior knowledge into learnable semantic features, and inject disease co-occurrence semantic information into the image feature channel space through the relationship enhancement channel attention module, realizing deep coupling between semantic knowledge and visual features, and improving feature discrimination and recognition stability in multi-disease coexistence scenarios.

[0049] This invention achieves simultaneous reasoning of defect category, location, and mask through a three-branch instance segmentation prediction head. Combined with the synergistic effect of structural enhancement, multi-scale fusion, and semantic attention, it can output pixel-level instance segmentation results with clear boundaries and accurate positioning. At the same time, the whole method maintains lightweight and efficient reasoning characteristics, and can be adapted to edge deployment scenarios such as drones and inspection robots, providing stable and reliable technical support for the automated, refined, and real-time detection of surface defects in concrete bridges.

[0050] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0051] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0053] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0054] Figure 1 The flowchart shows a method for segmenting apparent defects in concrete bridges based on the co-occurrence of defect tags.

[0055] Figure 2 A diagram illustrating the overall structure of a bridge defect instance segmentation method that integrates label vector co-occurrence relationships.

[0056] Figure 3This is a functional structure diagram of the CG-Former module described in this invention;

[0057] Figure 4 This is a schematic diagram of the RECA attention module structure described in this invention;

[0058] Figure 5 This is a visualization comparison chart of the comparative experiments on the self-built dataset in this embodiment;

[0059] Figure 6 This is a visualization comparison chart of the comparative experiment on the s2ds public dataset in this embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0061] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0062] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0063] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0064] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0065] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0066] Example 1

[0067] This embodiment mainly describes a method for segmenting apparent defects in concrete bridges based on the co-occurrence of defect tags, such as... Figure 1 , Figure 2 As shown, it specifically includes:

[0068] Acquire images of the concrete bridge surface, perform preprocessing and data augmentation, and generate standardized input images;

[0069] Standardized images are input into the backbone feature extraction network. During the feature extraction process, the CG-Former structure-aware feature enhancement module is embedded. The disease structure features are modeled through depthwise separable convolution and convolution-gated linear units, and multi-scale initial feature maps are output.

[0070] The adaptive multi-scale feature fusion module AFFM is used to perform lightweight reconstruction and learnable weighted fusion of shallow spatial features and deep semantic features to obtain a fused feature map.

[0071] Based on the statistical analysis of disease category co-occurrence frequency in the training dataset, a co-occurrence matrix based on disease label vectors is constructed and modeled using a graph convolutional network to obtain semantic features representing disease co-occurrence relationships.

[0072] The Relationship Enhancement Channel Attention Module (RECA) maps semantic features representing disease co-occurrence relationships to the channel space, and performs channel attention weighted enhancement on the fused feature map to obtain semantically enhanced features.

[0073] The semantic enhancement features are input into the three-branch instance segmentation prediction head, which outputs the disease category, location information, and pixel-level instance segmentation mask.

[0074] Furthermore, raw image data of the concrete bridge surface is acquired. These images can be obtained through drone inspection platforms, mobile inspection robots, or manual imaging equipment. During the acquisition process, high-resolution industrial cameras or RGB cameras can be used to photograph key structural components such as the bridge beams, bridge decks, and piers, thereby obtaining raw image data containing information on bridge surface defects.

[0075] To improve the robustness of the model under complex environmental conditions, the original images are preprocessed before being input into the neural network. This preprocessing mainly includes data augmentation operations, such as random horizontal flipping, random cropping, scaling, and brightness perturbation, to expand the diversity of training samples and thus enhance the model's adaptability to different shooting angles, lighting conditions, and local occlusion.

[0076] After the above data augmentation process, bridge appearance image data that meets the input requirements of the neural network is obtained and then input into the subsequent feature extraction network for processing to extract multi-scale feature information of bridge appearance defects.

[0077] Furthermore, a feature extraction network structure is constructed. After completing the image input and preprocessing steps, the preprocessed input image is fed into the backbone feature extraction network to extract multi-scale feature representations of apparent defects in concrete bridges. Let the input image be... The backbone feature extraction network encodes the input image through multi-layer convolution operations and progressive downsampling operations, forming multiple feature maps with different spatial resolutions to characterize the structural and semantic information of bridge surface defects at different scales.

[0078] like Figure 2 As shown, to enhance the network's ability to perceive diseased areas under conditions such as slender crack structures, irregular spalling boundaries, and complex backgrounds, this invention introduces a structure-aware feature enhancement mechanism during the backbone feature extraction process. This mechanism combines depthwise separable convolution with convolutionally gated linear units to jointly model the spatial structural information and channel semantic relationships of the features.

[0079] In the specific implementation, the input features are first extracted spatially using a depthwise separable convolutional structure, and the calculation process is as follows: ,in, and These represent pointwise convolution operations, This represents a depthwise separable convolution operation, σ This represents a nonlinear activation function. This calculation method effectively reduces computational complexity and improves the efficiency of the feature extraction process while maintaining the ability to perceive local spatial structure.

[0080] After completing the spatial feature modeling, to further enhance the nonlinear expressive power in the channel dimension, this invention employs convolutional gated linear units to perform channel modeling of the features. Specifically, the input features are divided into two parts in the channel dimension. One part of the features undergoes depthwise convolution and nonlinear transformation, and is then gated and fused with the other part of the features through element-wise multiplication. The calculation process is expressed as follows:

[0081]

[0082] in, and This represents two sets of features obtained by partitioning the input features along the channel dimension. This represents the activation function. This represents element-wise multiplication. This indicates the output mapping operation. This gating mechanism adaptively adjusts the importance of channel features based on the local spatial structure response, thereby enhancing the feature responses related to the diseased area and suppressing background noise.

[0083] To ensure the stability of the feature transfer process and avoid information degradation, the structure-aware feature enhancement process employs a residual connection approach, fusing the input features with the structure-enhanced features described above. This enhances the structure-awareness capability while preserving the original feature information. Through the aforementioned backbone feature extraction and structure-aware feature enhancement steps, a set of multi-scale feature maps that combine local spatial structural information and channel semantic discrimination capabilities is obtained. This provides high-quality input features for subsequent adaptive multi-scale feature fusion steps and disease label co-occurrence relationship modeling steps.

[0084] Furthermore, such as Figure 3 As shown, adaptive multi-scale feature fusion is performed. After completing the main feature extraction and structure-aware feature enhancement steps, feature mappings at different spatial resolutions are obtained. In order to achieve effective fusion of disease features at different scales, this invention performs adaptive multi-scale fusion processing on features from shallow and deep layers.

[0085] In the specific implementation, let the shallow features from the backbone network be... First, channel adjustment and feature reconstruction are performed on it. The calculation process is as follows:

[0086]

[0087] Then, supplementary features are generated:

[0088]

[0089] Then, the features are concatenated along the channel dimension to obtain the reconstructed shallow features:

[0090]

[0091] After completing the shallow feature reconstruction, it is compared with the deep features at the corresponding scale. The fusion process involves introducing learnable weight parameters λ, which are mapped to normalized weights α using the Sigmoid function. The fusion result is expressed as: Through the above adaptive multi-scale feature fusion steps, fused features that take into account both spatial detail information and high-level semantic information are obtained, providing input features for subsequent disease label co-occurrence relationship modeling and attention enhancement steps.

[0092] Furthermore, such as Figure 4 As shown, the co-occurrence relationship of disease labels is modeled and relationship-enhanced channel attention is applied. After completing the adaptive multi-scale feature fusion step, the fused feature F is obtained. Considering that the apparent diseases of concrete bridges often exhibit multi-category coexistence characteristics in actual scenarios, this invention proposes a RECA module. It extracts the co-occurrence relationship contained in the co-occurrence matrix based on the disease label vector through a graph convolutional network, and introduces the semantic prior information into the feature expression process through relationship-enhanced channel attention to improve the discrimination ability in multi-disease coexistence scenarios.

[0093] In the specific implementation, firstly, based on the annotation information in the training samples, the co-occurrence of different disease categories is statistically analyzed, and a disease label co-occurrence matrix is ​​constructed; let the set of disease categories be... The elements of the co-occurrence matrix are represented as:

[0094]

[0095] in, Indicates disease category and The number of times they appear simultaneously in the same image Indicates disease category The total number of occurrences. The resulting matrix M is used to describe the conditional co-occurrence relationships between disease categories.

[0096] Based on the co-occurrence matrix, a co-occurrence relationship graph of disease labels is constructed, and graph convolution is used to model the semantic relationships of diseases. Let the label embedding features be... The feature propagation process of the graph convolutional layer can be represented as:

[0097]

[0098] in, For learnable weight matrix, For activation function, The normalized co-occurrence matrix is ​​calculated as follows:

[0099]

[0100] Where I is the identity matrix and D is the degree matrix. Through multi-layer graph convolution operations, semantic features reflecting the co-occurrence relationships of diseases are obtained.

[0101] To incorporate the semantic features representing the co-occurrence relationship of diseases into the image feature channel space, a global max pooling operation is performed on the fused feature F, and channel weight mapping is performed in conjunction with the graph convolution output. The calculation process is as follows:

[0102]

[0103] in, This represents the global max pooling operation. Subsequently, channel attention weights are generated through a linear transformation and the sigmoid activation function, and the features are then weighted accordingly.

[0104]

[0105] in, and This represents a linear mapping operation. This represents the Sigmoid activation function.

[0106] Building upon this, to further model the local correlations between channels, global average pooling is performed on the weighted features, combined with one-dimensional convolution. The result is expressed as follows:

[0107]

[0108] in, This indicates a global average pooling operation.

[0109] Finally, to integrate the original features with the semantically enhanced feature representations, the two are channel-adjusted separately and then concatenated along the channel dimension to obtain the final output features:

[0110]

[0111] By modeling disease co-occurrence relationships and enhancing channel attention steps as described above, feature representations can incorporate disease semantic dependencies while maintaining the original spatial information, thereby improving the robustness of instance segmentation under conditions of coexistence of multiple disease categories.

[0112] This detailed implementation describes how, through steps such as image acquisition and preprocessing, feature extraction network structure construction, adaptive multi-scale feature fusion, disease co-occurrence relationship modeling, and relationship enhancement channel attention processing, bridge apparent disease instance segmentation results with high accuracy and good robustness are obtained.

[0113] Based on Example 1, this example details the training and evaluation of all models using a self-built dataset containing 1640 images of four types of bridge defects: cracks, exposed reinforcement, spalling, and water erosion. Approximately 46.21% of the data shows coexistence of multiple defects, reflecting the real-world bridge conditions. All models were trained and evaluated under a unified data partitioning and training configuration. The dataset was partitioned according to a training set:validation set ratio of 8:2. The input image size was uniformly set to 640×640 pixels, the training epochs were set to 300, the batch size to 8, and the number of data loading threads to 4. The model training employed the SGD optimizer with an initial learning rate of 0.01, a momentum parameter of 0.937, and a weight decay coefficient of 0.0005. Three warmup epochs were used to enhance stability in the early stages of training. These parameter configurations were maintained consistent throughout all comparative and ablation experiments in this chapter, and inference latency testing was performed on the same hardware.

[0114] To fully verify the generalization ability and stability of the proposed method under different data scenarios, this paper conducted comparative experiments on a self-built concrete bridge apparent defect dataset and the S2DS public dataset, and compared it with a variety of mainstream instance segmentation models such as Mask R-CNN, Cascade Mask R-CNN, SOLO, Swin Transformer, Mask2Former, Mask DINO, VMamba, and YOLOv8–YOLOv12 in the YOLO series.

[0115] Table 1. Comparative experimental results on the self-built dataset.

[0116] As shown in Table 1, on self-built datasets, traditional two-stage methods (Mask R-CNN, Cascade Mask R-CNN) exhibit relatively stable overall accuracy, but their large parameter and computational requirements result in significantly slower inference speeds, making them unsuitable for real-time detection. Transformer-type models (Swin Transformer, Mask DINO, etc.) demonstrate certain advantages in the mAP50-95 metric, but their generally large model size, high computational complexity, and high deployment costs contribute to their performance. While VMamba introduces a state-space modeling mechanism, its performance on this dataset is relatively weak, indicating that its structural advantages are not fully adapted to bridge surface defects. Lightweight single-stage models (YOLOv8–YOLOv12) show significant advantages in inference speed, with relatively low parameter counts and GFLOPs, but they have certain upper limits in Box and Mask accuracy. For example, YOLOv11 achieves 37.5% Box mAP50 and 35.7% Mask mAP50, still lagging behind two-stage methods. In comparison, our proposed method achieves 39.3% and 38.2% accuracy on Box mAP50 and MassmAP50, respectively, and also achieves the highest or near-highest results on the mAP50-95 metric. Meanwhile, with 9.7M model parameters and 10.9 GFLOPs, it maintains a good level of computational complexity control, and its inference speed is close to that of lightweight models. The results demonstrate that the proposed structural enhancement, multi-scale fusion, and semantic relationship modeling mechanisms can effectively improve instance segmentation accuracy and exhibit stronger discriminative and structural representation capabilities in complex bridge defect scenarios.

[0117] Table 2. Comparative experimental results on the s2ds dataset.

[0118] To verify the generalization ability of the model, we conducted comparative experiments on the publicly available s2ds dataset. This dataset contains six categories: cracks, spalling, corrosion, calcification, vegetation, and control points. However, as shown in Table 2, the performance gap between different methods remains significant. Mask DINO performs well on mAP50-95, especially at high IoU thresholds, but its inference time is relatively long. Lightweight models (YOLOv8–YOLOv12) have achieved high accuracy on Box mAP50, for example, YOLOv8 reaches 48.8% and YOLOv11 reaches 48.0%, but there is still room for improvement on the Mask mAP50-95 metric. Our proposed method achieves 50.3% Box mAP50 and 51.2% Mask mAP50 on this dataset, both the highest results among all compared methods; it also maintains a leading position on the mAP50-95 metric. Although the inference time is slightly increased compared to the ultra-lightweight model, it remains within an acceptable range. Experimental results show that the proposed method is not only effective on a single dataset, but also has good generalization ability in cross-data scenarios, and can steadily improve the boundary accuracy and overall consistency of instance segmentation.

[0119] To evaluate the model's performance in scenarios with sparse targets and co-occurring defects, we focused on two types of typical test images for visualization analysis: one type consisted of single-defect samples with extremely low pixel percentages (mainly long cracks), and the other type contained samples with two to three co-occurring defects. The model exhibited lower false positives and false negatives in multi-defect scenarios. This phenomenon further demonstrates that our proposed co-occurrence sensing mechanism is highly effective in complex detection scenarios involving multiple coexisting defects, while the performance improvement is relatively insignificant in images where a single defect type dominates.

[0120] Figure 5The presentation showcases some visual comparison results on a self-built dataset. As seen in row 1, in slender crack samples with a low pixel ratio, some traditional instance segmentation methods tend to exhibit discontinuous crack segmentation or blurred boundaries, even leading to local missed detections. The proposed method, however, better preserves the connectivity of the crack structure, making the segmentation results spatially closer to the actual annotations. This is mainly due to the enhanced representation of slender structural features by the structure-aware feature extraction module, enabling the model to more stably capture the spatial distribution of fine-scale defects. In complex scenarios with multiple defects coexisting, different types of defects often appear simultaneously in the same image region, as shown in rows 2, 3, 4, 5, and 6. Some comparison methods are prone to category confusion, false detections, or missed detections in such scenarios. The proposed method, however, can accurately distinguish between different types of defects and maintain clear segmentation boundaries between defect regions. This indicates that by introducing a label vector co-occurrence relationship modeling mechanism, the model can capture the semantic associations between different defects during feature learning, thereby achieving more stable instance segmentation in scenarios with multiple defects coexisting.

[0121] Figure 6 The visualization comparison results on the S2DS public dataset are further presented. As can be seen from the visualization results in rows 1, 3, and 7, under complex background conditions, some comparison methods easily misidentify background textures as defect regions or exhibit oversegmentation at defect boundaries. However, the proposed method can still accurately locate the true defect region in complex environments while maintaining good boundary integrity and class discrimination ability. This demonstrates that the proposed semantic co-occurrence relationship enhancement mechanism can effectively improve the model's feature representation ability in complex scenes, thereby enhancing the model's robustness and generalization ability in practical bridge detection tasks.

[0122] Through multiple sets of experiments, this invention has verified that the proposed method can effectively improve the accuracy and robustness of bridge defect segmentation. It can maintain clear and complete segmentation boundaries in both single defect and multiple defect co-occurrence scenarios, significantly reducing missed detections and false detections. At the same time, it maintains lightweight design and fast inference speed, and has good engineering applicability and generalization ability.

[0123] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A method for segmenting apparent defects in concrete bridges based on the co-occurrence of defect tags, characterized in that, include: Acquire images of the concrete bridge surface, perform preprocessing and data augmentation, and generate standardized input images; Standardized images are input into the backbone feature extraction network. During the feature extraction process, the CG-Former structure-aware feature enhancement module is embedded. The disease structure features are modeled through depthwise separable convolution and convolution-gated linear units, and multi-scale initial feature maps are output. The adaptive multi-scale feature fusion module AFFM is used to perform lightweight reconstruction and learnable weighted fusion of shallow spatial features and deep semantic features to obtain a fused feature map. Based on the statistical analysis of disease category co-occurrence frequency in the training dataset, a co-occurrence matrix based on disease label vectors is constructed and modeled using a graph convolutional network to obtain semantic features representing disease co-occurrence relationships. The Relationship Enhancement Channel Attention Module (RECA) maps semantic features representing disease co-occurrence relationships to the channel space, and performs channel attention weighted enhancement on the fused feature map to obtain semantically enhanced features. The semantic enhancement features are input into the three-branch instance segmentation prediction head, which outputs the disease category, location information, and pixel-level instance segmentation mask.

2. The method according to claim 1, characterized in that, The image preprocessing and data enhancement specifically include: performing random horizontal flipping, random cropping, scaling and brightness perturbation operations on the original image, while performing size normalization and pixel value standardization to remove interference from lighting and shooting angle, and generating a standardized image adapted to neural network input.

3. The method according to claim 1, characterized in that, The CG-Former structure-aware feature enhancement module first extracts spatial structure features through depthwise separable convolution, with the specific calculation formula as follows: in, and These represent pointwise convolution operations, This indicates a depthwise separable convolution operation. It is a non-linear activation function that reduces computational complexity while maintaining spatial structure perception capabilities.

4. The method according to claim 3, characterized in that, The CG-Former module uses convolutional gated linear units to adaptively adjust channel features after spatial feature extraction. The calculation formula is as follows: in, and The input features are divided into two sets of features along the channel dimension. For activation function, This is an element-wise multiplication operation. To achieve the output linear mapping operation, the disease-related channels are enhanced and background noise is suppressed.

5. The method according to claim 1, characterized in that, The adaptive multi-scale feature fusion module AFFM first performs channel reconstruction and supplementary feature generation on the shallow backbone features. The calculation process satisfies the following: Then, the adaptive weighted fusion of shallow and deep features is achieved through learnable weight parameters.

6. The method according to claim 5, characterized in that, The adaptive multi-scale feature fusion module achieves dynamic weighted fusion through learnable weights, which are obtained by normalization using the Sigmoid function. The fusion formula is as follows: in , These are learnable parameters that enable the model to automatically adjust the fusion ratio of shallow details and deep semantics according to the disease scale.

7. The method according to claim 1, characterized in that, The disease label co-occurrence matrix is ​​constructed based on the statistics of the training set. The matrix elements are used to characterize the conditional co-occurrence probability among diseases, and the calculation formula is: in, Disease category With category The number of times they appear simultaneously in the same image. Disease category The total number of occurrences is used to uncover the true symbiotic relationships between diseases.

8. The method according to claim 7, characterized in that, The graph convolutional network propagates and encodes disease semantic relationships based on the co-occurrence matrix. The feature propagation formula for the graph convolutional layer is as follows: in, This is the normalized co-occurrence matrix. It is the identity matrix. For degree matrix, For learnable weight matrix, This is the activation function.

9. The method according to claim 1, characterized in that, The Relationship Enhancement Channel Attention Module (RECA) maps the semantic features of disease co-occurrence to the image feature channel space. It first performs feature aggregation and weight mapping through global max pooling, calculated as follows: in, This is a global max pooling operation. and For linear mapping operations, This is the Sigmoid activation function.

10. The method according to claim 9, characterized in that, The RECA module further models channel correlation through global average pooling and one-dimensional convolution, and concatenates the original features with semantically enhanced features for output. The final output features are: in, , This is a global average pooling operation, which allows the output features to retain both spatial details and semantic information related to disease coexistence.

Images