An unmanned aerial vehicle bridge crack detection method based on structure perception cross-modal fusion, electronic equipment, computer readable storage medium and program

Abstract

This invention discloses a method, electronic device, computer-readable storage medium, and program for detecting bridge cracks using unmanned aerial vehicles (UAVs) based on structure-aware cross-modal fusion. The method includes acquiring visible light and infrared images of the bridge surface to form spatially and temporally aligned multimodal data pairs; extracting features to obtain an enhanced multimodal feature map; calculating illumination perception weights, structural consistency weights, and difference compensation weights for fusion to obtain fused features; performing bidirectional long-range dependency modeling on the fused features converted to a temporal sequence representation based on a state-updated sequence modeling approach to reconstruct global modeling features; then performing hierarchical refinement and weighted aggregation to obtain multi-scale aggregated features; finally, completing crack localization, structural segmentation, centerline extraction, and width calculation, and outputting the results. This invention can reduce crack feature fragmentation, improve the detection rate of small cracks, improve robustness under complex illumination, adapt to changes in UAV perspective, and exhibit good detection stability.

Description

Technical Field

[0001] This invention relates to the field of measurement technology, and in particular to a method for detecting cracks in bridges using unmanned aerial vehicles (UAVs) based on structure-aware cross-modal fusion, as well as electronic equipment, computer-readable storage media, and programs. Background Technology

[0002] Current bridge crack detection methods primarily rely on manual inspections or automated detection using single visible light images. However, in practical drone inspections, cracks are small in scale and elongated in shape, making them susceptible to interference from complex background noise; changes in viewing angle lead to geometrical instability; single-modal analysis exhibits poor robustness under complex lighting conditions; and conventional multimodal fusion lacks structural constraints, making it difficult to maintain the topological continuity of cracks. Therefore, existing methods cannot meet the demands for high-precision, stable, and continuous automated detection. Summary of the Invention

[0003] To address the shortcomings of the existing technologies, this invention provides a UAV bridge crack detection method based on structure-aware cross-modal fusion, solving problems such as illumination and viewing angle interference, insufficient robustness of single-modal methods, lack of structural constraints in multi-modal fusion, easy crack breakage, and difficulty in balancing accuracy and real-time performance. This invention also provides electronic equipment, computer-readable storage media, and computer program products for implementing this UAV bridge crack detection method.

[0004] The technical solution of the present invention is as follows:

[0005] A method for detecting cracks in bridges using unmanned aerial vehicles (UAVs) based on structure-aware cross-modal fusion includes:

[0006] Step 1: Acquire visible light and infrared images of the bridge surface to form spatially and temporally aligned multimodal data pairs;

[0007] Step 2: While maintaining spatial correspondence, extract feature representations that can characterize crack texture, edge and structural information from the visible light image and infrared image respectively, to obtain spatially strictly aligned enhanced multimodal feature maps;

[0008] Step 3: Generate illumination perception weights based on the response intensity of visible light and infrared features, calculate structural consistency weights based on the edge and direction information of the features, and construct difference compensation weights by combining the differences between modal features. The weighted weights calculated by illumination perception weights, structural consistency weights, and difference compensation weights are used to adaptively weight and fuse the enhanced multimodal features to obtain the fused features.

[0009] Step 4: The state-update-based sequence modeling method performs bidirectional long-range dependency modeling on the fusion features converted into time-series sequence representations, and reconstructs the global modeling features.

[0010] Step 5: First, perform structure-preserving refinement on the global modeling features at different levels, and then perform adaptive weighted aggregation to obtain multi-scale aggregated features that enhance the expression of crack structure.

[0011] Step 6: Based on multi-scale aggregation features, complete crack localization, structural segmentation, centerline extraction and width calculation, and finally output the detection results.

[0012] Furthermore, step 2 specifically includes: firstly, using a feature encoding network to perform feature mapping on the visible light image and the infrared image respectively to obtain initial features; then, using a multi-layer feature encoding unit to encode the initial features layer by layer to obtain multi-level feature representations; and finally, using linear mapping to perform a unified dimensional transformation on the features at each layer and at each scale to achieve scale normalization and channel alignment.

[0013] Furthermore, edge enhancement and high-frequency information enhancement operations are performed on the scale-normalized and channel-aligned features to obtain an enhanced multimodal feature map.

[0014] Furthermore, the illumination sensing weights are:

[0015] , , ;

[0016] The structural consistency weight is:

[0017] , , , ;

[0018] The difference compensation weight is:

[0019] , ;

[0020] in: For the Sigmoid function, These are preset parameters. After global average pooling, This is a visible light-enhanced multimodal feature map. This is an infrared-enhanced multimodal feature map. For convolution mapping, It is a constant. Edge extraction operator.

[0021] Furthermore, the weighted weights calculated from the illumination perception weight, structural consistency weight, and difference compensation weight are the visible light mode weight and the infrared mode weight. The visible light mode weight is the weighted sum of the illumination perception weight, structural consistency weight, and difference compensation weight, and the infrared mode weight is 1 minus the visible light mode weight.

[0022] Furthermore, in step 4, after reconstructing the long-range dependency model, the fused features and reconstructed features are enhanced by residual connection to obtain global modeling features.

[0023] Furthermore, step 5 specifically includes:

[0024] By performing structure-preserving upsampling on deep features, we obtain:

[0025]

[0026] in, It is an adaptive reconstruction kernel based on local content prediction. For local receptive fields The coordinates of the internal input feature pixels, For height direction index, Indexed in the width direction. This represents the deep features of the global modeling features obtained in step 4;

[0027] By performing orientation-aligned downsampling on the shallow features, we obtain:

[0028]

[0029] in, The reference coordinates are the center pixels of the downsampled target. For sampling neighborhood The relative offset coordinates within, For learnable offsets, For modulation weights, For sampling neighborhood, For sampling weights;

[0030] The identity mapping of the middle-layer features yields:

[0031]

[0032] in, It is an identity mapping;

[0033] Adaptive weighted fusion yields:

[0034]

[0035] in, , , For learnable weights, It is a constant;

[0036] Finally, perform multi-level recursive aggregation:

[0037]

[0038] in, For feature scale index, This represents the convolution operation. This indicates upsampling.

[0039] Another technical solution of the present invention is: an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor;

[0040] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to execute the aforementioned method for detecting cracks in UAV bridges based on structure-aware cross-modal fusion.

[0041] Another technical solution of the present invention is: a computer-readable storage medium storing a computer program, the computer program being used to enable a processor to execute the aforementioned method for detecting cracks in UAV bridges based on structure-aware cross-modal fusion.

[0042] Another technical solution of the present invention is: a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the aforementioned method for detecting cracks in UAV bridges based on structure-aware cross-modal fusion.

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] This invention complements visible light and infrared dual modes to improve robustness under complex lighting conditions; structure-aware weighted fusion preserves the continuous structure of cracks and reduces fractures; bidirectional long-range modeling enhances global continuity and adapts to changes in the UAV's perspective; and multi-scale adaptive aggregation takes into account both details and semantics to improve the detection rate of small cracks. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the UAV bridge crack detection method based on structure-aware cross-modal fusion, as an example. Detailed Implementation

[0046] The present invention will be further described below with reference to embodiments, but these are not intended to limit the scope of the invention.

[0047] Please combine Figure 1 As shown in the figure, the UAV bridge crack detection method based on structure-aware cross-modal fusion in this embodiment is as follows:

[0048] 1. Acquire visible light and infrared images of the bridge surface to form spatially and temporally aligned multimodal data pairs.

[0049] Visible light and infrared images of the bridge surface are acquired using visible light and infrared sensors mounted on an unmanned aerial vehicle (UAV) platform. The visible light and infrared images are time-aligned based on a hardware synchronization trigger signal. The images are spatially registered based on camera parameters to obtain spatially corresponding multimodal aligned image data. Resolution unification and contrast enhancement processing are then performed, and the output is a multi-scale image pair that is spatially and temporally aligned.

[0050] Specifically, it includes:

[0051] 1) Sensor Data Acquisition Steps. Based on the bridge's structural characteristics, refined flight path planning was conducted for key defect areas such as piers, the bottom of the bridge deck, and the sides of the beams. The UAV was controlled to fly stably at an altitude of 2m–10m and a speed of 0.5m / s–3m / s to ensure image clarity and overlap. The following operations were performed on the UAV platform: visible light images of the bridge surface were acquired using a visible light camera; infrared images of the corresponding areas were acquired using an infrared thermal imaging camera; and strict temporal synchronization of the two modal images was ensured through hardware synchronization trigger signals or software timestamp alignment. Simultaneously, the UAV flight control system recorded timestamps, GNSS positioning information, and IMU attitude information in real time, and saved camera intrinsic parameters K, extrinsic parameters R, T, etc., providing a basis for subsequent spatial registration.

[0052] Obtain the original visible light image: Raw infrared image: timestamp Camera parameters (internal parameters) External reference ), UAV pose information (GNSS / IMU).

[0053] 2) Spatial registration step. Map images of different modalities to a unified coordinate system, based on camera intrinsic and extrinsic parameters (intrinsic parameters). External reference Geometric correction is performed to obtain pixel-level aligned multimodal image pairs.

[0054] 3) Data preprocessing steps. This involves unifying the resolution and enhancing the contrast of images from different modalities.

[0055] The output is a spatially and temporally aligned multimodal data pair. , , It can further expand the output to include multi-scale image pairs: .

[0056] in For the first Visible light aligned images at various scales; For the first Infrared aligned images at several scales; S is the number of scales. Indicates scale index.

[0057] 2. While maintaining spatial correspondence, feature representations that can characterize crack texture, edge and structural information are extracted from visible light images and infrared images respectively, resulting in spatially strictly aligned enhanced multimodal feature maps, which provide a foundation for subsequent cross-modal fusion.

[0058] The specific steps include:

[0059] 1) Multimodal Independent Encoding Step. To avoid feature interference between visible light and infrared modes and to ensure the integrity and expressive power of each modality's information, independent and parallel feature encoding networks are used to extract features from the two types of images. The feature encoding network is used to learn shallow texture, mid-level structure, and deep semantic information from the original image, such as convolutional neural networks, visual Transformers, or state-space models.

[0060] Initial feature mapping is performed on the aligned visible light image and infrared image respectively using a feature encoding network to obtain the initial features of the visible light image. Compared with initial infrared features The two initial features maintain a pixel-level spatial correspondence.

[0061] 2) Hierarchical Feature Extraction Steps. Based on the initial features, a progressive feature extraction process is performed from shallow to deep using multi-layer feature encoding units, gradually strengthening semantic information while preserving detailed structure. Encoding operations are performed layer by layer on the initial features to form multi-layered feature representations:

[0062]

[0063] in, For the first Layer feature encoding unit. Used to improve the abstraction level and semantic expressive power of features; For feature-level indexing, It serves as a scale index; through progressive encoding, each layer of features contains crack structure information corresponding to the level of abstraction.

[0064] 3) Scale normalization and channel alignment steps. Since features at different levels and scales differ in channel dimensions and numerical distribution, this directly affects the stability and detection accuracy of subsequent cross-modal fusion. Therefore, a linear mapping is used to perform a uniform dimensionality transformation on features at each level and scale to achieve scale normalization and channel alignment.

[0065]

[0066] in, For the first The linear transformation matrix corresponding to the layer is used to map features from different sources to the same feature space; For the first Layer, First Normalized features after linear mapping at each scale can ensure consistent feature distribution and stable weight calculation during subsequent fusion processes.

[0067] 4) Bridge cracks are characterized by being slender, low-contrast, having prominent edges, and having concentrated high-frequency information. To improve the model's sensitivity to crack targets and suppress background noise and redundant information, edge enhancement and high-frequency information enhancement operations are introduced based on channel alignment features to highlight the linear structure and edge contours of the cracks. The high-frequency information enhancement includes at least one or a combination of gradient operator enhancement, Laplacian enhancement, high-pass filtering, frequency domain enhancement, and fuzzy difference-based sharpening operations.

[0068] After enhancement processing, the final crack-sensitive features are output: ,in, To enhance the visible light characteristics, Both enhanced infrared features possess stronger crack identification and structural expressiveness.

[0069] 3. Light perception weights are generated based on the response intensity of visible light and infrared features. Structural consistency weights are calculated based on the edge and direction information of the features. Difference compensation weights are constructed by combining the differences between modal features. Weighted weights are calculated from the light perception weights, structural consistency weights, and difference compensation weights to adaptively weight and fuse the enhanced multimodal features to obtain fused features.

[0070] To fully utilize the complementary advantages of visible and infrared modes while suppressing complex illumination interference and maintaining the continuity and consistency of the crack structure, this step adaptively weights and fuses the enhanced multimodal features, ultimately obtaining fused features that balance illumination robustness and structural continuity. This fusion strategy prioritizes retaining the fused features in regions with consistent structural responses across both modes, significantly improving the stability of crack detection.

[0071] The specific steps include:

[0072] 1) Light Sensing Weight Generation Steps. In UAV inspection scenarios, uneven lighting intensity, shadows, and reflections can cause single-modal features to fail. To dynamically select more reliable modal features under the current lighting conditions, the global response intensity of the enhanced visible light and infrared features is first estimated. The overall activation intensity of the visible light and infrared features is extracted separately through global average pooling.

[0073]

[0074] in, This is the visible light response value; is the infrared illumination response value; GAP represents the global average pooling operation, used to characterize the overall effective response of a feature under the corresponding mode.

[0075] Calculate illumination perception weights based on the response differences between the two modes:

[0076]

[0077] in: This is the Sigmoid function, used to normalize the weights to the (0,1) interval; These are preset parameters used to adjust the weight sensitivity. This illumination weight can adaptively enhance the contribution of high-reliability modes and suppress the influence of modes affected by illumination interference, based on real-time illumination conditions.

[0078] 2) Structural Consistency Analysis Steps. Cracks possess stable linear geometry and edge orientation. A true crack should exhibit a consistent structural response in both visible and infrared modes, while noise and interference typically do not show consistency across these two modes. To preserve the continuous and complete crack structure and filter artifact interference, structural feature extraction and consistency measurement are performed on the enhanced features.

[0079] Edge extraction operator Structural information of visible light and infrared features were extracted separately:

[0080]

[0081] in, , These are structural features, respectively, visible light and infrared characteristics, used to characterize crack edges, orientation, and continuity.

[0082] Calculate intermodal structural consistency based on structural features:

[0083]

[0084] in, Characterizes the degree of consistency between the structural orientation and edge response of two modes at the same spatial location; To prevent constants with a denominator of zero.

[0085] The consistency results are then processed through convolutional mapping and normalization to generate structural consistency weights. :

[0086]

[0087] in, This is a convolutional mapping used to enhance the correlation of local structures; This is a normalization function. The larger the structural consistency weight, the more consistent the structure is in both modes at that location, and the more likely it is to be a real crack.

[0088] 3) Difference Compensation Weighting Step (Enhancing Robustness). To further improve fusion robustness, compensate for information differences between modalities, and avoid detection failure due to missing information in a single modality, calculate the difference features between visible light and infrared features. for:

[0089]

[0090] Global pooling and Softmax normalization are performed on the differential features to obtain the differential compensation weights. for:

[0091]

[0092] By combining illumination weights, structural consistency weights, and difference compensation weights, visible light mode weights are generated respectively. With infrared mode weights :

[0093]

[0094]

[0095]

[0096] in, ; ; The structural consistency weight is relatively high to enhance the ability to express structural information at the crack edge.

[0097] The two modal features are then fused element-wise to obtain the final fused feature. :

[0098]

[0099] in: Element-by-element multiplication. Through the above fusion, both the structurally consistent real crack regions are preserved and modal differences are compensated, giving the fused features stronger illumination robustness and structural integrity.

[0100] 4. The state-update-based sequence modeling method performs bidirectional long-range dependency modeling on the fusion features converted into time-series sequence representations to obtain global modeling features.

[0101] The sequence modeling approach used in this step is designed to capture long-range dependencies while keeping computational complexity under control. The specific process is as follows:

[0102] 1) Feature Sequencing Step. To adapt to the input format of sequence modeling, the two-dimensional fused features are flattened into one-dimensional sequence features, converting spatial location information into temporal sequence information:

[0103]

[0104] in, The serialized feature sequence; The sequence length is equal to the total number of pixels in the feature map. Serialization preserves all spatial structure information, providing standard input for subsequent long-range dependency modeling.

[0105] 2) Perform bidirectional modeling of the sequence based on state updates.

[0106] The direction of crack extension is not fixed, and unidirectional sequence modeling will introduce directional bias, resulting in incomplete structural modeling in a certain direction. To eliminate directional bias and comprehensively model bidirectional structural correlations, bidirectional modeling (both forward and reverse) is performed on the feature sequences:

[0107] Forward modeling: Modeling along the original sequence order to obtain positive features. : ;

[0108] Reverse modeling: First, reverse the sequence, then perform the same modeling to obtain reverse features. : .

[0109] Bidirectional modeling can completely cover all extension directions of the crack, including left, right, up, and down, ensuring no structural modeling is missed. The specific modeling method described above uses a state-space model to perform long-range dependency modeling on the sequence, capturing long-distance pixel relationships along the crack's extension direction to solve the crack fracture problem. The state update and output mapping process is as follows:

[0110]

[0111]

[0112] in, For the first The hidden state at any given moment is used to store historical structural information; This is the state update function, used to update the structural representation based on the current input and historical states; This is the output mapping function, used to generate the structure output at the current time step.

[0113] 3) Bidirectional Feature Fusion Step. The forward and backward modeling results are fused to integrate bidirectional structural information and form a complete global structural representation.

[0114]

[0115] in, This is a fusion function, preferably achieved by combining feature concatenation with linear mapping, which preserves bidirectional information while unifying feature dimensions. , Indicates will , Channel dimension splicing, This represents a 1×1 convolution, used to implement linear mapping and channel compression. Represents a non-linear activation function; Output sequence for global modeling.

[0116] 4) Spatial Reconstruction Step. The global modeling results in one-dimensional sequence form are restored to a two-dimensional spatial structure to match the input format of subsequent multi-scale aggregation.

[0117]

[0118] The reconstructed global features satisfy: , , , These represent the feature map height, width, and number of channels, respectively. Spatial reconstruction can restore pixel-level spatial correspondences, ensuring that structural information is not lost.

[0119] 5) Residual Enhancement Step. During global structural modeling, some local details of cracks may be weakened. To preserve fine-grained crack information in the fused features and avoid detail loss, residual connections are used for enhancement:

[0120]

[0121] Residual enhancement enables global modeling features to possess both long-range continuous structure and local fine details, providing high-quality structural features for subsequent hierarchical feature aggregation.

[0122] 5. The global modeling features at different levels are first refined in a structure-preserving manner, and then adaptive weighted aggregation is performed to obtain multi-scale aggregated features that enhance the expression of crack structure.

[0123] Crack detection requires a balance between high-resolution detail and high semantic discriminativeness. Shallow features offer high resolution but weak semantics, while deep features offer strong semantics but low resolution. Direct multi-scale fusion can easily lead to crack structural breakage and blurred details. This step differs from traditional direct stitching methods, achieving efficient complementarity of detail and semantic information while preserving the crack topological continuity. Specific steps include:

[0124] 1) Hierarchical Feature Selection and Grouping Steps. Based on the differences in feature abstraction level and resolution, the globally modeled features are divided into three groups, each undertaking a different detection role:

[0125] Shallow features (high resolution) High resolution preserves fine-grained details such as crack edges and textures, but has weaker semantic information;

[0126] Mid-layer features It balances resolution and semantics, stably characterizes the local structure of cracks, and has strong noise robustness.

[0127] Deep features (high semantic level) It has high semantic and discriminative power, and can accurately distinguish between cracks and interference, but its resolution is low and it loses a lot of details.

[0128] By using layered processing, we can separate and optimize detailed information from semantic information, laying the foundation for subsequent differentiated refinement.

[0129] 2) Structure-preserving upsampling step. Deep features are semantically rich but small in size, and direct upsampling can easily lead to blurred crack edges and structural breaks. Therefore, a structure-preserving adaptive upsampling step is adopted to improve resolution while maintaining crack continuity and integrity:

[0130]

[0131] in, It is an adaptive reconstruction kernel based on local content prediction, which can dynamically calculate sampling weights according to the semantic information of the current location. This is a local receptive field. For local receptive fields The coordinates of the internal input feature pixels; For height direction index, It is indexed in the width direction; this upsampling method can avoid the detail blurring problem of traditional bilinear upsampling and accurately restore the crack line structure.

[0132] 3) Orientation-aligned downsampling step. The variable viewing angles of UAV inspections can easily cause geometric shifts and orientation distortions in cracks; direct downsampling would exacerbate these positional deviations. Therefore, orientation-aligned downsampling is performed on shallow high-resolution features to correct viewing distortions while reducing dimensionality, ensuring spatial alignment of the structure.

[0133]

[0134] in, The reference coordinates are the center pixels of the downsampled target; For sampling neighborhood The relative offset coordinates within; It is a learnable offset used to adaptively correct geometric misalignment caused by the viewpoint; The modulation weights are used to enhance the crack structure region and suppress background noise. The sampling neighborhood is used. Learnable offset alignment effectively improves detection stability under varying UAV perspectives. For sampling weights.

[0135] 4) Mid-level feature preservation step. Mid-level features achieve the best balance between detail and semantics, exhibiting structural stability, low noise, and requiring no significant transformations. Therefore, identity mapping is used to directly preserve them, avoiding structural distortion caused by additional transformations.

[0136] in, As an identity mapping, it directly outputs the original mid-layer features, preserving stable and reliable local structural information to the maximum extent.

[0137] 5) Adaptive weighted fusion step. The refined deep, mid-level, and shallow features are adaptively weighted and fused, with weights dynamically allocated based on feature contribution to achieve the optimal combination of multi-scale information.

[0138]

[0139] The contributions at different scales are adaptively adjusted based on feature importance. Among them, , , These are learnable weights that automatically learn the importance of features at different levels during training. The constant is minimized to avoid a denominator of 0. The fusion result simultaneously possesses deep semantics, mid-level structure, and shallow details, significantly improving the ability to express cracks.

[0140] 6) Multi-level recursive aggregation steps. To further strengthen the contextual association between multiple scales, a recursive aggregation path combining top-down and bottom-up approaches is constructed to achieve repeated interaction and enhancement of multi-level features:

[0141]

[0142] in, For recursive fusion functions, , For feature scale index, This represents the convolution operation. This indicates upsampling. By recursively passing structural information across scales, it ultimately outputs a set of multi-scale features with strong representational power. Scale indices 2–5 correspond to feature layers output at different downsampling stages in the backbone network. Layer 1, due to its excessively high resolution and insufficient semantic information, does not participate in cross-scale recursive fusion. This set can comprehensively cover crack targets of different scales and lengths, improving the detection completeness of both small and long cracks.

[0143] 6. Based on multi-scale aggregation features, this step performs integrated detection and structural analysis of bridge cracks, completes crack location, structural segmentation, centerline extraction and width calculation, and finally outputs complete detection results including spatial location, topological structure and geometric parameters, providing standardized data for subsequent bridge defect assessment and inspection report generation.

[0144] 1) To enable the detection head to utilize crack features at different scales simultaneously, taking into account the detection capabilities of micro-cracks, slender cracks, and large-area cracks, the multi-scale aggregated features are first scaled and spliced ​​together to construct the input features for the detection head:

[0145]

[0146] in: For scale alignment, upsampling is used to unify features at all scales to the same resolution; This is a channel-level splicing and fusion operation. The fusion features input to the detection head combine multi-scale details and semantic information.

[0147] 2) Crack Region Detection Steps. Crack target localization is performed based on fused features, enabling rapid bounding selection of the crack region. :

[0148]

[0149] in, This is a location regression function used to predict the bounding rectangle of the crack target; The output is a bounding box (x, y, w, h), which corresponds to the center coordinates, width, and height of the crack region, respectively, to achieve precise spatial positioning of the crack.

[0150] 3) Crack Structure Extraction Step. Based on the localization, pixel-level structural analysis is performed on the crack to obtain its complete topological morphology. express:

[0151]

[0152] in, The structure prediction function is output using a joint approach of semantic segmentation and skeleton extraction; It includes a binary segmentation mask for cracks and crack centerlines, which fully characterizes the continuous structure, orientation, and distribution pattern of cracks.

[0153] 4) Crack width estimation step. Based on the crack structure information, a distance transform algorithm is used for quantization calculation to obtain the crack geometric parameters:

[0154]

[0155] in, This is a distance-transform-based width calculation function. Using the crack centerline as a reference, it calculates and sums the distances from both edges to the centerline to obtain the point-by-point width; the output is... The width distribution and maximum width of cracks provide key quantitative indicators for assessing the severity of bridge defects.

[0156] Finally, this step outputs complete crack detection results: containing at least one crack target, with each target corresponding to a spatial location. Structural representation and geometric parameters This results in a structured output that can be directly used for engineering inspection.

[0157] In a preferred embodiment, the above-mentioned detection results can be transmitted back and visualized, including performing coordinate transformation on the detection results and mapping them to the original image or bridge engineering coordinate system; overlaying detection information such as crack location, centerline, and width onto the visible light image to achieve intuitive visualization. The system automatically generates a bridge inspection report, including key indicators such as crack location, length, width, and distribution pattern, which can be directly used for structural health assessment and maintenance decisions.

[0158] In a test of a specific embodiment, the key parameters of the model were set as follows:

[0159] Preset parameters for calculating illumination weights constants used in structural consistency calculations Learnable weights for fusing deep, mid, and shallow features , , Initialize to 1, 1, 1; use Adam as the training optimizer and set the learning rate to 10. −4 The loss function uses a combination of cross-entropy loss and Dice loss to improve segmentation accuracy; the model inference resolution is 1024×1024, achieving a frame rate of ≥15FPS and a detection mAP of ≥92% on the GPU platform, meeting the real-time inspection requirements of UAVs.

[0160] It is readily understood that the implementation of the methods described in the above embodiments can be based on a computer program, which is a set of instructions that can be executed by a computer (i.e., by a processor). When executed by the processor, the computer program implements a UAV bridge crack detection method based on structure-aware cross-modal fusion.

[0161] Furthermore, at least some of the computer programs associated with the methods of the embodiments can be distributed in a computer program product including a computer-readable storage medium carrying computer-usable instructions for one or more processors. This computer-readable storage medium can be provided in various forms, including non-transitory forms, such as, but not limited to, one or more disks, optical discs, magnetic tapes, chips, and magnetic and electronic storage. Further, the computer program can also be stored in a memory, which is part of an electronic device that also has a processor electrically connected to the memory. The computer program stored in the memory can be executed by the processor, thereby realizing a method for detecting cracks in UAV bridges based on structure-aware cross-modal fusion.

Claims

1. A method for detecting cracks in UAV bridges based on structure-aware cross-modal fusion, characterized in that, include: Step 1: Acquire visible light and infrared images of the bridge surface to form spatially and temporally aligned multimodal data pairs; Step 2: While maintaining spatial correspondence, extract feature representations that can characterize crack texture, edge and structural information from the visible light image and infrared image respectively, to obtain spatially strictly aligned enhanced multimodal feature maps; Step 3: Generate illumination perception weights based on the response intensity of visible light and infrared features, calculate structural consistency weights based on the edge and direction information of the features, and construct difference compensation weights by combining the differences between modal features. The weighted weights calculated by illumination perception weights, structural consistency weights, and difference compensation weights are used to adaptively weight and fuse the enhanced multimodal features to obtain the fused features. Step 4: The state-update-based sequence modeling method performs bidirectional long-range dependency modeling on the fusion features converted into time-series sequence representations, and reconstructs the global modeling features. Step 5: First, perform structure-preserving refinement on the global modeling features at different levels, and then perform adaptive weighted aggregation to obtain multi-scale aggregated features that enhance the expression of crack structure. Step 6: Based on multi-scale aggregation features, complete crack localization, structural segmentation, centerline extraction and width calculation, and finally output the detection results.

2. The method for detecting cracks in UAV bridges based on structure-aware cross-modal fusion according to claim 1, characterized in that, Step 2 specifically includes: first, using a feature coding network to perform feature mapping on the visible light image and the infrared image respectively to obtain initial features; then, using a multi-layer feature coding unit to encode the initial features layer by layer to obtain multi-level feature representations; and finally, using linear mapping to perform a unified dimensional transformation on the features of each layer and each scale to achieve scale normalization and channel alignment.

3. The method for detecting bridge cracks using unmanned aerial vehicles (UAVs) based on structure-aware cross-modal fusion according to claim 2, characterized in that, Enhanced multimodal feature maps are obtained by performing edge enhancement and high-frequency information enhancement operations on the scale-normalized and channel-aligned features.

4. The method for detecting cracks in UAV bridges based on structure-aware cross-modal fusion according to claim 1, characterized in that, The light perception weight is: ， ， ； The structural consistency weight is: ， ， ， ； The difference compensation weight is: ， ； in: For the Sigmoid function, These are preset parameters. After global average pooling, Visible light enhanced multimodal feature map, This is an infrared-enhanced multimodal feature map. For convolution mapping, It is a constant. Edge extraction operator.

5. The method for detecting cracks in UAV bridges based on structure-aware cross-modal fusion according to claim 1, characterized in that, The weighted weights are calculated by using the illumination perception weight, structural consistency weight, and difference compensation weight to calculate the visible light mode weight and the infrared mode weight. The visible light mode weight is the weighted sum of the illumination perception weight, structural consistency weight, and difference compensation weight, and the infrared mode weight is 1 minus the visible light mode weight.

6. The method for detecting cracks in UAV bridges based on structure-aware cross-modal fusion according to claim 1, characterized in that, In step 4, after reconstructing the long-range dependency model, the fused features and reconstructed features are enhanced by residual connection to obtain global modeling features.

7. The method for detecting cracks in UAV bridges based on structure-aware cross-modal fusion according to claim 1, characterized in that, Step 5 specifically includes: By performing structure-preserving upsampling on deep features, we obtain: ； in, It is an adaptive reconstruction kernel based on local content prediction. For local receptive fields The coordinates of the internal input feature pixels, For height direction index, Indexed in the width direction. This represents the deep features of the global modeling features obtained in step 4; By performing orientation-aligned downsampling on the shallow features, we obtain: ； in, The reference coordinates are the center pixels of the downsampled target. For sampling neighborhood The relative offset coordinates within, For learnable offsets, For modulation weights, For sampling neighborhood, For sampling weights; The identity mapping of the middle-layer features yields: ； in, It is an identity mapping; Adaptive weighted fusion yields: ； in, , , For learnable weights, It is a constant; Finally, perform multi-level recursive aggregation: ； in, For feature scale index, This represents the convolution operation. This indicates upsampling.

8. An electronic device, characterized in that, The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the UAV bridge crack detection method based on structure-aware cross-modal fusion as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that enables a processor to execute the UAV bridge crack detection method based on structure-aware cross-modal fusion as described in any one of claims 1 to 7.

10. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the UAV bridge crack detection method based on structure-aware cross-modal fusion as described in any one of claims 1 to 7.

Images