Bridge crack intelligent identification system and method based on image and radar multimodality

Abstract

The invention discloses a bridge crack intelligent identification system and method based on image and radar multimodality. The system comprises an image acquisition module; the image acquisition module acquires an apparent image of a bridge structure and transmits the apparent image to the apparent crack identification module; the apparent crack recognition module recognizes crack features in the image and outputs a recognition result; the system further comprises a radar data acquisition module which acquires geological radar images in the bridge structure and transmits the images to the internal crack identification module. The internal crack identification module extracts position, form and depth feature information of cracks in the geological radar image, carries out identification and outputs an identification result; the system further comprises a crack fusion evaluation module which outputs a crack risk level based on a fusion scoring function or a neural network. The method is suitable for structural health monitoring of multiple types of bridges, and the detection accuracy and the intelligent level are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge structure monitoring, and in particular to an intelligent bridge crack recognition system and method based on image and radar multimodality. Background Art

[0002] As a critical component of transportation infrastructure, the safe operation of bridges is crucial to the safety of people's lives and property, as well as national economic development. Cracks are one of the most common and typical defects in bridge structures, and can be primarily categorized as superficial cracks and internal structural cracks. Superficial cracks typically appear on the surface of concrete components such as bridge decks, piers, and beams, and can be identified using visible light imaging. Internal cracks, on the other hand, are hidden within the concrete structure and often require non-contact, nondestructive testing methods for detection.

[0003] Currently, surface crack detection generally uses deep learning methods based on image processing, such as CNN, YOLO, or the Transformer family of models. These methods are effective in detecting wide, clear cracks, but are prone to missed detections and false detections when detecting slender, curved, and low-contrast cracks. Furthermore, traditional self-attention mechanisms are insufficiently capable of modeling the directional characteristics of cracks, affecting detection accuracy and speed.

[0004] Internal crack detection, on the other hand, often relies on nondestructive testing equipment such as ground penetrating radar (GPR) to obtain images of underground structures. In GPR images, cracks typically appear as curved hyperbolic reflections. However, factors such as concrete heterogeneity, radar noise, and varying target depths can result in complex and discontinuous crack echoes. Traditional thresholding and edge detection algorithms are poorly adapted to these unstructured features, resulting in limited detection accuracy. While some existing deep learning methods can recognize GPR images, conventional convolution algorithms struggle to effectively perceive curved features and lack a mechanism to focus on weak edge information, resulting in insufficient recognition of shallow, fine cracks.

[0005] Furthermore, current methods for detecting surface cracks and internal cracks are typically handled as two separate processes, lacking a unified fusion framework. This prevents effective comparison and correlation of external and internal bridge disease information. This fragmented approach hinders comprehensive structural health assessments and fails to provide systematic support for subsequent maintenance decisions.

[0006] Therefore, there is an urgent need for a multimodal bridge crack intelligent identification system that integrates image recognition and radar signal analysis to improve the accuracy and adaptability of crack detection while realizing the fusion analysis and comprehensive evaluation of surface and internal crack data. Summary of the Invention

[0007] Based on the above technical problems, this application discloses an intelligent bridge crack recognition system based on surface image and radar multimodality, including an image acquisition module, an surface crack recognition module, a radar data acquisition module, an internal crack recognition module, and a crack fusion evaluation module;

[0008] The image acquisition module is used to acquire the apparent image of the bridge structure and transmit the image to the apparent crack recognition module;

[0009] The apparent crack recognition module includes a multi-scale strip attention module and an adaptive position query embedding module. The multi-scale strip attention module is used to perform attention enhancement in the horizontal and vertical directions to improve the modeling ability of slender cracks. The adaptive position query embedding module is used to dynamically generate position-sensitive query vectors based on global features to improve the positioning accuracy of complex spatial cracks. The apparent crack recognition module identifies crack features in the image and outputs the recognition results.

[0010] The radar data acquisition module is used to collect geological radar images inside the bridge structure and transmit the images to the internal crack identification module;

[0011] The internal crack recognition module automatically extracts the position, shape and depth feature information of the cracks in the geological radar image, recognizes the internal crack features in the geological radar image and outputs the recognition result;

[0012] The crack fusion assessment module is connected to the apparent crack identification module and the internal crack identification module respectively, and is used to perform spatial coordinate conversion and alignment on the apparent crack and internal crack identification results, and output the crack risk level based on the fusion scoring function or neural network.

[0013] Preferably, the multi-scale strip attention module calculates the attention weights in the horizontal and vertical directions respectively, and the formula is: F′=A h +A v =Softmax(W h F)+Softmax(W v ·F), where W h and W v are the convolution weights along the horizontal and vertical directions respectively, F is the input feature map, A h is the horizontal attention, A v It is vertical attention; the multi-scale strip attention module enhances the modeling ability of slender and directional crack features by decomposing the self-attention structure into horizontal and vertical strip attention.

[0014] Preferably, the adaptive location query embedding module is based on the global feature vector F global The query embedding vector is generated through a multi-layer perceptron network. The formula is: Where f(·) is a multi-layer perceptron network, N q is the preset number of queries used to control the density of location queries, and D is the embedding dimension to ensure the expressiveness of the feature space. The adaptive location query embedding module dynamically extracts global context information to generate a location-sensitive query vector that is strongly correlated with the spatial structure, thereby improving the spatial positioning of cracks under complex bridge deck geometry conditions.

[0015] Preferably, the radar data acquisition module is used to collect geological radar images inside the bridge structure, using a vehicle-mounted towed or handheld GPR radar device to perform multi-channel radar scanning on the target area of ​​the bridge, obtain a radar image sequence containing information on the internal structure of the bridge, and provide raw data input for the internal crack identification module.

[0016] Preferably, the internal crack recognition module includes a waveform-aware convolution module and an edge-guided attention module. The waveform-aware convolution module introduces a convolution kernel with a learnable offset to enable the convolution window to adaptively slide around the crack curve, thereby enhancing the modeling capability of curved cracks in GPR images. The edge-guided attention module generates a fused attention map by fusing the edge map and the spatial attention map, thereby guiding attention to focus on the fuzzy crack area and improving the recognition capability of low-contrast, shallow cracks. The combination of the two realizes the analysis of complex echo structures in geological radar images and the extraction of internal crack features.

[0017] Preferably, the waveform-aware convolution module uses a convolution kernel with a learnable offset, and its output formula is: Y(p0)=∑ k w(k)·F(p0+k+Δp k ), where Δp k is the learnable offset at position k, which is used to make the convolution window adaptively adjust to fit the crack curve shape, w(k) is the convolution kernel weight, F is the input feature map, and the waveform-aware convolution module enhances the modeling ability of curved cracks in geological radar images by introducing a learnable offset mechanism, effectively improving the extraction accuracy of crack features in complex radar echo structures.

[0018] Preferably, the edge-guided attention module generates a fused attention map A′ by fusing the edge map E with the spatial attention map A and calculates the enhanced output feature Y. The specific calculation formula is: A′=σ(W1·E+W2·A+b), Y=A′⊙F+F, where σ is the sigmoid activation function, ⊙ represents element-by-element multiplication, F is the input feature map, W1 and W2 are the convolution weight matrices of the edge map E and the spatial attention map A respectively, and b is the bias term; the edge-guided attention module focuses attention on suspected crack areas with blurred edges through a guided edge enhancement mechanism, thereby improving the recognition ability of low-contrast and shallow cracks.

[0019] Preferably, the crack fusion assessment module is used to perform spatial coordinate conversion and alignment on the apparent crack and internal crack identification results, and output the crack risk level based on the fusion scoring function or neural network. Specifically, the image coordinate system detection results of the apparent crack are projected onto the geological radar coordinate system through the multi-sensor coordinate transformation matrix to unify the spatial coordinates of the apparent and internal cracks. Then, the comprehensive risk value is calculated using the fusion scoring function by comprehensively considering factors such as detection confidence, crack size, spatial overlap and depth. The formula is: S = λ1·C img +λ2·C gpr +λ3·IoU 3D +λ4·D depth , where λ1, λ2, λ3, and λ4 are the corresponding adjustable fusion weight coefficients, C img 、C gpr Respectively, the apparent and internal recognition confidence, IoU 3D is the three-dimensional spatial overlap, D is the crack depth, and the input feature vector is automatically classified to output a structured assessment result containing location, type, and risk level information.

[0020] Preferably, the visualization and report output module is used to map the identification results to a three-dimensional bridge structure model and generate a structured inspection report, specifically: the identification results of apparent cracks and internal cracks are superimposed on the corresponding positions of the bridge model in a three-dimensional visualization form to intuitively display the spatial distribution of cracks, and at the same time output a structured report containing crack ID, type, spatial position, length, width, depth, risk level and color label, wherein the color label corresponds to different warning colors according to the risk level, which facilitates engineering maintenance personnel to quickly locate and evaluate the condition of bridge diseases.

[0021] The intelligent bridge crack recognition method based on surface image and radar multimodality includes the following steps:

[0022] S1. Collecting the surface image of the bridge structure through the image acquisition module and collecting the geological radar image inside the bridge structure through the radar data acquisition module;

[0023] S2. The apparent crack recognition module is used to identify crack features in the apparent image. The multi-scale strip attention module performs attention enhancement in the horizontal and vertical directions to improve the modeling capability of slender cracks. The adaptive position query embedding module dynamically generates position-sensitive query vectors based on global features to improve crack localization accuracy.

[0024] S3. The internal crack recognition module is used to identify the internal crack features in the geological radar image. The waveform-aware convolution module introduces a convolution kernel with a learnable offset to enhance the modeling capability of curved cracks. The edge-guided attention module fuses edge response and spatial attention features to improve the recognition capability of fuzzy crack areas.

[0025] S4. The crack fusion assessment module performs spatial coordinate conversion and alignment on the apparent crack and internal crack identification results, outputs the crack risk level based on the fusion scoring function, and the visualization and report output module maps the identification results to the three-dimensional bridge structure model and generates a structured inspection report including the location, type, size, and risk level.

[0026] Compared with the prior art, the technical solution of this application has the following technical effects:

[0027] The present invention realizes the joint detection of surface and internal cracks in bridges through the collaboration of multiple modules, which significantly improves the comprehensiveness of detection. In traditional methods, the detection of surface cracks and internal cracks is often carried out independently, and it is difficult to associate the spatial correspondence between the two, resulting in incomplete structural health assessment. The present invention obtains the surface image through the image acquisition module, combines the radar data acquisition module to obtain the internal geological radar image, and then uses the surface crack identification module and the internal crack identification module to respectively extract the crack characteristics under different modes, and the fusion assessment module realizes data alignment and risk rating. This multimodal fusion method breaks the fragmentation of traditional detection, can simultaneously provide disease information of the bridge surface and internal structure, form a complete crack distribution map, and provide comprehensive data support for the overall assessment of the health status of the bridge, avoiding the limitations of single modality detection.

[0028] The present invention effectively improves the recognition accuracy of crack features through innovative algorithm module design. The multi-scale strip attention module (MSA) in the apparent crack recognition module specifically improves the modeling ability of slender cracks by enhancing attention in the horizontal and vertical directions. Compared with the traditional self-attention mechanism, it can more accurately capture the directional characteristics of cracks and reduce missed detection and false detection of slender cracks. The waveform-aware convolution module (WaConv) in the internal crack recognition module introduces a convolution kernel with learnable offset, so that the convolution window can adaptively fit the curved crack morphology, solving the problem of insufficient modeling of hyperbolic reflection features in GPR images by traditional convolution; the edge-guided attention module (EGAM) enhances the recognition ability of fuzzy crack areas by fusing edge information with spatial attention, especially significantly improving the detection effect of shallow and small cracks. These algorithmic improvements have substantially improved the system's recognition accuracy for complex cracks.

[0029] The fusion assessment module of the present invention first unifies the detection results of surface and internal cracks into the same spatial coordinate system through a multi-sensor coordinate transformation matrix, thus solving the problem of spatial alignment of data from different modalities. Furthermore, by integrating a scoring function or neural network, it integrates multi-dimensional information such as detection confidence, crack size, spatial overlap, and depth to automatically output the crack risk level. This intelligent assessment method not only reduces the subjectivity of manual analysis but also adapts to the needs of different detection scenarios through adjustable weight coefficients. It supports both rule fusion and learning fusion modes, enhancing the system's flexibility and adaptability, and providing a scientific and quantitative basis for bridge maintenance decision-making.

[0030] The visualization and report output module of the present invention maps the crack identification results to a three-dimensional bridge structure model, displaying the spatial location, type, and risk level of the cracks in an intuitive manner. Engineering maintenance personnel can quickly locate the diseased area through the three-dimensional model without relying on professionals to interpret complex data. At the same time, the generated structured report contains detailed information such as crack ID, geometric properties, risk level, etc., and the format is standardized and unified to facilitate data archiving and subsequent tracking and analysis. This visual and structured output method greatly improves the practicality of the inspection report, shortens the process from inspection to decision-making, meets the engineering field's demand for efficient and intuitive inspection reports, and helps to make bridge maintenance work intelligent and standardized.

[0031] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application so that it can be implemented in accordance with the contents of the specification, and to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following is a detailed description of the preferred embodiment of the present application in conjunction with the accompanying drawings.

[0032] Based on the detailed description of the specific embodiments of the present application in conjunction with the accompanying drawings below, those skilled in the art will become more aware of the above and other objects, advantages and features of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without inventive work. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0034] Figure 1 This is a block diagram of the overall structure of the bridge crack identification system of the present invention;

[0035] Figure 2 This is a structural flow chart of the apparent crack identification module of the present invention;

[0036] Figure 3 This is a structural flow chart of the internal crack identification module of the present invention;

[0037] Figure 4 This is a flow chart of spatial mapping and scoring of the crack fusion assessment module of the present invention;

[0038] Figure 5 This is a schematic diagram of the visualization of the detection results of the present invention in the three-dimensional structure of the bridge. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the present application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. In addition, for clarity and brevity, the description of known functions and structures has been omitted in the embodiments.

[0040] It should be understood that references throughout this specification to "one embodiment" or "this embodiment" mean that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present application. Therefore, the appearance of "one embodiment" or "this embodiment" throughout this specification does not necessarily refer to the same embodiment. Furthermore, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0041] In addition, the present application may repeat reference numerals and / or letters in different examples. This repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or settings discussed.

[0042] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" in this article describes another type of association object relationship, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0043] The term "at least one" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, at least one of A and B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0044] It should also be noted that, in this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include," "comprises," or any other variations thereof are intended to cover non-exclusive inclusion.

[0045] Example 1

[0046] This embodiment mainly describes a bridge crack intelligent identification system based on surface image and radar multimodality. Figure 1 As shown, it includes image acquisition module, apparent crack recognition module, radar data acquisition module, internal crack recognition module, and crack fusion evaluation module;

[0047] The image acquisition module is used to acquire the apparent image of the bridge structure and transmit the image to the apparent crack recognition module;

[0048] The apparent crack recognition module includes a multi-scale strip attention module and an adaptive position query embedding module. The multi-scale strip attention module is used to perform attention enhancement in the horizontal and vertical directions to improve the modeling ability of slender cracks. The adaptive position query embedding module is used to dynamically generate position-sensitive query vectors based on global features to improve the localization accuracy of complex spatial cracks. The apparent crack recognition module identifies crack features in the image and outputs the recognition results.

[0049] The radar data acquisition module is used to collect geological radar images inside the bridge structure and transmit the images to the internal crack identification module;

[0050] The internal crack recognition module automatically extracts the location, shape and depth feature information of the cracks in the geological radar image, identifies the internal crack features in the geological radar image and outputs the recognition results;

[0051] The crack fusion assessment module is connected to the apparent crack identification module and the internal crack identification module respectively, and is used to perform spatial coordinate conversion and alignment on the apparent crack and internal crack identification results, and output the crack risk level based on the fusion scoring function or neural network.

[0052] Furthermore, the multi-scale strip attention module calculates the attention weights in the horizontal and vertical directions respectively, and the formula is: F′=A h +A v =Softmax(Wh F)+Softmax(W v ·F), where W h and W v are the convolution weights along the horizontal and vertical directions respectively, F is the input feature map, A h is the horizontal attention, A v The multi-scale strip attention module decomposes the self-attention structure into horizontal and vertical strip attention to enhance the modeling ability of slender and directional crack features.

[0053] Furthermore, the adaptive location query embedding module is based on the global feature vector F global The query embedding vector is generated through a multi-layer perceptron network. The formula is: Where f(·) is a multi-layer perceptron network, N q is the preset number of queries used to control the density of location queries, and D is the embedding dimension to ensure the expressiveness of the feature space. The adaptive location query embedding module dynamically extracts global context information to generate a location-sensitive query vector that is strongly correlated with the spatial structure, thereby improving the spatial positioning of cracks under complex bridge deck geometry conditions.

[0054] Furthermore, the radar data acquisition module is used to collect geological radar images inside the bridge structure. It uses a vehicle-mounted towed or handheld GPR radar device to perform multi-channel radar scanning on the target area of ​​the bridge to obtain a radar image sequence containing information about the internal structure of the bridge, providing raw data input for the internal crack identification module.

[0055] Furthermore, the internal crack recognition module includes a waveform-aware convolution module and an edge-guided attention module. The waveform-aware convolution module introduces a convolution kernel with learnable offset to make the convolution window slide adaptively around the crack curve, thereby enhancing the modeling ability of curved cracks in GPR images; the edge-guided attention module generates a fused attention map by fusing the edge map and the spatial attention map, guiding attention to focus on the fuzzy crack area, thereby improving the recognition ability of low-contrast and shallow cracks. The combination of the two realizes the analysis of complex echo structures in geological radar images and the extraction of internal crack features.

[0056] Furthermore, the waveform-aware convolution module uses a convolution kernel with a learnable offset, and its output formula is: Y(p0) = ∑ k w(k)·F(p0+k+Δp k ), where Δp kis the learnable offset at position k, which is used to make the convolution window adaptively adjust to fit the crack curve shape, w(k) is the convolution kernel weight, F is the input feature map, and the waveform-aware convolution module enhances the modeling ability of curved cracks in geological radar images by introducing a learnable offset mechanism, effectively improving the extraction accuracy of crack features in complex radar echo structures.

[0057] Furthermore, the edge-guided attention module generates a fused attention map A′ by fusing the edge map E with the spatial attention map A and calculates the enhanced output feature Y. The specific calculation formula is: A′=σ(W1·E+W2·A+b), Y=A′⊙F+F, where σ is the sigmoid activation function, ⊙ represents element-by-element multiplication, F is the input feature map, W1 and W2 are the convolution weight matrices of the edge map E and the spatial attention map A respectively, and b is the bias term; the edge-guided attention module focuses on suspected crack areas with blurred edges through a guided edge enhancement mechanism, thereby improving the recognition ability of low-contrast and shallow cracks.

[0058] Furthermore, the crack fusion assessment module is used to convert and align the spatial coordinates of the apparent cracks and internal cracks identification results, and output the crack risk level based on the fusion scoring function or neural network. Specifically, the image coordinate system detection results of the apparent cracks are projected into the geological radar coordinate system through the multi-sensor coordinate transformation matrix to achieve the unification of the spatial coordinates of the apparent and internal cracks. Then, the comprehensive risk value is calculated using the fusion scoring function based on the comprehensive detection confidence, crack size, spatial overlap and depth. The formula is: S = λ1·C img +λ2·C gpr +λ3·IoU 3D +λ4·D depth , where λ1, λ2, λ3, and λ4 are the corresponding adjustable fusion weight coefficients, C img 、C gpr Respectively, the apparent and internal recognition confidence, IoU 3D is the three-dimensional spatial overlap, D is the crack depth, and the input feature vector is automatically classified to output a structured assessment result containing location, type, and risk level information.

[0059] Furthermore, the visualization and report output module is used to map the identification results to a three-dimensional bridge structure model and generate a structured inspection report. Specifically, the identification results of apparent cracks and internal cracks are superimposed on the corresponding positions of the bridge model in a three-dimensional visualization form to intuitively display the spatial distribution of cracks. At the same time, a structured report containing crack ID, type, spatial position, length, width, depth, risk level and color labels is output. The color labels correspond to different warning colors according to the risk level, which facilitates engineering maintenance personnel to quickly locate and assess the condition of bridge diseases.

[0060] This embodiment describes in detail the multimodal bridge crack recognition system of the present invention that combines surface images with geological radar (GPR) data. Through the improved Transformer and convolutional network structures, it realizes the unified recognition and fusion evaluation of concrete surface cracks and internal structural cracks, thereby improving the comprehensiveness and accuracy of detection.

[0061] Based on Example 1, this embodiment describes in detail the specific bridge crack intelligent identification system of this application, specifically:

[0062] The multi-scale strip attention module MSA replaces the self-attention structure in the original Transformer encoder and decomposes it into horizontal and vertical strip attention to enhance the modeling ability of slender and directional crack features; let the input feature map be Then the MSA is calculated separately:

[0063] Horizontal attention: A h =Softmax(W h F); Vertical attention: A v =Softmax(W v ·F); where W h 、W v is the convolution kernel parameter. The final output fusion is expressed as: F′=A h +A v This mechanism can significantly improve the response capability to crack features with long strip morphology.

[0064] Adaptive Location Query Embedding (APQE) is a module that dynamically generates location query embeddings by extracting global context information through a neural network, thereby improving the model’s accuracy in locating complex spatial cracks. Specifically, let the global eigenvector be F global , generate query vectors through multi-layer perceptron: Where f(·) is the MLP network, N q is the number of queries, and D is the embedding dimension. This mechanism makes the query more closely related to the spatial structure and is suitable for crack identification under complex bridge deck geometry conditions.

[0065] Internal crack recognition module: including waveform-aware convolution module and edge-guided attention module;

[0066] The waveform-aware convolution module WaConv addresses the inadequacy of traditional convolution kernels in modeling curved cracks in radar images. WaConv introduces a convolution kernel with a learnable offset, which allows the receptive field to be adaptively aligned with the target shape. The input feature map is defined as F, and the learning offset vector is Δp. The convolution output is: Y(p0) = ∑ k w(k)·F(p0+k+Δp k), where k represents the position within the convolution window, w(k) is the kernel weight, and Δp k is the offset of the position; by introducing Δp k ,The convolution window can slide adaptively around the crack curve, enhancing the modeling capability of complex radar echo structures.

[0067] The edge-guided attention module (EGAM) addresses the problem of blurred edges in GPR images. EGAM uses a guided edge enhancement mechanism to fuse traditional spatial attention with edge information, guiding attention to focus on suspected crack areas.

[0068] Assume that the feature map is F, the edge map is E, and the spatial attention map is A. The fusion expression is: A′=σ(W1·E+W2·A+b), Y=A′⊙F+F, where σ is the sigmoid activation function and ⊙ represents element-by-element multiplication. This structure enhances the discrimination ability of low-contrast areas and helps to identify early or shallow cracks.

[0069] Crack Fusion Assessment Module,This module mainly includes the following two core steps;

[0070] Coordinate transformation and alignment: Camera-radar coordinates are unified. Since the apparent cracks are obtained from images, while the internal cracks are obtained from GPR data, there are differences in the coordinate systems between the two. It is necessary to establish a unified spatial reference frame. The principle is multi-sensor coordinate transformation. Assume that the detection center of the apparent crack in the image coordinate system is P img =[u,v,1] T ; The detection center of the internal crack in the GPR coordinate system is P gpr =[x,y,z] T ; The two coordinate systems are calibrated to obtain the transformation matrix The apparent detection results are projected into the GPR space coordinates as follows: Where K is the image intrinsic parameter matrix; T contains the rotation and translation components;

[0071] If the GPR is a 2D image, it can be simplified to a plane coordinate mapping. This process can be achieved through linear algebra to realize bilateral spatial point cloud or pixel-level reprojection, thus establishing a basis for consistency between the surface and internal crack spaces.

[0072] Coordinate transformation and alignment: The camera and radar coordinates are unified. After the coordinate unification is completed, the system further performs the fusion and grade evaluation of the recognition results. Principle 1 is a fusion scoring function (rule fusion example). The following fusion scoring function S can be constructed, which comprehensively considers factors such as detection confidence, crack size, and spatial overlap. The formula is: S = λ1·C img +λ2·C gpr +λ3·IoU 3D +λ4·D depth , where Cimg 、C gpr are the apparent and GPR detection confidence, IoU 3D The 3D overlap degree of the two types of cracks in the unified space, D depth GPR-inferred depth or fracture vertical scale, λ i Adjustable fusion weight coefficient.

[0073] Principle 2 is learning fusion, which can be further selected. If a neural network is introduced for automatic learning fusion strategy, the feature input can be defined as: f = [C img ,C gpr ,IoU 3D ,L,W,D], where L, W, and D are geometric attributes such as crack length, width, and depth, respectively. The evaluation level label is generated by lightweight MLP: Class risk =MLP(f). This approach can realize a closed-loop process from crack detection to intelligent assessment and output a structured inspection report containing information such as location, morphology, and grade for use by bridge maintenance personnel.

[0074] Further evaluating the output, the final output form is:

[0075] {

[0076] "id":"crack_014",

[0077] "type":"Appearance / Internal",

[0078] "location":[x,y,z],

[0079] "length":0.82,

[0080] "width":0.03,

[0081] "depth":0.16,

[0082] "risk_level":"Medium"

[0083] }

[0084] This implementation details the present invention's decomposition of horizontal and vertical attention through a multi-scale strip attention module to enhance the ability to model long and thin cracks; the adaptive position query embedding module dynamically generates position-sensitive query vectors to improve the accuracy of locating complex spatial cracks; the waveform-aware convolution module introduces a learnable offset convolution kernel to enhance the ability to model curved cracks; the edge-guided attention module fuses edge and spatial attention features to enhance the ability to identify fuzzy cracks; the fusion assessment module realizes coordinate alignment and risk rating, significantly improving the comprehensiveness, accuracy and intelligence of detection.

[0085] Example 2

[0086] This embodiment describes in detail a method for intelligently identifying bridge cracks based on surface images and radar multimodality, which includes the following steps, specifically:

[0087] S1. Collecting the surface image of the bridge structure through the image acquisition module and collecting the geological radar image inside the bridge structure through the radar data acquisition module;

[0088] S2. The apparent crack recognition module is used to identify crack features in the apparent image. The multi-scale strip attention module performs attention enhancement in the horizontal and vertical directions to improve the modeling capability of slender cracks. The adaptive position query embedding module dynamically generates position-sensitive query vectors based on global features to improve crack localization accuracy.

[0089] S3. The internal crack recognition module is used to identify the internal crack features in the geological radar image. The waveform-aware convolution module introduces a convolution kernel with a learnable offset to enhance the modeling capability of curved cracks. The edge-guided attention module fuses edge response and spatial attention features to improve the recognition capability of fuzzy crack areas.

[0090] S4. The crack fusion assessment module performs spatial coordinate conversion and alignment on the apparent crack and internal crack identification results, outputs the crack risk level based on the fusion scoring function, and the visualization and report output module maps the identification results to the three-dimensional bridge structure model and generates a structured inspection report including the location, type, size, and risk level.

[0091] like Figure 2 As shown, in the apparent crack recognition process in S1, high-resolution image sensors (which can be high-definition cameras, drone-mounted equipment, etc.) are deployed on-site to collect images of the bridge deck, beams, piers, and other areas. The collected images are pre-processed by normalization, noise reduction, and size scaling before being input into the apparent crack recognition module.

[0092] The apparent crack recognition module adopts an improved structure based on the RT-DETR network, which specifically includes the following two submodules:

[0093] Multi-scale strip attention module (MSA), which takes input feature maps Perform directional enhancement and calculate as follows:

[0094] Horizontal attention: A h =Softmax(W h F);

[0095] Vertical attention: A v =Softmax(Wv F);

[0096] Fusion output: F′=A h +A v , effectively capturing the horizontal and vertical distribution characteristics of slender cracks.

[0097] The adaptive position query embedding module APQE generates a position-sensitive query vector based on global features. The formula is: Where f(·) is an MLP network that improves the robustness of crack localization in complex images; this module ultimately outputs the location information (such as bounding box), type label, and confidence score of the crack target.

[0098] Further, if Figure 3 As shown in Figure 2, the internal crack identification process in S2 involves deploying a GPR radar device (either a vehicle-mounted towed device or a handheld device) on the bridge deck to perform radar scanning of the target area. A dual-channel or multi-channel GPR image sequence is obtained as input to the internal crack identification module. This module is built based on the improved EfficientDet-D3 model and integrates the following two submodules:

[0099] The waveform-aware convolution module (WaConv) is used to enhance the ability to extract curved reflection features in GPR images. It introduces an offset learnable convolution kernel, and its output is: Y(p0) = ∑ k w(k)·F(p0+k+Δp k ), where Δp k A learnable offset for the convolution kernel at position k to improve the adaptability to hyperbolic cracks;

[0100] The Edge-Guided Attention Module (EGAM) introduces edge responses to guide spatial attention generation. Specifically, it is calculated as follows: A′ = σ(W1·E+W2·A+b), Y = A′⊙F+F, where E is the edge map, A is the traditional spatial attention map, and ⊙ represents element-wise multiplication. This module is particularly suitable for detecting edge structures with blurred or shallow cracks.

[0101] Internal crack identification results include depth information, morphology estimation, and probability maps;

[0102] Further, if Figure 4 As shown in the fracture information fusion and evaluation process in S3, in order to achieve comprehensive analysis of multi-source fracture information, the system uses the following steps to complete the fusion:

[0103] Coordinate transformation, based on the previously completed external parameter calibration of the image sensor and radar sensor, obtains the transformation matrix Let the apparent detection coordinate P img Mapping to GPR coordinate system:

[0104] Crack fusion and grade evaluation, construction of fusion scoring function, S = λ1·C img +λ2·C gpr +λ3·IoU 3D +λ4·D depth , where C img , C gpr is the crack confidence, IoU 3D is the spatial overlap ratio, D depth is the crack depth, λ i Adjustable fusion weight; or by inputting feature f = [C img ,C gpr ,IoU 3D ,L,W,D] to MLP network to automatically classify crack levels.

[0105] Further, if Figure 5 As shown in the visualization and report generation module in S4, this system maps the crack identification and fusion results to the three-dimensional bridge structure diagram for visualization, and outputs the following information: crack type (apparent / internal), spatial position (X, Y, Z), geometric attributes (length, width, depth), risk level (high / medium / low), detection timestamp, forming a structured detection report for engineering maintenance personnel to make subsequent disposal decisions.

[0106] This embodiment describes in detail how the image acquisition module acquires bridge surface images through high-resolution sensors and improves their quality through preprocessing; the surface crack recognition module utilizes the MSA and APQE modules to effectively capture the characteristics of slender cracks and improve positioning robustness; the radar module acquires GPR images, and the internal recognition module enhances curve and fuzzy crack detection capabilities through WaConv and EGAM; the fusion module realizes multi-source data alignment and risk assessment through coordinate transformation and scoring functions; and the visualization module generates structured reports containing multi-dimensional information, improving detection accuracy and evaluation efficiency throughout the entire process and providing systematic support for bridge maintenance.

[0107] The above are only preferred embodiments of the present invention, which do not limit the scope of protection of the present invention. For those skilled in the art, the present invention can be modified and varied in various ways. Any changes, modifications, replacements, integrations and parameter changes to these embodiments through conventional substitutions or that can achieve the same functions without departing from the principles and spirit of the present invention fall within the scope of protection of the present invention.

Claims

1. The intelligent bridge crack recognition system based on image and radar multimodality is characterized by: It includes image acquisition module, apparent crack recognition module, radar data acquisition module, internal crack recognition module, and crack fusion assessment module; The image acquisition module is used to acquire the apparent image of the bridge structure and transmit the image to the apparent crack recognition module; The apparent crack recognition module includes a multi-scale strip attention module and an adaptive position query embedding module. The multi-scale strip attention module is used to perform attention enhancement in the horizontal and vertical directions to improve the modeling ability of slender cracks. The adaptive position query embedding module is used to dynamically generate position-sensitive query vectors based on global features to improve the positioning accuracy of complex spatial cracks. The apparent crack recognition module identifies crack features in the image and outputs the recognition results. The radar data acquisition module is used to collect geological radar images inside the bridge structure and transmit the images to the internal crack identification module; The internal crack recognition module automatically extracts the position, shape and depth feature information of the cracks in the geological radar image, recognizes the internal crack features in the geological radar image and outputs the recognition result; The crack fusion assessment module is connected to the apparent crack identification module and the internal crack identification module respectively, and is used to perform spatial coordinate conversion and alignment on the apparent crack and internal crack identification results, and output the crack risk level based on the fusion scoring function or neural network.

2. The bridge crack intelligent identification system based on image and radar multimodality according to claim 1 is characterized in that: The multi-scale strip attention module calculates the attention weights in the horizontal and vertical directions respectively, and the formula is: F′=A h +A v =Softmax(W h F)+Softmax(W v ·F), where W h and W v are the convolution weights along the horizontal and vertical directions respectively, F is the input feature map, A h is the horizontal attention, A v It is vertical attention; the multi-scale strip attention module enhances the modeling ability of slender and directional crack features by decomposing the self-attention structure into horizontal and vertical strip attention.

3. The bridge crack intelligent identification system based on image and radar multimodality according to claim 1 is characterized in that: The adaptive location query embedding module is based on the global feature vector F global The query embedding vector is generated through a multi-layer perceptron network. The formula is: Where f(·) is a multi-layer perceptron network, N q is the preset number of queries used to control the density of location queries, and D is the embedding dimension to ensure the expressiveness of the feature space. The adaptive location query embedding module dynamically extracts global context information to generate a location-sensitive query vector that is strongly correlated with the spatial structure, thereby improving the spatial positioning of cracks under complex bridge deck geometry conditions.

4. The bridge crack intelligent identification system based on image and radar multimodality according to claim 1 is characterized in that: The radar data acquisition module is used to collect geological radar images inside the bridge structure. It uses a vehicle-mounted towed or handheld GPR radar device to perform multi-channel radar scanning on the bridge target area, obtain a radar image sequence containing information about the internal structure of the bridge, and provide raw data input for the internal crack identification module.

5. The bridge crack intelligent identification system based on image and radar multimodality according to claim 1 is characterized in that: The internal crack recognition module includes a waveform-aware convolution module and an edge-guided attention module. The waveform-aware convolution module introduces a convolution kernel with a learnable offset to enable the convolution window to adaptively slide around the crack curve, thereby enhancing the modeling capability of curved cracks in GPR images. The edge-guided attention module generates a fused attention map by fusing the edge map and the spatial attention map, guiding attention to focus on the fuzzy crack area and improving the recognition ability of low-contrast and shallow cracks. The combination of the two realizes the analysis of complex echo structures in geological radar images and the extraction of internal crack features.

6. The bridge crack intelligent identification system based on image and radar multimodality according to claim 5 is characterized in that: The waveform-aware convolution module uses a convolution kernel with a learnable offset, and its output formula is: Y(p0)=∑ k w(k)·F(p0+k+Δp k ), where Δp k is the learnable offset at position k, which is used to make the convolution window adaptively adjust to fit the crack curve shape, w(k) is the convolution kernel weight, F is the input feature map, and the waveform-aware convolution module enhances the modeling ability of curved cracks in geological radar images by introducing a learnable offset mechanism, effectively improving the extraction accuracy of crack features in complex radar echo structures.

7. The bridge crack intelligent identification system based on image and radar multimodality according to claim 5 is characterized in that: The edge-guided attention module generates a fused attention map A′ by fusing the edge map E with the spatial attention map A and calculates the enhanced output feature Y. The specific calculation formula is: A′=σ(W1·E+W2·A+b), Y=A′⊙F+F, where σ is the sigmoid activation function, ⊙ represents element-by-element multiplication, F is the input feature map, W1 and W2 are the convolution weight matrices of the edge map E and the spatial attention map A respectively, and b is the bias term; the edge-guided attention module focuses attention on suspected crack areas with blurred edges through a guided edge enhancement mechanism, thereby improving the recognition ability of low-contrast and shallow cracks.

8. The bridge crack intelligent identification system based on image and radar multimodality according to claim 1 is characterized in that: The crack fusion assessment module is used to convert and align the spatial coordinates of the apparent cracks and internal cracks identification results, and output the crack risk level based on the fusion scoring function or neural network. Specifically, the image coordinate system detection results of the apparent cracks are projected into the geological radar coordinate system through the multi-sensor coordinate transformation matrix to achieve the unification of the spatial coordinates of the apparent and internal cracks. Then, the comprehensive risk value is calculated using the fusion scoring function based on the comprehensive detection confidence, crack size, spatial overlap and depth. The formula is: S = λ1·C img +λ2·C gpr +λ3·IoU 3D +λ4·D depth , where λ1, λ2, λ3, and λ4 are the corresponding adjustable fusion weight coefficients, C img 、C gpr Respectively, the apparent and internal recognition confidence, IoU 3D is the three-dimensional spatial overlap, D is the crack depth, and the input feature vector is automatically classified to output a structured assessment result containing location, type, and risk level information.

9. The bridge crack intelligent identification system based on image and radar multimodality according to claim 1 is characterized in that: The visualization and report output module is used to map the identification results to a three-dimensional bridge structure model and generate a structured inspection report. Specifically, the identification results of apparent cracks and internal cracks are superimposed on the corresponding positions of the bridge model in a three-dimensional visualization form to intuitively display the spatial distribution of cracks. At the same time, a structured report containing the crack ID, type, spatial location, length, width, depth, risk level and color labels is output. The color labels correspond to different warning colors according to the risk level, which facilitates engineering maintenance personnel to quickly locate and assess the condition of bridge diseases.

10. A bridge crack intelligent identification method based on image and radar multimodality, applicable to any of claims 1-9 above, characterized in that: The following steps are involved: S1. Collecting the surface image of the bridge structure through the image acquisition module and collecting the geological radar image inside the bridge structure through the radar data acquisition module; S2. The apparent crack recognition module is used to identify crack features in the apparent image. The multi-scale strip attention module performs attention enhancement in the horizontal and vertical directions to improve the modeling capability of slender cracks. The adaptive position query embedding module dynamically generates position-sensitive query vectors based on global features to improve crack localization accuracy. S3. The internal crack recognition module is used to identify the internal crack features in the geological radar image. The waveform-aware convolution module introduces a convolution kernel with a learnable offset to enhance the modeling capability of curved cracks. The edge-guided attention module fuses edge response and spatial attention features to improve the recognition capability of fuzzy crack areas. S4. The crack fusion assessment module performs spatial coordinate conversion and alignment on the apparent crack and internal crack identification results, outputs the crack risk level based on the fusion scoring function, and the visualization and report output module maps the identification results to the three-dimensional bridge structure model and generates a structured inspection report including the location, type, size, and risk level.

Images