Multi-modal bridge pier damage detection method and system based on image sonar adjustment

Abstract

The invention provides a multi-modal bridge pier damage detection method and system based on image sonar adjustment, and relates to an image processing method and system.The system comprises a holder used for carrying an image sonar; and receiving a stepping adjustment signal, and adjusting a sonar detection angle according to a preset stepping angle, the image sonar is used for collecting sonar images; the quality evaluation module is used for evaluating and obtaining at least one image index of the sonar image; the adjusting strategy module is used for gradually adjusting the sonar detection angle within the angle adjusting range according to a set stepping angle; all sonar images and corresponding image indexes in the angle adjustment range are obtained; selecting a sonar image with the image index or the maximum weighted sum of the image indexes; and the image processing module is used for processing the sonar image, and the damage identification module is used for carrying out bridge pier damage identification on the processed sonar image so as to complete bridge pier damage detection.

Description

Technical Field

[0001] The present invention relates to an image processing method and system, and in particular to a multi-modal bridge pier damage detection method and system based on image sonar adjustment. Background Art

[0002] In bridge structural health inspections, the integrity of piers, critical load-bearing structures, is directly related to the safe operation of the bridge. Existing inspection methods primarily rely on visual inspections using human divers or underwater robots equipped with optical cameras. However, in turbid waters or poor lighting conditions, optical image quality degrades significantly, making it difficult to accurately identify pier damage.

[0003] In recent years, imaging sonar technology has been increasingly used in underwater structure inspection due to its advantages in long-range imaging and penetration. However, limitations such as low resolution and significant geometric distortion limit its application in high-precision inspection. Furthermore, existing inspection systems lack effective fusion and calibration mechanisms for multimodal data (such as sonar images, optical images, and laser scales) in complex underwater environments, making it difficult to ensure the accuracy and consistency of inspection results. Summary of the Invention

[0004] In order to solve the technical problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a multi-modal bridge pier damage detection method and system based on image sonar adjustment, which improves the resolution of sonar images collected by image sonar, reduces geometric distortion, and increases detection accuracy and environmental adaptability. To achieve the above-mentioned object, the present invention provides a multimodal bridge pier damage detection method based on image sonar adjustment, comprising: Step S1, collecting sonar images of bridge piers; Step S2: evaluating and obtaining at least one image index of the sonar image; Step S3: comparing the image index with a corresponding image index threshold; When any image index is less than the corresponding image index threshold, the sonar detection angle is gradually adjusted at the set step angle, and steps S1 to S2 are executed after each adjustment of the sonar detection angle until all sonar images and corresponding image indicators within the angle adjustment range are obtained; Step S4: comparing the image index or the weighted sum of the image indexes of each sonar image, and selecting the sonar image with the largest image index or the weighted sum of the image indexes; Step S5: processing the sonar image; The processing includes at least one of reducing noise on the sonar image, performing information complementation and fusion enhancement on the sonar image through optical images, and correcting geometric distortion of the sonar image through distance data; Step S6: Identify bridge pier damage on the processed sonar image to complete the bridge pier damage detection.

[0005] The present invention also provides a multi-modal bridge pier damage detection system based on image sonar adjustment, comprising: The gimbal is used to carry the image sonar and receive the step adjustment signal to adjust the sonar detection angle according to the preset step angle; The gimbal serves as a motion carrier for multiple sensors, achieving flexible pointing of the sonar and camera through multi-degree-of-freedom rotation; the laser ruler independently provides a distance reference, and the three form multimodal detection capabilities through spatial calibration and time synchronization.

[0006] The image sonar is used to collect sonar images and send them to the quality assessment module; The quality assessment module is used to evaluate and obtain at least one image index of the sonar image and send it to the adjustment strategy module; The adjustment strategy module is used to compare the image index with the corresponding image index threshold; When any image indicator is less than the corresponding image indicator threshold, a step adjustment signal is sent to the pan-tilt head to gradually adjust the sonar detection angle by a set step angle within the angle adjustment range; and after each adjustment of the sonar detection angle, a sonar image is obtained by the image sonar and a corresponding image indicator is obtained by the quality assessment module until all sonar images and corresponding image indicators within the angle adjustment range are obtained; and, for comparing the image index or the weighted sum of the image indexes of each sonar image, selecting the sonar image with the largest image index or the weighted sum of the image indexes, and sending it to the image processing module; The image processing module is used to process the sonar image and send it to the damage identification module; The processing includes at least one of reducing noise on the sonar image, performing information complementation and fusion enhancement on the sonar image through optical images, and correcting geometric distortion of the sonar image through distance data; The damage identification module is used to identify bridge pier damage based on the processed sonar image to complete bridge pier damage detection.

[0007] According to a technical solution of the present invention, the adjustment strategy module is further configured to send the sonar detection angle corresponding to the sonar image with the largest image index or the weighted sum of the image indexes as the optimal detection angle to the pan / tilt platform; The pan-tilt platform is further used to receive the optimal detection angle and adjust to the optimal detection angle.

[0008] According to a technical solution of the present invention, the adjustment strategy module is further used to compare the image index with a corresponding image index threshold; When all image indicators are greater than the corresponding image indicator thresholds, the sonar image is sent to the image processing module.

[0009] According to a technical solution of the present invention, the image index includes at least one of gradient entropy and edge clarity.

[0010] According to a technical solution of the present invention, the image processing module includes: The denoising unit is used to perform multi-layer decomposition of the sonar image using wavelet transform, extract the low-frequency approximate coefficients and high-frequency detail coefficients after each decomposition layer; and perform soft threshold denoising on all high-frequency detail coefficients, retaining the low-frequency approximate coefficients; then, using the low-frequency approximate coefficients and the denoised high-frequency detail coefficients, reconstruct the image through inverse wavelet transform to obtain a denoised sonar image.

[0011] According to a technical solution of the present invention, it also includes an optical camera: The optical camera is used to collect optical images of the same detection area while collecting sonar images; The image processing module further includes a fusion enhancement unit, which is used to: Extracting feature maps of the sonar image and the optical image through a convolutional neural network; Then, a sonar image key vector is obtained from the feature map of the sonar image, and an optical image query vector and an optical image value vector are obtained from the feature map of the optical image; and an attention weight is calculated using the sonar image key vector, the optical image query vector, and a scaling factor; The optical image value vector is then weighted by the attention weight and then fused with the feature map of the sonar image to obtain a fused sonar image.

[0012] According to a technical solution of the present invention, it also includes: Laser ruler, used to collect distance data from the same detection area while collecting sonar images; The image processing module further includes a geometric correction unit, which is used to: registering the distance data with the pixel coordinates of the sonar image; Then, the real coordinates corresponding to each pixel point in the sonar image in the distance data are calculated, and the pixel coordinates of the sonar image are transformed into real space coordinates to obtain a geometrically corrected sonar image.

[0013] According to a technical solution of the present invention, the damage identification module includes: The consistency verification unit is configured to detect damaged cracks in the fused sonar image and the corrected sonar image respectively, and extract center lines of the damaged cracks respectively; The Hausdorff spatial distance of the two center lines is recalculated, and a consistency score is obtained based on a size relationship between the Hausdorff spatial distance and a preset spatial distance threshold value; The accuracy prompting unit is configured to calculate a comprehensive confidence based on the consistency score and an image quality index. When the comprehensive confidence is greater than or equal to a preset confidence threshold value, it is determined that the pier is damaged.

[0014] According to one of the technical solutions of the present application, the method further comprises: The synchronization module is configured to synchronize the time of the gimbal, the image sonar, the optical camera and the laser ruler.

[0015] The present application provides a kind of based on image sonar adjustment multimodal pier damage detection method and system, compared with prior art, with the following beneficial effects: The resolution of the sonar image collected by the image sonar is improved, the geometric distortion is reduced, the detection accuracy and environmental adaptability are increased, and through the key technologies such as dynamic angle adjustment of the image sonar, cross-modal data fusion enhancement and laser ruler auxiliary geometric correction, high-precision and high-robustness detection of pier damage is realized, and the detection performance and automation level of the system in complex underwater environment are improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 The flow chart of the multimodal pier damage detection method based on image sonar adjustment according to one embodiment of the present application is schematically represented; Figure 2 The structural diagram of the multimodal pier damage detection system based on image sonar adjustment according to one embodiment of the present application is schematically represented. DETAILED DESCRIPTION

[0018] The description of the embodiments in this specification should be combined with the corresponding drawings, which should be considered a complete part of this specification. In the drawings, the shapes and thicknesses of the embodiments may be exaggerated and indicated for simplicity or convenience. Furthermore, the various structural components in the drawings will be described separately. It is worth noting that components not shown in the drawings or not described in words are known to those of ordinary skill in the art.

[0019] The description of the embodiments herein and any references to directions and orientations are for ease of description only and are not to be construed as limiting the scope of the present invention. The following description of the preferred embodiments may involve combinations of features, which may exist independently or in combination. The present invention is not specifically limited to the preferred embodiments. The scope of the present invention is defined by the claims.

[0020] like Figures 1-2 As shown, the present invention provides a multi-modal bridge pier damage detection method based on image sonar adjustment, comprising: Step S1, collecting sonar images of bridge piers; Step S2: evaluating and obtaining at least one image index of the sonar image; Step S3: comparing the image index with the corresponding image index threshold; When any image index is less than the corresponding image index threshold, the sonar detection angle is gradually adjusted at the set step angle, and steps S1 to S2 are executed after each adjustment of the sonar detection angle until all sonar images and corresponding image indicators within the angle adjustment range are obtained; Step S4: comparing the image index or the weighted sum of the image indexes of each sonar image, and selecting the sonar image with the largest image index or the weighted sum of the image indexes; Step S5: processing the sonar image; The processing includes at least one of reducing noise on the sonar image, enhancing information complementation and fusion of the sonar image through optical images, and correcting geometric distortion of the sonar image through distance data; Step S6: Identify bridge pier damage on the processed sonar image to complete the bridge pier damage detection.

[0021] The multi-modal bridge pier damage detection system based on image sonar adjustment includes: The gimbal is used to carry the image sonar and receive the step adjustment signal to adjust the sonar detection angle according to the preset step angle; Image sonar, used to collect sonar images and send them to the quality assessment module; a quality assessment module, configured to assess and obtain at least one image index of the sonar image and send the index to the adjustment strategy module; An adjustment strategy module is used to compare the image index with the corresponding image index threshold; When any image indicator is less than the corresponding image indicator threshold, a step adjustment signal is sent to the gimbal to gradually adjust the sonar detection angle by the set step angle within the angle adjustment range. After each adjustment of the sonar detection angle, a sonar image is obtained through the image sonar and the corresponding image indicator is obtained through the quality assessment module until all sonar images and corresponding image indicators within the angle adjustment range are obtained. and, for comparing the image index or the weighted sum of the image indexes of each sonar image, selecting the sonar image with the largest image index or the weighted sum of the image indexes, and sending it to the image processing module; Image processing module, used to process sonar images and send them to damage identification module; The processing includes at least one of reducing noise on the sonar image, enhancing information complementation and fusion of the sonar image through optical images, and correcting geometric distortion of the sonar image through distance data; The damage identification module is used to identify bridge pier damage based on the processed sonar images and complete the damage detection of bridge piers.

[0022] This embodiment mainly involves an image sonar enhancement module, an image processing module, and a damage recognition module. The image sonar enhancement module is used to improve the imaging quality of the image sonar in complex underwater environments, and mainly includes an adaptive angle adjustment unit.

[0023] The adaptive angle adjustment unit includes a pan / tilt platform, a quality assessment module and an adjustment strategy module.

[0024] In the adaptive angle adjustment unit, the image sonar is connected through the pan-tilt system. The quality assessment module analyzes the gradient entropy and edge clarity of the sonar image, and then dynamically adjusts the pan-tilt angle (sonar detection angle) through the adjustment strategy module to form an optimal angle between the incident direction of the sound wave and the direction of the crack on the pier surface, thereby enhancing the reflection signal strength in the crack area and improving the imaging clarity of the crack area.

[0025] The specific working principle of the adaptive angle adjustment unit is as follows: The quality assessment module performs image quality assessment: The system collects current sonar images in real time , calculate its image index, namely gradient entropy and edge clarity ,in:

[0026] in is the gradient histogram of the sonar image i The probability of gray levels, Nis the gradient histogram level.

[0027] Edge clarity The Laplacian operator is used to calculate the sharpness of the sonar image:

[0028] in, M is the total number of sonar image pixels.

[0029] The adjustment strategy module implements the following angle adjustment strategy: or ,in 、 If the preset threshold is reached, the gimbal pitch angle adjustment is started, and the angle is adjusted each time (Step angle), and re-collect sonar images, recording the angle after each adjustment and , choose so that The largest angle is taken as the optimal detection angle, where is the weighting coefficient. If the image index is only one indicator, the above angle adjustment strategy can be performed by comparing the size of this indicator alone.

[0030] In the process of image acquisition, a dynamic window mechanism can also be used to calculate the local edge density of the image. Dynamically resize the evaluation window:

[0031] in is the window area, is the edge detection threshold. > , then use a small window (such as ); otherwise use a large window (such as ).

[0032] The image processing module processes the enhanced sonar image accordingly and then uses the damage recognition module to identify damage.

[0033] In the multimodal bridge pier damage detection system based on image sonar adjustment, the adjustment strategy module is further used to send the sonar detection angle corresponding to the sonar image with the largest image index or the weighted sum of image indexes as the optimal detection angle to the pan-tilt head; The gimbal is also used to receive the optimal detection angle and adjust it to the optimal detection angle.

[0034] This embodiment mainly involves the control and coordination subsystems.

[0035] The control and coordination subsystem is responsible for achieving closed-loop control of the pan-tilt angle adjustment.

[0036] The control logic incorporates an adaptive adjustment strategy based on image quality feedback. When sonar image clarity falls below a threshold (the image quality assessment module indicates a decrease in image clarity), it automatically triggers a gimbal angle adjustment process to find the optimal detection angle, improving overall inspection efficiency and automation. This adjustment is followed by a reassessment, forming a closed-loop feedback loop to ensure the detection angle remains optimal.

[0037] In the multimodal bridge pier damage detection system based on image sonar adjustment, the adjustment strategy module is also used to compare the image index with the corresponding image index threshold; When all image indicators are greater than the corresponding image indicator threshold, the sonar image is sent to the image processing module.

[0038] In the multimodal bridge pier damage detection system based on image sonar adjustment, the image index includes at least one of gradient entropy and edge clarity.

[0039] In the multimodal pier damage detection system based on image sonar adjustment, the image processing module includes: The denoising unit is used to perform multi-layer decomposition of the sonar image using wavelet transform, extract the low-frequency approximate coefficients and high-frequency detail coefficients after each decomposition layer; and perform soft threshold denoising on all high-frequency detail coefficients, retaining the low-frequency approximate coefficients; then, using the low-frequency approximate coefficients and the denoised high-frequency detail coefficients, reconstruct the image through inverse wavelet transform to obtain a denoised sonar image.

[0040] In this embodiment, the image processing module includes a noise reduction unit (multi-scale feature extraction unit). The multi-scale feature extraction unit uses wavelet transform or CNN convolutional neural network to perform multi-scale decomposition on the sonar image, separating high-frequency noise from low-frequency structural features (crack characteristics), effectively enhancing and denoising crack information, and improving the image signal-to-noise ratio.

[0041] If wavelet transform is used, the process is as follows: Wavelet transform is used to decompose the sonar image into three layers to extract the low-frequency approximate coefficients and high frequency detail coefficient (horizontally, vertically, and diagonally), perform soft threshold denoising on high-frequency coefficients to retain low-frequency structural information. Reconstructed image for:

[0042] The multimodal bridge pier damage detection system based on image sonar adjustment also includes optical cameras: An optical camera is used to collect optical images of the same detection area while collecting sonar images; The image processing module also includes a fusion enhancement unit, which is used to: Extract feature maps of sonar images and optical images through convolutional neural networks; Then, the sonar image key vector is obtained from the feature map of the sonar image, and the optical image query vector and optical image value vector are obtained from the feature map of the optical image. The attention weight is calculated using the sonar image key vector, the optical image query vector, and the scaling factor. The optical image value vector is then weighted by the attention weight and fused with the feature map of the sonar image to obtain a fused sonar image.

[0043] In this embodiment, the image processing module also implements multimodal data fusion, achieving information complementarity and enhanced fusion between sonar and optical images. It includes a fusion enhancement unit (implementing a cross-modal attention mechanism). This cross-modal attention mechanism, based on the Transformer architecture, establishes a pixel-level mapping relationship between sonar and optical images. It leverages the high-resolution edge information in the optical image to guide local enhancement of the sonar image, enhancing blurred areas in the sonar image and improving the clarity of crack edges.

[0044] The process of implementing the cross-modal attention mechanism by fusing the enhancement unit is as follows: 1. Sonar images and optical images Use CNN to extract feature maps respectively 、

[0045] Attention weight calculation: Attention weight is calculated by dot product :

[0046] in are the query and key vectors for optical images and sonar images respectively, is the scaling factor.

[0047] 2. Weighted fusion of optical image features into sonar images:

[0048] in V is the value vector of the optical image. The final output is the fused image .

[0049] The multimodal pier damage detection system based on image sonar adjustment also includes: Laser ruler, used to collect distance data from the same detection area while collecting sonar images; The image processing module also includes a geometric correction unit, which is used to: Registering distance data with pixel coordinates of sonar images; Then calculate the corresponding coordinates of each pixel point in the sonar image, transform the pixel coordinates of the sonar image into spatial coordinates, and obtain the geometrically corrected sonar image.

[0050] In this embodiment, the image processing module further includes a geometric correction unit for implementing geometric correction including laser ruler guidance.

[0051] Based on the distance data collected by the laser ruler, the geometric correction unit dynamically corrects the geometric distortion in the sonar image caused by the delay in sound wave propagation, thereby improving the accuracy of crack size measurement and the measurement accuracy of crack length and width.

[0052] The geometric correction unit implements the geometric correction process as follows: 1. Laser ruler collects distance points on the pier surface , aligned with the sonar image coordinates through the ICP algorithm.

[0053] 2. For each pixel in the sonar image ( x, y ), calculate the real coordinates of the corresponding points ( ), build a mapping function T , transform the image coordinates into space coordinates:

[0054] Get the corrected image Satisfy spatial consistency.

[0055] In the multimodal pier damage detection system based on image sonar adjustment, the damage identification module includes: A consistency verification unit, used to detect damage cracks in the fused sonar image and the corrected sonar image, and to extract the center lines of the damage cracks respectively; Then calculate the Hausdorff distance between the two center lines, and obtain the consistency score based on the relationship between the Hausdorff distance and the preset distance threshold. An accuracy prompt unit is used to calculate a comprehensive confidence level based on the consistency score and image quality indicators; And when the comprehensive confidence is greater than or equal to the preset confidence threshold, it is determined that the bridge pier is damaged.

[0056] This embodiment mainly involves a damage identification and accuracy assessment module.

[0057] The damage identification and accuracy assessment module is used to identify bridge pier damage based on the fused image data and to assess the credibility of the detection results.

[0058] The consistency verification unit implements bimodal consistency verification, comparing the location and direction of cracks in sonar and optical images to calculate a consistency score, which serves as the basis for confidence. This bimodal consistency verification mechanism improves the accuracy and reliability of damage identification, making it suitable for complex underwater pier inspection scenarios such as turbid waters and poor lighting.

[0059] The accuracy prompt unit determines the credibility of the current detection result based on the image quality index and the bimodal consistency result. If the crack position deviation exceeds the set tolerance threshold (such as 5%), it will be automatically marked as "low credibility" and trigger a manual review mechanism to ensure the reliability of the detection result. The consistency verification unit implements the bimodal consistency verification process as follows: 1. In the fusion image and geometrically corrected images Detect cracks separately and extract crack center lines and .

[0060] 2. Calculate the Hausdorff space distance between two center lines :

[0061] like < , the consistency score is high, otherwise it is low. The consistency score threshold is determined based on actual needs.

[0062] The process of the accuracy prompt unit to implement dynamic accuracy prompts is as follows: 1. Scoring based on consistency and image quality indicators , calculate the comprehensive confidence C :

[0063] in is the weight factor.

[0064] 2. If (preset confidence threshold), it will be marked as "low confidence" and prompted for manual review.

[0065] The multi-modal bridge pier damage detection system based on image sonar adjustment also includes: The synchronization module is used to synchronize the time of the pan / tilt, image sonar, optical camera and laser ruler.

[0066] In this embodiment, the control and coordination subsystem is also responsible for coordinating the synchronous acquisition and data alignment between the multi-sensor such as image sonar, optical camera, laser range finder, etc. Through timestamp synchronization and spatial coordinate registration technology, the time consistency of image sonar, optical camera, laser range finder and the angle adjustment of the gimbal is ensured, and the precise time and spatial consistency of the multi-modal data during acquisition is ensured.

[0067] Implementation data: In a certain underwater pier detection task, the system successfully identified a crack with a width of less than 10 cm in turbid water, and the crack length measurement error was controlled within ±1.2% through laser range finder correction. Dynamic angle adjustment improves image clarity by 32%, cross-modal fusion enhances crack contrast by 45%, and overall detection efficiency improves by 40%.

[0068] According to one aspect of the present application, a computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement a multi-modal pier damage detection method based on image sonar adjustment as in the above technical solution.

[0069] The computer-readable storage medium can include any medium capable of storing or transmitting information. Examples of computer-readable storage media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, and the like. Code segments can be downloaded via a computer network such as the Internet, an intranet, etc.

[0070] The multi-modal pier damage detection method and system based on image sonar adjustment of the present application, the system comprises: a gimbal for carrying an image sonar; and a receiving step adjustment signal, adjusting the sonar detection angle according to the preset step angle; an image sonar for acquiring sonar images; a quality assessment module for evaluating and obtaining at least one image index of the sonar images; an adjustment strategy module for adjusting the sonar detection angle step by step within the angle adjustment range at a set step angle; obtaining all sonar images within the angle adjustment range and corresponding image indexes; and selecting the sonar image with the maximum image index or image index weighted sum; an image processing module for processing the sonar images; a damage recognition module for recognizing the pier damage based on the processed sonar images, and completing the pier damage detection.

[0071] In addition, it should be noted that the present application can be provided as a method, device or computer program product. Therefore, the embodiments of the present application can adopt a completely hardware embodiment, a completely software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer usable storage media containing computer usable program code.

[0072] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0073] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0074] It should also be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal device comprising the element.

[0075] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be noted that although the preferred embodiment of the present invention has been described, it is clear that those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered as within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as covering the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A multimodal bridge pier damage detection method based on image sonar adjustment, characterized in that: include: Step S1, collecting sonar images of bridge piers; Step S2: evaluating and obtaining at least one image index of the sonar image; Step S3: comparing the image index with a corresponding image index threshold; When any image index is less than the corresponding image index threshold, the sonar detection angle is gradually adjusted at the set step angle, and steps S1 to S2 are executed after each adjustment of the sonar detection angle until all sonar images and corresponding image indicators within the angle adjustment range are obtained; Step S4: comparing the image index or the weighted sum of the image indexes of each sonar image, and selecting the sonar image with the largest image index or the weighted sum of the image indexes; Step S5: processing the sonar image; The processing includes at least one of reducing noise on the sonar image, performing information complementation and fusion enhancement on the sonar image through an optical image, and correcting geometric distortion of the sonar image through distance data of a laser rangefinder; Step S6: Identify bridge pier damage on the processed sonar image to complete the bridge pier damage detection.

2. A multi-modal bridge pier damage detection system based on image sonar adjustment, characterized in that: include: The gimbal is used to carry the image sonar and receive the step adjustment signal to adjust the sonar detection angle according to the preset step angle; The image sonar is used to collect sonar images and send them to the quality assessment module; The quality assessment module is used to evaluate and obtain at least one image index of the sonar image and send it to the adjustment strategy module; The adjustment strategy module is used to compare the image index with the corresponding image index threshold; When any image indicator is less than the corresponding image indicator threshold, a step adjustment signal is sent to the pan-tilt head to gradually adjust the sonar detection angle by a set step angle within the angle adjustment range; and after each adjustment of the sonar detection angle, a sonar image is obtained by the image sonar and a corresponding image indicator is obtained by the quality assessment module until all sonar images and corresponding image indicators within the angle adjustment range are obtained; and, for comparing the image index or the weighted sum of the image indexes of each sonar image, selecting the sonar image with the largest image index or the weighted sum of the image indexes, and sending it to the image processing module; The image processing module is used to process the sonar image and send it to the damage identification module; The processing includes at least one of reducing noise on the sonar image, performing information complementation and fusion enhancement on the sonar image through optical images, and correcting geometric distortion of the sonar image through distance data of a laser ruler; The damage identification module is used to identify bridge pier damage based on the processed sonar image to complete bridge pier damage detection.

3. The multi-modal bridge pier damage detection system based on image sonar adjustment according to claim 2 is characterized in that: The adjustment strategy module is further configured to send the sonar detection angle corresponding to the sonar image with the largest image index or the weighted sum of the image indexes as the optimal detection angle to the pan / tilt platform; The pan-tilt platform is further used to receive the optimal detection angle and adjust to the optimal detection angle.

4. The multi-modal bridge pier damage detection system based on image sonar adjustment according to claim 2 or 3 is characterized in that: The adjustment strategy module is further configured to compare the image index with a corresponding image index threshold; When all image indicators are greater than the corresponding image indicator thresholds, the sonar image is sent to the image processing module.

5. The multi-modal bridge pier damage detection system based on image sonar adjustment according to claim 4 is characterized in that: The image index includes at least one of gradient entropy and edge sharpness.

6. The multi-modal bridge pier damage detection system based on image sonar adjustment according to claim 5 is characterized in that: The image processing module includes: The denoising unit is used to perform multi-layer decomposition of the sonar image using wavelet transform, extract the low-frequency approximate coefficients and high-frequency detail coefficients after each decomposition layer; and perform soft threshold denoising on all high-frequency detail coefficients, retaining the low-frequency approximate coefficients; then, using the low-frequency approximate coefficients and the denoised high-frequency detail coefficients, reconstruct the image through inverse wavelet transform to obtain a denoised sonar image.

7. The multi-modal bridge pier damage detection system based on image sonar adjustment according to claim 6 is characterized in that: Also includes optical cameras: The optical camera is used to collect optical images of the same detection area while collecting sonar images; The image processing module further includes a fusion enhancement unit, which is used to: Extracting feature maps of the sonar image and the optical image through a convolutional neural network; Then, a sonar image key vector is obtained from the feature map of the sonar image, and an optical image query vector and an optical image value vector are obtained from the feature map of the optical image; and an attention weight is calculated using the sonar image key vector, the optical image query vector, and a scaling factor; The optical image value vector is then weighted by the attention weight and then fused with the feature map of the sonar image to obtain a fused sonar image.

8. The multi-modal bridge pier damage detection system based on image sonar adjustment according to claim 6 or 7, characterized in that: Also includes; Laser ruler, used to collect distance data from the same detection area while collecting sonar images; The image processing module further includes a geometric correction unit, which is used to: registering the crack distance data with the pixel coordinates of the sonar image; Then, the real coordinates corresponding to each pixel point in the sonar image in the distance data are calculated, and the pixel coordinates of the sonar image are transformed into real space coordinates to obtain a geometrically corrected sonar image.

9. The multi-modal bridge pier damage detection system based on image sonar adjustment according to claim 8 is characterized in that: The damage identification module includes: A consistency verification unit, used to detect damage cracks in the fused sonar image and the corrected sonar image, and to extract the center lines of the damage cracks respectively; Then, the Hausdorff spatial distance between the two center lines is calculated, and a consistency score is obtained based on the relationship between the Hausdorff spatial distance and a preset spatial distance threshold; an accuracy prompting unit, configured to calculate a comprehensive confidence level based on the consistency score and the image quality index; And when the comprehensive confidence is greater than or equal to a preset confidence threshold, it is determined that the bridge pier is damaged.

10. The multi-modal bridge pier damage detection system based on image sonar adjustment according to claim 8 is characterized in that: Also includes: The synchronization module is used to synchronize the time of the pan / tilt, image sonar, optical camera and laser ruler.

Images