Bridge engineering underwater foundation quality detection method and system based on machine vision

Abstract

This invention relates to the field of machine vision technology and discloses a machine vision-based method and system for inspecting the quality of underwater foundations in bridge engineering. The method involves acquiring visible light, near-infrared, and low-light images, fusing them to obtain the original image, performing enhancement processing to obtain a preprocessed image, adding a dual-branch attention module to the multi-head attention mechanism of the ViT model to extract defect features from the preprocessed image, inputting the defect features into a YOLOv8 model including deformable convolution modules and weighted feature fusion, and using the EIOU loss function to output the defect's category label and bounding box coordinates, performing pixel-level segmentation using the U-Net++ segmentation model based on the defect's category label and bounding box coordinates to obtain the defect contour, and calculating the defect's physical parameters by combining camera calibration parameters, associating the defect's physical parameters with a BIM model, and generating a visual report. This invention improves detection efficiency.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a machine vision-based method and system for inspecting the quality of underwater foundations for bridge engineering. Background Technology

[0002] Bridges are widely used in highway, water conservancy, and municipal engineering projects, and bridge construction is receiving increasing attention. Underwater bridge foundations are the core of the bridge structure, and are susceptible to defects such as cracks and exposed reinforcement due to long-term exposure to water flow, microbial erosion, and water quality changes, directly threatening bridge safety. Current detection technologies, such as sonar, have low resolution and cannot identify minute cracks less than 2mm wide; single-camera underwater robots rely on manual interpretation, resulting in blurred images and high false negative rates in low-light and high-turbidity environments; traditional machine vision methods use single enhancement algorithms, which are poorly adaptable to complex water environments and lack accurate defect feature extraction, making it difficult to support subsequent quantitative analysis. As bridges age, the requirements for detection accuracy and efficiency increase, necessitating a detection method that can adapt to complex underwater environments and achieve efficient defect identification. Summary of the Invention

[0003] The purpose of this invention is to solve the above-mentioned problems by designing a machine vision-based method and system for underwater foundation quality inspection of bridge engineering.

[0004] The first aspect of this invention provides a machine vision-based method for inspecting the quality of underwater foundations in bridge engineering, the method comprising the following steps:

[0005] An underwater robot equipped with a high-definition camera, a multispectral camera, and a low-light camera simultaneously acquires visible light images, near-infrared images, and low-light images, which are then fused to obtain the original image.

[0006] The low-turbidity water areas in the original image are enhanced by contrast enhancement and sharpening, and the high-turbidity water areas in the original image are enhanced by a dark channel prior-based dehazing algorithm and a multi-scale Retinex algorithm, resulting in a preprocessed image.

[0007] A dual-branch attention module is added to the multi-head attention mechanism of the ViT model to extract defect features from the preprocessed image;

[0008] The defect features are input into a YOLOv8 model that includes deformable convolutional modules and weighted feature fusion, and the EIOU loss function is used to output the defect category label and bounding box coordinates.

[0009] The U-Net++ segmentation model is used to perform pixel-level segmentation based on the defect category label and bounding box coordinates to obtain the defect contour. The physical parameters of the defect are then calculated in combination with the camera calibration parameters.

[0010] The physical parameters of the defects are associated with the BIM model, and a visualization report is generated.

[0011] Optionally, in a first implementation of the first aspect of the present invention, the underwater robot equipped with a high-definition camera, a multispectral camera, and a low-light camera simultaneously acquires visible light images, near-infrared images, and low-light images, and then fuses them to obtain the original image, including:

[0012] The SIFT algorithm is used to register the acquired visible light images, near-infrared images and low-light images. By calculating the feature point coordinate deviation between the images, the pixel coordinates of the near-infrared images and low-light images are calibrated to the coordinate system of the visible light images.

[0013] The information entropy of visible light images, near-infrared images, and low-light images is calculated by using grayscale variance, and the proportion of information entropy is used as the weight coefficient of the corresponding image.

[0014] The registered images are fused pixel-level based on weighting coefficients, and the fused original image is output.

[0015] Optionally, in a second implementation of the first aspect of the present invention, the enhancement processing of the low-turbidity water area in the original image using contrast enhancement and sharpening includes:

[0016] An adaptive histogram equalization algorithm is used to divide the low-turbidity water image into 16×16 sub-blocks, and a histogram is constructed and equalization is performed on each sub-block.

[0017] A 3×3 sharpening template is constructed using the Laplacian operator. The sharpening template is then convolved with the contrast-enhanced low-turbidity water image to calculate the grayscale difference between each pixel and its neighboring pixels. The grayscale difference is then superimposed onto the original pixel's grayscale value.

[0018] Optionally, in a third implementation of the first aspect of the present invention, the enhancement processing of the high-turbidity water area in the original image using a dark channel prior-based dehazing algorithm and a multi-scale Retinex algorithm includes:

[0019] Traverse each 15×15 local region of the high turbidity water image, extract the dark channel value of each region and obtain the minimum value, and calculate the dehazed image based on the minimum value;

[0020] The dehazed high-turbidity water area image was convolved with three Gaussian filter kernels of different scales to obtain three sets of illumination components of different scales. The enhanced high-turbidity water area image was obtained by subtracting the illumination components of each scale from the dehazed high-turbidity water area image and then performing a weighted summation.

[0021] Optionally, in a fourth implementation of the first aspect of the present invention, the addition of a dual-branch attention module to the multi-head attention mechanism of the ViT model to extract defect features from the preprocessed image includes:

[0022] The preprocessed image is divided into 16×16 pixel blocks, and each pixel block is used as an image token and input into the ViT model;

[0023] In the dual-branch attention module, the local branch calculates the similarity between the feature vector of each token and the token features in the 3×3 neighborhood to obtain the local attention weight.

[0024] In the dual-branch attention module, the global branch performs global pooling on the feature vectors of all tokens and then calculates the similarity to obtain the global attention weights.

[0025] The local attention weights and global attention weights are fused to obtain the final attention weight matrix;

[0026] The token feature vector is weighted and summed using the attention weight matrix, and then input into the fully connected layer for feature transformation, thus completing one operation of the encoder layer.

[0027] The output of the encoder layer is used as the input of the next layer. Attention calculation and feature transformation operations are repeated. Global average pooling is then performed on the final output to obtain the defect features.

[0028] Optionally, in a fifth implementation of the first aspect of the present invention, the step of inputting the defect features based on a YOLOv8 model including deformable convolutional modules and weighted feature fusion, and outputting the defect category label and bounding box coordinates using the EIOU loss function, includes:

[0029] The extracted defect features are converted into three-dimensional feature maps and input into the YOLOv8 model. The YOLOv8 model embeds deformable convolutional modules in the C2f module of the backbone network. By dynamically adjusting the sampling position of the convolutional kernels, it captures the irregular shape features of the defect region.

[0030] The 8×8, 16×16, and 32×32 scale feature maps output from the backbone network are input into the neck network. Weights are assigned to the channel dimensions of each scale feature map, and then feature fusion is performed through upsampling and downsampling operations.

[0031] The fused feature map is input into the detection head module. The classification branch calculates the probability value of the defect belonging to the categories of crack, erosion, exposed reinforcement, and spalling through a fully connected layer and the Softmax function, while the regression branch predicts the coordinate offset and width and height scaling factor of the defect bounding box.

[0032] The EIOU loss function is used to calculate the loss value between the predicted bounding box and the ground truth bounding box, and finally outputs the defect category label and bounding box coordinates.

[0033] Optionally, in the sixth implementation of the first aspect of the present invention, the defect-based category label and bounding box coordinates are segmented at the pixel level using the U-Net++ segmentation model to obtain the defect contour, and the defect physical parameters are calculated in combination with the camera calibration parameters, including:

[0034] Based on the defect category label and bounding box coordinates, a local image containing the complete defect is cropped from the preprocessed image and input into the U-Net++ segmentation model;

[0035] The encoder downsamples local images to extract defect features at different scales, while the decoder upsamples the images to fuse deep and shallow features.

[0036] The Sigmoid activation function is used in the output layer to generate a binary segmentation map of the defect region, and the binary segmentation map is then post-processed.

[0037] The camera calibration parameters of the camera are obtained, and the detection distance of the underwater robot's attitude sensor is combined to establish a mapping relationship between pixel coordinates and actual physical coordinates. The corresponding physical parameters are then calculated based on the type of defect.

[0038] A second aspect of the present invention provides a machine vision-based underwater foundation quality inspection system for bridge engineering, the system comprising:

[0039] The image acquisition module is used to simultaneously acquire visible light images, near-infrared images, and low-light images using an underwater robot equipped with a high-definition camera, a multispectral camera, and a low-light camera, and then fuse them to obtain the original image.

[0040] The image processing module is used to enhance the low-turbidity water areas in the original image by employing contrast enhancement and sharpening processing, and to enhance the high-turbidity water areas in the original image by employing a dark channel prior-based dehazing algorithm and a multi-scale Retinex algorithm, thereby obtaining a preprocessed image.

[0041] The feature extraction module is used to add a dual-branch attention module to the multi-head attention mechanism of the ViT model to extract defect features from the preprocessed image;

[0042] The defect output module is used to input the defect features into a YOLOv8 model that includes deformable convolutional modules and weighted feature fusion, and output the defect category label and bounding box coordinates using the EIOU loss function.

[0043] The parameter calculation module is used to perform pixel-level segmentation using the U-Net++ segmentation model based on the defect category label and bounding box coordinates to obtain the defect contour, and calculate the physical parameters of the defect in combination with the camera calibration parameters.

[0044] The visualization module is used to associate the physical parameters of the defects with the BIM model and generate a visualization report.

[0045] A third aspect of the present invention provides a machine vision-based underwater foundation quality inspection device for bridge engineering, the machine vision-based underwater foundation quality inspection device comprising a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to cause the machine vision-based underwater foundation quality inspection device for bridge engineering to perform the various steps of the machine vision-based underwater foundation quality inspection method for bridge engineering as described in any of the preceding claims.

[0046] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the machine vision-based underwater foundation quality inspection method for bridge engineering as described in any of the preceding claims.

[0047] The technical solution provided by this invention uses an underwater robot equipped with a high-definition camera, a multispectral camera, and a low-light camera to simultaneously acquire visible light images, near-infrared images, and low-light images, which are then fused to obtain the original image. Contrast enhancement and sharpening are used to enhance low-turbidity water areas in the original image, while a dehazing algorithm based on dark channel priors and a multi-scale Retinex algorithm are used to enhance high-turbidity water areas, resulting in a preprocessed image. A dual-branch attention module is added to the multi-head attention mechanism of the ViT model to extract defect features from the preprocessed image. The defect features are input into a YOLOv8 model that includes deformable convolutional modules and weighted feature fusion, and the EIOU loss function is used to output the defect category label and bounding box coordinates. Based on the defects... The category labels and bounding box coordinates are segmented at the pixel level using the U-Net++ segmentation model to obtain the defect contour. The physical parameters of the defect are calculated by combining the camera calibration parameters. The physical parameters of the defect are then associated with the BIM model to generate a visualization report. This invention solves the problem of insufficient information from a single image by using multi-source image fusion. The scene-specific enhancement strategy is adapted to waters with different turbidity levels, which greatly improves image quality. The dual-branch attention ViT model enhances defect feature extraction. Combined with the improved YOLOv8 model, the accuracy of defect identification is improved and the rate of missed detection of minor cracks is reduced. The combination of U-Net++ segmentation and camera calibration enables accurate quantification of defect parameters. The defect information is associated with the BIM model to generate a visualization report, replacing manual interpretation, improving detection efficiency, providing accurate data support for bridge maintenance, and reducing safety hazards. Attached Figure Description

[0048] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0049] Figure 1 A flowchart of a machine vision-based underwater foundation quality inspection method for bridge engineering provided in an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of the structure of a machine vision-based underwater foundation quality inspection system for bridge engineering provided in an embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram of the structure of a machine vision-based underwater foundation quality inspection device for bridge engineering, provided in an embodiment of the present invention. Detailed Implementation

[0052] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0053] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The flowchart of the underwater foundation quality inspection method for bridge engineering based on machine vision provided in this embodiment of the invention includes the following steps:

[0054] Step 101: Using an underwater robot equipped with a high-definition camera, a multispectral camera, and a low-light camera, visible light images, near-infrared images, and low-light images are simultaneously acquired and fused to obtain the original image;

[0055] In this embodiment, the SIFT algorithm is used to register the acquired visible light image, near-infrared image, and low-light image. By calculating the feature point coordinate deviation between the images, the pixel coordinates of the near-infrared image and the low-light image are calibrated to the coordinate system of the visible light image. The information entropy of the visible light image, near-infrared image, and low-light image is calculated by the gray-level variance, and the proportion of information entropy is used as the weight coefficient of the corresponding image. Based on the weight coefficient, the registered images are fused at the pixel level, and the fused original image is output.

[0056] In this embodiment, the SIFT algorithm is used to register the three types of images. First, Gaussian difference pyramids are constructed for the visible light, near-infrared, and low-light images respectively to generate a multi-scale space to detect stable feature points. Then, through key point localization and orientation assignment, a 128-dimensional descriptor is assigned to each feature point. This descriptor can effectively characterize the gray-scale texture information around the feature point. Subsequently, the similarity of feature point descriptors between different images is calculated by Euclidean distance to initially match feature point pairs. Then, the RANSAC algorithm is introduced to remove mismatched points and retain reliable matching pairs that satisfy the fundamental matrix constraints. Based on these matching pairs, the feature point coordinate deviations of the near-infrared and low-light images relative to the visible light image are calculated, and a coordinate transformation model is established. Finally, this model is used to calibrate all pixel coordinates of the near-infrared and low-light images to the coordinate system of the visible light image, realizing the spatial alignment of the three images.

[0057] The information entropy of the three images is calculated using grayscale variance. First, the registered visible light, near-infrared, and low-light images are all converted into single-channel grayscale images. Each image is divided into several non-overlapping 8×8 sub-blocks, and the grayscale variance of each sub-block is calculated. Valid sub-blocks with variances greater than a set threshold are selected to eliminate interference from uniform regions. Then, the information entropy of each valid sub-block is calculated based on the Shannon entropy formula. By statistically analyzing the probability of each gray level within a sub-block, the negative value of the sum of the logarithm of the probability and the product of the probabilities is taken as the sub-block entropy value. The total information entropy of a single image is obtained by summing the entropy values ​​of all valid sub-blocks. Finally, the proportion of the total information entropy of each image to the sum of the total information entropy of the three images is calculated. This proportion is the weight coefficient of the corresponding image during fusion; the image with higher information entropy has a greater weight.

[0058] Pixel-level fusion is performed on the registered images based on weighting coefficients. First, it is confirmed that the three images are completely spatially aligned, and each pixel has a unique corresponding position in the three images. For each pixel in the image, its grayscale value in the visible light, near-infrared, and low-light images is extracted respectively. Each grayscale value is multiplied by the weighting coefficient of its corresponding image to obtain three weighted grayscale values. Then, these three weighted values ​​are summed to obtain the fused grayscale value of the pixel. If it is a color image, the above weighted fusion operation is performed on the RGB three channels respectively. During the fusion process, the range of pixel grayscale values ​​is monitored in real time, and values ​​exceeding 0-255 are truncated to ensure that the image grayscale values ​​are valid. After traversing all pixels to complete the fusion, the fused original image that combines the advantages of each image is output.

[0059] Step 102: Enhance the low-turbidity water area in the original image by contrast enhancement and sharpening, and enhance the high-turbidity water area in the original image by dark channel prior dehazing algorithm and multi-scale Retinex algorithm to obtain the preprocessed image.

[0060] In this embodiment, an adaptive histogram equalization algorithm is used to divide the low-turbidity water image into 16×16 sub-blocks, construct a histogram for each sub-block and perform equalization operation; a 3×3 sharpening template is constructed using the Laplacian operator, and the sharpening template is convolved with the contrast-enhanced low-turbidity water image to calculate the gray-level difference value between each pixel and its neighboring pixels, and the gray-level difference value is superimposed on the original pixel gray-level value.

[0061] In this embodiment, each 15×15 local region of the high-turbidity water image is traversed, the dark channel value of each region is extracted and the minimum value is obtained. The dehazed image is calculated based on the minimum value. The dehazed high-turbidity water image is convolved with three Gaussian filter kernels of different scales to obtain three sets of illumination components of different scales. The illumination components of each scale are subtracted from the dehazed high-turbidity water image and then weighted and summed to obtain the enhanced high-turbidity water image.

[0062] Step 103: Add a dual-branch attention module to the multi-head attention mechanism of the ViT model to extract defect features from the preprocessed image;

[0063] In this embodiment, the preprocessed image is segmented into 16×16 pixel blocks, with each pixel block serving as an image token, which is then input into the ViT model. In the dual-branch attention module, the local branch calculates the similarity between the feature vector of each token and the token features within a 3×3 neighborhood, yielding local attention weights. The global branch in the dual-branch attention module performs global pooling on the feature vectors of all tokens and calculates their similarity, resulting in global attention weights. The local and global attention weights are then fused to obtain the final attention weight matrix. The token feature vectors are weighted and summed using the attention weight matrix, and then input into the fully connected layer for feature transformation, completing one encoder layer operation. The output of the encoder layer is used as the input to the next layer, and the attention calculation and feature transformation operations are repeated. The final output is then subjected to global average pooling to obtain defect features.

[0064] Step 104: Input the defect features into the YOLOv8 model, which includes deformable convolutional modules and weighted feature fusion, and use the EIOU loss function to output the defect category label and bounding box coordinates;

[0065] In this embodiment, the extracted defect features are converted into three-dimensional feature maps and input into the YOLOv8 model. The YOLOv8 model embeds a deformable convolutional module in the C2f module of the backbone network, which captures the irregular shape features of the defect region by dynamically adjusting the sampling position of the convolutional kernel. The 8×8, 16×16, and 32×32 scale feature maps output by the backbone network are input into the neck network. The channel dimension of each scale feature map is weighted, and then feature fusion is performed through upsampling and downsampling operations. The fused feature map is input into the detection head module. The classification branch calculates the probability value of the defect belonging to the category of crack, erosion, exposed reinforcement, and spalling through a fully connected layer and the Softmax function. The regression branch predicts the coordinate offset and width and height scaling factor of the defect bounding box. The EIOU loss function is used to calculate the loss value between the predicted bounding box and the ground truth bounding box. Finally, the category label of the defect and the bounding box coordinates are output.

[0066] Step 105: Based on the defect category label and bounding box coordinates, the U-Net++ segmentation model is used to perform pixel-level segmentation to obtain the defect contour, and the physical parameters of the defect are calculated in combination with the camera calibration parameters.

[0067] In this embodiment, based on the defect category label and bounding box coordinates, a local image containing the complete defect is cropped from the preprocessed image and input into the U-Net++ segmentation model. The local image is downsampled by the encoder to extract defect features at different scales, while the decoder is upsampled to fuse deep and shallow features. A binary segmentation map of the defect region is generated using the Sigmoid activation function in the output layer, and the binary segmentation map is post-processed. The camera calibration parameters of the camera are obtained, and combined with the detection distance of the underwater robot's attitude sensor, a mapping relationship between pixel coordinates and actual physical coordinates is established. The corresponding physical parameters are calculated according to the defect category.

[0068] Step 106: Associate the physical parameters of the defects with the BIM model and generate a visualization report.

[0069] Please see Figure 2 A schematic diagram of a machine vision-based underwater foundation quality inspection system for bridge engineering provided in this embodiment of the invention. The system includes:

[0070] The image acquisition module is used to simultaneously acquire visible light images, near-infrared images, and low-light images using an underwater robot equipped with a high-definition camera, a multispectral camera, and a low-light camera, and then fuse them to obtain the original image.

[0071] The image processing module is used to enhance the low-turbidity water areas in the original image by using contrast enhancement and sharpening processing, and to enhance the high-turbidity water areas in the original image by using a dark channel prior-based dehazing algorithm and a multi-scale Retinex algorithm, so as to obtain the preprocessed image.

[0072] The feature extraction module is used to add a dual-branch attention module to the multi-head attention mechanism of the ViT model to extract defect features from the preprocessed image;

[0073] The defect output module is used to input defect features into a YOLOv8 model that includes deformable convolutional modules and weighted feature fusion, and output the defect category label and bounding box coordinates using the EIOU loss function.

[0074] The parameter calculation module is used to perform pixel-level segmentation using the U-Net++ segmentation model based on the defect category label and bounding box coordinates to obtain the defect contour, and calculate the physical parameters of the defect in combination with the camera calibration parameters.

[0075] The visualization module is used to associate the physical parameters of defects with the BIM model and generate visualization reports.

[0076] Figure 3This is a schematic diagram of a machine vision-based underwater foundation quality inspection device for bridge engineering, provided in an embodiment of the present invention. The machine vision-based underwater foundation quality inspection device 300 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the machine vision-based underwater foundation quality inspection device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the machine vision-based underwater foundation quality inspection device 300 to implement the method provided in the above embodiment.

[0077] The machine vision-based underwater foundation quality inspection equipment 300 for bridge engineering may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The structure of the underwater foundation quality inspection equipment for bridge engineering based on machine vision shown does not constitute a limitation on the computer equipment provided by the present invention. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0078] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the various steps of the machine vision-based underwater foundation quality inspection method for bridge engineering provided in the above embodiments.

[0079] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the above-described equipment, apparatus, or unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A machine vision-based method for underwater foundation quality inspection of bridge engineering, characterized in that, The method includes the following steps: An underwater robot equipped with a high-definition camera, a multispectral camera, and a low-light camera simultaneously acquires visible light images, near-infrared images, and low-light images, which are then fused to obtain the original image. The low-turbidity water areas in the original image are enhanced by contrast enhancement and sharpening, and the high-turbidity water areas in the original image are enhanced by a dark channel prior-based dehazing algorithm and a multi-scale Retinex algorithm, resulting in a preprocessed image. A dual-branch attention module is added to the multi-head attention mechanism of the ViT model to extract defect features from the preprocessed image; The defect features are input into a YOLOv8 model that includes deformable convolutional modules and weighted feature fusion, and the EIOU loss function is used to output the defect category label and bounding box coordinates. The U-Net++ segmentation model is used to perform pixel-level segmentation based on the defect category label and bounding box coordinates to obtain the defect contour. The physical parameters of the defect are then calculated in combination with the camera calibration parameters. The physical parameters of the defects are associated with the BIM model, and a visualization report is generated. The defect-based category labels and bounding box coordinates are segmented at the pixel level using the U-Net++ segmentation model to obtain the defect contour. The physical parameters of the defect are then calculated using camera calibration parameters, including: Based on the defect category label and bounding box coordinates, a local image containing the complete defect is cropped from the preprocessed image and input into the U-Net++ segmentation model; The encoder downsamples local images to extract defect features at different scales, while the decoder upsamples the images to fuse deep and shallow features. The Sigmoid activation function is used in the output layer to generate a binary segmentation map of the defect region, and the binary segmentation map is then post-processed. The camera calibration parameters of the camera are obtained, and the detection distance of the underwater robot's attitude sensor is combined to establish a mapping relationship between pixel coordinates and actual physical coordinates. The corresponding physical parameters are then calculated based on the type of defect.

2. The machine vision-based underwater foundation quality inspection method for bridge engineering as described in claim 1, characterized in that, The underwater robot, equipped with a high-definition camera, a multispectral camera, and a low-light camera, simultaneously acquires visible light images, near-infrared images, and low-light images, which are then fused to obtain the original image, including: The SIFT algorithm is used to register the acquired visible light images, near-infrared images and low-light images. By calculating the feature point coordinate deviation between the images, the pixel coordinates of the near-infrared images and low-light images are calibrated to the coordinate system of the visible light images. The information entropy of visible light images, near-infrared images, and low-light images is calculated by using grayscale variance, and the proportion of information entropy is used as the weight coefficient of the corresponding image. The registered images are fused pixel-level based on weighting coefficients, and the fused original image is output.

3. The machine vision-based underwater foundation quality inspection method for bridge engineering as described in claim 1, characterized in that, The enhancement process for low-turbidity water areas in the original image using contrast enhancement and sharpening includes: An adaptive histogram equalization algorithm is used to divide the low-turbidity water image into 16×16 sub-blocks, and a histogram is constructed and equalization is performed on each sub-block. A 3×3 sharpening template is constructed using the Laplacian operator. The sharpening template is then convolved with the contrast-enhanced low-turbidity water image to calculate the grayscale difference between each pixel and its neighboring pixels. The grayscale difference is then superimposed onto the original pixel's grayscale value.

4. The machine vision-based underwater foundation quality inspection method for bridge engineering as described in claim 1, characterized in that, The enhancement process for high-turbidity water areas in the original image using a dark channel prior-based dehazing algorithm and a multi-scale Retinex algorithm includes: Traverse each 15×15 local region of the high turbidity water image, extract the dark channel value of each region and obtain the minimum value, and calculate the dehazed image based on the minimum value; The dehazed high-turbidity water area image was convolved with three Gaussian filter kernels of different scales to obtain three sets of illumination components of different scales. The enhanced high-turbidity water area image was obtained by subtracting the illumination components of each scale from the dehazed high-turbidity water area image and then performing a weighted summation.

5. The machine vision-based underwater foundation quality inspection method for bridge engineering as described in claim 1, characterized in that, The addition of a dual-branch attention module to the multi-head attention mechanism of the ViT model to extract defect features from the preprocessed image includes: The preprocessed image is divided into 16×16 pixel blocks, and each pixel block is used as an image token and input into the ViT model; In the dual-branch attention module, the local branch calculates the similarity between the feature vector of each token and the token features in the 3×3 neighborhood to obtain the local attention weight. In the dual-branch attention module, the global branch performs global pooling on the feature vectors of all tokens and then calculates the similarity to obtain the global attention weights. The local attention weights and global attention weights are fused to obtain the final attention weight matrix; The token feature vector is weighted and summed using the attention weight matrix, and then input into the fully connected layer for feature transformation, thus completing one operation of the encoder layer. The output of the encoder layer is used as the input of the next layer. Attention calculation and feature transformation operations are repeated. Global average pooling is then performed on the final output to obtain the defect features.

6. The machine vision-based underwater foundation quality inspection method for bridge engineering as described in claim 1, characterized in that, The step of inputting the defect features into a YOLOv8 model that includes deformable convolutional modules and weighted feature fusion, and using the EIOU loss function to output the defect category label and bounding box coordinates, includes: The extracted defect features are converted into three-dimensional feature maps and input into the YOLOv8 model. The YOLOv8 model embeds deformable convolutional modules in the C2f module of the backbone network. By dynamically adjusting the sampling position of the convolutional kernels, it captures the irregular shape features of the defect region. The 8×8, 16×16, and 32×32 scale feature maps output from the backbone network are input into the neck network. Weights are assigned to the channel dimensions of each scale feature map, and then feature fusion is performed through upsampling and downsampling operations. The fused feature map is input into the detection head module. The classification branch calculates the probability value of the defect belonging to the categories of crack, erosion, exposed reinforcement and spalling through the fully connected layer and the Softmax function, while the regression branch predicts the coordinate offset and width and height scaling factor of the defect bounding box. The EIOU loss function is used to calculate the loss value between the predicted bounding box and the ground truth bounding box, and finally outputs the defect category label and bounding box coordinates.

7. A machine vision-based underwater foundation quality inspection system for bridge engineering, characterized in that, The system includes: The image acquisition module is used to simultaneously acquire visible light images, near-infrared images, and low-light images using an underwater robot equipped with a high-definition camera, a multispectral camera, and a low-light camera, and then fuse them to obtain the original image. The image processing module is used to enhance the low-turbidity water area in the original image by using contrast enhancement and sharpening processing, and to enhance the high-turbidity water area in the original image by using a dark channel prior-based dehazing algorithm and a multi-scale Retinex algorithm, so as to obtain a preprocessed image. The feature extraction module is used to add a dual-branch attention module to the multi-head attention mechanism of the ViT model to extract defect features from the preprocessed image; The defect output module is used to input the defect features into a YOLOv8 model that includes deformable convolutional modules and weighted feature fusion, and output the defect category label and bounding box coordinates using the EIOU loss function. The parameter calculation module is used to perform pixel-level segmentation using the U-Net++ segmentation model based on the defect category label and bounding box coordinates to obtain the defect contour. It then calculates the physical parameters of the defect by combining the camera calibration parameters. Specifically, it crops a local image containing the complete defect from the preprocessed image based on the defect category label and bounding box coordinates, and inputs this image into the U-Net++ segmentation model. The encoder downsamples the local image to extract defect features at different scales, while the decoder upsamples the image, fusing deep and shallow features. The output layer uses a Sigmoid activation function to generate a binary segmentation map of the defect region, and then performs post-processing on the binary segmentation map. Finally, it obtains the camera calibration parameters, combines them with the detection distance of the underwater robot's attitude sensor, establishes a mapping relationship between pixel coordinates and actual physical coordinates, and calculates the corresponding physical parameters based on the defect category. The visualization module is used to associate the physical parameters of the defects with the BIM model and generate a visualization report.

8. A machine vision-based underwater foundation quality inspection device for bridge engineering, characterized in that, The machine vision-based underwater foundation quality inspection device for bridge engineering includes a memory and at least one processor. The memory stores instructions. The at least one processor invokes the instructions in the memory to cause the machine vision-based underwater foundation quality inspection device for bridge engineering to perform each step of the machine vision-based underwater foundation quality inspection method for bridge engineering as described in any one of claims 1-6.

9. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the various steps of the machine vision-based underwater foundation quality inspection method for bridge engineering as described in any one of claims 1-6.

Images