A wharf structure underwater damage visual detection method and system

Abstract

This application provides a visual inspection method and system for underwater damage to wharf structures, applied in the field of visual inspection technology. It involves projecting a reference beam array with a preset spatial arrangement to acquire visual images of the underwater target surface and simultaneously capturing the actual projection position of the reference beam array. The system calculates the displacement vector of the actual projection position relative to a preset ideal projection position and constructs distortion field data reflecting the transient refractive index distribution characteristics of the water medium based on the displacement vector. Furthermore, it establishes a reverse mapping relationship based on the distortion field data to geometrically reconstruct the visual image, thereby effectively eliminating optical distortion caused by the water medium and obtaining clear and accurate damage images. This method has the beneficial effect of improving the accuracy of damage identification.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology, and in particular to a visual inspection method and system for underwater damage to wharf structures. Background Technology

[0002] As a vital water transportation hub, the underwater support structure of a wharf is constantly subjected to the complex aquatic environment, making it susceptible to various types of damage. To ensure wharf safety, regular inspections using underwater visual inspection systems are crucial.

[0003] However, in actual port or estuary areas, some dock piles are often located in specific locations with rapid currents. For example, at river mouths, tidal channels, or narrow waterways, underwater currents can reach several meters per second, creating powerful currents. These strong currents pose significant challenges to underwater robot operations. First, lightweight underwater robots struggle to maintain a stable attitude and position, easily being pushed away from the target pile by the current, resulting in violently shaking and unfocused images, leading to blurry pictures. Second, even if the robot manages to get close, maintaining a constant distance and angle from the pile surface is difficult, significantly compromising the consistency of image acquisition and severely impacting the accuracy of subsequent damage analysis and quantitative assessment.

[0004] Traditional lightweight underwater robots are no longer sufficient; heavy-duty underwater robots equipped with high-powered thrusters and advanced positioning systems are now necessary. These robots typically possess precise attitude maintenance and hovering capabilities, and can adjust thruster power based on real-time water flow information, thereby mitigating the impact of water currents to a certain extent and ensuring relative stability of the robot body in space.

[0005] Despite significant improvements in the robot's stability, a more subtle and challenging problem was discovered. Even when the robot hovered steadily beside the piles, the images captured by the cameras still exhibited a peculiar fluctuation or distortion. This phenomenon was not caused by the robot's own minor wobbling, but rather by the influence of the strong water flow on the path of light. This transient optical distortion caused by water flow disturbance poses a serious challenge to identifying minute damage on the dock piles.

[0006] Therefore, in areas with rapid water flow, how to conduct high-precision visual damage detection on underwater piles of wharves to accurately identify and quantify minute structural damage on their surfaces, while overcoming the serious interference of transient, local, and irregular optical distortion caused by water flow disturbance on image clarity and geometric accuracy, avoiding missed detections or misjudgments due to image fluctuations and distortions, and without relying on physical intervention methods that may cause secondary damage to the structure or affect the aquatic environment, is a technical problem that urgently needs to be solved. Summary of the Invention

[0007] In view of the shortcomings of the prior art, this application provides a visual inspection method and system for underwater damage to wharf structures, which has the advantages of effectively eliminating transient optical distortion caused by the water medium, obtaining clear and accurate damage images, and improving the accuracy of damage identification.

[0008] A first aspect is a visual detection method for underwater damage to a wharf structure, the method comprising the following steps: S1: Project a reference beam array with a preset spatial arrangement onto the surface of the underwater target to be detected. The reference beam array is deflected as the refractive index of the water changes when it passes through the water medium. S2: Acquire visual images of the surface of the underwater target to be detected, and simultaneously capture the actual projection position of the reference beam array on the imaging plane after reflection from the water medium and the target surface; S3: Calculate the displacement vector of the actual projection position relative to the preset ideal projection position, and construct distortion field data reflecting the transient refractive index distribution characteristics of the water medium based on the displacement vector; S4: Establish a reverse mapping relationship based on the distortion field data, and perform geometric reconstruction on the visual image to eliminate optical distortion caused by the water medium.

[0009] Furthermore, step S2 includes: S21: Use a camera to acquire visual images of the surface of the underwater target to be detected; S22: Use a wavefront sensor to sense the wavefront distortion information of the reference beam array after passing through the water medium, and convert the wavefront distortion information into the actual projection position; the wavefront sensor is set in conjugate with the camera.

[0010] Furthermore, step S21 is followed by: S211: Use a camera to acquire an initial visual image of the surface of the underwater target to be detected, and analyze the average background brightness of the non-reference spot area and the halo width of the spot edge in the initial visual image to calculate the turbidity scattering index of the current water environment. S212: When the turbidity scattering index is higher than the preset scattering threshold, the working mode of the camera is switched to narrow pulse gating exposure mode; S213: Calculate the gating delay time based on the flight time distance of the reference beam array to the surface of the underwater target; S214: Control the camera to open the electronic shutter only within the time window during which the echo reflected from the target surface of the reference beam array reaches the imaging plane, so as to filter out the backscattered light noise generated by suspended particles located between the underwater moving carrier and the underwater target surface.

[0011] Furthermore, step S2 includes the following: S21: Obtain the set of actual projected position coordinates of all valid spots in the reference spot array in the current frame; S22: Calculate the global geometric transformation matrix of the actual projected position coordinate set relative to the theoretical coordinates, wherein the global geometric transformation matrix characterizes the rigid motion component of the underwater mobile vehicle relative to the underwater target surface; S23: Use the global geometric transformation matrix to perform coordinate transformation on the theoretical coordinates to generate dynamic reference coordinates; the dynamic reference coordinates include the preset ideal projection position.

[0012] Furthermore, step S3 includes: S31: Calculate the coordinate difference between the actual projection position and the preset ideal projection position corresponding to each of the reference beam arrays to obtain a set of discrete displacement vectors; S32: Using a spatial interpolation algorithm, the full pixel region of the visual image is fitted based on a set of discrete displacement vectors, and the estimated displacement vector of the non-reference beam projection region in the visual image is calculated, thereby generating dense distortion field data covering the entire visual image region.

[0013] Furthermore, step S4 includes: S41: Create a blank correction image; S42: Traverse each target pixel in the corrected image; S43: Based on the estimated displacement vector corresponding to the target pixel in the dense distortion field data, the source coordinate position of the target pixel in the original visual image is derived in reverse. S44: Read the pixel value at the source coordinate position and assign it to the target pixel in the corrected image to complete the geometric reconstruction.

[0014] Furthermore, the method also includes the step of: S5: Based on the corrected image after geometric reconstruction, feature extraction is performed on the corrected image using edge detection algorithms, morphological processing algorithms, or texture analysis algorithms to identify damage features such as cracks or peeling. S6: Obtain the pixel size data of the damage feature in the corrected image; S7: Call the pre-calibrated camera parameters to convert the pixel size data into actual physical size data, and output the quantitative evaluation result.

[0015] Furthermore, in step S1, the reference beam array is a laser beam arranged in a preset matrix or in concentric circles.

[0016] Furthermore, in step S1, the reference beam array is emitted by a miniature multi-point laser emission array, which employs a vertical cavity surface-emitting laser array and is configured to emit a green laser beam with a wavelength of 532 nanometers to maintain penetration in turbid water.

[0017] Secondly, a visual inspection system for underwater damage to wharf structures, the system comprising: Optical projection unit: projects a reference beam array with a preset spatial arrangement onto the surface of the underwater target to be detected. The reference beam array is deflected as it passes through the water medium due to the change in the refractive index of the water. Visual acquisition unit: Acquires visual images of the surface of the underwater target to be detected, and simultaneously captures the actual projection position of the reference beam array on the imaging plane after reflection from the water medium and the target surface; Distortion calculation unit: calculates the displacement vector of the actual projection position relative to the preset ideal projection position, and constructs distortion field data reflecting the transient refractive index distribution characteristics of the water medium based on the displacement vector; Geometric correction unit: Establishes a reverse mapping relationship based on the distortion field data, and performs geometric reconstruction on the visual image to eliminate optical distortion caused by the water medium.

[0018] Beneficial Effects: The present application proposes a visual detection method and system for underwater damage to wharf structures. By projecting a reference beam array with preset spatial arrangement characteristics, it acquires visual images of the surface of the underwater target to be detected and simultaneously captures the actual projection position of the reference beam array. It calculates the displacement vector of the actual projection position relative to the preset ideal projection position and constructs distortion field data reflecting the transient refractive index distribution characteristics of the water medium based on the displacement vector. Furthermore, it establishes a reverse mapping relationship based on the distortion field data to geometrically reconstruct the visual image, thereby effectively eliminating optical distortion caused by the water medium and obtaining clear and accurate damage images. This has the beneficial effect of improving the accuracy of damage identification. Attached Figure Description

[0019] Figure 1 This is a flowchart of a visual inspection method for underwater damage to a wharf structure proposed in this application.

[0020] Figure 2 This is a structural diagram of a visual inspection system for underwater damage to a wharf structure proposed in this application.

[0021] Figure 3 This is a schematic diagram of a visual inspection system for underwater damage to a wharf structure proposed in this application.

[0022] Labeling explanation: 201, Optical projection unit; 202, Visual acquisition unit; 203, Distortion calculation unit; 204, Geometric correction unit. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and marked in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0024] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] Please refer to Figure 1 This application proposes a visual detection method for underwater damage to wharf structures. This method aims to solve the optical distortion problem caused by transient changes in the refractive index of water in turbulent underwater environments, thereby achieving high-precision visual detection of underwater damage to wharf structures. The long-term safe operation of underwater support structures such as wharves and bridges, especially piles located in complex water flow areas such as estuaries and waterways, relies on regular visual inspections. However, strong water currents induce complex eddies and density fluctuations within the water body, resulting in an irregular transient distribution of the water's refractive index in time and space. When light passes through this medium, its propagation path is randomly deflected, ultimately forming visual distortions on imaging devices resembling water ripples or heat waves. This distortion severely reduces image clarity and geometric accuracy, making it difficult to accurately identify and quantify early damage at the millimeter or even sub-millimeter level, such as microcracks or concrete spalling. Traditional image enhancement or deblurring algorithms are ineffective against such non-uniform geometric distortions.

[0026] To overcome the aforementioned technical challenges, the method proposed in this application operates on the principle of no longer passively accepting and attempting post-processing of distorted images. Instead, it employs an active detection and real-time correction mechanism to accurately measure the transient optical distortion field caused by the water medium during imaging. This distortion field information is then used to perform reverse geometric reconstruction on the acquired visual image, thereby eliminating distortion at its source and restoring the true and clear visual appearance of the target object.

[0027] Specifically, the method includes the following steps: S1: Projecting a reference beam array with a preset spatial arrangement feature onto the surface of the underwater target to be detected, wherein the reference beam array is deflected as the refractive index of the water changes when passing through the water medium; S2: Acquire visual images of the surface of the underwater target to be detected, and simultaneously capture the actual projection position of the reference beam array on the imaging plane after reflection from the water medium and the target surface; S3: Calculate the displacement vector of the actual projection position relative to the preset ideal projection position, and construct distortion field data reflecting the transient refractive index distribution characteristics of the water medium based on the displacement vector; S4: Establish a reverse mapping relationship based on the distortion field data, and perform geometric reconstruction on the visual image to eliminate optical distortion caused by the water medium.

[0028] Throughout the workflow, an underwater mobile carrier equipped with the detection device described in this application, such as an underwater robot, will first dive down to the target detection area and approach the surface of the dock pile to be inspected.

[0029] In step S1, a reference beam array is projected onto the surface of the underwater target to be detected. This step is fundamental to the active detection of water distortion. The spatial arrangement characteristics of the projected reference beam array are precisely known beforehand. These beams act like a virtual, invisible calibration board, projected through the water area to be measured. When there is a refractive index inhomogeneity within the water, each beam of light will experience path deflection during propagation. The degree and direction of deflection directly reflect the optical properties of the water in the area traversed by the light path. Therefore, by subsequently capturing the projection positions of these beams on the target surface, the influence of the water medium on light propagation can be deduced.

[0030] In step S2, which involves acquiring a visual image and simultaneously capturing the actual projection position of the reference beam, this step is crucial for data acquisition. Acquiring the visual image is used to obtain the macroscopic appearance of the dock pile surface and potential damage information. Simultaneously capturing the actual projection position of the reference beam array is to obtain distortion information used to correct the visual image. Synchronization here is critical because the refractive index change of water is transient. Only by ensuring that the visual image and the reference beam position are captured at the same moment can the measured distortion field perfectly correspond to the distortion experienced by that frame of the image, thereby achieving accurate correction.

[0031] In step S3, which involves calculating displacement vectors and constructing distortion field data, the physical light deflection is transformed into a mathematical basis for correction. The preset ideal projection position refers to the theoretical coordinates on the imaging plane that the reference beam array should project onto under ideal conditions: no water distortion and a fixed relative position between the underwater moving vehicle and the target surface. By comparing the actual captured projection position point-by-point with this ideal position, a set of displacement vectors can be obtained. Each displacement vector precisely quantifies the magnitude and direction of the distortion experienced by the corresponding beam. Since the reference beam array is discrete, this set of displacement vectors is also discrete. To correct the entire image, these discrete measurement points need to be expanded into a continuous distortion field covering the entire field of view. The distortion field data is essentially a two-dimensional vector field, where each pixel position corresponds to a displacement vector, describing where the pixel at that point in the original image should be moved to eliminate distortion.

[0032] In step S4, which involves geometric reconstruction based on the distortion field data, this is the final step in image correction. Using the distortion field data constructed in the previous step, a reverse mapping relationship can be established from the distorted image to the corrected image. For each pixel in the corrected image, its original position in the original distorted image can be calculated using the distortion field data. Then, the pixel information at the original position is read and filled into the pixels of the current corrected image. By traversing all pixels, a clear and geometrically accurate corrected image is finally generated, eliminating the water ripple distortion.

[0033] Through the coordinated work of the above four steps, the method of this application can compensate for transient optical distortion caused by water flow disturbance in real time and dynamically, which significantly improves the reliability and accuracy of visual inspection in harsh underwater environments and provides a high-quality image basis for accurate identification and quantification of minor damage.

[0034] Furthermore, in some specific embodiments, the spatial arrangement characteristics of the projected reference beam array can be designed in various ways. For example, the reference beam array can be laser beams arranged in a predetermined matrix.

[0035] In one specific embodiment, the reference beam array is designed as a 10x10 orthogonal matrix arrangement containing 100 independent laser beams. The advantage of this matrix arrangement is that it provides uniform sampling points on a two-dimensional plane, facilitating subsequent distortion field interpolation calculations. Once these 100 light points are projected onto the target surface and captured by the imaging system, algorithms such as bilinear or bicubic interpolation can be conveniently used to quickly estimate the displacement vector of any pixel in the image based on these 100 known displacement vector reference points, thereby constructing a smooth and continuous dense distortion field. This method offers high computational efficiency and is suitable for detection scenarios with high real-time requirements.

[0036] In another specific embodiment, the reference beam array can be designed as laser beams arranged in concentric circles. For example, three concentric circles can be arranged, with four laser beams distributed in the inner circle, eight laser beams in the middle circle, and sixteen laser beams in the outer circle. This arrangement results in a higher sampling density in the central region of the field of view, gradually thinning outwards. This design may be more advantageous in certain specific water flow patterns, such as when the water flow mainly exhibits rotating eddies. The concentrically arranged beams can better capture this rotationally symmetric distortion pattern, thereby providing a more physically accurate constraint when fitting the distortion field and improving the accuracy of the correction.

[0037] By providing two different implementation methods, matrix arrangement and concentric circle arrangement, it is demonstrated that the concept of "reference beam array with preset spatial arrangement characteristics" protected by this application is not limited to a specific arrangement, but covers all regular arrangement forms that can provide effective spatial sampling to construct distortion fields.

[0038] To further improve the propagation performance of the reference beam underwater, especially in turbid waters with low visibility, the selection and configuration of the light source were optimized. Specifically, the reference beam array is emitted by a miniature multi-point laser emission array, which employs a vertical-cavity surface-emitting laser array and is configured to emit a green laser beam with a wavelength of 532 nanometers to maintain penetration in turbid waters.

[0039] Vertical-cavity surface-emitting laser arrays (VCSELs) are a type of semiconductor laser technology. Their advantage lies in integrating hundreds or thousands of miniature lasers onto a single, extremely small chip, achieving high-density, high-efficiency beam array output. This array structure is compact, easily integrated into the front end of underwater detection equipment, and allows for independent control of each laser point. More importantly, it allows for precise selection of the emission wavelength. Water exhibits selective absorption and scattering characteristics for different wavelengths of light, possessing a blue-green window—wavelengths between 450 and 550 nanometers exhibit minimal attenuation and the longest propagation distance in water. This application preferably uses a 532-nanometer green laser because this wavelength is near the central region of the blue-green window, possessing stronger penetration capabilities in typical turbid coastal or port waters compared to red or violet light. This ensures that even in environments with poor water quality, the reference beam can effectively reach the target surface and be clearly reflected back, thus guaranteeing the stability and reliability of the distortion measurement process.

[0040] In the steps described above for acquiring visual images and capturing the actual projection position of the reference beam, a specialized sensor configuration can be used to achieve high-precision distortion information acquisition. Specifically, this step includes: S21: Use a camera to acquire visual images of the surface of the underwater target to be detected; S22: Use a wavefront sensor to sense the wavefront distortion information of the reference beam array after passing through the water medium, and convert the wavefront distortion information into the actual projection position; the wavefront sensor and the camera are set in conjugate.

[0041] The cameras here are typically high-resolution industrial cameras, responsible for capturing detailed texture information of the pile surface. Wavefront sensors, on the other hand, are precision optical instruments specifically designed to measure the shape of light wavefronts.

[0042] In one specific embodiment, a Hartmann-Shack wavefront sensor is used. This sensor internally contains a microlens array and a high-frame-rate image sensor. When a reference beam array, distorted by the water medium, is incident on the microlens array, each microlens focuses its corresponding sub-beam onto the image sensor behind it, forming a focal array. If the incident wavefront is an ideal plane wave, all focal points will fall precisely at the center of the optical axis of each microlens. However, when the wavefront is distorted due to water disturbance, local tilting of the wavefront causes the focal points to deviate from their center positions. By accurately measuring the displacement of each focal point, the wavefront slope of the corresponding sub-region can be directly calculated. Integrating and reconstructing all slope data across the entire sensor yields high-precision wavefront distortion information across the entire beam cross-section. Finally, based on the wavefront distortion information, the actual projection position of each reference beam on the main imaging plane can be calculated, with a precision far exceeding that of directly finding the spot center in the main camera image using image processing algorithms.

[0043] Setting the wavefront sensor and the main camera in a conjugate configuration is crucial for ensuring calibration accuracy. This means that, through optical components such as beam splitters, the imaging plane of the main camera and the detection plane of the wavefront sensor are optically equivalent. In this way, the target scene observed by both and the optical path they traverse are exactly the same. The wavefront distortion accurately measured by the wavefront sensor can truly reflect the geometric distortion experienced by each pixel in the image captured by the main camera, thus providing physically accurate correction data for subsequent geometric reconstruction.

[0044] In practical underwater inspection operations, besides geometric distortion caused by water flow disturbance, water turbidity is another important factor affecting image quality. When there are many suspended particles in the water, light will be scattered during propagation, and some of the scattered light will directly return to the imaging device, forming background noise, i.e., backscattered light. This will severely reduce the image contrast and signal-to-noise ratio, as if a thick fog has formed underwater. To address this problem, this application further proposes an adaptive anti-scattering imaging strategy.

[0045] After acquiring visual images, the following steps are also included: S211: Use a camera to acquire an initial visual image of the surface of the underwater target to be detected, analyze the average background brightness of the non-reference spot area and the halo width of the spot edge in the initial visual image, and calculate the turbidity scattering index of the current water environment. S212: When the turbidity scattering index is higher than the preset scattering threshold, switch the camera's working mode to narrow pulse gating exposure mode; S213: Calculate the gating delay time based on the flight time distance of the reference beam array to the surface of the underwater target; S214: Control the camera to open the electronic shutter only within the time window during which the echo reflected from the target surface by the reference beam array reaches the imaging plane, in order to acquire a visual image and filter out the backscattered light noise generated by suspended particles located between the underwater moving vehicle and the underwater target surface.

[0046] Specifically, stronger backscattering results in higher background brightness in the image area far from the target; conversely, stronger forward scattering leads to a more blurred edge and wider halo of the reference spot projected onto the target surface. An empirical model can be established, for example: Turbidity Scattering Index = a × Mean Background Brightness + b × Halo Width of Spot Edge, where a and b are weighting coefficients. By calculating this index in real time, the turbidity of the current water body can be quantitatively assessed. The weighting coefficients a and b are typically determined through experimental calibration and data fitting. Specifically, obtaining the weighting coefficients a and b can include the following steps: First, a controlled experimental environment must be established. This typically involves a tank or container with adjustable turbidity. Within this tank, the turbidity of the water can be precisely controlled, for example, by adding a known concentration of a scattering agent (such as kaolin, diatomaceous earth, or a specific turbidity standard solution) and using a standard turbidimeter (such as a scattering turbidimeter) to obtain actual turbidity values ​​at different turbidity levels as a reference.

[0047] Next, data acquisition was performed. In the controlled experimental environment described above, images were acquired using the visual acquisition method described in this application for a series of different and known turbidity levels. For each turbidity level, the following operations were performed: Acquire visual images of the surface of the underwater target to be inspected.

[0048] From the acquired visual image, the average background brightness of the non-reference spot region is accurately calculated. This can be obtained by statistically analyzing the pixel brightness of the background region in the image that does not contain the reference spot and then calculating its average value. Simultaneously, the blurring width of the spot edge is measured. This can be achieved using image processing algorithms, such as edge detection and Gaussian fitting, to quantify the blurring degree or diffusion range of the reference spot edge.

[0049] Next, data fitting and model training are performed. After collecting a sufficient number of experimental data points, each data point contains a corresponding set of true turbidity values, mean background brightness, and spot edge blur width. This data is then input into a mathematical fitting or regression analysis algorithm. Since the turbidity scattering index model is a linear combination, multiple linear regression analysis can be used to solve for the weighting coefficients a and b. The goal of this algorithm is to find the optimal values ​​of a and b that minimize the error between the model-predicted turbidity scattering index and the experimentally measured true turbidity values. For example, a loss function, such as mean squared error, can be constructed and solved using the least squares method or other optimization algorithms. This process is a typical empirical model parameter determination process, the core of which lies in establishing the mapping relationship between actual measurement data and model output.

[0050] Next, when the calculated turbidity scattering index exceeds a preset scattering threshold, it indicates that the water turbidity has severely affected the image quality. At this point, the camera's operating mode is automatically switched to narrow pulse gating exposure mode. This is an active imaging technology based on the time-of-flight principle.

[0051] After switching to this mode, the gating delay time needs to be calculated based on the flight time distance of the reference beam array to the underwater target surface. This distance can be measured in real time using a laser ranging module or a sonar module. The formula for calculating the gating delay time is: Delay time = (2 × target distance) / speed of light in water. This time represents the total time required for the light pulse to travel from the transmitter to the target and then back to the receiver.

[0052] Finally, the camera is controlled to open its electronic shutter only within a precise time window when the echo reflected from the target surface reaches the imaging plane, thus acquiring the visual image. Specifically, the laser emitter emits an extremely narrow light pulse, simultaneously triggering a high-precision time delay. After a preset gating delay, the time delay generates a trigger signal, which controls the camera's electronic shutter to open and complete the exposure within an extremely short time (e.g., nanoseconds). Since the effective signal light reflected from the target surface has a flight time exactly equal to the gating delay time, it can be captured by the shutter. Backscattered light generated by suspended particles between the emitter and the target, due to its shorter propagation path, arrives at the receiver earlier, before the shutter opens, and is thus effectively filtered out. In this way, backscattered noise can be greatly suppressed, resulting in a clear image with significantly improved contrast.

[0053] Furthermore, during underwater detection, the underwater mobile carrier carrying the detection equipment may itself experience slight translational and attitude changes. This rigid movement of the carrier will also cause the reference spot to shift across the imaging plane. If this is misinterpreted as water distortion, it will introduce correction errors. To accurately separate carrier motion from water distortion, this application also proposes a dynamic reference calibration procedure.

[0054] After acquiring the actual projection position of the reference spot array, the method further includes: S21: Obtain the set of actual projected position coordinates of all valid spots in the reference spot array of the current frame; S22: Calculate the global geometric transformation matrix of the actual projected position coordinate set relative to the theoretical coordinates. The global geometric transformation matrix represents the rigid motion components of the underwater mobile vehicle relative to the underwater target surface. S23: Use the global geometric transformation matrix to transform the theoretical coordinates and generate dynamic reference coordinates; the dynamic reference coordinates include the preset ideal projection position.

[0055] The acquisition of the actual projected position coordinates of all valid spots in the current frame reference spot array refers to identifying and extracting the center coordinates of all clear and identifiable spots from the spot image captured by the visual acquisition unit after projecting the reference beam array onto the underwater target surface. This can be achieved using image processing algorithms. For example, the acquired visual image can be preprocessed, including noise reduction and contrast enhancement. Then, methods such as threshold segmentation and connected component analysis can be used to identify the spot regions. Finally, the precise actual projected position coordinates can be determined by calculating the centroid of the spot region or fitting the spot contour. These coordinates are typically represented in pixel coordinates.

[0056] Calculating the global geometric transformation matrix of the actual projected position coordinates relative to the theoretical coordinates refers to establishing a mapping relationship between the actually observed spot positions and the preset ideal spot positions through mathematical methods. Specifically, this can be achieved using the least squares method or the RANSAC (Random Sample Consensus) algorithm. For example, after obtaining a sufficient number of actual projected position coordinates and their corresponding theoretical coordinates, a system of linear equations can be constructed to solve for the geometric transformation matrix that optimally transforms the theoretical coordinates to the actual coordinates. This matrix is ​​typically a 2D affine transformation matrix or perspective transformation matrix, capable of describing rigid motion components such as translation, rotation, scaling, and shearing. The global geometric transformation matrix characterizes the rigid motion components of the underwater moving vehicle relative to the underwater target surface. This means that the matrix can quantify the overall displacement and attitude changes of the vehicle relative to the target surface during underwater movement, thereby distinguishing this overall motion from the local distortions caused by the water medium.

[0057] Generating dynamic reference coordinates by transforming theoretical coordinates using a global geometric transformation matrix involves correcting the preset ideal light spot position using the previously calculated global geometric transformation matrix. This can be achieved using matrix multiplication; for example, each theoretical coordinate point is represented in homogeneous coordinate form and then multiplied by the global geometric transformation matrix to obtain the transformed coordinates. These new coordinates are the dynamic reference coordinates, reflecting the position where the ideal light spot should appear under the current rigid motion state of the carrier. The dynamic reference coordinates include the preset ideal projection position, meaning that in the absence of rigid motion, the dynamic reference coordinates will coincide with the preset ideal projection position; while in the presence of rigid motion, the dynamic reference coordinates will be adjusted accordingly based on the motion, thus providing a reference that more closely matches the actual situation for subsequent displacement vector calculations.

[0058] This scheme addresses the inaccuracy of distortion field data caused by the rigid motion of an underwater mobile platform in dynamic underwater environments by introducing a compensation mechanism for such motion. First, the actual projected position coordinates of all effective spots in the current frame's reference spot array are obtained. This is the foundation for subsequent calculations, ensuring that all available measurement data is considered. Next, the global geometric transformation matrix of the actual projected position coordinates relative to the theoretical coordinates is calculated. This step is crucial, as it accurately characterizes the rigid motion components of the underwater mobile platform relative to the underwater target surface, thus separating the overall displacement caused by the platform's motion from the local distortion caused by the water medium. Finally, the theoretical coordinates are transformed using the global geometric transformation matrix to generate dynamic reference coordinates, which include a preset ideal projection position. In this way, the preset ideal projection position is no longer fixed but dynamically adjusted according to the real-time motion of the platform, providing a more accurate reference for subsequent displacement vector calculations. This effectively eliminates the interference of the rigid motion of the underwater mobile platform on the construction of the distortion field data and improves the accuracy of distortion correction.

[0059] After obtaining the accurate discrete displacement vector, it needs to be expanded into dense distortion field data covering the entire image region. Specifically, this step includes: S31: Calculate the coordinate difference between the actual projected position and the preset ideal projected position corresponding to each reference beam array to obtain a set of discrete displacement vectors; S32: Using a spatial interpolation algorithm, the full pixel region of the visual image is fitted based on a set of discrete displacement vectors, and the estimated displacement vector of the non-reference beam projection region in the visual image is calculated, thereby generating dense distortion field data covering the entire visual image region.

[0060] In one specific embodiment, a bilinear interpolation algorithm can be used. For any pixel at a non-reference spot location in the image, firstly, find the four nearest reference spots, which form a quadrilateral. Then, based on the pixel's relative position within this quadrilateral, calculate the estimated displacement vector of the pixel by weighted averaging the displacement vectors of these four reference points. This method is simple to calculate, fast, and can meet the needs of most real-time processing.

[0061] In another embodiment requiring higher accuracy, radial basis function interpolation can be employed. This algorithm assumes that the influence of the displacement vector of each reference point on its surrounding area decreases with increasing distance. By solving a system of linear equations, a smooth interpolation function can be constructed that accurately passes through the displacement vectors of all known reference points. Using this function, the displacement vector of any pixel in the image can be calculated. Radial basis function interpolation can better handle complex and nonlinear distortion field distributions, generating a smoother and more accurate distortion field, but its computational complexity is relatively high.

[0062] Through spatial interpolation, a two-dimensional vector field with the same size as the original visual image is finally obtained, namely dense distortion field data, which provides complete mapping information for subsequent pixel-level geometric reconstruction.

[0063] After generating dense distortion field data, the original visual image can be geometrically reconstructed. This process uses inverse mapping to ensure the quality of the corrected image. The specific steps are as follows: First, in step S41, a blank corrected image with the same size as the original image is created in memory.

[0064] Then, in step S42, each target pixel in this blank correction image is traversed, assuming its coordinates are ( ).

[0065] In step S43, for each target pixel, the corresponding estimated displacement vector (dx, dy) is found in the dense distortion field data based on its coordinates. This displacement vector describes how much the pixel in the original image has moved due to distortion. Therefore, to find the source of this target pixel in the original image, a reverse calculation is needed to determine its source coordinates in the original visual image (dx, dy). ) can be obtained through formula and It is derived that...

[0066] Finally, in step S44, the calculated source coordinate position is read ( The pixel value in the original visual image is used to assign the value to the current target pixel in the corrected image. Since the calculated source coordinates may be non-integer, methods such as bilinear interpolation are usually required to obtain sub-pixel precision pixel values ​​to avoid jagged edges or mosaic effects in the image.

[0067] By repeating the above process on all pixels in the corrected image, the geometric reconstruction of the entire image can be completed, ultimately resulting in a distortion-free and clear corrected image.

[0068] After obtaining high-quality corrected images through the aforementioned series of steps, the ultimate goal of the detection task is to identify and quantify the damage to the dock structure. Therefore, this method also includes subsequent damage analysis steps.

[0069] First, in step S5, based on the geometrically reconstructed corrected image, mature computer vision algorithms are used for feature extraction to identify damage features such as cracks or spalling. Since the corrected image has eliminated geometric distortion, the shape and edges of the damage features are very clear. For example, the Canney edge detection algorithm can be used to extract all edges in the image, combined with Hough transform or morphological processing algorithms, such as skeleton extraction, to identify cracks with linear features. For spalled concrete areas, since their surface texture is usually different from intact areas, texture analysis algorithms, such as local binary mode or gray-level co-occurrence matrix, can be used to segment the spalled areas.

[0070] After identifying the damage features, step S6 is to obtain the pixel size data of the damage features in the corrected image. For example, for a crack, its pixel length and average pixel width in the image can be calculated; for a peeling area, its pixel area can be calculated.

[0071] Finally, to obtain quantitative evaluation results with practical engineering significance, pixel dimensions need to be converted into actual physical dimensions. Step S7 requires calling pre-calibrated camera parameters. Before the equipment is launched into the water, by photographing standard checkerboard patterns or other calibration objects, the camera's internal parameters, such as focal length and principal point coordinates, can be accurately calculated. Combined with the real-time distance from the camera to the target surface provided by the ranging module, the actual physical dimension represented by each pixel in the current field of view can be calculated, i.e., the image resolution (unit: mm / pixel). Multiplying the pixel dimension data obtained in the previous step by this resolution coefficient converts the pixel dimension data into actual physical dimension data, such as outputting a crack length of 20.5 cm, a maximum width of 1.2 mm, or a spalling area of ​​15.3 square centimeters, etc. These precise quantitative data provide a reliable basis for the safety assessment and maintenance decisions of the dock structure.

[0072] For the methods described above, please refer to... Figure 2 , Figure 3This application also provides a visual inspection system for underwater damage to wharf structures, which integrates all the functional units required to implement the above-described method. The system includes: Optical projection unit 201: projects a reference beam array with a preset spatial arrangement feature onto the surface of the underwater target to be detected. The reference beam array is deflected as the refractive index of the water changes when it passes through the water medium. Visual acquisition unit 202: Acquires visual images of the surface of the underwater target to be detected, and simultaneously captures the actual projection position of the reference beam array on the imaging plane after reflection from the water medium and the target surface; Distortion calculation unit 203: calculates the displacement vector of the actual projection position relative to the preset ideal projection position, and constructs distortion field data reflecting the transient refractive index distribution characteristics of the water medium based on the displacement vector; Geometric correction unit 204: Establishes a reverse mapping relationship based on the distortion field data to perform geometric reconstruction on the visual image in order to eliminate optical distortion caused by the water medium.

[0073] The optical projection unit 201 can physically consist of a miniature multi-point laser emission array, such as a vertical-cavity surface-emitting laser array, along with corresponding driving circuitry and a collimating optical system. This unit is responsible for generating a stable, high-quality reference beam for detecting water distortion.

[0074] The visual acquisition unit 202 typically consists of a main imaging camera, a wavefront sensor, a beam splitter, and a synchronization trigger control circuit. The main camera is responsible for acquiring high-resolution target images, the wavefront sensor is responsible for accurately measuring wavefront distortion, and the synchronization circuit ensures that the data acquisition of both is strictly aligned in time.

[0075] The distortion calculation unit 203 is typically a high-performance embedded computing module, such as a field-programmable gate array or a graphics processor, which has algorithms embedded or running inside for spot position extraction, carrier motion compensation, spatial interpolation, etc., and can process sensor data into a dense distortion field in real time.

[0076] The geometric correction unit 204 can be integrated with the distortion calculation unit in the same calculation module, and is responsible for performing image processing operations such as reverse mapping and pixel interpolation, and finally outputting a clear, distortion-free corrected image.

[0077] These four units work together to form a complete closed loop from distortion detection and measurement to image correction, which can effectively achieve high-precision visual detection of underwater damage to the dock structure in harsh environments with turbulent water flow.

[0078] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for visual detection of underwater damage of a port structure, characterized in that, The method includes the following steps: S1: Project a reference beam array with a preset spatial arrangement onto the surface of the underwater target to be detected. The reference beam array is deflected as the refractive index of the water changes when it passes through the water medium. S2: Acquire visual images of the surface of the underwater target to be detected, and simultaneously capture the actual projection position of the reference beam array on the imaging plane after reflection from the water medium and the target surface; S3: Calculate the displacement vector of the actual projection position relative to the preset ideal projection position, and construct distortion field data reflecting the transient refractive index distribution characteristics of the water medium based on the displacement vector; S4: Establish a reverse mapping relationship based on the distortion field data, and perform geometric reconstruction on the visual image to eliminate optical distortion caused by the water medium.

2. The method according to claim 1, wherein, Step S2 includes: S21: Use a camera to acquire visual images of the surface of the underwater target to be detected; S22: Use a wavefront sensor to sense the wavefront distortion information of the reference beam array after passing through the water medium, and convert the wavefront distortion information into the actual projection position; the wavefront sensor is set in conjugate with the camera.

3. The method according to claim 1, wherein, Step S21 is followed by: S211: Use a camera to acquire an initial visual image of the surface of the underwater target to be detected, and analyze the average background brightness of the non-reference spot area and the halo width of the spot edge in the initial visual image to calculate the turbidity scattering index of the current water environment. S212: When the turbidity scattering index is higher than the preset scattering threshold, the working mode of the camera is switched to narrow pulse gating exposure mode; S213: Calculate the gating delay time based on the flight time distance of the reference beam array to the surface of the underwater target; S214: Control the camera to open the electronic shutter only within the time window during which the echo reflected from the target surface of the reference beam array reaches the imaging plane, so as to filter out the backscattered light noise generated by suspended particles located between the underwater moving carrier and the underwater target surface.

4. The method according to claim 1, wherein, Step S2 is followed by: S21: Obtain the set of actual projected position coordinates of all valid spots in the reference spot array in the current frame; S22: Calculate the global geometric transformation matrix of the actual projected position coordinate set relative to the theoretical coordinates, wherein the global geometric transformation matrix characterizes the rigid motion component of the underwater mobile vehicle relative to the underwater target surface; S23: Use the global geometric transformation matrix to perform coordinate transformation on the theoretical coordinates to generate dynamic reference coordinates; the dynamic reference coordinates include the preset ideal projection position.

5. The method according to claim 4, wherein, Step S3 includes: S31: Calculate the coordinate difference between the actual projection position and the preset ideal projection position corresponding to each of the reference beam arrays to obtain a set of discrete displacement vectors; S32: Using a spatial interpolation algorithm, the full pixel region of the visual image is fitted based on a set of discrete displacement vectors, and the estimated displacement vector of the non-reference beam projection region in the visual image is calculated, thereby generating dense distortion field data covering the entire visual image region.

6. The method according to claim 5, wherein, Step S4 includes: S41: Create a blank correction image; S42: Traverse each target pixel in the corrected image; S43: Based on the estimated displacement vector corresponding to the target pixel in the dense distortion field data, the source coordinate position of the target pixel in the original visual image is derived in reverse. S44: Read the pixel value at the source coordinate position and assign it to the target pixel in the corrected image to complete the geometric reconstruction.

7. The method for visual detection of underwater damage of a wharf structure according to claim 1, characterized in that, The method further includes the following steps: S5: Based on the corrected image after geometric reconstruction, feature extraction is performed on the corrected image using edge detection algorithms, morphological processing algorithms, or texture analysis algorithms to identify damage features such as cracks or peeling. S6: Obtain the pixel size data of the damage feature in the corrected image; S7: Call the pre-calibrated camera parameters to convert the pixel size data into actual physical size data, and output the quantitative evaluation result.

8. The method for visual detection of underwater damage of a wharf structure according to claim 1, characterized in that, In step S1, the reference beam array is a laser beam arranged in a preset matrix or in concentric circles.

9. The method according to claim 8, wherein, In step S1, the reference beam array is emitted by a miniature multi-point laser emission array, which is a vertical cavity surface-emitting laser array and configured to emit a green laser beam with a wavelength of 532 nanometers to maintain penetration in turbid water.

10. A visual underwater damage detection system for a wharf structure, characterised by, The system includes: Optical projection unit: projects a reference beam array with a preset spatial arrangement onto the surface of the underwater target to be detected. The reference beam array is deflected as it passes through the water medium due to the change in the refractive index of the water. Visual acquisition unit: Acquires visual images of the surface of the underwater target to be detected, and simultaneously captures the actual projection position of the reference beam array on the imaging plane after reflection from the water medium and the target surface; Distortion calculation unit: calculates the displacement vector of the actual projection position relative to the preset ideal projection position, and constructs distortion field data reflecting the transient refractive index distribution characteristics of the water medium based on the displacement vector; Geometric correction unit: Establishes a reverse mapping relationship based on the distortion field data, and performs geometric reconstruction on the visual image to eliminate optical distortion caused by the water medium.

Images