Tunnel crack inspection system and method based on scene camera and liquid lens

Abstract

This invention discloses a tunnel crack inspection system and method based on a scene camera and a liquid lens. The tunnel crack inspection system mimics the human eye's "first gaze, then fixation" mechanism, constructing a two-stage collaborative detection system consisting of a large-field-of-view scene camera and a liquid lens camera. It achieves automatic focusing and high-precision crack imaging of the target area through electronic control. The first stage, a scene perception module composed of a large-field-of-view scene camera and a depth camera, performs global scanning and spatial ranging to quickly locate suspected defect areas and their depth information. The second stage, a high-resolution liquid lens camera, automatically focuses based on the depth information to perform fine imaging and crack quantification analysis of suspected crack areas.

Description

Technical Field

[0001] This invention relates to the technical field of tunnel engineering structural health monitoring and machine vision, and in particular to a tunnel crack inspection system and method based on a scene camera and a liquid lens. Background Technology

[0002] During long-term use, tunnel structures develop cracks on their lining surfaces due to geological changes, material aging, and load fatigue. Micro-cracks (less than 0.2 mm in width) are often early signs of structural damage, and their timely detection and treatment are crucial for preventing catastrophic accidents. However, current tunnel crack detection technologies face significant limitations: 1. Manual inspection method: Relies on the experience of inspectors, is inefficient, highly subjective, prone to missing micro-cracks, and requires traffic interruption, posing a high safety risk.

[0003] 2. Vehicle detection method based on traditional cameras: Although it improves efficiency, it has the following inherent defects.

[0004] The conflict between global and detailed perspectives: To cover the entire tunnel cross-section, wide-angle lenses or multiple fixed-focus cameras are typically used. Wide-angle lenses produce distortion when shooting edge areas at close range, affecting measurement accuracy; while multi-camera systems are expensive and complex to calibrate.

[0005] Limitations of fixed focal length: The tunnel surface is not an ideal plane; it has undulations and curvatures. When a fixed focal length camera is mounted on a high-speed moving vehicle, due to the limited depth of field, it cannot guarantee that cracks at different distances across the entire cross-section will be clearly imaged. This can easily lead to fine cracks becoming blurry due to defocusing.

[0006] Redundant data processing: Processing the massive amount of high-definition images of the entire tunnel consumes huge amounts of computing resources and is inefficient.

[0007] 3. Laser scanning-based methods: Although they can accurately acquire three-dimensional geometric information, they are not good at distinguishing two-dimensional information such as crack texture and color, and it is difficult to identify early micro-cracks and their types (such as shrinkage cracks and stress cracks).

[0008] A liquid lens is an optical device that achieves millisecond-level rapid autofocus by precisely adjusting the lens curvature through changes in voltage or current. It boasts advantages such as no moving mechanical parts, high shock resistance, fast focusing speed, low power consumption, and small size. Currently, liquid lenses are mainly used in industrial endoscopes, microscopes, and barcode scanning. However, in large-scale, complex tunnel inspections, there is still no innovative application combining them with scene cameras to achieve a two-stage inspection process of "wide-area reconnaissance followed by precision strike." Summary of the Invention

[0009] To address the limitations and defects of existing technologies, this invention provides a tunnel crack inspection system based on a scene camera and a liquid lens, comprising a support platform, a primary detection unit, a secondary detection unit, and a control and processing unit; The support platform is used to simultaneously carry cameras, lighting sources, controllers, and positioning devices to move smoothly in the tunnel; The primary detection unit is used to perform visual acquisition of tunnel surface and spatial distance measurement to support the subsequent focusing of the liquid lens; the primary detection unit is also used to simultaneously acquire visual images and depth data, use visual algorithms to identify suspected crack areas in the panoramic image, and obtain the spatial depth information corresponding to the suspected crack areas through a depth camera. The secondary detection unit is used to perform local magnification processing on the suspected crack area for high-precision imaging. The secondary detection unit includes a liquid lens assembly, a focusing system, a driving system, a control system, an illumination system, and a signal transmission and layout system. The control and processing unit includes a main controller, an image processing module, a positioning and synchronization module, and a liquid lens control module.

[0010] Optionally, the carrying platform may include a rail-mounted mobile device, a vehicle-mounted mobile device, or a handheld mobile device.

[0011] Optionally, the primary detection unit includes a wide field-of-view scene camera and a depth camera. The wide field-of-view scene camera is used for visual detection to generate a panoramic image of the tunnel lining surface to identify the suspected crack area. The wide field-of-view scene camera includes a lens group, an imaging device, a shutter control component, an illumination system, and a signal interface.

[0012] Optionally, the lens assembly is a fisheye lens assembly, including a front lens, a correction lens and an infrared cutoff filter; The imaging device is equipped with a global shutter CMOS sensor to ensure no motion blur during high-speed travel; The shutter control component uses the main controller to send a trigger signal to ensure that the wide field-of-view scene camera and the lighting source are exposed synchronously. The lighting system uses high-brightness linear LED strip light sources to uniformly illuminate the tunnel walls; The signal interface communicates with the main control unit via Gigabit Ethernet or CameraLink interface to transmit real-time image data.

[0013] Optionally, the depth camera calculates the depth image by projecting infrared structured light and using echoes, and outputs the three-dimensional coordinates of each pixel. The depth camera and fisheye camera are calibrated together, and spatial registration of visual images and depth information is achieved through geometric mapping, providing target distance data for the focusing control of the liquid lens.

[0014] Optionally, the field of view of the fisheye lens group is ≥120°, the frame rate of the global shutter CMOS sensor is ≥200fps, and the working distance range of the depth camera is 0.5m to 5m.

[0015] Optionally, the liquid lens assembly employs a dual-liquid-layer electrowetting optical system, comprising a conductive liquid and a non-polar optical oil, wherein the conductive liquid and the non-polar optical oil are isolated by a transparent electrode to form a variable curvature interface, and the outer encapsulation of the liquid lens assembly comprises a high-transparency glass and an elastic sealing film. The focusing system is used to adjust the interface curvature by changing the control voltage between the electrodes to achieve a continuously adjustable focal length. The drive system includes a voltage drive module and a closed-loop current monitoring circuit. The control system is used to perform auxiliary fine-tuning via piezoelectric ceramics to improve focal length accuracy; The lighting system uses an adjustable intensity strobe light source, which is synchronously triggered with the liquid lens camera to freeze the moment of motion. The signal transmission and layout system outputs the voltage signal from the main controller to the electrode layer via a drive amplifier, and transmits the sensing signal back via USB 3.0 or a fiber optic interface.

[0016] Optionally, the main controller is used to coordinate the acquisition sequence of the primary detection unit and the secondary detection unit and generate a synchronization trigger signal; The image processing module is used to run a two-stage vision algorithm to extract the suspected crack area and complete depth mapping and crack identification. The positioning and synchronization module includes an encoder, an inertial measurement unit, and a real-time dynamic positioning system, used to label the visual image with spatial coordinates. The liquid lens control module is used to calculate the focal length corresponding to the target distance based on the target distance obtained by the depth camera, and output the corresponding control voltage or control current to achieve automatic focusing.

[0017] This invention also provides a tunnel crack inspection method based on a scene camera and a liquid lens, wherein the tunnel crack inspection method uses any of the tunnel crack inspection systems described above, and the tunnel crack inspection method includes: Joint calibration of fisheye camera, Kinect depth camera and liquid lens camera is performed to determine the extrinsic parameter matrix and projection relationship between fisheye camera, Kinect depth camera and liquid lens camera, establish digital surface model of tunnel inner wall, and record depth and coordinate information of each detection point; The carrying platform moves at a preset speed, the fisheye camera acquires panoramic images of the tunnel, the Kinect depth camera outputs depth images simultaneously, the image processing module uses a neural network to identify the suspected crack area, outputs the pixel coordinates and confidence level of the suspected crack area, and obtains the depth distance of the corresponding detection point based on the depth image. Based on the joint calibration results, the pixel coordinates and depth information of the suspected crack area are converted into the physical coordinate system of the liquid lens camera; The control unit calculates the target focal length of the liquid lens based on the depth distance, using the following expression: , in, Z1 is the initial focal length of the liquid lens, and Z2 is the depth distance. For reference plane distance, ; The control unit calculates the control voltage using the electrowetting model and empirical calibration curve, as shown in the following expression: , Where k is a system constant, The target focal length of the liquid lens; The control voltage is output to the electrode layer of the liquid lens to adjust the interface curvature of the liquid lens in order to achieve autofocus; The liquid lens camera performs local high-resolution imaging of the suspected crack area, triggers a strobe light source to suppress motion blur, and uses a deep network to segment, classify, and quantize the width of the crack. The measurement results are bound to the spatial coordinates obtained by the positioning module, and then annotated in the tunnel digital model to automatically generate a tunnel crack health report.

[0018] Optionally, the step of outputting the control voltage to the electrode layer of the liquid lens to adjust the interface curvature of the liquid lens includes: The control voltage is output to the electrode layer of the liquid lens, so that the interface curvature of the liquid lens is adjusted within 15ms.

[0019] The present invention has the following beneficial effects: Dual-vision fusion scene perception system: combining fisheye camera and depth camera to achieve integrated detection of "visual recognition + distance measurement".

[0020] Precision electronic focusing mechanism: The control voltage is directly calculated based on depth information to achieve automatic and precise focusing of the liquid lens.

[0021] Millisecond-level fast response: The liquid lens completes focus switching within 15ms, adapting to tunnel movement detection scenarios.

[0022] High robustness and scalability: No moving mechanical parts, shock and impact resistant, and scalable to infrared or thermal imaging modes.

[0023] High-precision quantitative analysis capability: Provides measurement results of crack width, length, and direction based on real spatial scale. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the tunnel crack inspection system based on a scene camera and a liquid lens provided in Embodiment 1 of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solution of the present invention, the tunnel crack inspection system and method based on scene camera and liquid lens provided by the present invention will be described in detail below with reference to the accompanying drawings. Example 1

[0026] This embodiment provides an intelligent tunnel crack inspection system and method based on the linkage of a scene camera and a liquid lens. The system aims to: solve the problem of difficult detection of micro-cracks due to defocusing; resolve the contradiction between large field-of-view coverage and high-precision detail capture; significantly improve detection efficiency and computing resource utilization, achieving intelligent and automated inspection; and enhance the system's adaptability and robustness in complex environments such as tunnel vibration and lighting changes.

[0027] Figure 1 This is a schematic diagram of the tunnel crack inspection system based on a scene camera and a liquid lens provided in Embodiment 1 of the present invention. Figure 1 As shown: 1. System Composition The system described in this embodiment mainly includes: a support platform, a primary detection unit, a secondary detection unit, and a control and processing unit.

[0028] (1) Bearing platform The carrying platform can be a rail-mounted, vehicle-mounted, or handheld mobile device with shock-resistant and vibration-damping design, used to move smoothly in tunnels and simultaneously carry cameras, lighting sources, controllers, and positioning equipment.

[0029] (2) Primary detection unit (scene perception module) The primary detection unit is responsible for visually acquiring and measuring spatial distances on the tunnel surface in a wide range of scenarios to support precise focusing of the subsequent liquid lens.

[0030] The primary detection unit consists of two parts: ① Wide field-of-view scene camera (visualization imaging part) This section is primarily used for visual inspection, generating panoramic images of the tunnel lining surface to facilitate rapid identification of suspected crack areas (ROI). Key components include: Lens assembly: It adopts a fisheye lens assembly (field of view ≥120°), which includes a front lens, a correction lens and an infrared cutoff filter.

[0031] Imaging device: Equipped with a global shutter CMOS sensor (frame rate ≥ 200fps) to ensure no motion blur at high speeds.

[0032] Shutter control unit: The main controller sends a trigger signal to achieve synchronous exposure with the illumination source.

[0033] Lighting system: High-brightness linear LED strip light source is used to evenly illuminate the tunnel wall.

[0034] Signal interface: Communicates with the main control unit via Gigabit Ethernet or CameraLink interface to transmit real-time image data.

[0035] ② Depth camera (spatial ranging component) Sensor type: Kinect depth camera or similar structured light / TOF depth sensor, working distance 0.5m to 5m.

[0036] Ranging principle: The depth image is calculated by projecting infrared structured light and using echoes, and the three-dimensional coordinates (X, Y, Z) of each pixel are output.

[0037] Function: The depth camera and fisheye camera are calibrated together to achieve spatial registration of visual images and depth information through geometric mapping, thereby providing accurate target distance data for focusing control of liquid lenses.

[0038] Summary of the functions of the first-level detection unit: This module simultaneously collects visual images and depth data during system operation. First, visual algorithms (such as YOLO and MobileNet) quickly identify suspected crack areas in the panoramic image. Then, the depth camera obtains the spatial depth information corresponding to the area, providing a basis for the liquid lens focusing of the second-level detection unit.

[0039] (3) Secondary detection unit (liquid lens camera module) The secondary detection unit is used to perform local magnification and high-precision imaging of the ROI area obtained from the primary detection.

[0040] The secondary detection unit consists of the following components: Liquid lens assembly: It adopts a dual-liquid-layer electrowetting optical system, which consists of a conductive liquid (electrolyte) and a non-polar optical oil. The two are isolated by transparent electrodes to form a variable curvature interface; the outer encapsulation consists of high-transparency glass and an elastic sealing film.

[0041] Focusing mechanism: By changing the control voltage between the electrodes, the curvature of the liquid-liquid interface is changed by utilizing the electrowetting effect, thereby achieving a continuously adjustable focal length from 0.5D to +15D.

[0042] Drive system: It consists of a high-precision voltage drive module (0-60V range) and a closed-loop current monitoring circuit, and supports PWM modulation control.

[0043] Control system: Higher focal length accuracy is achieved by selecting piezoelectric ceramics to assist fine adjustment.

[0044] Lighting system: It adopts an adjustable intensity strobe light source, which is triggered synchronously with the liquid lens camera to freeze the moment of motion.

[0045] Signal transmission and layout: The liquid lens and camera module are integrated into a single package. The voltage signal is output from the main controller to the electrode layer via a drive amplifier, and the sensing signal is transmitted back via USB 3.0 or fiber optic interface.

[0046] (4) Control and processing unit Main controller (MCU+FPGA architecture): coordinates the acquisition sequence of the first and second level units and generates synchronous trigger signals.

[0047] Image processing module: Runs a two-stage vision algorithm to complete ROI extraction, depth mapping and crack recognition.

[0048] Positioning and synchronization module: integrates an encoder, IMU (Inertial Measurement Unit), and RTK (Real-Time Kinematic) system to label images with spatial coordinates.

[0049] Liquid lens control module: Based on the target distance measured by the depth camera, it calculates the required focal length and outputs the corresponding control voltage / current to achieve fast and automatic focusing.

[0050] V. Work Process and Methods The tunnel crack inspection method based on scene camera and liquid lens provided in this embodiment includes the following steps: 1. System calibration and initialization Joint calibration was performed on the fisheye camera, Kinect depth camera, and liquid lens camera to determine their extrinsic parameter matrices and projection relationships. A digital surface model (DSM) of the tunnel interior wall was established, and the depth and coordinate information of each detection point was recorded.

[0051] 2. Level 1 Detection – Global Scan and Suspected Area Localization The platform moves forward at a certain speed (e.g., 0-60 km / h). A fisheye camera captures panoramic images of the tunnel; a Kinect simultaneously outputs a depth map. The image processing module runs a lightweight neural network (YOLO or MobileNet) to identify potential areas of interest (ROIs) with cracks, outputting the pixel coordinates (u1, v1) and confidence score of the ROIs, and simultaneously obtaining the depth distance Z1 of the corresponding points based on the depth map.

[0052] 3. Coordinate mapping and second-level unit scheduling: Using the joint calibration results, the ROI pixel coordinates and depth information are converted into the physical coordinate system of the liquid lens camera.

[0053] The control unit calculates the target focal length of the liquid lens based on the depth value Z1: , in This is the initial focal length of the liquid lens. For reference plane distance, .

[0054] 4. Liquid lens drive and fast focusing The control unit calculates the control voltage using the electrowetting model and the empirical calibration curve (fV relationship): , Where k is a system constant. This voltage is output to the liquid lens electrode layer, causing the lens interface curvature to adjust within 15ms, thus achieving autofocus.

[0055] 5. Secondary Detection – High-Precision Imaging and Analysis A liquid lens camera performs local high-resolution imaging of the Region of Interest (ROI) and triggers a strobe light source to suppress motion blur. Subsequently, a deep network (such as U-Net) is used to achieve pixel-level segmentation, classification, and width quantization of the crack.

[0056] 6. Data Fusion and Report Generation The measurement results are linked to the spatial coordinates obtained by the positioning module and annotated in the tunnel digital model. This automatically generates a crack distribution map and a structural health report.

[0057] The core idea of ​​this embodiment is to mimic the human eye's "first gaze, then fixation" mechanism, constructing a two-stage collaborative detection system consisting of a large field-of-view scene camera and a liquid lens camera. This system achieves automatic focusing and high-precision crack imaging of the target area through electronic control. The first stage, a scene perception module composed of a large field-of-view scene camera and a depth camera, performs global scanning and spatial ranging to quickly locate suspected defect areas and their depth information. The second stage, a high-resolution liquid lens camera, automatically focuses based on the depth information, performing detailed imaging and crack quantification analysis of the suspected crack area.

[0058] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A tunnel crack inspection system based on a scene camera and a liquid lens, characterized in that, It includes a support platform, a primary detection unit, a secondary detection unit, and a control and processing unit; The support platform is used to simultaneously carry cameras, lighting sources, controllers, and positioning devices to move smoothly in the tunnel; The primary detection unit is used to perform visual acquisition of tunnel surface and spatial distance measurement to support the subsequent focusing of the liquid lens; the primary detection unit is also used to simultaneously acquire visual images and depth data, use visual algorithms to identify suspected crack areas in the panoramic image, and obtain the spatial depth information corresponding to the suspected crack areas through a depth camera. The secondary detection unit is used to perform local magnification processing on the suspected crack area for high-precision imaging. The secondary detection unit includes a liquid lens assembly, a focusing system, a driving system, a control system, an illumination system, and a signal transmission and layout system. The control and processing unit includes a main controller, an image processing module, a positioning and synchronization module, and a liquid lens control module.

2. The tunnel crack inspection system based on a scene camera and a liquid lens according to claim 1, characterized in that, The carrier platform includes a rail-mounted mobile device, a vehicle-mounted mobile device, or a handheld mobile device.

3. The tunnel crack inspection system based on a scene camera and a liquid lens according to claim 2, characterized in that, The primary detection unit includes a wide field-of-view scene camera and a depth camera. The wide field-of-view scene camera is used for visual detection and generates a panoramic image of the tunnel lining surface to identify the suspected crack area. The wide field-of-view scene camera includes a lens group, an imaging device, a shutter control component, an illumination system, and a signal interface.

4. The tunnel crack inspection system based on a scene camera and a liquid lens according to claim 3, characterized in that, The lens assembly is a fisheye lens assembly, including a front lens, a correction lens and an infrared cutoff filter; The imaging device is equipped with a global shutter CMOS sensor to ensure no motion blur during high-speed travel; The shutter control component uses the main controller to send a trigger signal to ensure that the wide field-of-view scene camera and the lighting source are exposed synchronously. The lighting system uses high-brightness linear LED strip light sources to uniformly illuminate the tunnel walls; The signal interface communicates with the main control unit via Gigabit Ethernet or CameraLink interface to transmit real-time image data.

5. The tunnel crack inspection system based on a scene camera and a liquid lens according to claim 4, characterized in that, The depth camera calculates the depth image by projecting infrared structured light and using echoes, and outputs the three-dimensional coordinates of each pixel. The depth camera and fisheye camera are calibrated together, and spatial registration of visual images and depth information is achieved through geometric mapping, providing target distance data for the focusing control of the liquid lens.

6. The tunnel crack inspection system based on a scene camera and a liquid lens according to claim 5, characterized in that, The field of view of the fisheye lens group is ≥120°, the frame rate of the global shutter CMOS sensor is ≥200fps, and the working distance range of the depth camera is 0.5m to 5m.

7. The tunnel crack inspection system based on a scene camera and a liquid lens according to claim 6, characterized in that, The liquid lens assembly employs a dual-liquid-layer electrowetting optical system, comprising a conductive liquid and a non-polar optical oil. The conductive liquid and the non-polar optical oil are isolated by a transparent electrode to form a variable curvature interface. The outer encapsulation of the liquid lens assembly includes high-transparency glass and an elastic sealing film. The focusing system is used to adjust the interface curvature by changing the control voltage between the electrodes to achieve a continuously adjustable focal length. The drive system includes a voltage drive module and a closed-loop current monitoring circuit. The control system is used to perform auxiliary fine-tuning via piezoelectric ceramics to improve focal length accuracy; The lighting system uses an adjustable intensity strobe light source, which is synchronously triggered with the liquid lens camera to freeze the moment of motion. The signal transmission and layout system outputs the voltage signal from the main controller to the electrode layer via a drive amplifier, and transmits the sensing signal back via USB 3.0 or a fiber optic interface.

8. The tunnel crack inspection system based on a scene camera and a liquid lens according to claim 7, characterized in that, The main controller is used to coordinate the acquisition sequence of the primary detection unit and the secondary detection unit and generate a synchronization trigger signal; The image processing module is used to run a two-stage vision algorithm to extract the suspected crack area and complete depth mapping and crack identification. The positioning and synchronization module includes an encoder, an inertial measurement unit, and a real-time dynamic positioning system, used to label the visual image with spatial coordinates. The liquid lens control module is used to calculate the focal length corresponding to the target distance based on the target distance obtained by the depth camera, and output the corresponding control voltage or control current to achieve automatic focusing.

9. A method for inspecting tunnel cracks based on a scene camera and a liquid lens, characterized in that, The tunnel crack inspection method uses the tunnel crack inspection system according to any one of claims 1-8, and the tunnel crack inspection method includes: Joint calibration of fisheye camera, Kinect depth camera and liquid lens camera is performed to determine the extrinsic parameter matrix and projection relationship between fisheye camera, Kinect depth camera and liquid lens camera, establish digital surface model of tunnel inner wall, and record depth and coordinate information of each detection point; The carrying platform moves at a preset speed, the fisheye camera acquires panoramic images of the tunnel, the Kinect depth camera outputs depth images simultaneously, the image processing module uses a neural network to identify the suspected crack area, outputs the pixel coordinates and confidence level of the suspected crack area, and obtains the depth distance of the corresponding detection point based on the depth image. Based on the joint calibration results, the pixel coordinates and depth information of the suspected crack area are converted into the physical coordinate system of the liquid lens camera; The control unit calculates the target focal length of the liquid lens based on the depth distance, using the following expression: ， in, Z1 is the initial focal length of the liquid lens, and Z2 is the depth distance. For reference plane distance, ; The control unit calculates the control voltage using the electrowetting model and empirical calibration curve, as shown in the following expression: ， Where k is a system constant, The target focal length of the liquid lens; The control voltage is output to the electrode layer of the liquid lens to adjust the interface curvature of the liquid lens in order to achieve autofocus; The liquid lens camera performs local high-resolution imaging of the suspected crack area, triggers a strobe light source to suppress motion blur, and uses a deep network to segment, classify, and quantize the width of the crack. The measurement results are bound to the spatial coordinates obtained by the positioning module, and then annotated in the tunnel digital model to automatically generate a tunnel crack health report.

10. The tunnel crack inspection method based on a scene camera and a liquid lens according to claim 9, characterized in that, The step of outputting the control voltage to the electrode layer of the liquid lens to adjust the interface curvature of the liquid lens includes: The control voltage is output to the electrode layer of the liquid lens, so that the interface curvature of the liquid lens is adjusted within 15ms.

Images