Tunnel structure detection system and method integrating laser radar and rotary scanning camera

Abstract

The invention discloses a tunnel structure detection system and method fusing a laser radar and a rotary scanning camera, and the system employs a layered modular design, and comprises a bottom motion unit, a middle support and energy supply unit, an upper detection and synchronization unit, and a top rotary imaging unit. The synchronous control module of the upper layer detection and synchronization unit coordinates the work of each sensor, controls the self-walking servo motor of the bottom layer motion unit to drive the vehicle body to move at a constant speed, and accurately triggers the rotating motor of the top rotating imaging unit to drive the double mirror surfaces to rotate, thereby achieving the 360-degree high-definition imaging of a single camera. The laser radar scanning of the upper detection and synchronization unit is synchronously controlled, and a high-precision crystal oscillator is utilized to simulate GNSS second pulse time service, so that millisecond-level hard synchronization of multi-source data is realized; through high-precision matching fusion of three-dimensional laser point cloud data and image data, integrated high-precision detection of tunnel structure deformation and apparent diseases is completed, and full automation from data acquisition, synchronous fusion to disease identification is realized.

Description

Technical Field

[0001] This invention belongs to the field of detection technology for structural defects in rail transit tunnels, and more specifically, relates to a tunnel structure detection system and method that integrates lidar and a rotating scanning camera. Background Technology

[0002] With the rapid development of rail transit, the operation and maintenance inspection of tunnel structures has become particularly important. Traditional methods for detecting tunnel defects mainly rely on manual inspections and fixed-point measurements using single detection devices. This method is not only inefficient, but the results are also greatly affected by human factors, making it difficult to comprehensively and accurately reflect the actual condition of the tunnel.

[0003] In recent years, 3D laser scanning technology has been increasingly applied to tunnel structure inspection. Simultaneously, inspection methods based on image recognition from multiple area array cameras have also been widely adopted. However, current inspection technologies have several limitations: Firstly, the multi-area array camera technology currently used is limited by the imaging range of a single area array camera, typically requiring more than eight fixed area array cameras to cover the entire circumference of the tunnel. This multi-camera system is not only complex to implement, but also bulky and cumbersome, and the use of numerous sensors significantly increases equipment cost and power consumption. Secondly, most existing inspection methods separate 3D laser scanning from area array camera inspection, making it difficult to precisely align the acquired multi-scale point cloud and image data, or forcing alignment by selecting only some feature points, which limits accuracy. Existing technologies rely on software timing or time-division triggering, resulting in millisecond-level time drift, causing matching deviations between lidar point cloud data and image data, with varying deviations across different data segments. These limitations make it difficult to meet the high-efficiency, accurate, and low-cost requirements for detecting structural defects in rail transit tunnels.

[0004] Therefore, there is an urgent need for a detection method or system that can fuse point clouds of three-dimensional tunnel scenes with high-definition images to improve the efficiency, accuracy and integration of tunnel defect detection, reduce equipment costs and power consumption, and better meet the actual needs of operation and maintenance detection of rail transit tunnels. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a tunnel structure detection system and method integrating LiDAR and a rotating scanning camera. By fixing a single area array camera and flash structure, a small rotating motor controls the rotation of a reflector. A synchronous control program controls the rotating mirror to trigger image capture at specified rotation angles, enabling multi-angle imaging of the tunnel. Simultaneously, LiDAR scans to acquire 3D cross-sectional data, while inertial navigation and odometer components acquire relative pose information. A self-propelled motor drives the carrier forward, achieving simultaneous acquisition of LiDAR scanning data, high-definition image data, and positioning and attitude data. This invention significantly reduces equipment size, cost, and power consumption. By combining 3D laser scanning technology with a single area array camera through rotating imaging, and integrating an inertial navigation and positioning system, the highly integrated design greatly improves the matching degree between image data and 3D point cloud data without affecting image clarity. This enables integrated high-precision detection of tunnel structure deformation and surface defects, achieving full automation from data acquisition and synchronous fusion to defect identification.

[0006] To achieve the above objectives, one aspect of the present invention provides a tunnel structure detection system integrating lidar and a rotating scanning camera, comprising, from bottom to top, a bottom-level motion unit, a middle-level support and energy supply unit, an upper-level detection and synchronization unit, and a top-level rotating imaging unit; wherein...

[0007] The bottom-level motion unit includes a vehicle body, wheels symmetrically arranged on both sides of the vehicle body, a self-propelled servo motor on each wheel, a wheel encoder, and a push rod installed at the rear of the vehicle body for emergency braking; the middle-level support and energy supply unit includes a column vertically fixed to the center of the vehicle body and a power module integrated into the vehicle body frame or on the column base; the upper-level detection and synchronization unit includes a synchronization control module, an industrial control computer storage module, a laser scanning sensor, an inertial measurement sensor, and a GNSS satellite positioning sensor located at the bottom of the mounting frame; the top rotating imaging unit is vertically stacked at the very top of the main body of the equipment and includes an area array camera, a camera rotation motor, a flash and its control module, and a camera rotation mirror located at the top of the mounting frame;

[0008] The synchronous control module coordinates the data collected by each sensor, controls the self-propelled servo motor to drive the vehicle body to move at a constant speed, and precisely triggers the camera rotation motor to drive the double mirror of the camera rotation mirror to rotate, achieving 360° high-definition imaging by a single camera. It also synchronously controls the LiDAR scanning and uses the high-precision crystal module inside the synchronous control module to simulate GNSS second pulse time synchronization, achieving millisecond-level hard synchronization of multi-source data. Based on the attitude angle information output by the inertial measurement sensor, it performs attitude correction on the LiDAR point cloud data collected at each moment and position correction on the rotating camera image data. Through a pre-calibrated spatial transformation matrix, it unifies the data of each sensor to the WGS84 global coordinate system, achieving millimeter-level matching and fusion of point cloud and image data, and realizing integrated high-precision detection of tunnel structure deformation and surface defects.

[0009] Furthermore, the wheel encoder is fixed to the wheel in a concentric and coaxial manner. When the wheel rotates once during data acquisition, the odometer sends a certain pulse count value. The odometer frequency is such that at least one pulse signal should be sent for every millimeter the wheel travels.

[0010] Furthermore, the laser scanning sensor is used to receive simulated PPS pulses from the synchronization control module to achieve time synchronization and acquire three-dimensional laser scanning point cloud data of a large-scale scene;

[0011] The inertial measurement sensor is combined with the GNSS satellite positioning sensor to output the vehicle attitude information at each acquisition time.

[0012] The GNSS satellite positioning sensor, combined with the inertial measurement sensor, provides absolute vehicle positioning information, and simultaneously provides PPS second pulse signals and NEMA positioning data to the synchronization control module, which enables millisecond-level time synchronization of multiple sensors.

[0013] Furthermore, the synchronization control module includes an input control terminal, an FPGA synchronization board, and an output execution terminal; the FPGA synchronization board integrates a high-precision clock chip;

[0014] The input control terminal receives drive control signals from the camera's rotary motor, rotation feedback from the grating code disk, drive control commands from the self-propelled servo motor, gyroscope / accelerometer data from the inertial navigation system, UTC time reference from the clock chip, and distance pulse data from the wheel encoder, and transmits them to the processing terminal. The processing terminal's FPGA synchronization board uses a combination of crystal oscillator timing and PPS pulses to perform millisecond-level time synchronization of the laser scanning sensor, area scan camera, and inertial measurement sensor, and inputs and outputs these signals to the execution terminal. The output execution terminal generates time / distance trigger signals to control the camera's rotary motor angle rotation, strictly synchronize the area scan camera exposure and flash illumination, and simultaneously adjusts the speed of the self-propelled servo motor through a closed-loop signal from the wheel encoder. All raw sensor data and synchronization timescales are stored on the industrial control computer, and the operating frequency is configured through the parameter setting module, achieving spatiotemporal unification of motion control, 3D scanning, and rotational imaging.

[0015] Furthermore, the mounting frame includes a mounting platform located at the top of the column, and three fixed uprights arranged parallel to each other along the track direction on the mounting platform; partitions of different heights are horizontally arranged between adjacent fixed uprights.

[0016] Furthermore, the upper-level detection and synchronization unit is located on the top of the mounting platform; the synchronization control module, the industrial control computer storage module, and the inertial measurement sensor are located on the mounting platform; the synchronization control module and the industrial control computer storage module are located between two fixed upright plates in the direction of the front of the vehicle body, and the inertial measurement sensor is located between two fixed upright plates in the direction of the rear of the vehicle body.

[0017] Furthermore, the area array camera and the camera rotation motor are mounted on a partition near the front end of the vehicle body. One end of the camera rotation motor is connected to the area array camera, and the other end is fixed to the middle fixed plate. The output end of the camera rotation motor passes through the middle fixed plate and is fixed to the camera rotation mirror located on the other side. The flash is mounted on the fixed plate at the rear end of the vehicle body on the side away from the camera rotation mirror.

[0018] Furthermore, the camera rotating motor is a DC permanent magnet brushless motor, and its motor rotating shaft is concentrically and coaxially fixedly connected to the camera rotating mirror;

[0019] The camera rotary motor is equipped with a grating code disk, which controls the rotation of the motor in a programmed manner and feeds back the motor rotation to a specified angle to trigger the area array camera and flash to take pictures.

[0020] The camera rotating mirror includes two mirror surfaces at a 90° angle. The mirror surface closer to the area scan camera is the first mirror surface, and the mirror surface closer to the flash is the second mirror surface. Both mirror surfaces are at a 45° angle to the horizontal plane. The lens of the area scan camera is aligned with the first mirror surface, and the light outlet of the flash is aligned with the second mirror surface of the rotating mirror.

[0021] A second aspect of the present invention provides a tunnel structure detection method that integrates lidar and a rotating scanning camera, implemented using the aforementioned tunnel structure detection system that integrates lidar and a rotating scanning camera, comprising the following steps:

[0022] S1. System installation and initialization: Install the components of the bottom motion unit, middle support and energy supply unit, upper detection and synchronization unit, and top rotating imaging unit of the tunnel structure detection system integrating lidar and rotating scanning camera on the track to be detected according to the design requirements; connect the FPGA synchronization board, industrial control computer, and power module to ensure that all sensors of the system are powered normally; set the initial parameter values ​​of the rotating motor speed, camera trigger interval time, and vehicle forward speed.

[0023] S2. Data Acquisition and Synchronization Control: The industrial control computer sends commands to drive the camera rotation motor to rotate through the FPGA synchronization board of the synchronization control module. The grating code disk provides real-time feedback on the current mirror rotation angle. The FPGA synchronization board triggers the area array camera to take pictures and the flash to provide supplementary lighting based on the feedback value of the grating code disk. The FPGA synchronization board sends a synchronization pulse (PPS) to trigger the LiDAR scanner to scan and embeds the point cloud data with a timestamp, outputting the original sensor data with the timestamp.

[0024] S3. Rotational scanning imaging: The FPGA synchronization board controls the self-propelled servo motor to drive the vehicle forward at a constant speed according to the pulse of the wheel encoder, so that the images of adjacent circles of the camera rotating mirror are arranged in a spiral along the longitudinal axis of the tunnel. The flash intensity is dynamically adjusted according to the tunnel environment, and a high-definition image sequence covering the entire inner wall of the tunnel is output.

[0025] S4. Multi-sensor data fusion: The carrier pose is calculated by fusing inertial navigation and odometry, and the carrier coordinate system is output. The lidar coordinate system is converted to the carrier coordinate system by using known offsets and rotation matrices. The camera coordinate system is converted to the carrier coordinate system by using pre-calibrated sensor parameters. Point cloud and image coordinate transformations are used to unify point cloud and image data to the WGS84 global coordinate system, achieving high-precision matching and fusion of point cloud and image data, and outputting a spatiotemporally aligned tunnel structure detection dataset.

[0026] S5. Identify geometric deformation and apparent cracks based on the spatiotemporally aligned tunnel structure detection data, generate a defect report based on the geometric deformation and apparent crack data, and output a tunnel health status report including the location, type, and severity of the defect; complete the integrated high-precision detection of tunnel structure deformation and apparent defects.

[0027] Further, step S4 includes:

[0028] S41. Inertial Navigation and Odometer Data Fusion: Inertial navigation and odometer data are processed and calculated. Utilizing existing dead reckoning inertial measurement principles, differential calculations are used to obtain the relative mileage, position, and attitude information at each moment of the collected data, consisting of three offsets. Three rotation quantities Construct a 4x4 rotation matrix

[0029] S42. 3D Scanner Coordinate System Construction: Based on the structural design, determine the positional relationship between the phase center of the 3D scanner sensor 33 and the inertial navigation center to construct the carrier coordinate system, which consists of three offsets. Three rotation quantities Construct a 4x4 rotation matrix

[0030] S43. Camera Coordinate System Construction: Based on the structural design, determine the positional relationship between the camera's imaging center and the inertial navigation center to construct the carrier coordinate system, which consists of three offsets. Three rotation quantities Construct a 4x4 rotation matrix

[0031] S44. Image Rotation Angle Calculation: Based on the synchronization information of each frame of the rotating camera-triggered image, obtain the grating code disk value A at time t, with one circle of the grating scale being N, and construct the rotation matrix for the image rotation angle j.

[0032] S45. Image Center Coordinate Calculation: Calculate the absolute coordinates of the image center in the WGS84 system at the moment the camera rotates to trigger the frame, based on the camera coordinate system and the image rotation angle. Transform the image data to the global coordinate system; Image center coordinates P at moment t (moment of camera rotation trigger frame). cam84 The absolute coordinate expression in the WGS84 system is:

[0033]

[0034] S46. Coordinate Transformation of 3D Laser Scanning Point Cloud Data: The absolute coordinates of any original point P acquired by the 3D laser scanner at any given time are calculated using a rotation matrix transformation, converting the 3D scanning data to a global coordinate system; the coordinates of any original point P acquired by the 3D laser scanner at any given time t are... laser =(X tlas ,Y tlas Z tlas The matrix is ​​represented as Rlaser, and its absolute coordinate calculation expression is:

[0035]

[0036] The absolute coordinates P of the 3D laser scanning point in the WGS84 system laser84 The expression is:

[0037]

[0038] The matching and fusion of rotating camera images and 3D scanning point cloud data are realized through the calculation of formulas (4) to (6), and a spatiotemporally aligned tunnel structure detection dataset is output.

[0039] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0040] (1) The tunnel structure detection system and method of the present invention, which integrates lidar and rotating scanning camera, addresses the problem that existing technologies typically require more than eight fixed area array cameras to cover the entire circumference of the tunnel for imaging, resulting in bulky equipment, high power consumption, and high cost. Instead of the traditional multi-camera array, it adopts a combination of a single area array camera, a rotating mirror, and a flash. By integrating a camera rotation motor and a grating code disk, combined with a specific style of rotating mirror module, the rotating mirror is driven by the motor to rotate, achieving 360° full-scene imaging. This enables dynamic acquisition of full-scene image data with supplementary lighting effect while the camera module remains stationary. This method achieves the effect of a single camera acquiring full-tunnel scene image data through rotating mirror supplementary lighting, significantly reducing the number of hardware components, significantly reducing system hardware costs, and reducing hardware size, making the system lighter, more compact, and easier to move for inspection, further improving the practicality of the product application. In addition, by rigidly fixing sensors such as lidar, inertial navigation (IMU), GNSS, and rotating cameras onto the same platform, the position and angle relationship of each sensor are fixed, and they are calibrated once at the factory, avoiding the repeated calibration problems caused by the separate installation of traditional methods, thus improving the accuracy of data fusion; and the system complexity is reduced through highly integrated design.

[0041] (2) The tunnel structure detection system and method of the present invention, which integrates lidar and rotating scanning camera, addresses the problem of millisecond-level time drift caused by the reliance on software timing or time-division triggering in the prior art, which leads to the deviation in the matching between point cloud and image. It utilizes the high-precision characteristics of a unified high-precision crystal oscillator timing system to achieve hard synchronization, so that the positional relationship between image and point cloud at any acquisition time is fixed. It adopts FPGA synchronization board + crystal oscillator timing + GNSS PPS calibration to achieve microsecond-level time synchronization, ensuring strict alignment of lidar scanning, camera exposure, and inertial navigation data. The rotation angle is fed back in real time by the grating code disk, and the FPGA dynamically adjusts the trigger timing to eliminate accumulated errors, so that the spatial correspondence between each image and point cloud is accurately fixed. That is, the spatiotemporal uniformity is ensured by the hard synchronization mechanism of FPGA + high-precision crystal oscillator and the closed-loop feedback control of grating code disk.

[0042] (3) The tunnel structure detection system and method of the present invention, which integrates lidar and rotating scanning camera, takes spiral images along the tunnel axis when the vehicle moves at a constant speed. Combined with encoder displacement feedback, it ensures that the images are complete and do not overlap, covering the entire inner wall of the tunnel. The position and attitude angle information of the platform carrier in the absolute coordinate system (WGS84 relative coordinate system) can be obtained at any time through inertial measurement sensor. The position and angle relationship between each sensor is determined through high integration calibration, so that the rotating camera image data and lidar point cloud data achieve a high degree of matching data fusion effect. The carrier pose is calculated by inertial navigation + odometer fusion. Combined with the pre-calibrated sensor offset matrix, the point cloud and image are unified to the WGS84 global coordinate system to achieve millimeter-level precision data matching and avoid the error of relying on manual feature point alignment in traditional methods.

[0043] (4) The tunnel structure detection system and method of the present invention, which integrates lidar and rotating scanning camera, can simultaneously detect tunnel structure deformation (such as misalignment and encroachment) and surface defects (such as cracks and spalling) through high-precision matching point cloud and image data, avoiding the missed detection problem of traditional step-by-step detection; realizing full automation from data acquisition, synchronous fusion to defect identification; the present invention solves the core problems of existing technology equipment such as bulky, large synchronization error, high cost and low matching accuracy through the innovative design of single-camera rotating scanning + hard synchronization + high integration calibration, providing an efficient, accurate and low-cost integrated solution for rail transit tunnel detection, and has significant engineering application value. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the overall structure of a tunnel structure detection system that integrates lidar and a rotating scanning camera, according to an embodiment of the present invention.

[0045] Figure 2 This is an enlarged structural diagram of the upper detection and synchronization unit and the top rotating imaging unit of a tunnel structure detection system integrating lidar and rotating scanning camera according to an embodiment of the present invention.

[0046] Figure 3 This is a schematic diagram of the grating code disk scale of a tunnel structure detection system integrating lidar and rotating scanning camera according to an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of a rotating mirror supplementary lighting photograph of a tunnel structure detection system that integrates lidar and a rotating scanning camera, according to an embodiment of the present invention.

[0048] Figure 5 This is a schematic diagram of the overlapping images of adjacent circles of a rotating mirror in a tunnel structure detection system that integrates lidar and a rotating scanning camera, according to an embodiment of the present invention.

[0049] Figure 6 This is a schematic diagram of the contents of the area array camera trigger synchronization information file of a tunnel structure detection system that integrates lidar and rotating scanning camera according to an embodiment of the present invention.

[0050] Figure 7 This is a schematic diagram of the synchronization control principle of a synchronization control module in a tunnel structure detection system that integrates lidar and a rotating scanning camera, according to an embodiment of the present invention.

[0051] Figure 8 This is a flowchart illustrating a tunnel structure detection method that integrates lidar and a rotating scanning camera, according to an embodiment of the present invention.

[0052] In all the accompanying drawings, the same reference numerals denote the same technical features, specifically: 1-bottom motion unit, 11-vehicle body, 12-wheel, 13-self-propelled servo motor, 14-wheel encoder, 15-push rod, 2-middle support and energy supply unit, 21-column, 22-power module, 23-mounting bracket, 231-mounting platform, 232-fixed upright plate, 233-partition, 3-upper detection and synchronization unit, 31-synchronization control module, 32-industrial computer storage module, 33-laser scanning sensor, 34-inertial measurement sensor, 4-top rotating imaging unit, 41-area array camera, 42-camera rotation motor, 43-flash lamp, 44-camera rotating mirror. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0054] In this invention, the industrial control computer, as an industrial control computer, has powerful data processing capabilities and a stable operating environment, and can process and analyze data from different sensors in real time; the industrial control computer storage module is used to realize data storage and processing.

[0055] Laser scanning sensors, also known as lidar, are used to collect three-dimensional scanning data of tunnel structures. They measure the distance to objects by emitting laser beams and receiving the reflected light, thus constructing a three-dimensional point cloud model of the tunnel's interior. This data is crucial for detecting deformation, cracks, and other defects in tunnel structures.

[0056] An inertial measurement unit (IMU) is a core component of an inertial navigation system (INS), providing raw acceleration and angular velocity data. This data forms the basis for the INS to calculate position and attitude. Through the IMU, the INS can calculate the changes in position and attitude of the carrier in the absence of external reference signals, providing precise positioning and navigation information for tunnel inspection equipment.

[0057] A GNSS (Global Navigation Satellite System) positioning sensor is a device that uses a global satellite navigation system for positioning and navigation. It determines the receiver's precise position, velocity, and time information by receiving signals from multiple satellites. In tunnel structure inspection systems, GNSS positioning sensors provide the absolute position information of the equipment relative to the Earth's coordinate system; used in conjunction with an inertial measurement unit (IMU), GNSS can provide an external position reference for the IMU to correct for accumulated errors.

[0058] A high-precision crystal oscillator module is an electronic component capable of generating highly accurate and stable frequency signals. A crystal oscillator can provide a high-precision time reference for time synchronization. GNSS satellite positioning sensors can receive precise time information from satellites, i.e., pulses per second (PPS). This pulse signal can be used to synchronize all sensors and systems on the device. In tunnels or other environments where GNSS signals cannot reach, the crystal oscillator module inside the synchronization control module 31 can simulate this pulse-per-second signal, providing a unified time reference for the system.

[0059] like Figure 1 and Figure 2As shown, one aspect of the present invention provides a tunnel structure detection system integrating lidar and a rotating scanning camera, which adopts a layered modular design, including a bottom-level motion unit 1, a middle-level support and energy supply unit 2, an upper-level detection and synchronization unit 3, and a top rotating imaging unit 4 arranged sequentially from bottom to top; the bottom-level motion unit 1 includes a vehicle body 11, wheels 12 symmetrically arranged on both sides of the vehicle body 11, a self-propelled servo motor 13 on each wheel 12, a wheel encoder 14, and a push rod 15 installed at the rear of the vehicle body 11 for emergency braking; the middle-level support and energy supply unit 2 includes a column 21 vertically fixed to the center of the vehicle body, integrated into the vehicle body frame. The power module 22 is located inside the frame or on the column base; the column 21 ensures the stability of the upper detection equipment during movement and isolates it from track vibration; the power module 22 supplies power to all electronic modules (synchronization board, industrial computer, sensors) via cables; the upper detection and synchronization unit 3 is fixed to the top of the column and includes core sensors and controllers, including a synchronization control module 31, an industrial computer storage module 32, a laser scanning sensor 33, an inertial measurement sensor 34, and a GNSS satellite positioning sensor located at the bottom of the mounting frame 23; the top rotating imaging unit 4 is vertically stacked at the very top of the main body of the equipment and includes an area array camera 41 and a camera rotation motor located at the top of the mounting frame 23. 42. Flash lamp and its control module 43, and camera rotating mirror 44; The tunnel structure detection system of the present invention achieves stable movement of the equipment on the track through the bottom motion unit 1, the middle support and energy supply unit 2 provides necessary support and power supply for the entire system, the upper detection and synchronization unit 3 achieves accurate data acquisition and synchronization by integrating multiple sensors and controllers, and the top rotating imaging unit 4 acquires high-definition images of the tunnel inner wall by rotating the camera and flash lamp; the synchronization control module 31 coordinates the work of each sensor: controls the servo motor to drive the vehicle body to move at a constant speed, and accurately triggers the camera rotating motor 42 to drive the double mirror of the camera rotating mirror 44 to rotate, realizing single The camera provides 360° high-definition imaging; simultaneously, it performs synchronous LiDAR scanning and utilizes the high-precision crystal oscillator module inside the synchronization control module 31 to simulate GNSS second pulse timing, achieving millisecond-level hard synchronization of multi-source data; based on the attitude angle information output by the inertial measurement sensor 14, it performs attitude correction on the LiDAR point cloud data collected at each moment and position correction on the rotated camera image data; through a pre-calibrated spatial transformation matrix, it unifies the data from each sensor to the WGS84 global coordinate system, achieving millimeter-level matching and fusion of point cloud and image data, realizing integrated high-precision detection of tunnel structure deformation and apparent defects, and achieving full automation from data acquisition, synchronous fusion to defect identification.

[0060] Furthermore, the vehicle body 11, serving as a basic load-bearing platform, includes crossbeams and vertical beams, and is installed on a track to provide physical support for the equipment mounted on it. It runs on the track by being fixed to the main body of the equipment via columns. The wheels 12 are rigidly connected to the crossbeams of the vehicle body 11 via bearings, serving as the wheels for the vehicle body's movement on the track. Each wheel is equipped with a wheel encoder 14 to record forward pulses, and a self-propelled servo motor 13 controls the vehicle's self-movement. Typically, there are four or more wheels. The self-propelled servo motor 13 is installed on a single wheel 12, achieving single-point drive. The self-propelled servo motor 13 is fixedly connected to the rotating shaft of the wheel 12 in a concentric and coaxial manner. The motor is programmed with a driver to achieve forward and reverse rotation, thereby realizing the vehicle's forward or backward movement. The wheel encoder 14 is fixedly connected to the wheel in a concentric and coaxial manner. During data acquisition, when the wheel rotates one revolution, the odometer sends a certain pulse count value. The odometer frequency should ensure that at least one or more pulse signals are sent for every millimeter the wheel travels, so as to achieve accurate mileage feedback. The push rod 15 is used to control the vehicle to stop or move when the equipment is not powered on or in an emergency.

[0061] Furthermore, the mounting frame 23 includes a mounting platform 231 located on the top of the column 21, and three fixed uprights 232 arranged parallel to each other along the track direction on the mounting platform 231; and partitions 233 of different heights are horizontally arranged between two adjacent fixed uprights 232.

[0062] Furthermore, the upper detection and synchronization unit 3 is located on the top of the mounting platform 231; the synchronization control module 31, the industrial control computer storage module 32, and the inertial measurement sensor 34 are located on the mounting platform 231; the synchronization control module 31 and the industrial control computer storage module 32 are located between two fixed upright plates 232 in the direction of the front of the vehicle body, and the inertial measurement sensor 34 is located between two fixed upright plates 232 in the direction of the rear of the vehicle body.

[0063] Furthermore, the area array camera 41 and the camera rotation motor 42 are mounted on a partition 233 near the front end of the vehicle body. One end of the camera rotation motor 42 is connected to the area array camera 41, and the other end is fixed to the middle fixed plate 232. The output end of the camera rotation motor 42 passes through the middle fixed plate 232 and is fixed to the camera rotation mirror 44 located on the other side. The flash 43 is mounted on the fixed plate 232 in the direction of the rear end of the vehicle body, away from the camera rotation mirror 44.

[0064] Furthermore, the camera viewport of the area array camera 41 faces the direction of the camera rotating mirror 44, and is used to receive instructions from the synchronization control module 31 to take pictures and transmit image data to it; the camera rotating motor 42 is a DC permanent magnet brushless motor, and its motor rotation shaft is concentrically and coaxially fixedly connected to the camera rotating mirror 44; the camera rotating motor 42 is equipped with a grating code disk, which controls the rotation of the motor in a programmed manner and feeds back the motor rotation to a specified angle to trigger the area array camera 41 and the flash 43 to take pictures. The flash 43 is used to receive the trigger signal from the synchronization control module 31 to trigger the flash, thereby achieving the fill light effect for the camera in the dark environment of the tunnel. One end of the camera rotating mirror 44 is coaxially fixed to the camera rotating motor 42 via a coupling, and rotates coaxially with the motor. The camera rotating mirror 44 includes two mirror surfaces at a 90° angle, the one closer to the area scan camera 41 is the first mirror surface, and the one closer to the flash 43 is the second mirror surface. Both mirror surfaces are at a 45° angle to the horizontal plane. The lens of the area scan camera 41 is aligned with the first mirror surface; the light outlet of the flash 43 is aligned with the second mirror surface of the rotating mirror. The mirror surface; during operation, the flash lamp 43 is controlled by the synchronization control module 31 to emit light, which is reflected by the 45° second mirror surface and illuminates the tunnel wall. The light reflected from the tunnel wall is reflected by the 45° first mirror surface and enters the lens of the area array camera 41. The rotation of the camera rotation motor 42 is controlled by the synchronization control module 31 to drive the rotation of the camera rotation mirror 44 and the grating code disk mounted on the camera rotation motor 42 to provide real-time angle feedback, so that the FPGA synchronization board of the synchronization control module 31 triggers the area array camera 41 and the flash lamp 43 to expose synchronously at the target angle (e.g., every 5000 grating values).

[0065] Furthermore, the synchronization control module 31 controls the self-propelled servo motors 13 on the wheels to drive the vehicle body 11 forward or backward, achieving self-propelled movement; it coordinates the data acquisition of each sensor through electrical signals; it controls the camera rotation motor 42 to precisely rotate to a set angle, triggering the area array camera 41 to take pictures and the flash 43 to flash, and records the specific time of the camera taking pictures, acquiring high-definition images of the tunnel wall; the synchronization control module 31 is equipped with a high-precision crystal oscillator module, which generates a time base, and achieves millisecond-level precise time synchronization of the data collected by the inertial measurement sensor 34, odometer, area array camera 41, and laser scanning sensor 33 by combining active synchronization and passive time synchronization strategies; this process ensures the synchronous acquisition of all sensor data, providing accurate timestamp information for subsequent data processing and analysis. Hard synchronization refers to achieving synchronization of multiple data sources through precise time control at the hardware level; in this invention, it means that the data acquisition of all sensors in the system is based on the same precise time base, thereby achieving data synchronization within millisecond-level time accuracy. In this way, even in the absence of GNSS signals, the system can ensure the temporal consistency of all sensor data. When the lidar scans the tunnel structure and generates point cloud data, while the camera captures images of the tunnel's inner walls, these data need to be precisely synchronized in time to correctly fuse the image data with the point cloud data, thereby achieving accurate detection of tunnel structural deformation and apparent defects. This synchronization method improves the accuracy and reliability of data processing and is one of the key technologies for realizing automated tunnel structure inspection.

[0066] Furthermore, the industrial control computer storage module 32, adjacent to the synchronization control module 31, is used for user interaction to collect and store data from various sensors; the laser scanning sensor 33, fixed to the side or top of the main body of the equipment, facing the tunnel wall, is used to receive simulated PPS pulses from the synchronization control module 31 to achieve time synchronization and acquire three-dimensional laser scanning point cloud data of a large-scale scene; the inertial measurement sensor 34 is rigidly fixed at the geometric center of the main body of the equipment, defining the origin of the vehicle coordinate system; combined with the GNSS satellite positioning sensor, it outputs the vehicle attitude information at each acquisition moment; the GNSS satellite positioning sensor, combined with the inertial measurement sensor 34, provides absolute vehicle positioning information, and simultaneously provides PPS second pulse signals and NEMA positioning data to the synchronization control module 31, which achieves millisecond-level time synchronization of multiple sensors. NEMA positioning data is a standard GPS data format that includes information such as position, speed, and heading.

[0067] Furthermore, the synchronization control module 31, through its built-in high-precision crystal oscillator module, uses the system's own time as a reference to fuse the vehicle's position, distance, and time information obtained from the laser scanning sensor 33, inertial measurement sensor 34, and GNSS satellite positioning sensor. Simultaneously, it controls the camera rotation motor 42 to rotate, triggering the area array camera 41 to take pictures and the flash 43 to flash, and controls the self-propelled servo motor 13 to rotate forward and backward to achieve the vehicle's forward or backward movement. The synchronization logic of the synchronization control module 31 is as follows: Figure 7 As shown; the synchronization control module 31 includes an input control terminal, an FPGA synchronization board, and an output execution terminal; the FPGA synchronization board is the hard synchronization hub of the entire system, integrating a high-precision clock chip; it receives the drive control signal of the camera rotary motor 42, the rotation feedback of the grating code disk, the drive control command of the self-propelled servo motor 13, the gyroscope / accelerometer data of the inertial navigation system, the UTC time reference of the clock chip, and the distance pulse data of the wheel encoder 14 through the input control terminal and transmits them to the processing terminal; the processing terminal FPGA synchronization board adopts a combination of crystal oscillator timing and PPS pulse to control the laser scanning. Sensor 33, area scan camera 41, and inertial measurement sensor 34 are synchronized at the millisecond level and input to the output execution terminal; the output execution terminal generates a time / distance trigger signal to control the angle rotation of camera rotation motor 42 (triggered every 5000 grating values), strictly synchronize the exposure of area scan camera 41 and the supplementary lighting of flash lamp 43, and simultaneously adjust the speed of self-propelled servo motor 13 through the signal closed loop of wheel encoder 14; all sensor raw data and synchronization time scales are stored in the industrial control computer, and the working frequency is configured through the parameter setting module to achieve spatiotemporal unification of motion control, 3D scanning and rotation imaging.

[0068] During operation, the industrial control computer sends rotation commands to the camera rotary motor 42 via the FPGA synchronization board of the synchronization control module 31. Simultaneously, the camera rotary motor 42 rotates, and its grating code disk provides feedback on the current mirror rotation angle. Figure 3 As shown, one rotation of the camera rotary motor 42 can be considered as the grating code disk being evenly divided into 80000 scales (note that different grating code disks have different scale divisions, but they can all be expressed as a 360° even division).

[0069] Next, the FPGA synchronization board determines whether the camera has reached the trigger time for taking a picture based on the current feedback value from the raster code disk. Here, camera rotation for taking pictures means that the camera rotation motor rotates one revolution and takes a fixed number of uniformly captured image data, such as... Figure 3 As shown, when the raster value feedback is 0, 5000, 10000..., the FPGA synchronization board sends a trigger signal to the camera and flash, causing the flash to fire and the camera to capture an image. Simultaneously, it stores a camera trigger synchronization information file composed of the time information and raster value information at that moment. The synchronization file content information is as follows... Figure 6As shown in the figure, each column represents the trigger frame information, which includes: frame number, trigger time (date, hour, minute, second, millisecond, microsecond), grating code disk value, and wheel encoder value information, separated by commas;

[0070] Subsequently, the camera rotation motor 42 rotates, causing the camera rotation mirror 44 to rotate coaxially. A schematic diagram of the fill light from the camera rotation mirror 44 is shown below. Figure 4 As shown, the instantaneous high-brightness light emitted by the flash 43 is reflected at a 45° angle by the mirror adjacent to the flash and applied to the target area; at the same time, the area scan camera 41 reflects the target area highlighted by the flash 43 back to the camera lens through a 45° angle reflection by the mirror adjacent to the area scan camera, thereby realizing the camera rotation supplementary lighting for taking pictures; according to the above operation, the camera achieves high-brightness supplementary lighting for taking pictures at the position corresponding to the fixed grating code value, and the camera rotation motor achieves the acquisition of a fixed number of image data during one rotation;

[0071] When the vehicle stops, the area array camera 41 will continuously rotate and take pictures along the fixed cross-section with the camera rotation motor 42; in order to achieve full-scene image acquisition of the tunnel, the self-propelled servo motor 13 on the wheel 12 is controlled by the instructions of the FPGA synchronization board of the synchronization control module 31 to drive the detection vehicle to move forward at a constant speed, ensuring the spiral acquisition of multi-angle image data of the tunnel structure; wherein, the acquired multiple images are unfolded along the tunnel cross-section, and the adjacent circles of images are arranged in a spiral pattern, such as Figure 5 As shown, N is the maximum number of photos taken in one revolution of the camera's rotating motor; in order to minimize the overlap and missing values ​​of images at the same raster value in adjacent revolutions, the forward speed of the wheel self-propelled motor is adjusted as needed.

[0072] The self-propelled servo motor 13 controls the movement of the detection vehicle, and collects odometer data with synchronized board crystal oscillator time, inertial navigation data, image data and trigger time stamp data of camera-triggered photos, as well as large scene cross-sectional point cloud data with synchronized board crystal oscillator time information of 3D laser scanner with the scanner phase center as the origin.

[0073] Finally, the collected point cloud, image, and pose data are processed, fused, and unified into a global coordinate system through the FPGA synchronization board of the synchronization control module 31, and a spatiotemporally aligned tunnel structure detection dataset is output and applied to subsequent tunnel structure defect detection and identification.

[0074] The FPGA synchronization board of the synchronization control module 31 processes and fuses the collected point cloud, image, and pose data and unifies them to a global coordinate system, outputting a spatiotemporally aligned tunnel structure detection dataset, including:

[0075] Inertial navigation and odometry data fusion: Inertial navigation and odometry data are processed and calculated. Utilizing existing dead reckoning inertial measurement principles, differential calculations are used to obtain the relative mileage, position, and attitude information at each moment (at a given data output frequency). The position and attitude information is regularized into a right-handed coordinate system with the inertial navigation geometric center as the origin, the Z-axis pointing upwards, and the Y-axis pointing in the forward direction—the carrier coordinate system. This yields the position and attitude relationship of the inertial navigation carrier coordinate system at any time t within the global coordinate system, i.e., the precise position and attitude information of the tunnel structure. Specifically, this is achieved through three offsets... Three rotation quantities Construct a 4x4 rotation matrix Offset refers to the translational distance of the phase center of a sensor (such as a 3D scanner or camera) relative to the inertial navigation center in the carrier coordinate system; rotation refers to the rotation angle of the phase center of the sensor relative to the inertial navigation center in the carrier coordinate system, usually expressed as Euler angles.

[0076] 3D Scanner Coordinate System Construction: Based on the structural design, the positional relationship between the phase center of the 3D scanner sensor 33 and the inertial navigation center is determined, and a 4*4 rotation matrix is ​​constructed. To provide an accurate coordinate system for 3D scanning data; specifically, with the carrier coordinate system as the reference, the relationship between the cross-sectional coordinate system of the 3D laser scanner and the carrier coordinate system defined by the positioning and orientation system is determined by three offsets. Three rotation quantities Construct a 4x4 rotation matrix

[0077]

[0078] in,

[0079]

[0080] d1 = 0

[0081] d2=0

[0082] d3 = 0

[0083] d4=1

[0084] Camera coordinate system construction: Based on the structural design, determine the positional relationship between the camera's imaging center and the inertial navigation center in the carrier coordinate system, and construct a 4x4 rotation matrix. Specifically, taking the carrier coordinate system as a reference, the relationship between the camera image coordinate system and the carrier coordinate system defined by the positioning and attitude determination system consists of three offsets. Three rotation quantities Construct a 4x4 rotation matrix

[0085]

[0086] in,

[0087]

[0088] h1 = 0

[0089] h2 = 0

[0090] h3 = 0

[0091] h4=1

[0092] Image rotation angle calculation: Based on the synchronization information of each frame of the rotating camera-triggered image, the grating code disk value A at time t is obtained, and one circle of the grating is N. Then, the image rotation angle j is expressed as a rotation matrix. The coordinates are expressed as follows:

[0093]

[0094] Image center coordinate calculation: Based on the camera coordinate system and image rotation angle, calculate the absolute coordinates of the image center in the WGS84 system at the moment the camera rotates to trigger the frame, and transform the image data to the global coordinate system; image center coordinates P at moment t of the camera rotation trigger frame. cam84 The absolute coordinate expression in the WGS84 system is:

[0095]

[0096] 3D laser scanning point cloud data coordinate transformation: The absolute coordinates of any original point P of the 3D laser scanner at any time t are calculated using a rotation matrix transformation, and the 3D scanning data is transformed into a global coordinate system; laser =(X tlas ,Y tlas Z tlas The matrix is ​​represented as Rlaser, and its absolute coordinate calculation expression is:

[0097]

[0098] The absolute coordinates of the 3D laser scanning point in the WGS84 system are obtained by using the rotation matrix of the inertial navigation carrier coordinate system in the global coordinate system, the rotation matrix of the 3D scanner coordinate system in the carrier coordinate system, and the rotation matrix of any original point coordinates of the 3D laser scanner.

[0099] The absolute coordinates P of the 3D laser scanning point in the WGS84 system laser84 The expression is:

[0100]

[0101] in, The rotation matrix is ​​used to transform the inertial navigation vehicle coordinate system to the WGS84 system; To convert the 3D scanner coordinate system to the carrier coordinate system using the rotation matrix R, laser Rotation matrix of arbitrary original point coordinates of a 3D laser scanner;

[0102] The matching and fusion of rotating camera images and 3D scanning point cloud data are realized by formulas (4) to (6), and a spatiotemporally aligned tunnel structure detection dataset is output for subsequent tunnel structure defect detection and identification.

[0103] This invention provides a tunnel structure detection system integrating LiDAR and a rotating scanning camera. Through multi-sensor integrated technology, it rigidly connects LiDAR, area array camera, inertial measurement unit (IMU), GNSS, and other sensors to the same carrier platform. A one-time factory calibration determines the spatial positional relationship of each sensor, ensuring the consistency of the spatial system and solving the problems of repeated calibration and spatial matching errors caused by traditional separate installations. A synchronization control module receives the PPS (pulse per second) signal from the GNSS and uses FPGA hardware triggering to achieve microsecond-level time synchronization of each sensor. This provides precise hardware trigger signals for the area array camera and LiDAR, and records accurate moments, ensuring all sensor data have a unified time reference. An innovatively designed dual-mirror rotating mirror module is coaxially connected to a rotating motor, integrating a high-precision grating code disk to provide real-time feedback of the rotation angle. Precise angle control is achieved through FPGA programming. The system acquires key information such as the timing and angle of camera exposure and flash illumination, and performs attitude correction on the lidar point cloud data collected at each moment based on the attitude angle information output by the inertial measurement sensor and position correction on the rotating camera image data. A pre-calibrated spatial transformation matrix unifies the data from each sensor to the WGS84 global coordinate system, achieving millimeter-level matching accuracy between point cloud and image data. Based on spatiotemporally unified multi-source data, it synchronously detects tunnel lining structure deformation (misalignment, encroachment, etc.) and surface defects (cracks, spalling, water leakage, etc.), enabling a comprehensive assessment of the tunnel structure's health status. Through the organic combination of multi-sensor integrated technology, high-precision hard synchronization timing technology, rotating scanning imaging technology, and multi-source data fusion processing technology, it solves key problems in existing technologies such as bulky equipment, large synchronization errors, and low matching accuracy, achieving high precision, high efficiency, and low cost in tunnel structure detection.

[0104] like Figure 8 As shown, a second aspect of the present invention provides a tunnel structure detection method that integrates lidar and a rotating scanning camera, comprising the following steps:

[0105] S1. System Installation and Initialization: Install the components of the tunnel structure detection system integrating LiDAR and rotating scanning camera—including the bottom motion unit 1, the middle support and energy supply unit 2, the upper detection and synchronization unit 3, and the top rotating imaging unit 4—on the track to be inspected according to the design requirements; connect the FPGA synchronization board, industrial computer, and power module 22 to ensure that all sensors of the system are powered normally; start the system and check the communication status of each sensor (such as LiDAR point cloud output, camera exposure test, and inertial navigation data feedback); set the initial parameter values ​​for the rotating motor speed, camera trigger interval time, and vehicle forward speed: for example, rotating motor speed (e.g., 10 rpm), camera trigger interval (e.g., 1 image taken every 5000 raster values), and vehicle forward speed (e.g., 0.2 m / s, matching the shooting frequency).

[0106] S2. Data Acquisition and Synchronization Control: The industrial control computer sends commands to the FPGA synchronization board of the synchronization control module 31 to drive the camera rotation motor 42 to rotate. The grating code disk provides real-time feedback on the current mirror rotation angle. The FPGA synchronization board triggers the area array camera 41 to take pictures and the flash 43 to provide supplementary light based on the feedback value from the grating code disk. The FPGA synchronization board sends a synchronization pulse (PPS) to trigger the lidar scanner to scan and embeds the point cloud data with a timestamp, outputting the original sensor data with the timestamp.

[0107] S3. Rotational scanning imaging: The FPGA synchronization board controls the self-propelled servo motor 13 to drive the vehicle body 11 forward at a constant speed according to the pulse of the wheel encoder 14, so that the adjacent circles of the rotating mirror of the camera are spirally arranged along the longitudinal axis of the tunnel. The intensity of the flash lamp is dynamically adjusted according to the tunnel environment (brightness and darkness changes), and a high-definition image sequence covering the entire inner wall of the tunnel is output. Each image is associated with: timestamp, grating angle and encoder displacement.

[0108] S4. Multi-sensor data fusion: The carrier pose (position + attitude) is calculated by fusing inertial navigation and odometry data, and the carrier coordinate system is output (origin: inertial navigation center, Z-axis points to the sky, Y-axis points to the forward direction); the lidar coordinate system is converted to the carrier coordinate system using known offsets and rotation matrices; the camera coordinate system is converted to the carrier coordinate system using pre-calibrated sensor parameters; point cloud coordinate transformation and image coordinate transformation are used to unify the point cloud and image to WGS84 global coordinates, achieving high-precision matching and fusion of point cloud and image data, and outputting a spatiotemporally aligned tunnel structure detection dataset, including 3D point cloud (including absolute coordinates) and high-resolution images (including shooting position and angle);

[0109] S5. Identify geometric deformation and apparent cracks based on the spatiotemporally aligned tunnel structure detection data, generate a defect report (such as crack location, width, and deformation amount) based on the geometric deformation and apparent crack data, and output a tunnel health status report including defect location, type, and severity; complete the integrated high-precision detection of tunnel structure deformation and apparent defects.

[0110] Further, step S2 specifically includes: generating a system time reference based on the high-precision crystal oscillator (±1ppm) inside the FPGA synchronization board of the synchronization control module 31; receiving the PPS second pulse from GNSS to calibrate the UTC time (outside the tunnel); and relying on the crystal oscillator for autonomous timekeeping inside the tunnel; the industrial control computer sends a rotation command to the camera rotation motor 42 through the FPGA synchronization board of the synchronization control module 31; while the camera rotation motor 42 rotates, it feeds back the current mirror rotation angle through the grating code disk on it; the FPGA synchronization board determines whether the camera trigger shooting time has been reached based on the feedback value of the grating code disk; at a specific grating value, it sends a trigger signal to the camera and flash, causing the flash to fire and the camera to capture an image, and simultaneously stores the time. The camera trigger synchronization information file is composed of time information and grating value information; the camera rotation motor rotates, driving the rotating mirror to rotate coaxially, and the instantaneous high-brightness light emitted by the flash is reflected by the rotating mirror to the target area for photography; at the same time, the area array camera reflects the high-brightness target area back to the camera lens through the rotating mirror, realizing the camera rotation supplementary lighting for photography; during the process of the camera rotation motor rotating one revolution, a fixed frame of image data is captured, acquiring multi-angle image data of the tunnel structure; the FPGA synchronization board sends a synchronization pulse (PPS) to trigger the LiDAR scanner to scan, and embeds the point cloud data with timestamps, outputting the original sensor data with timestamps, including LiDAR point cloud, camera image, inertial navigation attitude, grating code disk angle, and encoder displacement.

[0111] Furthermore, step S4 specifically includes:

[0112] S41. Inertial Navigation and Odometer Data Fusion: Inertial navigation and odometer data are processed and calculated. Utilizing existing dead reckoning inertial measurement principles, differential calculations are used to obtain the relative mileage, position, and attitude information for each moment (at a given data output frequency). The position and attitude information is regularized into a right-handed coordinate system with the inertial navigation geometric center as the origin, the Z-axis pointing upwards, and the Y-axis pointing in the forward direction—the carrier coordinate system. This yields the position and attitude relationship of the inertial navigation carrier coordinate system at any given time t within the global coordinate system, i.e., the precise position and attitude information of the tunnel structure. Specifically, this is achieved through three offsets... Three rotation quantities Construct a 4x4 rotation matrix

[0113] S42. 3D Scanner Coordinate System Construction: Based on the structural design, determine the positional relationship between the phase center of the 3D scanner sensor 33 and the inertial navigation center to construct the carrier coordinate system, and build a 4*4 rotation matrix. To provide an accurate coordinate system for 3D scanning data; specifically, with the carrier coordinate system as the reference, the relationship between the cross-sectional coordinate system of the 3D laser scanner and the carrier coordinate system defined by the positioning and orientation system is determined by three offsets. Three rotation quantities Construct a 4x4 rotation matrix

[0114]

[0115] in,

[0116]

[0117] d1 = 0

[0118] d2=0

[0119] d3 = 0

[0120] d4=1

[0121] S43. Camera Coordinate System Construction: Based on the structural design, determine the positional relationship between the camera's imaging center and the inertial navigation center in the carrier coordinate system, and construct a 4x4 rotation matrix. Specifically, taking the carrier coordinate system as a reference, the relationship between the camera image coordinate system and the carrier coordinate system defined by the positioning and attitude determination system consists of three offsets. Three rotation quantities Construct a 4x4 rotation matrix

[0122]

[0123] in,

[0124]

[0125] h1 = 0

[0126] h2 = 0

[0127] h3 = 0

[0128] h4=1

[0129] S44. Image Rotation Angle Calculation: Based on the synchronization information of each rotating camera-triggered image frame, obtain the grating code disk value A at time t. The grating's one-circle scale is N. The image rotation angle j is then expressed as a rotation matrix. The coordinates are expressed as follows:

[0130]

[0131] S45. Image Center Coordinate Calculation: Calculate the absolute coordinates of the image center in the WGS84 system at the moment the camera rotates to trigger the frame, based on the camera coordinate system and the image rotation angle. Transform the image data to the global coordinate system; Image center coordinates P at moment t (moment of camera rotation trigger frame). cam84 The absolute coordinate expression in the WGS84 system is:

[0132]

[0133] S46. Coordinate Transformation of 3D Laser Scanning Point Cloud Data: The absolute coordinates of any original point P acquired by the 3D laser scanner at any given time are calculated using a rotation matrix transformation, converting the 3D scanning data to a global coordinate system; the coordinates of any original point P acquired by the 3D laser scanner at any given time t are... laser =(X tlas ,Y tlas Z tlas The matrix is ​​represented as Rlaser, and its absolute coordinate calculation expression is:

[0134]

[0135] The absolute coordinates of the 3D laser scanning point in the WGS84 system are obtained by using the rotation matrix of the inertial navigation carrier coordinate system in the global coordinate system, the rotation matrix of the 3D scanner coordinate system in the carrier coordinate system, and the rotation matrix of any original point coordinates of the 3D laser scanner.

[0136] The absolute coordinates P of the 3D laser scanning point in the WGS84 system laser84 The expression is:

[0137]

[0138] in, The rotation matrix is ​​used to transform the inertial navigation vehicle coordinate system to the WGS84 system; To convert the 3D scanner coordinate system to the carrier coordinate system using the rotation matrix R, laser Rotation matrix of arbitrary original point coordinates of a 3D laser scanner;

[0139] The matching and fusion of rotating camera images and 3D scanning point cloud data are realized by formulas (4) to (6), and a spatiotemporally aligned tunnel structure detection dataset is output for subsequent tunnel structure defect detection and identification.

[0140] In step S5, geometric deformation detection includes comparing the current point cloud with the design model and calculating deviations such as misalignment and encroachment. If the deviation is >5mm, an alarm is triggered. When identifying apparent cracks, a deep learning model (such as U-Net) is used to automatically detect cracks and peeling in the image.

[0141] This invention achieves millisecond-level synchronization using an FPGA and crystal oscillator, solving the timing drift problem of traditional solutions; it replaces multi-camera arrays with single-camera rotating scanning, achieving a lightweight design and reducing power consumption and cost; it automates the entire process from data acquisition to defect analysis, requiring no human intervention; and it aligns point clouds and images within millimeter-level errors, achieving high-precision fusion. The method of this invention can be widely applied to the rapid detection and maintenance management of subway, railway, and highway tunnels.

[0142] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A tunnel structure detection system integrating lidar and a rotating scanning camera, characterized in that, The system includes, from bottom to top, a bottom-level motion unit (1), a middle-level support and energy supply unit (2), an upper-level detection and synchronization unit (3), and a top-level rotating imaging unit (4); wherein, the bottom-level motion unit (1) includes a vehicle body (11), wheels (12) symmetrically arranged on both sides of the vehicle body (11), a self-propelled servo motor (13) on each wheel (12), a wheel encoder (14), and a push rod (15) installed at the rear of the vehicle body (11) for emergency braking; the middle-level support and energy supply unit (2) includes a column (2) vertically fixed to the center of the vehicle body. 1) A power module (22) integrated into the vehicle frame or on the column base; the upper detection and synchronization unit (3) includes a synchronization control module (31), an industrial control computer storage module (32), a laser scanning sensor (33), an inertial measurement sensor (34) and a GNSS satellite positioning sensor located at the bottom of the mounting frame (23); the top rotating imaging unit (4) is vertically stacked at the top of the main body of the equipment, including an area array camera (41), a camera rotation motor (42), a flash lamp and its control module (43) and a camera rotation mirror (44) located at the top of the mounting frame (23); The synchronization control module (31) coordinates the data collected by each sensor, controls the self-propelled servo motor (13) to drive the vehicle body (11) to move at a constant speed, and precisely triggers the camera rotation motor (42) to drive the double mirror of the camera rotation mirror (44) to rotate, so as to realize 360° high-definition imaging of a single camera; synchronously controls the laser radar scanning, and uses the high-precision crystal module inside the synchronization control module (31) to simulate GNSS second pulse timing, so as to realize millisecond-level hard synchronization of multi-source data; based on the attitude angle information output by the inertial measurement sensor (14), the attitude correction is performed on the laser radar point cloud data collected at each moment, and the position correction is performed on the rotating camera image data. The data of each sensor is unified to the WGS84 global coordinate system through the pre-calibrated spatial transformation matrix, so as to realize the millimeter-level matching and fusion of point cloud and image data, and realize the integrated high-precision detection of tunnel structure deformation and surface defects.

2. The tunnel structure detection system integrating lidar and rotating scanning camera according to claim 1, characterized in that: The wheel encoder (14) is fixed to the wheel in a concentric and coaxial manner. When the wheel rotates once during data acquisition, the odometer sends a certain pulse count value. The odometer frequency can ensure that at least one or more pulse signals are sent when the wheel travels one millimeter.

3. The tunnel structure detection system integrating lidar and rotating scanning camera according to claim 2, characterized in that: The laser scanning sensor (33) is used to receive the simulated PPS pulse from the synchronization control module (31) to achieve time synchronization and acquire three-dimensional laser scanning point cloud data of a large-scale scene; The inertial measurement sensor (34) is combined with the GNSS satellite positioning sensor to output the vehicle attitude information at each acquisition time. The GNSS satellite positioning sensor, combined with the inertial measurement sensor (34), provides absolute vehicle positioning information and provides PPS second pulse signal and NEMA positioning data to the synchronization control module (31). The synchronization control module (31) realizes millisecond-level time synchronization of multiple sensors.

4. The tunnel structure detection system integrating lidar and rotating scanning camera according to claim 3, characterized in that: The synchronization control module (31) includes an input control terminal, an FPGA synchronization board, and an output execution terminal; the FPGA synchronization board integrates a high-precision clock chip; The input control terminal receives the drive control signal of the camera rotary motor (42), the rotation feedback of the grating code disk, the drive control command of the self-propelled servo motor (13), the gyroscope / accelerometer data of the inertial navigation system, the UTC time reference of the clock chip, and the distance pulse data of the wheel encoder (14) and transmits them to the processing terminal. The FPGA synchronization board of the processing terminal adopts a combination of crystal oscillator timing and PPS pulse to perform millisecond-level time synchronization of the laser scanning sensor (33), the area array camera (41), and the inertial measurement sensor (34) and input / output the execution terminal. The output execution terminal generates time / distance trigger signals to control the strict synchronization of the angle rotation of the camera rotary motor (42), the exposure of the area array camera (41), and the illumination of the flash lamp (43). At the same time, the speed of the self-propelled servo motor (13) is adjusted through the signal closed loop of the wheel encoder (14). All sensor raw data and synchronization time scales are stored in the industrial control computer, and the working frequency is configured through the parameter setting module to realize the spatiotemporal unity of motion control, three-dimensional scanning and rotation imaging.

5. A tunnel structure detection system integrating lidar and a rotating scanning camera according to any one of claims 1-4, characterized in that: The mounting frame (23) includes a mounting platform (231) on the top of the column (21), and three fixed uprights (232) arranged parallel to each other along the track direction on the mounting platform (231); and partitions (233) of different heights are horizontally arranged between two adjacent fixed uprights (232).

6. The tunnel structure detection system integrating lidar and rotating scanning camera according to claim 5, characterized in that: The upper detection and synchronization unit (3) is located on the top of the mounting platform (231); the synchronization control module (31), the industrial control computer storage module (32), and the inertial measurement sensor (34) are located on the mounting platform (231); the synchronization control module (31) and the industrial control computer storage module (32) are located between two fixed uprights (232) in the direction of the front of the vehicle body, and the inertial measurement sensor (34) is located between two fixed uprights (232) in the direction of the rear of the vehicle body.

7. A tunnel structure detection system integrating lidar and rotating scanning camera according to claim 6, characterized in that: The area array camera (41) and camera rotation motor (42) are mounted on a partition (233) near the front end of the vehicle body. One end of the camera rotation motor (42) is connected to the area array camera (41), and the other end is fixed to the middle fixed plate (232). The output end of the camera rotation motor (42) passes through the middle fixed plate (232) and is fixed to the camera rotation mirror (44) on the other side. The flash (43) is mounted on the fixed plate (232) at the rear end of the vehicle body, away from the camera rotation mirror (44).

8. A tunnel structure detection system integrating lidar and a rotating scanning camera according to any one of claims 1-4, 6, and 7, characterized in that: The camera rotating motor (42) is a DC permanent magnet brushless motor, and its motor rotating shaft is coaxially and fixedly connected to the camera rotating mirror (44). The camera rotating motor (42) is equipped with a grating code disk, which controls the rotation of the motor in a programmed manner and feeds back the motor rotation to a specified angle to trigger the area array camera (41) and flash (43) to take pictures; The camera rotating mirror (44) includes two mirror surfaces at a 90° angle. The one closer to the area scan camera (41) is the first mirror surface, and the one closer to the flash (43) is the second mirror surface. Both mirror surfaces are at a 45° angle to the horizontal plane. The lens of the area scan camera (41) is aligned with the first mirror surface, and the light outlet of the flash (43) is aligned with the second mirror surface of the rotating mirror.

9. A method for detecting tunnel structures by integrating lidar and a rotating scanning camera, characterized in that, The tunnel structure detection system based on the fusion lidar and rotating scanning camera as described in any one of claims 1-8 includes: S1. System installation and initialization: Install the components of the bottom motion unit (1), middle support and energy supply unit (2), upper detection and synchronization unit (3), and top rotating imaging unit (4) of the tunnel structure detection system integrating lidar and rotating scanning camera on the track to be detected according to the design requirements; connect the FPGA synchronization board, industrial control computer, and power module (22) to ensure that all sensors of the system are powered normally; set the initial parameter values ​​of the camera rotating motor speed, camera trigger interval time, and vehicle forward speed. S2. Data acquisition and synchronization control: The industrial control computer sends instructions through the FPGA synchronization board of the synchronization control module (31) to drive the camera rotation motor (42) to rotate. The grating code disk provides real-time feedback on the current mirror rotation angle. The FPGA synchronization board triggers the area array camera (41) to take pictures and the flash (43) to provide supplementary light based on the feedback value of the grating code disk. The FPGA synchronization board sends a synchronization pulse PPS to trigger the laser radar scanner to scan and embeds the point cloud data with a timestamp, outputting the original sensor data with the timestamp. S3. Rotational scanning imaging: The FPGA synchronization board controls the self-propelled servo motor (13) to drive the vehicle body (11) forward at a constant speed according to the pulse of the wheel encoder (14), so that the adjacent circle images of the rotating mirror (44) of the camera are arranged in a spiral along the longitudinal axis of the tunnel. The intensity of the flash lamp is dynamically adjusted according to the tunnel environment, and a high-definition image sequence covering the entire inner wall of the tunnel is output. S4. Multi-sensor data fusion: The carrier pose is calculated by fusing inertial navigation and odometry, and the carrier coordinate system is output. The lidar coordinate system is converted to the carrier coordinate system by using known offsets and rotation matrices. The camera coordinate system is converted to the carrier coordinate system by using pre-calibrated sensor parameters. Point cloud and image coordinate transformations are used to unify point cloud and image data to the WGS84 global coordinate system, achieving high-precision matching and fusion of point cloud and image data, and outputting a spatiotemporally aligned tunnel structure detection dataset. S5. Identify geometric deformation and apparent cracks based on the spatiotemporally aligned tunnel structure detection data, generate a defect report based on the geometric deformation and apparent crack data, and output a tunnel health status report including the location, type, and severity of the defect; complete the integrated high-precision detection of tunnel structure deformation and apparent defects.

10. The tunnel structure detection method integrating lidar and rotating scanning camera according to claim 9, characterized in that, Step S4 includes: S41. Inertial Navigation and Odometer Data Fusion: Inertial navigation and odometer data are processed and calculated. Utilizing existing dead reckoning inertial measurement principles, differential calculations are used to obtain the relative mileage, position, and attitude information at each moment of the collected data, consisting of three offsets. Three rotation quantities Construct a 4x4 rotation matrix S42. 3D Scanner Coordinate System Construction: Based on the structural design, determine the positional relationship between the phase center of the 3D scanner sensor 33 and the inertial navigation center to construct the carrier coordinate system, which consists of three offsets. Three rotation quantities Construct a 4x4 rotation matrix S43. Camera Coordinate System Construction: Based on the structural design, determine the positional relationship between the camera's imaging center and the inertial navigation center to construct the carrier coordinate system, which consists of three offsets. Three rotation quantities Construct a 4x4 rotation matrix S44. Image Rotation Angle Calculation: Based on the synchronization information of each frame of the rotating camera-triggered image, obtain the grating code disk value A at time t, with one circle of the grating scale being N, and construct the rotation matrix for the image rotation angle j. S45. Image Center Coordinate Calculation: Calculate the absolute coordinates of the image center in the WGS84 system at the moment the camera rotates to trigger the frame, based on the camera coordinate system and the image rotation angle. Transform the image data to the global coordinate system; Image center coordinates P at moment t (moment of camera rotation trigger frame). cam84 The absolute coordinate expression in the WGS84 system is: S46. Coordinate Transformation of 3D Laser Scanning Point Cloud Data: The absolute coordinates of any original point P acquired by the 3D laser scanner at any given time are calculated using a rotation matrix transformation, converting the 3D scanning data to a global coordinate system; the coordinates of any original point P acquired by the 3D laser scanner at any given time t are... laser =(X tlas ,Y tlas Z tlas The matrix is ​​represented as Rlaser, and its absolute coordinate calculation expression is: The absolute coordinates of the 3D laser scanning point in the WGS84 system are expressed by the Plaser84 expression: The rotation camera image and the three-dimensional scanning point cloud data are matched and fused by the calculation of formulas (4) to (6), and a spatiotemporally aligned tunnel structure detection dataset is output.

Images