Method and device for detecting girder web diseases of box girder

Abstract

The invention relates to the technical field of bridge engineering, and particularly discloses a box girder web disease detection method and device, and the method comprises the steps: S1, driving an unmanned plane to scan along a preset route; s2, the unmanned aerial vehicle collects a multispectral image, thermal imaging data and point cloud in the box girder; s3, judging whether supplementary shooting is needed or not; s4, performing geometric registration on the multispectral image and the point cloud, and mapping the point cloud to a world coordinate system where the magnetic chassis is located; step S5, constructing a four-dimensional voxel library; s6, determining a candidate disease area based on a gray field of the thermal imaging data; s7, inputting the four-dimensional voxel library and the candidate disease region into the neural network model, and outputting a disease tetrad; and step S8, mapping the coordinates of the disease tetrad in a visual platform. The method can provide a high-quality decision basis for bridge operation and maintenance, and is suitable for concrete and steel box girder bridges.

Description

Technical Field

[0001] This invention relates to the field of bridge engineering technology, specifically to a method and device for detecting defects in the web of a box girder. Background Technology

[0002] Box girders are widely used in long-span cable-stayed bridges, suspension bridges, and continuous beam bridges for railways and highways due to their light weight, high rigidity, and good overall integrity. However, the web of the girder is subjected to complex external forces such as reciprocating loads from trains or vehicles, alternating temperature and humidity, electrochemical erosion from wind, rain, and occasional earthquakes, often resulting in corrosion, coating deterioration, weld fatigue cracks, and localized warping. Once these defects develop, they not only weaken the load-bearing capacity of the main structure but also pose operational safety hazards, necessitating frequent, comprehensive, and traceable condition monitoring throughout the service life.

[0003] Current inspection methods primarily rely on manual entry into the enclosure using handheld lighting and equipment such as magnetic particle and ultrasonic sensors for visual or contact flaw detection. Tracked or wheeled robots equipped with monocular visible light cameras along beam bottoms or guide rails perform localized imaging, while drones autonomously fly inside the enclosure for aerial photography. Additionally, single ultrasonic, eddy current, and magnetic flux leakage non-destructive testing sensors are also used for scanning localized hidden defects. These methods are constrained by factors such as the narrow and elongated enclosure, obstruction by partitions, lack of GPS signals, poor lighting conditions, and high metal reflectivity. They generally suffer from slow inspection speed, insufficient coverage, lack of depth information, low positioning accuracy, inconsistent data formats, and severe interference from lighting conditions in defect identification. These methods fail to meet the operational needs for high-precision quantification of defect size, type, and spatial coordinates, as well as periodic comparative analysis of their development trends.

[0004] Therefore, there is an urgent need for a new type of intelligent inspection equipment and method that can achieve rapid inspection, precise three-dimensional positioning, multi-spectral high-resolution imaging, and automatic identification of disease type and size in the narrow web space of a box girder, combined with deep learning algorithms. This would overcome the shortcomings of existing technologies in terms of efficiency, accuracy, data traceability, and adaptability, and provide reliable support for the whole life cycle health inspection and preventive maintenance of bridges. Summary of the Invention

[0005] To address the aforementioned problems, the purpose of this invention is to provide a method and apparatus for detecting defects in the web of box girders. This method enables rapid, comprehensive, and sub-millimeter-level detection and quantification of defects (cracks, corrosion, coating peeling, weld cracking, etc.) within the enclosed and narrow web space of a box girder. By utilizing a rail-fly collaborative mobile platform, multi-spectral three-dimensional vision fusion, and deep learning cross-modal recognition, it overcomes the limitations of existing manual inspection, single-platform single-spectral imaging, and single-sensor NDT methods in terms of efficiency, positioning accuracy, identification robustness, and data traceability. It is applicable to both concrete and steel box girders and can provide scientific decision-making basis and key data support for bridge operation and maintenance management.

[0006] This invention provides a method for detecting defects in the web of a box girder, applied to a detection device. The detection device includes: a guide rail, a transport mechanism, and a motorized carrier. The motorized carrier includes: a magnetic chassis and a drone fixed to the magnetic chassis. The two ends of the guide rail are respectively fixed to the inlet and outlet of the box girder, connecting the entire box girder in series. The transport mechanism slides on the guide rail, and the motorized carrier is adsorbed onto the bottom end of the transport mechanism by the magnetic force of the magnetic chassis. The detection method includes: Step S1: Use the positioning module mounted on the magnetic chassis to obtain the three-dimensional initial attitude of the UAV, and drive the UAV to scan along a preset route based on the three-dimensional initial attitude; Step S2: Use the multimodal multispectral imaging unit carried by the UAV to collect multispectral images, thermal imaging data and point clouds inside the box girder; Step S3: Calculate the confidence level based on the multispectral image, and determine whether reshooting is needed by comparing the confidence level with a preset confidence threshold; Step S4: Use the camera's extrinsic calibration matrix to perform geometric registration of the multispectral image and the point cloud, and map the point cloud to the world coordinate system where the magnetic chassis is located to obtain the target point cloud in the world coordinate system; Step S5: Voxelize the target point cloud at a preset resolution to construct a four-dimensional voxel library; Step S6: Calculate the temperature rise anomaly index based on the grayscale field of the thermal imaging data, and determine the candidate disease area by comparing the temperature rise anomaly index with the preset anomaly threshold. Step S7: Input the four-dimensional voxel library and the candidate disease area into the neural network model and output the disease quadruple; Step S8: Map the coordinates of the disease quadruple onto the visualization platform.

[0007] In one possible implementation, the disease quadruple is uploaded to the cloud; the cloud is used to optimize the neural network model based on the disease quadruple, and the optimized neural network model is sent back to the detection device through a parameter distribution mechanism.

[0008] In one possible implementation, when reshooting is required, the magnetic chassis pauses and ejects the drone. After the multimodal multispectral imaging unit on the drone acquires multispectral images, thermal imaging data, and point clouds of the area, the drone is reattached to the bottom of the transport mechanism via the magnetic chassis. The magnetic chassis then drives the drone to continue scanning along a preset route. In one possible implementation, step S3 includes: Calculate the confidence level using the following formula. : in, For the first The weights corresponding to each feature For the feature number, For the first Each feature in coordinates The value of For the Sigmoid function, For natural constants, Let's establish coordinates.

[0009] In one possible implementation, step S4 includes: The flight trajectory of the drone used for reshooting is determined using the following formula. : in, To be at the trajectory position Confidence level at the location For the velocity of the trajectory, For the maximum allowable acceleration threshold, For the weighting coefficient, For time.

[0010] In one possible implementation, step S4 includes: Geometric registration of the multispectral image and the point cloud is performed using the camera's extrinsic calibration matrix. Based on acquiring a preset number of images using the Zhang calibration plate; Based on the preset number of images, the camera's extrinsic parameters are solved using least squares and optimized using the Random Sample Consensus Algorithm (RANSAC) to obtain the optimized extrinsic parameters. Based on the optimized extrinsic parameters, the point cloud is mapped to the world coordinate system of the magnetic chassis to obtain the target point cloud in the world coordinate system.

[0011] In one possible implementation, step S6 includes: Calculate the temperature rise anomaly index using the following formula. : in, For thermal imaging data, For coordinates, For reference area temperature average, This represents the standard deviation of temperature in the reference region. In one possible implementation, the disease quadruple includes: disease type, disease width, disease depth, and the location of the disease in the world coordinate system.

[0012] In one possible implementation, when the UAV's signal status is normal, the UAV uses UWB positioning and LoRa communication and / or Wi-Fi wireless communication technology to perform high-speed batch backhaul in the cabin window area. When the signal status of the drone is extremely weak or interrupted, the drone performs lossless compression caching of the multispectral image and the point cloud in its local solid-state storage (SSD). When the drone's signal status returns to normal, the drone will transmit the multispectral image and the point cloud back to the vehicle in segments.

[0013] The present invention also provides a detection device for defects in the web of a box girder, for implementing the detection method described in any of the above claims, comprising: a guide rail, a transport mechanism, and a motorized carrier; the motorized carrier comprises: a magnetic chassis and a drone fixed on the magnetic chassis; The two ends of the guide rail are fixed to the inlet and outlet of the box girder, respectively, and the entire box girder is connected in series; the transport mechanism is slidably mounted on the guide rail, and the motorized carrier is attracted to the bottom end of the transport mechanism by the attraction force of the magnetic chassis.

[0014] This invention provides a method and apparatus for detecting defects on the web of box girders. Addressing the challenges of limited internal space, complex structure, and low efficiency and high missed detection rate of traditional manual inspections in box girders, this invention designs a track-fly collaborative detection device that integrates a traveling mechanical platform and a drone. It combines multispectral vision-point cloud fusion technology with an edge-cloud incremental learning closed loop to achieve rapid detection and accurate identification of defects on the surface of the box girder web. The traveling mechanical platform, equipped with a multispectral imaging module and a laser point cloud sensor, performs a preliminary scan along a preset track. If blind spots or low-confidence areas are found in the scan results, the drone is automatically triggered to fill in the gaps. An edge computing unit fuses multi-source data in real time, identifying cracks, corrosion, and coating defects through a collaborative attention mechanism. This achieves sub-millimeter-level quantification of defect size and location, and utilizes edge-cloud collaborative incremental learning to continuously optimize the model. This method is applicable to the inspection of bridges such as steel box girders and concrete box girders. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the detection device provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the movement route of the detection device provided in an embodiment of the present invention; Figure 3 A schematic flowchart illustrating the detection method for defects in the web of a box girder provided in an embodiment of the present invention. Detailed Implementation

[0016] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. The following detailed description of the embodiments and the accompanying drawings are used to illustrate the principles of the present invention by way of example, but should not be used to limit the scope of the present invention. That is, the present invention is not limited to the described preferred embodiments, and the scope of the present invention is defined by the claims.

[0017] In the description of this invention, it should be noted that, unless otherwise stated, "a plurality of" means two or more; the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance; those skilled in the art can understand the specific meaning of the above terms in this invention as appropriate.

[0018] Figure 1 This is a schematic diagram of the detection device provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the detection device provided by the present invention includes: a guide rail, a transport mechanism, and a motorized carrier; the motorized carrier includes: a magnetic chassis, a power supply and a drone fixed on the magnetic chassis; the two ends of the guide rail are respectively fixed to the inlet and outlet of the box girder, connecting the entire box girder in series; the transport mechanism slides on the guide rail, and the motorized carrier is adsorbed to the bottom end of the transport mechanism by the adsorption force of the magnetic chassis. Figure 2 This is a schematic diagram of the movement route of the detection device provided in an embodiment of the present invention.

[0019] Specifically, this invention installs guide rails or steel cables above the inlet and outlet positions, connecting the entire container. The guide rails minimize space occupation, and the drone moves longitudinally within the box girder's web via the guide rails. The drone's position is recorded based on the transmission distance, aiding in its positioning. Furthermore, at the center of each container, the drone is activated to collect visible light and infrared images of the entire container's interior, completing the inspection work. Based on a dual-flight cooperative flight optimal path planning method, the drone first flies forward to check the overall status, and then flies laterally with the camera at a 45° angle upwards and downwards to check key areas, achieving efficient, comprehensive coverage detection and focused inspection.

[0020] In one possible implementation, the magnetic attraction force of the magnetic chassis is expressed by the following formula. : in, The magnetic flux density of the guide rail is (0.18–0.25 T). For the effective area of ​​the iron core, is the vacuum permeability.

[0021] The magnetic chassis ensures stability under tilt angles of ±60° and acceleration disturbances of 0.5g. Wheel drive is powered by a brushless DC motor with a 60mm wheel diameter and a reduction ratio of 30:1, providing... Achieving 0.2–0.5 ms -1 Uniform cruising speed. A magnetically mounted UWB / millimeter-wave triple base station is used for positioning via a two-step method: first, a rough position is obtained via UWB. Then, the high-precision mileage is corrected by aligning the correlation peaks of the millimeter-wave FMCW radar: in, For cross-correlation operators. The launch-recovery landing seat at the end of the magnetic chassis generates its initial velocity through a compression spring. The system, in conjunction with IMU-visual fusion control, elevates the drone to a preset altitude of 0.8–1.2 m; the recovery phase employs a funnel-magnetic ring mechanism, with a recovery window... Tolerance for yaw angle The landing pad also incorporates a Litz-wire induction coil, providing 30 W / 37 V magnetic coupling wireless charging; charging current... It can restore 80% of the battery within 8 minutes.

[0022] This invention addresses the challenges of limited internal space, complex structure, and low efficiency and high missed detection rate of traditional manual inspections for box girders. It designs a track-fly collaborative inspection device that integrates a traveling mechanical platform and a drone, combining multispectral vision-point cloud fusion technology with an edge-cloud incremental learning closed loop to achieve rapid detection and accurate identification of surface defects on the box girder web. The traveling mechanical platform, equipped with a multispectral imaging module and a laser point cloud sensor, performs a preliminary scan along a preset track. If blind spots or low-confidence areas are found in the scan results, the drone is automatically triggered to fill in the gaps. An edge computing unit fuses multi-source data in real time, identifying cracks, corrosion, and coating defects through a collaborative attention mechanism. This achieves sub-millimeter-level quantification of defect size and location, and continuous model optimization is achieved using edge-cloud collaborative incremental learning. This device is suitable for the inspection of bridges such as steel box girders and concrete box girders. Real bridge tests show that the detection efficiency of this invention is more than 7 times higher than that of traditional methods, the minimum detectable crack width is 0.05 mm, the three-dimensional spatial positioning error is less than 5 mm, and the missed detection rate is less than 2%. It significantly improves the efficiency, accuracy and safety of box girder web defect detection and has broad engineering application prospects.

[0023] Figure 3 A flowchart illustrating the detection method for web defects in box girders provided in an embodiment of the present invention is shown below. Figure 3 As shown, the method for detecting defects in the web of a box girder provided by this invention is applied to a detection device, including: Step S1: Use the positioning module mounted on the magnetic chassis to obtain the three-dimensional initial attitude of the UAV, and drive the UAV to scan along the preset route based on the three-dimensional initial attitude. In one possible implementation, before the detection operation begins, the positioning module of the magnetic chassis achieves an initial three-dimensional attitude through RTK-GNSS and IMU fusion. Accurate acquisition, attitude error controlled within This step provides a unified world coordinate reference for subsequent multi-sensor spatiotemporal alignment.

[0024] Step S2: Use the multimodal multispectral imaging unit carried by the UAV to collect multispectral images, thermal imaging data and point clouds inside the box girder; In one possible implementation, the UAV is equipped with a multimodal multispectral imaging unit to acquire multispectral images. Thermal imaging data and high-precision LiDAR point clouds The data is transmitted in real time to the computing unit on the vehicle. The drone executes a preset route on both sides of the track, performing cross-view scanning.

[0025] Visible light (VIS, 550 nm) is used to acquire high-resolution texture and true color information; near-infrared (NIR, 950 nm) is used to enhance defect contrast in low light and under some coating conditions; short-wave infrared (SWIR, 1550 nm) is used to penetrate thin layers of rust, paint and contaminants to detect potential defects; thermal infrared (LWIR, 8–14 μm) band can be optionally added for thermal anomaly detection.

[0026] Therefore, the three-spectrum fusion camera of the modal multispectral imaging unit adopts a coaxial beam-splitting prism layout with three center wavelengths. Synchronous exposure error <100 µs. Images from each stream undergo spectrally guided super-resolution: in, A 4× subpixel convolutional upsampling network is used. Depth data is reconstructed from structured ray lasers using phase unrolling and triangulation. in, As baseline, For focal length, For pixel parallax, Let be the launch angle. The final point cloud, expressed in the camera coordinate system, is... . Step S3: Calculate the confidence level based on the multispectral image, and determine whether reshooting is needed by comparing the confidence level with a preset confidence threshold; In one possible implementation, the confidence level is calculated according to the following formula. : in, For the first The weights corresponding to each feature For the feature number, For the first Each feature in coordinates The value of For the Sigmoid function, The natural constant is 2.71828. Let's establish coordinates.

[0027] When reshooting is needed, the magnetic chassis pauses and ejects the drone. After the multimodal multispectral imaging unit on the drone collects multispectral images, thermal imaging data and point clouds of the area, the drone is reattached to the bottom of the transport mechanism via the magnetic chassis. The magnetic chassis then drives the drone to continue scanning along the preset route. In one possible implementation, Super-BiFPN generates multi-scale features. The confidence heatmap is obtained by linear combination with Sigmoid. Chassis tracking curve If it exists This will trigger the drone to fill the blind spot.

[0028] Flight trajectory The solution is obtained using constrained optimal control. The flight trajectory for UAV re-enhancing images is determined according to the following formula. : in, To be at the trajectory position Confidence level at the location For the velocity of the trajectory, For the maximum allowable acceleration threshold, For the weighting coefficient, For time.

[0029] Step S4: Perform geometric registration on the multispectral image and point cloud, and map the point cloud to the world coordinate system of the magnetic chassis to obtain the target point cloud in the world coordinate system; In one possible implementation, the computing unit utilizes the camera's extrinsic calibration matrix. Geometric registration is performed on multispectral images and point clouds; the camera in the multimodal multispectral imaging unit acquires a preset number of images using a Zhang calibration plate; based on the preset number of images, the camera's extrinsic parameters are solved using least squares and optimized using the Random Sample Consensus Algorithm (RANSAC) to obtain optimized extrinsic parameters that minimize reprojection error. Based on the optimized extrinsic parameters, the point cloud is mapped to the world coordinate system of the magnetic chassis, resulting in the target point cloud in the world coordinate system.

[0030] Specifically, the improved Zhang's calibration plate was used for shooting. The extrinsic parameters of the group of images are obtained through least squares solution. ; In the calibration residual After performing RANSAC iteration, it satisfies Using chained homogeneous coordinates Step S5: Voxelize the target point cloud at a preset resolution to construct a four-dimensional voxel library; The four-dimensional voxel library includes: spectral bands and the three-dimensional position of the UAV.

[0031] The point cloud is mapped to the chassis world system, and then voxelized at 0.5 mm to generate a four-dimensional voxel library. .in, For three-dimensional position, For spectral bands. To support multimodal fusion, the voxel attribute vector is defined as: Step S6: Calculate the temperature rise anomaly index based on the gray field of thermal imaging data, and determine the candidate disease area by comparing the temperature rise anomaly index with the preset anomaly threshold. In one possible implementation, the temperature rise anomaly index is calculated according to the following formula. : in, For thermal imaging data, For coordinates, For reference area temperature average, This represents the standard deviation of temperature in the reference region. when At that time, the area is marked as a candidate disease area, and a ROI mask is generated. This reduces the computational load of subsequent registration and inference.

[0032] Step S7: Input the four-dimensional voxel library and candidate disease areas into the neural network model and output disease quadruplets; In one possible implementation, the constructed four-dimensional voxel library and candidate disease regions are input into the Vision-GNN neural network model to extract spatial neighborhood relationships and structural semantic features, and output disease quadruples. .

[0033] Disease Quadruple The definition is as follows: Disease category; : Horizontal width of the disease; Vertical depth of the disease; : World coordinate system pose.

[0034] Image branches use Mask2Former to output pixel masks. And return to the width of the disease The point cloud branch constructs a K-NN dynamic graph using EdgeConv (edge ​​convolution) to extract local geometric features.

[0035] Alignment is achieved using a collaborative attention mechanism, as shown in the following formula: in, For image mask Query, For point cloud Key, For point cloud Value, For channel dimensions.

[0036] The comprehensive loss is expressed by the following formula. : in, For cross-entropy loss, For Dice's loss, To smooth out L1 loss. The model uses INT8 quantization and is deployed on an 8-TOPS edge SoC (such as NVIDIA Jetson Orin NX), with a single-frame inference latency of 42 ms, which meets the requirements for real-time detection.

[0037] Step S8: Map the coordinates of the disease quadruple onto the visualization platform.

[0038] In one possible implementation, the identification results are mapped to a BIM / GIS platform to achieve accurate visualization and attribute association of defects in the 3D model. The resulting data is uploaded to the cloud in GeoJSON format for multi-terminal sharing.

[0039] In one possible implementation, the computing unit uploads the disease quadruple set to the cloud; the cloud is used to optimize the neural network model based on the disease quadruple set, and then sends the optimized neural network model back to the intelligent detection terminal through a parameter distribution mechanism.

[0040] When end-side inference confidence Below the preset threshold At that time, the terminal will cut the original four-dimensional voxel slices Uploaded to the cloud; the cloud actively mines hard samples based on the strategy network, and generates a lightweight Student model through cyclic distillation. Its optimization objective is as follows: The edge supports OTA non-interrupted hot updates with an update latency of less than 5 minutes, enabling seamless switching of inference tasks.

[0041] The cloud performs model enhancement and updates for labeled samples, and the optimized model is sent back to the terminal through a parameter synchronization mechanism, thereby continuously improving detection performance and forming a closed-loop model of "collection-inference-feedback-reuse".

[0042] In one possible implementation, when the drone's signal is normal, UWB high-precision positioning and LoRa long-distance communication / local wireless LAN high-speed backhaul are used for model deployment and data synchronization.

[0043] When the drone's signal weakens or is interrupted, the drone caches the data on the onboard solid-state drive (SSD) and performs segmentation and caching strategies on the segments. When the drone's signal returns to normal, the drone automatically resumes the interrupted transmission and uploads the data segment by segment to the cloud to complete the backhaul.

[0044] The following implementation method, combined with a real-world bridge inspection scenario of the steel box girder web of an existing four-track railway super-large bridge (main span 2.3km) in Suzhou, fully illustrates the present invention. All components are selected from publicly available models, and the parameters and calculation processes are based on on-site measurements or manufacturer technical manuals, and can be directly reproduced by those skilled in the art. Unless explicitly limited, the listed values ​​are preferred rather than unique.

[0045] I. Traveling Machinery Platform and Positioning Reference In this embodiment, austenitic stainless steel guide rails with a diameter of 16 mm are first laid on the axis of the top plate of the beam web. The dimensions of the traveling mechanism platform are 1.20 m long, 0.60 m wide, and 0.45 m high, with a self-weight of 58 kg. Eight sets of rubber-coated steel wheels are configured on each side. The differential drive motor has a gear ratio of 30:1, and after deceleration, it outputs a continuous traction force of 90 N, which can maintain the platform at a speed of 0.10 m / s. -1 cruise at a constant speed.

[0046] The platform integrates a permanent magnet-electromagnetic composite adsorption component, with magnetic induction intensity... T, effective surface area of ​​the board This generates adsorption force. in, is the vacuum permeability.

[0047] The platform maintains its rubber-coated wheels and rails even under ±60 roll rotations and 0.5g impact conditions. It undergoes geometric calibration at the factory using NIST traceability standard blocks, ensuring a straightness error of ≤1.5mm·m. -1 .

[0048] The platform uses RTK-GNSS+MEMS-IMU integrated navigation to provide outdoor reference coordinates; after entering the GNSS-shielded area of ​​the enclosure, it adaptively switches to the odometry + 16-line Velodyne VLP-16 laser SLAM mode, with a measured cumulative drift of 2.4mm·5m. -1 .

[0049] The platform travels to the starting point of the 3rd–4th section of the beam web (chainage K47+123.60m) and stops for 3 seconds. After the gimbal performs closed-loop self-calibration, a local coordinate system is established.

[0050] Experiments show that the platform can maintain track alignment under ±60° tilt angles and 0.5g dynamic disturbance conditions; after calibration with NIST standard blocks, its straightness error does not exceed 1.5 mm·m. -1 After switching to laser SLAM under abnormal GNSS conditions, the cumulative drift was 2.4 mm·5 m. -1 This meets the high-precision inspection requirements of the narrow space in the beam web.

[0051] II. Preliminary Multi-Spectral Scan and Confidence Heatmap Generation The platform's pan-tilt angle is set to 12°, and the Zenmuse H20T payload is enabled for simultaneous visible light acquisition (20MP, 4K, 10fps). PX thermal infrared data.

[0052] Velodyne VLP-16 lidar outputs approximately [missing information] points / s point cloud. The platform uses... A 5m high-speed scan of the work section was conducted, acquiring a total of 150 frames of multimodal images and... Point cloud; after calibration with 25 sets of chessboard grids, the mean square error of reprojection is 0.62mm.

[0053] The multi-spectral fusion features are extracted from five layers of semantic features by the Super-BiFPN codec. The fusion coefficient of adaptive learning in the experiment is: Calculate the confidence heatmap using the following formula: In this section The center coordinates of the low confidence region are It is located in the dead corner behind the beam.

[0054] Based on this, the system triggers a blind spot reshooting strategy for the drone to complete the data.

[0055] III. Reshooting blind spots using a folding quadcopter drone The platform's catapult-recovery landing seat can store 22 J of energy, and the initial launch velocity was measured to be 2.1 m / s. -1 The folding quadcopter UAV has a diagonal diameter of 0.48 m, a payload of 0.9 kg, and an IP55 rating. Its UWB-IMU integrated positioning accuracy is ±1.6 cm, and it achieves a steady-state hovering state within 1.2 seconds of takeoff. The UAV utilizes a B-spline trajectory. The measured horizontal displacement was 2.4 m, and the vertical displacement was... It reached the work point in 7 seconds, with a distance of 0.8 m, and hovered at a working distance of 0.9 m. The gimbal had a 12× optical zoom field of view of 0.40 m × 0.23 m. It stitched together 8 20 MP RGB images and 8 thermal images, with a reshoot time of 38 seconds. The return-to-home and magnetic ring recovery took 9 seconds, for a total of 54 seconds for a single mission, with 84% battery remaining.

[0056] IV. Multi-source fusion registration and disease identification The UAV imagery was first processed using SIFT + EPnP to solve for camera extrinsic parameters, and then aligned across modal spaces with the original voxel library; the mean square error of single-frame matching was maintained at 0.62 mm. Subsequently, a resolution of [missing information - likely a specific value] was constructed. Three-dimensional voxel set .

[0057] Image branches use Mask2Former to output pixel masks. The point cloud branch uses a GNN based on DynamicEdgeConv to extract geometric features. The two branches achieve feature alignment through a collaborative attention mechanism, the calculation formula of which is as follows: in, For channel dimensions.

[0058] After alignment, a disease quadruple is generated. .

[0059] in, Disease type; Crack width; Crack length / depth; The location of the disease in the world coordinate system.

[0060] The recognition results are as follows: The evaluation metrics are: Dice=0.87, IoU=0.79.

[0061] The actual crack width was then verified using an MT-A600 magnetic particle flaw detector, and was found to be 0.10–0.12 mm, with a width error of [missing information]. 3% ~ +9%, verifying recognition accuracy.

[0062] V. End-to-Cloud Incremental Learning Closed Loop This invention constructs a closed-loop incremental learning system of "inference-backtracking-retraining-distillation-hot update" between the edge (mobile machinery platform and drone) and the cloud (centralized training server), enabling the disease identification model to have online self-adaptation and lifelong evolution capabilities.

[0063] To address the severe signal attenuation and interference issues within the narrow space of the beam web, this invention employs a dual-channel redundancy design at the data transmission level: on one hand, low-frequency UWB (Ultra-Wideband) positioning and LoRa communication ensure stable transmission of low-bit-rate control commands and critical metadata; on the other hand, Wi-Fi 6 / 5G modules are used for high-speed batch backhaul in the cabin window area. When the signal is extremely weak or interrupted, the edge automatically activates a cache-delay synchronization mechanism, which performs lossless compression and cache of multispectral imagery and point cloud data in the UAV's local SSD and transmits it back in segments when the signal is restored, avoiding data loss due to link instability.

[0064] Inference and confidence assessment: After multispectral-point cloud fusion is completed at the edge, the Mask2Former-EdgeConv-GNN network is run in quantized INT8 mode; for each disease candidate instance... Output class probability vector Calculate the confidence level using the following formula: and confidence levels below the threshold The samples are labeled as difficult samples.

[0065] Edge-side sample acquisition and encrypted upload: Original RGB-TIR slices, local point cloud fragments, and location metadata of difficult samples The automatic segmentation mask is packaged into VOX-ZIP (approximately 1 MB / frame) at the edge. The file is encrypted with AES-256 symmetric encryption and uploaded to the cloud via a TLS-1.3 channel; the daily limit is 500 MB, far lower than the 300 Mbps bandwidth of a 5G NR link.

[0066] Cloud-based active learning and retraining: Incremental deep training triggered by a TPU-v3-8 cluster in the cloud. Sample selection: using a core set based on cosine similarity Maintain sample diversity; Fine-tuning strategy: Freeze the first three layers of the backbone, and only apply the learning rate to the higher-order semantic layer and the decoding head. Iterate for 10 epochs; Regularization: Add Focal-Loss To prevent minority class overfitting, the overall loss function Fine-tuning can be completed on 3,000 difficult samples within 20 minutes, with an improvement of 0.6–1.2 pct at mAP@0.5.

[0067] Model distillation and quantization compression To balance edge computing power and accuracy, temperature is used. KL distillation: After distillation, the student model maintains ≥97% accuracy while reducing the number of parameters by 18%, and the final weight is ≤30 MB, making it suitable for edge computing deployment.

[0068] The cloud generates and signs a differential update packet, which is then pushed via MQTT-TLS. The edge verifies the SHA-256 hash and completes the restart cascading. The entire process takes approximately 13 seconds.

[0069] The first 100 frames after the update are compared in parallel with the old model output using "shadow mode". After passing the consistency threshold check, the mode is switched to ensure zero missing frames and zero interruptions.

[0070] During continuous on-site inspections, this invention achieved unsupervised self-evolution for 17 newly added coating defect forms and 5 rare crack morphologies. The model performance improved from mAP@0.5=95.2% to 98.1%, and the false negative rate decreased from 4.7% to 1.8%, verifying the effectiveness and engineering feasibility of the edge-cloud collaborative incremental learning system.

[0071] VI. Results and Benefits of Full-Span Testing This invention employs a "5-m segment-based cyclic initial scan—blind spot detection and supplementary scan" strategy, completing 19 segments of scanning and 4 UAV supplementary scans, covering a steel box girder spanning 2.3 km. The total inspection time was 2000 min. The overall performance is as follows: minimum detectable crack width 0.2 mm, mAP 98%, mIoU 0.82, 3D coordinate projection error ≤ 5 mm, and a missed detection rate of 1.8%. Compared with manual suspended platform inspection (approximately 8 hours, two workers) and traditional single UAV aerial photography (approximately 2 hours, 18% blind spot), this invention improves operational efficiency by 7–32 times, demonstrating significant superiority in safety, coverage, and accuracy. The system hardware uses all mature market models (H20T, VLP-16, u-blox F9P RTK, etc.), and the algorithm is based on Mask2Former and EdgeConv-GNN. It can complete single-frame inference in 42 ms on an RTX A2000 industrial PC, and has the ability to be deployed immediately and scaled up for engineering promotion.

[0072] In the actual experiment on the 2.3 km main span of the aforementioned four-line railway super bridge, the system of the present invention was executed in sections according to the box chambers. The system used a traveling platform combined with a drone to achieve full-span box chamber inspection. The total operation time was 16.67 h. The comparison with the benchmark scheme is shown in Table 1 below.

[0073] Table 1 *Estimated based on 4 routine inspections per year for the same bridge type, including manpower, traffic closures, and equipment depreciation.

[0074] The system identified 13 cracks (0.05–0.21 mm wide), 6 instances of coating peeling and corrosion (0.32–1.24 m² area), and 4 early-stage fissures (0.20–0.38 mm deep) on-site. The crack detection rate was 100%, with a width error of [missing information]. 9%~+6%; crack depth error ±0.05 mm; spatial positioning error ≤ 5 mm, meeting the Level III accuracy requirements of the "Railway Bridge Defect Detection Code".

[0075] Comprehensive tests show that this invention, through collaborative scanning of a walking platform and a drone, multi-spectral + point cloud fusion perception, and an edge-cloud incremental learning closed loop, achieves rapid and comprehensive detection of defects in the web of box girders, enabling sub-millimeter-level crack localization and adaptive model evolution. It is applicable to both steel and concrete box girders. Compared to 8 hours of manual suspended platform inspection, the total inspection time of this invention is reduced to 16 hours and 45 minutes, increasing efficiency by 7 times. Unmanned entry into the enclosed box girder completely avoids the risks of high-altitude operations. Annual maintenance costs are reduced by approximately 44%, while the minimum detectable crack size is reduced to 0.05 mm, the 3D projection error is controlled within 5 mm, and the missed detection rate is 1.8%. This achievement meets the high-precision and digital requirements for periodic surveys and defect trend assessments of high-grade steel box girders and can be widely applied in intelligent maintenance scenarios for large bridges.

[0076] Therefore, this invention achieves the integrated goal of "efficient inspection - accurate identification - self-evolutionary learning - closed-loop data management" in a complex and closed environment, which has significant engineering and economic advantages compared with existing single-machine vision or manual inspection solutions.

[0077] The present invention provides a method and apparatus for detecting defects in the web of box girders. This method employs a rail-fly collaborative multispectral visual fusion detection system to perform unmanned aerial vehicle (UAV) inspections within a single compartment of a single-span (2.3 km) box girder web. The flight time for a single compartment is approximately 5 minutes, representing a more than 5-fold increase in detection efficiency compared to manual inspection. The system relies on VIS / NIR / SWIR point cloud precision registration and the Vision-GNN cross-modal recognition framework. The minimum detectable crack width reaches 0.2 mm, the overall mAP is improved to 98%, the mIoU is improved to 0.82, and the missed detection rate is reduced to below 2%. The 3D coordinate projection error is controlled within 5 mm, achieving sub-millimeter-level quantification and precise BIM / GIS mapping of defect type, size, and spatial location. The overall technical performance is significantly superior to existing manual inspection, single-spectral visual, or single NDT detection methods, meeting the high-precision and digital requirements for bridge maintenance cycle surveys and defect trend assessments.

[0078] In actual bridge testing, the detection efficiency of this invention is more than 7 times higher than that of traditional methods, the minimum detectable crack width is 0.05 mm, the three-dimensional spatial positioning error is less than 5 mm, and the missed detection rate is less than 2%. It significantly improves the efficiency, accuracy and safety of box girder web defect detection and has broad engineering application prospects.

[0079] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting defects in the web of a box girder, applied to a detection device, the detection device comprising: Guide rails, transport mechanisms, and motorized vehicles; The mobile carrier includes: a magnetic chassis and a drone fixed on the magnetic chassis; the two ends of the guide rail are respectively fixed to the inlet and outlet of the box girder, connecting the entire box girder in series; the transport mechanism is slidably mounted on the guide rail, and the mobile carrier is adsorbed to the bottom end of the transport mechanism by the adsorption force of the magnetic chassis; characterized in that the detection method includes: Step S1: Use the positioning module mounted on the magnetic chassis to obtain the three-dimensional initial attitude of the UAV, and drive the UAV to scan along a preset route based on the three-dimensional initial attitude; Step S2: Use the multimodal multispectral imaging unit carried by the UAV to collect multispectral images, thermal imaging data and point clouds inside the box girder; Step S3: Calculate the confidence level based on the multispectral image, and determine whether reshooting is needed by comparing the confidence level with a preset confidence threshold; Step S4: Perform geometric registration on the multispectral image and the point cloud, and map the point cloud to the world coordinate system where the magnetic chassis is located to obtain the target point cloud in the world coordinate system; Step S5: Voxelize the target point cloud at a preset resolution to construct a four-dimensional voxel library; Step S6: Calculate the temperature rise anomaly index based on the grayscale field of the thermal imaging data, and determine the candidate disease area by comparing the temperature rise anomaly index with the preset anomaly threshold. Step S7: Input the four-dimensional voxel library and the candidate disease area into the neural network model and output the disease quadruple; Step S8: Map the coordinates of the disease quadruple onto the visualization platform.

2. The detection method according to claim 1, characterized in that, Also includes: Upload the aforementioned disease quadruple to the cloud; The cloud is used to optimize the neural network model based on the disease quadruple set, and to send the optimized neural network model back to the detection device through a parameter distribution mechanism.

3. The detection method according to claim 1, characterized in that, Also includes: When reshooting is required, the magnetic chassis pauses and ejects the drone. After the multimodal multispectral imaging unit on the drone reacquires the multispectral images, thermal imaging data, and point clouds of the area, the drone is reattached to the bottom of the transport mechanism via the magnetic chassis. The magnetic chassis then drives the drone to continue scanning along the preset route.

4. The detection method according to claim 1, characterized in that, Step S3 includes: Calculate the confidence level using the following formula. : in, For the first The weights corresponding to each feature For the feature number, For the first Each feature in coordinates The value of For the Sigmoid function, For natural constants, Let's establish coordinates.

5. The detection method according to claim 3, characterized in that, Also includes: The flight trajectory of the drone used for reshooting is determined using the following formula. : ； in, To be at the trajectory position Confidence level at the location For the velocity of the trajectory, For the maximum allowable acceleration threshold, For the weighting coefficient, For time.

6. The detection method according to claim 1, characterized in that, Step S4 includes: Geometric registration of the multispectral image and the point cloud is performed using the camera's extrinsic calibration matrix. A preset number of images were acquired using the Zhang calibration plate; Based on the preset number of images, the camera's extrinsic parameters are solved using least squares and optimized using the Random Sample Consensus Algorithm (RANSAC) to obtain the optimized extrinsic parameters. Based on the optimized extrinsic parameters, the point cloud is mapped to the world coordinate system of the magnetic chassis to obtain the target point cloud in the world coordinate system.

7. The detection method according to claim 1, characterized in that, Step S6 includes: Calculate the temperature rise anomaly index using the following formula. : ； in, For thermal imaging data, For coordinates, For reference area temperature average, This represents the standard deviation of temperature in the reference region.

8. The detection method according to claim 1, characterized in that, The disease quadruple includes: disease type, disease width, disease depth, and the disease's position in the world coordinate system.

9. The detection method according to claim 1, characterized in that, Also includes: When the drone's signal is normal, the drone uses UWB positioning and LoRa communication or local wireless LAN to transmit data back for model deployment and data synchronization. When the drone's signal weakens or is interrupted, the drone will cache the data in segments on the onboard solid-state drive (SSD). When the drone's signal returns to normal, the drone will automatically resume the interrupted transmission and upload the data segment by segment to the cloud to complete the backhaul.

10. A device for detecting defects in the web of a box girder, used to implement the detection method as described in any one of claims 1-9, characterized in that, include: Guide rails, transport mechanisms, and motorized vehicles; The mobile carrier includes: a magnetic chassis and a drone fixed on the magnetic chassis; The two ends of the guide rail are fixed to the inlet and outlet of the box girder, respectively, and the entire box girder is connected in series; the transport mechanism is slidably mounted on the guide rail, and the motorized carrier is attracted to the bottom end of the transport mechanism by the attraction force of the magnetic chassis.

Images