Tower crane safety inspection method and system based on unmanned aerial vehicle and AI image recognition

Abstract

The invention discloses a tower crane safety inspection method and system based on an unmanned aerial vehicle and AI image recognition, hardware comprises an unmanned aerial vehicle and a control platform, and software comprises a path generation algorithm of an unmanned aerial vehicle inspection tower crane and an AI image recognition system in the control platform. During working, the unmanned aerial vehicle inspects all key parts of the tower crane according to an optimal path of an algorithm and takes photos, the photos are transmitted to the control platform in real time, the safety state of the tower crane is integrated and analyzed by the AI image recognition system, and a safety evaluation conclusion and rectification opinions are given. The system has the beneficial effects that manpower is saved, safety personnel and safety experts do not need to climb the tower crane in the safety inspection work of the tower crane after the system is used, the inspection efficiency is higher, the inspection is safer, meanwhile, compared with naked eye inspection and subjective judgment of people, the safety inspection process is more comprehensive, blind spot omission is avoided, and the safety inspection efficiency is improved. And the inspection result is more accurate.

Description

Technical Field

[0001] This invention relates to the field of intelligent construction and safety monitoring technology, and in particular to a tower crane safety inspection method and system based on "drone + AI image recognition". Background Technology

[0002] Traditional tower crane safety inspections heavily rely on manual climbing to high altitudes to inspect critical components such as the tower body and boom, which has significant drawbacks: First, working at heights is extremely risky, with a high risk of falls, and is limited by weather and lighting conditions, resulting in short inspection windows. Second, it is inefficient, requiring 2-3 hours for a single tower crane inspection, and has a high rate of missing hidden areas. Third, it is highly subjective, relying on the inspector's experience, making defect judgment susceptible to human influence, and the results are difficult to quantify and trace. Furthermore, traditional methods cannot achieve digital management, and hazard rectification lacks visual evidence support, failing to meet the intelligent requirements of modern construction safety supervision. Therefore, there is an urgent need for an intelligent inspection technology that eliminates the need for manual climbing, provides full coverage, and offers high precision to address the problems of poor safety, low efficiency, and insufficient reliability of traditional methods. Summary of the Invention

[0003] The main technical problem solved by this invention is to provide a tower crane safety inspection method and system based on "drone + AI image recognition", which solves one or more of the above-mentioned prior art problems.

[0004] To address the aforementioned technical problems, the present invention employs the following technical solution: a tower crane safety inspection method based on "drone + AI image recognition," the innovation of which includes the following steps: S1: Input the basic parameters and environmental parameters of the tower crane. Based on the parameters, the path planning algorithm generates a UAV flight path covering the key components of the tower crane foundation, tower body, attachment device, tower top slewing mechanism, lifting boom, and counterweight boom. The flight strategy is automatically switched according to the building height: the area below the building height adopts a 270° fan-shaped circling combined with a zigzag return and ascent, and the area above the building height adopts a 360° spiral ascent and circling. S2: The drone flies autonomously along the path, locks onto the main body of the tower crane for video recording throughout the process, and automatically hovers at preset key component waypoints to capture high-definition close-up images of bolts, pins, wire ropes, and welds from multiple angles; S3: Input the collected image data into a pre-trained deep learning model. The model automatically identifies tower crane components and diagnoses safety hazards, including loose or missing bolts, rusted or deformed structural components, broken wire ropes, and malfunctioning safety devices. S4: Generate a structured inspection report that includes the location, type, severity, and visual evidence of potential hazards, and push it to management personnel.

[0005] In some implementations, the basic parameters of the tower crane in step S1 include: standard section height a, number of standard sections c, foundation top surface height s, building height h, attachment device heights f1 and f2, boom length d, counterweight boom length e, and wire rope length l.

[0006] In some implementations, the design principles of path planning algorithms include: Safety principles: Drones should maintain a safe distance from tower crane structures, buildings and the surrounding environment, and fly only within a 270° fan-shaped area below the height of buildings; Full coverage principle: The inspection is carried out in layers and sections according to the following steps: “overall and environment → foundation → tower body → lifting boom → counterweight boom → tower top”. No key parts are missed and hidden parts are photographed from multiple angles. Efficiency principle: Shortest path, use a zigzag pattern to ascend below the building height.

[0007] In some implementations, the deep learning model in step S3 can simultaneously complete the tasks of target detection, segmentation and classification, and output the defect type, location, confidence level and defect region mask.

[0008] In some implementations, the inspection report generated in step S4 can be bound to the tower crane's identity information, geographical location, and timestamp, and automatically connected to a smart construction site management platform or building information modeling system.

[0009] A tower crane safety inspection system based on "drone + AI image recognition" includes a control platform and a drone; the control platform includes drone inspection path generation software and a tower crane safety hazard AI image recognition system; the drone is used to fly along the optimal path planned by the path generation software and collect image data, and the AI ​​image recognition system is used to analyze the image data and output safety hazard diagnosis results.

[0010] In some implementations, the drone inspection path generation software automatically generates a path containing the following modules based on the input tower crane basic parameters: Basic inspection: Take high-definition close-up photos of anchor bolts and foundation cracks at a 45° downward angle; Inspection of standard tower sections: Take close-up photos of the connecting bolts at a 90° angle; Attachment device inspection: Fly along the attachment rod direction and photograph the pin and wall support points; Inspection of the lifting boom and counterweight boom: including taking a 30° upward angle photo from the side of the trolley's traveling wheels, taking segmented photos of the wire rope, and taking a side photo of the counterweight's fixing bolts.

[0011] In some implementations, the tower crane safety hazard AI image recognition system employs a two-stage deep learning model: The first stage is a lightweight target detection model, which outputs the bounding boxes and preliminary classifications of key hidden danger components; The second stage is an encoder-decoder architecture based on Transformer, which performs defect classification and pixel-level segmentation on the cropped component sub-images.

[0012] In some implementations, the training strategy for the two-stage model includes: Loss functions: CIOU Loss + Focal Loss is used for object detection, weighted cross-entropy loss is used for classification, and Dice Loss + BCE Loss is used for segmentation. Optimizer: AdamW is used, which performs transfer learning fine-tuning based on pre-trained weights from ImageNet or COCO datasets.

[0013] In some implementations, the drone is equipped with a high-definition camera gimbal to continuously lock onto and record the main body of the tower crane during flight, and performs multi-angle hovering shooting at preset waypoints, with the image data being transmitted back to the control platform in real time.

[0014] The beneficial effects of this invention are: Safety: Replaces manual climbing, eliminating the risk of falls from heights; drones maintain a safe distance of ≥2m from tower cranes to avoid collisions; Full coverage: The path covers 100% of key components, and hidden parts (such as attachment rod connection points) are photographed from multiple angles without omission; High precision: The AI ​​model has a defect recognition accuracy of ≥95% (loose bolts) and ≥90% (broken wires in steel wire ropes), which is better than manual visual inspection; High efficiency: Inspection time for a single tower crane is ≤20 minutes (traditional manual inspection takes 2 hours), and automatic report generation takes ≤5 minutes. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a flowchart of a tower crane safety inspection method based on "drone + AI image recognition" according to the present invention.

[0016] Figure 2 This is a schematic diagram of a tower crane safety inspection system based on "drone + AI image recognition" according to the present invention. Detailed Implementation

[0017] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] To make the objectives, technical solutions, and effects of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0019] I. Method Implementation Examples (e.g.) Figure 1 ) S1: Inspection parameter input and automatic 3D path planning Operators input the basic parameters and environmental parameters of the tower crane to be inspected through the control platform, specifically including: Tower crane structural parameters: standard section height Standard section side length Standard section quantity Section, crane boom length Balance arm length Height of the top surface of the foundation from the ground Wire rope length ; Environmental parameters: height of surrounding buildings Attachment device height , .

[0020] The path planning algorithm generates a flight path covering the entire structure of the tower crane based on the above parameters. The specific strategy is as follows: 1. Areas below the building height ( ): 1) Employing a 270° fan-shaped circumference (avoiding the 90° area on the side of buildings) combined with a zigzag return ascent, the flight radius... Each time the height increases by one standard section ( Turn back once to ensure that no connecting bolts of the standard tower section are missed in the photos.

[0021] 2) Example of a critical waypoint: Top surface of the base (Away from the ground) ), the third standard section of the tower body ( (The angle within the sector area).

[0022] 2. Areas above the building height ( The crane employs a 360° spiral ascent and circumference, with a circumference radius R=2.4m. It completes one circumference every two standard sections (5.6m) of ascent, focusing on covering the tower top slewing mechanism and the boom tie rod.

[0023] 3. Specialized pathways for specific components: 1) Attachment device: generated along the direction of the attachment rod to Close-up shooting path (distance) ), photograph the pin shaft and the wall support point; 2) Crane boom trolley: Generate a 30° lateral elevation path. ( The furthest position of the car. (Total height of the tower crane), photograph the traveling wheels and rope breakage protection device; 3) Steel wire rope: generated in segments along its length. to The path was inspected, and the mid-section kinks and hook anti-detachment devices were checked.

[0024] S2: Autonomous Driving and Multimodal Data Acquisition by Unmanned Aerial Vehicles 1. The drone is equipped with a 4K high-definition camera gimbal (resolution 3840×2160, frame rate 30fps) and flies autonomously along the planned path: 1) Full-process video recording: During flight, the gimbal continuously locks onto the main body of the tower crane through a visual tracking algorithm and records panoramic video (bitrate 10Mbps). 2) Hovering at key waypoints: Automatically hover at preset waypoints (such as standard section bolts, attachment device pins) for 3-5 seconds, and capture close-up images at the following angles: a. Bolts / pins: 0° (direct view), ±45° (side view), shooting distance 0.5-1m, to ensure the thread texture is clearly visible; b. Steel wire rope: 0° (front view), 90° (side view), using a zoom lens (focal length 24-70mm) to capture details of broken wires; c. Welds: Take photos at a 45° downward angle to highlight cracks or rusted areas.

[0025] The image data is transmitted back to the control platform in real time via a 5G wireless link, and the storage formats are JPEG (image) and MP4 (video).

[0026] S3: AI-based image recognition and security vulnerability diagnosis The AI ​​image recognition system for tower crane safety hazards in the control platform employs a two-stage deep learning model: 1. First stage: Component target inspection 1) Model: Lightweight YOLOv8s, input image size 640×640, output bounding boxes of key components (such as bolts, wire ropes, pins) and primary classification (confidence threshold 0.7). 2) Training data: Includes 50,000 labeled images (20,000 bolts, 15,000 wire ropes, 10,000 pins, and 5,000 others), which have been preprocessed with Mosaic enhancement and random rotation (±15°); 3) Loss function: CIOU Loss (bounding box regression) + Focal Loss (class imbalance handling), optimizer AdamW (initial learning rate 0.001).

[0027] 2. Second Stage: Fine-grained Defect Identification 1) Model: Transformer-based encoder-decoder architecture (ViT-Base as encoder), with the input being the component sub-images (256×256) cropped in the first stage. 2) Task: a. Classification: Output status labels for bolts (normal / loose / missing / corroded) and wire ropes (normal / broken / kinked), using weighted cross-entropy loss (the weight of defective samples is 5 times that of normal samples). b. Segmentation: Output pixel-level masks of defective areas (such as broken wire locations), using Dice Loss + BCE Loss; 3) Training strategy: Fine-tune the weights based on ImageNet pre-training, iterate 100 times, batch size=32.

[0028] 3. Diagnostic Logic: 1) Bolt loosening determination: The relative positional offset between the bolt head and the nut is compared by template matching (threshold > 2mm). 2) Wire rope broken wire identification: The number of broken wires is calculated by dividing the mask (more than 3 broken wires in a single strand are considered a potential hazard). 3) Structural deformation detection: The component contour is extracted using an edge detection algorithm (Canny operator) and the deviation is compared with the standard template (threshold > 5°).

[0029] S4: Inspection Report Generation and Early Warning The system automatically integrates the recognition results and generates a structured report: 1. Report Content: 1) Basic information: Tower crane number (e.g., QTZ80-001), inspection time (2023-10-01 14:30), geographical location (39.9°N, 116.3°E); 2) Details of the potential hazard: Potential hazards type Severity Evidence Images Tower body, 5th standard section Loose bolts middle (Close-up view from 0°) Crane boom tie rod pin rust Low (45° side close-up) middle section of steel wire rope Broken wires (2 strands) high (Segmentation mask markings broken wire areas) 3) Push method: Push to the mobile terminal of the management personnel through the API of the smart construction site management platform and trigger an audible and visual alarm (severity level ≥ medium).

[0030] II. System Implementation Examples (e.g.) Figure 2 ) Control Platform 1. Hardware configuration: Industrial-grade server (Intel i7-12700K CPU, NVIDIA RTX 3090 GPU, 32GB RAM), supporting parallel processing of 10 drone data streams; 2. Software Modules: 1) Drone inspection path generation software: developed based on Python, integrating OpenCV (image processing) and NumPy (parameter calculation), providing a visual interface (Qt framework) for parameter input and path preview; 2) Tower crane safety hazard AI image recognition system: Deployed in Docker containers, using the TensorFlow framework to implement model inference, with an inference speed of ≥20fps (single image).

[0031] drones 1. Model: Quadcopter industrial drone (takeoff weight 5kg, flight time 35 minutes), equipped with a three-axis mechanical stabilization gimbal (±0.01° anti-shake) and RTK-GPS module (positioning accuracy 1cm). 2. Flight control: Supports Waypoint mode (waypoint positioning accuracy ±0.5m), built-in obstacle avoidance sensors (front / rear / downward lidar, detection range 0.5-30m).

[0032] The advantages of this technical solution are: Safety: Replaces manual climbing, eliminating the risk of falls from heights; drones maintain a safe distance of ≥2m from tower cranes to avoid collisions; Full coverage: The path covers 100% of key components, and hidden parts (such as attachment rod connection points) are photographed from multiple angles without omission; High precision: The AI ​​model has a defect recognition accuracy of ≥95% (loose bolts) and ≥90% (broken wires in steel wire ropes), which is better than manual visual inspection; High efficiency: Inspection time for a single tower crane is ≤20 minutes (traditional manual inspection takes 2 hours), and automatic report generation takes ≤5 minutes.

[0033] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0034] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A tower crane safety inspection method based on "drone + AI image recognition", characterized in that: The method comprises the following steps: S1: input the basic parameters and environmental parameters of the tower crane, and generate a flight path of a UAV covering the key components of the tower crane foundation, tower body, attachment device, tower top rotating mechanism, hoisting arm and balance arm based on the path planning algorithm, wherein the flight path automatically switches the flight strategy according to the building height: the 270° sector around combined with the zigzag shape of the area below the building height is adopted for ascending, and the 360° spiral ascending around is adopted for the area above the building height; S2: the UAV autonomously flies along the path, locks the main body of the tower crane for recording, and automatically hovers at the preset key component navigation point to take multi-angle high-definition close-up images of the bolts, pin shafts, steel wires and welds; S3: input the collected image data into a pre-trained deep learning model, and the model automatically identifies the tower crane components and diagnoses the safety hazards, including loose or missing bolts, rusted or deformed structural parts, broken steel wires and failed safety devices; S4: generate a structured inspection report containing the location, type, severity and visual evidence of the hidden dangers, and push it to the management personnel.

2. The tower crane safety inspection method based on "drone + AI image recognition" according to claim 1, characterized in that: The basic parameters of the tower crane in the step S1 include: standard section height a, standard section number c, foundation top surface height from ground s, building height h, attachment device height f1 and f2, hoisting arm length d, balance arm length e, and steel wire length l.

3. The tower crane safety inspection method based on "drone + AI image recognition" according to claim 1, characterized in that: The design principles of the path planning algorithm include: Safety principle: the UAV maintains a safe distance from the tower crane structure, the building and the surrounding environment, and only flies in the 270° sector area below the building height; Whole coverage principle: check in layers and sections according to “whole and environment -> foundation -> tower body -> hoisting arm -> balance arm -> tower top”, and no key parts are missed and multi-angle shooting is performed on hidden parts; Efficiency principle: shortest path, zigzag shape of the area below the building height is adopted for ascending.

4. The tower crane safety inspection method based on "drone + AI image recognition" according to claim 1, characterized in that: The deep learning model in the step S3 can simultaneously complete the target detection, segmentation and classification tasks, and output the defect type, location, confidence and defect area mask.

5. The tower crane safety inspection method based on "drone + AI image recognition" according to claim 1, characterized in that: The inspection report generated in the step S4 can bind the tower crane identity information, geographical location and timestamp, and automatically interface with the intelligent construction site management platform or the building information model system.

6. A tower crane safety inspection system based on "drone + AI image recognition", characterized in that: The UAV inspection path generation software automatically generates a path containing the following modules according to the input tower crane basic parameters:

7. The tower crane safety inspection system based on "drone + AI image recognition" according to claim 6, characterized in that: Foundation inspection: 45° downward angle high-definition close-up shooting of foundation bolts and foundation cracks; Tower body standard section inspection: close-range front shooting of connecting bolts, one close-up shot every 90° angle; Attachment device inspection: fly along the attachment rod direction and shoot the pin shafts and wall body support points; Hoisting arm and balance arm inspection: include 30° upward angle shooting from the side of the trolley running wheels, segmented shooting of steel wires and side shooting of counterweight block fixing bolts. The tower crane safety hazard AI image recognition system adopts a two-stage deep learning model:

8. The tower crane safety inspection system based on "drone + AI image recognition" according to claim 6, characterized in that: ​ The first stage is a lightweight target detection model, which outputs the bounding box and primary classification of the key hidden hazard components; The second stage is a Transformer-based encoder-decoder architecture, which classifies defects and performs pixel-level segmentation on the cropped component sub-image.

9. The tower crane safety inspection system based on "drone + AI image recognition" according to claim 8, characterized in that: The training strategy of the two-stage model includes: Loss function: CIOU Loss + Focal Loss for target detection task, weighted cross-entropy loss for classification task, and Dice Loss + BCE Loss for segmentation task; Optimizer: AdamW, transfer learning fine-tuning based on ImageNet or COCO dataset pre-training weights.

10. The tower crane safety inspection system based on "drone + AI image recognition" according to claim 6, characterized in that: The UAV is equipped with a high-definition camera gimbal, which continuously locks the main body of the tower crane during flight and performs multi-angle hovering shooting at the preset waypoints. The image data is transmitted back to the control platform in real time.

Images