A crane weld defect intelligent identification method and system based on UAV technology

Abstract

This invention discloses an intelligent identification method and system for weld defects in crane equipment based on UAV technology. The UAV autonomously cruises and collects images of crane welds, obtaining the three-dimensional coordinates of the image acquisition points. An onboard edge module performs CUDA preprocessing of the images, calls a modified YOLOv8n model with an embedded DAMFA module for heterogeneous inference, and restores the original pixel coordinates of the defect image. Multi-source data is encapsulated with timestamps and MD5 checksums, and real-time transmission or local caching and breakpoint resumption are achieved through network discrimination. The terminal parses the data packets, progressively converting the defect pixel coordinates into world three-dimensional coordinates, and uses a digital twin model for hierarchical labeling, early warning, and generation of maintenance work orders. This invention solves industry pain points such as unstable communication due to steel structure obstruction in cranes, false positives and false negatives of slender cracks, inability to locate defects in three dimensions, and high risks associated with manual high-altitude inspections. It achieves intelligent operation of the entire process of weld defect inspection, identification, transmission, location, and maintenance.

Description

Technical Field

[0001] This invention belongs to the field of safety inspection technology for lifting equipment, specifically relating to a method and system for intelligent identification of weld defects in lifting equipment based on UAV technology. Background Technology

[0002] Tower cranes, gantry cranes, and bridge cranes, among other large lifting equipment, are widely used in critical sectors such as construction, port loading and unloading, and industrial production, serving as key equipment to ensure the normal operation of industries. These machines operate under continuous high loads in harsh environments with high temperatures, high humidity, and strong winds, subjecting their metal structures to repeated alternating stresses. This makes the welds in critical load-bearing components highly susceptible to fatigue defects such as cracks, porosity, undercut, and lack of fusion. If these weld defects are not detected and addressed in a timely manner, they will continuously deteriorate the overall structural strength of the equipment, potentially leading to major safety accidents such as boom breakage and tower collapse, threatening the lives of on-site personnel and causing significant economic losses.

[0003] Currently, the industry's inspection of welds in lifting equipment mainly relies on manual visual inspection and traditional non-destructive testing methods. The supporting machine vision inspection solutions also have obvious technical deficiencies, and the overall technical system is difficult to meet the requirements of modern safety inspection. The specific problems are as follows:

[0004] Firstly, manual inspection is risky and inefficient. The core welds of lifting equipment are mostly located in areas that are difficult for personnel to access, such as high altitudes, narrow chambers, and the inside of trusses. If workers climb to the site for inspection, there is a great risk of falling from heights.

[0005] Secondly, existing visual algorithms suffer from poor feature extraction specificity and insufficient anti-interference capabilities. Current weld seam detection models generally reuse common components such as Spatial Pyramid Pooling with Holes (ASPP) and Convolutional Block Attention (CBAM), simply building the network through concatenation without customizing it to suit the elongated linear geometric features of the weld seam. For example, existing technology, patent publication number CN116229077A, entitled "A Mathematical Function Image Instance Segmentation Method Based on an Improved Mask-R-CNN Network," uses isotropic convolution for feature extraction. This type of convolution lacks signal selectivity and cannot distinguish between high-frequency noise such as point rust and oil stains on the crane surface and the linear low-frequency features of the weld seam, resulting in a mixture of defects and noise signals within the feature map. Even with subsequent general attention modules, only secondary filtering of the already mixed features is possible, failing to suppress interference at its source. Ultimately, the model is prone to misclassifying rust patches as weld defects, leading to significant false detection problems.

[0006] Third, general loss functions are unsuitable for detecting slender defects, resulting in poor sample imbalance. Most existing target detection models use general loss functions such as CIoU and Wise-IoU. These functions are designed for conventional targets and do not consider the morphological characteristics of slender defects with high aspect ratios, such as weld cracks. The aspect ratio constraint term in general IoU loss functions uses a linear weighting form. When the defect aspect ratio is greater than 5:1, the gradient corresponding to the slender defect will be masked by a large number of approximately square conventional targets, causing a significant decrease in bounding box regression accuracy, and the predicted box cannot fit the true contour of the defect. Furthermore, industrial field data suffers from severe sample imbalance, with a large number of simple background samples and a very low proportion of difficult samples such as microcracks and ultra-long welds. Traditional FocalLoss relies on manually fixing hyperparameters γ and α to adjust sample weights, which involves a large amount of parameter tuning work, has weak generalization ability, and makes it difficult to achieve dynamic gradient reconciliation.

[0007] Fourth, there is a lack of real-time end-to-cloud collaborative monitoring and breakpoint resume mechanisms. Existing detection methods are often based on post-event analysis, and data loss is prone to occur in signal-obstructed areas such as crane box-type main beams, making it impossible to achieve strict spatiotemporal synchronization of defect evidence and immediate hierarchical early warning. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for intelligent identification of weld defects in lifting equipment based on UAV technology, thereby solving the problems mentioned in the background art.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following solution:

[0010] A method for intelligent identification of weld defects in lifting equipment based on UAV technology includes the following steps:

[0011] Step S1, Physical Data Acquisition: The UAV autonomously flies to the target weld area of ​​the lifting equipment according to the preset inspection route, and collects physical image data of the weld through the airborne industrial camera. At the same time, it uses multi-source sensors to synchronously record the spatial three-dimensional coordinates of the acquisition points.

[0012] Step S2, Image Preprocessing and Edge Intelligent Inference: The edge computing module performs noise reduction and distortion correction preprocessing on the acquired physical image, calls the lightweight improved YOLOv8n target detection model for real-time inference, and outputs a structured data package containing defect category, confidence level, and defect bounding box pixel coordinates.

[0013] Step S3, Data Encoding and Resume Transmission: The structured data packet, the corresponding original image frame, and the spatial location coordinates are uniformly encoded and then sent to the remote monitoring terminal through the wireless communication link;

[0014] Step S4, Terminal 3D Mapping and Graded Early Warning: The remote monitoring terminal receives and parses the data, completes the quantitative grading of defects according to the preset evaluation rules; converts the defect pixel coordinates into 3D coordinates in the world coordinate system through coordinate transformation, performs mesh matching with the digital twin model of the lifting equipment, and marks the defect location and grade; triggers corresponding early warning and handling actions according to the defect grade.

[0015] Further optimization is achieved in step S1, where the multi-source sensors include an RTK high-precision positioning module, an inertial measurement unit, a barometer, and a visual odometry. When satellite signals are blocked by the structure of the lifting equipment and RTK positioning fails, an extended Kalman filter algorithm is used to fuse the data from the inertial measurement unit, barometer, and visual odometry to achieve joint positioning and output continuous pose information.

[0016] Further optimization, step S2 specifically includes:

[0017] Step S2.1, Image Preprocessing: Image decoding, multi-level denoising, distortion correction, and size normalization are performed sequentially. Image decoding converts the raw YUY2 format bitstream output from the industrial camera into an RGB three-channel image matrix. Multi-level denoising uses Gaussian filtering with a 5×5 convolution kernel and bilateral filtering with a neighborhood diameter of 7, a spatial domain of 20, and a pixel value domain of 25. Distortion correction includes lens radial distortion and tangential distortion correction based on pre-stored camera calibration parameters, as well as perspective distortion correction combined with the gimbal attitude angle. Size normalization scales the corrected image proportionally and pads the edges to a uniform resolution of 640×640, while recording the scaling ratio and padding parameters. All image preprocessing operators are implemented using CUDA parallel computing.

[0018] Step S2.2, Model Deployment and Inference Scheduling: Build and train the improved YOLOv8n object detection model, export the trained improved object detection model offline to ONNX format and compile it into a TensorRT engine file, perform INT8 model quantization and DAMFA module operator fusion, and then deploy it to the edge computing module; adopt a CPU+GPU heterogeneous scheduling method, with the CPU responsible for process scheduling and data reading and writing, and the GPU carrying out all feature extraction, convolution operation and feature fusion calculation tasks.

[0019] Step S2.3, Post-inference processing and coordinate mapping: After the model inference is completed, a confidence threshold of 0.5 is set to remove low-confidence prediction boxes, and duplicate detection boxes of the same target are eliminated by non-maximum suppression; combined with the scaling ratio and edge-padding parameters recorded in the size normalization stage, the 640×640 normalized bounding box coordinates output by the improved YOLOv8n target detection model are inversely mapped to the original 4K image pixel coordinate system to obtain the true pixel coordinates of the defects.

[0020] An anomaly tolerance mechanism has been added to directly skip inference for abnormal image frames such as black screens and overexposed images. At the same time, a watchdog timer has been set to automatically restart the inference engine when the model inference times out.

[0021] Further optimization involves the following steps in step S2.2: Improving the construction and offline training of the YOLOv8n object detection model:

[0022] Step S2.2.1: Dataset Construction and Preprocessing: Collect and expand crane weld images to construct a labeled dataset containing four types of defects: cracks, holes, spatter, and weld line type. Perform random flipping, grayscale perturbation, and tensile deformation data online to enhance and expand the difficult sample of slender defects. Divide the dataset into training set, validation set, and test set in a 7:2:1 ratio.

[0023] Step S2.2.2: Lightweight model network construction: Based on the original YOLOv8n, a three-layer network architecture consisting of a backbone network, a neck fusion network, and a multi-scale detection head is constructed;

[0024] Among them, the neck fusion network construction: a DAMFA module was designed and replaced the C2f module in the original neck network to improve the fusion performance of weld defect recognition. The interaction between deep and shallow features was completed through upsampling and downsampling, and a total of four DAMFA feature aggregation nodes were set up.

[0025] Multi-scale detection head construction: The differentiated features output by each DAMFA are connected to an independent Detect head. Each group of detection heads is set up in parallel with classification and regression dual branches to output the category, confidence and pixel bounding box of four types of defects: crack, hole, spatter and weld line type.

[0026] Step S2.2.3: Model training, specifically including:

[0027] Step S2.2.3.1: Training basic parameters and loss function constraints: The training process is built based on the PyTorch framework, using the AdamW optimizer, with the batch size set to 8 or 16, the initial learning rate of 0.001, and the total number of iterations in the range of 200.

[0028] The entire training process uses the weld geometry-aware gradient harmonic loss (WG-HLoss) as the optimization objective, and the total loss formula is as follows: ;in, Based on IoU loss, L AR For aspect ratio adaptive penalty term, L GDH For gradient density harmonic terms, , Weighting coefficient; Aspect ratio penalty factor w gt h gtHere, represents the width and height of the ground truth bounding box, and γ is the scale adjustment coefficient; single-sample gradient weights. P(g) is the probability density function of the gradient norm of the current batch. To prevent division by zero from minimizing the constant.

[0029] Step S2.2.3.2: Two-stage hierarchical iterative training:

[0030] 1) Freeze pre-training phase: Lock the parameters of the four-level backbone network, and only update the neck network, four-way DAMFA modules, and multi-scale detection head; after forward propagation to calculate the total loss, only backpropagate the parameters to the updatable modules;

[0031] 2) Global Fine-tuning Phase: Unfreeze all network parameters, enable the learning rate decay strategy, and minimize the learning rate throughout the process. The loss is updated via global gradient backpropagation; an early stopping mechanism for validation set accuracy is configured, which is triggered if there is no improvement in validation set mAP@0.5 for 15 consecutive rounds to prevent model overfitting.

[0032] Step S2.2.3.3: Model performance verification: After training, an independent test set is used for evaluation. The core evaluation indicators include overall mAP@0.5, accuracy of thin crack recognition, and false detection rate of rust scene.

[0033] Further optimization involves using the DAMFA module, a lightweight improved YOLOv8n model core feature aggregation unit, as input to perform the following steps sequentially:

[0034] 1) Strip Pooling Directional Encoding: The input feature map F is divided into two parallel branches, horizontal and vertical. The horizontal branch uses a 1×N strip pooling kernel to extract the horizontal linear context features of the weld, and the vertical branch uses an N×1 strip pooling kernel to extract the vertical linear context features of the weld. The pooling results from the two branches are concatenated and then fed into a 1×1 convolutional layer to compress the channel dimension, generating a one-dimensional directional encoding vector that encodes the main extension direction of the weld. .

[0035] 2) Affine Transform Offset Modulation: Using directional encoded vectors As input, through the affine transformation formula Solving for convolution sampling offset The system constrains the displacement variance of sampling points in the main direction parallel to the weld to be greater than the displacement variance in the direction perpendicular to the weld, and outputs elliptical sampling grid control parameters.

[0036] 3) Multi-branch anisotropic feature extraction: Configure three parallel dilated convolution branches with dilation rates of r1, r2, and r3, with all branches sharing the modulated sampling offset. Each branch stretches and convolves the receptive field along the main direction of the weld to form an elliptical anisotropic sampling region. The passband is widened along the weld direction to retain continuous crack features, and the passband is narrowed perpendicular to the weld to filter point corrosion noise. The three branches output differentiated feature maps F1, F2, and F3 respectively.

[0037] 4) Adaptive Gated Weighted Fusion: Global average pooling is performed on the three feature maps F1, F2, and F3 respectively to obtain global feature vectors; the three sets of vectors are input into a shared-weight multilayer perceptron, and dynamic gating weights α1, α2, and α3 are generated after Softmax normalization; according to the formula... Weighted aggregation of three-way features outputs an enhanced feature map to improve anti-interference capabilities. And send it into the model's subsequent fusion or detection process.

[0038] Further optimization, step S3, data encoding and breakpoint resumption, includes the following sub-steps:

[0039] Step S3.1, Unified Encapsulation of Multi-Source Data: The defect structured data packet output in step S2, the original weld image, and the three-dimensional spatial coordinates of the image points synchronously acquired in step S1 are uniformly encoded and encapsulated. A globally unified timestamp is bound to the entire set of data and an MD5 checksum is calculated to generate a standardized transmission data packet.

[0040] Step S3.2, Real-time monitoring of communication link status: The airborne edge computing module continuously monitors the transmission quality of the industrial wireless communication link and determines whether the current network is in a stable signal state or an abnormal signal state.

[0041] Step S3.3, Real-time transmission processing: If the network signal is stable, the standardized transmission data packets are directly pushed to the remote monitoring terminal via the wireless link.

[0042] Step S3.4, Local Cache Storage: If the network signal is abnormal, the complete data packet carrying the timestamp and MD5 checksum will be stored in the edge local database, and a cache queue will be constructed in order according to the timestamp.

[0043] Step S3.5, Automatic Resumption of Transmission: When the communication link is restored to normal, the edge computing module reads the data packets in the cache queue in ascending order of timestamp, and transmits them to the remote terminal one by one after MD5 integrity verification. The terminal reorders the received data according to the timestamp to make up for the inspection data missing during the disconnection period.

[0044] Further optimization, step S4, which involves terminal 3D mapping and hierarchical early warning, includes the following steps:

[0045] Step S4.1, Data packet reception and parsing: The remote monitoring terminal receives and verifies the detection result data packet from the UAV, and parses it to obtain the defect category, confidence level, and defect pixel bounding box coordinates.

[0046] Step S4.2, Defect Quantification and Grading Judgment: Retrieve the preset quantitative assessment rule library, and conduct a comprehensive score based on the defect type, size, and structural location to determine the severity level of the defect, which is divided into Grade A severe defects, Grade B general defects, and Grade C minor defects. Grade A defect definition: Cracks in critical load-bearing areas, through-weld cracks; Grade B defect definition: Shallow cracks, voids, minor undercut defects; Grade C defect definition: Micro-porosity, minor beveling defects.

[0047] Step S4.3, tiered matching and corresponding handling procedures:

[0048] If the defect is determined to be a Class A serious defect, immediately stop the machine for inspection and repair, trigger an audible and visual warning, generate an emergency repair work order and push it to the safety manager;

[0049] If the defect is classified as a Class B general defect, the following measures will be taken: increase the frequency of inspection and monitoring, develop a regular maintenance and repair plan, record and track the entire process, and conduct defect trend analysis.

[0050] If the defect is determined to be a minor C-level defect, the defect record will be archived, and routine continuous monitoring will be implemented. The area will be listed as a key area for the next inspection and long-term trend analysis will be conducted.

[0051] Step S4.4, 3D Digital Twin Visualization Mapping: Convert the two-dimensional pixel coordinates of the defect into three-dimensional coordinates in the world coordinate system, match them with the three-dimensional digital twin model of the lifting equipment, and mark the defect location; highlight and annotate Class A defects, use medium-level ordinary annotations for Class B defects, and use light-level annotations for Class C defects.

[0052] S4-5. Generate a structured inspection report: Summarize the complete defect list, defect level distribution, 3D location screenshots, and graded handling suggestions, and output a complete inspection report for archiving.

[0053] Further optimization, step S4.4, which converts the two-dimensional pixel coordinates of the defect into three-dimensional coordinates in the world coordinate system and matches them with the digital twin model annotations, includes the following steps:

[0054] 1) Parameter pre-configuration: The camera intrinsic parameter matrix, the camera extrinsic parameter matrix relative to the UAV body, the registration parameters of the digital twin model of the lifting equipment and the triangular mesh topology information are pre-fixed locally on the remote monitoring terminal;

[0055] 2) Pixel coordinates to camera coordinates: Based on the camera imaging model and pre-stored camera intrinsic parameter matrix, the two-dimensional pixel coordinates of the defect are converted. By performing inverse mapping calculations and combining the prior depth dimension values ​​of the weld structure of the lifting equipment, the three-dimensional coordinates of the defect in the camera coordinate system are obtained.

[0056] 3) Camera coordinates to body coordinates: Call the pre-stored camera-body external parameter matrix, complete the coordinate transformation through rigid body transformation, and obtain the three-dimensional coordinates of the defect in the UAV body coordinate system;

[0057] 4) Converting drone coordinates to world coordinates: Based on the real-time position of the drone transmitted synchronously. From attitude data, construct the transformation matrix from the body coordinate system to the world coordinate system, and calculate the absolute three-dimensional coordinates of the defect in the world coordinate system. ;

[0058] 5) Digital twin model mesh matching: Construct a spatial bounding box centered on the three-dimensional coordinates of the defect world, roughly screen out the overlapping local triangular meshes in the digital twin model, find the nearest target triangular mesh through spatial distance calculation, and project the defect coordinates onto the mesh surface to complete the precise matching;

[0059] 6) Complete the spatial reference registration between the three-dimensional coordinates of the defect and the digital twin model, set differentiated visual labels at the intersection points according to the defect level, and bind the defect category, confidence level, and handling suggestion information to the three-dimensional annotation points.

[0060] A lifting equipment weld defect intelligent identification system based on UAV technology is used to perform the above-mentioned intelligent identification method, including a UAV inspection subsystem, a weld defect intelligent identification subsystem, and a remote monitoring terminal subsystem.

[0061] The UAV inspection subsystem is used to autonomously cruise along a preset route, collect original images of the weld seams of the lifting equipment, and simultaneously collect spatial three-dimensional positioning data of the corresponding points in the images; it integrates a multi-source positioning fusion unit, and outputs continuous six-degree-of-freedom UAV pose by fusing inertial, barometric, and visual odometry data through extended Kalman filtering in satellite signal blocking scenarios.

[0062] The intelligent weld defect recognition subsystem is deployed on an UAV-borne edge computing hardware platform. It is used to complete image preprocessing, real-time inference of the lightweight improved YOLOv8n model, and reconstruction of defect pixel coordinates after inference. It has a built-in local cache database and a breakpoint resume control module, and binds timestamps and MD5 checksums to complete multi-source data caching, verification, and retransmission. The lightweight improved YOLOv8n model includes a backbone, neck, and detection head network structure. All feature fusion nodes in the neck are embedded in the DAMFA module. The improved YOLOv8n model is trained in stages and completes INT8 quantization and operator fusion to accelerate deployment.

[0063] The remote monitoring terminal subsystem and the UAV inspection subsystem interact bidirectionally via an industrial wireless communication link. The remote monitoring terminal subsystem receives and verifies inspection data packets, performs defect quantification and grading, defect pixel-world 3D coordinate step-by-step conversion, and digital twin model visualization annotation. Based on the A / B / C level defects, it performs graded early warning, work order generation, trend analysis, and outputs structured inspection reports.

[0064] Further optimization involves the intelligent weld defect identification subsystem and the remote monitoring terminal subsystem interacting with each other according to the following communication transmission protocol:

[0065] S1. Unified encapsulation of data at the sending end: The edge computing module integrates defect structured data packets, 4K raw image frames, and RTK / EKF fused spatial coordinates, binds a unified timestamp to the entire set of data and generates an MD5 checksum, and constructs a structured data packet {timestamp T, detection data packet D, image frame I, location coordinates P, MD5 checksum H}.

[0066] S2, Network Status Branch Transmission Judgment: Real-time detection of wireless communication link signal quality;

[0067] If the network signal is stable, the structure data packets will be sent to the remote monitoring terminal in real time via the 5G / 4G industrial link.

[0068] If the network signal is abnormal, the complete structure will be stored in an orderly manner in the local SQLite database to form a timestamp cache queue, which will be continuously cached until the network is restored.

[0069] S3. Automatic resume transmission after network recovery: After the communication link is restored, the edge computing module reads the data packets in the cache queue based on the timestamp and MD5 check mechanism and automatically retransmits them to the remote terminal.

[0070] S4. Data verification and reassembly at the receiving end: After receiving the data packet, the remote monitoring terminal first performs MD5 integrity verification; if the verification fails, it initiates a retransmission request to the edge computing module; if the verification passes, it reorders multiple batches of data according to the timestamp, completes time-series reassembly, parses the data, and sequentially performs defect classification assessment and three-dimensional coordinate mapping processing.

[0071] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the above-described intelligent identification method for weld defects in lifting equipment based on UAV technology.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] 1. The invention adopts fully automated drone patrol to replace manual climbing for high-altitude operations, avoiding the risks of falls and component collisions caused by manual high-altitude climbing from the source; relying on the onboard edge terminal to complete image processing and defect identification locally, without cloud forwarding computing power, the single frame weld inference time is short, it is suitable for dynamic drone patrol operations, greatly reducing the inspection time of the whole machine welds, and is suitable for the routine compliance inspection of large numbers of tower cranes and gantry cranes in factories and construction sites.

[0074] 2. This invention improves the YOLOv8n neck fusion architecture, abandons the simple channel splicing mode of the native BiFPN, embeds the self-developed DAMFA direction sensing module into the fully fused node, relies on strip pooling to extract the weld seam extension prior, and generates an elliptical anisotropic receptive field through affine modulation, which is suitable for the slender weld seam and high aspect ratio morphology; it can retain the characteristics of continuous cracks and filter the point noise of rust and oil stains. The measured AP recognition accuracy of slender cracks is improved by 13.9%, and the false detection rate of rust scene is reduced by 12.4%, which is suitable for complex interference real-world working conditions on construction sites.

[0075] 3. Paired with a dedicated weld geometry-aware gradient harmonic loss function, on the one hand, the aspect ratio exponential penalty term is used to optimize the fitting accuracy of the bounding box of slender cracks, solving the problem of poor fit of slender boxes in conventional loss; on the other hand, the gradient density harmonic term is used to adaptively allocate sample weights, reduce the weight of invalid background samples, and enhance the learning ability of difficult samples of small cracks; combined with the two-stage training strategy of trunk freezing + global fine-tuning, the model's generalization ability in the field is improved and the probability of missing small defects is reduced.

[0076] 4. Relying on CUDA parallel acceleration preprocessing operators and combined with CPU+GPU heterogeneous computing power scheduling; the model completes INT8 global quantization and DAMFA internal convolution operator fusion, reducing inference latency with only a 0.2% loss in mAP50 accuracy; the whole is based on the native lightweight YOLOv8n modification, with network changes concentrated in the neck layer, small increase in parameter volume, and is compatible with Jetson Orin NX low-power airborne edge hardware, which can be deployed without high computing power.

[0077] 5. To address the issues of wireless signal shielding by crane metal trusses and network fluctuations in the field, a closed-loop communication mechanism is constructed, which includes multi-source data encapsulation, link identification, local caching, and time-series retransmission. Under weak network conditions, inspection data packets are automatically cached, and when the network recovers, they are automatically sorted in time sequence, and integrity is verified for continued transmission. This prevents the loss or tampering of inspection images, defect data, and positioning coordinates, and meets the industry compliance requirements for traceability, archiving, and verification of special equipment inspection data.

[0078] 6. Establish a four-level coordinate system transformation logic from pixel to camera to machine body to world, and combine it with the twin model mesh intersection algorithm to achieve accurate mapping of two-dimensional detection results to three-dimensional spatial points of the equipment; equipped with A / B / C three-level defect judgment rules, linking twin differentiated annotation, hierarchical sound and light early warning, and automatic work order generation functions, to change the shortcomings of traditional visual inspection that only recognizes, does not locate, and does not classify, and to build a fully intelligent closed loop of inspection-identification-transmission-location-maintenance. Attached Figure Description

[0079] Figure 1 This is an overall flowchart of the intelligent recognition method of the present invention;

[0080] Figure 2 To improve the network structure diagram of the target detection model;

[0081] Figure 3 This is a structural diagram of the Direction-Aware Adaptive Multi-Scale Feature Aggregation Module (DAMFA).

[0082] Figure 4 A flowchart for defect classification assessment and early warning handling;

[0083] Figure 5 Images showing the weld inspection results before and after the improvement;

[0084] Figure 6 To improve the PR change curve of the YOLOv8 algorithm before and after;

[0085] Figure 7 This is a diagram of the overall system architecture of the present invention;

[0086] Figure 8 This diagram illustrates the data communication, encoding, and breakpoint resume protocols. Detailed Implementation

[0087] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0088] Example 1:

[0089] In this embodiment, the inspection targets are the main beams and boom welds of tower cranes and gantry cranes on construction sites. The drone used is a six-rotor industrial drone equipped with a three-axis stabilization gimbal, a 4K industrial camera, an RTK+IMU+barometer+VIO multi-source positioning module, and the airborne edge hardware is NVIDIA Jetson Orin NX. The remote terminal deploys a three-dimensional digital twin model of the lifting equipment, along with defect classification assessment, three-dimensional visualization, and early warning work order modules.

[0090] like Figure 1 As shown, a method for intelligent identification of weld defects in lifting equipment based on UAV technology specifically includes the following steps:

[0091] Step S1: Physical Data Acquisition: Import the 3D CAD model of the lifting equipment, extract the weld coordinates, and automatically generate the UAV's autonomous inspection route. The UAV cruises along the route, the three-axis gimbal adjusts the shooting angle to align with the weld, and the onboard 4K industrial camera acquires the original YUY2 format weld image stream; the multi-source positioning fusion module uses an extended Kalman filter (EKF) to fuse RTK, IMU, barometer, and visual odometry data, synchronously recording the UAV's global spatial 3D coordinates and gimbal attitude angle at the moment of each image acquisition; if the crane's metal truss blocks the satellite RTK signal, continuous positioning output is maintained by relying on the IMU+VIO visual odometry to ensure strict time synchronization of images and spatial points.

[0092] Step S2: Image preprocessing and edge intelligent inference, specifically including:

[0093] Step S2.1, Image Preprocessing: Decode the raw bitstream of the industrial camera YUY2 into an RGB three-channel image; perform multi-level noise reduction by sequentially applying a 5×5 Gaussian filter, a bilateral filter with a neighborhood diameter of 7, a spatial domain of 20, and a pixel value domain of 25; correct the radial and tangential distortion of the lens based on the pre-stored camera calibration parameters, and correct the perspective distortion by combining the gimbal attitude angle; scale the corrected image proportionally and patch the edges to a standard resolution of 640×640, and record the scaling ratio and edge offset parameters simultaneously; accelerate all preprocessing operators through CUDA parallel computing.

[0094] Step S2.2, model deployment and heterogeneous inference scheduling, specifically includes:

[0095] Step S2.2.1: Improve the YOLOv8n object detection model construction, such as... Figure 2As shown, the backbone network takes a 640×640 three-channel RGB image as input, and stacks two layers of 3×3 standard Conv, three sets of Conv+C2f lightweight residual units, and SPPF fast spatial pyramid pooling module in sequence. The C2f module achieves lightweight feature extraction through Split channel splitting, multi-path Bottleneck residual extraction, and Concat channel compression. The SPPF module is connected in series with multiple MaxPool2d layers to fuse global contextual features. The backbone outputs three sets of multi-scale basic features (shallow, middle, and deep layers) and feeds them into the neck network.

[0096] Neck fusion network construction: A bidirectional fusion pathway is constructed, consisting of top-down upsampling and bottom-up sampling. A self-developed DAMFA module is embedded in all feature splicing nodes, with a total of four DAMFA feature aggregation nodes. Deep features are upsampled and spliced ​​with mid-level features and then fed into DAMFA. Further upsampling is performed and splicing is performed with shallow features and then fed into DAMFA. Shallow features are downsampled and spliced ​​with mid-level fused features and then fed into DAMFA. Mid-level features are further downsampled and spliced ​​with deep original features and then fed into DAMFA.

[0097] Detect multi-scale detection head: The differential enhancement features output by the four end DAMFA modules are respectively connected to the independent detection head; each group of detection heads is set with a defect classification branch and a bounding box regression branch in parallel, and outputs the category, confidence level and pixel bounding box coordinates of four types of defects: crack, porosity, spatter and weld line type.

[0098] like Figure 3 As shown, the DAMFA module internally executes the following process: The DAMFA module takes the spliced ​​weld feature map F as input and executes four levels of sub-units sequentially:

[0099] 1) Strip pooling directional encoding: Parallel 1×N horizontal and N×1 vertical strip pooling is used to extract linear context. After channel concatenation, the main direction encoding vector of the weld is generated by 1×1 convolution. ;

[0100] 2) Affine transformation offset modulation: through Solve for the convolution sampling offset, constrain the displacement variance in the parallel weld direction to be larger, and generate an elliptical sampling mesh;

[0101] 3) Multi-branch anisotropic dilated convolution: Configure three parallel dilated convolutions with dilation rates r1, r2, and r3, share modulation offset, stretch the receptive field along the weld, retain fine cracks in the wide passband, filter corrosion noise in the narrow passband, and output three feature maps F1, F2, and F3.

[0102] 4) Adaptive Gated Fusion: After global average pooling of the three features, they are fed into a shared MLP. Softmax generates dynamic weights α1, α2, and α3, which are then processed... Weighted fusion outputs enhanced feature maps.

[0103] Step S2.2.2, the phased training process of the model, is as follows:

[0104] 1) Dataset creation: Collect at least 6420 real-world crane weld images, divided into training, validation, and test sets in a 7:2:1 ratio; label four types of defects: cracks, porosity, spatter, and weld line type; ensure that the bounding boxes for slender cracks fit the real contours; use random flipping, brightness perturbation, Gaussian noise, and stretching transformation to expand the difficult sample of slender defects through online data augmentation; scale all images to 640×640; based on the PyTorch framework and AdamW optimizer, batch size=8, initial learning rate 0.001, and a total of 200 iterations.

[0105] 2) Loss function constraint: Geometrically sensed gradient harmonic loss is used throughout the entire process. L AR Aspect Ratio Penalty Factor Constraining the boundaries of slender defects; L GDH Single sample weights The difficulty of samples is adaptively balanced based on the batch gradient probability density.

[0106] 3) Two-stage iterative training: ① Freeze pre-training (1~50 rounds): Lock all convolutional, C2f, and SPPF weights in the backbone, and only update the neck DAMFA module and the detector head; calculate the total loss in forward propagation, and update only the internal structure of DAMFA and the parameters of the detector head in backpropagation; ② Global fine-tuning (51~200 rounds): Unfreeze all network parameters, enable learning rate decay, and update all weights in global backpropagation; calculate mAP50, mAP50:95, precision, and recall every 10 rounds on the validation set, and configure an early stop mechanism to save the optimal FP32 floating-point weights.

[0107] Accelerated model deployment: The optimal weights are exported to ONNX and compiled into a Tensor RT engine, INT8 quantization compression is performed, and operator fusion is performed on the deformable dilated convolutions inside DAMFA; a CPU+GPU heterogeneous scheduling is adopted, with the CPU controlling process scheduling and data reading and writing, and the GPU carrying out all convolution and feature fusion operations; after deployment on Jetson Orin NX, the single-frame inference time is 22ms, and the mAP50 accuracy is only lost by 0.2%, which meets the requirements of airborne real-time detection.

[0108] Step S2.3, Post-inference processing and pixel coordinate restoration: The model performs inference on a 640×640 scaled image, outputs the normalized bounding box of the defect target, sets a confidence threshold of 0.5 to remove low confidence prediction boxes, and eliminates duplicate boxes of the same defect by non-maximum suppression (NMS); retrieves the scaling and edge-padding parameters recorded in the preprocessing stage, reverses the conversion to map the normalized bounding box to the original 4K image, obtains the true pixel coordinates of the defect, and generates a structured data package containing defect category, confidence level, and original image pixel coordinates.

[0109] Step S3: Data encoding and breakpoint resumption, specifically:

[0110] Step S3.1, Unified Encapsulation at the Sending End: The edge computing module integrates three types of data: Step S2 outputs defect structured data packets, 4K raw image frames, and Step S1 synchronously acquires RTK / EKF spatial three-dimensional coordinates; binds a globally unified timestamp to the entire set of data, calculates the MD5 integrity check code, and constructs a standardized structured data packet: {timestamp T, detection data packet D, image frame I, location coordinates P, MD5 check code H}.

[0111] Step S3.2, Network Status Branch Transmission, specifically:

[0112] The edge computing module monitors the signal-to-noise ratio and packet loss rate of 5G / 4G industrial CPE wireless links in real time.

[0113] (1) Stable network signal: The complete structure is pushed to the remote monitoring terminal in real time through the industrial wireless link;

[0114] (2) Network signal abnormality: Store the structure with timestamp and MD5 code into the onboard SQLite local database, and build a cache queue in ascending order of timestamp for continuous storage.

[0115] Step S3.3, Automatic Resumption of Network Data Transmission: After the communication link is restored to normal, the edge computing module reads the cache queue, sorts it according to the timestamp, verifies the integrity of the data packets using MD5, and automatically retransmits the cached data to the remote terminal in batches.

[0116] Step S3.4: Receiver verification and time sequence reconstruction. After receiving the data packet, the remote terminal performs MD5 integrity verification; if the verification fails, it initiates a retransmission request to the airborne edge computing module; if the verification passes, it reorders the multiple batches of inspection data according to the timestamp, completes the time sequence reconstruction, and sends it to the downstream defect classification and analysis process.

[0117] Step S4: Terminal 3D mapping and hierarchical early warning, such as Figure 4 As shown, it specifically includes:

[0118] Step S4.1, Data Packet Parsing and Defect Classification: The remote terminal receives the reconstructed complete data packet, parses the defect category, confidence level, and original image defect pixel coordinates; retrieves the built-in quantitative evaluation rule base, and conducts a comprehensive score based on the defect type, defect size, and location of the weld in the critical load-bearing area of ​​the crane, classifying the defects into three categories: Grade A (Severe Defect), Grade B (General Defect), and Grade C (Minor Defect). Grade A: Cracks, through-welds, defects in critical load-bearing areas; Grade B: Shallow cracks, porosity, slight undercut; Grade C: Micro-porosity, slight beveling defects.

[0119] Step S4.2, graded differentiated handling process, including: 1) Grade A serious defects: immediately push down the machine for maintenance prompts, local sound and light warnings, automatically generate emergency maintenance work orders that comply with national safety standards, and push them to the on-site safety manager at the same time;

[0120] 2) Class B general defects: Increase the frequency of inspection and monitoring in this weld area, formulate periodic maintenance and repair plans, record defect changes throughout the process, and conduct long-term trend analysis;

[0121] 3) Class C minor defects: Defect information is archived and stored, and is monitored continuously on a routine basis. The weld is marked as a key area for the next drone inspection, and the evolution trend of the defect is tracked in real time.

[0122] Step S4.3: Convert pixel coordinates to world 3D coordinates and perform digital twin annotation:

[0123] 1) Read calibration and pose data: retrieve the pre-stored industrial camera intrinsic parameters and gimbal-camera extrinsic parameter transformation matrix, and synchronously read the UAV's global 3D position, gimbal attitude angle, and defect 2D pixel coordinates in the data packet;

[0124] 2) Pixel back projection: Based on the inverse operation of camera intrinsic parameters, the coordinates of defective pixels are solved as spatial rays in the camera coordinate system;

[0125] 3) Multi-level coordinate system transformation: The camera coordinate system ray is transformed to the global unified world coordinate system sequentially through the gimbal attitude transformation matrix and the UAV body pose matrix;

[0126] 4) Mesh intersection calculation of 3D coordinates: Collision calculation of the world coordinate system spatial ray with the 3D digital twin mesh model of the lifting equipment, the intersection point is the real world 3D coordinates of the defect;

[0127] 5) Digital twin model hierarchical visualization annotation: The three-dimensional coordinates of the defect are spatially registered with the digital twin model, and the defects are annotated differently according to their level: Level A is marked with a high-brightness mark, Level B is marked with a medium-level mark, and Level C is marked with a simple and slight mark; the defect category, confidence level, and handling suggestions are simultaneously bound to the three-dimensional annotation points, and the full set of original inspection images and detection data can be retrieved by clicking on the points.

[0128] Step S4.4: Generate a structured inspection report. This process includes a complete list of defects, statistical distribution of defect levels, 3D twin-annotated screenshots, and corresponding graded handling suggestions. The report is automatically generated and archived locally, supporting export, printing, and background traceability queries.

[0129] Algorithm ablation performance comparison verification:

[0130] 1) Hardware environment:

[0131] Drone platform: Hexacopter industrial drone, equipped with Jetson Orin NX edge computing board, 24GB memory;

[0132] GPU training environment: Linux system, RTX 3090 graphics card, Intel Xeon Platinum 8358P CPU, Python 3.8.19, Torch 2.3.1, CUDA 12.2;

[0133] Data acquisition equipment: 3-axis stabilized gimbal + 4K industrial camera, LED fill light, RTK positioning module, IMU, barometer, VIO visual odometry;

[0134] Remote terminal: Industrial workstation equipped with a 3D digital twin rendering engine.

[0135] 2) Self-built weld defect dataset:

[0136] A dataset of 6420 crane weld images was obtained through a combination of on-site industrial data collection and acquisition using Python network tools. The dataset includes four defect features: cracks, porosity, spatters, and weldinglines. Images were annotated using the Labelimg tool and divided into a 7:2:1 ratio: 4815 images for training, 805 images for validation, and 800 images for testing.

[0137] Training hyperparameters: learning rate 0.01, momentum 0.937, weight decay 0.0005, batch size=8, iterations 200.

[0138] 3) Comparison of test results

[0139] like Figure 5 As shown, Figure 5 In the diagram, (a), (c), and (e) are the detection outputs of the baseline native YOLOv8n model, while (b), (d), and (f) are the detection results of the improved model of this invention (YOLOv8n + DAMFA feature module + WG-HLoss aspect ratio harmonic loss). Figure 5In (a), the baseline model outputs a confidence level of only 0.91 for long crack defects; Figure 5 In (b), the confidence level of the model based on the same crack target is increased to 0.96; Figure 5 In (c), the baseline model outputs confidence scores of 0.87 and 0.86 for the two pore defects, respectively. Figure 5 In the figure, (d) represents the confidence level of the model of the present invention based on the corresponding hole of the same weld seam, which is increased to 0.89 and 0.88. Figure 5 In the figure, (e) represents the baseline model with a confidence level of 0.87 for welding line defects. Figure 5 In the figure (f), the confidence level of the model of the present invention is improved to 0.97 based on the same weld seam to address welding line defects.

[0140] 4) Ablation experiment:

[0141] To verify the individual and combined application effects of DAMFA and WG-HLoss in crane weld defect detection, four sets of ablation comparison experiments were conducted using native YOLOv8n as the baseline. The quantitative indicators were mAP50, mAP50:95, FPS, precision (P), and recall (R). The ablation test results are shown in Table 1.

[0142] Table 1 Ablation Test Results

[0143]

[0144] Through ablation experiments, compared to the basic YOLOv8n algorithm, adding two modules respectively improved the mAP0.5 and mAP0.5:0.95 metrics. The best results were achieved when all three modules were used together. Ultimately, the improved algorithm model achieved mAP0.5 and mAP0.5:0.95 of 86.3.1% and 64.5% respectively, representing improvements of 3.85 and 9.3 percentage points compared to the original algorithm, fully demonstrating the superiority of the improved algorithm in detection accuracy.

[0145] Visual verification of PR curves: such as Figure 6 As shown, where Figure 6 (a) shows the precision-recall curve of the YOLOv8n algorithm. Figure 6 (b) shows the precision-recall curves of the YOLOv8n+DAMFA algorithm. Figure 6 In the middle (c), the precision-recall curves of the YOLOv8n+WG-HLoss algorithm are shown. Figure 6 In the middle (d), the precision-recall curves of the Yolov8n+DAMFA+WG-HLoss algorithm are shown. Figure 6As can be seen, the average accuracy values ​​of the four types of defects in the improved YOLOv8n model described in this invention are all higher than those of the baseline model. The average accuracy of weld line defects reaches 0.987, the average accuracy of cracks reaches 0.948, and the recall rate of defects such as pores and spatter is improved simultaneously. The detection effect diagrams show that the defect detection confidence of the improved model is significantly higher than that of the original YOLOv8n, with no missed detection of small spatter and slender cracks, and no false detection of corrosion.

[0146] Compared to the baseline native YOLOv8n, this invention adds a DAMFA orientation-aware feature aggregation module and a WG-HLoss aspect ratio harmonic loss function:

[0147] 1) For slender cracks with high aspect ratio: the confidence level of defect detection is greatly improved, the ability to extract linear defect features is significantly enhanced, and the problem of low confidence level and easy missed detection of long cracks in the original model is solved.

[0148] 2) For tiny spatter, hole and point defects: The overall confidence of various small target defects is improved, the recall performance of tiny defects is improved, and small weld defects with low confidence and easy to miss in the original image can be detected.

[0149] 3) For regular weld lines: the recognition accuracy is not reduced, and no additional false positives are generated due to rust background or weld texture throughout the process, which improves the recall rate while ensuring the precision rate;

[0150] 4) The visualized detection results corroborate the PR curve and single-type AP quantitative index mentioned above, intuitively proving that the improved scheme of this invention has positive detection gain for all four types of weld defects.

[0151] Example 2:

[0152] like Figure 7 As shown, this embodiment provides an intelligent identification system for weld defects in lifting equipment based on UAV technology. The system is divided into three subsystems: UAV inspection subsystem, intelligent weld defect identification subsystem, and remote monitoring terminal subsystem. It fully implements a weld identification method that includes UAV autonomous data acquisition, real-time defect detection at the airborne edge, weak network interruption resume transmission, and terminal digital twin hierarchical early warning.

[0153] The UAV inspection subsystem is used to autonomously fly along a preset route, collect physical image data of weld seams, and complete the spatial positioning of inspection points through multi-source sensors. In this embodiment, the hardware uses an industrial hexacopter UAV, equipped with a three-axis stabilization gimbal, a 4K industrial global shutter camera, and adjustable LED fill lights; it integrates an RTK positioning module, an IMU inertial unit, a barometer, and a visual odometry, and burns in an extended Kalman filter fusion positioning program; it is equipped with an industrial 5G communication module and flight control unit, imports the inspection route program, and debugs the autonomous obstacle avoidance and fixed-point hovering shooting functions.

[0154] The intelligent weld defect recognition subsystem is deployed on an UAV-borne edge computing hardware platform. It performs image preprocessing, real-time inference of a lightweight improved YOLOv8n model, and reconstruction of defect pixel coordinates after inference. It includes a built-in local cache database and a breakpoint resume control module, binding timestamps and MD5 checksums to cache, verify, and retransmit multi-source data. The lightweight improved YOLOv8n model comprises a backbone, neck, and detection head network structure, with all feature fusion nodes in the neck embedded in the DAMFA module. The improved YOLOv8n model is trained in stages and undergoes INT8 quantization and operator fusion for accelerated deployment. In this embodiment, the airborne edge hardware uses NVIDIA Jetson OrinNX; a CUDA-accelerated preprocessing program and a TensorRT inference engine are deployed; the trained improved target detection ONNX model is imported, and INT8 quantization and DAMFA module internal operator fusion are performed, compiling to generate a .trt inference engine; MD5 encoding, a local cache database, and a breakpoint resume scheduler are deployed, and watchdog timeout restart parameters are configured.

[0155] The remote monitoring terminal subsystem and the UAV inspection subsystem interact bidirectionally via an industrial wireless communication link. The remote monitoring terminal subsystem receives and verifies inspection data packets, performs defect quantification and grading, defect pixel-world 3D coordinate step-by-step conversion, and digital twin model visualization annotation. Based on the A / B / C level defects, it performs graded early warning, work order generation, trend analysis, and outputs structured inspection reports.

[0156] like Figure 8 As shown, the intelligent weld defect identification subsystem and the remote monitoring terminal subsystem complete data interaction according to the following communication transmission protocol:

[0157] S1. Unified encapsulation of data at the sending end: The edge computing module integrates defect structured data packets, 4K raw image frames, and RTK / EKF fused spatial coordinates, binds a unified timestamp to the entire set of data and generates an MD5 checksum, and constructs a structured data packet {timestamp T, detection data packet D, image frame I, location coordinates P, MD5 checksum H}.

[0158] S2, Network Status Branch Transmission Judgment: Real-time detection of wireless communication link signal quality;

[0159] If the network signal is stable, the structure data packets will be sent to the remote monitoring terminal in real time via the 5G / 4G industrial link.

[0160] If the network signal is abnormal, the complete structure will be stored in an orderly manner in the local SQLite database to form a timestamp cache queue, which will be continuously cached until the network is restored.

[0161] S3. Automatic resume transmission after network recovery: After the communication link is restored, the edge computing module reads the data packets in the cache queue based on the timestamp and MD5 check mechanism and automatically retransmits them to the remote terminal.

[0162] S4. Data verification and reassembly at the receiving end: After receiving the data packet, the remote monitoring terminal first performs MD5 integrity verification; if the verification fails, it initiates a retransmission request to the edge computing module; if the verification passes, it reorders multiple batches of data according to the timestamp, completes time-series reassembly, parses the data, and sequentially performs defect classification assessment and three-dimensional coordinate mapping processing.

[0163] Example 3:

[0164] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the above-described intelligent identification method for weld defects in lifting equipment based on UAV technology.

[0165] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for intelligent identification of weld defects in lifting equipment based on UAV technology, characterized in that, Includes the following steps: Step S1, Physical Data Acquisition: The UAV autonomously flies to the target weld area of ​​the lifting equipment according to the preset inspection route, and collects physical image data of the weld through the airborne industrial camera. At the same time, it uses multi-source sensors to synchronously record the spatial three-dimensional coordinates of the acquisition points. Step S2, Image Preprocessing and Edge Intelligent Inference: The edge computing module performs denoising and distortion correction preprocessing on the acquired physical image, calls the improved YOLOv8n target detection model for real-time inference, and outputs a structured data package containing defect category, confidence level, and defect bounding box pixel coordinates; specifically including: Step S2.1, Image Preprocessing: Sequentially perform image decoding, multi-level denoising, distortion correction, and size normalization; Step S2.2, Model Deployment and Inference Scheduling: Build and train the improved YOLOv8n object detection model, export the trained improved object detection model offline to ONNX format and compile it into a TensorRT engine file, perform INT8 model quantization and DAMFA module operator fusion, and then deploy it to the edge computing module; adopt a CPU+GPU heterogeneous scheduling method, with the CPU responsible for process scheduling and data reading and writing, and the GPU carrying out all feature extraction, convolution operation and feature fusion calculation tasks; The construction and offline training of the improved YOLOv8n object detection model include the following steps: Step S2.2.1: Dataset Construction and Preprocessing: Collect and expand crane weld images, construct a labeled dataset containing four types of defects: cracks, holes, spatter, and weld line type, perform random flipping, grayscale perturbation, and tensile deformation data online, enhance and expand the difficult sample of slender defects, and divide it into training set, validation set, and test set in a 7:2:1 ratio; Step S2.2.2: Lightweight model network construction: Based on the original YOLOv8n, a three-layer network architecture consisting of a backbone network, a neck fusion network, and a multi-scale detection head is constructed; Among them, the construction of the neck fusion network: a DAMFA module was designed and replaced with the C2f module in the original neck network. Deep and shallow feature interaction was completed through upsampling and downsampling. A total of four DAMFA feature aggregation nodes were set up. Multi-scale detection head construction: The differential features output by each DAMFA are respectively connected to an independent Detect head. Each group of detection heads is set up in parallel with classification and regression dual branches, and outputs the category, confidence and pixel bounding box of four types of defects: crack, hole, spatter and weld line type. Step S2.2.3: Model training, specifically including: Step S2.2.3.1: Training basic parameters and loss function constraints: The training process is built based on the PyTorch framework, the AdamW optimizer is selected, the batch size is set to 8 or 16, the initial learning rate is 0.001, and the total number of iterations is 200. Throughout the training process, the harmonic loss based on weld geometry perception gradient is used as the optimization objective, and the total loss formula is: ;in, Based on IoU loss, L AR L is an aspect ratio adaptive penalty term. GDH For gradient density harmonic terms, , Weighting coefficient; Aspect ratio penalty factor w gt h gt Here, represents the width and height of the ground truth bounding box, and γ is the scale adjustment coefficient; single-sample gradient weights. P(g) is the probability density function of the gradient norm of the current batch. It is a constant; Step S2.2.3.2: Two-stage hierarchical iterative training: 1) Freeze pre-training phase: Lock the parameters of the four-level backbone network, and only update the neck network, four-way DAMFA modules, and multi-scale detection head; after forward propagation to calculate the total loss, only backpropagate the parameters to the updatable modules; 2) Global Fine-tuning Phase: Unfreeze all network parameters, enable the learning rate decay strategy, and minimize the learning rate throughout the process. The loss is updated via global gradient backpropagation; an early stopping mechanism for validation set accuracy is configured to prevent model overfitting. Step S2.2.3.3: Model performance verification: After training, an independent test set is used for evaluation. The core evaluation indicators include overall mAP@0.5, accuracy of thin crack recognition, and false detection rate in rust scene. The DAMFA module is the core feature aggregation unit of the improved YOLOv8n model. It takes the spliced ​​and fused weld feature map F as input and executes the following steps in sequence: 1) Strip Pooling Directional Encoding: The input feature map F is divided into two parallel branches, horizontal and vertical. The horizontal branch uses a 1×N strip pooling kernel to extract the horizontal linear context features of the weld, and the vertical branch uses an N×1 strip pooling kernel to extract the vertical linear context features of the weld. The pooling results from the two branches are concatenated and then fed into a 1×1 convolutional layer to compress the channel dimension, generating a one-dimensional directional encoding vector that encodes the main extension direction of the weld. ; 2) Affine Transform Offset Modulation: Encoding vectors by direction As input, through the affine transformation formula Solving for convolution sampling offset And constrain the displacement variance of the sampling points in the main direction parallel to the weld to be greater than the displacement variance in the direction perpendicular to the weld, and output the control parameters of the elliptical sampling grid; 3) Multi-branch anisotropic feature extraction: Configure three parallel dilated convolution branches with dilation rates of r1, r2, and r3, with all branches sharing the modulated sampling offset. Each branch stretches and convolves the receptive field along the main direction of the weld to form an elliptical anisotropic sampling region. The passband is widened along the weld direction to retain continuous crack features, and the passband is narrowed perpendicular to the weld to filter point corrosion noise. The three branches output differential feature maps F1, F2, and F3 respectively. 4) Adaptive Gated Weighted Fusion: Global average pooling is performed on the three feature maps F1, F2, and F3 respectively to obtain global feature vectors; the three sets of vectors are input into a shared-weight multilayer perceptron, and dynamic gating weights α1, α2, and α3 are generated after Softmax normalization; according to the formula... Weighted aggregation of three-way features outputs an enhanced feature map to improve anti-interference capabilities. And send it into the model's subsequent fusion or detection process; Step S2.3, Post-inference processing and coordinate mapping: After the model inference is completed, a confidence threshold of 0.5 is set to remove low-confidence prediction boxes, and duplicate detection boxes of the same target are eliminated by non-maximum suppression; combined with the scaling ratio and edge-padding parameters recorded in the size normalization stage, the 640×640 normalized bounding box coordinates output by the improved YOLOv8n target detection model are inversely mapped to the original image pixel coordinate system to obtain the true pixel coordinates of the defects; Step S3, Data Encoding and Resume Transmission: The structured data packet, the corresponding original image frame, and the spatial location coordinates are uniformly encoded and then sent to the remote monitoring terminal through the wireless communication link; Step S4, Terminal 3D Mapping and Graded Early Warning: The remote monitoring terminal receives and parses the data, completes the quantitative grading of defects according to the preset evaluation rules; converts the defect pixel coordinates into 3D coordinates in the world coordinate system through coordinate transformation, performs mesh matching with the digital twin model of the lifting equipment, and marks the defect location and grade; triggers corresponding early warning and handling actions according to the defect grade.

2. The intelligent identification method for weld defects in lifting equipment based on UAV technology according to claim 1, characterized in that, In step S1, the multi-source sensor includes an RTK high-precision positioning module, an inertial measurement unit, a barometer, and a visual odometry. When the satellite signal is blocked by the structure of the lifting equipment and the RTK positioning fails, the extended Kalman filter algorithm is used to fuse the data from the inertial measurement unit, the barometer, and the visual odometry to achieve joint positioning and output continuous pose information.

3. The intelligent identification method for weld defects in lifting equipment based on UAV technology according to claim 1, characterized in that, Step S3, data encoding and breakpoint resumption, includes the following sub-steps: Step S3.1, Unified Encapsulation of Multi-Source Data: The defect structured data packet output in step S2, the original weld image, and the three-dimensional spatial coordinates of the image points synchronously acquired in step S1 are uniformly encoded and encapsulated. A globally unified timestamp is bound to the entire set of data and an MD5 checksum is calculated to generate a standardized transmission data packet. Step S3.2, Real-time monitoring of communication link status: The airborne edge computing module continuously monitors the transmission quality of the industrial wireless communication link and determines whether the current network is in a stable signal state or an abnormal signal state. Step S3.3, Real-time transmission processing: If the network signal is stable, the standardized transmission data packets are directly pushed to the remote monitoring terminal via the wireless link; Step S3.4, Local Cache Storage: If the network signal is abnormal, the complete data packet carrying the timestamp and MD5 checksum is stored in the edge local database, and a cache queue is constructed in order according to the timestamp. Step S3.5, Automatic Resumption of Transmission: When the communication link is restored to normal, the edge computing module reads the data packets in the cache queue in ascending order of timestamp, and transmits them to the remote terminal one by one after MD5 integrity verification. The terminal reorders the received data according to the timestamp to make up for the inspection data missing during the disconnection period.

4. The intelligent identification method for weld defects in lifting equipment based on UAV technology according to claim 1, characterized in that, Step S4, the terminal 3D mapping and hierarchical early warning, includes the following steps: Step S4.1, Data packet reception and parsing: The remote monitoring terminal receives and verifies the detection result data packet from the UAV, and parses it to obtain the defect category, confidence level, and defect pixel bounding box coordinates; Step S4.2, Defect Quantification and Grading Judgment: Retrieve the preset quantitative assessment rule library, combine the defect type, size, and location in the structure to conduct a comprehensive score, and determine the severity level of the defect, which is divided into Grade A severe defect, Grade B general defect, and Grade C minor defect; Class A defects are defined as: cracks in the load-bearing critical area and through-weld cracks. Class B defects are defined as: shallow cracks, holes, and minor undercut defects. Class C defects are defined as: minor pores and slight beveling defects. Step S4.3, tiered matching and corresponding handling procedures: If the defect is determined to be a Class A serious defect, immediately stop the machine for inspection and repair, trigger an audible and visual warning, generate an emergency repair work order and push it to the safety manager; If the defect is classified as a Class B general defect, the following measures will be taken: increase the frequency of inspection and monitoring, develop a regular maintenance and repair plan, record and track the entire process, and conduct defect trend analysis. If the defect is determined to be a Class C minor defect, the defect record will be archived, and routine continuous monitoring will be carried out. The area will be listed as a key area for the next inspection and a long-term trend analysis will be conducted. Step S4.4, 3D digital twin visualization mapping: Convert the two-dimensional pixel coordinates of the defect into three-dimensional coordinates in the world coordinate system, match them with the three-dimensional digital twin model of the lifting equipment, and mark the location of the defect; Class A defects are highlighted and clearly annotated; Class B defects are marked with medium-level ordinary annotations; and Class C defects are marked with light and slight annotations. S4-5. Generate a structured inspection report: Summarize the complete defect list, defect level distribution, 3D location screenshots, and graded handling suggestions, and output a complete inspection report for archiving.

5. The intelligent identification method for weld defects in lifting equipment based on UAV technology according to claim 4, characterized in that, Step S4.4, converting the two-dimensional pixel coordinates of the defect to three-dimensional coordinates in the world coordinate system and matching them with the digital twin model annotations, includes the following steps: 1) Parameter pre-configuration: The camera intrinsic parameter matrix, the camera extrinsic parameter matrix relative to the UAV body, the registration parameters of the lifting equipment digital twin model and the triangular mesh topology information are pre-fixed locally on the remote monitoring terminal; 2) Pixel coordinates to camera coordinates: Based on the camera imaging model and pre-stored camera intrinsic parameter matrix, the two-dimensional pixel coordinates of the defect are converted. By performing inverse mapping calculations and combining the prior depth dimension values ​​of the weld structure of the lifting equipment, the three-dimensional coordinates of the defect in the camera coordinate system are obtained. 3) Camera coordinates to body coordinates: Call the pre-stored camera-body external parameter matrix, complete the coordinate transformation through rigid body transformation, and obtain the three-dimensional coordinates of the defect in the UAV body coordinate system; 4) Converting drone coordinates to world coordinates: Based on the real-time position of the drone transmitted synchronously. From attitude data, construct the transformation matrix from the body coordinate system to the world coordinate system, and calculate the absolute three-dimensional coordinates of the defect in the world coordinate system. ; 5) Digital twin model mesh matching: Construct a spatial bounding box centered on the three-dimensional coordinates of the defect world, roughly screen out the overlapping local triangular meshes in the digital twin model, find the nearest target triangular mesh through spatial distance calculation, and project the defect coordinates onto the mesh surface to complete the precise matching; 6) Complete the spatial reference registration between the three-dimensional coordinates of the defect and the digital twin model, set differentiated visual labels at the intersection points according to the defect level, and bind the defect category, confidence level, and handling suggestion information to the three-dimensional annotation points.

6. A smart identification system for weld defects in lifting equipment based on UAV technology, characterized in that, The method for performing the intelligent identification method according to any one of claims 1 to 5 includes an unmanned aerial vehicle inspection subsystem, a weld defect intelligent identification subsystem, and a remote monitoring terminal subsystem. The UAV inspection subsystem is used to autonomously cruise along a preset route, collect original images of the weld seams of the lifting equipment, and simultaneously collect spatial three-dimensional positioning data of the corresponding points in the images; it integrates a multi-source positioning fusion unit, and outputs continuous six-degree-of-freedom UAV pose by fusing inertial, barometric, and visual odometry data through extended Kalman filtering in satellite signal blocking scenarios. The intelligent weld defect recognition subsystem is deployed on an UAV-borne edge computing hardware platform. It is used to complete image preprocessing, real-time inference of the lightweight improved YOLOv8n model, and reconstruction of defect pixel coordinates after inference. It has a built-in local cache database and a breakpoint resume control module, and binds timestamps and MD5 checksums to complete multi-source data caching, verification, and retransmission. The lightweight improved YOLOv8n model includes a backbone, neck, and detection head network structure. All feature fusion nodes in the neck are embedded in the DAMFA module. The improved YOLOv8n model is trained in stages and completes INT8 quantization and operator fusion to accelerate deployment. The remote monitoring terminal subsystem and the UAV inspection subsystem interact bidirectionally via an industrial wireless communication link. The remote monitoring terminal subsystem receives and verifies inspection data packets, performs defect quantification and grading, defect pixel-world 3D coordinate step-by-step conversion, and digital twin model visualization annotation. Based on the A / B / C level defects, it performs graded early warning, work order generation, trend analysis, and outputs structured inspection reports.

7. The intelligent identification system for weld defects in lifting equipment based on UAV technology according to claim 6, characterized in that, The intelligent weld defect identification subsystem and the remote monitoring terminal subsystem complete data interaction according to the following communication transmission protocol: S1. Unified encapsulation of data at the sending end: The edge computing module integrates defect structured data packets, 4K raw image frames, and RTK / EKF fused spatial coordinates, binds a unified timestamp to the entire set of data and generates an MD5 checksum, and constructs a structured data packet {timestamp T, detection data packet D, image frame I, location coordinates P, MD5 checksum H}. S2, Network Status Branch Transmission Judgment: Real-time detection of wireless communication link signal quality; If the network signal is stable, the structure data packets will be sent to the remote monitoring terminal in real time via the 5G / 4G industrial link. If the network signal is abnormal, the complete structure will be stored in an orderly manner in the local SQLite database to form a timestamp cache queue, which will be continuously cached until the network is restored. S3. Automatic resume transmission after network recovery: After the communication link is restored, the edge computing module reads the data packets in the cache queue based on the timestamp and MD5 check mechanism and automatically retransmits them to the remote terminal. S4. Data verification and reassembly at the receiving end: After receiving the data packet, the remote monitoring terminal first performs MD5 integrity verification; if the verification fails, it initiates a retransmission request to the edge computing module; if the verification passes, it reorders multiple batches of data according to the timestamp, completes time-series reassembly, parses the data, and sequentially performs defect classification assessment and three-dimensional coordinate mapping processing.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the intelligent identification method for weld defects of lifting equipment based on UAV technology as described in any one of claims 1 to 5.

Images