A system and method for detecting and quantifying diseases of a drone sling based on multi-source asynchronous data alignment

Abstract

A kind of unmanned aerial vehicle sling disease detection quantification system and method based on multi-source asynchronous data alignment, system includes: unmanned aerial vehicle platform is used to carry gimbal and camera system, laser ranging module, flight control and fusion height acquisition module, airborne edge computing module and data storage module, carry out inspection flight;Gimbal and camera system are used to collect the surface video stream image of sling target component;Laser ranging module is used to generate ranging snapshot;Flight control and fusion height acquisition module are used to output height data;Height correction input module is used to set detection starting point as relative height zero point;Airborne edge computing module is used to process data;Data storage module is used to save image and metadata file;Remote control display terminal is used to display inspection picture and superimposed information in real time;Post-processing software module is used to carry out disease area calculation processing;Report generation module is used to generate detection report;User interactive display module is used to carry out result preview processing.

Description

Technical Field

[0001] This invention relates to the field of bridge cable inspection technology, and more specifically, to a quantitative system and method for detecting cable defects using unmanned aerial vehicles (UAVs) based on multi-source asynchronous data alignment. Background Technology

[0002] The suspension cable is a key load-bearing component of a suspension bridge, and its sheath, outer covering and appearance directly affect the bridge's durability and operational safety. The main appearance defects of the suspension cable include paint peeling, flaking, corrosion, broken wires, anchor head damage, rubber aging and protective layer damage. At present, the following technical routes are mainly used for the appearance inspection of suspension cables: (1) Manual close-range inspection: Through bridge inspection vehicles, high-altitude equipment, rope operations, etc., the inspectors approach the suspension cable and record it by visual inspection and photography. (2) Cable climbing robot inspection: A special cable climbing robot is used to climb along the suspension cable and collect images of the suspension cable surface through sensors such as cameras to achieve automatic inspection. (3) UAV image inspection: UAVs are used to collect images or videos of the suspension cable surface, and then manual offline interpretation or defect detection is performed through deep learning models. (4) Automatic identification of bridge appearance defects based on deep learning: In recent years, some technical solutions have attempted to use target detection algorithms to identify bridge cracks, peeling, corrosion and other appearance defects, and output defect categories and detection boxes.

[0003] However, existing technologies still have the following shortcomings: manual inspection is inefficient, risky, and subjective, making it difficult to meet the needs of batch and standardized bridge inspection; cable-climbing robots are not adaptable to different cable diameters, sheath types, and suspension vibration damping frame structures, making on-site deployment difficult; existing drone solutions mostly focus on image acquisition or defect identification, failing to directly obtain the actual size and precise location of defects; existing automatic identification solutions are mostly based on pixel features, lacking collaborative processing with laser ranging and camera intrinsic parameters, making it difficult to achieve real-time quantification of defect sizes; suspension cables are slender components, and engineering inspection requires recording the position of defects along the cable direction, but existing technologies typically only save image or time information, failing to directly generate usable location records; during drone inspections, image, ranging, and altitude data are usually updated asynchronously by different threads or callbacks, and without a lightweight real-time alignment mechanism, errors in size quantification and location calibration are easily caused; existing inspection results lack an automatic connection mechanism with post-processing reports, still requiring manual processing of defect quantity, area, grade, and score, resulting in low efficiency.

[0004] Therefore, there is a need for an integrated technical solution that can automatically generate inspection reports by identifying defects, quantifying dimensions, calibrating locations, saving images, and post-processing them during UAV inspections, and that can adapt to the real-time fusion of multi-source asynchronous data. Summary of the Invention

[0005] The present invention aims to overcome at least one of the defects (deficiencies) of the prior art and provide a quantification system and method for UAV sling defect detection based on multi-source asynchronous data alignment. This system integrates defect identification, size quantification, position calibration, image saving, and post-processing to automatically generate inspection reports during UAV inspections, and can adapt to the effect of real-time fusion of multi-source asynchronous data.

[0006] On one hand, the technical solution adopted by the present invention is a quantitative system for detecting defects in UAV slings based on multi-source asynchronous data alignment, the system comprising:

[0007] Unmanned aerial vehicle (UAV) platform: used to carry gimbal and camera system, laser ranging module, flight control and fusion altitude acquisition module, airborne edge computing module and data storage module, and used to perform inspection flights along sling target components; Gimbal and camera system: used to acquire surface video stream images of the target components of the suspension cable and transmit them to the airborne edge computing module; Laser ranging module: used to acquire distance information of the current target area to continuously update the ranging value and form a ranging snapshot; Flight control and fusion altitude acquisition module: used to output altitude data corresponding to the current UAV platform or gimbal and camera system; Altitude correction input module: connected to the airborne edge computing module, used to process the detection starting point altitude correction amount input in the remote control display terminal, and set the detection starting point to the relative altitude zero point; Airborne edge computing module: connected to the gimbal and camera system, laser ranging module, flight control and fusion altitude acquisition module, altitude correction input module and data storage module respectively, for data processing; Data storage module: Connected to the airborne edge computing module, used to store image and metadata files; Remote control display terminal: Communicates with the UAV platform and is used to display inspection footage and overlay information in real time; Post-processing software module: Connected to the data storage module, it is used to read the data exported from the data storage module and perform scale / height information extraction, disease verification and annotation, area calculation, disease statistics and table generation. Report generation module: Connects to the post-processing software module and is used to generate test reports; User interaction display module: Connected to the post-processing software module, it is used for disease annotation, disease type selection, template selection, result preview, and report export.

[0008] This invention proposes a UAV-based quantitative system for detecting cable defects in bridges, based on multi-source asynchronous data alignment. It is suitable for specialized inspections of slender cable components. The system integrates a unified hardware architecture, with a UAV platform carrying camera, ranging, height measurement, and edge computing components. These modules work collaboratively, offering greater adaptability compared to cable-climbing robots and avoiding the risks of manual inspections at heights. A height correction input module calibrates the cable detection zero point, enabling precise location of defects along the cable's axial direction based on height data. An onboard edge computing module coordinates the processing of multi-source asynchronous data (images, ranging, and height), achieving data linkage and matching to ensure accurate quantification of defect dimensions. The system dual-stores original images and metadata files, utilizing onboard local storage to avoid data loss during wireless transmission. A remote control display terminal allows for real-time verification of inspected defect information, improving the convenience of on-site operations. With the support of post-processing software, report generation, and interactive modules, it can automatically extract detection parameters, verify defects, calculate defect areas, and statistically analyze defect data. It eliminates the need for manual record keeping and generates compliant inspection reports with one click, ensuring standardized operation and maintenance inspection of bridge suspension cables, significantly reducing manual workload, and improving the efficiency of the entire process of cable defect detection, assessment, and archiving.

[0009] On the other hand, the present invention also provides a quantitative method for detecting defects in UAV slings based on multi-source asynchronous data alignment according to the above-described system, the method comprising the following steps: S1: Input inspection parameters to create an inspection task; S2: Obtain the current millisecond-level timestamp as a unified time reference, and mark the image frame, ranging snapshot, and altitude data with their own dedicated timestamps; S3: The airborne edge computing module decodes the H.264 encoded video stream of the UAV collected by the gimbal and camera system to obtain RGB image frames, and performs callback and enqueue processing; S4: Perform distance snapshot update and altitude data update; S5: The disease detection model is used to detect RGB image frames. The output detection results are aligned with the image frames and corresponding metadata is generated. S6: Approximately align the distance measurement with the latest image value, and then perform size quantization; S7: Align the height with the image freshness threshold, and then perform position calibration; S8: Overlay detection and quantization information onto the processed image and forward it to the remote control display terminal for real-time display; S9: Simultaneously save the original image without superimposed information, the image with superimposed information, and the metadata file to the data storage module, and then export the data to the post-processing software module for processing; S10: The post-processing software module automatically reads the file information, extracts the height and scale data, then performs disease labeling and area calculation, and finally generates a disease table and an inspection report.

[0010] This invention provides a quantitative method for detecting defects in UAV slings adapted to the aforementioned system. Utilizing multi-source asynchronous data alignment logic, it uses millisecond-level unified timestamps to complete the temporal marking of three types of heterogeneous data: image, ranging, and altitude. This solves the problem of misalignment in asynchronous data updates across multiple modules and improves detection linkage. Through video stream decoding and queuing, ranging snapshot binding, and real-time altitude acquisition, it achieves frame-by-frame alignment between defect detection results and image frames, ensuring accurate defect identification and tracing. Defect size is quantified based on ranging temporal alignment, and sling axial calibration is completed based on altitude freshness threshold alignment, achieving integrated airborne quantification of defect size and location, eliminating the need for manual ranging verification. Simultaneously, defect quantification information is overlaid in real-time and transmitted back to the terminal, facilitating real-time verification of defect status by on-site personnel. Original images, overlaid images, and metadata files are stored in categories to ensure traceability of detection data. Finally, the post-processing software automatically analyzes parameters, verifies defects, calculates areas, and automatically generates defect statistics tables and inspection reports. It connects the entire process of airborne data acquisition, edge computing, and backend report generation, reducing manual intervention, improving the accuracy of defect detection and inspection efficiency of slender suspension cables, and adapting to routine bridge operation and maintenance.

[0011] Preferably, in step S1, before performing the inspection task, the following is also included: Record the takeoff altitude of the drone, then operate the drone to fly to the sling detection starting point and record the altitude of the detection starting point; input the altitude correction value through the altitude correction input module so that the relative altitude corresponding to the sling detection starting point is 0 m.

[0012] This preferred procedure, by calibrating the starting point of the sling inspection to the relative height zero point, can unify the coordinate system for calculating the height of the entire area, eliminate the positioning deviation caused by the take-off altitude of the UAV and the difference in elevation between the take-off and landing terrain, standardize the calculation benchmark for the vertical position of defects, accurately calculate the relative position of defects along the sling axis, and improve the uniformity of the sling defect point calibration and the applicability of the project.

[0013] Preferably, step S3 includes: S31: The decoded RGB image frame is output to the application layer in the airborne edge computing module via a callback method. After receiving the RGB image frame, the application layer records the image timestamp t_img and assigns a unique image number frame_id to the image frame. S32: The application layer copies the RGB image frame and determines whether it is currently in a frozen display state. If it is in a frozen display state, it skips the enqueueing and detection processing of the new image frame and encodes and sends the frozen frame back for display. If it is not in a frozen display state, it pushes the copied RGB image frame into the image queue for asynchronous consumption by the independent detection thread.

[0014] This optimized process binds traceability data to image-specific timestamps and unique frame numbers, enabling traceable and correlated image data. Furthermore, it incorporates a frozen image branch logic, which blocks new image detection operations while frozen, conserving onboard edge computing resources. By relying on independent threads for asynchronous image processing, it decouples image transmission from the disease detection process, avoiding image stuttering and detection delays. This balances the real-time performance of the inspection images with the adaptability to onboard computing power, improving overall system stability.

[0015] Preferably, step S4 includes: In the laser ranging module, the ranging snapshot is updated periodically in an independent thread or an independent callback. The ranging snapshot includes the ranging value D and the ranging timestamp t_range. When an RGB image frame enters the detection queue, the system latches the latest ranging snapshot and binds the ranging snapshot to the RGB image frame to update the ranging snapshot. In the flight control and fusion altitude acquisition module, altitude data is updated in real time through a subscription callback method, and the altitude timestamp t_height is recorded.

[0016] The above steps employ independent threads and callback mechanisms to periodically update ranging snapshots and real-time altitude data, simultaneously marking their respective timestamps. When an image is enqueued, the latest ranging snapshot is latched to complete frame data binding. This method decouples multiple acquisition channels, preventing interference and ensuring continuous updating of ranging and altitude data. By binding image frames one-to-one with ranging snapshots, a temporal correlation is established, providing reliable data pairing relationships for subsequent asynchronous data timeliness verification and accurate quantification of defect dimensions. This avoids mismatches between image and distance information, significantly reducing detection errors caused by misalignment of multi-source asynchronous data and improving the reliability of quantification results.

[0017] Preferably, step S5 includes: S51: Retrieve RGB image frames from the image queue in a first-in-first-out manner and convert them into BGR images to meet the input format requirements of the model; S52: The detection thread calls the disease detection model to perform inference on the BGR image, outputs the disease category, confidence level and detection box, and the output detection result forms a strict frame-aligned relationship with the RGB image frame; S53: The detection thread overlays the detection box, disease category, confidence level, distance value, corrected height, height correction value, scale information, time information and actual size onto the BGR image, then converts the overlaid BGR image back to RGB format and constructs the metadata corresponding to the RGB image frame.

[0018] This optimized process adapts and converts image formats to meet the input standards of the disease detection model, ensuring its normal inference operation. A first-in, first-out (FIFO) queue is used to process images, guaranteeing consistent detection sequence and ensuring that disease identification results are strictly bound to the original image frame-by-frame for clear traceability. A complete set of detection quantification parameters is overlaid on the image, and corresponding metadata is generated simultaneously, enabling on-site visualization. All disease, distance, location, and scale information are archived along with the image, providing complete raw data support for backend verification, area calculation, and report generation, thus improving the completeness and intuitiveness of inspection data.

[0019] Preferably, step S6 includes: S61: Perform a timeliness check on the latched ranging snapshots, when If the distance exceeds the preset effective range threshold T_range, the distance snapshot will not be used for size quantization, and the size result will be marked as unquantizable; otherwise, the size result will be marked as quantizable, and step S62 will be executed. S62: Convert the pixel size of the disease target into the actual size based on the pixel coordinates of the detection box corresponding to the RGB image frame, the valid ranging snapshot bound to the image frame, and the camera intrinsic data; wherein, when the ranging snapshot corresponding to the detection box is invalid, the ranging value is missing, or the camera intrinsic data is missing, the actual size calculation is not performed, the size_valid field is set to false, and the invalid_reason field is written with the corresponding reason.

[0020] This preferred procedure establishes a timeliness verification mechanism for ranging snapshots. It judges the validity of ranging data by comparing the timestamp difference with a preset threshold, filtering out invalid ranging data with excessive time-series deviations. Only qualified ranging snapshots are used in conjunction with camera intrinsic parameters to convert pixel dimensions to actual physical dimensions, simultaneously setting validity indicators and failure reason recording fields. In scenarios where ranging data or camera parameters are missing, the size calculation is automatically stopped and failure records are retained, avoiding the output of erroneous or defective size values. This balances data quantification accuracy and fault tracing capabilities, ensuring the authenticity and reliability of airborne quantification results.

[0021] More preferably, step S62 includes: S621: Obtain camera intrinsic parameter data through checkerboard calibration or other camera calibration methods; S622: Correct the distortion of the pixel coordinates of the detection box to obtain the corrected detection box; S623: Calculate the horizontal and vertical pixel equivalents of the corrected detection box based on the ranging value and camera intrinsic data; S624: Calculate the actual size of the defect target based on the corrected detection frame data and the horizontal and vertical pixel equivalent data.

[0022] This step first obtains complete intrinsic parameters through camera calibration to eliminate pixel deviations caused by inherent lens distortion. Then, distortion correction is applied to the detection frame coordinates to restore the true pixel contours of the lesions. Next, based on the distance measurement values ​​and intrinsic parameters, the pixel equivalents in each direction are calculated to establish the conversion relationship between pixels and actual physical lengths. Based on the corrected detection frame, the actual length and width of the lesions are accurately calculated. This process eliminates quantization errors caused by lens distortion and imaging scale deviations at each stage, significantly improving the accuracy of lesion size calculation. The quantization results closely match engineering measurement standards, providing accurate basic data for post-processing lesion area calculation and lesion level assessment.

[0023] Preferably, step S7 includes: S71: Read the latest height data and determine its freshness. If the time difference between the current time t_now and the height timestamp t_height does not exceed the preset height freshness threshold T_height, then proceed to step S72 and use the height data to participate in the disease location calculation; otherwise, consider the height data invalid, do not participate in the location calculation, and mark the location result as unusable. In particular, when the height data is invalid, the height timestamp has expired, or the height value is missing, the location calculation is also not performed, and the position_valid field is set to false. S72: Calculate the relative position of the defect target along the sling direction using the takeoff point altitude, detection starting point altitude, and current effective altitude data, thereby calibrating the defect location.

[0024] This optimized procedure sets up height data freshness verification logic, filtering out outdated and invalid data by using time difference and height freshness thresholds to select valid height data. Based on the takeoff point, detection starting point, and real-time valid height, the relative position of defects along the sling axis is calculated, achieving accurate vertical calibration of defects. When height data times out or is missing, the position calculation is automatically terminated and the valid position field is marked as false, completely avoiding point calibration deviations caused by invalid height data. Valid / invalid position data is distinguished and identified, ensuring the reliability of defect location results and providing standardized and usable positioning data for defect sorting and ledger archiving in post-processing software.

[0025] Preferably, step S10 includes: in the post-processing software module, manually verifying the diseased areas in the original image, selecting or inputting the disease type, and performing rectangular box annotation, polygon annotation, or curve annotation to calculate the actual area of ​​the disease.

[0026] This optimized process supports multiple annotation methods, including rectangles, polygons, and curves, to adapt to various irregular cable defects. Manual verification corrects for deviations in the automatic airborne identification system, synchronously matching the corresponding defect type. Based on previously stored dimensional parameters, the actual defect area is automatically calculated, avoiding the inefficiency and subjective errors of manual measurement. The use of multiple annotation methods ensures complete coverage of irregular defect boundaries such as peeling, rust, and broken wires, improving the accuracy of area calculations. This provides precise quantitative data for defect severity assessment and statistical report generation, meeting bridge inspection standards.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows: First, to achieve real-time identification of bridge cable defects; Second, it enables online quantization of the pixel size of the defect detection frame to the actual engineering size; Third, to determine the relative position of the diseased target along the direction of the suspension cable; Fourth, a lightweight real-time fusion of multi-source asynchronous data is achieved by adopting a unified time reference, image frame alignment, approximate alignment of the latest ranging values, and high freshness threshold judgment. Fifth, an unusable rollback mechanism is adopted to reduce erroneous dimensions and positions caused by expired height data, invalid ranging data, or abnormal parameters; Sixth, the corrected height, distance measurement value, scale information, detection frame, defect type, confidence level and actual size are displayed in real time on the remote control display terminal, which is convenient for on-site verification; Seventh, it simultaneously saves the overlaid information image, the original image, and the metadata file, facilitating automatic extraction and result traceability in post-processing; Eighth, the post-processing software module can automatically generate disease tables and export test reports according to templates, improving report generation efficiency and standardization. Ninth, the use of drones for inspection does not rely on a specific cable-climbing robot structure and has good adaptability to different bridge slings; Tenth, by using RGB image frames as common association units, detection results, ranging results, height results, size results, position results, and validity markers are uniformly encapsulated, reducing engineering quantization errors caused by mismatch of multi-source asynchronous data. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the overall system structure of the present invention.

[0029] Figure 2 This is a schematic diagram illustrating the spatial relationship between the drone and the sling inspection system according to the present invention.

[0030] Figure 3 This is a schematic diagram of the lightweight time alignment of multi-source asynchronous data according to the present invention.

[0031] Figure 4 This is a flowchart of the method of the present invention.

[0032] Figure 5 This is a schematic diagram illustrating the conversion principle of the actual size of the detection frame in this invention.

[0033] Figure 6 This is a flowchart of the validity determination and rollback control of the present invention.

[0034] Figure 7 This is a flowchart illustrating the image / metadata saving, filename parsing, and report generation process of this invention.

[0035] Figure 8 This is a schematic diagram of the real-time overlay display interface of the present invention.

[0036] Figure 9 This is a schematic diagram of the post-processing software interface of the present invention.

[0037] Figure descriptions: 1. UAV platform; 2. Gimbal and camera system; 3. Laser ranging module; 4. Flight control and fusion altitude acquisition module; 5. Altitude correction input module; 6. Airborne edge computing module; 7. Data storage module; 8. Remote control display terminal; 9. Post-processing software module; 10. Report generation module; 11. User interaction display module; 12. Sling target component; 13. Damaged area; 14. RGB image frame; 15. Ranging snapshot; 16. Altitude data; 17. Detection box; 18. Original image; 19. Overlay information image; 20. Metadata file. Detailed Implementation

[0038] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the invention. To better illustrate the following embodiments, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; it is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0039] Example 1 like Figure 1 As shown, this embodiment provides a quantitative system for detecting defects in UAV slings based on multi-source asynchronous data alignment. The system includes: The UAV platform 1 is used to carry a gimbal and camera system 2, a laser ranging module 3, a flight control and fusion altitude acquisition module 4, an airborne edge computing module 6, and a data storage module 7, and is used to perform inspection flights along the sling target component 12; The gimbal and camera system 2 are mounted on the drone platform 1, such as Figure 2 As shown, during the UAV inspection process, the gimbal and camera system 2 moves towards the target component 12 of the sling to collect video stream images of its surface and outputs H.264 encoded video streams, which are then transmitted to the airborne edge computing module 6. The laser ranging module 3 is installed on the UAV platform 1. Its ranging direction corresponds to the optical axis of the gimbal and camera system 2. It is used to obtain the distance information of the current shooting target area to continuously update the ranging value and form a ranging snapshot 15. The flight control and fusion altitude acquisition module 4 is installed in the UAV platform 1 and is used to continuously update and output the altitude data 16 corresponding to the current UAV platform 1 or gimbal and camera system 2. The height correction input module 5 is connected to the airborne edge computing module 6 and is used to process the detection starting point height correction amount input in the remote control display terminal 8 and set the detection starting point to the relative height zero point. The airborne edge computing module 6 is connected to the gimbal and camera system 2, the laser ranging module 3, the flight control and fusion altitude acquisition module 4, the altitude correction input module 5, and the data storage module 7, respectively. It is used to process data, including: performing video stream reception, H.264 decoding, RGB image frame callback, image copying and queuing, freeze display judgment, RGB / BGR format conversion, target detection, detection box and status information overlay, RGB to H.264 encoding back transmission, timestamp marking, asynchronous data alignment, size quantization, position calibration, validity judgment, rollback control, and result encapsulation, etc. The airborne edge computing module 6 acquires the H.264 encoded video stream through a high-speed data channel; it decodes the H.264 to RGB compact format image frames by calling the video stream interface, and outputs the decoded RGB image frame 14 to the application layer via image callback. Upon receiving the RGB image frame, the application layer immediately records the image timestamp and copies the RGB image frame. Then, the application layer determines whether it is in a frozen display state. If it is, it directly sends the frozen frame to the encoding interface for H.264 encoding and transmission, skipping the current new image frame. Subsequent detection and processing: If not in a frozen display state, the copied RGB image frame 14 is sent to the image queue; when the RGB image frame 14 is enqueued, the system latches the latest ranging snapshot 15 and binds the ranging snapshot 15 to the RGB image frame 14; the target detection thread retrieves the RGB image frame 14 from the image queue in a first-in-first-out manner, converts the RGB image frame 14 into a BGR image, performs target detection inference, and outputs the detection results. Since the detection results are directly derived from the corresponding image frame 14, the detection box 17, the disease category, and the confidence score are strictly aligned with the image frame 14. In addition, the system also determines whether the difference between the ranging timestamp of ranging snapshot 15 and the image timestamp of image frame 14 exceeds the preset ranging validity threshold. If it does not exceed the threshold, the ranging snapshot 15 is used for size quantization. If it exceeds the threshold, the ranging snapshot 15 is not used, and the size result is marked as unquantizable. When the ranging value is valid, the system calculates the actual size of the defect based on the pixel coordinates of the detection box 17, the ranging value D, and the camera intrinsic parameters. Then, before performing position calibration, the system reads the latest height data 16 and determines whether the difference between the height timestamp and the current time exceeds the preset height freshness threshold. If it does not exceed the threshold, the height data 16 is used for position calculation. If it exceeds the threshold, the height data 16 is not used, and the position result is marked as unusable. When the height value is valid, the system calculates the relative position of the defect target along the sling direction based on the current valid height value, the take-off point height, the detection starting point height, and the height correction amount.

[0040] The data storage module 7 is connected to the airborne edge computing module 6 and is used to save the image and metadata file 20, specifically including: saving the original image 18, the overlaid information image 19 and the metadata file 20; The remote control display terminal 8 is communicatively connected to the UAV platform 1, used for real-time display of inspection footage and overlay information, as well as on-site verification; it is not used as the main transmission channel for post-processing data. For example... Figure 8 As shown, the system overlays and displays the corrected height, height correction value, distance measurement value, scale information, current actual time, detection frame, disease type, confidence level, and actual length and width of the detection frame on the real-time screen of the remote control display terminal 8. Post-processing software module 9 is connected to data storage module 7, such as... Figure 7 and Figure 9 As shown, after image acquisition is completed, the post-processing software module 9 reads the original image 18 and automatically extracts the height information, scale information, sling number and image number from the image file name and / or metadata file 20. The operator manually reviews and annotates the diseased area 13 in the original image 18 in the post-processing software module 9 and selects or inputs the disease type. The post-processing software module 9 automatically calculates the actual area of ​​the disease based on the pixel size and scale information of the annotation box, and determines the scale and score according to the inspection specifications.

[0041] Finally, the report generation module 10 generates an inspection report based on the disease table, labeled images, and preset report templates; and uses the user interaction display module 11 to perform disease labeling, disease type selection, template selection, result preview, and report export.

[0042] This system achieves synchronized distance acquisition through integrated hardware deployment and matching of the ranging optical path with the camera optical axis. Airborne edge computing uniformly handles video decoding, time-series marking, multi-source data binding, and timeliness verification to quantify defect dimensions. Combined with height correction, it accurately calibrates the axial defect location of the suspension cable. Failure data is automatically marked to avoid erroneous output. Furthermore, all detection parameters can be visualized in real-time on the terminal, with complete images and metadata stored locally. The primary data source for post-processing is the exported airborne storage data, rather than data directly transmitted via real-time wireless link after acquisition, thus reducing the impact of wireless link interruptions or delays on the integrity of post-processing results. Simultaneously, the backend software can automatically parse parameters, annotate and calculate defect areas in multiple formats, and generate standardized inspection reports with one click, minimizing manual intervention throughout the process and balancing inspection safety, detection accuracy, and operational efficiency.

[0043] Example 2 like Figure 4 As shown, this embodiment provides a quantitative method for detecting defects in UAV slings based on multi-source asynchronous data alignment according to the system described in Embodiment 1. The method includes the following steps: Step S1: Input inspection parameters to create an inspection task; Preferably, before the inspection begins, the operator inputs image naming prefix information such as bridge number and sling number through the height correction input module 5 and the task initialization interface, and the system records the drone's takeoff altitude. After the UAV platform 1 flies to the detection starting point of the sling target component 12, the system records the height of the detection starting point. And calculate the height correction:

[0044] in, To correct for height, the relative height corresponding to the starting point of the sling detection is 0 m at this time. , and subsequent current height The same altitude coordinate system is used. If the flight control and fusion altitude acquisition module 4 can output the relative altitude at the time of taking the picture (relative to the takeoff point altitude), then... 0 is acceptable.

[0045] like Figure 2 As shown, when the UAV platform 1 inspects the target component 12 along the sling, the flight control and fusion altitude acquisition module 4 outputs the current altitude. The airborne edge computing module 6 is based on , , and Calculate the relative position of the damaged target along the direction of the sling:

[0046] Right now:

[0047] in, This refers to the relative starting point height of the disease.

[0048] Step S2: Obtain the current millisecond-level timestamp as a unified time reference, and mark the image frame, ranging snapshot, and altitude data with their own unique timestamps; Preferably, this method obtains the current millisecond-level timestamp as a unified time base. For example... Figure 3 As shown, RGB image frame 14, ranging snapshot 15, and height data 16 all record corresponding timestamps during acquisition, reception, or callback, for subsequent alignment determination and data binding. These timestamps include at least the image timestamp t_img, the ranging timestamp t_range, and the height timestamp t_height.

[0049] Step S3: The airborne edge computing module 6 decodes the H.264 encoded video stream of the UAV collected by the gimbal and camera system 2 to obtain RGB image frame 14, and performs callback and enqueue processing. Preferably, such as Figure 4 As shown, step S3 includes: Step S31: The decoded RGB image frame 14 is output to the application layer in the airborne edge computing module via a callback method. After receiving the RGB image frame 14, the application layer records the image timestamp t_img and assigns a unique image number frame_id to the image frame. Step S32: The application layer copies the RGB image frame and determines whether it is currently in a frozen display state. If it is in a frozen display state, it skips the queuing and detection processing of the new image frame and encodes and sends the frozen frame back for display. If it is not in a frozen display state, it pushes the copied RGB image frame into the image queue for asynchronous consumption by the independent detection thread.

[0050] The subsequent detection results, ranging data, size data, location data, and validity markers are all associated with the frame_id.

[0051] Step S4: Update the ranging snapshot and altitude data; Preferably, step S4 includes: In the laser ranging module 3, the ranging snapshot 15 is periodically updated using an independent thread or an independent callback. The ranging snapshot includes at least the ranging value D and the ranging timestamp t_range, such as... Figure 3As shown, when RGB image frame 14 enters the detection queue, the system latches the latest ranging snapshot and binds the ranging snapshot to the RGB image frame, thereby updating the ranging snapshot; In the flight control and fusion altitude acquisition module 4, altitude data 16 is updated in real time via a subscription callback method, and an altitude timestamp t_height is recorded. The altitude data 16 includes at least the current altitude value. , height timestamp t_height.

[0052] Step S5: Use the disease detection model to detect RGB image frames, and the output detection results are aligned with the image frames in the same frame, and corresponding metadata is generated; Specifically, step S5 includes: Step S51: Take RGB image frame 14 from the image queue in a first-in-first-out manner and convert it into a BGR image to meet the input format requirements of the model; Step S52: The detection thread calls the disease detection model to perform inference on the BGR image and outputs the disease category. Confidence level and detection frame 17,

[0053] in, This indicates detection box 17. This indicates the coordinates of the top-left corner of the detection box. This indicates the coordinates of the bottom right corner of the detection box.

[0054] Since the detection results are derived directly from the current image frame, the detection box, disease category, and confidence level are strictly aligned with the image frame.

[0055] Furthermore, the disease detection model is not limited to a specific model and can employ YOLO series networks, SSD networks, Faster R-CNN networks, Transformer detection networks, or other lightweight object detection networks. In this embodiment, the disease detection model uses the YOLOv8s model, with a confidence threshold P_min of 0.40. When the confidence level P of the detection result is lower than 0.40, the system does not output the detection box.

[0056] Step S53: The detection thread overlays the detection box, disease category, confidence level, distance measurement value, corrected height, height correction value, scale information, time information, and actual size onto the BGR image. Then, the overlaid BGR image is converted back to RGB format, and metadata corresponding to the RGB image frame is constructed. This metadata includes at least the image number, target coordinates, disease category, confidence level, size result, location result, and validity marker.

[0057] Step S6: Approximately align the distance measurement with the latest value of the image, and then perform size quantization; Preferably, such as Figure 5 and Figure 6 As shown, step S6 includes: Step S61: Perform a timeliness check on the latched ranging snapshot. When the distance exceeds the preset effective distance measurement threshold T_range, the distance measurement snapshot is not used for size quantization, and the size result is marked as unquantizable; otherwise, the size result is marked as quantizable, and step S62 is executed. In this specific embodiment, T_range can be 100 ms to 500 ms; preferably, T_range is 300 ms.

[0058] Step S62: Convert the pixel size of the disease target into its actual size based on the pixel coordinates of the detection box corresponding to the RGB image frame, the valid ranging snapshot bound to the image frame, and the camera intrinsic data; wherein, when the ranging snapshot corresponding to the detection box is invalid, the ranging value is missing, or the camera intrinsic data is missing, the actual size calculation is not performed, the size_valid field is set to false, and the invalid_reason field is written to "range invalid".

[0059] Similarly, when the distance value D If the distance measurement is missing, the range_status is abnormal, or the distance measurement value D is less than the minimum effective range or greater than the maximum effective range, the system will not use the distance measurement value D for size quantization, and will write "distance measurement missing" to the invalid_reason field respectively.

[0060] More preferably, step S62 includes: Step S621: Obtain camera intrinsic parameter data using the checkerboard calibration method or other camera calibration methods; the obtained camera intrinsic parameter data includes: horizontal focal length parameter... The vertical focal length parameter is Principal point coordinates , Radial distortion coefficient , , Tangential distortion coefficient , .

[0061] Step S622: Perform distortion correction on the pixel coordinates of the detection box to obtain the corrected detection box; the corrected detection box :

[0062] Step S623: Calculate the horizontal and vertical pixel equivalents of the corrected detection frame based on the ranging value and camera intrinsic data; this invention calculates the horizontal pixel equivalents separately. and vertical pixel equivalent This can avoid area errors caused by the default assumption that the horizontal and vertical dimensions are completely consistent.

[0063] When the unit of the distance measurement value D is meters, and When the unit is pixels, the horizontal pixel equivalent and vertical pixel equivalent The calculations are as follows: The actual dimensions of the detection frame are calculated as follows:

[0064]

[0065] in, and The unit is mm / pixel.

[0066] Step S624: Calculate the actual size of the defect target based on the corrected detection frame data and the horizontal and vertical pixel equivalent data.

[0067] Actual width of the disease target and actual length The calculation is as follows:

[0068]

[0069] in: W and L The unit is m.

[0070] Specifically, in this embodiment, the camera's horizontal focal length parameter The vertical focal length is 22513.890 pixels. The value is 22511.701 pixels. For a certain RGB image frame 14, the effective range value D latched by the system is 6.200 m.

[0071] The system calculates the horizontal pixel equivalent. and vertical pixel equivalent : =1000×D / =1000×6.20 / 22513.890=0.275 mm / pixel; =1000×D / =1000×6.20 / 22511.701=0.275 mm / pixel; =150 pixels; =300 pixels; The actual width W of the diseased target is: W =150×0.275 / 1000=0.041 m; The actual length L of the target disease is: L =300×0.275 / 1000=0.083 m; The actual size of the diseased target can then be calculated using the above formula.

[0072] Step S7: Align the height with the image freshness threshold, and then perform position calibration; Preferably, such as Figure 3 and Figure 6 As shown, step S7 includes: Step S71: Read the latest height data 16 and determine the freshness of the height data 16. If the time difference between the current time t_now and the height timestamp t_height does not exceed the preset height freshness threshold T_height, then execute step S72 and use the height data to participate in the disease location calculation; otherwise, consider the height data 16 invalid, not to participate in the location calculation, and mark the location result as unusable. Among them, when the height data is invalid, the height timestamp has timed out, or the height value is missing, the location calculation is not performed, and the position_valid field is set to false.

[0073] In this embodiment, the system sets the high freshness threshold T_height to 2 seconds. If the current time t_now and the height timestamp t_height satisfy: t_now t_height≤T_height The height data 16 is considered valid and allowed to be used in the calculation of the relative position H of the disease. If the following conditions are met: t_now t_height>T_height If the height data 16 is deemed invalid, the system will not use this height data 16 in position calculation, will set the position_valid field to false, and write "height timeout" to the invalid_reason field.

[0074] In addition, when height data 16 is missing or the height status is abnormal, the system will not perform position calibration and will write "height missing" to the invalid_reason field.

[0075] like Figure 6 As shown, when the ranging data is invalid but the height data is valid, the system retains the disease category. C Confidence level P Detection box 17 and relative position of disease H However, the actual length L Actual width W The relevant fields are marked as unquantifiable.

[0076] When altitude data is invalid but distance measurement data is valid, the system retains the disease category. C Confidence level P Detection frame 17, actual length L and actual width W However, the relative location of the disease H Marked as unavailable.

[0077] When both the ranging and height data are invalid, the system retains the original image 18, the overlaid information image 19, the detection box 17, the disease category C, and the confidence level. P However, it does not output size quantization results or position calibration results.

[0078] By employing the above methods, complex interpolation, resampling, and large-scale cache backtracking are avoided, enabling low-computational-load, low-latency multi-source asynchronous data alignment on the UAV edge computing platform. Simultaneously, when ranging or altitude data is unreliable, the system uses validity marking and backoff control to prevent erroneous size or location results from entering subsequent defect statistics tables and inspection reports.

[0079] Step S72: Calculate the relative position of the defect target along the sling direction using the takeoff point height, detection starting point height, and current effective height data, thereby calibrating the defect location; Specifically, such as Figure 2 , 3 As shown in Figure 6, the airborne edge computing module 6 calculates the altitude of the takeoff point. Detection starting point height and current effective height Calculate the relative position of the damaged target along the direction of the sling. : Correct height for:

[0080] The relative position of the damaged target along the direction of the suspension cable for:

[0081] Substitution We can then conclude that:

[0082] in, This indicates the height of the target defect relative to the starting point of the sling inspection. If the sling inspection direction is from bottom to top, then... This can indicate the upward distance of the disease from the detection starting point; if the inspection direction of the sling is from top to bottom, then it can be adjusted according to the inspection direction. The sign of the disease is agreed upon; thus, the location of the disease is marked in the above manner.

[0083] Step S8: Overlay detection and quantization information onto the processed image and forward it to the remote display terminal for real-time display; like Figure 8 As shown, detection and quantization information are overlaid on the image processed by the detection thread. The upper left corner of the image displays at least the corrected height. H Altitude correction amount h Distance value D Scale information, current actual time, and validity markers; at least the detection frame, disease type, confidence level, and actual length should be displayed at the target location of the disease. L and actual width W .

[0084] After the overlay is completed, the overlaid RGB image and its corresponding metadata are sent to the encoding interface to encode the RGB image into H.264 data. This data is then transmitted back to the UAV platform 1 via the video stream sending interface, and the UAV platform 1 forwards it to the remote control display terminal 8 for display. When the system is in a frozen display state, it does not perform detection processing on newly received RGB image frames, but instead encodes and transmits the frozen frame to maintain the frozen image on the remote control display terminal 8. This effectively prevents incorrect size or position results from entering subsequent defect statistics tables and inspection reports.

[0085] Step S9: Synchronously save the original image without superimposed information, the image with superimposed information, and the metadata file to the data storage module, and then export the data to the post-processing software module for processing; wherein, in this embodiment, the metadata includes at least the following fields: frame_id; image_path; overlay_path; t_img; t_range; t_height; D; λx; λy; bbox; class; confidence; W; L; H; range_valid; height_valid; size_valid; position_valid; invalid_reason.

[0086] like Figure 7 As shown, in this embodiment, after image acquisition is completed, the original image 18, the overlaid information image 19, and the metadata file 20 are exported from the data storage module 7 to the post-processing software module 9. It should be noted that the video displayed on the remote control display terminal 8 is generated by the airborne edge computing module 6 encoding the overlaid RGB image into H.264 data and then transmitting it back. This transmitted video is mainly used for real-time on-site viewing and verification. The primary data source used by the post-processing software module 9 is the original image 18, the overlaid information image 19, and the metadata file 20 exported from the data storage module 7, rather than the real-time transmitted video stream received by the remote control display terminal 8. This ensures that the primary data source for post-processing is the exported airborne stored data, rather than the data directly transmitted via the real-time wireless link after acquisition, effectively reducing the impact of wireless link interruptions or delays on the integrity of the post-processing results.

[0087] Step S10: The post-processing software module 9 automatically reads the file information, extracts the height and scale data, then performs disease labeling and area calculation, and finally generates a disease table and a test report.

[0088] like Figure 7 and Figure 9 As shown, post-processing software module 9 reads the original image 18 and automatically parses the information in the image file name. This information includes at least: bridge number; upstream and downstream locations, sling number; height information; and scale information. and The height information is used for disease sorting and location, and the scale information is used for disease area conversion.

[0089] For example, for each original image 18, the post-processing software module 9 prioritizes reading the height and scale information from the name of image 18; the metadata file 20 is mainly used for tracing and verification.

[0090] For filenames: BridgeA_S_Cable03_H12.350m_piexl_(2.207,2.211)_raw.jpg Post-processing software module 9 can parse the following: The bridge is designated Bridge A. Upstream and downstream side number S; The sling number is Cable03; The corrected height is 12.350 m; Horizontal pixel equivalent It is 2.207 mm / pixel; Vertical pixel equivalent It is 2.211 mm / pixel.

[0091] Preferably, such as Figure 5 , Figure 7 and Figure 9 As shown, step S10 includes: in the post-processing software module, manually verifying the diseased area 13 in the original image 18, selecting or inputting the disease type, and performing rectangular box annotation, polygon annotation, or curve annotation to calculate the actual area of ​​the disease.

[0092] In this embodiment, for rectangular box annotations, the software automatically calculates the number of pixels in the horizontal direction as 'a' and the number of pixels in the vertical direction as 'b'. Based on the scale information... and Calculate the actual area of ​​the disease, A (m²), as follows: 2 ):

[0093] Where A is in units of m². and When they are equal or approximately equal, they can be simplified to:

[0094] Specifically, in this embodiment, if the horizontal pixel count (a) of a certain lesion annotation box is 150 pixels and the vertical pixel count (b) is 350 pixels, It is 2.207 mm / pixel. If the density is 2.211 mm / pixel, then the affected area A is: A=150×2.207×350×2.211=0.26m² In subsequent implementations, AI models can be used for automatic labeling first, followed by manual review and confirmation. The reviewed labeling results can be converted into the label format required for training the object detection model for iterative optimization.

[0095] Preferably, the post-processing software module 9 determines the disease scale and score based on the disease type, disease area, and preset detection specifications. The detection specifications are pre-written into the system as template parameters. Different disease types correspond to different area thresholds, scaling rules, and scoring rules.

[0096] Post-processing software module 9 sorts the defect records based on the sling number and height information, generating a defect table. The defect table includes at least the following fields: sling number; height; defect type; defect area; scale; score; image number.

[0097] When the size_valid field of a disease record is false, the post-processing software module 9 marks the disease area as "unquantifiable" in the disease table and displays invalid_reason in the remarks. When the position_valid field of a disease record is false, the post-processing software module 9 marks the height or relative position field as "unavailable".

[0098] Finally, as Figure 7 and Figure 9 As shown, the report generation module 10 reads the defect table, the annotated images, and the preset report template to generate an inspection report. The inspection report includes basic project information, equipment information, inspection method description, sling number, defect table, annotated defect images, defect quantity statistics, defect area statistics, grading results, scoring results, and inspection conclusions.

[0099] Furthermore, in this embodiment, the post-processing software module 9 can also export the manually reviewed annotation results as training labels for the target detection model, such as YOLO format labels. Thus, the manually reviewed results generated during post-processing can be used for subsequent iterative training of the defect detection model, improving the model's recognition accuracy in similar bridge cable-stayed bridge scenarios.

[0100] Through the above methods, this embodiment achieves a complete closed loop from on-site data collection by UAVs to disease verification, area calculation, disease table generation, and export of inspection reports. It also clearly distinguishes between real-time H.264 video transmission and post-processing data sources: real-time H.264 video transmission is used for on-site display, while the post-processing software mainly reads the original image 18, overlaid information image 19, and metadata file 20 exported from the data storage module 7, thereby ensuring good integrity and traceability of the post-processing results.

[0101] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A quantitative system for detecting defects in UAV slings based on multi-source asynchronous data alignment, characterized in that, The system includes: Unmanned aerial vehicle (UAV) platform: used to carry gimbal and camera system, laser ranging module, flight control and fusion altitude acquisition module, airborne edge computing module and data storage module, and used to perform inspection flights along sling target components; Pan-tilt unit and camera system: used to acquire surface video stream images of the target component of the suspension cable and transmit them to the edge computing module; Laser ranging module: used to acquire distance information of the current target area to continuously update the ranging value and form a ranging snapshot; Flight control and fusion altitude acquisition module: used to output altitude data corresponding to the current UAV platform or gimbal and camera system; Altitude correction input module: connected to the airborne edge computing module, used to process the detection starting point altitude correction amount input in the remote control display terminal, and set the detection starting point to the relative altitude zero point; Airborne edge computing module: connected to the gimbal and camera system, laser ranging module, flight control and fusion altitude acquisition module, altitude correction input module and data storage module respectively, for data processing; Data storage module: Connected to the airborne edge computing module, used to store image and metadata files; Remote control display terminal: Communicates with the UAV platform and is used to display inspection footage and overlay information in real time; Post-processing software module: Connected to the data storage module, it is used to read the data exported from the data storage module and perform scale / height information extraction, disease verification and annotation, area calculation, disease statistics and table generation. Report generation module: Connects to the post-processing software module and is used to generate test reports; User interaction display module: Connected to the post-processing software module, it is used for disease annotation, disease type selection, template selection, result preview, and report export.

2. A quantitative method for detecting UAV sling defects based on multi-source asynchronous data alignment according to the system described in claim 1, characterized in that, The method includes the following steps: S1: Input inspection parameters to create an inspection task; S2: Obtain the current millisecond-level timestamp as a unified time reference, and mark the image frame, ranging snapshot, and altitude data with their own dedicated timestamps; S3: The airborne edge computing module decodes the H.264 encoded video stream of the UAV collected by the gimbal and camera system to obtain RGB image frames, and performs callback and enqueue processing; S4: Perform distance snapshot update and altitude data update; S5: The disease detection model is used to detect RGB image frames. The output detection results are aligned with the image frames and corresponding metadata is generated. S6: Approximately align the distance measurement with the latest image value, and then perform size quantization; S7: Align the height with the image freshness threshold, and then perform position calibration; S8: Overlay detection and quantization information onto the processed image and forward it to the remote control display terminal for real-time display; S9: Simultaneously save the original image without superimposed information, the image with superimposed information, and the metadata file to the data storage module, and then export the data to the post-processing software module for processing; S10: The post-processing software module automatically reads the file information, extracts the height and scale data, then performs disease labeling and area calculation, and finally generates a disease table and an inspection report.

3. The method for quantitative detection of UAV sling defects based on multi-source asynchronous data alignment according to claim 2, characterized in that, In step S1, the following steps are included before performing the inspection task: Record the takeoff altitude of the drone, then operate the drone to fly to the sling detection starting point and record the altitude of the detection starting point; input the altitude correction value through the altitude correction input module so that the relative altitude corresponding to the sling detection starting point is 0 m.

4. The method for quantitative detection of UAV sling defects based on multi-source asynchronous data alignment according to claim 2, characterized in that, Step S3 includes: S31: The decoded RGB image frame is output to the application layer in the airborne edge computing module via a callback method. After receiving the RGB image frame, the application layer records the image timestamp t_img and assigns a unique image number frame_id to the image frame. S32: The application layer copies the RGB image frame and determines whether it is currently in a frozen display state. If it is in a frozen display state, it skips the enqueueing and detection processing of the new image frame and encodes and sends the frozen frame back for display. If it is not in a frozen display state, it pushes the copied RGB image frame into the image queue for asynchronous consumption by the independent detection thread.

5. The method for quantitative detection of UAV sling defects based on multi-source asynchronous data alignment according to claim 4, characterized in that, Step S4 includes: In the laser ranging module, the ranging snapshot is updated periodically in an independent thread or an independent callback. The ranging snapshot includes the ranging value D and the ranging timestamp t_range. When an RGB image frame enters the detection queue, the system latches the latest ranging snapshot and binds the ranging snapshot to the RGB image frame to update the ranging snapshot. In the flight control and fusion altitude acquisition module, altitude data is updated in real time through a subscription callback method, and the altitude timestamp t_height is recorded.

6. The method for quantitative detection of UAV sling defects based on multi-source asynchronous data alignment according to claim 5, characterized in that, Step S5 includes: S51: Retrieve RGB image frames from the image queue in a first-in-first-out manner and convert them into BGR images to meet the input format requirements of the model; S52: The detection thread calls the disease detection model to perform inference on the BGR image, outputs the disease category, confidence level and detection box, and the output detection result forms a strict frame-aligned relationship with the RGB image frame; S53: The detection thread overlays the detection box, disease category, confidence level, distance value, corrected height, height correction value, scale information, time information and actual size onto the BGR image, then converts the overlaid BGR image back to RGB format and constructs the metadata corresponding to the RGB image frame.

7. The method for quantitative detection of UAV sling defects based on multi-source asynchronous data alignment according to claim 6, characterized in that, Step S6 includes: S61: Perform a timeliness check on the latched ranging snapshots, when If the distance exceeds the preset effective range threshold T_range, the distance snapshot will not be used for size quantization, and the size result will be marked as unquantizable; otherwise, the size result will be marked as quantizable, and step S62 will be executed. S62: Convert the pixel size of the disease target into the actual size based on the pixel coordinates of the detection box corresponding to the RGB image frame, the valid ranging snapshot bound to the image frame, and the camera intrinsic data; wherein, when the ranging snapshot corresponding to the detection box is invalid, the ranging value is missing, or the camera intrinsic data is missing, the actual size calculation is not performed, the size_valid field is set to false, and the invalid_reason field is written with the corresponding reason.

8. The method for quantitative detection of UAV sling defects based on multi-source asynchronous data alignment according to claim 7, characterized in that, Step S62 includes: S621: Obtain camera intrinsic parameter data through checkerboard calibration or other camera calibration methods; S622: Correct the distortion of the pixel coordinates of the detection box to obtain the corrected detection box; S623: Calculate the horizontal and vertical pixel equivalents of the corrected detection box based on the ranging value and camera intrinsic data; S624: Calculate the actual size of the defect target based on the corrected detection frame data and the horizontal and vertical pixel equivalent data.

9. A quantitative method for detecting UAV sling defects based on multi-source asynchronous data alignment according to claim 7, characterized in that, Step S7 includes: S71: Read the latest height data and determine its freshness. If the time difference between the current time t_now and the height timestamp t_height does not exceed the preset height freshness threshold T_height, then proceed to step S72 and use the height data to participate in the disease location calculation; otherwise, consider the height data invalid, do not participate in the location calculation, and mark the location result as unusable. In particular, when the height data is invalid, the height timestamp has expired, or the height value is missing, the location calculation is also not performed, and the position_valid field is set to false. S72: Calculate the relative position of the defect target along the sling direction using the takeoff point altitude, detection starting point altitude, and current effective altitude data, thereby calibrating the defect location.

10. A quantitative method for detecting UAV sling defects based on multi-source asynchronous data alignment according to claim 9, characterized in that, Step S10 includes: in the post-processing software module, manually verifying the diseased areas in the original image, selecting or inputting the disease type, and performing rectangular box annotation, polygon annotation, or curve annotation to calculate the actual area of ​​the disease.

Images