A self-inspection infrared unmanned aerial vehicle inspection method and device

Abstract

The application discloses a kind of self-inspection infrared unmanned aerial vehicle inspection method and device, it is related to unmanned aerial vehicle damage detection technical field.The self-inspection infrared unmanned aerial vehicle inspection method obtains the structure information of the tower crane arm support to be detected, and analysis generates the inspection path containing several inspection points;The inspection path is input to the infrared unmanned aerial vehicle and is flown and is inspected;When each inspection point is reached by infrared unmanned aerial vehicle, the infrared thermal imaging image and the visible light image corresponding to the tower crane arm support to be detected are respectively collected, and multi-source fusion is carried out, and the corresponding multi-source fusion image is obtained;Damage identification is carried out on the multi-source fusion image based on improved YOLOv5 damage detection algorithm, and outputs inspection report, the structure information of the tower crane arm support is obtained in the application, the key position and the main component coordinate are used as inspection point, and the optimal inspection path is generated, and this path planning ensures that connecting point, corner, stiffener and other high-risk areas are certainly covered, fundamentally avoid the possibility of missing detection.

Description

Technical Field

[0001] This invention relates to the field of drone damage detection technology, specifically to a self-inspection infrared drone inspection method and device. Background Technology

[0002] With the continuous expansion of modern engineering construction, large tower cranes are increasingly used in high-rise building and bridge construction. As a key load-bearing and force-transmitting component, the operational safety of the tower crane jib directly affects the safety and efficiency of the construction site. Traditional tower crane inspections rely heavily on manual methods, typically involving operators using telescopes, cameras, and other tools to observe the tower crane's structural condition from the ground or at close range. This method is not only limited by factors such as the operator's perspective and ambient lighting, but also has low inspection efficiency, making it difficult to meet the demands of high-frequency and large-scale inspections.

[0003] The limitations of existing technologies include at least the following problems: Firstly, existing tower crane boom inspection routes are mostly based on fixed waypoints or experience templates, failing to establish mandatory inspection points for high-risk areas such as connection points and corners based on the specific structural information of the boom. This results in blind spots and structural omissions. Secondly, infrared thermal imaging images and visible light images are typically acquired and interpreted separately from different sources, lacking pixel-level alignment and fusion. This makes it difficult to corroborate temperature anomalies with surface textures, and they are easily obscured by background heat, reflections, or shadows, amplifying the probability of missed detections at the perception level. Thirdly, conventional target detection configurations on single-source images lack sufficient separability for small-scale, slender defects, resulting in positioning drift and low confidence levels. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a self-inspection infrared drone inspection method and device, which solves the problems of high missed detection rate, numerous false detections, and inaccurate defect location in existing tower crane boom inspections.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A self-inspection infrared drone inspection method includes the following steps: acquiring structural information of the tower crane boom to be inspected and analyzing and generating an inspection path containing several inspection points; inputting the inspection path into the infrared drone for flight inspection; when the infrared drone reaches each inspection point, acquiring infrared thermal imaging images and visible light images of the corresponding tower crane boom to be inspected, and performing multi-source fusion to obtain the corresponding multi-source fused image; performing damage identification on the multi-source fused image based on the improved YOLOv5 damage detection algorithm, and outputting an inspection report.

[0007] Furthermore, the structural information includes the three-dimensional coordinates of each component and the three-dimensional coordinates of each key part.

[0008] Furthermore, the specific steps for generating an inspection path containing several inspection points are as follows: The three-dimensional coordinates of each key part of the tower crane boom to be inspected are set as inspection points, and supplementary inspection points are generated according to the three-dimensional coordinates of each component and preset rules; the three-dimensional coordinates of all inspection points are read, and under preset constraints, a path optimization algorithm is used to determine the access order of the inspection points and generate a connecting trajectory to form an inspection path.

[0009] Furthermore, the infrared thermal imaging image includes several infrared pixels, and each infrared pixel corresponds to an apparent temperature pixel value; the visible light image includes several visible light pixels, and each visible light pixel corresponds to a reflected light brightness pixel value; the multi-source fusion image includes several fused pixels, and each fused pixel corresponds to a fused pixel value.

[0010] Further, the specific steps to obtain the corresponding multi-source fusion image are as follows: Geometric registration is performed between the infrared thermal imaging image and the visible light image of the tower crane boom to be tested, so that the pixels of the two correspond one-to-one under the same pixel grid; On the common pixel grid, the apparent temperature pixel value and the reflected light brightness pixel value corresponding to the fusion pixel value are read respectively, and the fusion pixel value is analyzed to form a multi-source fusion image.

[0011] Further, the specific steps for analyzing the fused pixel value of each fused pixel are as follows: For each infrared pixel, based on a set neighborhood range, the mean and standard deviation of the corresponding neighborhood apparent temperature pixels are identified and analyzed, and the temperature weight of each fused pixel is analyzed; For each visible light pixel, the corresponding brightness gradient is analyzed, and based on a set neighborhood range, the maximum brightness gradient of the corresponding neighborhood is identified and analyzed, and the brightness weight of each fused pixel is analyzed; For each fused pixel, a weighted analysis is performed based on the apparent temperature pixel value, temperature weight, reflected light brightness pixel value, and brightness weight to obtain the corresponding fused pixel value.

[0012] Furthermore, the specific steps for damage identification of multi-source fusion images based on the improved YOLOv5 damage detection algorithm are as follows: The multi-source fusion image of the tower crane boom to be inspected is scaled and pixel normalized; the processed multi-source fusion image is input into the improved YOLOv5 model, which outputs candidate boxes, categories, and confidence scores; non-maximum suppression or weighted non-maximum suppression is performed on the candidate boxes, and they are filtered according to the confidence threshold and the intersection-union ratio threshold to obtain the set of damage targets.

[0013] Furthermore, improvements to YOLOv5 include: introducing a channel attention module into the backbone network to enhance fine-grained feature representation; performing multi-scale feature fusion in the feature pyramid and path aggregation structure, and setting learnable weights for the fusion branch to suppress invalid scales; employing a multi-scale detection head with decoupled classification and regression branches; adaptively updating anchor box size and aspect ratio based on sample statistics; and using weighted non-maximum suppression to deduplicate and fuse overlapping candidate boxes in the output stage.

[0014] Furthermore, the generation of the inspection report includes: mapping the image coordinates of the damaged target to the structural coordinates of the tower crane boom; outputting the location, category, confidence level, size and timestamp of each damaged target in a preset format, and associating it with the corresponding inspection point identifier; classifying the severity according to threshold rules and marking retest points, generating an inspection report containing a list of results and suggestions for re-inspection tasks, and archiving it.

[0015] Based on the same inventive concept, a self-inspection infrared drone inspection device is also proposed, applied to the aforementioned self-inspection infrared drone inspection method. The device includes: a path planning module for acquiring structural information of the tower crane boom to be inspected and analyzing and generating an inspection path containing several inspection points; a flight execution module for inputting the inspection path into the infrared drone for flight inspection; an image fusion module for acquiring infrared thermal imaging images and visible light images of the corresponding tower crane boom when the infrared drone reaches each inspection point, and performing multi-source fusion to obtain the corresponding multi-source fused image; and a damage identification module for identifying damage in the multi-source fused image based on an improved YOLOv5 damage detection algorithm and outputting an inspection report.

[0016] The present invention has the following beneficial effects:

[0017] (1) The self-inspection infrared UAV inspection method obtains the structural information of the tower crane boom, takes the coordinates of key parts and main components as inspection points, and generates the optimal inspection path on this basis. Unlike the traditional manual setting or fixed waypoint method, this path planning ensures that high-risk areas such as connection points, corners and stiffeners are covered, fundamentally avoiding the possibility of missed inspections. At the same time, when generating the path, the actual constraints such as flight power, no-fly zones and safety gaps are comprehensively considered, making the flight trajectory more reasonable and feasible. After deduplication, merging and path optimization, the UAV inspection no longer has invalid repeated coverage, reduces unnecessary maneuvering operations, and significantly shortens the operation time. This not only improves the inspection efficiency, but also reduces flight energy consumption and personnel operation costs.

[0018] (2) The self-inspection infrared UAV inspection method collects infrared thermal imaging images and visible light images at the inspection point, performs geometric registration and pixel-level alignment of the two types of data, and then fuses them into a unified image through weight allocation. This multi-source fusion method makes temperature anomaly information and surface texture information complement each other, solving the problem that a single infrared image is easily affected by ambient temperature and a single visible light image is easily affected by illumination and shadow interference. On this basis, the improved YOLOv5 detection algorithm is introduced, which uses channel attention to enhance detailed features, multi-scale fusion to enhance the recognition ability of small targets, and combined with decoupled detection head and weighted nonmaximum suppression, further improves the recognition accuracy of fine cracks and defects in complex backgrounds. The final result shows that the false alarm rate and false negative rate are significantly reduced.

[0019] (3) After the inspection is completed, the system will map the image coordinates of the identified damaged target to the actual tower crane boom structure coordinates and bind them with the inspection point identifier and timestamp to form standardized data entries. This information is integrated into the inspection report. The report not only includes the location, type, size and confidence level of the damage, but also classifies the severity of the damage according to preset rules and marks the parts that need to be retested. The generated report can directly guide the subsequent maintenance and re-inspection work, reduce the ambiguity of manual secondary interpretation, and at the same time, the historical report data can be archived and tracked, which is convenient for long-term trend analysis of the tower crane boom operation status and maintenance plan formulation.

[0020] (4) The self-inspection infrared UAV inspection device integrates four modules—path planning, flight execution, image fusion, and damage identification—into a single system. The modules interact directly through standardized interfaces, forming a fully automated link from data acquisition to report output. Compared with the existing method of using multiple devices and manual analysis, this device eliminates the tedious manual operations in path design, data alignment, and result summarization, significantly lowering the barrier to entry. Users only need to input the structural information of the tower crane boom to be inspected to automatically complete the inspection path generation, UAV flight control, image acquisition and fusion, damage identification, and report output, reducing the uncertainty in the operation process and the errors caused by manual intervention.

[0021] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0022] Figure 1 This is a flowchart of a self-inspection infrared drone inspection method according to the present invention.

[0023] Figure 2 This is a flowchart illustrating the specific steps involved in obtaining the corresponding multi-source fusion image in a self-inspection infrared drone inspection method according to the present invention.

[0024] Figure 3 This is a block diagram of a self-inspection infrared drone inspection device according to the present invention. Detailed Implementation

[0025] Please see Figure 1 This invention provides a self-inspection infrared drone inspection method, comprising the following steps: acquiring structural information of the tower crane boom to be inspected and analyzing and generating an inspection path containing several inspection points; inputting the inspection path into the infrared drone for flight inspection; when the infrared drone reaches each inspection point, acquiring infrared thermal imaging images and visible light images of the corresponding tower crane boom to be inspected, and performing multi-source fusion to obtain the corresponding multi-source fused image; performing damage identification on the multi-source fused image based on the improved YOLOv5 damage detection algorithm, and outputting an inspection report.

[0026] Specifically, the structural information includes the three-dimensional coordinates of each component (such as boom, connecting section, stiffener, etc.) and the three-dimensional coordinates of each key part (such as connection point, support node, corner point, etc.).

[0027] The specific steps for generating an inspection path containing several inspection points are as follows: The three-dimensional coordinates of each key component of the tower crane boom to be inspected are set as inspection points. Supplementary inspection points are generated according to preset rules based on the three-dimensional coordinates of each component. Specifically: the three-dimensional coordinate sets of each key component and the three-dimensional coordinate sequences of each component are read; the coordinate system and units are unified; duplicate and abnormal coordinate points are eliminated; the three-dimensional coordinates of each key component are directly recorded as mandatory inspection points, and a "high priority" identifier and target area label (such as connection points, corner points) are assigned to them. For each mandatory inspection point, a set of feasible shooting poses (including UAV position and camera attitude) is calculated according to preset imaging parameters (field of view, target resolution, working distance) and safety clearance. Occluded poses are eliminated. At least one pose is selected from the feasible set as the default shooting pose for that inspection point.

[0028] The rules for generating supplementary points include:

[0029] Equidistant point selection: Inspection points are generated along the center line of the component at a set step distance, and the ends must be selected.

[0030] Corner reinforcement: Set up reinforcement zones on both sides of the corner of the component and add inspection points in smaller increments.

[0031] Joint / weld coverage: Inspection points are generated at fixed intervals along the length of the connection section or weld.

[0032] Resolution calibration: When the ground pixel resolution of the target area is not up to standard at the default working distance, the camera will automatically insert supplementary points or adjust the shooting pose and distance.

[0033] Safety constraints: The minimum distance from all inspection points to the component surface shall not be less than the safety clearance threshold.

[0034] Deduplication and merging: Calculate the visual overlap of adjacent candidate inspection points, merge those exceeding a set threshold into a single inspection point, and retain the better view pose.

[0035] Output: A set of inspection points is generated, including the 3D coordinates, default shooting pose, target area label and priority of each inspection point, which is used for subsequent path planning.

[0036] Read the 3D coordinates of all inspection points. Under preset constraints, use a path optimization algorithm to determine the access order of the inspection points and generate a connecting trajectory to form the inspection path, which is as follows:

[0037] Constraint modeling: Set constraints such as battery power and maximum range, maximum mission duration, no-fly zones and altitude-restricted zones, minimum / maximum flight altitude, safety clearance, maximum turn angle / rate of climb, and environmental thresholds (wind speed, rainfall).

[0038] Graph modeling: Construct a connected graph within the flyable space with all inspection points as necessary nodes; the edge weights of the graph are defined by comprehensive costs (flight distance, energy consumption, wind direction penalty, turning cost); edges that cross no-fly zones or do not meet safety clearance requirements are removed or added as obstacle avoidance relay points.

[0039] Access order optimization: The problem is modeled as a constrained traveling salesman / vehicle routing problem. A heuristic solution process is adopted (nearest neighbor initialization + two / three swap local optimization or genetic / tacit search iteration). When the battery and duration thresholds are not met, the trip is automatically split into multiple flights and a return / charging / battery swapping node is inserted.

[0040] Segment generation: Based on the optimized access order, use 3D path search (such as A*, RRT* or PRM) to generate collision-free segments for adjacent inspection points within the flyable space.

[0041] Trajectory smoothing and executability: Perform B-spline or multi-segment polynomial smoothing on discrete flight segments, and apply velocity, acceleration, angular velocity and curvature boundaries; configure hovering or deceleration passage time at each inspection point to meet imaging stability.

[0042] Verification and Output: Verify whether the coverage (target area imaging resolution and viewing angle requirements), safety gaps, and no-fly avoidance meet the standards; if not, return to the step access sequence optimization / segment generation replanning; if compliant, output the inspection path file (including waypoint sequence, attitude, speed, and timestamp) and task list for execution.

[0043] In this implementation plan, the scientific layout of inspection points and intelligent optimization of paths are achieved through the precise acquisition and processing of tower crane boom structure information. This avoids blind spots and redundant coverage caused by traditional manual experience-based point layout. By ensuring the inspection of key parts, generating supplementary component points, eliminating obstructions, verifying resolution, and constraining safety gaps, the comprehensiveness and safety of inspections are guaranteed. At the same time, various practical constraints such as power consumption, no-fly zones, maximum range, and flight altitude are introduced in the path planning stage. Combined with the optimization solution of the traveling salesman problem and 3D path search, the inspection path is ensured to be not only feasible but also efficient. The trajectory smoothing and executable processing further improve the stability of UAV flight and imaging quality. The final verification and output stage enables the path plan to have self-checking and error correction capabilities. This entire mechanism significantly improves the inspection coverage and reliability, reduces the risk of missed inspections and flight risks, and shortens the operation time and the need for manual intervention.

[0044] Specifically, an infrared thermal imaging image includes several infrared pixels, and each infrared pixel corresponds to an apparent temperature pixel value; a visible light image includes several visible light pixels, and each visible light pixel corresponds to a reflected light brightness pixel value; and a multi-source fusion image includes several fusion pixels, and each fusion pixel corresponds to a fusion pixel value.

[0045] like Figure 2 As shown, the specific steps to obtain the corresponding multi-source fusion image are as follows: Geometrically register the infrared thermal imaging image and the visible light image of the tower crane boom to be tested, so that the pixels of the two correspond one-to-one under the same pixel grid; On the common pixel grid, taking the fused pixel as the unit, read the apparent temperature pixel value and the reflected light brightness pixel value corresponding to the fused pixel, and analyze the fused pixel value to form a multi-source fusion image.

[0046] In this implementation scheme, pixel-level registration and fusion of infrared thermal imaging images and visible light images ensures point-to-point correspondence between the two types of images on the same grid, enabling different information sources to complement each other. Infrared images reflect the temperature distribution on the surface of components, while visible light images present the texture and shape characteristics of components. The combination of the two forms a fused pixel value at the same pixel point, which not only retains the abnormal sensitivity of thermal features but also incorporates the intuitive expression of visual details. This multi-source fusion avoids the defects of single image analysis that are easily affected by lighting, ambient temperature, etc., making defect identification and location more reliable. At the same time, the geometric registration and fusion steps are executed automatically, reducing the workload of manual alignment and interpretation, improving the efficiency and consistency of data processing, and finally generating a multi-source fused image with higher information density and interpretability.

[0047] Specifically, the steps for analyzing the fused pixel value of each fused pixel are as follows: For each infrared pixel, based on a set neighborhood range, the mean and standard deviation of the apparent temperature pixels in the neighborhood are identified and analyzed, and the temperature weight of each fused pixel is analyzed; For each visible light pixel, the corresponding brightness gradient is analyzed, and based on a set neighborhood range, the maximum brightness gradient in the neighborhood is identified and analyzed, and the brightness weight of each fused pixel is analyzed; For each fused pixel, a weighted analysis is performed based on the apparent temperature pixel value, temperature weight, reflected light brightness pixel value, and brightness weight to obtain the corresponding fused pixel value.

[0048] The specific formulas for calculating the temperature weight and brightness weight of a certain fused pixel are as follows:

[0049] ;

[0050] in, For the temperature weight of a certain fused pixel, The apparent temperature pixel value corresponding to a certain fused pixel. The average apparent temperature of the neighboring pixels corresponding to a given fused pixel. Let be the standard deviation of the apparent temperature pixels in the neighborhood corresponding to a given fused pixel. This is a temperature adjustment factor stored in the database to prevent the denominator from being zero. For the brightness weight of a certain fused pixel, For a given merged pixel, the brightness gradient is... The maximum brightness gradient in the neighborhood of a given fused pixel. This is a brightness adjustment factor stored in the database to prevent the denominator from being zero.

[0051] In this implementation scheme, dynamic calculation of weights achieves a more scientific and reliable fusion. For infrared data, the analysis of neighborhood mean and standard deviation is introduced to highlight the difference between outliers and the overall temperature distribution, thereby enhancing the salience of abnormal areas. For visible light data, the comparison of brightness gradient and neighborhood maximum value can better capture edge and texture features, making small damages such as cracks and corrosion stand out. Through weight normalization, the two types of data achieve complementary advantages in the same fused pixel, retaining the sensitivity of infrared in thermal anomaly detection and combining the intuitiveness of visible light in shape recognition. At the same time, the adjustment factor introduced in the formula can avoid calculation problems such as zero denominator, improving the stability of the algorithm in real complex scenarios. Overall, this step makes the information density of the fused image higher, the target features clearer, and effectively reduces false detections and false negatives caused by environmental interference.

[0052] Specifically, the steps for damage identification of multi-source fused images based on the improved YOLOv5 damage detection algorithm are as follows: First, the multi-source fused image of the tower crane boom to be inspected is scaled and pixel normalized. This involves scaling the fused image according to the network input size and adding necessary padding, recording the scaling factor and boundary offset for result back-mapping; second, pixel values ​​are normalized and channel order is adjusted, maintaining a one-to-one correspondence with the corresponding inspection point markers; third, the processed multi-source fused image is input into the improved YOLOv5 model, outputting candidate boxes, categories, and confidence scores. This involves loading the weights and category set of the improved YOLOv5 model, setting the inference threshold, and so on. The preprocessed fused image is input to obtain the original detection results containing candidate box coordinates, damage category labels, and confidence scores. The candidate box coordinates are then restored to the original image scale according to the recorded scaling / offset parameters. Non-maximum suppression or weighted non-maximum suppression is performed on the candidate boxes, and they are filtered according to the confidence score threshold and the intersection-union ratio (IU / R) threshold to obtain the damage target set. Specifically, candidate boxes of the same category are sorted by confidence score and non-maximum suppression or weighted non-maximum suppression is performed on each one. Candidate boxes with an IU / R or confidence score below the threshold are removed, and the deduplicated damage target set is output. For each target, the image coordinates, category, confidence score information, and corresponding inspection point identifier are retained.

[0053] In this implementation scheme, scale unification and pixel normalization ensure consistency of fused images from different sources and of different sizes when entering the model, avoiding target detection bias caused by scale inconsistencies. Simultaneously, scaling and boundary information are recorded to ensure accurate back-mapping of detection results to the original image. Secondly, the preprocessed image is input into the improved YOLOv5 model. With the help of trained weights and class sets, candidate boxes, classes, and confidence scores can be quickly output, significantly improving recognition efficiency. Finally, non-maximum suppression or weighted non-maximum suppression is introduced in the candidate box output stage. Through dual filtering using confidence scores and intersection-union ratio (IU / U), redundant boxes and low-quality detection results are effectively eliminated. The final target set not only removes duplicate detections but also balances detection coverage and accuracy.

[0054] Specifically, improvements to YOLOv5 include: introducing a channel attention module into the backbone network to enhance fine-grained feature representation; specifically, setting channel recalibration units after feature maps at each stage of the backbone to adjust the weights of each channel response to highlight channels related to subtle damage and suppress irrelevant channels; performing multi-scale feature fusion in the feature pyramid and path aggregation structure, and setting learnable weights for the fusion branches to suppress invalid scales; specifically, employing top-down and bottom-up bidirectional fusion to weight and merge features of different scales according to branch weights, with the branch weights automatically learned during training to reduce the impact of invalid or redundant scales; and using a multi-scale detection head with decoupled classification and regression branches. Specifically, in each scale of the detection head, a classification branch for category and confidence prediction and a regression branch for target location and size prediction are set up independently to reduce task interference. The anchor frame size and aspect ratio are adaptively updated based on sample statistics. Specifically, the size parameters and aspect ratio parameters of the anchor frame are statistically updated according to the size and aspect ratio distribution of the target in the training samples to improve the matching degree between the anchor frame and the target. In the output stage, weighted nonmaximum suppression is used to deduplicate and fuse overlapping candidate frames. Specifically, candidate frames of the same category and overlapping with each other are weighted and merged according to confidence to obtain a single output frame, thereby reducing the interference between multiple frames of the same defect and adjacent defects.

[0055] In this implementation, a channel attention module is added to the backbone network, enabling the model to automatically enhance feature channels related to defects such as fine cracks and corrosion, and suppress background noise channels, thereby better distinguishing effective information from interference information. Secondly, learnable weights are introduced into the feature pyramid and path aggregation to ensure that the model can highlight truly effective scale information when fusing multi-scale features, reducing the interference of redundant and invalid features on the results. Furthermore, the decoupling design of the classification and regression branches avoids the conflict between the target category and the location regression task, making the model more robust in both category recognition and location localization. The adaptive update of the anchor frame size and aspect ratio ensures that the model can better fit the real distribution of tower crane boom defect targets, improving the consistency between the predicted box and the target. Finally, weighted nonmaximum suppression can reasonably fuse overlapping candidate boxes, reducing the mutual interference between multi-box detection and adjacent targets.

[0056] Specifically, the generation of the inspection report includes: mapping the image coordinates of the damaged target to the structural coordinates of the tower crane boom. Specifically, based on the aforementioned geometric registration relationship and the three-dimensional coordinates / default shooting pose of the corresponding inspection point, the image coordinates of the detection frame are projected back to the structural coordinate system of the tower crane boom to obtain the position description of each damaged target in the structural coordinate system; outputting the position, category, confidence level, size, and timestamp of each damaged target in a preset format, and associating it with the corresponding inspection point identifier. Specifically, generating a record entry for each damaged target, including structural coordinate position, damage category, confidence level, frame size, acquisition timestamp, and inspection point identifier, and organizing it into a tabular list according to task number; classifying the severity according to threshold rules and marking retest points, generating an inspection report containing a result list and re-inspection task suggestions, and archiving it. Specifically, classifying the severity of the target and determining the treatment suggestions according to preset thresholds, marking the targets that need to be retested and generating re-inspection task entries; summarizing to form an inspection report, including task information, result list, and re-inspection suggestions, and completing archiving and tracking association.

[0057] In this implementation plan, by projecting the image coordinates of the damaged target back onto the structural coordinate system of the tower crane boom, the spatial location of the defect on the actual component can be accurately described, solving the problem that it is difficult to match the results of simple image detection on site. Secondly, each defect is output with its location, category, confidence level, size, and timestamp in a unified format and associated with the corresponding inspection point identifier to form a standardized data entry. This not only facilitates subsequent retrieval and comparison but also avoids data chaos caused by inconsistencies in manual records. Thirdly, the introduction of severity grading and retest marking mechanisms enables the inspection results to be directly converted into operation and maintenance suggestions, helping managers to quickly distinguish between minor defects and issues that require key tracking. Finally, through the summarization, archiving, and tracking of reports, a complete inspection archive is established, supporting long-term trend analysis and multiple retest comparisons.

[0058] Please see Figure 3 Based on the same inventive concept, this invention provides a self-inspection infrared drone inspection device, applied to the aforementioned self-inspection infrared drone inspection method, comprising: a path planning module for acquiring structural information of the tower crane boom to be inspected and analyzing and generating an inspection path containing several inspection points; a flight execution module for inputting the inspection path into the infrared drone for flight inspection; an image fusion module for acquiring infrared thermal imaging images and visible light images of the corresponding tower crane boom when the infrared drone reaches each inspection point, and performing multi-source fusion to obtain a corresponding multi-source fused image; and a damage identification module for identifying damage in the multi-source fused image based on an improved YOLOv5 damage detection algorithm and outputting an inspection report.

[0059] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0060] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A self-inspection infrared drone inspection method, characterized in that, Includes the following steps: Obtain the structural information of the tower crane boom to be inspected, and analyze and generate an inspection path containing several inspection points; Input the inspection path into the infrared drone for flight inspection; When the infrared drone arrives at each inspection point, it collects infrared thermal imaging images and visible light images of the corresponding tower crane boom to be inspected, and performs multi-source fusion to obtain the corresponding multi-source fused image. Damage identification is performed on multi-source fused images based on the improved YOLOv5 damage detection algorithm, and an inspection report is output. Infrared thermal imaging images include several infrared pixels, and each infrared pixel corresponds to an apparent temperature pixel value. Visible light images include several visible light pixels, and each visible light pixel corresponds to a reflected light brightness pixel value. Multi-source fusion images include several fusion pixels, and each fusion pixel corresponds to a fusion pixel value. The specific steps to obtain the corresponding multi-source fused image are as follows: The infrared thermal imaging image of the tower crane boom to be inspected is geometrically registered with the visible light image so that the pixels of the two correspond one-to-one under the same pixel grid. On the common pixel grid, taking the fused pixel as the unit, the apparent temperature pixel value and the reflected light brightness pixel value corresponding to the fused pixel are read respectively, and the fused pixel value is analyzed to form a multi-source fused image; The specific steps for analyzing the merged pixel value of each merged pixel are as follows: For each infrared pixel, based on a set neighborhood range, the mean value of the apparent temperature pixels in the neighborhood and the standard deviation of the apparent temperature pixels in the neighborhood are identified and analyzed, and the temperature weight of each fused pixel is analyzed. For each visible light pixel, the corresponding brightness gradient is analyzed, and based on the set neighborhood range, the maximum brightness gradient in the neighborhood is identified and analyzed, and the brightness weight of each fused pixel is analyzed. For each fused pixel, a weighted analysis is performed based on the apparent temperature pixel value, temperature weight, reflected light brightness pixel value, and brightness weight to obtain the corresponding fused pixel value. The specific formulas for calculating the temperature weight and brightness weight of a certain fused pixel are as follows: ； in, , The weights are, in order, temperature and brightness weights for a given pixel in the fusion process. , , , , The values ​​are, in order, the apparent temperature pixel value corresponding to a certain fused pixel, the mean apparent temperature pixel value of its neighbors, the standard deviation of the apparent temperature pixel value of its neighbors, the brightness gradient, and the maximum brightness gradient of its neighbors. , These are, in order, the temperature adjustment factor and the brightness adjustment factor stored in the database.

2. The self-inspection infrared UAV inspection method according to claim 1, characterized in that, The structural information includes the three-dimensional coordinates of each component and the three-dimensional coordinates of each key part.

3. The self-inspection infrared UAV inspection method according to claim 2, characterized in that, The specific steps for generating an inspection path that includes several inspection points are as follows: The three-dimensional coordinates of each key part of the tower crane boom to be inspected are set as inspection points, and supplementary inspection points are generated according to the three-dimensional coordinates of each component and preset rules. Read the three-dimensional coordinates of all inspection points, and under preset constraints, use a path optimization algorithm to determine the access order of the inspection points and generate a connecting trajectory to form an inspection path.

4. The self-inspection infrared UAV inspection method according to claim 1, characterized in that, The specific steps for damage identification of multi-source fused images based on the improved YOLOv5 damage detection algorithm are as follows: Scale unification and pixel normalization are performed on the multi-source fusion images of the tower crane boom to be inspected; The processed multi-source fused image is input into the improved YOLOv5 model, which outputs candidate boxes, categories, and confidence scores. The candidate boxes are subjected to non-maximum suppression or weighted non-maximum suppression, and then filtered according to the confidence threshold and the intersection-union threshold to obtain the set of damaged targets.

5. The self-inspection infrared UAV inspection method according to claim 4, characterized in that, Improvements to YOLOv5 include: A channel attention module is introduced into the backbone network to enhance fine-grained feature representation; Multi-scale feature fusion is performed in the feature pyramid and path aggregation structure, and learnable weights are set for the fusion branches to suppress invalid scales. A multi-scale detection head with decoupled classification and regression branches is used; The anchor frame dimensions and aspect ratio are adaptively updated based on sample statistics. In the output stage, weighted nonmaximum suppression is used to deduplicatize and fuse overlapping candidate boxes.

6. The self-inspection infrared UAV inspection method according to claim 5, characterized in that, The generation of the inspection report includes: Map the image coordinates of the damaged target to the structural coordinates of the tower crane boom; Output the location, category, confidence level, size and timestamp of each damaged target in a preset format, and associate it with the corresponding inspection point identifier; Severity is graded according to threshold rules and retest points are marked. An inspection report containing a list of results and suggestions for re-inspection tasks is generated and archived.

7. A self-inspection infrared drone inspection device, employing the self-inspection infrared drone inspection method according to any one of claims 1-6, characterized in that, include: The path planning module is used to obtain the structural information of the tower crane boom to be inspected and analyze and generate an inspection path containing several inspection points. The flight execution module is used to input the inspection path into the infrared UAV for flight inspection; The image fusion module is used to collect infrared thermal imaging images and visible light images of the corresponding tower crane boom to be inspected when the infrared UAV arrives at each inspection point, and perform multi-source fusion to obtain the corresponding multi-source fused image. The damage identification module is used to identify damage in multi-source fused images based on the improved YOLOv5 damage detection algorithm and output an inspection report.

Images