Inspection method and device of self-inspection infrared unmanned aerial vehicle

Abstract

The invention discloses a self-inspection infrared unmanned aerial vehicle inspection method and device, and relates to the technical field of unmanned aerial vehicle damage detection. The self-inspection infrared unmanned aerial vehicle inspection method comprises the following steps: acquiring structural information of a to-be-detected tower crane boom, and analyzing to generate an inspection path comprising a plurality of inspection points; inputting the inspection path into the infrared unmanned aerial vehicle for flight inspection; when the infrared unmanned aerial vehicle arrives at each inspection point, respectively acquiring an infrared thermal imaging image and a visible light image at the boom of the corresponding to-be-detected tower crane, and carrying out multi-source fusion to obtain a corresponding multi-source fusion image; the multi-source fusion image is subjected to damage identification based on an improved YOLOv5 damage detection algorithm, and an inspection report is output. Structural information of the tower crane boom is obtained, coordinates of key parts and main components serve as inspection points, and an optimal inspection path is generated; the path planning ensures that high-risk areas such as connecting points, corners, stiffeners and the like are certainly covered, and the possibility of missing detection is fundamentally avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle damage detection, in particular to a self-inspection infrared unmanned aerial vehicle inspection method and device. BACKGROUND

[0002] With the continuous expansion of modern engineering construction scale, large tower cranes are increasingly used in high-rise building construction, bridge construction and other scenarios. As a key bearing and force transmission component, the operation safety of the tower crane arm directly relates to the safety and efficiency of the construction site. Traditional tower crane inspection relies on manual methods, usually through the use of tools such as telescopes and cameras by operators to observe the tower crane structure from the ground or at close range. This method is not only limited by factors such as personnel perspective and environmental lighting, but also has low inspection efficiency, making it difficult to meet the detection needs of high frequency and large range.

[0003] The limitations of the prior art include at least the following problems. The inspection path of the existing tower crane arm inspection is usually based on fixed waypoints or experience templates, and does not arrange must-visit inspection points for high-risk parts such as connection points and corners based on the structural information of the specific arm, resulting in structural blind spots and structural missing inspections. On the other hand, infrared thermal images and visible light images are usually obtained asynchronously and interpreted separately, lacking pixel-level alignment and fusion, making it difficult to verify temperature anomalies and surface textures, and easily being overwhelmed by background heat, reflection or shadow interference, which magnifies the probability of missing detection from the perception level. At the same time, the conventional target detection is configured on a single source image, which is insufficient for small-scale and slender defects in terms of separability, and has low positioning drift and confidence. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides a self-inspection infrared unmanned aerial vehicle inspection method and device, which solves the problems of high missing inspection rate, high false detection rate and inaccurate defect positioning in the prior art.

[0005] To achieve the above purpose, the present application realizes the following technical solutions: A self-inspection infrared unmanned aerial vehicle inspection method, comprising the following steps: obtaining the structural information of a tower crane arm to be detected, and analyzing and generating an inspection path containing a plurality of inspection points; inputting the inspection path into an infrared unmanned aerial vehicle for flight inspection; when the infrared unmanned aerial vehicle reaches each inspection point, respectively collecting infrared thermal images and visible light images corresponding to the tower crane arm to be detected, and performing multi-source fusion to obtain corresponding multi-source fusion images; performing damage identification on the multi-source fusion images based on an improved YOLOv5 damage detection algorithm, and outputting an inspection report.

[0006] Further, the structural information includes the three-dimensional coordinates of each component and the three-dimensional coordinates of each key part.

[0007] Further, the specific steps of generating the inspection path including a plurality of inspection points are as follows: the three-dimensional coordinates of each key part of the tower crane jib to be detected are set as inspection points respectively, and supplementary inspection points are generated according to the three-dimensional coordinates of each component according to a preset rule; the three-dimensional coordinates of all the inspection points are read, the visiting order of the inspection points is determined and the connected track is generated under the preset constraint condition by using a path optimization algorithm, and the inspection path is formed.

[0008] Further, the infrared thermal imaging image includes a plurality of infrared pixel points, and each infrared pixel point corresponds to an apparent temperature pixel value; the visible light image includes a plurality of visible light pixel points, and each visible light pixel point corresponds to a reflected light brightness pixel value; and the multi-source fusion image includes a plurality of fusion pixel points, and each fusion pixel point corresponds to a fusion pixel value.

[0009] Further, the specific steps of obtaining the corresponding multi-source fusion image are as follows: the infrared thermal imaging image and the visible light image of the tower crane jib to be detected are geometrically registered so that the pixel points correspond to each other under the same pixel grid; the apparent temperature pixel value and the reflected light brightness pixel value corresponding to the fusion pixel point are read respectively on the common pixel grid in units of the fusion pixel point, and the fusion pixel value is analyzed to form the multi-source fusion image.

[0010] Further, the specific steps of analyzing the fusion pixel value of each fusion pixel point are as follows: for each infrared pixel point, the corresponding neighborhood apparent temperature pixel mean value, neighborhood apparent temperature pixel standard deviation are identified and analyzed based on a set neighborhood range, and the temperature weight of each fusion pixel point is analyzed; for each visible light pixel point, the corresponding brightness gradient is analyzed, and the corresponding neighborhood maximum brightness gradient is identified and analyzed based on a set neighborhood range, and the brightness weight of each fusion pixel point is analyzed; for each fusion pixel point, the corresponding fusion pixel value is obtained by weighted analysis based on the apparent temperature pixel value, the temperature weight, the reflected light brightness pixel value and the brightness weight.

[0011] Further, the specific steps of identifying damage of the multi-source fusion image based on the improved YOLOv5 damage detection algorithm are as follows: the multi-source fusion image of the tower crane jib to be detected is processed by scale unification and pixel normalization; the processed multi-source fusion image is input into the improved YOLOv5 model to output the candidate box, the category and the confidence; the non-maximum suppression or weighted non-maximum suppression is performed on the candidate box, and the damage target set is obtained according to the confidence threshold and the intersection over union threshold.

[0012] Further, the improved YOLOv5 includes: introducing a channel attention module in the backbone network to enhance fine-grained feature expression; multi-scale feature fusion is performed in the feature pyramid and the path aggregation structure, and a learnable weight is set for the fusion branch to suppress invalid scales; a multi-scale detection head decoupled by the classification branch and the regression branch is adopted; the anchor box size and the aspect ratio are adaptively updated based on sample statistics; and in the output stage, a weighted non-maximum suppression is used to remove and fuse overlapping candidate boxes.

[0013] Further, the generation of the inspection report includes: mapping the image coordinates of the damage target to the structural coordinates of the tower crane jib; outputting the position, category, confidence, size and timestamp of each damage target in a preset format, and associating the corresponding inspection point identifier; performing severity grading according to threshold rules and marking the re-inspection points, generating an inspection report containing a result list and a re-inspection task suggestion and archiving.

[0014] Based on the same inventive concept, a self-inspection infrared unmanned aerial vehicle inspection device is also proposed, which is applied to the above-mentioned self-inspection infrared unmanned aerial vehicle inspection method, and includes: a path planning module for obtaining structural information of a tower crane jib to be detected and analyzing and generating an inspection path containing a plurality of inspection points; a flight execution module for inputting the inspection path into the infrared unmanned aerial vehicle for flight inspection; an image fusion module for respectively collecting infrared thermal imaging images and visible light images of the tower crane jib to be detected at each inspection point when the infrared unmanned aerial vehicle arrives at each inspection point, and performing multi-source fusion to obtain corresponding multi-source fusion images; and a damage identification module for identifying damages in the multi-source fusion images based on the improved YOLOv5 damage detection algorithm and outputting an inspection report.

[0015] The present application has the following beneficial effects: (1) The self-inspection infrared unmanned aerial vehicle inspection method, by obtaining the structural information of the tower crane jib, taking the coordinates of the key parts and main components as the inspection points, and generating the optimal inspection path on this basis, unlike the traditional manual point setting or fixed waypoint method, this path planning ensures that the high-risk areas such as connection points, corners and stiffeners are definitely covered, fundamentally avoiding the possibility of missed inspection, and at the same time, when generating the path, the actual constraints such as flight power, no-fly zones and safety gaps are considered, making the flight trajectory more reasonable and feasible, after de-duplication, merging and path optimization, the unmanned aerial vehicle inspection no longer appears invalid and repeated coverage, reducing unnecessary maneuvering operation, significantly shortening the operation time, which not only improves the inspection efficiency, but also reduces the flight energy consumption and personnel operation cost.

[0016] (2) The self-inspection infrared unmanned aerial vehicle inspection method, after collecting infrared thermal imaging images and visible light images at the inspection points, performs geometric registration and pixel-level alignment on the two types of data, and then fuses them into a unified image through weight distribution. This multi-source fusion method makes the temperature anomaly information and the surface texture information complementary to each other, solves the problems that a single infrared image is easily affected by the environment temperature and a single visible light image is easily affected by light and shadow interference. On this basis, the improved YOLOv5 detection algorithm is introduced, the channel attention is used to strengthen the detail features, the multi-scale fusion is used to enhance the small target recognition ability, the decoupled detection head and the weighted non-maximum suppression are combined, and the recognition accuracy of defects in small cracks and complex backgrounds is further improved. Finally, the false positive rate and the false negative rate are obviously reduced.

[0017] (3) The self-inspection infrared unmanned aerial vehicle inspection method, after the inspection is completed, the system maps the identified damage target image coordinates to the actual tower crane boom structure coordinates, binds them with the inspection point identifier and the time stamp, and forms standardized data entries. These information is integrated into the inspection report. The report not only contains the damage position, category, size and confidence, but also classifies the damage severity according to the preset rules and marks the parts that need to be retested. The generated report can directly guide the subsequent maintenance and retest work, reducing the ambiguity of manual secondary interpretation. At the same time, the historical report data can be archived and tracked, which is convenient for long-term trend analysis and maintenance plan development of the tower crane boom operation state.

[0018] (4) The self-inspection infrared unmanned aerial vehicle inspection device integrates the path planning, flight execution, image fusion and damage identification modules into an integrated system. The modules interact directly through standardized interfaces, forming a full-process automation link from data acquisition to report output. Compared with the existing scattered use of multiple devices and manual auxiliary analysis, the device eliminates the tedious operations of manual path design, data alignment and result summarization, significantly reduces the use threshold, and users only need to input the structure information of the tower crane boom to be detected. The device can automatically complete the inspection path generation, unmanned aerial vehicle flight control, image acquisition and fusion, damage identification and report output, reducing the uncertainty and errors caused by manual intervention in the operation link.

[0019] Of course, implementing any product of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A self-inspection infrared unmanned aerial vehicle inspection method flow chart.

[0021] Figure 2 A specific step flow chart for obtaining corresponding multi-source fusion images in the self-inspection infrared unmanned aerial vehicle inspection method.

[0022] Figure 3 A self-inspection infrared unmanned aerial vehicle inspection device block diagram of the present application. DETAILED DESCRIPTION

[0023] Please refer to Figure 1 The embodiment of the present application provides a kind of self-inspection infrared unmanned aerial vehicle inspection method, comprising the following steps: obtaining the structure information of the tower crane arm frame to be detected, and analysis generates the inspection path containing several inspection points;The inspection path is input to the infrared unmanned aerial vehicle to carry out flight inspection;When each inspection point is reached by the infrared unmanned aerial vehicle, the infrared thermal imaging image and the visible light image corresponding to the tower crane arm frame to be detected are collected respectively, and multi-source fusion is carried out to obtain the corresponding multi-source fusion image;Damage identification is carried out on the multi-source fusion image based on improved YOLOv5 damage detection algorithm, and inspection report is output.

[0024] Specifically, the structure information includes the three-dimensional coordinates of each component (such as arm frame, connecting section, stiffener, etc.), and the three-dimensional coordinates of each key position (such as connecting point, support node, corner point, etc.).

[0025] The specific steps of generating the inspection path containing several inspection points are as follows: the three-dimensional coordinates of each key position of the tower crane arm frame to be detected are set as inspection points respectively, and supplementary inspection points are generated according to the three-dimensional coordinates of each component according to a preset rule, which is specifically: reading the three-dimensional coordinate set of each key position and the three-dimensional coordinate sequence of each component, unifying the coordinate system and unit, eliminating repeated and abnormal coordinate points, directly recording the three-dimensional coordinates of each key position as mandatory inspection points, and giving them "high priority" mark and target area label (such as connecting point, corner point), for each mandatory inspection point, according to the preset imaging parameters (field of view, target resolution, working distance) and safety gap, a set of feasible shooting poses (including unmanned aerial vehicle position and camera attitude) is solved;The poses with occlusion are eliminated;At least one of the feasible set is selected as the default shooting pose of the inspection point.

[0026] The supplementary point generation rule includes: Equidistant point taking: generate inspection points along the center line of the component at a set step distance, and the end part must be taken.

[0027] Corner encryption: set an encryption interval on both sides of the component corner, and add inspection points at a smaller step distance.

[0028] Joint / weld coverage: generate inspection points at fixed intervals along the length direction of the connecting section or weld.

[0029] Resolution checking: when the ground pixel resolution of the target area by the camera at the default working distance is not up to standard, automatically insert front and rear supplementary points or adjust the shooting pose distance.

[0030] Safety constraint: The minimum distance between all inspection points and the component surface must not be less than the safety gap threshold.

[0031] Deduplication and merging: Calculate the visual overlap of adjacent candidate inspection points, merge those exceeding the set threshold into a single inspection point, and retain the better visual pose.

[0032] Output: A set of inspection points is formed, including the three-dimensional coordinates, default shooting posture, target area label and priority of each inspection point, which is used for subsequent path planning.

[0033] The 3D coordinates of all inspection points are read. Under the preset constraints, a path optimization algorithm is used to determine the order in which the inspection points are visited and generate connection trajectories to form an inspection path, which is as follows: Constraint modeling: Set constraints such as battery power and maximum range, maximum mission duration, no-fly zones and restricted altitude zones, minimum / maximum flight altitude, safety clearance, maximum turn angle / climb rate, and environmental thresholds (wind speed, rainfall).

[0034] Graph modeling: A connected graph is constructed within the flyable space, with all inspection points as required nodes. The edge weights of the graph are defined by the comprehensive cost (flight distance, energy consumption, wind direction penalty, and turning cost). Edges that cross no-fly zones or do not meet safety clearance requirements are removed or replaced with obstacle-avoidance relay waypoints.

[0035] Visit sequence optimization: The problem is modeled as a constrained traveling salesman / vehicle routing problem, and a heuristic solution process (nearest neighbor initialization + two / three exchange local optimization or genetic / tabu search iteration) is adopted. When the battery and duration thresholds are not met, the trip is automatically split into multiple flights and return / charging / swapping nodes are inserted.

[0036] Segment generation: Generate collision-free segments for adjacent inspection points within the flyable space using a three-dimensional path search (such as A*, RRT*, or PRM) in an optimized visit order.

[0037] Trajectory smoothing and executable: B-spline or multi-segment polynomial smoothing is performed on discrete flight segments, and velocity, acceleration, angular velocity, and curvature boundaries are applied. The hovering or deceleration duration is configured at each inspection point to ensure imaging stability.

[0038] Verification and output: Verify coverage (target area imaging resolution and viewing angle requirements), safety clearance, and no-fly avoidance to see if they meet the standards. If not, return to the step of visit sequence optimization / segment generation and replanning. If they meet the standards, output the inspection path file (including waypoint sequence, attitude, speed, and timestamp) and task list for distribution and execution.

[0039] In the embodiment, through accurate acquisition and processing of the tower crane jib structure information, scientific arrangement of the inspection points and intelligent optimization of the path are realized, blind areas and repeated coverage caused by traditional manual experience arrangement are avoided, through key part must-inspect, component supplementary point generation, occlusion elimination, resolution checking and safety gap constraint, the comprehensiveness and safety of the inspection are ensured, meanwhile, various practical constraints such as electric quantity, flight prohibited area, maximum flight range, flight height are introduced in the path planning stage, and the optimization solution of the traveling salesman problem and three-dimensional path search are combined to ensure that the inspection path is not only feasible but also efficient, the trajectory smoothing and executable processing further improve the stability of the unmanned aerial vehicle flight and the imaging quality, the final checking and output link makes the path scheme have self-checking and error correction capability, this whole mechanism significantly improves the inspection coverage rate and reliability, reduces the risk of missed inspection and flight risk, and shortens the operation time and the demand for manual intervention.

[0040] Specifically, the infrared thermal imaging image includes a plurality of infrared pixel points, and each infrared pixel point corresponds to an apparent temperature pixel value; the visible light image includes a plurality of visible light pixel points, and each visible light pixel point corresponds to a reflected light brightness pixel value; and the multi-source fusion image includes a plurality of fusion pixel points, and each fusion pixel point corresponds to a fusion pixel value.

[0041] As shown in Figure 2 The specific steps of obtaining the corresponding multi-source fusion image are as follows: geometrically registering the infrared thermal imaging image and the visible light image of the tower crane jib to be detected, so that the pixel points correspond to each other under the same pixel grid; reading the apparent temperature pixel value and the reflected light brightness pixel value corresponding to the fusion pixel point on the common pixel grid in units of fusion pixel points, respectively, and analyzing the fusion pixel value to form the multi-source fusion image.

[0042] In the embodiment, through pixel-level registration and fusion of the infrared thermal imaging image and the visible light image, it is ensured that the two types of images correspond to each other point by point under the same grid, so that different information sources can complementarily play a role, the infrared image can reflect the temperature distribution of the component surface, the visible light image can present the texture and topographic features of the component, and the fusion pixel value is formed on the same pixel point, which not only retains the abnormal sensitivity of the thermal features, but also integrates the intuitive expression of the visual details. This multi-source fusion avoids the defects that a single image analysis is easily disturbed by light, environmental temperature and the like, so that defect identification and positioning are more reliable. Meanwhile, the geometric registration and fusion steps are automatically executed, the workload of manual alignment and interpretation is reduced, the efficiency and consistency of data processing are improved, and the multi-source fusion image finally generated has higher information density and readability.

[0043] Specifically, the specific steps of analyzing the fusion pixel value of each fusion pixel point are as follows: for each infrared pixel point, the corresponding neighborhood apparent temperature pixel mean and neighborhood apparent temperature pixel standard deviation are identified and analyzed based on the set neighborhood range, and the temperature weight of each fusion pixel point is analyzed; for each visible light pixel point, the corresponding brightness gradient is analyzed, and the corresponding neighborhood maximum brightness gradient is identified and analyzed based on the set neighborhood range, and the brightness weight of each fusion pixel point is analyzed; for each fusion pixel point, the corresponding fusion pixel value is obtained by weighted analysis based on the apparent temperature pixel value, the temperature weight, the reflected light brightness pixel value and the brightness weight.

[0044] wherein the specific formula for calculating the temperature weight and the brightness weight of a certain fusion pixel point is as follows: ; wherein, the temperature weight of a certain fusion pixel point, the apparent temperature pixel value corresponding to a certain fusion pixel point, the neighborhood apparent temperature pixel mean corresponding to a certain fusion pixel point, the neighborhood apparent temperature pixel standard deviation corresponding to a certain fusion pixel point, the temperature adjustment factor stored in the database, used to prevent the denominator from being 0, the brightness weight of a certain fusion pixel point, the brightness gradient corresponding to a certain fusion pixel point, the neighborhood maximum brightness gradient corresponding to a certain fusion pixel point, the brightness adjustment factor stored in the database, used to prevent the denominator from being 0.

[0045] In the present embodiment, the dynamic calculation of the weight realizes more scientific and reliable fusion. For infrared data, the analysis of the neighborhood mean and standard deviation can highlight the difference between the abnormal points and the overall temperature distribution, thereby enhancing the saliency of the abnormal area. For visible light data, through the comparison of the brightness gradient and the neighborhood maximum value, the edge and texture features can be better captured, so that small damages such as cracks and corrosion can be highlighted. Through weight normalization, the two types of data realize complementary advantages in the same fusion pixel, which not only retains the sensitivity of infrared in thermal anomaly detection, but also combines the intuitiveness of visible light in topographic identification. At the same time, the adjustment factor introduced in the formula can avoid the calculation problem of the denominator being zero, improve the stability of the algorithm in actual complex scenes, and overall, this step makes the information density of the fusion image higher and the target features clearer, effectively reducing the false detection and missed detection caused by environmental interference.

[0046] Specifically, the specific steps of the improved YOLOv5 damage detection algorithm for damage identification of the multi-source fusion image are as follows: scale unification and pixel normalization are performed on the multi-source fusion image of the tower crane jib to be detected, specifically: the fusion image is scaled and padded according to the network input size, and the scaling coefficient and boundary offset are recorded for result remapping; the pixel value is normalized and the channel order is arranged, and the one-to-one correspondence with the corresponding inspection point identifier is maintained; the processed multi-source fusion image is input into the improved YOLOv5 model, and the output is the candidate box, the class and the confidence, specifically: the improved YOLOv5 model weight and the class set are loaded, the inference threshold is set, the preprocessed fusion image is input, the original detection result containing the candidate box coordinates, the damage class label and the confidence is obtained, and the candidate box coordinates are restored to the original image scale according to the recorded scaling / offset parameters; non-maximum suppression or weighted non-maximum suppression is performed on the candidate box, and screening is performed according to the confidence threshold and the intersection over union threshold to obtain a damage target set, specifically: the same class candidate boxes are sorted by confidence and non-maximum suppression or weighted non-maximum suppression is performed one by one; the candidate boxes with an intersection over union lower than the threshold or a confidence lower than the threshold are removed, and the de-duplicated damage target set is output, and the image coordinates, the class and the confidence information of each target and the corresponding inspection point identifier are retained.

[0047] In the embodiment, scale unification and pixel normalization are used to ensure the consistency of fusion images of different sources and different sizes when entering the model, avoid target detection deviation caused by inconsistent scales, record scaling and boundary information to ensure that the detection result can be accurately remapped to the original image, secondly, the preprocessed image is input into the improved YOLOv5 model, the trained weight and class set are used to quickly output the candidate box, the class and the confidence, which significantly improves the identification efficiency, finally, non-maximum suppression or weighted non-maximum suppression is introduced in the candidate box output link, through the double screening of confidence and intersection over union, redundant boxes and low-quality detection results are effectively removed, and the final retained target set not only removes repeated detection, but also considers the coverage and accuracy of detection.

[0048] Specifically, the improved YOLOv5 includes introducing a channel attention module in the backbone network to enhance fine-grained feature expression, which specifically includes setting a channel re-rating unit after the feature map of each stage of the backbone, adjusting the weight of each channel response to highlight the channels related to the fine damage and suppress irrelevant channels; multi-scale feature fusion is performed in the feature pyramid and path aggregation structure, and a learnable weight is set for the fusion branch to suppress invalid scales, which specifically includes using top-down and bottom-up bidirectional fusion to weight the features of different scales according to branch weights, and the branch weights are automatically learned during the training process to reduce the influence of invalid or redundant scales; a multi-scale detection head decoupled from the classification branch and the regression branch is used, which specifically includes setting a classification branch for class and confidence prediction and a regression branch for target position and size prediction in each scale detection head, and the two are independent of each other to reduce task interference; the anchor box size and aspect ratio are adaptively updated based on sample statistics, which specifically includes updating the size parameters and aspect ratio parameters of the anchor box according to the size and aspect ratio distribution of the target in the training sample to improve the matching degree of the anchor box and the target; weighted non-maximum suppression is used in the output stage to fuse overlapping candidate boxes, which specifically includes merging the same class and overlapping candidate boxes according to the confidence to obtain a single output box, thereby reducing the multi-box of the same defect and the mutual interference of adjacent defects.

[0049] In the present embodiment, a channel attention module is added to the backbone network, enabling the model to automatically enhance the feature channels related to defects such as fine cracks and rust, and suppress background noise channels, thereby better distinguishing between effective information and interference information. Secondly, a learnable weight is introduced in the feature pyramid and path aggregation to ensure that the model can highlight truly effective scale information when fusing multi-scale features, reducing the interference of redundant and invalid features on the results. Thirdly, the decoupling design of the classification and regression branches avoids the conflict between the target class and position regression tasks, making the model more robust in both class recognition and position positioning. The adaptive update of the anchor box size and aspect ratio ensures that the model can better fit the real distribution of the tower crane boom defect target, improving the consistency of the prediction box and the target. Finally, weighted non-maximum suppression can reasonably fuse overlapping candidate boxes, reducing the mutual interference of multi-box detection and adjacent targets.

[0050] Specifically, the generation of the inspection report comprises: mapping the image coordinates of the damage target to the structural coordinates of the tower crane jib, specifically: according to the aforementioned geometric registration relationship and the three-dimensional coordinates / default shooting pose of the corresponding inspection point, the image coordinates of the detection frame are back-projected to the structural coordinate system of the tower crane jib to obtain the position description of each damage target in the structural coordinate system; outputting the position, category, confidence, size and timestamp of each damage target in a preset format, and associating the corresponding inspection point identifier, specifically: generating a record entry for each damage target, including the structural coordinate position, damage category, confidence, frame size, acquisition timestamp and inspection point identifier, and organizing them into a tabular list according to the task number; according to the threshold rule, the severity classification and the re-inspection point marking are performed, and the inspection report containing the result list and the re-inspection task suggestion is generated and archived, specifically: according to the preset threshold, the severity classification and disposal suggestion of the target are determined, and the targets that need to be re-inspected are marked and a re-inspection task entry is generated; the inspection report is formed by summarizing the task information, result list and re-inspection suggestion, and the archive and tracking association are completed.

[0051] In the embodiment, by back-projecting the image coordinates of the damage target to the structural coordinate system of the tower crane jib, the spatial position of the defect on the actual component can be accurately described, solving the problem that the image detection result alone is difficult to correspond in the field, secondly, the position, category, confidence, size and timestamp of each defect are output in a unified format and associated with the corresponding inspection point identifier to form a standardized data entry, which not only facilitates subsequent retrieval and comparison, but also avoids data confusion caused by inconsistent manual records, thirdly, the severity classification and re-inspection marking mechanism are introduced, so that the inspection result can be directly converted into operation and maintenance suggestions to help management personnel quickly distinguish between minor defects and problems that need to be tracked, and finally, through the summary, archiving and tracking of the report, a complete inspection archive is established to support long-term trend analysis and multiple re-inspection comparisons.

[0052] Please refer to Figure 3 , based on the same inventive concept, the embodiment of the present application provides a self-inspection infrared unmanned aerial vehicle inspection device, which is applied to the self-inspection infrared unmanned aerial vehicle inspection method, comprising: a path planning module for acquiring the structural information of the tower crane jib to be detected and analyzing and generating an inspection path containing a plurality of inspection points; a flight execution module for inputting the inspection path into the infrared unmanned aerial vehicle for flight inspection; an image fusion module for respectively collecting infrared thermal imaging images and visible light images of the tower crane jib to be detected at each inspection point when the infrared unmanned aerial vehicle arrives at each inspection point, and performing multi-source fusion to obtain corresponding multi-source fusion images; a damage identification module for identifying damages in the multi-source fusion images based on the improved YOLOv5 damage detection algorithm and outputting an inspection report.

[0053] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims include all such modifications and variations as fall within the scope of the present application.

[0054] It is apparent that those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. It is therefore intended that the present application cover all such changes and modifications that are within its scope.

Claims

1. A self-inspection infrared unmanned aerial vehicle inspection method, characterized in that, The method comprises the following steps: Obtain the structure information of the tower crane jib to be detected, and analyze and generate an inspection path containing a plurality of inspection points; Input the inspection path into an infrared unmanned aerial vehicle for flight inspection; When the infrared unmanned aerial vehicle reaches each inspection point, respectively collect infrared thermal imaging images and visible light images of the corresponding tower crane jib to be detected, and perform multi-source fusion to obtain corresponding multi-source fusion images; Perform damage identification on the multi-source fusion images based on an improved YOLOv5 damage detection algorithm, and output an inspection report; The infrared thermal imaging images comprise a plurality of infrared pixel points, and each infrared pixel point corresponds to an apparent temperature pixel value. The visible light images comprise a plurality of visible light pixel points, and each visible light pixel point corresponds to a reflected light brightness pixel value. The multi-source fusion images comprise a plurality of fusion pixel points, and each fusion pixel point corresponds to a fusion pixel value. The specific steps of obtaining the corresponding multi-source fusion images are as follows: Geometrically register the infrared thermal imaging images and the visible light images of the tower crane jib to be detected, so that the pixel points correspond one by one under the same pixel grid; On the common pixel grid, read the apparent temperature pixel value and the reflected light brightness pixel value corresponding to each fusion pixel point, and analyze the fusion pixel value to form a multi-source fusion image; The specific steps of analyzing the fusion pixel value of each fusion pixel point are as follows: For each infrared pixel point, based on a set neighborhood range, identify and analyze the neighborhood apparent temperature pixel mean value and the neighborhood apparent temperature pixel standard deviation, and analyze the temperature weight of each fusion pixel point; For each visible light pixel point, analyze the corresponding brightness gradient, and based on a set neighborhood range, identify and analyze the neighborhood maximum brightness gradient, and analyze the brightness weight of each fusion pixel point; For each fusion pixel point, based on the apparent temperature pixel value, the temperature weight, the reflected light brightness pixel value, and the brightness weight, perform weighted analysis to obtain the corresponding fusion pixel value. The specific formulas for calculating the temperature weight and the brightness weight of a certain fusion pixel point are as follows: ； wherein, , the temperature weight, the brightness weight, , , , , the apparent temperature pixel value corresponding to the certain fusion pixel point, the neighborhood apparent temperature pixel mean value, the neighborhood apparent temperature pixel standard deviation, the brightness gradient, the neighborhood maximum brightness gradient, , the temperature adjustment factor stored in the database, the brightness adjustment factor.

2. The self-inspection infrared drone inspection method of claim 1, wherein, The structure information includes the three-dimensional coordinates of each component and the three-dimensional coordinates of each key part.

3. The self-inspection infrared drone inspection method of claim 2, wherein, The specific steps of generating the inspection path containing a plurality of inspection points are as follows: Set the three-dimensional coordinates of each key part of the tower crane jib to be detected as an inspection point, and generate supplementary inspection points according to the three-dimensional coordinates of each component according to a preset rule; Read the three-dimensional coordinates of all inspection points, determine the access order of the inspection points and generate a connected track under the preset constraint condition by using a path optimization algorithm to form an inspection path.

4. The self-inspection infrared drone inspection method of claim 1, wherein, The specific steps of performing damage identification on the multi-source fusion images based on the improved YOLOv5 damage detection algorithm are as follows: Perform scale unification and pixel normalization on the multi-source fusion images of the tower crane jib to be detected; Input the processed multi-source fusion images into the improved YOLOv5 model to output candidate boxes, classes, and confidence levels; Perform non-maximum suppression or weighted non-maximum suppression on the candidate boxes, and filter them according to the confidence threshold and the intersection over union threshold to obtain a damage target set.

5. The self-inspection infrared drone inspection method of claim 4, wherein, The improved YOLOv5 includes: Introduce a channel attention module in the backbone network to enhance fine-grained feature expression; Multi-scale feature fusion is performed in the feature pyramid and path aggregation structure, and a learnable weight is set for the fusion branch to suppress invalid scales; A multi-scale detection head is adopted, which decouples the classification branch and the regression branch; The anchor box size and aspect ratio are adaptively updated based on sample statistics; In the output stage, a weighted non-maximum suppression is used to remove and fuse overlapping candidate boxes.

6. The self-inspection infrared drone inspection method of claim 5, wherein, The generation of the inspection report includes: Mapping the image coordinates of the damaged target to the structural coordinates of the tower crane jib; Outputting the location, category, confidence, size, and timestamp of each damaged target in a preset format, and associating the corresponding inspection point identifier; According to the threshold rule, the severity is classified and the retest point is marked, and the inspection report containing the result list and the re-inspection task suggestion is generated and archived.

7. A self-inspection infrared unmanned aerial vehicle inspection device, applying the self-inspection infrared unmanned aerial vehicle inspection method of any one of claims 1-6, characterized in that, It includes: A path planning module for obtaining the structural information of the tower crane jib to be detected and analyzing and generating an inspection path containing a plurality of inspection points; A flight execution module for inputting the inspection path into the infrared unmanned aerial vehicle for flight inspection; An image fusion module for collecting infrared thermal imaging images and visible light images of the corresponding tower crane jib to be detected when the infrared unmanned aerial vehicle reaches each inspection point, and performing multi-source fusion to obtain the corresponding multi-source fusion image; A damage identification module for identifying the damage of the multi-source fusion image based on the improved YOLOv5 damage detection algorithm and outputting the inspection report.

Images