Aircraft surface damage detection algorithm and system based on improved YOLOv12

Abstract

The invention discloses an improved YOLOv12-based aircraft surface damage detection algorithm and system, and relates to the technical field of aircraft surface damage, and the method comprises the following steps: S1, data collection and preprocessing: employing an unmanned plane and an unmanned vehicle to collect the related data of the aircraft surface damage, precisely marking the aircraft surface image, and increasing the data diversity; s2, model improvement: improving a YOLOv12 aircraft surface damage detection model according to an aircraft surface detection task; a high resolution input size is employed to capture small defect features. According to the method, the YOLOv12 aircraft surface damage detection model is improved, the unmanned aerial vehicle and the unmanned vehicle can comprehensively record key structural components on the surface of the aircraft under various illumination conditions, the data integrity is guaranteed, and through the CLAHE algorithm, diversified data and the imaging effect in various environments, the detection accuracy of the aircraft surface damage detection model is improved, and the detection accuracy of the aircraft surface damage detection model is improved. And the robustness of subsequent model training can be obviously improved, so that the detection algorithm has higher environmental adaptability.

Description

Technical Field

[0001] The present invention relates to the technical field of aircraft surface damage, and in particular to an aircraft surface damage detection algorithm and system based on an improved YOLOv12. Background Art

[0002] In the field of aircraft safety maintenance, accurate detection of surface damage such as corrosion, scratches, and cracks is a core link in ensuring flight safety. Traditional manual visual inspection relies on the experience of technicians and has disadvantages such as low detection efficiency, insufficient blind spot coverage, and highly subjective standards. According to statistics, in the past five years, approximately 58% of aircraft airworthiness issues caused by missed surface damage can be attributed to the limitations of traditional inspection methods. Although automated inspection technology has improved coverage to a certain extent, existing algorithms still face shortcomings. For example, the traditional YOLO series model has a fixed grid division strategy, resulting in a missed detection rate of more than 30% for minor defects. The detection accuracy drops by 15%-20% in complex backgrounds with reflective skin and shadows. In addition, the large number of parameters and slow inference speed make it difficult to meet the real-time needs of drones and unmanned vehicles.

[0003] In this regard, patent CN119831940A proposes a quartz optical component defect detection method based on improved YOLOv10, which includes the following steps: obtaining a digitized image through a quartz optical component defect detection device and preprocessing it; building a quartz optical component defect database; using Labelme to label the images in the database; training and testing the improved YOLOv10 model; loading the trained quartz optical component defect detection model to detect quartz optical component defects.

[0004] However, the current aircraft surface damage detection algorithms and systems have the following defects: usually under different lighting conditions, drones and unmanned vehicles cannot fully record the key structures on the aircraft surface and cannot ensure the integrity of the data. In addition, the imaging effects of diversified data and multiple environments cannot significantly improve the robustness of subsequent model training, reducing the adaptability of the detection algorithm to the environment.

[0005] Therefore, it is necessary to propose an aircraft surface damage detection algorithm and system based on improved YOLOv12 to solve the above problems. Summary of the Invention

[0006] The purpose of the present invention is to provide an aircraft surface damage detection algorithm and system based on improved YOLOv12 to solve the problems raised in the above background technology.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an aircraft surface damage detection algorithm and system based on an improved YOLOv12, comprising the following steps:

[0008] S1. Data collection and preprocessing: Use drones and unmanned vehicles to collect data related to aircraft surface damage, accurately annotate aircraft surface images, and increase data diversity;

[0009] S2. Model Improvement: The YOLOv12 aircraft surface damage detection model was improved for aircraft surface inspection tasks; a high-resolution input size was adopted to capture small defect features; a CBAM module was added to enhance the attention mechanism; confidence thresholds and IoU thresholds were set to accommodate irregularly shaped damage detection; and input images were analyzed in real time during model inference to automatically identify and classify various surface defects.

[0010] S3. Model training optimization: A two-stage optimization method is adopted. In the first stage, the improved K-means clustering method is used to optimize the anchor box ratio for hyperparameter search; in the second stage, the triple optimization method of "label allocation optimization + mixed precision training + gradient accumulation" is used to fine-tune the parameters.

[0011] Preferably, the drone and the unmanned vehicle are equipped with high-resolution cameras, and the drone and the unmanned vehicle systematically photograph key parts through the high-resolution cameras, and the systematic photographing requires collecting images under different lighting conditions.

[0012] Preferably, the data related to the damage on the aircraft surface is obtained by enhancing the image contrast through the CLAHE algorithm; the data diversity is enhanced by implementing geometric transformation strategies such as rotation and flipping; and the aircraft surface image is obtained by performing brightness adjustment to simulate imaging effects in various environments.

[0013] Preferably, after the data configuration file of the YOLOv12 aircraft surface damage detection model is modified, the pixels are 640×640, and the sampling strategy is adjusted to ensure feature map alignment; the confidence threshold is set to above 0.5, and the IoU threshold is set to 0.45.

[0014] Preferably, the model training optimization is specifically as follows:

[0015] S3.1. Use an improved K-means clustering method to optimize the anchor frame ratio for hyperparameter search, targeting the aircraft's aspect ratio characteristics. Implement a dynamic adjustment strategy to automatically scale the anchor frame size based on the altitude changes of the drone's aerial photography. Then, use local occlusion simulation and Gaussian noise injection to specifically enhance the aircraft's local features. Finally, optimize positioning accuracy by increasing the bounding box regression loss weight.

[0016] S3.2. A triple optimization method of "label assignment optimization + mixed precision training + gradient accumulation" is used to fine-tune parameters. Label assignment optimization uses Task-Aligned Assigner + dynamic sample weighting to improve the positive sample matching rate of small targets. Mixed precision training can speed up training and reduce video memory usage. Gradient accumulation can simulate large-scale training and enhance gradient stability. A label assignment optimization strategy is introduced, and mixed precision training and gradient accumulation technology are used to significantly improve the detection sensitivity of small defects, increasing the training speed by 40% while maintaining accuracy.

[0017] The aircraft surface damage detection system based on the improved YOLOv12 also includes:

[0018] Data acquisition module: UAVs and unmanned vehicles are equipped with high-resolution cameras to collect data on the aircraft surface and systematically photograph key parts of the aircraft from multiple angles; images are collected under different lighting conditions;

[0019] The data processing module uses the CLAHE algorithm to enhance image contrast and highlight surface features; increases data diversity through geometric transformation; and performs brightness adjustment to simulate imaging effects in different environments;

[0020] The model detection module uses the introduced high-resolution input size, adds the CBAM module, sets the confidence threshold and IoU threshold of the YOLOv12 optimization architecture, recognizes the data, detects defects on the aircraft surface in real time, and outputs structured results;

[0021] The software system module designs an interactive interface for users, realizes real-time communication with drones and unmanned vehicles, automatically generates inspection reports and provides corresponding maintenance suggestions.

[0022] Preferably, the software system module supports detection of images, image batches and video inputs, and the detection efficiency of the software system module is above 20 frames per second.

[0023] Preferably, the structured results specifically include visual marking of defect locations, damage classification information, and confidence value display of corresponding detection results.

[0024] Technical effects and advantages of the present invention:

[0025] 1. This invention improves the YOLOv12 aircraft surface damage detection model, enabling drones and unmanned vehicles to comprehensively record key structural components on aircraft surfaces under various lighting conditions, ensuring data integrity. Furthermore, through the CLAHE algorithm, diversified data, and imaging effects in various environments, the robustness of subsequent model training can be significantly improved, giving the detection algorithm greater environmental adaptability.

[0026] 2. The aircraft surface damage detection method and system based on the improved YOLOv12 provided by the present invention introduces the R-ELAN residual efficient layer aggregation network and the Scaling Factor scaling factor to adjust the residual path weight, optimizes gradient flow, and adopts a cross-layer feature fusion strategy similar to the bottleneck structure to improve the utilization of multi-scale information. Compared with YOLOv10, it solves the problems of gradient vanishing and training instability in large-scale attention models.

[0027] 3. The present invention provides an aircraft surface damage detection method and system based on the improved YOLOv12. YOLOv12 completely abandons the traditional CNN architecture for the first time, focuses on attention, adopts the pure attention mechanism Vision Transformer as the backbone network, and introduces a regional attention module during feature extraction, enabling the model to more accurately capture tiny defects on the aircraft surface. Compared with the local attention mechanism adopted by YOLOv10 and its predecessors, the regional attention module performs well in reducing computational complexity and cost, thereby significantly improving speed performance.

[0028] 4. The aircraft surface damage detection method and system based on improved YOLOv12 provided by the present invention, in addition to the traditional Transformer position encoding, helps regional attention perceive position information by introducing large separable convolutions, improves GPU video memory utilization by reducing the number of memory accesses in attention calculations, and achieves a 10-fold increase in SRAM read and write speed. It is oriented to high-resolution complex scenes and relies on high-performance GPUs for real-time detection. The performance is significantly improved under low-light conditions, with an ExDark dataset mAP of 47.3%, an increase of 9.2% over YOLOv11.

[0029] 5. The aircraft surface damage detection method and system based on the improved YOLOv12 provided by the present invention can automatically generate detailed maintenance recommendation reports with an integrated intelligent decision-making system, reducing the manual intervention cost by more than 60%. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The figure is a schematic diagram of the aircraft surface damage detection method and system flow based on the improved YOLOv12 of the present invention.

[0031] Figure 2 Schematic diagram of the CBAM module of the present invention.

[0032] Figure 3 Schematic diagram of the intelligent detection of drones and unmanned vehicles according to the present invention.

[0033] Figure 4The figure is a schematic diagram of the aircraft surface damage detection method and system data acquisition process based on the improved YOLOv12 of the present invention.

[0034] Figure 5 Schematic diagram of the detection results of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the embodiments described are part of the embodiments of the present invention, not all of the embodiments, and they should not be understood as limitations on the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0036] The present invention provides Figure 1-Figure 5 The aircraft surface damage detection algorithm based on the improved YOLOv12 shown includes the following steps:

[0037] S1. Data collection and preprocessing: Use drones and unmanned vehicles to collect data related to aircraft surface damage, accurately mark aircraft surface images and increase data diversity; use drones and unmanned vehicles to collect data (the collection process is shown in the attached Figure 4 By accurately labeling the aircraft surface images and increasing data diversity, the environmental adaptability is improved.

[0038] S2. Model improvement: Improve the YOLOv12 aircraft surface damage detection model for aircraft surface inspection tasks; use high-resolution input size to capture small defect features; add CBAM module to enhance the attention mechanism; set confidence threshold and IoU threshold to adapt to irregular shape damage detection; analyze input images in real time during model inference, and automatically identify and classify various surface defects; When improving the model, it is necessary to load the specially optimized YOLOv12 aircraft damage detection model. This model is based on the standard YOLOv12 architecture and has been targeted for aircraft surface inspection tasks. The YOLOv12 aircraft surface damage detection model will analyze the input image in real time during the inference process, automatically identify and classify various surface defects (such as Figure 2 including but not limited to common damage types such as corrosion, cracks, and scratches.

[0039] S3. Model training optimization: A two-stage optimization method is adopted. In the first stage, the improved K-means clustering method is used to optimize the anchor box ratio for hyperparameter search; in the second stage, the triple optimization method of "label allocation optimization + mixed precision training + gradient accumulation" is used to fine-tune the parameters. While this method significantly improves the sensitivity of small damage detection, it can adjust the box / cls loss weight ratio (0.05:0.5) to avoid sacrificing the detection performance of large defects when optimizing small targets.

[0040] It should be noted that drones and unmanned vehicles are equipped with high-resolution cameras, which systematically photograph key parts. Systematic photography requires collecting images under different lighting conditions. To collect images of aircraft surface damage, drones and unmanned vehicles equipped with high-resolution cameras are used to conduct comprehensive photographic recording of key structural components such as the aircraft's wings, tail and skin. To ensure the integrity of the data, image collection must be carried out under a variety of lighting conditions, including direct strong light, shadow areas, and low-light environments.

[0041] It should also be noted that the relevant data on aircraft surface damage are obtained by enhancing image contrast through the CLAHE algorithm; data diversity is enhanced by implementing geometric transformation strategies such as rotation and flipping; aircraft surface images are obtained by performing brightness adjustment to simulate imaging effects in various environments; the CLAHE algorithm is used to enhance image contrast to highlight the characteristics of subtle damage on the aircraft surface, and geometric transformation strategies such as rotation and flipping are implemented to enhance data diversity, and brightness adjustment is performed to simulate the imaging effects of aircraft surface images in various environments. These preprocessing measures can significantly improve the robustness of subsequent model training, making the detection algorithm more adaptable to the environment.

[0042] Furthermore, after the data configuration file of the YOLOv12 aircraft surface damage detection model was modified, the pixels were changed to 640×640, and the sampling strategy was adjusted to ensure feature map alignment; the confidence threshold was set to above 0.5, and the IoU threshold was set to 0.45; by modifying the data configuration file, the pixels were changed to 640×640, and then the sampling strategy was adjusted to ensure feature map alignment to achieve a higher resolution input size. In order to enhance the attention mechanism, the CBAM module was added; the confidence threshold was set to above 0.5 to ensure the reliability of the detection results, and the IoU threshold of 0.45 was used to adapt to irregular shape damage detection.

[0043] It should be noted that YOLOv12 completely abandons the traditional CNN architecture for the first time, focusing on attention. It adopts the pure attention mechanism Vision Transformer as the backbone network. During feature extraction, it introduces a regional attention module, which divides the feature map into vertical and horizontal regions and performs attention calculations within each region. By reducing the length of the sequence to be processed, it reduces computational complexity and cost, significantly improving speed performance. It also enables the model to more accurately capture subtle defects on the aircraft surface, such as fine cracks and superficial scratches. Compared with the local attention mechanisms used in YOLOv10 and its predecessors, such as Crisis-cross attention, Window attention, and Axial attention, the regional attention module excels in reducing computational complexity and cost, thereby significantly improving speed performance.

[0044] It should also be noted that in addition to the traditional Transformer's positional encoding, the introduction of large separable convolutions, such as 7×7, effectively helps regional attention perceive positional information. The FlashAttention technology introduced optimizes the memory access pattern in the attention mechanism. By reducing the number of memory accesses in attention calculations, it improves GPU memory utilization and achieves a 10x increase in SRAM read and write speeds. This helps to complete inference tasks faster when processing high-resolution images, and enables real-time detection on high-performance GPUs. For low-light conditions, the image loss is enhanced using a frequency domain contrast loss and a reconstruction loss. A dynamic label assignment strategy is adopted to improve the recall of small and blurred objects. A specific loss is designed for the S channel in HSV space to enhance the model's lighting-insensitive feature learning and localization capabilities for partially occluded objects. On the ExDark dataset, by adjusting the gamma value for sensitivity analysis and selecting the optimal enhancement parameters, experiments show that YOLOv12 significantly improves its performance in low-light conditions, achieving a mAP of 47.3% on the ExDark dataset, a 9.2% improvement over YOLOv11.

[0045] In this invention, the R-ELAN residual efficient layer aggregation network is introduced into the YOLOv12 architecture to modify the original network layer; then the Scaling Factor is introduced into the block-level residual parameter connection to adjust the weight of the residual path, thereby optimizing the gradient flow. In the experiment, the value of the scaling factor is set starting from 0.1, and gradually adjusted according to the results of the preliminary experiment. If the model performs poorly, the scaling factor is increased. If the model is overfitting or the training is unstable, the scaling factor is reduced. After determining the approximate range, more detailed tuning is performed to achieve the best gradient flow effect. To ensure the reliability of the results, the model is evaluated using the cross-validation method. Through multiple training and verification, the impact of the scaling factor on the model performance is studied; a cross-layer feature fusion strategy similar to the bottleneck structure is adopted, and the specific design includes a bottleneck structure of convolution layer, activation function and pooling layer. Feature maps of different scales are selected for fusion, and maximum pooling is used on the feature maps at the selected level to match the feature map sizes of different levels. The parameters of the bottleneck structure and fusion strategy are then optimized through the backpropagation algorithm to improve the utilization of multi-scale information. Compared with YOLOv10, it solves the problems of gradient disappearance and training instability in large-scale attention models.

[0046] Going further, model training optimization is as follows:

[0047] S3.1. Use an improved K-means clustering method to optimize anchor frame ratios for hyperparameter search, generating specific anchor frames based on aircraft aspect ratio characteristics, such as the wing-to-fuselage ratio. Implement a dynamic adjustment strategy to automatically scale the anchor frame size based on the altitude of the drone. Then, use local occlusion simulation and Gaussian noise injection to specifically enhance the local features of the aircraft. Finally, optimize positioning accuracy by increasing the bounding box regression loss weight.

[0048] S3.2. A triple optimization method of "label assignment optimization + mixed precision training + gradient accumulation" is used to fine-tune parameters. Label assignment optimization uses Task-Aligned Assigner + dynamic sample weighting to improve the positive sample matching rate of small targets. Mixed precision training can speed up training and reduce video memory usage. Gradient accumulation can simulate large-scale training and enhance gradient stability. A label assignment optimization strategy is introduced, and mixed precision training and gradient accumulation technology are used to significantly improve the detection sensitivity of small defects, increasing the training speed by 40% while maintaining accuracy.

[0049] The aircraft surface damage detection system based on the improved YOLOv12 in the present invention also includes:

[0050] Data acquisition module: UAVs and unmanned vehicles are equipped with high-resolution cameras to collect aircraft surface data and systematically photograph key aircraft parts from multiple angles. To ensure the comprehensiveness of the data, images must be collected under different lighting conditions, such as direct sunlight, shadow areas, and low-light environments.

[0051] The data processing module uses the CLAHE algorithm to enhance image contrast and highlight subtle surface defects; it increases data diversity through geometric transformations such as rotation and flipping; and it adjusts brightness to simulate imaging effects in different environments;

[0052] The model detection module uses a YOLOv12 optimized architecture with a higher-resolution input size, the addition of the CBAM module, and the setting of confidence and IoU thresholds. This allows for high-precision defect recognition on data, detecting defects such as corrosion and cracks on aircraft surfaces in real time, and outputting structured results including location coordinates, type classification, and confidence score.

[0053] The software system module is designed with an interactive interface for users, which integrates data input, parameter configuration, result display and data storage functions, realizes real-time communication with drones and unmanned vehicles, automatically generates inspection reports and provides corresponding maintenance suggestions.

[0054] In this paper, we developed an intelligent inspection software system using the Python programming language and development framework. The system consists of the following modules: a data input module that supports image, image batch, and video input; a model inference module that integrates an optimized YOLOv12 model; a result display module that displays defect location, classification information, and confidence level; and a data storage module that stores inspection results and related data.

[0055] It should be noted that the software system module supports the detection of images, image batches and video inputs. The detection efficiency of the software system module is more than 20 frames per second. The structured results specifically include visual marking of defect locations, damage classification information and confidence value display of corresponding detection results.

[0056] This invention is based on the improvement of YOLOv12's aircraft surface damage detection system. The integrated intelligent decision-making system can automatically generate detailed maintenance recommendation reports, reducing the cost of manual intervention by more than 60%.

[0057] This study deeply integrates drones and unmanned vehicle platforms with an intelligent inspection software system. This integrated solution ensures that image data collected by drones during flight is transmitted to the intelligent inspection software system in real time with high efficiency and low latency. After receiving this data, the software system uses pre-trained models to perform precise inference analysis, enabling rapid detection of surface damage on target objects. After the inspection is complete, the system generates detailed inspection results and provides real-time visual feedback to the user, ensuring that the user has a timely understanding of the inspection object's condition. In practical application scenarios, the operator can flexibly control the drone's flight path and data collection process through the intelligent inspection software system's user interface, while also monitoring data transmission status and inspection progress in real time. Through this process, operators can efficiently obtain comprehensive and accurate inspection reports, providing a solid and scientific basis for subsequent aircraft maintenance work, significantly improving maintenance efficiency and safety.

Claims

1. Aircraft surface damage detection algorithm based on improved YOLOv12, characterized by: The following steps are involved: S1. Data collection and preprocessing: Use drones and unmanned vehicles to collect data related to aircraft surface damage, accurately annotate aircraft surface images, and increase data diversity; S2. Model Improvement: Improve the YOLOv12 aircraft surface damage detection model for aircraft surface inspection tasks; use high-resolution input size to capture subtle defect features; Add the CBAM module to enhance the attention mechanism; set the confidence threshold and IoU threshold to adapt to irregularly shaped damage detection; analyze the input image in real time during model inference to automatically identify and classify various surface defects; S3. Model training optimization: A two-stage optimization method is used. In the first stage, an improved K-means clustering method is used to optimize the anchor box ratio for hyperparameter search. In the second stage, a triple optimization method of "label assignment optimization + mixed precision training + gradient accumulation" is used for fine-tuning parameters.

2. The aircraft surface damage detection algorithm based on improved YOLOv12 according to claim 1 is characterized in that: The drone and the unmanned vehicle are equipped with high-resolution cameras, and the drone and the unmanned vehicle systematically photograph key parts through the high-resolution cameras. The systematic photographing requires collecting images under different lighting conditions.

3. The aircraft surface damage detection algorithm based on improved YOLOv12 according to claim 1 is characterized in that: The data related to the damage on the aircraft surface is obtained by enhancing the image contrast through the CLAHE algorithm; the data diversity is enhanced by implementing geometric transformation strategies such as rotation and flipping; and the aircraft surface image is obtained by performing brightness adjustment to simulate imaging effects in various environments.

4. The aircraft surface damage detection algorithm based on improved YOLOv12 according to claim 1 is characterized in that: After the data configuration file of the YOLOv12 aircraft surface damage detection model is modified, the pixels are 640×640, and the sampling strategy is adjusted to ensure feature map alignment; the confidence threshold is set to be greater than 0.5, and the IoU threshold is set to 0.

45.

5. The aircraft surface damage detection algorithm based on improved YOLOv12 according to claim 1 is characterized in that: The model training optimization is specifically as follows: S3.

1. Use an improved K-means clustering method to optimize the anchor frame ratio for hyperparameter search, targeting the aircraft's aspect ratio characteristics. Implement a dynamic adjustment strategy to automatically scale the anchor frame size based on the altitude changes of the drone's aerial photography. Then, use local occlusion simulation and Gaussian noise injection to specifically enhance the aircraft's local features. Finally, optimize positioning accuracy by increasing the bounding box regression loss weight. S3.

2. A triple optimization method of "label assignment optimization + mixed-precision training + gradient accumulation" is used to fine-tune parameters. Label assignment optimization uses Task-Aligned Assigner + dynamic sample weighting to improve the positive sample matching rate of small targets. Mixed-precision training can speed up training and reduce video memory usage. Gradient accumulation can simulate large-scale training and enhance gradient stability. The introduction of a label assignment optimization strategy uses mixed-precision training and gradient accumulation technology to significantly improve the detection sensitivity of small defects, while maintaining accuracy while increasing training speed by 40%.

6. Aircraft surface damage detection system based on improved YOLOv12, characterized by: The aircraft surface damage detection method based on the improved YOLOv12 according to any one of claims 1 to 5 further includes: Data acquisition module: UAVs and unmanned vehicles are equipped with high-resolution cameras to collect data on the aircraft surface and systematically photograph key parts of the aircraft from multiple angles; images are collected under different lighting conditions; The data processing module uses the CLAHE algorithm to enhance image contrast and highlight surface features; increases data diversity through geometric transformation; and performs brightness adjustment to simulate imaging effects in different environments; The model detection module uses the introduced high-resolution input size, adds the CBAM module, sets the confidence threshold and IoU threshold of the YOLOv12 optimization architecture, recognizes the data, detects defects on the aircraft surface in real time, and outputs structured results; The software system module designs an interactive interface for users, realizes real-time communication with drones and unmanned vehicles, automatically generates inspection reports and provides corresponding maintenance suggestions.

7. The aircraft surface damage detection system based on improved YOLOv12 according to claim 6 is characterized in that: The software system module supports detection of images, image batches and video inputs, and the detection efficiency of the software system module is above 20 frames per second.

8. The aircraft surface damage detection system based on improved YOLOv12 according to claim 6, characterized in that: The structured results specifically include visual marking of defect locations, damage classification information, and confidence value display of corresponding detection results.

Images