Aircraft surface defect detection algorithm and system based on improved YOLOv8

Abstract

The invention discloses an improved YOLOv8-based aircraft surface defect detection algorithm and system, and belongs to the technical field of aircraft detection. The method comprises the following steps: acquiring data by using an unmanned aerial vehicle carried with a high-resolution camera, and carrying out standardized labeling and enhanced preprocessing on the data; a double-layer routing attention mechanism is introduced on the basis of a YOLOv8 algorithm, a BiFormer universal visual converter is constructed, and the multi-scale feature extraction capability is enhanced; adopting a two-stage hyper-parameter optimization strategy and a mixed precision training technology to optimize the model; and a user-friendly software system is developed, and real-time docking between the unmanned aerial vehicle and software is realized. The method can be widely applied to the field of civil aviation, can accurately and efficiently detect various defects such as corrosion, scratches and cracks on the surface of an aircraft, automatically generates a detection report, provides decision support for maintenance personnel, improves the safety and reliability of an air transportation system, and can also be popularized to other fields such as surface detection of unmanned aerial vehicles and spacecrafts.

Description

Technical Field

[0001] The present invention relates to the field of aircraft detection technology, and in particular to an aircraft surface defect detection algorithm and system based on an improved YOLOv8. Background Art

[0002] In the aviation sector, aircraft surface defect detection is a critical component in ensuring flight safety. Traditional manual visual inspection methods have numerous drawbacks, including numerous blind spots, difficulty standardizing standards, and significant environmental impact. In the past three years, 65% of aircraft airworthiness issues caused by surface defects can be attributed to the inadequate performance of traditional inspection methods. Inspection solutions based on wall-climbing robots and wheeled robots are limited by efficiency and spatial constraints and have yet to be widely adopted. While drone-based inspection methods hold promise, existing computer vision-based inspection algorithms still suffer from low accuracy and limited ability to detect complex defects, failing to meet the aviation industry's stringent requirements for aircraft safety and reliability. Although patent CN117252839A proposes a fiber prepreg defect detection method based on an improved YOLO-v7 model, primarily using an industrial line array camera to capture surface defect images of fiber prepregs and utilizing the improved YOLO-v7 model for defect detection, its application scenarios are limited, it does not address the technical solution of using drone-mounted cameras for data acquisition, and its ability to detect complex defects is limited. Patent CN118505670A discloses a method for detecting aircraft skin defects by combining drones with deep learning. The method performs feature learning through an improved Encoder-Decoder network based on the ELAN module, and introduces the SimAM three-dimensional attention mechanism and the SPPCSPC module. However, the method is limited in technical details and does not involve optimization methods such as the dual-layer routing attention mechanism (BRA), the two-stage hyperparameter optimization strategy, and the mixed precision training technology. Furthermore, the method lacks a user-friendly software system, making it impossible to achieve real-time docking between drones and software, automatically generate inspection reports, and provide maintenance recommendations.

[0003] Therefore, there is an urgent need to develop an aircraft surface defect detection technology that is efficient, accurate and suitable for complex environments, so as to improve the inspection efficiency and safety of aircraft and reduce the safety risks caused by missed defects. Summary of the Invention

[0004] The present invention aims to provide an aircraft surface defect detection algorithm and system based on an improved YOLOv8, so as to improve the automation level, accuracy and efficiency of aircraft surface defect detection, reduce detection costs, and reduce safety hazards caused by missed defects, thereby meeting the safety and reliability requirements of the aviation transportation system.

[0005] The aircraft surface defect detection algorithm based on the improved YOLOv8 includes the following steps:

[0006] Step S1, data acquisition and preprocessing, collecting aircraft surface images, accurately annotating the collected images using image annotation tools, and using data enhancement techniques such as rotation, flipping, brightness adjustment, and CLAHE preprocessing to expand the data set;

[0007] Step S2: Model improvement: Based on the YOLOv8 model, a dual-layer routing attention mechanism is introduced to construct the BiFormer universal visual transformer. The dual-layer routing attention mechanism is also expressed as the BRA mechanism. BRA is used to perform feature map region division. According to the BRA network structure, the feature map is divided into S×S non-overlapping regions and linearly mapped. Attention weights are dynamically allocated through coarse-grained screening and fine-grained focusing. The weight calculation in the coarse-grained stage is based on the significance of the overall regional features, and the weight in the fine-grained stage is driven by data and adaptively optimized through end-to-end training.

[0008] Step S3, model training optimization, adopts a two-stage hyperparameter optimization strategy. In the first stage, the random search method is used to explore the preliminary configuration of hyperparameters, and in the second stage, the Bayesian optimization method is used for fine tuning. At the same time, the batch size is reduced to increase the frequency of model training feedback, and mixed precision training technology is used to speed up training.

[0009] Furthermore, in step S1, the acquisition of the aircraft surface image utilizes a high-resolution camera mounted on a drone to acquire the aircraft surface image from multiple angles and under different lighting conditions. The multi-angle acquisition targets difficult-to-reach parts of the aircraft, including the aircraft wings, aircraft tail, and upper part of the aircraft skin, and adopts low-altitude flight and rotating lens to obtain front and side images.

[0010] Furthermore, the data enhancement technology in step S1 includes: rotation, flipping, brightness adjustment, and CLAHE preprocessing, wherein the brightness adjustment changes the image brightness by setting an adjustment coefficient, and the CLAHE preprocessing is used to adjust the image brightness and contrast.

[0011] Furthermore, the coarse-grained screening method of step S2 is: dividing the feature map into non-overlapping local areas, each area corresponding to a spatial position; calculating the importance score of each area, specifically generating the initial attention weight through global average pooling; sorting according to the score, retaining the top 30% of the most important areas, and resetting the weights of the remaining areas to zero.

[0012] Furthermore, the fine-grained focusing method of step S2 is: for each retained area, extract its multi-scale features; introduce a channel attention module, and generate channel weights through three steps of feature compression, weight generation, and dynamic adjustment.

[0013] Furthermore, step S3 adopts a two-stage hyperparameter optimization strategy, specifically:

[0014] Step S3-1, using the random search method, explore the learning rate, batch size, momentum, weight decay, and loss function in the range of 0.0001-0.01, 32-128, 0.9-0.99, 0.0001-0.001, and 1.0-5.0, and select a parameter combination with average precision ≥ 0.7, recall rate ≥ 70%, inference speed ≥ 50ms / frame, and model parameter number ≤ 6M;

[0015] Step S3-2, use the Bayesian optimization method to perform fine tuning at learning rate = 0.0007, batch size = 64, momentum = 0.937, and weight decay = 0.0005, continuously adjust the hyperparameters, and observe the changes in the performance indicators of the model on the validation set until the detection effect is achieved. The learning effect is specifically: at the learning rate = 0.0005, the average precision, recall rate, and inference speed are 0.712, 70.3%, and 47.2ms / frame, respectively; at the learning rate = 0.0007, the average precision, recall rate, and inference speed are 0.725, 72%, and 46.5ms / frame, respectively; at the learning rate = 0.001, the average precision, recall rate, and inference speed are 0.698, 68.5%, and 45.8ms / frame, respectively.

[0016] In addition, the present invention also proposes an aircraft surface defect detection system based on an improved YOLOv8, wherein the aircraft surface defect detection system based on the improved YOLOv8 comprises:

[0017] The data acquisition module, consisting of a drone and a high-resolution camera, is used to collect images of the aircraft surface according to the preset flight path, altitude, and camera parameters;

[0018] The data processing module uses image annotation tools to accurately annotate the collected images and uses data enhancement technology to expand the data set to provide data support for model training;

[0019] The model detection module integrates the improved YOLOv8 model, receives preprocessed data for defect detection, and outputs defect location, classification information, and confidence level;

[0020] The software system module provides users with an operation interface with data input, parameter setting, result display, and data saving functions. It can achieve real-time docking with drones, automatically generate inspection reports, and provide maintenance suggestions.

[0021] Furthermore, the software system module supports image, image batch, and video input detection, with a detection speed of more than 30FPS.

[0022] Furthermore, it is characterized in that the result display includes visual annotation of the defect location, defect classification information and the confidence value of each detection result.

[0023] Compared with the prior art, the present invention adopts the above technical solution and has the following beneficial effects:

[0024] (1) The aircraft surface defect detection method and system based on improved YOLOv8 provided by the present invention are improved by applying the traditional YOLOv8 algorithm combined with the BRA mechanism. They can accurately identify and locate a variety of complex defects. The average precision on the validation set is 0.83, and the recall rate is 0.72, which is more than 40% higher than the traditional method. This improves the detection accuracy and effectively solves the problem of missed defect detection in complex backgrounds.

[0025] (2) The aircraft surface defect detection method and system based on the improved YOLOv8 provided by the present invention improves the detection accuracy of the model by 35% in complex environments such as low light, rain and fog through multi-illumination, multi-angle data acquisition and data enhancement technology, enhances environmental adaptability, and significantly improves generalization ability.

[0026] (3) The aircraft surface defect detection method and system based on improved YOLOv8 provided by the present invention, by combining the UAV platform with the deep learning model, can improve the detection efficiency by more than 5 times compared with manual visual inspection, reduce the cost of a single inspection by 60%, reduce the aircraft downtime, improve the operating efficiency of airlines, and significantly improve the detection efficiency.

[0027] (4) The aircraft surface defect detection method and system based on the improved YOLOv8 provided by the present invention has a user-friendly software system interface that is simple and easy to use, convenient to operate, and can directly output maintenance suggestions, providing maintenance personnel with efficient decision-making support. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is an aircraft surface defect detection method and system flow chart based on improved YOLOv8.

[0029] Figure 2 Figure 2 is a schematic diagram of the BRA network structure.

[0030] Figure 3 Schematic diagram of the BiFormer universal visual transformer built based on BRA.

[0031] Figure 4 This is a screenshot of the intelligent detection software interface.

[0032] Figure 5 This is a diagram of the aircraft surface defect detection method and system data acquisition process based on improved YOLOv8.

[0033] Figure 6A schematic diagram of the test results. DETAILED DESCRIPTION

[0034] Aircraft surface defect detection method based on improved YOLOv8, such as Figure 1 As shown, the following steps are included:

[0035] Step S1, data acquisition and preprocessing, collecting aircraft surface images, the acquisition process is as follows Figure 5 As shown, the collected images are accurately annotated using image annotation tools, and data enhancement techniques such as rotation, flipping, brightness adjustment, and CLAHE preprocessing are used to expand the dataset;

[0036] Step S2, model improvement, introduces a two-layer routing attention mechanism based on the YOLOv8 model to construct the BiFormer universal visual transformer, such as Figure 3 As shown, the introduction of the dual-layer routing attention mechanism is also expressed as the BRA mechanism, using BRA to perform feature map region division, according to the BRA network structure, as shown Figure 2 As shown in the figure, the feature map is divided into S×S non-overlapping regions and linearly mapped. Attention weights are dynamically assigned through coarse-grained screening and fine-grained focusing. The weight calculation in the coarse-grained stage is based on the significance of the overall features of the region, while the weight in the fine-grained stage is data-driven and adaptively optimized through end-to-end training.

[0037] The weight allocation logic is as follows: Weight calculations in the coarse-grained stage are based on the overall significance of the region's features, such as by measuring regional importance through activation strength or gradient response. The retained Topk regions dominate subsequent calculations, ensuring that key information is not overwhelmed by background noise. Weights in the fine-grained stage are data-driven and adaptively optimized through end-to-end training.

[0038] Step S3, model training optimization, adopts a two-stage hyperparameter optimization strategy. In the first stage, the random search method is used to explore the preliminary configuration of hyperparameters, and in the second stage, the Bayesian optimization method is used for fine tuning. At the same time, the batch size is reduced to increase the frequency of model training feedback, and mixed precision training technology is used to speed up training.

[0039] Furthermore, in step S1, the aircraft surface image is collected by using a high-resolution camera mounted on a drone to collect aircraft surface images from multiple angles and under different lighting conditions. The multi-angle collection is targeted at difficult-to-reach parts of the aircraft, including the aircraft wings, aircraft tail, and the upper part of the aircraft skin, by flying at low altitude and rotating the lens to obtain front and side images.

[0040] Furthermore, the data enhancement technology in step S1 includes: rotation, flipping, brightness adjustment, and CLAHE preprocessing, wherein the brightness adjustment changes the image brightness by setting an adjustment coefficient, and the CLAHE preprocessing is used to adjust the image brightness and contrast.

[0041] Furthermore, in step S1, a drone with stable flight performance and precise flight control capabilities is selected, equipped with a camera with high resolution, high frame rate, and good low-light performance. Based on a preset flight path, altitude, and camera shooting angle, all-around image capture is performed on the aircraft surface, ensuring coverage of all parts, including the wings, fuselage, and tail, and capturing image data under various lighting conditions.

[0042] Furthermore, in step S1, the Labelimg tool is used to accurately annotate the defects in the captured image, and the annotation content includes the defect type (corrosion, scratches, cracks, coating peeling, dents, etc.), location and bounding box information.

[0043] Furthermore, in step S1, the annotated data is enhanced using techniques such as rotation, flipping, brightness adjustment, and CLAHE preprocessing. For example, the image is rotated by a specific angle (30°, 90°, 150°, etc.), the brightness is adjusted (e.g., setting the brightness adjustment coefficient to 0.6-1.4), and adaptive histogram equalization is performed to increase the diversity of the dataset.

[0044] Furthermore, the coarse-grained screening method of step S2 is: dividing the feature map into non-overlapping local areas, each area corresponding to a spatial position; calculating the importance score of each area, specifically generating the initial attention weight through global average pooling; sorting according to the score, retaining the top 30% of the most important areas, namely the Topk areas, and resetting the weights of the remaining areas to zero.

[0045] Furthermore, the fine-grained focusing method of step S2 is: for each retained area, extract its multi-scale features; introduce a channel attention module, and generate channel weights through three steps of feature compression, weight generation, and dynamic adjustment.

[0046] Furthermore, the weight allocation logic is as follows: Weight calculations in the coarse-grained stage are based on the overall significance of the region's features, for example, by measuring regional importance through activation strength or gradient response. The retained Topk regions dominate subsequent calculations, ensuring that key information is not overwhelmed by background noise. Weights in the fine-grained stage are data-driven and adaptively optimized through end-to-end training.

[0047] Furthermore, step S3 adopts a two-stage hyperparameter optimization strategy. Specifically, first, a random search method is used to explore hyperparameters such as learning rate, batch size, and loss function over a large range to screen out parameter combinations with better performance. Then, a Bayesian optimization method is used to fine-tune the parameters near the optimal parameters, continuously adjusting the hyperparameters and observing the changes in the performance indicators of the model on the validation set until the optimal detection effect is achieved. Specifically:

[0048] Step S3-1, using the random search method, explore the learning rate, batch size, momentum, weight decay, and loss function in the range of 0.0001-0.01, 32-128, 0.9-0.99, 0.0001-0.001, and 1.0-5.0, and select a parameter combination with average precision ≥ 0.7, recall rate ≥ 70%, inference speed ≥ 50ms / frame, and model parameter number ≤ 6M;

[0049] Step S3-2, use the Bayesian optimization method to perform fine tuning at learning rate = 0.0007, batch size = 64, momentum = 0.937, and weight decay = 0.0005, continuously adjust the hyperparameters, and observe the changes in the performance indicators of the model on the validation set until the detection effect is achieved. The learning effect is specifically: at the learning rate = 0.0005, the average precision, recall rate, and inference speed are 0.712, 70.3%, and 47.2ms / frame, respectively; at the learning rate = 0.0007, the average precision, recall rate, and inference speed are 0.725, 72%, and 46.5ms / frame, respectively; at the learning rate = 0.001, the average precision, recall rate, and inference speed are 0.698, 68.5%, and 45.8ms / frame, respectively.

[0050] Furthermore, in step S3, mixed precision training is used during the training process, utilizing GPU-accelerated computation. The enhanced dataset is divided into a training set, a validation set, and a test set according to a certain ratio. The model is trained on the training set, and the model performance is evaluated on the validation set. The model parameters are adjusted based on the evaluation results, and finally the generalization ability of the model is tested on the test set.

[0051] In addition, the present invention also proposes an aircraft surface defect detection system based on an improved YOLOv8, wherein the aircraft surface defect detection system based on the improved YOLOv8 comprises:

[0052] The data acquisition module, consisting of a drone and a high-resolution camera, is used to collect images of the aircraft surface according to the preset flight path, altitude, and camera parameters;

[0053] The data processing module performs pre-processing operations such as annotation and data enhancement on the collected images to provide data support for model training;

[0054] The model detection module integrates the improved YOLOv8 model, receives preprocessed data for defect detection, and outputs defect location, classification information, and confidence level;

[0055] The software system module provides users with an operation interface with functions such as data input, parameter setting, result display, and data saving. It can realize real-time docking with drones, automatically generate inspection reports, and provide maintenance suggestions.

[0056] Furthermore, the software system module supports image, image batch, and video input detection, with a detection speed of more than 30FPS;

[0057] Furthermore, it is characterized in that the result display includes visual annotation of the defect location, defect classification information and the confidence value of each detection result.

[0058] Furthermore, using a suitable programming language (such as Python) and development framework, an intelligent inspection software system was developed. The system includes a data input module (supporting image, image batch, and video input), a model inference module (integrating a trained improved YOLOv8 model), a result display module (displaying defect location, classification information, confidence level, etc.), and a data storage module (saving inspection results and related data).

[0059] Furthermore, the UAV platform is integrated with the intelligent detection software system, such as Figure 4 As shown, this ensures that image data collected by drones can be transmitted to the software system in real time. After receiving the data, the software system uses model reasoning to perform defect detection and provides real-time feedback to the user. In actual applications, operators use the software system to control drone flight and data collection, and obtain inspection reports to provide a basis for aircraft maintenance.

Claims

1. Aircraft surface defect detection algorithm based on improved YOLOv8, characterized by: The steps include: Step S1, data collection and preprocessing, collecting aircraft surface images, accurately annotating the collected images using image annotation tools, and expanding the data set using data enhancement technology; Step S2: Model improvement: Based on the YOLOv8 model, a dual-layer routing attention mechanism is introduced to construct the BiFormer universal visual transformer. The dual-layer routing attention mechanism is also expressed as the BRA mechanism. BRA is used to perform feature map region division. According to the BRA network structure, the feature map is divided into S×S non-overlapping regions and linearly mapped. Attention weights are dynamically allocated through coarse-grained screening and fine-grained focusing. The weight calculation in the coarse-grained stage is based on the significance of the overall regional features, and the weight in the fine-grained stage is driven by data and adaptively optimized through end-to-end training. Step S3, model training optimization, adopts a two-stage hyperparameter optimization strategy. In the first stage, the random search method is used to explore the preliminary configuration of hyperparameters, and in the second stage, the Bayesian optimization method is used for fine tuning. At the same time, the batch size is reduced to increase the frequency of model training feedback, and mixed precision training technology is used to speed up training.

2. The aircraft surface defect detection algorithm based on improved YOLOv8 according to claim 1 is characterized in that: In step S1, the aircraft surface image is collected using a high-resolution camera mounted on a drone to collect aircraft surface images from multiple angles and under different lighting conditions. The multi-angle collection targets difficult-to-reach parts of the aircraft, including the aircraft's wings, aircraft tail, and the upper part of the aircraft skin, and uses low-altitude flight and rotating lenses to obtain front and side images.

3. The aircraft surface defect detection algorithm based on improved YOLOv8 according to claim 1 is characterized in that: The data enhancement technology in step S1 includes: rotation, flipping, brightness adjustment, and CLAHE preprocessing, wherein brightness adjustment changes the image brightness by setting an adjustment coefficient, and CLAHE preprocessing is used to adjust the image brightness and contrast.

4. The aircraft surface defect detection algorithm based on improved YOLOv8 according to claim 1 is characterized in that: The coarse-grained screening method of step S2 is as follows: dividing the feature map into non-overlapping local regions, each region corresponding to a spatial position; calculating the importance score of each region, specifically generating the initial attention weight through global average pooling; sorting by score, retaining the top 30% of the regions with the highest importance, and resetting the weights of the remaining regions to zero.

5. The aircraft surface defect detection algorithm based on improved YOLOv8 according to claim 1 is characterized in that: The fine-grained focusing method of step S2 is: for each retained area, extract its multi-scale features; The channel attention module is introduced to generate channel weights through three steps: feature compression, weight generation, and dynamic adjustment.

6. The aircraft surface defect detection algorithm based on improved YOLOv8 according to claim 1, characterized in that: The step S3 adopts a two-stage hyperparameter optimization strategy, specifically: Step S3-1, using the random search method, explore the learning rate, batch size, momentum, weight decay, and loss function in the range of 0.0001-0.01, 32-128, 0.9-0.99, 0.0001-0.001, and 1.0-5.0, and select a parameter combination with average precision ≥ 0.7, recall rate ≥ 70%, inference speed ≥ 50ms / frame, and model parameter number ≤ 6M; Step S3-2, use the Bayesian optimization method to perform fine tuning at learning rate = 0.0007, batch size = 64, momentum = 0.937, and weight decay = 0.0005, continuously adjust the hyperparameters, and observe the changes in the performance indicators of the model on the validation set until the detection effect is achieved. The learning effect is specifically: at the learning rate = 0.0005, the average precision, recall rate, and inference speed are 0.712, 70.3%, and 47.2ms / frame, respectively; at the learning rate = 0.0007, the average precision, recall rate, and inference speed are 0.725, 72%, and 46.5ms / frame, respectively; at the learning rate = 0.001, the average precision, recall rate, and inference speed are 0.698, 68.5%, and 45.8ms / frame, respectively.

7. Aircraft surface defect detection system based on improved YOLOv8, characterized by: The aircraft surface defect detection system based on the improved YOLOv8 is applied to the aircraft surface defect detection method based on the improved YOLOv8 according to any one of claims 1 to 6, and is characterized by comprising: The data acquisition module, consisting of a drone and a high-resolution camera, is used to collect images of the aircraft surface according to the preset flight path, altitude, and camera parameters; The data processing module uses image annotation tools to accurately annotate the collected images and uses data enhancement technology to expand the data set to provide data support for model training; The model detection module integrates the improved YOLOv8 model, receives preprocessed data for defect detection, and outputs defect location, classification information, and confidence level; The software system module provides users with an operation interface with data input, parameter setting, result display, and data saving functions. It can achieve real-time docking with drones, automatically generate inspection reports, and provide maintenance suggestions.

8. The aircraft surface defect detection system based on improved YOLOv8 according to claim 7, characterized in that: The software system module supports image, image batch, and video input detection, with a detection speed of over 30FPS.

9. The aircraft surface defect detection system based on improved YOLOv8 according to claim 7, characterized in that: The result display includes visual annotations showing the defect location, defect classification information, and confidence values ​​for each detection result.

Images