An aircraft skin defect detection method based on deep learning
By using a defect detection model based on Transformer and convolutional neural networks, and combining brightness generation and multi-scale detection branches, the robustness problem in complex lighting and multi-scale target detection is solved, achieving high-precision and high-recall aircraft skin defect detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
- Filing Date
- 2026-04-07
- Publication Date
- 2026-07-03
AI Technical Summary
Existing general-purpose target detection algorithms have poor robustness in complex lighting and multi-scale target detection, making it difficult to meet the requirements of high accuracy and high recall in the field of aviation safety.
A defect detection model based on Transformer and convolutional neural networks is adopted, which combines a brightness generation branch network and a multi-scale detection branch network. By training the model with an augmented dataset, the detection accuracy and recall rate of aircraft skin defects are improved.
It significantly improves the model's detection precision and recall under complex lighting and multi-scale targets, reduces the false negative rate, and improves the performance of the detection algorithm.
Smart Images
Figure CN122335752A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of defect detection technology, and in particular to a method, device, medium, and product for detecting defects in aircraft skin based on deep learning. Background Technology
[0002] Pre-flight inspections of aircraft require rapid and accurate visual confirmation of damage to the external fuselage skin. Traditional manual visual inspection methods are inefficient, susceptible to fatigue, and have a high risk of missed detections. Utilizing computer vision technology to achieve automated inspection is an inevitable trend.
[0003] However, automated visual inspection in the tarmac environment faces two major challenges: 1. Extremely complex lighting changes: strong glare, shadows, low illumination, metal reflections and other conditions seriously affect image quality and feature consistency; 2. Significant differences in target scale: the size of the parts to be inspected ranges from about 2 centimeters in diameter to several square meters in area, requiring the algorithm to have both fine local detail perception and global semantic understanding capabilities.
[0004] Existing general-purpose object detection algorithms (such as the YOLO series and DETR series) are mainly designed for general scenarios and have shortcomings in dealing with the specific challenges mentioned above: the YOLO series has poor robustness under complex lighting conditions and limited ability to detect small targets; the DETR series has slow training convergence, performs poorly on low-resolution small targets, and is sensitive to changes in lighting. These algorithms are difficult to meet the stringent requirements of high accuracy and high recall in the field of aviation safety. Summary of the Invention
[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an aircraft skin defect detection algorithm for complex lighting and multi-scale targets. This algorithm can effectively improve the accuracy, recall rate and robustness of aircraft skin defect detection in complex tarmac environments.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0007] A deep learning-based method for detecting aircraft skin defects includes:
[0008] Acquire the target image; the target image is an image of an aircraft skin defect;
[0009] The target image is input into the defect detection model to obtain an image with defect boundaries; the defect detection model is trained using an augmented dataset based on Transformer and convolutional neural networks; the augmented dataset is determined based on an aircraft skin defect dataset and a brightness generation branch network.
[0010] Optionally, the training process of the initial defect detection model includes:
[0011] Obtain the aircraft skin defect dataset; the aircraft skin defect dataset includes multiple training aircraft skin defect images and corresponding defect boundaries;
[0012] Construct the brightness generation branch network;
[0013] Supervised learning based on public datasets is used to train the brightness generation branch network on the aircraft skin defect dataset to obtain a data augmentation model;
[0014] The aircraft skin defect dataset is input into the data augmentation model to obtain the augmented dataset;
[0015] Construct a multi-scale detection branch network;
[0016] A defect detection model was obtained by training the model on the Transformer and convolutional neural network using an augmented dataset.
[0017] Optionally, the brightness generation branch network includes: two brightness denoising modules and two brightness fitting modules;
[0018] The brightness denoising module includes: a fast spatial pyramid pooling unit, a residual block, and an upsampling unit;
[0019] The brightness fitting module includes: a pooling unit, a residual block, and an activation unit.
[0020] The brightness denoising module is used to convert the input RGB image to the YUV color space and separate the brightness channel Y and the chroma channels U and V. For the brightness channel Y, a lightweight convolutional neural network is used for enhancement processing to improve the details in dark areas and suppress overexposed areas. For the chroma channels U and V, a denoising network is used to process them to eliminate color noise.
[0021] The brightness fitting module is used to merge the processed Y, U, and V channels, convert them back to RGB space, and output an image optimized for lighting conditions.
[0022] Optionally, the multi-scale detection branch network includes: an illumination sensing modulation module, a hybrid feature extraction module, a global context coding module, and a multi-scale feature fusion and prediction module;
[0023] The illumination sensing modulation module is used to receive the original image and the image after brightness sensing enhancement branch processing. By introducing a weighted grayscale channel based on physical illumination prior, a four-channel feature representation that is more robust to illumination changes is constructed.
[0024] The hybrid feature extraction module adopts a hierarchical structure that combines standard convolution and wavelet convolution to extract multi-scale deep features from the modulated features, thereby enhancing the feature representation capability while suppressing high-frequency noise.
[0025] The global context encoding module is used to encode deep features based on the Transformer encoder architecture and to establish feature dependencies in the global scope of the image using a self-attention mechanism.
[0026] The multi-scale feature fusion and prediction module is used to fuse the global features output by the Transformer encoder with the mid-layer features of the hybrid multi-scale feature extraction network, and then output the final key component detection results through a Transformer decoder, including class labels, bounding box coordinates and state attributes.
[0027] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the deep learning-based aircraft skin defect detection method described above.
[0028] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the deep learning-based aircraft skin defect detection method described above.
[0029] A computer program product includes a computer program that, when executed by a processor, implements the steps of the deep learning-based aircraft skin defect detection method described above.
[0030] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0031] This invention discloses a deep learning-based method for detecting aircraft skin defects. First, a target image is acquired; the target image is an image of an aircraft skin defect. The target image is then input into a defect detection model to obtain an image with defect boundaries. The defect detection model is trained using an augmented dataset based on Transformer and convolutional neural networks. The augmented dataset is determined based on an aircraft skin defect dataset and a brightness generation branch network. By introducing a collaborative mechanism between the brightness generation branch and the multi-scale detection branch, this invention overcomes the dual challenges of extreme lighting variations and significant target scale differences in complex apron environments. This significantly improves the model's accuracy and recall for detecting aircraft skin defects, reducing the false negative rate while maintaining high inference speed, thus enhancing the performance of the detection algorithm. Attached Figure Description
[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0033] Figure 1 This is a schematic diagram of the deep learning-based aircraft skin defect detection method provided in Embodiment 1 of the present invention.
[0034] Figure 2 A schematic diagram illustrating the specific principles of the aircraft skin defect detection process;
[0035] Figure 3 This is a schematic diagram of the first data in the aircraft skin defect dataset.
[0036] Figure 4 This is a schematic diagram of the second data in the aircraft skin defect dataset.
[0037] Figure 5 A schematic diagram of the third data in the aircraft skin defect dataset;
[0038] Figure 6 This is a schematic diagram of the fourth data point in the aircraft skin defect dataset.
[0039] Figure 7 This is a schematic diagram of the fifth data point in the aircraft skin defect dataset.
[0040] Figure 8 This is a schematic diagram of the sixth data point in the aircraft skin defect dataset.
[0041] Figure 9 This is a schematic diagram of the seventh data point in the aircraft skin defect dataset.
[0042] Figure 10 This is a schematic diagram of the eighth data point in the aircraft skin defect dataset.
[0043] Figure 11 This is a schematic diagram of the ninth data point in the aircraft skin defect dataset.
[0044] Figure 12 This is a schematic diagram of the tenth data point in the aircraft skin defect dataset.
[0045] Figure 13 This is a schematic diagram of the eleventh data point in the aircraft skin defect dataset.
[0046] Figure 14 This is a schematic diagram of the twelfth data point in the aircraft skin defect dataset.
[0047] Figure 15This is a schematic diagram of a multi-scale detection network structure;
[0048] Figure 16 A schematic diagram of the branch network structure for brightness generation;
[0049] Figure 17 Image showing the damage detection results;
[0050] Figure 18 This is a diagram showing the damage detection results;
[0051] Figure 19 This is a diagram showing the damage detection results;
[0052] Figure 20 This is a diagram showing the damage detection results;
[0053] Figure 21 This is a diagram showing the damage detection results. Detailed Implementation
[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0055] The purpose of this invention is to provide a method, device, medium, and product for detecting aircraft skin defects based on deep learning, aiming to improve the accuracy of aircraft skin defect detection.
[0056] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0057] Example 1
[0058] like Figures 1-2 As shown, the deep learning-based aircraft skin defect detection method in this embodiment includes:
[0059] Step 101: Obtain the target image; the target image is an image of the aircraft skin defect.
[0060] Step 102: Input the target image into the defect detection model to obtain an image with defect boundaries.
[0061] The defect detection model is trained using an augmented dataset based on Transformer and convolutional neural networks; the augmented dataset is determined based on an aircraft skin defect dataset and a brightness generation branch network.
[0062] As an optional implementation method, the training process of the defect detection model includes:
[0063] Step 1021: Obtain the aircraft skin defect dataset; the aircraft skin defect dataset includes multiple training images of aircraft skin defects and their corresponding defect boundaries.
[0064] Specifically, image acquisition equipment is used to obtain a dataset of aircraft skin defects. A portion of this dataset is shown below. Figures 3-14 As shown.
[0065] When it comes to deep learning models, the dataset plays a crucial role. A high-quality dataset that reflects real-world scenarios will significantly improve the accuracy of deep learning models. Therefore, the aircraft skin defect dataset to be used should accurately describe the damage caused by aircraft skin defects.
[0066] Step 1022: Construct the luminance generation branch network.
[0067] Step 1023: Using supervised learning based on a public dataset, train the brightness generation branch network on the aircraft skin defect dataset to obtain a data augmentation model.
[0068] Specifically, the loss function of the luminance generation branch network is as follows:
[0069]
[0070] in to To balance the hyperparameters of different loss components, Indicates smoothing loss. Perceptual loss is used to measure the similarity of high-level features. To calculate the histogram distribution loss of pixel intensity in the predicted image and the real image, The noise suppression loss represents the peak signal-to-noise ratio constraint. This represents the loss for minimizing the mean difference of the RGB channels. This represents the multi-scale structural similarity index loss.
[0071] The training phase is essentially a supervised learning phase. By using public datasets to establish supervisory information, the parameters of the brightness generation branch network are continuously optimized, and the loss function of the brightness generation branch network is reduced. When the loss function is reduced to a certain threshold, the dataset augmentation of the aircraft skin defect dataset can be completed based on the trained model.
[0072] Step 1024: Input the aircraft skin defect dataset into the data augmentation model to obtain the augmented dataset.
[0073] Step 1025: Construct a multi-scale detection network.
[0074] Step 1026: Use the augmented dataset to train the Transformer and convolutional neural network to obtain the defect detection model.
[0075] As an optional implementation method, such as Figure 16 As shown, the brightness generation branch network includes two brightness denoising modules and a brightness fitting module.
[0076] The brightness denoising module includes: a fast spatial pyramid pooling unit, a residual block, and an upsampling unit;
[0077] The brightness fitting module includes: a pooling unit, a residual block, and an activation unit.
[0078] Specifically, the luminance generation branch network proposed in this invention is similar to classic image processing networks, consisting of a forward propagation path, and achieves image enhancement through hierarchical feature transformation. However, unlike most single-path enhancement networks, the luminance generation branch adopts a dual-path parallel processing architecture, optimizing luminance and chrominance information separately. The two processing paths are the luminance enhancement path and the chrominance denoising path, respectively.
[0079] The brightness enhancement path specifically handles the brightness component of the image. First, the input RGB image is converted to the YUV color space, separating the brightness channel Y. The brightness channel then passes through a 3×3 convolutional layer to extract shallow features, followed by a lightweight convolutional attention module to enhance key feature representations. Finally, an optimized brightness component is output through a 3×3 convolutional layer. This path preserves the main structural information of the image while significantly improving the illumination distribution.
[0080] The chroma denoising path is specifically designed to handle the chroma component. The U and V channels, separated from the YUV space, are input into a lightweight denoising module. This module consists of cascaded 3×3 standard convolutional layers and depthwise separable convolutional layers, employing a U-shaped encoder-decoder structure and integrating spatial pyramid pooling and channel attention mechanisms as bottleneck layers. This path effectively suppresses noise interference in the chroma channels through multi-scale feature fusion and attention guidance.
[0081] The optimized luminance component and the denoised chrominance component are recombine in the YUV space, and the final enhanced RGB image is obtained through color space conversion. Through the synergistic effect of contrast enhancement in the luminance channel and noise suppression in the chrominance channel, the output image significantly improves visual quality and its support for subsequent detection tasks while maintaining natural colors.
[0082] As an optional implementation method, such as Figure 15As shown, the multi-scale detection network includes: an illumination-sensing modulation module, a hybrid feature extraction module, a global context encoding module, and a multi-scale feature fusion and prediction module;
[0083] The illumination sensing modulation module is used to receive the original image and the image processed by... Figure 16 The image processed by the brightness perception enhancement branch is shown to construct a four-channel feature representation that is more robust to changes in illumination by introducing a weighted grayscale channel based on physical illumination prior.
[0084] The hybrid feature extraction module adopts a hierarchical structure that combines standard convolution and wavelet convolution to extract multi-scale deep features from the modulated features, thereby enhancing the feature representation capability while suppressing high-frequency noise.
[0085] The global context encoding module is used to encode deep features based on the Transformer encoder architecture and to establish feature dependencies in the global scope of the image using a self-attention mechanism.
[0086] The multi-scale feature fusion and prediction module is used to fuse the global features output by the Transformer encoder with the mid-level features of the hybrid multi-scale feature extraction network, and then pass them through a Transformer decoder to output the final detection result, including class label, bounding box coordinates, and state attributes. The detection result is as follows: Figures 17-21 As shown.
[0087] Example 2
[0088] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the deep learning-based aircraft skin defect detection method in Embodiment 1.
[0089] Example 3
[0090] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the deep learning-based aircraft skin defect detection method in Embodiment 1.
[0091] Example 4
[0092] A computer program product includes a computer program that, when executed by a processor, implements the steps of the deep learning-based aircraft skin defect detection method in Embodiment 1.
[0093] Example 5
[0094] A computer device, which may be a database, includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores pending transactions. The I / O interfaces facilitate information exchange between the processor and external devices. The communication interface allows communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the deep learning-based aircraft skin defect detection method described in Embodiment 1.
[0095] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0096] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0097] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0098] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for detecting aircraft skin defects based on deep learning, characterized in that, The method includes: Acquire the target image; the target image is an image of an aircraft skin defect; The target image is input into the defect detection model to obtain an image with defect boundaries; the defect detection model is trained using an augmented dataset based on Transformer and convolutional neural networks; the augmented dataset is determined based on an aircraft skin defect dataset and a brightness generation branch network.
2. The aircraft skin defect detection method based on deep learning according to claim 1, characterized in that, The training process of the defect detection model includes: Obtain an aircraft skin defect dataset; the aircraft skin defect dataset includes multiple training images of aircraft skin defects and corresponding defect boundaries; Construct a luminance generation branch network; By utilizing supervised learning based on public datasets, a brightness generation branch network is trained on an aircraft skin defect dataset to obtain a data augmentation model; The aircraft skin defect dataset is input into the data augmentation model to obtain the augmented dataset; Construct a multi-scale detection network; A defect detection model was obtained by training the model on the Transformer and convolutional neural network using an augmented dataset.
3. The aircraft skin defect detection method based on deep learning according to claim 1, characterized in that, The brightness generation branch network includes: two brightness denoising modules and a brightness fitting module; The brightness denoising module includes: a fast spatial pyramid pooling unit, a residual block, and an upsampling unit; The brightness fitting module includes: pooling units, residual blocks, and activation units.
4. The aircraft skin defect detection method based on deep learning according to claim 1, characterized in that, The multi-scale detection network includes: an illumination sensing modulation module, a hybrid feature extraction module, a global context coding module, and a multi-scale feature fusion and prediction module; The illumination-sensing modulation module is used to receive the original image and the image after brightness-sensing enhancement branch processing. By introducing a weighted grayscale channel based on physical illumination prior, it constructs a four-channel feature representation that is more robust to illumination changes. The hybrid feature extraction module adopts a hierarchical structure that combines standard convolution and wavelet convolution to extract multi-scale deep features from the modulated features, thereby enhancing the feature expressive power while suppressing high-frequency noise. The global context encoding module is used to encode deep features based on the Transformer encoder architecture and to establish feature dependencies in the global scope of the image using a self-attention mechanism. The multi-scale feature fusion and prediction module is used to fuse the global features output by the Transformer encoder with the mid-level features of the hybrid multi-scale feature extraction network, and then through a Transformer decoder, output the final key component detection results, including class labels, bounding box coordinates and state attributes.
5. A computer device, comprising: The memory, the processor, and the computer program stored in the memory and executable on the processor are characterized in that the processor executes the computer program to implement the steps of the deep learning-based aircraft skin defect detection method according to any one of claims 1-6.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the deep learning-based aircraft skin defect detection method according to any one of claims 1-6.
7. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the deep learning-based aircraft skin defect detection method according to any one of claims 1-6.