Unmanned aerial vehicle paint surface defect detection method, device and equipment based on deep learning and storage medium

Abstract

The application discloses a deep learning-based unmanned aerial vehicle paint surface defect detection method and device, equipment and a storage medium, relates to the field of unmanned aerial vehicles, and is applied to a terminal device carrying a target detection model and comprises the following steps: acquiring original image data of a target unmanned aerial vehicle; processing the original image data by using a feature extraction network of the target detection model based on a target reflection suppression mechanism to obtain a target feature map; determining a target candidate frame corresponding to the target feature map by using a region candidate network of the target detection model based on a target anchor frame generation mechanism, so as to determine a corresponding target classification result and a target regression result based on the target candidate frame, and determine a corresponding target region of interest; and processing the target region of interest by using a target detection mechanism corresponding to a defect morphology of the target region of interest based on the target detection model, so as to obtain a paint surface defect detection result corresponding to the target unmanned aerial vehicle. The application realizes high-precision detection of the paint surface defects of the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicles (UAVs), and in particular to a method, apparatus, equipment, and storage medium for detecting defects in the painted surfaces of UAVs based on deep learning. Background Technology

[0002] Current paint defect assessment technologies primarily rely on manual visual inspection, which is not only inefficient and lacks real-time performance, but also suffers from high subjectivity due to differences in operator experience. This makes it difficult to meet the quality and efficiency requirements of modern mass production lines. Furthermore, existing deep learning-based paint defect detection technologies (such as the SSD (Single Shot MultiBox Detector) and YOLO (You Only Look Once) algorithms) directly predict defect categories and locations through regression, failing to accurately extract defect boundary contours (e.g., difficulty in distinguishing overlapping or close defects). This results in incomplete defect feature descriptions, impacting detection accuracy.

[0003] In conclusion, how to achieve high-precision detection of defects in the paint surface of drones is an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for detecting defects in the paint surface of drones based on deep learning, capable of achieving high-precision detection of defects in the paint surface of drones. The specific solution is as follows:

[0005] In a first aspect, this application discloses a deep learning-based method for detecting defects in the painted surfaces of drones, applied to a terminal device equipped with a target detection model, comprising:

[0006] The original image data of the target drone is acquired, and the original image data is processed by the feature extraction network of the target detection model based on the target reflection suppression mechanism to obtain the corresponding target feature map; the original image data is the painted surface image data of the target drone.

[0007] Based on the target anchor box generation mechanism, the region candidate network of the target detection model is used to determine the target candidate box corresponding to the target feature map, so as to determine the target classification result and target regression result corresponding to the target feature map based on the target candidate box;

[0008] The target detection model is used to map the target candidate box to the target feature map based on the target classification result and the target regression result to determine the corresponding target region of interest. The target detection model is then used to process the target region of interest based on the target detection mechanism corresponding to the defect morphology of the target region of interest to obtain the paint surface defect detection result corresponding to the target UAV.

[0009] Optionally, the target reflection suppression mechanism utilizes the feature extraction network of the target detection model to process the original image data to obtain a corresponding target feature map, including:

[0010] The original image data is converted to the target color space to determine the converted image data, and the corresponding brightness distribution data and scene brightness mean are determined.

[0011] Based on the brightness distribution data, the converted image data is segmented using a dual threshold to determine the highlight area image data and shadow area image data corresponding to the converted image data.

[0012] The target Gamma coefficient is determined based on the average scene brightness. The highlight area image data is compressed and mapped based on the target Gamma coefficient to obtain the corresponding first balanced data. The shadow area image data is enhanced and mapped based on the target Gamma coefficient to obtain the corresponding second balanced data. The balanced image data corresponding to the original image data is obtained based on the first balanced data and the second balanced data.

[0013] The balanced image data is processed using the feature extraction network of the target detection model to obtain the corresponding target feature map.

[0014] Optionally, the step of processing the original image data using the feature extraction network of the target detection model to obtain the corresponding target feature map includes:

[0015] The first feature extraction network of the target detection model is used to perform cross-layer connection processing on the original image data to obtain the corresponding initial feature map;

[0016] The second feature extraction network of the target detection model is used to fuse all the initial feature maps corresponding to the original image data to obtain the corresponding target feature map;

[0017] The feature extraction network includes a first feature extraction network and a second feature extraction network, wherein the first feature extraction network is a residual neural network and the second feature extraction network is a feature pyramid network.

[0018] Optionally, the target anchor box generation mechanism utilizes the region candidate network of the target detection model to determine the target candidate box corresponding to the target feature map, including:

[0019] Based on the target anchor box generation mechanism, the region candidate network of the target detection model is used to determine the first candidate box and the first confidence score corresponding to the target feature map;

[0020] The target feature map is processed using a pre-defined Anchor-Free detection model to determine the corresponding second candidate box and second confidence score;

[0021] The target candidate box corresponding to the target feature map is determined by using a preset weighting mechanism based on the first candidate box, the first confidence score, the second candidate box, and the second confidence score.

[0022] Optionally, the target anchor box generation mechanism utilizes the region candidate network of the target detection model to determine the first candidate box and the first confidence score corresponding to the target feature map, including:

[0023] Based on the target anchor box generation mechanism, the region candidate network of the target detection model is used to determine the initial candidate box corresponding to the target feature map;

[0024] The initial candidate box is calibrated based on the target reflection suppression mechanism to obtain the first candidate box corresponding to the target feature map, and the first confidence score corresponding to the first candidate box is determined.

[0025] Optionally, the step of using the target detection model based on the target detection mechanism corresponding to the defect morphology of the target region of interest to process the target region of interest to obtain the paint surface defect detection result corresponding to the target UAV includes:

[0026] The target detection model is used to classify and regress the region of interest of the target to obtain the corresponding classification output and bounding box regression output.

[0027] Using the target detection model based on the target detection mechanism corresponding to the defect morphology of the target region of interest, a masking process is performed on the target region of interest to obtain the masking output result corresponding to the target UAV.

[0028] The target total loss function is used to determine the paint surface defect detection result corresponding to the target UAV based on the classification output result, the bounding box regression output result, and the mask output result.

[0029] Optionally, the step of using the target detection model based on the target detection mechanism corresponding to the defect morphology of the target region of interest to perform masking processing on the target region of interest includes:

[0030] If the defect morphology of the target region of interest is a drip morphology, then a directional gradient regularization term is added to the target region of interest to perform masking processing on the target region of interest based on the added directional gradient regularization term;

[0031] If the defect morphology of the target region of interest is orange peel, then the local texture statistical features and frequency domain features of the target region of interest are fused to obtain fused texture features, and the target region of interest is masked based on the fused texture features.

[0032] If the defect morphology of the target region of interest is a scratch morphology, then the target region of interest is masked using Sobel gradient sparsity constraints;

[0033] If the defect morphology of the target region of interest is a bubble morphology or a shrinkage cavity morphology, then the target region of interest is masked using cyclic consistency loss.

[0034] Secondly, this application discloses a deep learning-based device for detecting defects in the painted surfaces of drones, applied to a terminal device equipped with a target detection model, comprising:

[0035] The feature map acquisition module is used to acquire the original image data of the target drone, and process the original image data using the feature extraction network of the target detection model based on the target reflection suppression mechanism to obtain the corresponding target feature map; the original image data is the painted surface image data of the target drone.

[0036] The feature map processing module is used to determine the target candidate box corresponding to the target feature map using the region candidate network of the target detection model based on the target anchor box generation mechanism, so as to determine the target classification result and target regression result corresponding to the target feature map based on the target candidate box;

[0037] The defect detection module is used to map the target candidate box and the target feature map based on the target classification result and the target regression result using the target detection model to determine the corresponding target region of interest, and to process the target region of interest based on the target detection mechanism corresponding to the defect morphology of the target region of interest using the target detection model to obtain the paint surface defect detection result corresponding to the target UAV.

[0038] Thirdly, this application discloses an electronic device, including:

[0039] Memory, used to store computer programs;

[0040] A processor is used to execute the computer program to implement the aforementioned deep learning-based method for detecting defects in the painted surfaces of drones.

[0041] Fourthly, this application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned deep learning-based method for detecting defects in the painted surfaces of unmanned aerial vehicles.

[0042] In this application, when detecting defects in the painted surface of a drone based on deep learning, a terminal device equipped with a target detection model acquires the original image data of the target drone, and processes the original image data using the feature extraction network of the target detection model based on a target reflection suppression mechanism to obtain a corresponding target feature map; the original image data is the painted surface image data of the target drone; a target candidate box corresponding to the target feature map is determined using the region candidate network of the target detection model based on a target anchor box generation mechanism, and a target classification result and a target regression result corresponding to the target feature map are determined based on the target candidate box; the target detection model maps the target candidate box and the target feature map based on the target classification result and the target regression result to determine the corresponding target region of interest, and processes the target region of interest based on the target detection mechanism corresponding to the defect morphology of the target region of interest to obtain the paint surface defect detection result corresponding to the target drone. As can be seen, after acquiring the original image data of the UAV, this application utilizes a target reflection suppression mechanism to enhance the defect texture features in the image and suppress false reflection features. It then uses the feature extraction network of the target detection model to achieve multi-scale feature extraction, thereby obtaining the corresponding target feature map. Next, based on the target anchor box generation mechanism, the region candidate network of the target detection model is used to determine the target candidate box corresponding to the target feature map, achieving high recall detection of small targets and thin scratches. Then, based on the target classification and regression results, the target candidate box and the target feature map are mapped to determine the corresponding target region of interest, achieving feature alignment. Finally, the target detection model uses a target detection mechanism based on the defect morphology corresponding to the target region of interest to process the target region of interest, obtaining the detection results of the paint surface defects of the target UAV, thus achieving accurate detection of paint surface defects on the UAV. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0044] Figure 1 This application discloses a flowchart of a deep learning-based method for detecting defects in the painted surfaces of drones.

[0045] Figure 2 This is a schematic diagram of the specific architecture of an object detection model disclosed in this application;

[0046] Figure 3 This is a schematic diagram of the specific structure of the backbone extraction network of an object detection model disclosed in this application;

[0047] Figure 4 This is a schematic diagram of a specific candidate box mapping process disclosed in this application;

[0048] Figure 5 This is a schematic diagram illustrating a specific intersection-union ratio (IURR) solution process disclosed in this application;

[0049] Figure 6 This is a schematic diagram of a specific confusion matrix between the predicted result and the actual result disclosed in this application;

[0050] Figure 7 This is a schematic diagram illustrating a specific target detection model training process disclosed in this application;

[0051] Figure 8 This is a schematic diagram of the structure of a deep learning-based drone paint surface defect detection device disclosed in this application;

[0052] Figure 9 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Existing paint defect assessment technologies primarily rely on manual visual inspection, which is not only inefficient and lacks real-time performance, but also suffers from high subjectivity due to differences in operator experience. This makes it difficult to meet the quality and efficiency requirements of modern mass production lines. Furthermore, existing deep learning-based paint defect detection technologies (such as the SSD and YOLO algorithms) directly predict defect categories and locations through regression, failing to accurately extract defect boundary contours (e.g., difficulty in distinguishing overlapping or close defects), resulting in incomplete defect feature descriptions and affecting detection accuracy. To address these technical problems, this application discloses a deep learning-based method for detecting paint defects on UAVs, enabling high-precision detection of paint defects on UAVs.

[0055] See Figure 1 As shown, this embodiment of the invention discloses a method for detecting defects in the painted surface of a drone based on deep learning, applied to a terminal device equipped with a target detection model, including:

[0056] Step S11: Obtain the original image data of the target drone, and process the original image data using the feature extraction network of the target detection model based on the target reflection suppression mechanism to obtain the corresponding target feature map; the original image data is the painted surface image data of the target drone.

[0057] In this embodiment, as Figure 2 The diagram illustrates the specific architecture of an object detection model. Based on a region-based convolutional neural network (Faster R-CNN), the RoI Pooling feature extraction algorithm is replaced with RoIAlign. Furthermore, a fully convolutional network (FCN) layer is added to the object detection layer to couple detection and segmentation, resulting in a Mask R-CNN (Mask Region-based Convolutional Neural Network) model. Further improvements to the Mask R-CNN model lead to the object detection model. This improved Mask R-CNN model can be deployed on GPU servers or embedded AI acceleration platforms to achieve pixel-level defect segmentation and multi-class defect detection on the painted surfaces of drones, solving the problems of blurred defect boundaries and difficulty in distinguishing overlapping defects.

[0058] In one specific implementation, the target detection model includes the following modules:

[0059] (1) Image input module: Receives the painted surface image data of the target UAV as the raw image data; wherein, the raw image data can be the raw image of the painted surface obtained by using a high-resolution industrial camera (resolution not less than 1024×1024) with an adjustable LED light source and an anti-reflective filter. The camera and the painting robot arm are triggered synchronously to ensure that the imaging angle corresponds to the painting angle.

[0060] (2) Backbone extraction network module: also known as feature extraction network module, which uses ResNet50-FPN to extract multi-scale features and embeds CCAM (Color Channel Attention and Reflection Suppression Module) in each layer of FPN;

[0061] (3) Region Proposal Network Module: Combining the DA-RPN (Dynamic-Anchor Region Proposal Network) mechanism, candidate boxes (RoI) are generated and preliminary classification and regression are performed;

[0062] (4) RoI Align module: Eliminates quantization error and improves feature alignment accuracy;

[0063] (5) Header network module: Outputs defect classification, bounding box and segmentation mask;

[0064] (6) Morphological branching module: Performs differentiated mask learning for different defect types;

[0065] (7) Results output module: Displays the test results and associates them with the spraying process parameters.

[0066] In this embodiment, a color channel attention and reflection suppression module is introduced between the input end of the traditional Mask R-CNN model and the feature layer of FPN (Feature Pyramid Networks). By converting the RGB image to color spaces such as HSV, YCbCr, or Lab, the brightness component is extracted and dynamic Gamma correction and reflection filtering are performed to suppress false detections caused by strong reflections. Thus, brightness modeling and adaptive correction are performed on input images with different lighting conditions, spraying angles, and surface brightness to achieve a target reflection suppression mechanism. In one specific implementation, the original image data is processed using a feature extraction network of a target detection model based on a target reflection suppression mechanism to obtain a corresponding target feature map. This includes: converting the original image data to a target color space to determine the converted image data, and determining the corresponding brightness distribution data and the average scene brightness; performing dual-threshold segmentation on the converted image data based on the brightness distribution data to determine the corresponding highlight area image data and shadow area image data; determining the target Gamma coefficient based on the average scene brightness, compressing and mapping the highlight area image data based on the target Gamma coefficient to obtain corresponding first balanced data, and enhancing and mapping the shadow area image data based on the target Gamma coefficient to obtain corresponding second balanced data; and obtaining balanced image data corresponding to the original image data based on the first and second balanced data; and processing the balanced image data using the feature extraction network of the target detection model to obtain a corresponding target feature map.

[0067] In one specific implementation, the target reflection suppression mechanism is implemented as follows:

[0068] (1) Brightness modeling and specular detection:

[0069] The input RGB image (i.e., the raw image data) is converted to the YCbCr or Lab color space, the luminance channel Y / L is extracted, and the global average luminance is calculated. and standard deviation Perform dual-threshold segmentation on the brightness distribution; values ​​higher than... The pixels below are considered highlight areas, and those below are considered highlight areas. The pixels are considered as shadow areas.

[0070] (2) Adaptive Gamma correction:

[0071] Calculate the Gamma coefficient based on the average scene brightness. :

[0072] ;

[0073] in For reference brightness, The adjustment factor is (0.5~1.0). Apply to the highlight areas. Compression mapping with a value greater than 1 yields the first balanced data, which is then applied to the shaded area. The enhanced mapping with a value less than 1 yields the second balanced data, achieving dynamic balance of overall exposure.

[0074] (3) Local reflection suppression and normalization:

[0075] A guided filter is applied to the highlight areas (r ranges from 4 to 8, ε=1e). 3) Alternatively, perform bilateral filtering for local smoothing, preserving edge details while reducing reflection saturation, and then perform local brightness normalization on the compensated image:

[0076] ;

[0077] in, Brightness before normalization This is the normalized brightness.

[0078] Finally, the normalized result is recombine with the original chroma channel to form balanced image data, which is then used as input features for the backbone extraction network to extract stable features. Based on this target reflection suppression mechanism, the illumination distribution of the input image can be modeled and adaptively corrected in real time using the brightness features of the sprayed surface as constraints. This effectively solves problems such as unstable image brightness, local overexposure, or underexposure caused by changes in incident angle, ambient light differences, and uneven coating thickness during the spraying process. It achieves brightness normalization from the original imaging domain to the feature domain, ensuring the stability and consistency of the input features. In addition, this embodiment can also perform image denoising, geometric normalization, and standardization operations, resampling the image data by size and normalizing the pixel intensity to generate enhanced image input in conjunction with the target reflection suppression mechanism. This provides the backbone extraction network with optimal feature quality, achieving standardized input of image brightness and color.

[0079] In this embodiment, as Figure 3 The diagram shows a schematic of the backbone extraction network of a target detection model. The backbone extraction network (also known as the feature extraction network) is used to extract the depth features of the original image data of the target UAV. It adopts the ResNet50-FPN structure and improves the multi-scale detection performance through cross-layer connections and feature pyramid fusion.

[0080] In one specific implementation, the ResNet50 network of the backbone extraction network extracts image features through cross-layer connections, and the image undergoes five convolution operations (e.g., ...) in the ResNet50 network. Figure 3 (As shown in C1-C5) Extract target features to obtain 5 feature maps of different sizes (e.g.) Figure 3As shown in pages 2-6, this method deepens the neural network without compromising feature extraction performance, making training easier. Typically, high-resolution images containing more detailed information are used as input to improve object detection performance. The FPN introduces a top-down path, fully utilizing features extracted from different layers. It samples the same size as the next layer on the high-level feature map and adds it to the next layer, resulting in a new feature layer that fuses high and low-level features. This combines low-resolution, large receptive field features (semantic information) with high-resolution, small receptive field features (detail information) to detect objects at different scales. In other words, the feature extraction network of the object detection model processes the original image data to obtain corresponding target feature maps. Specifically, this can include: using the first feature extraction network of the object detection model to perform cross-layer connection processing on the original image data to obtain the corresponding initial feature map; and using the second feature extraction network of the object detection model to fuse all the initial feature maps corresponding to the original image data to obtain the corresponding target feature map. The feature extraction network includes a first feature extraction network and a second feature extraction network. The first feature extraction network is a residual neural network, and the second feature extraction network is a feature pyramid network.

[0081] Step S12: Based on the target anchor box generation mechanism, the region candidate network of the target detection model is used to determine the target candidate box corresponding to the target feature map, so as to determine the target classification result and target regression result corresponding to the target feature map based on the target candidate box.

[0082] In this embodiment, the Region Proposal Network (RPN) of the target detection model is used to generate candidate target regions on the entire feature map and is a key component of the entire detection framework. Its main function is to generate candidate boxes through a sliding window and combine these candidate regions with a convolutional network for classification and bounding box regression, thereby selecting high-confidence regions of interest (RoIs) to provide accurate target locations for the subsequent RoI Align module. After inputting the multi-scale features (i.e., the target feature map) extracted by the backbone extraction network in the previous steps into the RPN, the RPN uses a multi-layer convolutional structure to generate multiple candidate boxes (anchors) and simultaneously completes bounding box regression and foreground / background binary classification tasks. Specifically, the RPN uses a sliding window method to divide the input feature map into image blocks of different scales, performs binary classification (determining whether it contains a target) and regression (predicting the coordinate offset of the defect bounding box) on each block, and finally selects the candidate region (RoI) with the highest score as the input of the subsequent RoI Align module, realizing the connection between the initial localization of candidate boxes and feature extraction.

[0083] In one specific implementation, the method of determining the target candidate box corresponding to the target feature map using the region candidate network of the target detection model based on the target anchor box generation mechanism includes: determining the first candidate box and the first confidence score corresponding to the target feature map using the region candidate network of the target detection model based on the target anchor box generation mechanism; processing the target feature map using a preset Anchor-Free detection model to determine the corresponding second candidate box and the second confidence score; and determining the target candidate box corresponding to the target feature map based on the first candidate box, the first confidence score, the second candidate box, and the second confidence score using a preset weighting mechanism.

[0084] In this embodiment, the target anchor box generation mechanism can be a multi-scale dynamic anchor mechanism (Dynamic-Anchor RPN, DA-RPN). This mechanism uses IoU-k-means clustering to statistically analyze the real defect boxes in the training samples, automatically generating 9 to 12 initial anchor templates covering extreme aspect ratios such as 1:1, 1:3, 1:5, and 1:7 to accommodate various morphological defects on painted surfaces (such as long scratches and round bubbles). During DA-RPN training, the anchor size and ratio are adaptively updated based on the prediction residual distribution of each batch of samples (the exponential moving average coefficient α ranges from 0.6 to 0.9), achieving dynamic self-learning of the anchor box shape. This allows the model to automatically adjust the sampling strategy of the region of interest for different painting materials, lighting, and angle conditions. A priority sampling strategy is implemented for candidate boxes of small targets (area < 32*32 pixels), increasing their weight during training and improving the recall rate of minor defects. Meanwhile, to address the practical difficulties in detecting small-sized defects, fine scratches, and complex reflective areas on painted surfaces, this embodiment also utilizes a pre-defined Anchor-Free detection model to implement a hybrid detection strategy combining a multi-scale dynamic anchor mechanism (Dynamic-Anchor RPN, DA-RPN) and Anchor-Free, thereby achieving an innovative improvement over the traditional RPN. Specifically, a lightweight Anchor-Free detection branch (similar to the FCOS (Fully Convolutional One-Stage Object Detection) structure) is introduced in parallel at the P2 layer (high-resolution feature layer) of the FPN to detect extremely small or irregularly shaped targets. This branch predicts the target probability and regression offset through pixel center points, avoiding anchor quantization errors. During the inference stage, Anchor-Free and Anchor-Based results are fused through score weighting and Non-Maximum Suppression (NMS) to ensure a balance between recall and precision.

[0085] In this embodiment, the region candidate network of the target detection model is used to determine the first candidate box and the first confidence score corresponding to the target feature map based on the target anchor box generation mechanism. This includes: determining the initial candidate box corresponding to the target feature map using the region candidate network of the target detection model based on the target anchor box generation mechanism; calibrating the initial candidate box based on the target reflection suppression mechanism to obtain the first candidate box corresponding to the target feature map, and determining the first confidence score corresponding to the first candidate box. That is, in this embodiment, after obtaining the candidate box output by RPN for the first time, CCAM can be used to calibrate the candidate box, and the calibrated candidate box is used as the first candidate box finally output by the RPN network. Then, the first candidate box and the second candidate box output by the Anchor-Free auxiliary detection branch are weighted based on the confidence score to obtain the final candidate box. After the candidate box output by RPN is corrected again by CCAM, false candidates caused by reflection or brightness gradient can be further suppressed. At the same time, the high-scoring candidate box generated by DA-RPN and the multi-scale features of FPN are jointly input into the RoI Align module through cross-layer fusion (Top-Down + Lateral Path) to make the feature hierarchy and spatial position more accurately aligned.

[0086] In this embodiment, after obtaining the target candidate bounding box, the corresponding target classification result and target regression result can be determined based on the target candidate bounding box. In a specific implementation, the classification task uses Focal Loss (… The value range is 1.5 to 2.5 to address the imbalance between positive and negative samples and enhance the model's sensitivity to defects in small samples; bounding box regression uses GIoU or CIoU loss functions to improve the stability of fitting the boundaries of slender targets; the positive and negative sample thresholds can be set as follows: IoU not less than 0.6 is considered a positive sample, IoU not greater than 0.3 is considered a negative sample, and the rest are ignored; different types of defects are weighted according to their categories. An adaptive _class adjustment strategy is used to reduce misclassification of easily confused categories (such as dust and dirt). Multi-scale fusion detection is achieved by combining anchor-free branching with Focal Loss classification loss.

[0087] Step S13: Using the target detection model, the target candidate box and the target feature map are mapped based on the target classification result and the target regression result to determine the corresponding target region of interest. The target region of interest is then processed using the target detection model based on the target detection mechanism corresponding to the defect morphology of the target region of interest to obtain the paint surface defect detection result corresponding to the target UAV.

[0088] In this embodiment, as Figure 4As shown, when the object detection model maps candidate bounding boxes to the object feature map based on object classification and regression results to determine the corresponding region of interest (ROI), the RoI Align module is responsible for mapping the candidate bounding boxes to the original image, generating the ROI. The RoI Align process is as follows: Figure 3 As shown, the black dashed grid represents the feature map, the solid lines represent RoIs (a 2×2 grid in this example), and the blue dots represent the four sampling points in each grid. In one specific implementation, the ROI Align module eliminates the two quantization operations (candidate box boundary clipping and cell partitioning rounding) in RoI Pooling, accurately mapping candidate boxes to the feature map with floating-point numbers. It then uses bilinear interpolation to calculate the feature values ​​of the four sampling points within each k×k cell partition, and finally integrates the cell features through max pooling to determine the target region of interest. This approach, by preserving boundary continuity, introducing bilinear interpolation, and multi-point sampling, completely eliminates quantization errors, achieving precise alignment between candidate boxes and the feature map. This solves the region mismatch problem caused by boundary offset in RoI Pooling, significantly improving the accuracy of object detection and instance segmentation.

[0089] In this embodiment, after the target region of interest (ROI) enters the head network, detection output is achieved through classification, bounding box regression, and masking branches. Specifically, the target detection model utilizes a target detection mechanism based on the defect morphology of the ROI to process the ROI and obtain the paint surface defect detection result for the target UAV. This can include: classifying and performing bounding box regression on the ROI using the target detection model to obtain corresponding classification and bounding box regression output results; masking the ROI using the target detection model based on the defect morphology of the ROI to obtain the mask output result for the target UAV; and determining the structured paint surface defect detection result for the target UAV using the total loss function based on the classification output result, bounding box regression output result, and mask output result, thus achieving end-to-end optimization of the classification, localization, and segmentation tasks. The total loss function L consists of classification, regression, segmentation mask, and morphological regularization terms.

[0090] ;

[0091] in, For classifying losses, To regress the loss, For segmentation mask loss, For morphological branching loss, The weighting coefficient is 0.1~0.3.

[0092] In one specific implementation, a target detection model is used to perform masking processing on the target region of interest (ROI) based on the defect morphology of the target region of interest. This includes: if the defect morphology of the target region of interest is a drip morphology, a directional gradient regularization term is added to the target region of interest to perform masking processing, thereby ensuring the mask has continuity in the vertical direction; if the defect morphology of the target region of interest is an orange peel morphology, the local texture statistical features and frequency domain features of the target region of interest are fused to obtain fused texture features, and the target region of interest is masked based on the fused texture features to improve the accuracy of micro-texture recognition; if the defect morphology of the target region of interest is a scratch morphology, Sobel gradient sparsity constraints are used to perform masking processing on the target region of interest to maintain the continuity of the slender structure; if the defect morphology of the target region of interest is a bubble morphology or a pinhole morphology, cyclic consistency loss (such as roundness regularization and morphology consistency loss) is used to perform masking processing on the target region of interest to prevent missegmentation. These branches share features in the first two layers and learn corresponding morphological features in the final output layer, enabling the object detection model to independently learn for multiple types of defects. By designing a multi-branch mask segmentation structure in the RoI output stage, and applying direction, texture, or roundness regularization based on different defect morphologies, adaptive pixel-level instance segmentation and morphological constraint learning are achieved for different types of defects such as drips, orange peel, scratches, and bubbles. This allows the final paint surface defect detection result to simultaneously include structured information such as multiple defect categories, the bounding box coordinates and pixel-level segmentation mask corresponding to the defect, confidence scores, and morphological feature vectors.

[0093] In this embodiment, if the target detection model is currently in the model training or optimization learning phase, and the original image data of the target UAV is training data with known real defects in the painted surface, then after obtaining the paint surface defect detection result output by the target detection model, the paint surface defect detection result can be used as the model prediction result and compared with the corresponding real defects in the painted surface to optimize the target detection model. Specifically, the prediction results can be filtered by the IoU threshold, the classification performance can be quantified by combining accuracy and recall, and AP / mAP can be used as the final comprehensive index to construct a complete evaluation system from positioning accuracy to classification ability to multi-class detection effect, ensuring that the target detection model balances high accuracy and high recall in practical applications.

[0094] In one specific implementation, during the training process of object detection, the matching degree is quantified and the localization accuracy is measured by calculating the ratio of the overlap area to the union area between the predicted bounding box and the ground truth bounding box (or the predicted mask and the ground truth mask). In the detection domain, IoU represents the ratio of the intersection and union between the ground truth bounding box and the predicted bounding box. In the segmentation domain, IoU represents the overlap area between the ground truth mask and the predicted mask. A schematic diagram of IoU calculation is shown below. Figure 5 As shown. In practical applications, IoU > 0.5 and confidence > 0.5 can be set as the standard for correct prediction, ensuring that the model's prediction results meet the threshold requirements in both location and confidence, thus providing a basis for subsequent accuracy and recall calculations. Based on the IoU threshold screening, accuracy and recall are calculated using statistics such as TP, FP, and FN to further evaluate the classification ability of the object detection model. Accuracy reflects the proportion of correctly predicted positive samples, measuring the object detection model's ability to suppress false detections; recall reflects the proportion of correctly predicted real positive samples, measuring the object detection model's ability to avoid missed detections. The confusion matrix between the predicted results and the actual results is shown below. Figure 6 As shown. Accuracy corresponds to the ratio of correctly identified defects to the total number of defects identified, while recall corresponds to the ratio of correctly identified defects to the total number of defects. The two, together with the F1 score (the harmonic mean of the two), characterize the overall performance of the model.

[0095] It is understood that in this embodiment, positive samples for spray coating defect detection refer to painted surface images containing defects such as bubbles, dust, dirt, pinholes, runs, scratches, and orange peel. Negative samples refer to painted surface images that do not contain these seven defects, but may contain other types of defects or have no defects at all. The meanings of TP, FP, TN, and FN in this document are shown in Table 1 below:

[0096]

[0097] Building upon this, in this embodiment, the average precision performance of the target detection model under different recall rates can also be evaluated by plotting a precision-recall curve and calculating the area under the curve. A higher AP value for a single class indicates better performance of the target detection model. If the area under the precision-recall curve is 1, it signifies that the target detection model has reached its optimal performance. For multi-class detection tasks, this invention uses the mean precision (mAP) as a comprehensive evaluation metric. mAP comprehensively reflects the target detection model's ability to detect different classes of targets; a higher mAP value indicates stronger generalization and robustness of the target detection model.

[0098] It is understandable that painted surfaces, due to their specular reflective properties, will exhibit localized bright areas (i.e., specular highlights) under strong light or oblique incidence. Traditional detection algorithms easily misclassify these specular edges or reflective pseudo-textures as defects, leading to distortion of the overall detection results. Therefore, this embodiment utilizes the false detection rate index for specular scenes to quantify the anti-reflective capability of the target detection model. Specifically, specular regions in an image can be detected using the brightness channel thresholding method: Let the brightness of the input image be L(x,y). When L(x,y) >> μ + 2σ (μ is the global brightness mean, σ is the standard deviation), the pixel is considered a specular region. The number of false detections in the model output within the specular region is then counted. The number of truly defect-free samples corresponding to the area of ​​the highlight region. Perform ratio calculation:

[0099] ;

[0100] in, This represents the number of pixels or instances within the highlight area that are mistakenly identified as defects. This represents the number of real, defect-free samples corresponding to the area of ​​the highlight region. This reflects the model's discrimination accuracy in highly reflective areas; a lower value indicates stronger reflection suppression and brightness normalization capabilities. In a typical test set, this embodiment introduces a color channel attention module to... The percentage dropped from 17.3% to 11.9%, a reduction of approximately 31%, significantly improving the detection stability in reflective scenarios.

[0101] It is understood that the output paint surface defect detection results in this embodiment include the defect category, defect bounding box, segmentation mask, and confidence level. Furthermore, this embodiment can also construct a target paint detection system based on the target detection model and interface it with a corresponding target visualization platform to achieve data synchronization. On the target visualization platform, the detected visualization image can be displayed in real time, and defect statistical reports can be exported. If the model is currently in the training or optimization learning phase, the defect statistical report can also include indicators such as the false detection rate and average accuracy for highlight scenes, used for subsequent process optimization and production quality analysis, realizing a quantitative evaluation of the detection accuracy and stability of the target detection model. In addition, based on key indicators such as mAP and FP_highlight, the model weights and learning rate of the target detection model can be dynamically adjusted to achieve adaptive optimization of the algorithm layer, thereby realizing a complete closed loop of dynamic model adjustment and application. It should be noted that the target paint inspection system implemented based on this embodiment can realize multimodal application expansion. For example, by using the detected defect type (category label) and defect boundary features (mask image) as input to the paint surface defect detection results, it supports defect cause analysis and automatic generation of process documents, thus promoting the extension of paint quality inspection to intelligent management.

[0102] As can be seen, after acquiring the original image data of the UAV, this application utilizes a target reflection suppression mechanism to enhance the defect texture features in the image and suppress false reflection features. It then uses the feature extraction network of the target detection model to achieve multi-scale feature extraction, thereby obtaining the corresponding target feature map. Next, based on the target anchor box generation mechanism, the region candidate network of the target detection model is used to determine the target candidate box corresponding to the target feature map, achieving high recall detection of small targets and thin scratches. Then, based on the target classification and regression results, the target candidate box and the target feature map are mapped to determine the corresponding target region of interest, achieving feature alignment. Finally, the target detection model uses a target detection mechanism based on the defect morphology corresponding to the target region of interest to process the target region of interest, obtaining the detection results of the paint surface defects of the target UAV, thus achieving accurate detection of paint surface defects on the UAV.

[0103] As described in the previous embodiment, this application discloses a deep learning-based method for detecting defects in the paint surface of drones, which can achieve accurate detection of defects in the paint surface of drones. Next, the specific target detection model training process will be explained; see [link to relevant documentation]. Figure 7 As shown.

[0104] In one specific embodiment, images are first synchronously acquired on the painting production line using an industrial camera and a light source system. The camera employs a fixed or robotic arm dynamic scanning method to acquire image samples covering different spraying angles, lighting conditions, and material surfaces. The acquired raw images are normalized for brightness using CCAM to eliminate the effects of highlights and shadows. Subsequently, image denoising, geometric correction, and normalization are performed to form the network input image. Then, instance segmentation and annotation are performed using annotation tools such as Labelme to delineate defect target areas and generate JSON label files. The samples are divided into training, validation, and test sets in a ratio of 7:2:1. The training set is used to train the model, the validation set is used to evaluate model performance during training, and the test set is used to finally evaluate the model's accuracy. Next, using transfer learning, different batches and iterations of model training are performed based on the pre-trained model. After each batch of training, the model accuracy is verified using the validation sample set, and the weight parameters are adjusted. This yields the training model corresponding to each training batch and the accuracy index of the validation sample set. Model training includes the following three stages:

[0105] Phase 1: Freeze the backbone network and train only the RPN and mask head branches;

[0106] Phase 2: Unfreeze FPN and lower layers of the backbone, and jointly train CCAM and DA-RPN modules;

[0107] Phase 3: Fine-tune the morphological branch detection head to achieve defect morphology differentiation.

[0108] After each training batch, the learning rate and weight parameters are dynamically adjusted based on metrics such as validation set mAP. After training, the mAP of each trained model on the validation sample set is calculated, and the model with the highest mAP accuracy is selected as the optimal model and applied to defect detection on the test sample set. Specifically, the accuracy of the selected optimal model is evaluated using the test sample set, and the weights of the optimal model are loaded for testing on the test set. Precision, recall, and F1 score are used to evaluate model performance. If the model passes the evaluation, it is deployed to the corresponding defect detection platform. After deployment, a model performance monitoring system can be established to periodically update the model with new data and maintain its effectiveness through online learning or incremental training.

[0109] In one specific implementation, addressing the shortcomings of traditional manual visual inspection, which relies on experience and is prone to missed detections, and the insufficient edge localization accuracy of single-stage detection algorithms such as YOLO / SSD, this embodiment constructs a multimodal defect dataset containing 2468 samples as a test set. On this test set, the target detection model provided in this embodiment achieves a pixel-level segmentation accuracy (88.00%) for scratch-type defects that is 32.6% higher than traditional Canny edge detection, and its recall rate for small defects (<1mm) is increased to 94.5%. This precise boundary modeling capability solves the technical problem of "high false detection rate due to ambiguous defect localization." In other words, the multi-scale feature fusion and pixel-level segmentation capabilities of the target detection model in this embodiment overcome the boundary ambiguity problem of traditional detection algorithms.

[0110] In one specific implementation, this embodiment addresses the problem of variable defect features caused by changes in spraying process parameters (such as spray gun pressure and paint viscosity) and insufficient generalization ability of traditional models by constructing a multimodal defect dataset containing 2468 samples. A pre-trained ResNet50 model is then used for transfer learning. Through the design of dynamic data augmentation strategies (including illumination change simulation and spray texture perturbation), the model has been exposed to 12 variant features of 7 typical defect types during the training phase. Ablation experiments demonstrate that this strategy improves the model's F1 score by 19.3% under unseen conditions and maintains a cross-domain detection accuracy of over 85% for different spraying substrates (such as metal / plastic). This transfer learning and dynamic data augmentation strategy enhances the model's generalization ability under complex conditions and effectively reduces the impact of environmental interference on the robustness of the target detection model.

[0111] Furthermore, this embodiment proposes a feature reuse mechanism based on FPN. In one specific implementation, the original computational complexity of Mask R-CNN can be reduced by 42%, while the TensorRT acceleration technology of the NVIDIA Jetson AGX Xavier platform can control the single-frame detection time to within 123ms (meeting the line-speed requirements of industrial detection). Further, by designing a defect region pruning preprocessing module, the computational load of invalid regions is reduced by 68%, enabling the system to maintain a 93% recall rate while achieving a throughput of 420 frames per second. This hardware and software co-optimization scheme solves the engineering challenge of balancing high-precision detection with real-time requirements, thereby achieving a balance between detection speed and accuracy through a lightweight inference architecture and hardware co-optimization.

[0112] As can be seen, after acquiring the original image data of the UAV, this application utilizes a target reflection suppression mechanism to enhance the defect texture features in the image and suppress false reflection features. It then uses the feature extraction network of the target detection model to achieve multi-scale feature extraction, thereby obtaining the corresponding target feature map. Next, based on the target anchor box generation mechanism, the region candidate network of the target detection model is used to determine the target candidate box corresponding to the target feature map, achieving high recall detection of small targets and thin scratches. Then, based on the target classification and regression results, the target candidate box and the target feature map are mapped to determine the corresponding target region of interest, achieving feature alignment. Finally, the target detection model uses a target detection mechanism based on the defect morphology corresponding to the target region of interest to process the target region of interest, obtaining the detection results of the paint surface defects of the target UAV, thus achieving accurate detection of paint surface defects on the UAV.

[0113] See Figure 8 As shown, this application discloses a deep learning-based device for detecting defects in the painted surfaces of drones, applied to a terminal device equipped with a target detection model, comprising:

[0114] The feature map acquisition module 11 is used to acquire the original image data of the target drone, and process the original image data using the feature extraction network of the target detection model based on the target reflection suppression mechanism to obtain the corresponding target feature map; the original image data is the painted surface image data of the target drone.

[0115] Feature map processing module 12 is used to determine the target candidate box corresponding to the target feature map using the region candidate network of the target detection model based on the target anchor box generation mechanism, so as to determine the target classification result and target regression result corresponding to the target feature map based on the target candidate box;

[0116] The defect detection module 13 is used to map the target candidate box and the target feature map based on the target classification result and the target regression result using the target detection model to determine the corresponding target region of interest, and to process the target region of interest based on the target detection mechanism corresponding to the defect morphology of the target region of interest using the target detection model to obtain the paint surface defect detection result corresponding to the target UAV.

[0117] As can be seen, after acquiring the original image data of the UAV, this application utilizes a target reflection suppression mechanism to enhance the defect texture features in the image and suppress false reflection features. It then uses the feature extraction network of the target detection model to achieve multi-scale feature extraction, thereby obtaining the corresponding target feature map. Next, based on the target anchor box generation mechanism, the region candidate network of the target detection model is used to determine the target candidate box corresponding to the target feature map, achieving high recall detection of small targets and thin scratches. Then, based on the target classification and regression results, the target candidate box and the target feature map are mapped to determine the corresponding target region of interest, achieving feature alignment. Finally, the target detection model uses a target detection mechanism based on the defect morphology corresponding to the target region of interest to process the target region of interest, obtaining the detection results of the paint surface defects of the target UAV, thus achieving accurate detection of paint surface defects on the UAV.

[0118] In one specific embodiment, the feature map acquisition module 11 may include:

[0119] The data conversion submodule is used to convert the original image data to a target color space to determine the converted image data, and to determine the corresponding brightness distribution data and the average scene brightness.

[0120] The threshold segmentation submodule is used to perform dual threshold segmentation on the converted image data based on the brightness distribution data to determine the highlight area image data and shadow area image data corresponding to the converted image data.

[0121] The image balancing submodule is used to determine the target Gamma coefficient based on the average brightness of the scene, compress and map the highlight area image data based on the target Gamma coefficient to obtain the corresponding first balanced data, and enhance and map the shadow area image data based on the target Gamma coefficient to obtain the corresponding second balanced data, so as to obtain the balanced image data corresponding to the original image data based on the first balanced data and the second balanced data.

[0122] The feature map acquisition submodule is used to process the balanced image data using the feature extraction network of the target detection model to obtain the corresponding target feature map.

[0123] In one specific embodiment, the feature map acquisition module 11 may include:

[0124] The initial feature map acquisition submodule is used to perform cross-layer connection processing on the original image data using the first feature extraction network of the target detection model to obtain the corresponding initial feature map;

[0125] The target feature map acquisition submodule is used to fuse all the initial feature maps corresponding to the original image data using the second feature extraction network of the target detection model to obtain the corresponding target feature map.

[0126] The feature extraction network includes a first feature extraction network and a second feature extraction network, wherein the first feature extraction network is a residual neural network and the second feature extraction network is a feature pyramid network.

[0127] In one specific embodiment, the feature map processing module 12 may include:

[0128] The first candidate box determination submodule is used to determine the first candidate box and the first confidence score corresponding to the target feature map based on the target anchor box generation mechanism and the region candidate network of the target detection model.

[0129] The second candidate box determination submodule is used to process the target feature map using a preset Anchor-Free detection model to determine the corresponding second candidate box and second confidence score;

[0130] The target candidate box determination submodule is used to determine the target candidate box corresponding to the target feature map based on the first candidate box, the first confidence score, the second candidate box, and the second confidence score using a preset weighting mechanism.

[0131] In one specific implementation, the first candidate box determining submodule may specifically include:

[0132] The initial candidate box determination unit is used to determine the initial candidate box corresponding to the target feature map based on the target anchor box generation mechanism and the region candidate network of the target detection model.

[0133] The candidate box calibration unit is used to calibrate the initial candidate box based on the target reflection suppression mechanism to obtain the first candidate box corresponding to the target feature map, and to determine the first confidence score corresponding to the first candidate box.

[0134] In one specific embodiment, the defect detection module 13 may include:

[0135] The first output result acquisition submodule is used to classify and regress the target region of interest using the target detection model to obtain the corresponding classification output result and bounding box regression output result.

[0136] The second output result acquisition submodule is used to perform masking processing on the target region of interest by the target detection model based on the target detection mechanism corresponding to the defect morphology of the target region of interest, so as to obtain the mask output result corresponding to the target UAV.

[0137] The defect detection result acquisition submodule is used to determine the paint surface defect detection result corresponding to the target UAV based on the classification output result, the bounding box regression output result, and the mask output result using the target total loss function.

[0138] In one specific implementation, the second output result acquisition submodule may specifically include:

[0139] The first defect processing unit is used to add a directional gradient regularization term to the target region of interest if the defect morphology of the target region of interest is a drip morphology, so as to perform mask processing on the target region of interest based on the added directional gradient regularization term.

[0140] The second defect processing unit is used to fuse the local texture statistical features and frequency domain features of the target region of interest to obtain fused texture features if the defect morphology of the target region of interest is orange peel morphology, and to perform masking processing on the target region of interest based on the fused texture features.

[0141] The third defect processing unit is used to perform masking processing on the target region of interest using Sobel gradient sparsity constraints if the defect morphology of the target region of interest is a scratch morphology.

[0142] The fourth defect processing unit is used to perform masking processing on the target region of interest using cyclic consistency loss if the defect morphology of the target region of interest is a bubble morphology or a shrinkage cavity morphology.

[0143] Furthermore, embodiments of this application also disclose an electronic device, Figure 9 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0144] Figure 9This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the deep learning-based UAV paint surface defect detection method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0145] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0146] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon can include an operating system 221, computer programs 222, etc., and the storage method can be temporary storage or permanent storage.

[0147] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the deep learning-based UAV paint surface defect detection method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0148] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned deep learning-based method for detecting defects in the painted surfaces of unmanned aerial vehicles. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0149] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0150] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0151] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0152] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0153] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting defects on a painted surface of a UAV based on deep learning, characterized in that, The application is applied to a terminal device carrying a target detection model, and comprises: obtaining original image data of a target unmanned aerial vehicle, and processing the original image data by using a feature extraction network of the target detection model based on a target reflection suppression mechanism to obtain a corresponding target feature map; the original image data is paint surface image data of the target unmanned aerial vehicle; determining a target candidate box corresponding to the target feature map by using a region candidate network of the target detection model based on a target anchor box generation mechanism, to determine a target classification result and a target regression result corresponding to the target feature map based on the target candidate box; mapping the target candidate box and the target feature map based on the target classification result and the target regression result by using the target detection model to determine a corresponding target region of interest, and processing the target region of interest based on a target detection mechanism corresponding to a defect morphology of the target region of interest by using the target detection model to obtain a paint surface defect detection result corresponding to the target unmanned aerial vehicle; wherein the processing of the target region of interest based on the target detection mechanism corresponding to the defect morphology of the target region of interest by using the target detection model to obtain the paint surface defect detection result corresponding to the target unmanned aerial vehicle comprises: classifying and processing the target region of interest by using the target detection model to obtain a classification output result and a bounding box regression output result; masking the target region of interest based on the target detection mechanism corresponding to the defect morphology of the target region of interest by using the target detection model to obtain a mask output result corresponding to the target unmanned aerial vehicle; determining the paint surface defect detection result corresponding to the target unmanned aerial vehicle based on the classification output result, the bounding box regression output result and the mask output result by using a target total loss function; wherein the masking of the target region of interest based on the target detection mechanism corresponding to the defect morphology of the target region of interest by using the target detection model comprises: if the defect morphology of the target region of interest is a sagging morphology, adding a directional gradient regularization term to the target region of interest to mask the target region of interest based on the added directional gradient regularization term; if the defect morphology of the target region of interest is an orange peel morphology, fusing local texture statistical features and frequency domain features of the target region of interest to obtain fused texture features, and masking the target region of interest based on the fused texture features; if the defect morphology of the target region of interest is a scratch morphology, masking the target region of interest by using Sobel gradient sparse constraint; if the defect morphology of the target region of interest is a bubble morphology or a shrinkage hole morphology, masking the target region of interest by using a cycle consistency loss. 2.The deep learning-based unmanned aerial vehicle paint surface defect detection method of claim 1, wherein, the processing of the original image data by using the feature extraction network of the target detection model based on the target reflection suppression mechanism to obtain the corresponding target feature map comprises: convert the original image data to a target color space to determine converted image data, and determine corresponding brightness distribution data and a scene brightness mean value; perform double-threshold segmentation on the converted image data based on the brightness distribution data to determine high-light region image data and shadow region image data corresponding to the converted image data; determine a target Gamma coefficient based on the scene brightness mean value, perform compression mapping on the high-light region image data based on the target Gamma coefficient to obtain corresponding first balancing data, and perform enhancement mapping on the shadow region image data based on the target Gamma coefficient to obtain corresponding second balancing data, so as to obtain balancing image data corresponding to the original image data based on the first balancing data and the second balancing data; process the balancing image data by using a feature extraction network of the target detection model to obtain corresponding target feature maps. 3.The deep learning-based unmanned aerial vehicle paint surface defect detection method of claim 1, wherein, The processing of the original image data by using the feature extraction network of the target detection model to obtain corresponding target feature maps comprises: performing cross-layer connection processing on the original image data by using a first feature extraction network of the target detection model to obtain corresponding initial feature maps; performing fusion processing on all the initial feature maps corresponding to the original image data by using a second feature extraction network of the target detection model to obtain corresponding target feature maps; The feature extraction network comprises the first feature extraction network and the second feature extraction network, the first feature extraction network is a residual neural network, and the second feature extraction network is a feature pyramid network. 4.The deep learning-based unmanned aerial vehicle paint surface defect detection method of claim 1, wherein, The determination of the target candidate box corresponding to the target feature map by using the region proposal network of the target detection model based on the target anchor box generation mechanism comprises: determining a first candidate box and a first confidence score corresponding to the target feature map by using the region proposal network of the target detection model based on the target anchor box generation mechanism; processing the target feature map by using a preset Anchor-Free detection model to determine a second candidate box and a second confidence score; determining a target candidate box corresponding to the target feature map by using a preset weighting mechanism based on the first candidate box, the first confidence score, the second candidate box, and the second confidence score. 5.The deep learning-based unmanned aerial vehicle paint surface defect detection method of claim 4, wherein, The determination of the first candidate box and the first confidence score corresponding to the target feature map by using the region proposal network of the target detection model based on the target anchor box generation mechanism comprises: determining an initial candidate box corresponding to the target feature map by using the region proposal network of the target detection model based on the target anchor box generation mechanism; calibrating the initial candidate box based on the target highlight suppression mechanism to obtain the first candidate box corresponding to the target feature map, and determining a first confidence score corresponding to the first candidate box. 6.A deep learning-based unmanned aerial vehicle (UAV) paint surface defect detection device, characterized in that, Applied to a terminal device carrying a target detection model, comprising: The feature map acquisition module is configured to acquire original image data of a target UAV and process the original image data based on a target anti-reflection mechanism by using a feature extraction network of the target detection model to obtain a corresponding target feature map. The original image data is a paint surface image data of the target UAV. The feature map processing module is configured to determine a target candidate box corresponding to the target feature map by using a region candidate network of the target detection model based on a target anchor box generation mechanism, and determine a target classification result and a target regression result corresponding to the target feature map based on the target candidate box. The defect detection module is configured to map the target candidate box and the target feature map based on the target classification result and the target regression result by using the target detection model to determine a corresponding target region of interest, and process the target region of interest based on a target detection mechanism corresponding to a defect morphology of the target region of interest by using the target detection model to obtain a paint surface defect detection result corresponding to the target UAV. The defect detection module specifically includes: The first output result acquisition submodule is configured to perform classification processing and bounding box regression processing on the target region of interest by using the target detection model to obtain a corresponding classification output result and a bounding box regression output result. The second output result acquisition submodule is configured to perform mask processing on the target region of interest based on a target detection mechanism corresponding to a defect morphology of the target region of interest by using the target detection model to obtain a mask output result corresponding to the target UAV. The defect detection result acquisition submodule is configured to determine a paint surface defect detection result corresponding to the target UAV based on the classification output result, the bounding box regression output result, and the mask output result by using a target total loss function. The second output result acquisition submodule specifically includes: The first defect processing unit is configured to add a directional gradient regularization term to the target region of interest if the defect morphology of the target region of interest is a sagging morphology, and perform mask processing on the target region of interest based on the added directional gradient regularization term. The second defect processing unit is configured to fuse local texture statistical features and frequency domain features of the target region of interest to obtain fused texture features if the defect morphology of the target region of interest is an orange peel morphology, and perform mask processing on the target region of interest based on the fused texture features. The third defect processing unit is configured to perform mask processing on the target region of interest by using Sobel gradient sparse constraint if the defect morphology of the target region of interest is a scratch morphology. The fourth defect processing unit is configured to perform mask processing on the target region of interest by using cyclic consistency loss if the defect morphology of the target region of interest is a bubble morphology or a shrinkage hole morphology.

7. An electronic device, comprising: The memory is configured to save a computer program. The processor is configured to execute the computer program to implement the deep learning-based UAV paint surface defect detection method according to any one of claims 1 to 5. ​ 8. A computer-readable storage medium, characterized in that, A computer program product for saving a computer program, wherein the computer program, when executed by a processor, implements the deep learning based unmanned aerial vehicle paint surface defect detection method according to any one of claims 1 to 5.

Images