A sar image airport foreign object target real-time detection method based on yolov5

Abstract

The application discloses a kind of SAR image airport extraneous object target real-time detection method based on YOLOv5, steps are as follows: obtaining SAR image dataset, by imaging in test field and obtaining real SAR image airport extraneous object dataset, data set is carried out data enhancement, after obtaining the expanded sample set and is divided into training sample set, verification sample set and test sample set;The expanded sample set is carried out Gaussian low-pass filter noise reduction, and the pixel value of SAR image is weighted average;Establish improved real-time YOLOv5 model;The training data set, verification sample set are input into improved real-time YOLOv5 model;Get trained improved real-time YOLOv5 model;Test data set is input into trained improved real-time YOLOv5 model, and detection recognition result is obtained.The improved real-time YOLOv5 model proposed in the application can more quickly identify the airport extraneous object target in SAR image, greatly reduce the test time of model, significantly improve detection precision.

Description

Technical Field

[0001] This invention belongs to the field of SAR image target detection technology, and in particular relates to a real-time detection method for foreign object targets in SAR images of airports based on YOLOv5. Background Technology

[0002] Foreign Object Debris (FOD) refers to any foreign material that may endanger airport runway safety, damage aircraft and equipment, or threaten the lives of airport staff and passengers. These foreign objects are mainly divided into two categories: airborne foreign objects and ground-based foreign objects. Airborne foreign objects include substances attached to aircraft surfaces, engine air intakes, windshields, etc., such as small birds, dust, rain, and snow; while ground-based foreign objects refer to various foreign objects existing on the airport ground, such as items lost or damaged by passengers, parts generated during maintenance, and debris from runway aging or impacts.

[0003] Synthetic Aperture Radar (SAR) is a high-resolution two-dimensional imaging radar system, typically used on ground, airborne, and airborne platforms. It utilizes the platform's motion to synthesize a virtual large-aperture array antenna, achieving high-resolution azimuth imaging. Arc-scanning Synthetic Aperture Radar (AS-SAR) is a high-resolution two-dimensional imaging radar system that synthesizes echo signals acquired during circular scanning. When performing high-resolution imaging over large areas, circular scanning can more effectively cover the entire region, improving imaging efficiency and coverage. It has all-weather, long-range target detection capabilities. Compared to visible light imaging equipment, AS-SAR is unaffected by adverse weather conditions such as clouds, fog, rain, and snow, and can operate stably in harsh environments. It has been widely used in military and civilian applications for target detection. Among these applications, the detection and identification of foreign objects at airports is a crucial problem in SAR image processing.

[0004] The imaging characteristics of SAR images enable real-time acquisition of high-resolution images, but also introduce problems such as persistent speckle noise and sensitivity to angle changes. The noise characteristics in SAR images differ from typical Gaussian noise, exhibiting structural coherence, also known as speckle noise. Traditional algorithms cannot effectively address these issues. However, with the development of deep learning models in recent years, they can automatically learn and extract high-level features from data without the need for manual feature extractor design. This is extremely useful for complex SAR image data, as SAR images often contain intricate interference and noise that traditional methods struggle to handle effectively. Deep learning models excel in processing large-scale datasets. Deep learning models can automatically learn and extract high-level features from data without the need for manual feature extractor design. This is particularly useful for processing complex SAR image data, which typically contains significant noise and interference. In recent years, the YOLO series of algorithms, represented by YOLOv5, has made significant progress in target detection. These algorithms, through multi-scale feature fusion strategies, can achieve high accuracy in complex scenes and possess high real-time performance. Therefore, they are widely used in various real-time video detection, edge computing, and other scenarios. While the YOLO series of algorithms can improve accuracy and efficiency, as a general-purpose algorithm for target detection, it requires more adaptability in SAR image target detection.

[0005] SAR image target detection is mostly applied to ship and aircraft detection, but less so to foreign object detection at airports. Moreover, there is a lack of real SAR image datasets for FOD detection on the market. Compared to strong edge targets such as ships and aircraft, FOD targets have weaker textures and less obvious size features, making them more difficult to detect on complex backgrounds such as airport runways. Summary of the Invention

[0006] The purpose of this invention is to provide a real-time detection method for foreign objects in airport SAR images based on YOLOv5, in order to solve the technical problem that foreign objects in airports have weaker textures and less obvious size features, making them more difficult to detect on complex background airport runways.

[0007] To achieve the above objectives, the specific technical solution of the present invention is as follows:

[0008] A real-time detection method for foreign objects in airport SAR images based on YOLOv5 includes the following steps:

[0009] Step 1: Obtain SAR image dataset. Obtain real SAR image dataset of foreign objects at the airport by taking images at the test site. Perform data augmentation on the dataset to obtain an expanded sample set, which is then divided into training sample set, validation sample set and test sample set.

[0010] Step 2: Perform Gaussian low-pass filtering on the expanded sample set to reduce noise, and perform pixel-weighted averaging on the image to reduce the energy of the high-frequency part caused by speckle noise in the SAR image and reduce the influence of random noise in the image.

[0011] Step 3: Build an improved real-time YOLOv5 model, that is, replace the feature enhancement module of the YOLOv5 backbone network with a module based on SPPELAN, replace the activation function with the ReLU activation function, add a residual branch to the backbone network, add the subsequent feature pyramid fusion structure, and replace the loss function with Focal-ELoU.

[0012] Step 4: Input the training dataset and validation dataset into the improved real-time YOLOv5 model for training, and obtain the trained improved real-time YOLOv5 model;

[0013] Step 5: Input the test dataset into the trained, improved real-time YOLOv5 model to obtain the detection and recognition results.

[0014] Therefore, by acquiring a dataset of foreign objects in airport SAR images through imaging at the test site, data augmentation is performed on the dataset to obtain an expanded sample set, which is then divided into training, validation, and test sample sets. Gaussian low-pass filtering is applied to the expanded sample set for noise reduction, and pixel-weighted averaging is performed on the SAR images. An improved real-time YOLOv5 model is then established. The training and validation datasets are input into the improved real-time YOLOv5 model, resulting in a trained improved real-time YOLOv5 model. The test dataset is then input into the trained improved real-time YOLOv5 model to obtain the detection and recognition results. The improved real-time YOLOv5 model proposed in this invention can identify ships in SAR images more quickly, significantly reducing the model's testing time and significantly improving detection accuracy.

[0015] Furthermore, in step 1, the airport foreign object dataset is augmented to obtain an expanded sample set, including: performing horizontal and vertical flipping operations on each SAR image in the sample set to enhance the robustness of the network and obtain the expanded sample set.

[0016] Furthermore, in step 2, the formula for the Gaussian low-pass filter is as follows:

[0017]

[0018] In the formula, x and y are the coordinates of the pixel, and σ is the standard deviation of the normal distribution.

[0019] Furthermore, in step 2, the standard deviation parameter of the Gaussian low-pass filter is set to 5-8.

[0020] Furthermore, in step 3, the formula for calculating Focal-EIoU is as follows:

[0021] L Focal-EIoU =α·FL(p t )+(1-α)·EIoU (2)

[0022] Where α is a weighting factor used to balance the effects of Focal Loss and EIoU, while FL(p t The formula for calculating EIoU is as follows:

[0023] FL(p t )=-α t (1-p t ) γ log(p t (3)

[0024] Where, p t To predict the probability of the correct class; α t γ is a balancing factor used to balance positive and negative samples; γ is an adjustment factor used to adjust the weights of easy and difficult samples.

[0025]

[0026] Where IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box; ρ(b,b) g ) represents the distance between the center point of the predicted bounding box and the center point of the ground truth bounding box; c represents the diagonal length of the containing box; w and h represent the width and height of the predicted bounding box, respectively; w and w g These are the width and height of the actual bounding box, respectively; c w and c h These are the maximum values ​​for width and height, respectively.

[0027] Furthermore, in step 3, DIoU-NMS is introduced as a suppression criterion to post-process the target prediction results. The calculation formula for DIoU-NMS is as follows:

[0028]

[0029] -1≤DIoU≤1

[0030] Where b is the center of the predicted bounding box, b gt ρ is the center of the ground truth bounding box, d is the distance between the centers of the ground truth bounding box and the predicted bounding box, and ρ is the center of the ground truth bounding box. 2 (b,b gt ) is the square of the center distance between the ground truth box and the predicted box, and c is the diagonal length of the smallest bounding box containing both bounding boxes.

[0031] In addition, its performance evaluation metrics include accuracy, recall, average precision, and average test duration.

[0032] The present invention has the following advantages:

[0033] 1) When the method is applied to the identification of foreign objects in airport runways and building areas, it can effectively extract the features of airport lights, buildings and foreign objects and distinguish the information of foreign objects. It has a good recognition rate for foreign objects in complex airport scenarios.

[0034] 2) The proposed method can accurately identify metal objects of different sizes in simulated airport foreign objects in SAR images, and it also achieves high accuracy on both dense and sparse airport foreign object datasets. Compared with existing algorithms, it offers superior performance, a higher recognition rate for airport foreign objects, and improves the efficiency of target recognition for airport foreign objects.

[0035] 3) The present invention has a significant improvement in detection speed compared to the original model based on YOLOv5, while taking into account both high accuracy and real-time performance, thus meeting the real-time requirements for foreign object detection in airport application scenarios. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the implementation of the present invention;

[0037] Figure 2 This is a physical diagram of the FOD (Field-Oriented Discharge) device used for experimental simulation according to the present invention.

[0038] Figure 3 This is a training result image of the image before it undergoes the filtering algorithm in this invention;

[0039] Figure 4 This is a training result image of the image after the filtering algorithm in this invention;

[0040] Figure 5 This is a diagram of the network structure for improving real-time performance based on YOLOv5 in this invention.

[0041] Figure 6 This is a structural diagram of the SPPFELAN module of the present invention;

[0042] Figure 7 The image shows the detection results of the algorithm of this invention on a sparse sample set of images;

[0043] Figure 8 The image shows the detection results of the algorithm of this invention on a dense sample set of images. Detailed Implementation

[0044] To better understand the purpose, structure, and function of this invention, the invention will be described in further detail below with reference to the accompanying drawings.

[0045] Reference Figure 1 The flowchart shown below illustrates the implementation of this invention, which includes the following steps:

[0046] Step 1: Obtain SAR image dataset. Obtain real SAR image dataset of foreign objects at the airport by taking images at the test site. Perform data augmentation on the dataset to obtain an expanded sample set, which is then divided into training sample set, validation sample set and test sample set.

[0047] Preferably, the airport foreign object dataset is augmented to obtain an expanded sample set, including: performing horizontal and vertical flipping operations on each SAR image in the sample set, thereby expanding the dataset by three times without compromising the authenticity of the measured dataset, enhancing the robustness of the network, and obtaining the expanded sample set.

[0048] Step 2: Perform Gaussian low-pass filtering on the expanded sample set to reduce noise, and perform pixel-weighted averaging on the image to reduce the energy of the high-frequency part caused by speckle noise in the SAR image and reduce the influence of random noise in the image.

[0049] Preferably, the expanded sample set is subjected to Gaussian low-pass filtering for noise reduction, and the image is weighted by pixel value averaging to reduce the energy of the high-frequency components caused by speckle noise in the SAR image and reduce the influence of random noise in the image. The formula for the Gaussian low-pass filter is shown below:

[0050]

[0051] In the above formula, x and y are the coordinates of the pixel, and σ is the standard deviation of the normal distribution. The magnitude of σ affects the image processing effect; the larger the σ, the more obvious the effect on random noise, but the image will also become smoother and blurrier. Using the imgaussfilt function to perform adaptive windowing and Gaussian filtering on the pixel values ​​within the window eliminates the influence of random noise to a certain extent, making the target features in the sample set more prominent. Airport foreign object targets in the sample set account for approximately one-thousandth of the entire high-resolution image, making them extremely difficult to detect. Therefore, considering the characteristics of airport foreign object targets in this sample set, image filtering is applied to reduce the impact of random noise on the model's extraction of target features. A simulated airport foreign object image is shown below. Figure 2 As shown, these are hollow metal cylinders with diameters of 2cm and 4cm, simulating small parts inside an aircraft or engine that might fall onto an airport runway. A comparison of the training results before and after the image filtering algorithm is provided for reference. Figure 3 and Figure 4 , Figure 3Input the training result image of the YOLOv5 model as the original image. Figure 4 The image shows the training results of the YOLOv5 model, with the filtered image input as input. The model trained using the filtered image shows better performance and stability across metrics such as training box loss (train / box_loss), training object loss (train / obj_loss), training classification loss (train / cls_loss), precision (metrics / precision), recall (metrics / recall), and mean precision (metrics / mAP_0.5). Particularly at higher IoU thresholds (metrics / mAP_0.5:0.95), this indicates that the model trained with the filtered image exhibits more stable performance across different IoU thresholds.

[0052] Step 3: Establish an improved real-time YOLOv5 model, namely, replace the feature enhancement module of the YOLOv5 backbone network with a SPPELAN-based module, replace the activation function with the ReLU activation function, add a residual branch to the backbone network, incorporate the subsequent feature pyramid fusion structure, replace the loss function with Focal-EIoU, and introduce DIoU-NMS as a suppression criterion for post-processing the target prediction results.

[0053] Preferably, a YOLOv5 model reference is established to improve real-time performance. Figure 2As shown, the real-time detection method for foreign objects in SAR images at airports based on YOLOv5 proposed in this invention consists of three parts: a backbone network, a fusion layer (Neck), and a detection head. The backbone network comprises modules such as CBS, C3, and SPPFELAN. CBS mainly includes Conv, BN, and SiLU activation functions. This module performs convolution operations on the feature maps, progressively extracting feature maps of different sizes for feature fusion in the fusion layer. The original model extracted three feature layers, P3, P4, and P5, which are then fed into the fusion layer. Now, a four-fold downsampling branch is used, and the newly extracted feature layer P2 corresponds to a feature map size of 160*160 pixels, which can detect targets as small as 4*4 pixels, making it more suitable for detecting weak targets such as foreign objects at airports. P2, P3, P4, and P5 are then incorporated into the Neck layer for feature fusion. The Neck layer consists of an FPN-PAN structure. It fuses high- and low-level features through upsampling, and the laterally connected PAN structure enhances both the semantic and global information of the features, enabling multi-scale feature fusion. Finally, four detection heads are set in the Head layer. By setting anchor boxes of different sizes and aspect ratios on feature maps at different scales, multi-scale object detection is achieved. By incorporating surrounding contextual information, the feature representation capability of small targets can be improved, thereby enhancing detection performance.

[0054] Specifically, the SPPFELAN module is as follows: Figure 3 As shown, the SPPFELAN module combines the advantages of SPP and ELAN, enhancing feature representation capabilities through multi-scale feature fusion and efficient layer aggregation. It captures features at different scales through pooling operations of varying scales (e.g., 1x1, 5x5, 9x9, 13x13); aggregates these pooled features with the original feature map to form multi-scale fused features; and performs convolution operations on the aggregated features to further extract and enhance them. Referring to the design of the SimSPPF module in YOLOv6, the CBS module is improved by replacing the activation function with ReLU, resulting in the CBR module. ReLU has a simpler computation method; this non-linear transformation allows ReLU to accelerate training and reduce computational complexity, while effectively mitigating the gradient vanishing problem.

[0055] Focal Loss is an improved cross-entropy loss method primarily used to address class imbalance. It reduces the loss weights of easily distinguishable negative examples, preventing the network from being biased by a large number of negative samples. EIoU is an improved IoU loss method that uses a smoothing function to approximate IoU calculation. Compared to traditional IoU loss calculation, this approximation reduces computational complexity and memory requirements, aiming to better optimize the localization accuracy of bounding boxes. Focal-EIoU combines the advantages of Focal Loss and EIoU, enabling the optimization of bounding box localization accuracy while handling class imbalance, thereby improving overall detection performance. It performs particularly well in class imbalance and complex scenes, making it suitable for the sample set scenario of this invention. The calculation formula for Focal-EIoU is shown below:

[0056] L Focal-EIoU =α·FL(p t )+(1-α)·EloU (2)

[0057] Where α is a weighting factor used to balance the effects of Focal Loss and EIoU; while FL(p t The formula for calculating EIoU is as follows:

[0058] FL(p t )=-α t (1-p t ) γ log(p t (3)

[0059] Where, p t To predict the probability of the correct class; α t γ is a balancing factor used to balance positive and negative samples; γ is an adjustment factor used to adjust the weights of easy and difficult samples.

[0060]

[0061] Where IoU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box; ρ(b,b) g ) represents the distance between the center point of the predicted bounding box and the center point of the ground truth bounding box; c represents the diagonal length of the containing box; w and h represent the width and height of the predicted bounding box, respectively; w and w g These are the width and height of the actual bounding box, respectively; c w and c h These are the maximum values ​​for width and height, respectively.

[0062] DIoU-NMS, or Non-maximum suppression for short, removes redundant detection boxes by discarding those that overlap significantly with maxima. However, traditional NMS relies solely on IoU for candidate box filtering and makes predictions for overlapping boxes, often resulting in false suppression in cases of occlusion or blurred edges. DIoU-NMS, by incorporating center point distance, considers the distance information between bounding boxes, making the final retained boxes more accurate in location. This allows for better differentiation of adjacent but non-overlapping targets, reducing false detections. The calculation formula for DIoU-NMS is as follows:

[0063]

[0064] -1≤DIoU≤1

[0065] Where b is the center of the predicted bounding box, b gt ρ is the center of the ground truth bounding box, d is the distance between the centers of the ground truth bounding box and the predicted bounding box, and ρ is the center of the ground truth bounding box. 2 (b,b gt ) is the square of the center distance between the ground truth box and the predicted box, and c is the diagonal length of the smallest bounding box containing both bounding boxes.

[0066] Step 4: Input the training dataset and validation dataset into the improved real-time YOLOv5 model for training, and obtain the trained improved real-time YOLOv5 model.

[0067] Step 5: Input the test dataset into the trained, improved real-time YOLOv5 model to obtain the detection and recognition results.

[0068] Preferably, in steps 4 and 5, the experimental process divides the sample set into a training sample set, a validation sample set, and a test sample set in an 8:1:1 ratio. The training sample set contains 936 SAR images, the validation set contains 105 SAR images, and the test set contains 106 SAR images, all of which are high-resolution images with a size of 9601×4094. The input image size is set to 640×640, the batch size is set to 4, and the training iterations are 200. The ablation experiments of this embodiment are compared in Table 1. In the table, P, R, mAP, Para, and Inference time refer to accuracy, recall, average precision, number of parameters, and average test time, respectively.

[0069] Table 1 Comparison of ablation test results using this method

[0070]

[0071] This invention proposes a real-time detection method for foreign object targets in airport SAR images based on YOLOv5. The process of locating the target range in the SAR image is commonly referred to as "target localization." This method enables all-weather, full-range real-time monitoring of areas on airport runways where foreign objects may pose a safety hazard. Furthermore, it extracts the target's location information and estimates its approximate range in the complex environment of an airport. Once the target's approximate range is determined in the radar image, it can be further refined by using an optical camera to ensure the safety of the airport's flight environment.

[0072] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A real-time detection method for foreign object targets at airports based on YOLOv5 SAR images, characterized in that, The steps include the following: Step 1: Obtain SAR image dataset. Obtain real SAR image dataset of foreign objects at the airport by taking images at the test site. Perform data augmentation on the dataset to obtain an expanded sample set, which is then divided into training sample set, validation sample set and test sample set. Step 2: Perform Gaussian low-pass filtering on the expanded sample set to reduce noise, and perform pixel-weighted averaging on the image to reduce the energy of the high-frequency part caused by speckle noise in the SAR image and reduce the influence of random noise in the image. Step 3: Establish an improved real-time YOLOv5 model. This involves replacing the feature enhancement module of the YOLOv5 backbone network with a SPPELAN-based module, replacing the activation function with ReLU, adding a residual branch to the backbone network, incorporating the subsequent feature pyramid fusion structure, and replacing the loss function with Focal-ELoU. The calculation formula is shown below: L Focal-EIoU= α∙FL(p t )+(1−α)∙EIoU (2) Where α is a weighting factor used to balance the effects of Focal Loss and EIoU, while FL(p t The formula for calculating EIoU is as follows: FL(p t )=−α t (1−p t ) γ log(p t ) (3) Where, p t To predict the probability of the correct class; α t γ is a balancing factor used to balance positive and negative samples; γ is an adjustment factor used to adjust the weights of easy and difficult samples. EIoU=1−IoU+ρ 2 (b,b g ) / c 2 +ρ 2 (w,w g ) / c w 2 +ρ 2 (h,h g ) / c h 2 (4) Where IoU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box; b is the center point of the predicted bounding box, b g Let ρ(b,b) be the center point of the true bounding box. g ) represents the distance between the center point of the predicted bounding box and the center point of the ground truth bounding box; c represents the diagonal length of the containing box; w and h represent the width and height of the predicted bounding box, respectively; w g and h g These are the width and height of the actual bounding box, respectively; c w and c h These are the maximum values ​​for width and height, respectively. Step 4: Input the training dataset and validation dataset into the improved real-time YOLOv5 model for training, and obtain the trained improved real-time YOLOv5 model; Step 5: Input the test dataset into the trained, improved real-time YOLOv5 model to obtain the detection and recognition results.

2. The method for real-time detection of foreign objects in airport SAR images based on YOLOv5 according to claim 1, characterized in that, In step 1, the airport foreign object dataset is augmented to obtain an expanded sample set, including: performing horizontal and vertical flipping operations on each SAR image in the sample set to enhance the robustness of the network and obtain the expanded sample set.

3. The method for real-time detection of foreign objects in airport SAR images based on YOLOv5 according to claim 1, characterized in that, In step 2, the formula for the Gaussian low-pass filter is as follows: h(x,y,σ)=1 / 2πσ 2 exp(−x 2 +y 2 / 2σ 2 ) (1) In the formula, x and y are the coordinates of the pixel, and σ is the standard deviation of the normal distribution.

4. The method for real-time detection of foreign objects in airport SAR images based on YOLOv5 according to claim 1, characterized in that, In step 2, the standard deviation parameter of the Gaussian low-pass filter is set to 5-8.

5. The method for real-time detection of foreign objects in airport SAR images based on YOLOv5 according to claim 1, characterized in that, In step 3, DIoU-NMS is introduced as a suppression criterion to post-process the target prediction results. The calculation formula for DIoU-NMS is as follows: DIoU=IoU−ρ 2 (b,b g ) / c 2 =IoU−d 2 / c 2 −1≤DIoU≤1 (5) Where b is the center of the predicted bounding box, b g ρ is the center of the ground truth bounding box, d is the distance between the centers of the ground truth bounding box and the predicted bounding box, and ρ is the center of the ground truth bounding box. 2 (b,b g ) is the square of the center distance between the ground truth box and the predicted box, and c is the diagonal length of the smallest bounding box containing both bounding boxes.

6. The method for real-time detection of foreign objects in airport SAR images based on YOLOv5 according to claim 1, characterized in that, Its performance evaluation metrics include accuracy, recall, average precision, and average test duration.

Images