A method and system for identifying foreign objects at airports based on an improved yolov8s model

Abstract

The present invention discloses an airport foreign object recognition method and system based on an improved yolov8s model, belonging to the field of computer vision technology. The method trains an airport foreign object small target detection model using airport images and corresponding labels in multiple scenes and multiple weather conditions, thereby ensuring that the model has a large object recognition range and high detection accuracy. The airport images are resized to a standard size to meet the input requirements of the model. The yolov8s model has the characteristics of strong feature extraction capability, high model robustness, strong generalization capability, fast running speed, and high inference efficiency, and can effectively detect objects of different sizes. The airport foreign object small target detection model constructed using the yolov8s model can well identify foreign objects of different sizes in airport images, thereby avoiding the harm of foreign objects to airport aircraft. The introduction of a visual attention mechanism helps the model focus more on important features, further improving the accuracy and robustness of the model in target detection tasks.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to an airport foreign object recognition method and system based on an improved YOLOv8s model. Background Art

[0002] Foreign objects on airport pavements can damage aircraft. These objects, such as screws, nuts, washers, springs, and stones, are foreign objects within the flight zone that could potentially damage or endanger aircraft. These objects pose a serious threat to flight safety and can cause significant financial losses to airlines. For this reason, some airports rely on manual inspections or radar-based foreign object detection systems to detect foreign objects. Radar detection technology is characterized by high resolution and long detection range, but it lacks color signatures or intuitive video images of detected objects, complicating subsequent foreign object removal.

[0003] The emergence of deep learning algorithms has brought a new approach to foreign object detection research. In recent years, deep learning research has made significant progress, with a growing number of deep learning-based object detection algorithms becoming increasingly mature. Many general-purpose object detection algorithms are now being applied in everyday life and production, achieving promising results. However, foreign objects in airport images are relatively small and vary in size. Directly using existing general-purpose object detection algorithms for foreign object detection results in low accuracy, and even small objects among them cannot be accurately detected. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to address the deficiencies in the above-mentioned prior art and provide an airport foreign object identification method and system based on an improved yolov8s model. The method and system have novel and reasonable designs, can accurately detect small target foreign objects, are highly practical, and are easy to promote and use.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A method for identifying foreign objects in airports based on an improved YOLOv8s model comprises the following steps: resizing airport images of multiple scenes and multiple weather conditions to a standard size, performing image enhancement processing on the resized airport images, marking foreign objects on the processed airport images, generating corresponding labels based on the labels, and constructing a data set using the airport images of multiple scenes and multiple weather conditions and the corresponding labels; constructing an airport foreign object small target detection model, and training the airport foreign object small target detection model using the data set; using the airport images to be identified as input to the trained airport foreign object small target detection model, outputting corresponding labels, and completing the identification of foreign objects in the airport images; and constructing the airport foreign object small target detection model based on the YOLOv8s model.

[0007] The above-mentioned airport foreign object small target detection model is constructed based on the yolov8s model. The above-mentioned yolov8s model includes a backbone network. The above-mentioned backbone network uses DSConv and C2F modules to complete feature extraction of the input image and input the extracted features into the neck layer;

[0008] A visual attention mechanism is added between the DSConv and C2F modules. The visual attention mechanism assigns different attention weights to different spatial positions of the input image according to the trained weight vector. The working process of the visual attention mechanism is as follows: the output image u of DSConv is used as the input of the positioning network, a transformation parameter θ is output, the grid generator creates a grid G according to the transformation parameter θ, the sampler samples from the output image u of DSConv according to the grid G, and the spatial transformer processes the output image u according to the sampling result and outputs a new image v.

[0009] Furthermore, the yolov8s model also includes a neck layer and a head detection head;

[0010] DSConv includes depthwise convolution and point-by-point convolution, which are used to extract features from images, obtain feature maps and use them as input to the neck layer. The neck layer extracts complex features from the input feature maps, obtains complex feature maps and uses them as input to the head detection head; the head detection head performs forward propagation calculations on the input complex feature maps to obtain the location information and category information of the foreign object, thereby determining the corresponding label and outputting it.

[0011] Furthermore, DSConv includes depthwise convolution and pointwise convolution. The number of depthwise convolutions = convolution kernel size × number of input channels; the parameter data of pointwise convolution = 1 × 1 × number of input channels × number of output channels.

[0012] Furthermore, the neck layer adopts a cross-scale feature fusion module. The above-mentioned cross-scale feature fusion model refers to: feature extraction of the input feature map through feature frames of multiple sizes, superimposition of the extracted feature maps of different scales according to the channel dimension, and then feature fusion through a convolutional layer to obtain a complex feature map.

[0013] Furthermore, the head detection head includes 4 detect detectors, and the above 4 detect detectors are used to detect feature maps of different sizes respectively.

[0014] Furthermore, a direct connection path is introduced into the forward propagation path, and the gradient is propagated through the direct connection path when gradient disappearance occurs.

[0015] Furthermore, the airport images are collected in all directions using high-resolution multi-channel cameras; the image enhancement includes: random cropping, random flipping, scaling and rotation, adjusting brightness and contrast, adding noise, and mosaic data enhancement; the labels are annotated on the airport images using the labelimg annotation tool.

[0016] Furthermore, two models are obtained during training, the best model and the last model, and the best model is used as the trained airport foreign object small target detection model.

[0017] Furthermore, when training the airport foreign object small target detection model, the airport foreign object data set is divided into a training set, a validation set, and a test set according to a set ratio, and the set ratio is 7:2:1.

[0018] The present invention also discloses an airport foreign object recognition system based on an improved Yolov8s model. The system includes a processor, and the processor is used to process the embodiment of the airport foreign object recognition method based on the improved Yolov8s model.

[0019] Compared with the prior art, the present invention has the following advantages:

[0020] The airport foreign object recognition method based on the improved yolov8s model trains the airport foreign object small target detection model with airport images and corresponding labels in multiple scenes and multiple weather conditions, ensuring that the model has a large recognition range and high detection accuracy. The airport images are resized to a standard size, which can meet the input requirements of the model without losing image data. The model is constructed based on the yolov8s model, which has the characteristics of strong feature extraction capability, high model robustness, strong generalization capability, fast running speed and high reasoning efficiency, and can effectively detect objects of different sizes. Therefore, the airport foreign object small target detection model constructed using the yolov8s model can well identify foreign objects of different sizes in airport images, thereby avoiding the harm of foreign objects to airport aircraft. By introducing the visual attention mechanism, the model is helped to focus more on important features, further improving the accuracy and robustness of the model in target detection tasks.

[0021] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of an embodiment of a method for identifying foreign objects at an airport based on an improved yolov8s model according to the present invention;

[0023] Figure 2 Schematic diagram of the structure of the yolov8s model of an embodiment of the airport foreign object identification method based on the improved yolov8s model of the present invention;

[0024] Figure 3 Schematic diagram of the DSConv structure of an embodiment of the airport foreign object identification method based on the improved yolov8s model of the present invention;

[0025] Figure 4 Schematic diagram of the standard convolution structure of the prior art;

[0026] Figure 5 This is a schematic diagram of the spatial attention mechanism structure of an embodiment of the airport foreign object recognition method based on the improved yolov8s model of the present invention;

[0027] Figure 6 This is a schematic diagram of the confusion matrix of the target recognition results of the yolov8 model in the prior art;

[0028] Figure 7 This is a schematic diagram of the confusion matrix of target recognition results of an embodiment of the airport foreign object recognition method based on the improved yolov8s model of the present invention. DETAILED DESCRIPTION

[0029] Example of airport foreign object identification method based on improved yolov8s model:

[0030] like Figure 1 As shown in FIG, the airport foreign object recognition method based on the improved yolov8s model includes the following steps:

[0031] First, airport images are collected in all directions using high-resolution multi-channel cameras. The multi-scene, multi-weather airport images are resized to a standard size, and image enhancement processing is performed on the resized airport images. Foreign objects on the processed airport images are annotated, and corresponding labels are generated based on the annotations. A dataset is constructed using the multi-scene, multi-weather airport images and corresponding labels. Resizing involves filling the image with a transparent or black background, while preserving the original image data. Image enhancement includes random cropping, random flipping, scaling and rotation, adjusting brightness and contrast, adding noise, and mosaic data enhancement. By diversifying and expanding the data, the richness and diversity of the dataset are increased. Labels are annotated using the labelimg annotation tool. The camera resolution is 5120. Labels refer to the category of foreign objects.

[0032] Construct an airport foreign object small target detection model and use the dataset to train the airport foreign object small target detection model; use the airport image to be identified as the input of the trained airport foreign object small target detection model, output the corresponding label, and complete the identification of foreign objects in the airport image; the airport foreign object small target detection model is constructed based on the yolov8s model. When training the airport foreign object small target detection model, the airport foreign object dataset is divided into a training set, a validation set, and a test set according to a set ratio. Preferably, the ratio is set to 7:2:1. Two models are obtained during training, the best model and the last model. The best model is used as the trained airport foreign object small target detection model. The best model refers to the optimal model, and the last model refers to the last model trained. The forward reasoning generates output feature maps of four scales: 240×240, 120×120, 60×60, and 30×30. The classification category prediction and bbox prediction at the four scales are concatenated and dimensionally transformed before threshold filtering. After filtering, NMS (non-maximum suppression) is performed and the confidence level is adjusted to obtain the final airport foreign object target that meets the requirements.

[0033] like Figure 2As shown, specifically, the yolov8s model includes a backbone network, a neck layer, and a head detection head. The backbone network uses DSConv and C2F modules to complete feature extraction of the input image, and inputs the extracted features into the neck layer; DSConv (Depthwise Separable Convolution) includes depthwise convolution and point-by-point convolution, which are used to extract features from the image, obtain feature maps and use them as input to the neck layer. The neck layer extracts complex features from the input feature maps, obtains complex feature maps and uses them as input to the head detection head; the head detection head performs forward propagation calculations on the input complex feature maps to obtain the location information and category information of the foreign object, thereby determining the corresponding label and outputting it.

[0034] DSConv has the following features: DSConv splits standard convolution into depthwise convolution and pointwise convolution, allowing each input channel to be convolved with a different filter, reducing the number of parameters. Compared to standard convolution, with the same input and output channels, DSConv can significantly reduce the number of parameters, reduce model size, and lower computational costs. Because DSConv has fewer parameters, it is more computationally efficient. In resource-constrained scenarios such as mobile devices, DSConv can effectively reduce the amount of computation and accelerate model inference. DSConv maintains the spatial information in the input feature map, and features between different channels are learned independently, which helps to extract richer and more accurate features. DSConv performs well in resource-constrained scenarios such as mobile devices and embedded devices. In addition to being widely used in lightweight models, DSConv is often used in combination with other modules to provide the model with better feature expression capabilities.

[0035] In the backbone network, DSConv includes depthwise convolution and pointwise convolution. The number of depthwise convolutions = convolution kernel size × number of input channels; the parameter data of pointwise convolution = 1 × 1 × number of input channels × number of output channels. This is equivalent to replacing the standard convolution in yolov8s with DSConv. Compared with the standard convolution module, Figure 3 As shown in , a convolution kernel of DSConv is responsible for one channel, and one channel corresponds to one convolution kernel convolution. Figure 4 As shown in the figure, the standard convolution module is that each convolution kernel operates on each channel of the input image simultaneously. When the convolution kernel size is 5×5, the input channels are 16, and the output channels are 32. The parameter calculation is as follows:

[0036] Parameter count for a standard convolution: Each kernel needs to be convolved with each input channel, so the number of parameters is the kernel size multiplied by the number of input channels. In this case, each kernel has 5 × 5 × 16 = 400 parameters. Since there are 32 output channels, the total number of parameters is 400 × 32 = 12,800.

[0037] DSConv splits standard convolution into depthwise convolution and pointwise convolution. The number of parameters for depthwise convolution is: For depthwise convolution, each output channel is convolved with only one input channel, so the number of parameters is: kernel size × number of input channels. Therefore, the number of parameters for depthwise convolution is: 5 × 5 × 16 = 400. The number of parameters for pointwise convolution is: Pointwise convolution is a standard 1 × 1 convolution. For each output channel, there is a convolution kernel of size 1 × 1 × number of input channels, so the number of parameters is: 1 × 1 × 16 × 32 = 512. Therefore, the total number of parameters is the sum of the number of parameters for depthwise convolution and pointwise convolution, total number of parameters = 400 + 512 = 912.

[0038] Replacing the standard 5x5 convolution kernel with a depthwise separable convolution reduces the number of parameters from 12,800 to 912. This demonstrates that, while achieving the same output, replacing standard convolution with depthwise separable convolution significantly reduces the number of parameters. This can improve real-time detection for foreign object detection at airports.

[0039] In the backbone network, a spatial attention mechanism is added between the DSConv and C2F modules. This mechanism assigns different attention weights to different spatial locations in the input image based on a trained weight vector. This spatial attention mechanism leverages the visual spatial attention mechanism to improve the model's robustness and generalization, enabling better detection of airport foreign objects in diverse scenarios. Specifically, the spatial attention mechanism focuses on specific spatial regions in a two-dimensional image, enhancing the model's feature extraction capabilities. By assigning different attention weights to different spatial locations in the image, the model can better capture important regions. When assigning attention weights, the model learns a weight vector corresponding to each spatial location in the image. The magnitude of the weight indicates the model's attention to that location. Weight assignment is typically implemented using a learnable parameter matrix, which is convolved with the feature map to produce an attention map. Applying the attention weights to the feature map allows the model to enhance the feature response of a region while suppressing others. This is typically achieved through element-wise multiplication, where the feature map is multiplied element-wise with the attention map.

[0040] like Figure 5As shown in the figure, this visual attention mechanism processes the output image u of the DSConv convolutional layer. Output u serves as the input to the localization network, which extracts features from the input image. The localization network outputs a transformation parameter θ, which defines the spatial transformation of the input image. Based on the transformation parameter θ, the grid generator creates a grid G containing the target location information for each pixel in the output image. These locations are calculated based on the transformation parameter θ. The sampler samples from the input image u based on the grid G. Based on the coordinate information in grid G, the sampler finds the corresponding pixel value in the input image u and copies it to the corresponding location in the output image. After processing by the spatial transformer, the input image u is transformed into a new image v. The spatial transformer dynamically adjusts the spatial structure of the input image based on the transformation parameters predicted by the localization network. This helps the model focus more closely on important features, further improving its accuracy and robustness in the airport foreign object detection task.

[0041] The neck layer uses a cross-scale feature fusion module. This cross-scale feature fusion model extracts features from the input feature map using feature boxes of multiple sizes, then superimposes the extracted feature maps of different scales along the channel dimension, and then fuses the features through a convolutional layer to produce a complex feature map. This cross-scale feature fusion module is a technique used to integrate feature representations of different scales to improve model performance. In image processing and computer vision tasks, multi-scale feature maps are used to capture diverse image information. The cross-scale feature fusion module fuses these multi-scale features, superimposing the feature maps of different scales along the channel dimension and then fusing them through a convolutional layer. This produces a more comprehensive and richer feature representation.

[0042] In the head detection head, the head detection head includes 4 detect detectors, which are used to detect feature maps of different sizes. The yolov8s model without p2 improvement should have 3 detection heads in this embodiment. After p2 improvement, it is equivalent to adding a detection head. The added detection head is Figure 2 The first detect detector in . This is used to enhance the model's ability to detect small objects of different sizes.

[0043] In the detection head, a direct connection is introduced into the forward propagation path. This path is used to propagate information when vanishing gradients occur. This is effectively a residual connection. Its purpose is to transmit information through this direct path to address the vanishing gradient problem during deep neural network training and improve model training and convergence speed. Vanishing gradient refers to the situation where, during the network's backpropagation, the gradient used to update the network weights becomes so small that it barely produces any noticeable update. Therefore, when vanishing gradients occur, the output is directly transmitted through the direct connection path. The output of a residual connection network = main path (input) + input.

[0044] like Figure 6 、 Figure 7 As shown, the confusion matrix summarizes the records in the data set in the form of a matrix according to the two criteria of the real category and the category predicted by the classification model. The rows of the matrix represent the real values, the columns of the matrix represent the predicted values, and the values on the main diagonal of the confusion matrix represent the number of correct predictions. It can be found that Figure 7 The predicted value in Figure 6 The predicted correct value in indicates that the effect of this embodiment is better than the recognition effect of the traditional yolov8 model. Figure 6 、 Figure 7 In the example, confusionmatrix normalized is the normalized confusion matrix, predicted is the prediction, ture is the truth, background is the background category, stone is stone, spring_washer is spring, gasket is gasket, nut is nut, and screw is screw. Specifically, compared with the existing yolov8 network structure, this method in this embodiment obtains higher accuracy and recall rate in target recognition tasks, thereby improving detection accuracy. The DSConv and spatial attention mechanisms reduce parameters, improve computational efficiency, and thus improve recognition speed. Compared with other yolov8 models, the network structure of the yolov8s model adds an output head. These four output heads are usually used to detect targets of different scales and aspect ratios to enhance the detection capability of the model, so that the model can more comprehensively capture targets in the image and perform better in small target detection.

[0045] Example of airport foreign object recognition system based on improved yolov8s model:

[0046] The present invention also discloses an airport foreign object identification system based on an improved YOLOv8s model, which includes a processor for processing the embodiment of the above-mentioned airport foreign object identification method based on the improved YOLOv8s model. This embodiment is implemented with reference to the embodiment of the above-mentioned airport foreign object identification method based on the improved YOLOv8s model, and will not be repeated here.

[0047] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for identifying foreign objects at airports based on an improved yolov8s model, characterized by: The method includes the following steps: The multi-scene, multi-weather airport images were resized to a standard size, and image enhancement processing was performed on the resized airport images. Foreign objects in the processed airport images were annotated, and corresponding labels were generated based on the annotations. A dataset was constructed using the multi-scene, multi-weather airport images and the corresponding labels. A small foreign object detection model for airports was constructed and trained using the dataset. The airport image to be identified is used as the input of the trained airport foreign object small target detection model, and the corresponding label is output to complete the identification of foreign objects in the airport image; The airport foreign object small target detection model is constructed based on the yolov8s model, which includes a backbone network. The backbone network uses DSConv and C2F modules to complete feature extraction of the input image and input the extracted features into the neck layer; A visual attention mechanism is added between the DSConv and C2F modules. The visual attention mechanism assigns different attention weights to different spatial locations of the input image based on the trained weight vector. The visual attention mechanism works as follows: the DSConv output image u is used as the input of the localization network, a transformation parameter θ is output, a grid generator creates a grid G based on the transformation parameter θ, a sampler samples from the DSConv output image u based on the grid G, and a spatial transformer processes the output image u based on the sampling result and outputs a new image v. The yolov8s model also includes a neck layer and a head detection head; DSConv includes depth convolution and point-by-point convolution, which are used to extract features from images to obtain feature maps and serve as input to the neck layer. The neck layer extracts complex features from the input feature maps to obtain complex feature maps and serve as input to the head detection head; the head detection head performs forward propagation calculations on the input complex feature maps to obtain the location information and category information of the foreign object, thereby determining the corresponding label and outputting it; The neck layer adopts a cross-scale feature fusion module. The cross-scale feature fusion model refers to: extracting features from the input feature map through feature frames of multiple sizes, superimposing the extracted feature maps of different scales according to the channel dimension, and then fusing the features through a convolutional layer to obtain a complex feature map; The head detection head includes 4 detect detectors, and the 4 detect detectors are used to detect feature maps of different sizes respectively; A direct connection path is introduced into the forward propagation path, and propagation is carried out through the direct connection path when gradient disappearance occurs.

2. The method for identifying foreign objects at airports based on the improved Yolov8s model according to claim 1, characterized in that: The DSConv includes depthwise convolution and pointwise convolution. The number of depthwise convolutions = convolution kernel size × number of input channels; the parameter data of pointwise convolution = 1 × 1 × number of input channels × number of output channels.

3. The method for identifying foreign objects at airports based on the improved Yolov8s model according to claim 1, characterized in that: The airport images are collected in all directions using high-resolution multi-channel cameras; the image enhancement includes: random cropping, random flipping, scaling and rotation, adjusting brightness and contrast, adding noise, and mosaic data enhancement; the labels are annotated on the airport images using the labelimg annotation tool.

4. The method for identifying foreign objects at airports based on the improved Yolov8s model according to claim 1, characterized in that: During the training, two models are obtained, namely the best model and the last model, and the best model is used as the trained airport foreign object small target detection model.

5. The method for identifying foreign objects at airports based on the improved Yolov8s model according to claim 1, characterized in that: When training the airport foreign object small target detection model, the airport foreign object data set is divided into a training set, a validation set, and a test set according to a set ratio, and the set ratio is 7:2:

1.

6. An airport foreign object recognition system based on an improved yolov8s model, characterized by: The system includes a processor, which is used to process the airport foreign object identification method based on the improved yolov8s model described in any one of claims 1 to 5.

Images