Airport runway foreign matter detection method, system and device based on convolutional neural network

Abstract

The application discloses an airport runway foreign matter detection method, system and equipment based on a convolutional neural network, first, airport runway image data is acquired; then, the airport runway image data is input into the convolutional neural network, and a detection result is output, the detection result contains a category to which a runway foreign matter target belongs and a position of the runway foreign matter target on an image; the application improves the YOLOv5 convolutional neural network on the basis of the YOLOv5 convolutional neural network, improves the detection precision for small runway foreign matter targets while ensuring the detection speed, can meet the requirements of real-time detection of runway foreign matters, provides an efficient detection method for runway foreign matter detection, and reduces the consumption of artificial cost.

Description

Technical Field

[0001] This invention belongs to the field of airport safety control technology, and relates to a method, system and equipment for detecting foreign objects on airport runways, and more particularly to a method, system and equipment for detecting foreign objects on airport runways based on an improved YOLOv5 convolutional neural network. Background Technology

[0002] With the continuous development of the global economy, air transport volume is also increasing rapidly, and the number of airports around the world is also growing. This means that the number of aircraft taking off from airports every day will increase, and the difficulty of ensuring aircraft safety will also increase accordingly. Aviation safety has always been a focus of public attention, and foreign object debris (FOD) on airport runways poses a significant threat to aviation safety.

[0003] FOD (Foreign Object Debris) refers to foreign objects within the airspace that may cause damage or harm to aircraft. Examples include screws, wrenches, metal strips, and rubber parts of tires. These foreign objects seriously endanger flight safety and can cause significant economic losses to airlines. Currently, airport managers worldwide are strengthening runway safety to prevent foreign objects from posing a threat to flights during takeoff and landing.

[0004] Some airports detect FOD (Fouling Objects) through manual inspections, while other current FOD detection systems utilize radar detection and video image recognition technologies. Radar detection technology is characterized by high resolution and long detection range, but it cannot provide color characteristics of the detected objects or intuitive video images, hindering subsequent FOD removal efforts. In practical applications, due to the diverse types and characteristics of FOD, image recognition technology struggles to effectively describe and extract their features, severely impacting detection accuracy.

[0005] The emergence of deep learning algorithms has brought a different solution to the problem of FOD detection. In recent years, research in deep learning has made significant progress, with a growing number of increasingly sophisticated and mature object detection algorithms. Many general-purpose object detection algorithms are now applied in daily life and production, achieving good results. However, FODs are irregular in shape and vary in size. Directly using existing general-purpose object detection algorithms for FOD detection results in relatively low accuracy, and small targets within FODs cannot be accurately detected. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, this invention provides a method, system, and device for detecting foreign objects on airport runways based on an improved YOLOv5 convolutional neural network. This method can better solve the problem of low efficiency in clearing foreign objects from airport runways and has extremely wide application scenarios and market value.

[0007] The technical solution adopted by the method of the present invention is: a method for detecting foreign objects on airport runways based on convolutional neural networks, comprising the following steps:

[0008] Step 1: Acquire airport runway image data;

[0009] Step 2: Input the airport runway image data into the convolutional neural network and output the detection results, which include the category of the foreign object on the runway and its location in the image;

[0010] The convolutional neural network includes a backbone network, a neck end, and a prediction end;

[0011] The backbone network adopts the CSPDarknet53 structure, which consists of a Focus layer, a first CSP layer, a second CSP layer, a third CSP layer, a standard convolutional layer CBL, a spatial pyramid pooling operation layer SPP, and a coordinate attention module CA arranged sequentially. The Focus layer is used to perform interlaced sampling and stitching on the input image, converting the information of the image plane to the channel dimension. The CSP layer consists of consecutive standard convolutional layers CBL and CSP1_X. The spatial pyramid pooling operation layer SPP uses a method of extracting and fusing feature maps of different scales to enhance the detection accuracy of the model.

[0012] The Neck includes a first CSP module, a second CSP module, a third CSP module, a fusion layer Concat, a first CSP sub-module, a second CSP sub-module, a third CSP sub-module, and CSP2_1; each CSP module consists of CSP2_1, a standard convolutional layer CBL, and an upsampling layer; each CSP sub-module consists of sequentially connected CSP2_1 and standard convolutional layers CBL; the output of the coordinate attention module CA passes through the first CSP module, and after passing through the fusion layer Concat with the output of the third CSP layer, it is input into the second CSP module; the output of the second CSP module and the output of the second CSP layer pass through the fusion layer Co After being ncat'd, the output is input into the third CSP module; the output of the third CSP module and the output of the first CSP layer are concatted together by a fusion layer and then input into the first CSP sub-module; the output of the first CSP sub-module and the standard convolutional layer CBL output of the third CSP module are concatted together by a fusion layer and then input into the second CSP sub-module; the output of the second CSP sub-module and the standard convolutional layer CBL output of the second CSP module are concatted together by a fusion layer and then input into the third CSP sub-module; the output of the third CSP sub-module and the standard convolutional layer CBL output of the first CSP module are concatted together by a fusion layer and then input into CSP2_1;

[0013] The prediction end receives four CSP2_1 outputs from the Neck end, which are then processed by a convolutional layer (Conv) to obtain feature maps of different sizes. These feature maps are then input into the Detect module for subsequent target detection operations.

[0014] The technical solution adopted by the system of the present invention is: an airport runway foreign object detection system based on convolutional neural network, including the following data acquisition module and detection module;

[0015] The data acquisition model is used to acquire airport runway image data;

[0016] The detection module is used to input airport runway image data into a convolutional neural network and output detection results, which include the category of the foreign object on the runway and its location on the image.

[0017] The convolutional neural network includes a backbone network, a neck end, and a prediction end;

[0018] The backbone network adopts the CSPDarknet53 structure, which consists of a Focus layer, a first CSP layer, a second CSP layer, a third CSP layer, a standard convolutional layer CBL, a spatial pyramid pooling operation layer SPP, and a coordinate attention module CA arranged sequentially. The Focus layer is used to perform interlaced sampling and stitching on the input image, converting the information of the image plane to the channel dimension. The CSP layer consists of consecutive standard convolutional layers CBL and CSP1_X. The spatial pyramid pooling operation layer SPP uses a method of extracting and fusing feature maps of different scales to enhance the detection accuracy of the model.

[0019] The Neck includes a first CSP module, a second CSP module, a third CSP module, a fusion layer Concat, a first CSP sub-module, a second CSP sub-module, a third CSP sub-module, and CSP2_1; each CSP module consists of CSP2_1, a standard convolutional layer CBL, and an upsampling layer; each CSP sub-module consists of sequentially connected CSP2_1 and standard convolutional layers CBL; the output of the coordinate attention module CA passes through the first CSP module, and after passing through the fusion layer Concat with the output of the third CSP layer, it is input into the second CSP module; the output of the second CSP module and the output of the second CSP layer pass through the fusion layer Co After being ncat'd, the output is input into the third CSP module; the output of the third CSP module and the output of the first CSP layer are concatted together by a fusion layer and then input into the first CSP sub-module; the output of the first CSP sub-module and the standard convolutional layer CBL output of the third CSP module are concatted together by a fusion layer and then input into the second CSP sub-module; the output of the second CSP sub-module and the standard convolutional layer CBL output of the second CSP module are concatted together by a fusion layer and then input into the third CSP sub-module; the output of the third CSP sub-module and the standard convolutional layer CBL output of the first CSP module are concatted together by a fusion layer and then input into CSP2_1;

[0020] The prediction end receives four CSP2_1 outputs from the Neck end, which are then processed by a convolutional layer (Conv) to obtain feature maps of different sizes. These feature maps are then input into the Detect module for subsequent target detection operations.

[0021] The technical solution adopted by the device of the present invention is: an airport runway foreign object detection device based on convolutional neural networks, comprising:

[0022] One or more processors;

[0023] A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the convolutional neural network-based foreign object detection method for airport runways.

[0024] This invention improves upon the YOLOv5 convolutional neural network by adding a coordinate attention module (CA) at the end of the YOLOv5 convolutional neural network backbone. By fusing an attention mechanism into the YOLOv5 convolutional neural network, it improves detection accuracy, recall, and the network's mAP value, while reducing inference time and computational cost. A small target detection layer is added at the prediction end to improve the head network structure for detecting smaller foreign objects on airport runways, including screws and stones. An image segmentation layer is added to the YOLOv5 Detect module, decomposing the large target image into several smaller images for detection within the YOLOv5 convolutional neural network. Although this increases computational cost, it significantly improves the accuracy of small foreign object detection. This invention improves the detection accuracy of small foreign objects on airport runways while maintaining detection speed, meeting the requirements for real-time foreign object detection on airport runways, providing an efficient detection method, and reducing manual labor costs. Attached Figure Description

[0025] Figure 1 This is a diagram of the YOLOv5 convolutional neural network structure according to an embodiment of the present invention;

[0026] Figure 2 This is a diagram of the CSP1_X network structure according to an embodiment of the present invention;

[0027] Figure 3 This is a diagram of the residual layer Res unit network structure according to an embodiment of the present invention;

[0028] Figure 4 This is a diagram of the CSP2_2 network structure according to an embodiment of the present invention;

[0029] Figure 5 This is a diagram of the Focus network structure according to an embodiment of the present invention;

[0030] Figure 6 This is a diagram of the SPP network structure according to an embodiment of the present invention;

[0031] Figure 7 This is a CBL network structure diagram according to an embodiment of the present invention;

[0032] Figure 8 This is a diagram of the attention module CA network structure in an embodiment of the present invention. Detailed Implementation

[0033] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0034] This invention provides a method for detecting foreign objects on airport runways based on convolutional neural networks, comprising the following steps:

[0035] Step 1: Acquire airport runway image data;

[0036] Step 2: Input the airport runway image data into the convolutional neural network and output the detection results, which include the category of the foreign object on the runway and its location on the image.

[0037] The convolutional neural network structure in this embodiment is divided into three parts: backbone, neck, and head.

[0038] The backbone network of this implementation adopts the CSPDarknet53 structure, which consists of a Focus layer, a first CSP layer, a second CSP layer, a third CSP layer, a standard convolutional layer CBL, a spatial pyramid pooling operation layer SPP, and a coordinate attention module CA arranged sequentially. The Focus layer is used to perform interlaced sampling and stitching on the input image, transforming the information of the image plane to the channel dimension. The CSP layer consists of consecutive standard convolutional layers CBL and CSP1_X. The spatial pyramid pooling operation layer SPP uses the method of extracting and fusing feature maps of different scales to enhance the detection accuracy of the model.

[0039] The Focus layer in this implementation performs interlaced sampling and stitching on the input image, transforming the information of the image plane to the channel dimension. This allows for downsampling without losing feature information, while also reducing the computational complexity of the model and accelerating its training speed. The CSP layer mainly consists of consecutive standard convolutional layers CBL and CSP1_X, used to enhance the model's learning ability and reduce the number of parameters. The SPP layer employs a method of extracting and fusing feature maps of different scales to enhance the model's detection accuracy. The backbone network's main function is feature extraction;

[0040] The neck in this implementation includes a first CSP module, a second CSP module, a third CSP module, a fusion layer (Concat), a first CSP sub-module, a second CSP sub-module, a third CSP sub-module, and CSP2_1. Each CSP module consists of CSP2_1, a standard convolutional layer (CBL), and an upsampling layer. Each CSP sub-module consists of sequentially connected CSP2_1 and standard convolutional layers (CBL). The output of the coordinate attention module (CA) passes through the first CSP module and is then fused with the output of the third CSP layer (Concat) before being input into the second CSP module. The output of the second CSP module and the output of the second CSP layer... After being concatenated by the fusion layer, the output is input into the third CSP module; the output of the third CSP module and the output of the first CSP layer are concatenated by the fusion layer and input into the first CSP sub-module; the output of the first CSP sub-module and the standard convolutional layer CBL output of the third CSP module are concatenated by the fusion layer and input into the second CSP sub-module; the output of the second CSP sub-module and the standard convolutional layer CBL output of the second CSP module are concatenated by the fusion layer and input into the third CSP sub-module; the output of the third CSP sub-module and the standard convolutional layer CBL output of the first CSP module are concatenated by the fusion layer and input into CSP2_1;

[0041] This embodiment can transmit semantic information from top to bottom. It fuses the results of CSP modules at different levels through six Concat operations. The Neck end is responsible for mixing and combining features and passing these features to the prediction end.

[0042] In this implementation, the prediction end receives four CSP2_1 outputs from the Neck end, which are then processed by the Conv convolutional layer to obtain feature maps of different sizes. These feature maps are then input into the Detect module for subsequent object detection operations.

[0043] In this implementation, the prediction layer utilizes grid-based anchors to perform object detection on feature maps at different scales. The prediction layer receives the output from each layer of the Neck layer, performs Conv convolution operations to obtain feature maps of different sizes, and inputs these maps into the Detect module for subsequent object detection operations. After adding anchor boxes for small object detection layers, the input image size is 640*640, resulting in feature map sizes of 160x160, 80x80, 40x40, and 20x20 at the four scales.

[0044] Please see Figure 2In the convolutional neural network of this embodiment, CSP1_X divides the input into two branches. One branch passes through a sequentially connected convolutional layer CBL, X residual layers Resunit, and a single convolutional operation layer Conv. The other branch directly performs a convolutional operation layer Conv. The results from the two branches are sequentially connected to a fusion layer Concat, a normal distribution layer BN, an activation function layer LeakyReLU, and a convolutional layer CBL to obtain the output result. The value of X is selected according to the default configuration of different versions of YOLOv5.

[0045] Please see Figure 3 The residual layer Res unit in this embodiment consists of two sequentially connected CBL and ADD operations. The input is the output after passing through two convolutional layers CBL, and then the ADD operation is performed between the input and the output.

[0046] Please see Figure 4 In the convolutional neural network of this embodiment, CSP2_X divides the input into two branches. One branch passes through a sequentially connected convolutional layer CBL, 2*X convolutional layers CBL, and a single convolutional operation layer Conv. The other branch directly inputs into the convolutional operation layer Conv. The results from the two branches are then fed into a sequentially connected fusion layer Concat, a normal distribution layer BN, an activation function layer LeakyReLU, and a convolutional layer CBL to obtain the output. The value of X is selected according to the default configuration of different versions of YOLOv5.

[0047] Please see Figure 5 In this embodiment, the Focus module in the convolutional neural network includes four parallel slice layers, followed by a concatenation layer (CBL) and a convolutional layer (CBL). For the original 640*640*3 image input, a slice operation is first performed, transforming it into a 320*320*12 feature map. After concatenating the resulting feature maps (CBL), a final convolutional CBL operation is performed, resulting in a 320*320*64 feature map. In this embodiment, the Focus module in the convolutional neural network, for the original 640*640*3 image input, first performs a slice operation, transforming it into a 320*320*12 feature map. After concatenating the resulting feature maps (CBL), a final convolutional CBL operation is performed, resulting in a 320*320*64 feature map.

[0048] Please see Figure 6In this embodiment, the Spatial Pyramid Pooling (SPP) layer in the convolutional neural network includes a sequentially connected convolutional layer (CBL), four parallel slice layers (slice), a fusion layer (Concat), and another convolutional layer (CBL). Max pooling is performed on the maximum value within each bounding box of the 1x1, 2x2, and 4x4 sub-images, resulting in 1*1 + 2*2 + 4*4 = 21 extracted feature values. These features are then concatted, and finally passed through a CBL layer. The SPP structure can convert feature maps of arbitrary size into feature vectors of fixed size.

[0049] Please see Figure 7 In this embodiment, the CBL is a standard convolutional layer, which consists of a sequentially connected Conv, BN (normal distribution) layer and activation function layer (Leaky ReLU).

[0050] Please see Figure 8 In this embodiment, the coordinate attention module (CA) first divides the input feature map into width and height directions and performs global average pooling (XAvgPool, YAvgPool) separately to obtain feature maps in the width and height directions. Then, the feature maps obtained from the global receptive field in the width and height directions are concatenated. These concatenated feature maps are then fed into a shared 1x1 convolutional module (Conv2d) to reduce the dimension to C / r of the original. Next, Batch Normalization (BN) and Non-linear methods are used to encode the spatial information in the vertical and horizontal directions. The module is then split, and convolutions with 1x1 kernels are performed according to the original height and width to obtain the same number of channels as the input. After passing through a sigmoid activation function, attention weights gh in the height and width directions and attention weight gw in the width direction are obtained. Finally, the weights are normalized and re-weighted to obtain the final feature map with attention weights in the width and height directions.

[0051] The convolutional neural network used in this embodiment is a pre-trained model; the specific training process includes the following sub-steps:

[0052] Step 1: Obtain the dataset of foreign object images on the airport runway;

[0053] In this embodiment, the airport runway foreign object dataset is obtained by collecting images containing various runway foreign objects on-site at the airport, as well as images of similar objects collected online. The categories and locations of the foreign object targets in the images are labeled. Airport foreign object categories include stones, metals (screws, nuts, wrenches, etc.), and plastics (plastic bags, plastic bottles, etc.). A detection dataset containing multiple categories of airport foreign objects is established, and the dataset is divided into training and testing sets.

[0054] This embodiment first clarifies the types and sizes of foreign objects (FOOs) to be detected on the runway. Images are collected through on-site airport photography and online sources of similar objects. Then, the Labelimg object detection and annotation software is used to annotate the collected image dataset, generating XML files. The annotations primarily include: the VOC format of the airport runway FEO dataset, the FEO category of the object in the image, and its coordinates and dimensions within the image. Next, after annotation, the size of the dataset is calculated. A script written in Python converts all annotated files to TXT files and normalizes the coordinates, resulting in the airport runway FEO detection dataset. This dataset is then divided into training and testing sets in an 8:2 sample ratio.

[0055] Step 2: Input the foreign object images from the airport runway foreign object image dataset in Step 1 into the improved YOLOv5 convolutional neural network for training, and iterate and save the model every preset conditions. Finally, save the optimal model and the last model. The dataset in the training set is trained for 1000 iterations. The image size is set to 640×640. The batch size for each model iteration is set to 16. The initial learning rate is set to 0.01. Iterative training is performed using GPU in the created virtual environment.

[0056] In this embodiment, a coordinate attention module (CA) is added to the end of the YOLOv5 backbone network, and the SPP module completes the spatial pyramid pooling operation.

[0057] In this embodiment, the coordinate attention module (CA) divides the feature map obtained from the previous convolutional layer into two parts: width and height. Global average pooling is then performed on each part separately to obtain feature maps in the width and height directions. These two feature maps, embedding specific directional information, are encoded into two attention maps, storing the positional information within them. These two attention maps are then weighted and applied to the input feature map to enhance its representational power. In this embodiment, the SPP module performs max pooling on the features of each channel, extracts each feature, concatenates them by channel, and then performs a 1x1 convolution to achieve feature fusion.

[0058] This embodiment improves the head network structure by adding a small target detection layer to the prediction head of the YOLOv5 network model.

[0059] In this embodiment, the small object detection layer adds a set of anchors downsampled by four times to the initial anchors (anchor boxes, the initial parameters in YOLOv5), making it suitable for detecting small objects on runways. These anchors serve as the initial values ​​for the adaptive anchor box calculation built into YOLOv5. Based on the initial anchor boxes, the predicted boxes are output and then compared with the ground truth boxes (GroundTruth). The difference between the two is calculated, and then the network parameters are updated in reverse iteratively. In this embodiment, the anchor box values ​​of the improved YOLOv5 network model after adding the small object detection layer are set to [3,5,6,8,9,11], [10,13,16,30,33,23], [30,61,62,45,59,119], and [116,90,156,198,373,326].

[0060] The small target detection layer in this embodiment also includes, based on the original prediction head of the YOLOv5 network, after the 18th layer, continuing to perform upsampling and other processing on the feature map to further expand the feature map. At the same time, at the 21st layer, the obtained 160x160 feature map is concat-fused with the feature map of the second layer in the backbone network to obtain a larger feature map for foreign object small target detection.

[0061] This embodiment addresses the issue of small-sized foreign object targets on airport runways by adding an image segmentation layer to the YOLOv5 detect module. This layer decomposes the large target image into several smaller images, which are then fed into the YOLOv5 network for detection. Finally, all images are collected, and the predicted bounding boxes are placed onto the large image based on the cropping positions of the smaller images. Non-maximum suppression (NMS) is then applied to the entire large image to obtain the output result.

[0062] In this embodiment, when a large target image is decomposed into several smaller images, in order to avoid some targets being truncated, an Overlap region is set between two smaller images with an overlap percentage of 20%. For example, if the smaller images are 960*960 pixels in size, the overlap can be set to 960*20%=192 pixels.

[0063] This invention has the advantage of improving the detection accuracy of small foreign objects on airport runways while ensuring detection speed.

[0064] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A method for detecting foreign objects on airport runways based on convolutional neural networks, characterized in that, Includes the following steps: Step 1: Acquire airport runway image data; Step 2: Input the airport runway image data into the convolutional neural network and output the detection results, which include the category of the foreign object on the runway and its location in the image; The convolutional neural network includes a backbone network, a neck end, and a prediction end; The backbone network adopts the CSPDarknet53 structure, which consists of a Focus layer, a first CSP layer, a second CSP layer, a third CSP layer, a standard convolutional layer CBL, a spatial pyramid pooling operation layer SPP, and a coordinate attention module CA arranged sequentially. The Focus layer is used to perform interlaced sampling and stitching on the input image, converting the information of the image plane to the channel dimension. The CSP layer consists of consecutive standard convolutional layers CBL and CSP1_X. The spatial pyramid pooling operation layer SPP uses a method of extracting and fusing feature maps of different scales to enhance the detection accuracy of the model. The Neck includes a first CSP module, a second CSP module, a third CSP module, a fusion layer (Concat), a first CSP sub-module, a second CSP sub-module, a third CSP sub-module, and a CSP2_X. Each CSP module consists of a CSP2_X, a standard convolutional layer (CBL), and an upsampling layer. Each CSP sub-module consists of sequentially connected CSP2_X and standard convolutional layers (CBL). The output of the coordinate attention module (CA) passes through the first CSP module, is fused with the output of the third CSP layer, and then input into the second CSP module via a fusion layer (Concat). The output of the second CSP module is fused with the output of the second CSP layer and then input into the third CSP module via a fusion layer (Concat). The output of the third CSP module is fused with the output of the first CSP layer and then input into the first CSP sub-module. The output of the first CSP submodule and the standard convolutional layer CBL output of the third CSP module are concatted by a fusion layer and then input into the second CSP submodule. The output of the second CSP submodule and the standard convolutional layer CBL output of the second CSP module are concatted by a fusion layer and then input into the third CSP submodule. The output of the third CSP submodule and the standard convolutional layer CBL output of the first CSP module are concatted by a fusion layer and then input into CSP2_X. The prediction end receives four CSP2_X outputs from the Neck end, which are then processed by the Conv convolutional layer to obtain feature maps of different sizes. These feature maps are then input into the Detect module for subsequent target detection operations. In the convolutional neural network CSP1_X, the input is divided into two branches. One branch passes through a sequentially connected convolutional layer CBL, X residual layers Resunit, and a single convolutional operation layer Conv. The other branch directly performs a convolutional operation layer Conv. The results from the two branches are then sequentially connected to a fusion layer Concat, a normal distribution layer BN, an activation function layer LeakyReLU, and a convolutional layer CBL to obtain the output. The value of X is selected according to the default configuration of different versions of YOLOv5. In the convolutional neural network CSP2_X, the input is divided into two branches. One branch passes through a sequentially connected convolutional layer CBL, 2*X convolutional layers CBL, and a single convolutional operation layer Conv. The other branch directly inputs into the convolutional operation layer Conv. The results from the two branches are then fed into a sequentially connected fusion layer Concat, a normal distribution layer BN, an activation function layer LeakyReLU, and a convolutional layer CBL to obtain the output. The value of X is selected according to the default configuration of different versions of YOLOv5.

2. The method for detecting foreign objects on airport runways based on convolutional neural networks according to claim 1, characterized in that: The residual component Res unit takes the input as the output of two convolutional layers (CBL), and then performs an ADD operation with the input before outputting the result.

3. The method for detecting foreign objects on airport runways based on convolutional neural networks according to claim 1, characterized in that: The Focus module in the convolutional neural network described in step 2 includes four parallel slice operation layers, followed by a concatenation layer (Concat) and a convolutional layer (CBL). For the original 640*640*3 image input, the slice operation is first applied to transform it into a 320*320*12 feature map. After concatenating the obtained feature map with Concat, it undergoes another convolutional CBL operation to finally transform it into a 320*320*64 feature map.

4. The method for detecting foreign objects on airport runways based on convolutional neural networks according to claim 1, characterized in that: The Spatial Pyramid Pooling (SPP) layer in the convolutional neural network described in step 2 includes a sequentially connected convolutional layer (CBL), four parallel slice layers (slice), a fusion layer (Concat), and another convolutional layer (CBL). Max pooling is performed on the maximum value within each bounding box of the 1x1, 2x2, and 4x4 sub-images, resulting in a total of 1*1 + 2*2 + 4*4 = 21 extracted features. These features are then subjected to a Concat operation, and finally pass through a CBL layer.

5. The method for detecting foreign objects on airport runways based on convolutional neural networks according to claim 1, characterized in that: The convolutional layer CBL consists of a sequentially connected convolutional operation layer Conv, a normal distribution layer BN, and an activation function layer Leaky ReLU.

6. The method for detecting foreign objects on airport runways based on convolutional neural networks according to any one of claims 1-5, characterized in that: The convolutional neural network mentioned in step 2 is a trained convolutional neural network; the specific training process includes the following sub-steps: Step 2.1: Collect an image dataset of foreign objects on the airport runway, label the categories and locations of the foreign objects in the images, and divide the dataset into a training set and a detection set; Step 2.2: Input the training set into the convolutional neural network, train the convolutional neural network, and iterate and save the model every preset condition. When the number of training times reaches the preset value, finally save the optimal model to obtain the trained convolutional neural network model.

7. A foreign object detection system for airport runways based on convolutional neural networks, characterized in that, This includes the following data acquisition module and detection module; The data acquisition model is used to acquire airport runway image data; The detection module is used to input airport runway image data into a convolutional neural network and output detection results, which include the category of the foreign object on the runway and its location on the image. The convolutional neural network includes a backbone network, a neck end, and a prediction end; The backbone network adopts the CSPDarknet53 structure, which consists of a Focus layer, a first CSP layer, a second CSP layer, a third CSP layer, a standard convolutional layer CBL, a spatial pyramid pooling operation layer SPP, and a coordinate attention module CA arranged sequentially. The Focus layer is used to perform interlaced sampling and stitching on the input image, converting the information of the image plane to the channel dimension. The CSP layer consists of consecutive standard convolutional layers CBL and CSP1_X. The spatial pyramid pooling operation layer SPP uses a method of extracting and fusing feature maps of different scales to enhance the detection accuracy of the model. The Neck includes a first CSP module, a second CSP module, a third CSP module, a fusion layer (Concat), a first CSP sub-module, a second CSP sub-module, a third CSP sub-module, and a CSP2_X. Each CSP module consists of a CSP2_X, a standard convolutional layer (CBL), and an upsampling layer. Each CSP sub-module consists of sequentially connected CSP2_X and standard convolutional layers (CBL). The output of the coordinate attention module (CA) passes through the first CSP module, is fused with the output of the third CSP layer, and then input into the second CSP module via a fusion layer (Concat). The output of the second CSP module is fused with the output of the second CSP layer and then input into the third CSP module via a fusion layer (Concat). The output of the third CSP module is fused with the output of the first CSP layer and then input into the first CSP sub-module. The output of the first CSP submodule and the standard convolutional layer CBL output of the third CSP module are concatted by a fusion layer and then input into the second CSP submodule. The output of the second CSP submodule and the standard convolutional layer CBL output of the second CSP module are concatted by a fusion layer and then input into the third CSP submodule. The output of the third CSP submodule and the standard convolutional layer CBL output of the first CSP module are concatted by a fusion layer and then input into CSP2_X. The prediction end receives four CSP2_X outputs from the Neck end, which are then processed by the Conv convolutional layer to obtain feature maps of different sizes. These feature maps are then input into the Detect module for subsequent target detection operations. In the convolutional neural network CSP1_X, the input is divided into two branches. One branch passes through a sequentially connected convolutional layer CBL, X residual layers Resunit, and a single convolutional operation layer Conv. The other branch directly performs a convolutional operation layer Conv. The results from the two branches are then sequentially connected to a fusion layer Concat, a normal distribution layer BN, an activation function layer LeakyReLU, and a convolutional layer CBL to obtain the output. The value of X is selected according to the default configuration of different versions of YOLOv5. In the convolutional neural network CSP2_X, the input is divided into two branches. One branch passes through a sequentially connected convolutional layer CBL, 2*X convolutional layers CBL, and a single convolutional operation layer Conv. The other branch directly inputs into the convolutional operation layer Conv. The results from the two branches are then fed into a sequentially connected fusion layer Concat, a normal distribution layer BN, an activation function layer LeakyReLU, and a convolutional layer CBL to obtain the output. The value of X is selected according to the default configuration of different versions of YOLOv5.

8. A foreign object detection device for airport runways based on convolutional neural networks, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the airport runway foreign object detection method based on a convolutional neural network as described in any one of claims 1 to 6.

Images