Foreign matter identification and detection method for unmanned motor sweeper in airport
By introducing the ECA-CA hybrid attention mechanism and the DIoU edge box regression loss function into the YOLOv7 network model, combined with data augmentation strategies, the problem of insufficient foreign object detection accuracy on airport runways was solved, achieving efficient foreign object identification and detection and improving detection accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENYANG RUIMA TECH CO LTD
- Filing Date
- 2026-01-27
- Publication Date
- 2026-05-22
AI Technical Summary
The existing YOLOv7 network structure has insufficient detection accuracy in foreign object detection on airport runways, and it is difficult to meet the model requirements when the dataset is small, thus failing to achieve efficient foreign object recognition.
By introducing the ECA-CA hybrid attention mechanism and the DIoU edge box regression loss function into the YOLOv7 network model, and combining random and mosaic data augmentation strategies, an airport runway foreign object detection model is constructed to improve the sensitivity and positioning accuracy of small target information detection.
It has achieved accurate identification and detection of foreign objects on airport runways, improved detection accuracy, and reached an average accuracy of 98.69%, meeting the high accuracy and high efficiency requirements of airport foreign object detection.
Smart Images
Figure CN122073045A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent vehicle airport foreign object identification and detection technology, and specifically relates to a foreign object identification and detection method for airport unmanned cleaning vehicles. Background Technology
[0002] Foreign objects (FOOs) on airport runways typically exhibit diversity, varying sizes, concealment, and dynamic characteristics. To accurately assess the hazard level of these FEOs and implement appropriate countermeasures, high-performance FEO detection on airport runways requires a high-quality dataset and demands high detection speed and accuracy from the FEO identification algorithm. Furthermore, mandatory civil aviation regulations necessitate accurate recording of FEOs at airports. Therefore, accurate identification of FEOs on airport runways is essential.
[0003] The YOLOv7 network structure is a target detection algorithm that mainly consists of four parts: the input end, the core network, the feature integration network, and the prediction end. The algorithm has a fast detection speed, but its detection accuracy is insufficient for foreign object detection and recognition on airport runways.
[0004] In addition, training with airport foreign object data requires a large dataset, and when the available dataset is relatively small, it cannot meet the needs of the model. Summary of the Invention
[0005] The purpose of this invention is to address the diversity and concealment of foreign objects on airport runways by providing a method for identifying and detecting foreign objects using an unmanned airport cleaning vehicle. This method can identify foreign objects on airport runways using an unmanned airport cleaning vehicle and improve the accuracy of foreign object identification on airport runways.
[0006] The technical solution provided by this invention is as follows: A method for foreign object identification and detection in an airport unmanned cleaning vehicle includes: Images of foreign objects on airport runways were collected and labeled to construct an airport runway foreign object detection dataset. The YOLOv7 network model was improved, and the improved YOLOv7 network model was trained based on the airport runway foreign object detection dataset to obtain the airport runway foreign object detection model. The method to improve the YOLOv7 network model is as follows: introduce an ECA-CA hybrid attention module into the feature output of each ELAN module in the Backbone of the YOLOv7 network model, and use the DIoU bounding box regression loss function as the bounding box loss function for model training. The airport runway images are collected in real time by cameras on the airport unmanned cleaning vehicle, and the airport runway images are input into the airport runway foreign object detection model. The airport runway foreign object detection model outputs the type of foreign object detected.
[0007] Preferably, the model training process also includes: The acquired airport runway foreign object images were augmented, and the augmented airport runway foreign object images and the acquired airport runway foreign object images were used together as the training dataset. The methods for enhancing the acquired images of foreign objects on airport runways include: random data augmentation strategy and mosaic data augmentation strategy.
[0008] Preferably, the method for enhancing the acquired airport runway foreign object images is determined according to the following rules; If the random number is less than 0.5 and the training generations are less than 70, a mosaic augmentation strategy is used; otherwise, a random data augmentation strategy is used.
[0009] Preferably, the DIoU edge box regression loss function The expression is: ; in, This represents the coordinate regression loss. Indicates the target confidence loss. This represents the category confidence loss.
[0010] Preferably, the formula for calculating the coordinate regression loss is: ; in, and These represent the center points of the predicted bounding box and the ground truth bounding box, respectively. yes and The Euclidean distance between them Indicates the size of the feature map. This indicates the number of predicted bounding boxes for each cell. This represents the cell index on the feature map. Indicates the bounding box index. Indicates the first The first cell Does a bounding box contain an object? =1 indicates that the target is included; otherwise... =0; Indicates intersection, union, and ratio. This indicates the number of indices in the feature map.
[0011] Preferably, the formula for calculating the target confidence loss is: ; in, Indicates the size of the feature map. This indicates the number of predicted bounding boxes for each cell. This represents the cell index on the feature map. Indicates the bounding box index. Indicates the first The first cell Does a bounding box contain an object? =1 indicates that the target is included; otherwise... =0; This indicates the confidence level of the predicted bounding box. This indicates the confidence level of the labeled bounding box. Represents the sample weight coefficient. Indicator function representing negative samples.
[0012] Preferably, the formula for calculating the category confidence loss is: ; in, Indicates the size of the feature map. This represents the cell index on the feature map. Indicates the bounding box index. Indicates the target category index. Represents the target category dataset. This represents the class probability of the labeled target bounding box. Indicates the first The first cell Does a bounding box contain an object? =1 indicates that the target is included; otherwise... =0.
[0013] Preferably, the airport unmanned cleaning vehicle foreign object identification and detection method further includes: When collecting images of foreign objects on airport runways, each type of foreign object is collected multiple times under different lighting and weather conditions. The images collected under various lighting and weather conditions are then combined to form an airport runway foreign object detection dataset.
[0014] The beneficial effects of this invention are: The airport unmanned cleaning vehicle foreign object identification and detection method provided by the present invention improves the sensitivity of small target information detection by introducing the CA attention mechanism into the YOLOv7 network model, and enhances the positioning accuracy of the airport unmanned cleaning vehicle foreign object identification and detection by adopting the DIoU edge box regression loss function, which can realize the accurate detection and identification of foreign objects on airport runways by unmanned cleaning vehicles. Attached Figure Description
[0015] Figure 1 This is a diagram illustrating the effect of the random data enhancement described in this invention.
[0016] Figure 2 This is an illustration of the effect of mosaic data enhancement described in this invention.
[0017] Figure 3 This is a flowchart illustrating the hybrid data enhancement described in this invention.
[0018] Figure 4 This is a structural diagram of the CA attention mechanism described in this invention.
[0019] Figure 5 This is the mAP graph of the YOLOv7 network model for foreign object recognition on various airport runways.
[0020] Figure 6 This is the mAP diagram of the improved network model in this invention for the identification of foreign objects on various airport runways. Detailed Implementation
[0021] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.
[0022] This invention provides a method for foreign object identification and detection in airport unmanned cleaning vehicles, the specific implementation process of which is as follows.
[0023] I. Construction and annotation of the foreign object detection dataset for airport runways.
[0024] 1. The foreign object detection dataset considers foreign object type, lighting conditions, and weather conditions. Specifically, images of various foreign objects are collected under different lighting and weather conditions, and these images are combined to form a dataset. The collected dataset is then labeled to obtain the airport runway foreign object detection dataset. The labeling includes marking the location and type of foreign objects on the airport runway. The dataset includes a total of 31 classification labels, each with a unique appearance and size, covering most of the unusual items that may be encountered on airport runways. Specific foreign object types are shown in Table 1.
[0025] Table 1. English-Chinese Glossary of Foreign Object Types
[0026] Second, a hybrid data augmentation strategy combining random data augmentation and mosaic data augmentation is used to improve the richness of training data and enhance the generalization effect of the model.
[0027] Random data augmentation aims to generate more and richer data samples by performing a series of transformations on the original data, thereby enhancing the model's generalization ability and robustness. Common methods include data augmentation, which involves geometric transformations to ensure the image's length and width are distorted to match the input size, followed by color manipulations such as brightness, contrast, and hue, and finally adding gray bars to redundant parts of the image. The effects of random data augmentation include... Figure 1 As shown.
[0028] Mosaic data augmentation divides an image into small patches and replaces the original pixel values with the average color value of each patch, thus achieving a blurring effect. Four images are randomly selected from the dataset, denoted as... n 1 , n 2 , n 3 , n 4 and set its label to t 1 , t 2 , t 3 , t 4 Then, geometric transformations are performed on the image, including various random operations such as flipping, rotating, shearing, deforming, and compressing. The original four images are then arranged sequentially at the top left, bottom left, bottom right, and top right. Fixed regions of the four images are cropped and stitched together to form a new image, using matrix operations. The remaining portion is then filled with gray borders, denoted as . N The expression is:
[0029] The corresponding real-world label location label is:
[0030] in, n 1, n 2, n 3, n 4 represents four images randomly selected from the dataset. t 1, t 2, t 3, t 4 represents the true labels corresponding to the four images. The mosaic enhancement effect is as follows: Figure 2 As shown.
[0031] This invention employs a combination of random data augmentation and mosaic data augmentation. Mosaic data augmentation is removed in the last 30% of the training generations to improve the training effect of the network. Finally, the coordinates of the image and bounding box are obtained after data augmentation, and the obtained image is normalized.
[0032] In one embodiment, the flowchart of the hybrid data augmentation strategy is as follows: Figure 3 As shown, when the random number is less than 0.5 and the training generations are less than 70, the mosaic data augmentation strategy is used; otherwise, the random data augmentation strategy is used.
[0033] Third, the YOLOv7 network model is improved, and the improved YOLOv7 network model is trained based on the enhanced airport runway foreign object detection dataset to obtain the airport runway foreign object detection model.
[0034] This invention uses the YOLOv7 network model as the basis for foreign object detection in airport unmanned cleaning vehicles. It introduces an ECA-CA hybrid attention mechanism and a DIoU edge box regression loss function into the YOLOv7 network architecture. An ECA-CA attention mechanism is embedded at the output of the ELAN module in the backbone to improve the sensitivity of small target information detection, and the DIoU edge box regression loss function is used to enhance the localization accuracy of foreign object detection in the airport unmanned cleaning vehicle.
[0035] The ECA-CA hybrid attention mechanism first performs weighted processing on the channel attention through ECA channel attention calculation, that is: ; in, X Represents the features of the input. X ECA This indicates the weighted characteristics of the ECA mechanism. w This is the channel weight vector.
[0036] ; in, R^( k One-dimensional convolution kernel weights of (×1×1) This indicates global average pooling.
[0037] For adaptive convolution kernels k The size is: ; in, k The result is taken as the odd number closest to the calculated value. C Indicates the number of channels. Indicates the offset. This represents the scaling factor. In this embodiment, it is taken as...b =1, =2.
[0038] Global average pooling can be expressed as: ; in, , Indicates spatial dimension, Indicates the feature map at location The value of .
[0039] The weighted channel attention is imported into the CA module. The CA attention process is as follows: Figure 4 As shown, the CA attention mechanism fuses the input feature image coordinate data into a single input. x The data is then pooled globally according to a preset path, resulting in a pooled output C×H×1 in the x-direction and a pooled output C×1×W in the y-direction. The pooled information is then concatenated to obtain an output C×1×(H+W), with a compression ratio of r=16. The concatenated data is then convolved to obtain C / r×1×(H+W), with a compression ratio of r=16. Batch normalization is then performed to accelerate convergence. Finally, coordinate attention is generated through fully connected operations to obtain the height feature C / r×H×1 and the width feature C / r×1×W.
[0040] The process of applying two different global average pooling methods to the input feature image (C×H×W) is called coordinate information embedding. Global average pooling allows the attention module to encode global spatial information, enabling remote information interaction and obtaining accurate location coordinates and regions of interest. This coordinate data is then fused into a single input. x When the pooling kernels are (m,1) and (1,n), global average pooling is performed according to a preset path, where m and n represent the height and width of the feature map, respectively.
[0041] Global average pooling results include: Height is h The c Pooling results for each channel Z c h : ; Width is w The c Pooling results for each channel Z c w : ; Output of coordinate attention f for: ; in, F 1 indicates the pooled result Z h , Z w Perform matrix join. δ This represents a nonlinear activation function; , These represent the pooling results for height h and width w, respectively. This represents a coordinate in the feature map.
[0042] By embedding an ECA-CA hybrid attention mechanism at the output of the ELAN module in the backbone and adopting a serial structure, the input feature map passes through the ECA module. This module generates channel attention weights through one-dimensional fast convolution and the sigmoid function, improving the feature response of important channels with extremely low computational cost. The feature map output from the ECA module is then fed into the CA module for precise coordinate information encoding, capturing long-range spatial dependencies. This achieves the collaborative work of two-level attention—first coarse channel selection, then precise position marking—to improve sensitivity to small targets. The DIoU loss function directly shortens the distance between two boxes, significantly enhancing target recognition. The DIoU bounding box regression loss function consists of class confidence loss, coordinate regression loss, and target confidence loss. Loss The value is: ; Coordinate Regression Loss L DIoU for: ; Target confidence loss L conf for: ; Category confidence loss L class for: ; in, and These represent the center points of the predicted bounding box and the ground truth bounding box, respectively. yes and The Euclidean distance between them Indicates the size of the feature map. This indicates the number of predicted bounding boxes for each cell. This represents the cell index on the feature map. Indicates the bounding box index. Indicates the first The first cell Does a bounding box contain an object? =1 indicates that the target is included; otherwise... =0; Indicates intersection, union, and ratio. Indicates the number of indices in the feature map; This indicates the confidence level of the predicted bounding box. This indicates the confidence level of the labeled bounding box. =0.5 indicates the sample weight coefficient. Indicator function representing negative samples; Indicates the target category index. Represents the target category dataset. This represents the class probability of the labeled target bounding box.
[0043] In one embodiment, setting B =3.
[0044] Fourth, real-time images of the airport runway are captured by cameras installed on the airport unmanned cleaning vehicle, and these images are input into the airport runway foreign object detection model. The airport runway foreign object detection model outputs the category of the detected foreign object. In addition, the foreign object recognition system (including cameras and detection models) installed on the airport unmanned cleaning vehicle can detect foreign objects while cleaning, eliminating the need for a dedicated detection device and making the detection process more convenient and efficient.
[0045] This invention optimizes the positioning accuracy of small targets by using the center distance penalty of DIoU and enhances the extraction of foreign object features by combining the ECA-CA hybrid attention mechanism. It can realize FOD detection and retrieval in the complex environment of airports and improve the accuracy of foreign object identification on airport runways.
[0046] Tests were conducted using the improved model and YOLOv7 network model from this invention, from... Figure 5-6 As can be seen from the results, the performance on the validation set is good. Furthermore, the mean average accuracy (mAP) of the YOLOv7 network model for identifying various types of foreign objects is 97.55% (e.g., ...). Figure 5 As shown), the improved network model in this invention achieves a mean average accuracy (mAP) of 98.69% for identifying various types of foreign objects (e.g., ...). Figure 6 As shown in the figure, the improved model in this invention improves the mean average accuracy (mAP) by 1.14% compared to the YOLOv7 network model, proving that the method in this paper can effectively detect and identify foreign object debris (FOD) at airports. The improved network model in this invention has better recognition accuracy than the YOLOv7 network model.
[0047] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.
Claims
1. A method for foreign object identification and detection in an airport unmanned cleaning vehicle, characterized in that, include: Images of foreign objects on airport runways are collected and labeled to construct an airport runway foreign object detection dataset. The YOLOv7 network model was improved, and the improved YOLOv7 network model was trained based on the airport runway foreign object detection dataset to obtain the airport runway foreign object detection model. The method to improve the YOLOv7 network model is as follows: introduce an ECA-CA hybrid attention module into the feature output of each ELAN module in the Backbone of the YOLOv7 network model, and use the DIoU bounding box regression loss function as the bounding box loss function for model training. The airport runway images are collected in real time by cameras on the airport unmanned cleaning vehicle, and the airport runway images are input into the airport runway foreign object detection model. The airport runway foreign object detection model outputs the type of foreign object detected.
2. The method for foreign object identification and detection of airport unmanned cleaning vehicles according to claim 1, characterized in that, The model training process also includes: The acquired airport runway foreign object images were augmented, and the augmented airport runway foreign object images and the acquired airport runway foreign object images were used together as the training dataset. The methods for enhancing the acquired images of foreign objects on airport runways include: random data augmentation strategy and mosaic data augmentation strategy.
3. The method for foreign object identification and detection of airport unmanned cleaning vehicles according to claim 2, characterized in that, The method for enhancing the acquired airport runway foreign object images is determined according to the following rules; If the random number is less than 0.5 and the training generations are less than 70, a mosaic augmentation strategy is used; otherwise, a random data augmentation strategy is used.
4. The method for foreign object identification and detection of an airport unmanned cleaning vehicle according to any one of claims 1-3, characterized in that, The DIOU edgebox regression loss function The expression is: ； in, This represents the coordinate regression loss. Indicates the target confidence loss. This represents the category confidence loss.
5. The method for foreign object identification and detection of an airport unmanned cleaning vehicle according to claim 4, characterized in that, The formula for calculating the coordinate regression loss is as follows: ； in, and These represent the center points of the predicted bounding box and the ground truth bounding box, respectively. yes and The Euclidean distance between them Indicates the size of the feature map. This indicates the number of predicted bounding boxes for each cell. This represents the cell index on the feature map. Indicates the bounding box index. Indicates the first The first cell Does a bounding box contain an object? =1 indicates that the target is included; otherwise... =0; Indicates intersection, union, and ratio. This indicates the number of indices in the feature map.
6. The method for foreign object identification and detection of an airport unmanned cleaning vehicle according to claim 5, characterized in that, The formula for calculating the target confidence loss is as follows: ； in, Indicates the size of the feature map. This indicates the number of predicted bounding boxes for each cell. This represents the cell index on the feature map. Indicates the bounding box index. Indicates the first The first cell Does a bounding box contain an object? =1 indicates that the target is included; otherwise... =0; This indicates the confidence level of the predicted bounding box. This indicates the confidence level of the labeled bounding box. Represents the sample weight coefficient. Indicator function representing negative samples.
7. The method for foreign object identification and detection of an airport unmanned cleaning vehicle according to claim 6, characterized in that, The formula for calculating the category confidence loss is as follows: ； Where represents the size of the feature map. This represents the cell index on the feature map. Indicates the bounding box index. Indicates the target category index. Represents the target category dataset. This represents the class probability of the labeled target bounding box. Indicates the first The first cell Does each bounding box contain a target? A value of 1 indicates that the bounding box contains a target; otherwise... =0.
8. The method for foreign object identification and detection of an airport unmanned cleaning vehicle according to claim 1, characterized in that, Also includes: When collecting images of foreign objects on airport runways, each type of foreign object is collected multiple times under different lighting and weather conditions. The images collected under various lighting and weather conditions are then combined to form an airport runway foreign object detection dataset.