Airport parking apron target detection method based on Faster-RCNN
Patent Information
- Application Number
- CN202510924318.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-04
- Publication Date
- 2025-10-28
Smart Images
Figure CN120852746A_ABST
Abstract
Description
Technical Field
[0001] This invention provides a method for airport apron target detection based on Faster-RCNN, belonging to the field of computer vision. Background Art
[0002] With the rapid development of the air transport industry, airport aprons, as the core area for aircraft takeoff, landing, parking, and ground support, directly affect the operation of the entire airport in terms of safety and management efficiency. Due to the complex environment of airport aprons, where multiple targets such as aircraft, ground vehicles, and personnel coexist, and which are affected by factors such as weather, lighting, obstruction, and scale changes, traditional target detection methods are difficult to achieve high detection accuracy in this scenario. Summary of the Invention
[0003] This invention discloses an airport apron target detection method based on Faster-RCNN, aiming to solve the problem of scarce target detection models and low detection accuracy in the current airport apron scenario.
[0004] An airport apron target detection method based on Faster-RCNN is proposed. This method utilizes the Faster-RCNN framework and optimizes it for the complex environment of airport aprons, improving the accuracy and efficiency of detecting key targets within the apron area and ensuring the system's efficiency and adaptability. The method comprises four steps: data acquisition, data processing, data analysis, and model building. Step 1: In the data acquisition step, the airport surveillance cameras continuously capture images of the tarmac area, including various targets such as aircraft, staff, and ground support vehicles.
[0005] In the data acquisition step, airport surveillance cameras continuously film the tarmac area to acquire image data under different environments. Keyframes are extracted using random sampling and interval sampling, and then expanded by incorporating images from some publicly available datasets to form the original dataset. The specific steps are as follows: Step 1.1: Airport surveillance cameras continuously capture images of the tarmac area to obtain data from different time periods, lighting conditions, and weather conditions, ensuring the diversity and representativeness of the data.
[0006] Step 1.2: For the surveillance video collected in Step 1.1, the video is processed using random frame extraction and interval frame extraction methods to obtain representative keyframe images.
[0007] Step 1.3: For the keyframe images obtained in Step 1.2, some publicly available datasets are introduced to supplement them, and the datasets of existing airport target detection tasks are combined to expand them, thereby enhancing the richness and generalization ability of the datasets.
[0008] Step 2: In the data processing step, the image data collected in Step 1 is processed using methods such as data cleaning, data augmentation, and target annotation.
[0009] In the data processing step, the original dataset is preprocessed to remove blurry images and redundant data, image targets are labeled, and geometric transformations are used to adjust lighting to enhance the original data, resulting in a processed dataset. The specific steps are as follows: Step 2.1: Data cleaning mainly involves quality screening of the raw image data collected in Step 1, removing blurry or noisy images to ensure the clarity and usability of the dataset.
[0010] Step 2.2: Data augmentation mainly involves using ensemble geometric transformations, illumination adjustments, noise additions, and other methods to enhance the diversity of the image data selected in Step 2.1 and improve the model's detection capabilities in complex environments.
[0011] Step 2.3: Data annotation mainly involves a combination of manual and semi-automatic annotation of the image data from Step 2.2. Target annotation is performed on the data, including targets such as aircraft, ground support vehicles, and personnel, to ensure the accuracy and high quality of the training data.
[0012] Step 3: In the data analysis step, perform detailed statistical analysis on the dataset processed in Step 2 to optimize the model training strategy.
[0013] In the data analysis step, the processed dataset is analyzed to determine the target categories, target sizes, and environmental factors within the airport, providing data support for the subsequent construction of a target detection model. The specific steps are as follows: Step 3.1: For the dataset processed in Step 2, perform target category distribution analysis to count the number distribution of targets such as airplanes, ground support vehicles, and staff.
[0014] Step 3.2: For the dataset processed in Step 2, perform target size analysis and statistically analyze the distribution of the number of large, medium, and small targets.
[0015] Step 3.3: Perform environmental factor analysis on the dataset processed in Step 2 to analyze the impact of different weather conditions (such as sunny days, cloudy days, and nighttime) on target detection accuracy.
[0016] Step 4: In the model building step, based on the data analysis results of Step 3, optimize the target detection framework based on Faster-RCNN to build an airport apron target detection model.
[0017] In the model building step, based on the Faster-RCNN framework, and considering the complex and ever-changing environment of airport aprons, the original Faster-RCNN framework is optimized to establish a target detection model suitable for airport aprons. This includes backbone network optimization, the use of deformable convolutions for feature extraction, and the introduction of a smoothing function into the loss function.
[0018] The backbone network of Faster R-CNN is optimized by constructing a TD module, which includes a Transformer attention mechanism and a Dropout layer, enhancing the model's ability to extract contextual information while avoiding overfitting.
[0019] The feature extraction method is optimized by using deformable convolution to replace the traditional convolution method. The model is randomly offset from the traditional sampling point and samples around the traditional sampling point to improve the model's adaptability to target deformation.
[0020] To optimize the loss function, a smoothing function is introduced to make gradient changes smoother.
[0021] The specific steps are as follows: Step 4.1: Optimize the backbone network by constructing the TD module, which includes the Transformer attention mechanism and Dropout layer. This expands the receptive field of the model while avoiding overfitting, thereby improving model performance and generalizability.
[0022] Step 4.2: Convolution method optimization. Deformable convolution method is introduced to extract image features to adapt to changes in image size and improve the model's adaptability to targets with large size changes and deformation.
[0023] Step 4.3: Loss function optimization. A smoothing function is introduced into the loss function to enhance smoothness and avoid sharp fluctuations in gradients.
[0024] This invention proposes an improved Faster-RCNN object detection method to enhance object detection performance in apron scenarios, thereby providing effective support for airport operation management and security monitoring. Attached Figure Description
[0025] Figure 1 This is a schematic diagram of the overall architecture of the detection model of the present invention; Figure 2 This is a schematic diagram of the TD module of the present invention; Figure 3 This is a schematic diagram of the image processing flow of the Transformer of the present invention. Detailed Implementation
[0026] As the preferred embodiment of the present invention, the present invention includes an airport apron target detection method based on Faster-RCNN.
[0027] Refer to the instruction manual Figures 1-3 The aforementioned airport apron target detection method based on Faster-RCNN specifically includes the following steps: Step S1: Data Acquisition. Utilizing airport tarmac surveillance cameras, the tarmac area was continuously filmed 24 / 7 to acquire video footage under different time periods, lighting conditions, and weather conditions, ensuring data diversity and representativeness. In this example, 90 video footage segments were acquired at a frame rate of 30 frames per second, each approximately 2 hours long, with a resolution of 1280×720. Due to the high frame rate of airport video acquisition, directly storing all video frames would incur enormous data storage and computational costs. Therefore, an interval frame extraction method was used to process the video to obtain representative keyframe images. Interval frame extraction involved extracting one frame every 800 frames as image data. To further increase dataset diversity, random frame extraction was also used, randomly selecting 15,000 frames from a video segment as image data. Furthermore, to enhance the richness and generalization ability of the dataset, some publicly available datasets were introduced to supplement it, expanding upon existing airport target detection datasets to form the original dataset, providing a solid data foundation for subsequent models.
[0028] Step S2: Data processing, which involves processing the original dataset formed in step S1, mainly including data cleaning, data augmentation, and target labeling.
[0029] Data cleaning mainly involves using methods such as manual management and automatic quality assessment algorithms to screen the quality of the current original dataset, removing blurry or low-quality images. At the same time, due to the long video shooting time, duplicate frames are inevitable in the image frames extracted in step S1, so it is necessary to remove duplicate data frames and eliminate redundant data.
[0030] Data augmentation primarily enriches image datasets and enhances their diversity through geometric transformations, lighting adjustments, and noise addition. Geometric transformations, such as random rotation, scaling, cropping, and mirroring, enrich images from different perspectives. Lighting adjustments enhance or reduce brightness and contrast to simulate lighting changes under different weather conditions, thus adding images under varying lighting conditions. Noise addition uses Gaussian noise or salt-and-pepper noise to simulate signal interference and compression noise that may occur in surveillance videos, adding noise to the images.
[0031] Data annotation mainly employs a combination of manual and semi-automatic annotation methods to label targets in images, ensuring the accuracy and consistency of the annotations. Manual annotation primarily uses annotation tools such as labelImg, labelme, Labelbox, VGG Image Annotator, RectLabel, and CVAT to annotate image data. The semi-automatic annotation method mainly combines deep learning pre-trained models with manual correction for annotation. The labeled target categories include aircraft, ground support vehicles (refueling trucks, shuttle buses, baggage carts, etc.), and staff (personnel wearing safety vests).
[0032] Step S3: Data Analysis. Based on the dataset processed in Step S2, perform statistical analysis and organize the overall situation and detailed information of the target distribution in the dataset to understand the detailed situation of the dataset and provide reference value for subsequent model building, including target category statistics, target size analysis, and the influence of lighting and weather factors.
[0033] The target category statistics mainly involve calculating the proportion of different target categories, including aircraft, ground vehicles, and staff, which account for 54.3%, 32.6%, and 13.1% respectively. Among them, staff accounts for the largest proportion of targets, followed by ground vehicles, while aircraft accounts for the smallest proportion. In subsequent model training, a target category balancing strategy will be considered to avoid the impact of category imbalance on model performance.
[0034] Target size analysis mainly involves calculating the size distribution of each type of target. Aircraft are large targets, usually occupying most of the image area. Ground support vehicles are medium-sized targets, while staff are small targets. These two types of targets only occupy a small portion of the image area, but are numerous and usually distributed in various regions of the image. In the subsequent model training process, attention mechanisms and multi-scale feature extraction strategies will be introduced to optimize the model and improve the detection capability of targets of different sizes.
[0035] The analysis of the impact of illumination and weather mainly focuses on the influence of illumination and weather factors on image data. Since airport lighting is relatively sufficient compared to other scenes, there is a lack of image data under insufficient illumination conditions. In addition, the dataset also lacks images for diverse meteorological conditions such as rain and fog. Therefore, we consider using GAN to generate some synthetic data under nighttime and different weather conditions to supplement the training samples.
[0036] Step S4: Model building steps, based on and optimized from the Faster-RCNN framework. First, a DT module is built in the backbone network, which includes a Transformer attention mechanism and Dropout layers. Second, deformable convolutions are used in the convolutional neural network to extract features. Finally, a smoothing function is introduced into the loss function, with the structure as follows: Figure 1 As shown.
[0037] A TD module is constructed in the backbone network of Faster R-CNN, with the following structure: Figure 2 As shown, this module consists of a Transformer attention mechanism and a Dropout layer. After the image is processed by the backbone network, it will be processed by the Transformer attention mechanism and the Dropout layer in sequence to output the final result.
[0038] The Transformer image processing workflow is as follows: Figure 3 As shown, after the image is processed by the backbone network, it is divided into a series of fixed-size image patches, and then these image patches are flattened into one-dimensional vectors. This vector is used as input to the Transformer. A learnable positional encoding is added to each flattened vector to preserve spatial information, allowing the Transformer to directly perceive local relationships using the convolutional structure. The positionally encoded vector X is input into the Transformer's encoder, which maps it to a query vector Q, a key vector K, and a value vector V through three independent linear transformations. These transformations are used to calculate the attention score. The query vector Q is then multiplied by all key vectors K and divided by a scaling factor. Then, through Softmax normalization, the attention weight distribution is obtained. The attention weights are used to weight the corresponding value vector V, as shown in equation (1): (1) Then, the Q, K, and V vectors are projected into multiple different subspaces. Each subspace calculates independent attention weights and interacts with information. The subspaces are then fused through linear transformations to form the final multi-head attention output.
[0039] The output of the Transformer serves as the input to the Dropout layer. The Dropout layer randomly masks certain neurons in the neural network, rendering them ineffective during forward propagation. The masking probability of each neuron is independently sampled based on a Bernoulli distribution to determine whether the neuron is temporarily disabled. When a neuron is masked, its output value is set to zero, meaning that the neuron will not affect subsequent layers in the current training iteration.
[0040] Deformable convolution is used to extract features. An offset obtained by dynamic learning by the network is introduced. Each sampling point is expanded or shifted outward based on the original sampling point with the offset as the reference, so as to extract image features in a wider area.
[0041] Finally, a smoothing function is introduced into the GIoU loss function to make the gradient changes smoother, as shown in equation (2): (2) in, It is the base of the natural logarithm. It is a positive constant used to control the degree of smoothness. When When it approaches -1, This will provide a smooth and gradual transition.
Claims
1. A method for airport apron target detection based on Faster-RCNN, characterized in that, It includes four steps: data acquisition, data processing, data analysis, and model building. Step 1: In the data acquisition step, the airport surveillance cameras continuously capture images of the tarmac area, including various targets such as aircraft, staff, and ground support vehicles. Step 2: In the data processing step, the image data collected in Step 1 is processed using methods such as data cleaning, data augmentation, and target annotation. Step 3: In the data analysis step, perform detailed statistical analysis on the dataset processed in Step 2 to optimize the model training strategy; Step 4: In the model building step, based on the data analysis results of Step 3, optimize the target detection framework based on Faster-RCNN to build an airport apron target detection model.
2. The airport apron target detection method based on Faster-RCNN according to claim 1, characterized in that, Step 1 includes the following sub-steps: Step 1.1: Airport surveillance cameras continuously capture images of the tarmac area to obtain data from different time periods, lighting conditions, and weather conditions, ensuring the diversity and representativeness of the data; Step 1.2: For the surveillance video collected in Step 1.1, the video is processed using random frame extraction and interval frame extraction methods to obtain representative keyframe images; Step 1.3: For the keyframe images obtained in Step 1.2, some publicly available datasets are introduced to supplement them, and the datasets of existing airport target detection tasks are combined to expand them, thereby enhancing the richness and generalization ability of the datasets.
3. The airport apron target detection method based on Faster-RCNN according to claim 1, characterized in that, Step 2 includes the following sub-steps: Step 2.1: Data cleaning mainly involves quality screening of the raw image data collected in Step 1, removing blurry or noisy images to ensure the clarity and usability of the dataset; Step 2.2: Data augmentation mainly targets the image data selected in Step 2.1, using ensemble geometric transformations, illumination adjustments, and noise additions to improve data diversity and enhance the model's detection capabilities in complex environments; Step 2.3: Data annotation mainly involves a combination of manual and semi-automatic annotation of the image data from Step 2.
2. Target annotation is performed on the data, including targets such as aircraft, ground support vehicles, and staff, to ensure the accuracy and high quality of the training data.
4. The airport apron target detection method based on Faster-RCNN according to claim 1, characterized in that, Step 3 includes the following sub-steps: Step 3.1: For the dataset processed in Step 2, perform target category distribution analysis and count the number distribution of targets such as aircraft, ground support vehicles, and staff. Step 3.2: For the dataset processed in Step 2, perform target size analysis and statistically analyze the distribution of large, medium, and small targets. Step 3.3: Perform environmental factor analysis on the dataset processed in Step 2 to analyze the impact of different weather conditions on target detection accuracy.
5. The airport apron target detection method based on Faster-RCNN according to claim 1, characterized in that, Step 4 includes the following sub-steps: Step 4.1: Optimize the backbone network and build the TD module, which includes the Transformer attention mechanism and Dropout layer. This expands the receptive field of the model while avoiding overfitting, thereby improving the model's performance and generalizability. Step 4.2: Convolution method optimization. Deformable convolution method is introduced to extract image features to adapt to changes in image size and improve the model's adaptability to targets with large size changes and deformations. Step 4.3: Loss function optimization. A smoothing function is introduced into the loss function to enhance smoothness and avoid sharp fluctuations in gradients.