An airport target detection method based on combination of global and local features

Abstract

The application discloses an airport target detection method based on combination of global and local features, which comprises the following steps: collecting data; performing data labeling on the collected data; performing data enhancement to form an airport remote sensing image dataset; constructing a local feature extraction network, including detection models of three types of targets of a runway, a parking apron and a control tower; training the detection models of the three types of targets of the runway, the parking apron and the control tower; constructing a global feature determination model, which is used for determining whether all the runway, parking apron and control tower targets are airport targets; training the global feature determination network; and performing airport target detection on images by using the trained local feature extraction model and the global feature determination model. The method firstly constructs a local feature extraction network to extract important components of the airport, and then constructs a global feature determination network to integrally determine the features between different components, so that the accuracy of airport target detection is improved.

Description

Technical Field

[0001] This invention belongs to the field of target detection, and specifically relates to an airport target detection method based on a combination of global and local features. Background Technology

[0002] While deep learning techniques for object detection and recognition are becoming increasingly mature, such as in face recognition, pedestrian detection, and vehicle detection, less attention has been paid to the detection of large topological targets like airports and ports. Existing airport detection methods mainly fall into two categories: one detects the airport as a whole, and the other detects only the most characteristic runway portion. When detecting the airport as a whole, the model's robustness and generalization ability are weak due to the large area covered by airports and the varying shapes and runway / apron combinations of airports in different cities. Accuracy is high for airports seen during training, but false negatives are common for unseen airports. Conversely, when detecting the most characteristic runway portion, the wide coverage of satellite remote sensing imagery means the runway's features are primarily a straight line, making it easy to confuse with other straight targets like roads or bridges, resulting in a high false alarm rate. Summary of the Invention

[0003] Purpose of the invention: The technical problem to be solved by the present invention is to provide an airport target detection method based on the combination of global and local features, which addresses the shortcomings of the existing technology.

[0004] To address the aforementioned technical problems, this invention discloses an airport detection method that combines global and local features, comprising the following steps:

[0005] Step 1: Collect data by collecting satellite remote sensing images of airports from different geographical locations, types, and historical periods;

[0006] Step 2: Label the collected data;

[0007] Step 3: Data augmentation. Perform data augmentation operations on the labeled remote sensing images to form an airport remote sensing image dataset.

[0008] Step 4: Construct a local feature extraction model. The local feature extraction network is used to detect all runways, aprons, and control tower targets, including a detection model for three types of targets: runways, aprons, and control towers.

[0009] Step 5: Train the local feature extraction model. Using the airport remote sensing image dataset, train the detection model for three types of targets: runway, apron, and control tower.

[0010] Step 6: Construct a global feature determination model, which is used to determine whether a target is an airport based on all runways, aprons and control towers.

[0011] Step 7: Train the global feature determination model;

[0012] Step 8: Model testing. Use the trained local feature extraction model and global feature determination model to perform airport target detection on the image.

[0013] Furthermore, the data annotation in step 2 uses a four-point slanted rectangular box for annotation, and the annotation categories include three types of data: airport runway, airport apron, and airport control tower.

[0014] Airport runways, aprons, and control towers are often irregular targets. If a rectangular frame is used, it is easy to include a lot of background, which can lead to false detections. Using an slanted rectangular frame can better remove the background and increase the accuracy of detection.

[0015] Furthermore, step 3 includes the following steps:

[0016] Step 3-1: Rotate the labeled remote sensing image multiple times, by 15 degrees, 30 degrees, 45 degrees, 60 degrees, 75 degrees and 90 degrees respectively. While rotating the image, calculate the coordinates in the corresponding label file to generate the label file corresponding to the rotated image. Adjust the label to fit the target. No fine-tuning is needed for the 90-degree rotation.

[0017] Step 3-2: Perform random scaling operations on the remote sensing images annotated in Step 2 and the remote sensing images generated by rotation in Step 3-1. The scaling ratio ranges from 70% to 130%. At the same time as each scaling operation, the coordinates of the corresponding annotation files are calculated synchronously to generate the corresponding annotation files. The annotation files generated after the scaling operation do not require fine-tuning.

[0018] Step 3-3: Randomly adjust the brightness of the remote sensing images annotated in Step 2, the remote sensing images rotated in Step 3-1, and the remote sensing images scaled in Step 3-2 to generate images with different brightness and contrast. At the same time, generate corresponding annotation files. When generating new annotation files, the coordinates of the target do not need to be recalculated and adjusted.

[0019] The purpose of data augmentation through scaling is primarily to reduce the size of targets in images at different resolutions, thereby reducing false positives or false negatives. The purpose of data augmentation through rotation is primarily to reduce false negatives or false negatives in images taken from different angles, due to the varying orientation of targets.

[0020] Furthermore, in step 4, the detection models for the three target types—runway, apron, and control tower—are built using the Oriented-RCNN (Region Convolutional Neural Network for Rotating Target Detection) tilt box detection model. The Oriented-RCNN detection model comprises two modules: Qriented-RPN (Region Proposal Network) and RRoI-Aligh. The construction steps for the three detection models are as follows:

[0021] Step 4-1: Construct a runway detection model using the Oriented-RCNN model. The initial aspect ratios of the candidate boxes are 1:5, 5:1, 1:7, and 7:1, for a total of 4 types. The initial sizes of the candidate boxes are 1000*200, 3000*600, and 2000*500.

[0022] Step 4-2: Construct a tarmac detection model using the Oriented-RCNN model. The initial aspect ratio of the candidate boxes is 1:2, 1:1, and 2:1, for a total of 3 types. The initial size of the candidate boxes is 64*64, 128*128, 256*256, and 512*512, for a total of 4 types.

[0023] Step 4-3: Construct a command tower detection model using the Oriented-RCNN model. The initial aspect ratio of the candidate boxes is 1:2, 1:1, and 2:1, for a total of 3 types. The initial size of the candidate boxes is 64*64, 128*128, and 256*256, for a total of 3 types.

[0024] The scale design of the candidate boxes is based on analyzing the dimensions and sizes of targets in the training dataset. In this application, the most frequent aspect ratios and sizes of three target types—runways, aprons, and control towers—are statistically analyzed and used as parameters for the candidate boxes. The Oriented-RCNN model currently boasts the best candidate box detection performance.

[0025] Furthermore, in step 5, when training the detection models for the three types of targets—runway, apron, and control tower—the training epoch is 30, the batch is 1, the learning rate is 0.0001 to 0.001, and the loss function is the cross-entropy function.

[0026] Furthermore, the global feature determination model described in step 6 sequentially includes a feature extraction network, a feature fusion network, and a classification network.

[0027] Furthermore, the feature extraction network described in step 6 adopts the EfficientNet network. For the three types of targets, namely the runway, the apron, and the control tower, the EfficientNet network is used for feature extraction. The output of the last convolutional layer is flattened to generate feature vectors.

[0028] Feature extraction is a necessary step in classification, and the EfficientNet network is the optimal network selected after comparing the performance of several basic networks.

[0029] The feature fusion network is used to add and average the feature values ​​of similar targets to form the category features of that target class. For example, if there are two runways, the features of the two runways are averaged to form the category features of the runways. If there are three aprons, the features of the three aprons are averaged to form the category features of the aprons. The category features of each category are then concatenated to form the overall feature of the target.

[0030] An airport may have multiple aprons or multiple runways. By first averaging the features, we can combine the features of multiple individuals of each type of target to avoid feature redundancy. Then, by using feature concatenation, we can combine the features of all categories to reduce feature loss and achieve the best classification results.

[0031] The classification network uses a ResNet152 network for classification. The overall features of the target are input into the ResNet152 network, and then the target is predicted to be an airport after passing through a fully connected layer. The loss function is the cross-entropy function.

[0032] The reason for prioritizing feature extraction, fusion, and classification is that detecting an airport as a single target lacks common features and is prone to overfitting. By extracting features to detect local targets, and then fusing them, some common local features of the airport are extracted. For example, an airport must have aprons and runways. Further fusion and classification allow for a more effective determination of whether these common features constitute the conditions for an airport. This approach can significantly improve detection accuracy.

[0033] Further, in step 7, the global feature determination network is trained. The EfficientNet network is trained with Epoch = 60, Batch = 10, learning rate = 0.005–0.01, and cross-entropy loss function. The ResNet152 network is trained with Epoch = 60, Batch = 1, learning rate = 0.0001–0.005, and cross-entropy loss function.

[0034] Furthermore, step 8 includes the following steps:

[0035] Step 8-1: Detect the image to be detected using the runway detection model and output all runway targets;

[0036] Step 8-2: Detect the image to be detected using the helipad detection model and output all helipad targets;

[0037] Step 8-3: Detect the image to be detected using the command tower detection model and output all command tower targets;

[0038] Step 8-4: Classify all runway targets, apron targets, and control tower targets using a global feature determination network to determine whether they are airport targets.

[0039] Furthermore, the data collection in step 1 includes finding and capturing remote sensing images of airports by entering the airport's latitude and longitude on Google Earth. This includes airports of different geographical locations and types (including military, civilian, and dual-use airports). Images of the same airport at different time periods are captured to cover scenes from different periods and seasons, thus increasing data diversity.

[0040] Beneficial effects:

[0041] This paper addresses the problem of insufficient common features of airports as a whole, and the lack of obvious features of individual components. It proposes an airport target detection method based on a combination of global and local features. Considering the similarity of components among airports of different types and locations, a local feature extraction network is first constructed to detect components such as runways, aprons, and control towers. Then, a global feature judgment network is constructed, combining the feature similarity of multiple components in the training set to determine whether an object is an airport. This method avoids the overfitting problem caused by detecting the airport as a whole, and also avoids the high false alarm rate caused by detecting only runways. It effectively improves the accuracy of airport detection.

[0042] Compared with existing technologies, this invention (1) has a stronger generalization ability, can process remote sensing images of multiple resolutions, and can also accurately detect airports that have not appeared in the training set. (2) The model algorithm has a low false alarm rate. By using a global feature discrimination network to filter out similar targets, the detection accuracy of the model is improved. Attached Figure Description

[0043] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0044] Figure 1 This is a test flowchart of an airport target detection method based on a combination of global and local features, provided for an embodiment of this application.

[0045] Figure 2 This document presents a flowchart illustrating the model training process for an airport target detection method based on a combination of global and local features, as provided in an embodiment of this application.

[0046] Figure 3This is a schematic diagram illustrating the workflow of the global feature determination model in an airport target detection method based on a combination of global and local features, provided in an embodiment of this application. Detailed Implementation

[0047] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0048] While airports share similar basic components, including runways, aprons, and navigation towers, the layout of these components varies from city to city. Treating an airport as a whole lacks common features, while the features of individual components are not readily apparent. To address this issue, this invention provides an airport target detection method based on a combination of global and local features. First, a dataset for airport detection is constructed. For example, we select data from 100 airports nationwide, using data from 70 airports as the training set and data from 30 airports as the test set. The specific airport detection steps are as follows:

[0049] Step 1: Data Collection. Collect latitude and longitude data for 100 airports. Use Google Earth to collect data on each airport at different times and under different weather conditions using latitude and longitude. Collect about 10 images for each airport, with a resolution of 0.3m-0.8m and the size of each image is between 7000*7000 and 10000*10000.

[0050] Step 2: Data Labeling. Label the data for each airport's runways, aprons, and control towers using slanted rectangles.

[0051] Step 3: Data Augmentation. After labeling, data augmentation operations such as brightness adjustment, rotation, and scaling are performed on the data to create an airport remote sensing image dataset. This mainly includes:

[0052] Step 3-1: Rotate the remote sensing image multiple times, by 15 degrees, 30 degrees, 45 degrees, 60 degrees, 75 degrees and 90 degrees respectively. While rotating the image, calculate the coordinates in the corresponding annotation file to generate the annotation file corresponding to the rotated image. Adjust the annotation to fit the target exactly. No fine-tuning is needed for the 90-degree rotation.

[0053] Step 3-2: Perform random scaling on the remote sensing image (including those generated by rotation), with a scaling ratio ranging from 70% to 130%. Simultaneously calculate the coordinates of the corresponding annotation file each time the image is scaled, and generate the corresponding annotation file. The annotation file generated after the scaling operation does not require fine-tuning.

[0054] Step 3-3: Perform random brightness adjustments on the remote sensing images (including those generated by rotation and scaling) to generate images with different brightness and contrast, and generate corresponding annotation files at the same time. When generating new annotation files, the coordinates of the target do not need to be recalculated and adjusted.

[0055] Step 4: Construct a local feature extraction network. This network is used to detect all runways, aprons, and control tower targets, including a detection network for these three target types. The main construction steps are as follows:

[0056] Step 4-1: Construct a runway detection model using the Oriented-RCNN model. The initial aspect ratios of the candidate boxes are 1:5, 5:1, 1:7, and 7:1, for a total of 4 types. The initial sizes of the candidate boxes are 1000*200, 3000*600, and 2000*500.

[0057] Step 4-2: Construct a tarmac detection model using the Oriented-RCNN model. The initial aspect ratio of the candidate boxes is 1:2, 1:1, and 2:1, for a total of 3 types. The initial size of the candidate boxes is 64*64, 128*128, 256*256, and 512*512, for a total of 4 types.

[0058] Step 4-3: Construct a command tower detection model using the Oriented-RCNN model. The initial aspect ratio of the candidate boxes is 1:2, 1:1, and 2:1, for a total of 3 types. The initial size of the candidate boxes is 64*64, 128*128, and 256*256, for a total of 3 types.

[0059] Step 5: Train the detection networks for three target types: runway, apron, and control tower. Using the labeled data obtained in Step 3, train the runway detection model using the labeled runway data. The training iterations are 100,000, with an initial learning rate of 0.001. After 60,000 iterations, the learning rate is set to 0.0001. The cross-entropy function is used as the loss function. Train the apron detection model using the labeled apron data, with the same number of iterations and initial learning rate as the runway detection model. Train the control tower detection model using the labeled control tower data, with the same number of iterations and initial learning rate as the control tower detection model.

[0060] Step 6: Construct a global feature determination model. This model is used to determine whether a target is an airport based on all runways, aprons, and control towers. It includes a feature extraction network, a feature fusion network, and a classification network. For example... Figure 3 As shown, it mainly includes the following steps:

[0061] Step 6-1: Feature Extraction: For the three types of targets detected by the local feature extraction network, namely the runway, the apron and the control tower, the EfficientNet network is used for feature extraction. The output of the last convolutional layer is flattened to generate feature vectors.

[0062] Step 6-2: Generate fused features from runway features, apron features, and control tower features through dot product. Specifically, this includes: adding and averaging the feature values ​​of targets of the same type to form the category features of that type of target; and splicing the category features of each category to form the overall feature of the target.

[0063] Step 6-3: The classification network uses the ResNet152 network for classification. The overall features of the target are input into the ResNet152 network, and then the fully connected layer is used to predict whether it is an airport. The loss function is the cross-entropy function.

[0064] Step 7: Train the global feature determination network. The EfficientNet network is trained with Epoch = 60, Batch = 10, learning rate = 0.005–0.01, and cross-entropy loss function. The ResNet152 network is trained with Epoch = 60, Batch = 1, learning rate = 0.0001–0.005, and cross-entropy loss function.

[0065] A flowchart illustrating the training process for the local feature extraction network and the global feature determination network is shown below. Figure 2 As shown.

[0066] Step 8: Test the model on the airport test dataset, such as... Figure 1 As shown, it mainly includes the following steps:

[0067] Step 8-1: Detect the image to be detected using the runway detection model, and output all runway targets.

[0068] Step 8-2: Detect the image to be inspected using the helipad detection model, and output all helipad targets.

[0069] Step 8-3: Detect the image to be detected using the command tower detection model, and output all command tower targets.

[0070] Step 8-4: All runway targets, apron targets, and control tower targets are processed through the trained EfficientNet feature extraction network. After all features are fused, they are classified and determined to be airport targets by the ResNet152 network.

[0071] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding an airport target detection method based on a combination of global and local features, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0072] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MUU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0073] This invention provides an airport target detection method based on a combination of global and local features. Many methods and approaches exist for implementing this technical solution; the above description is merely a specific embodiment of this invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. An airport target detection method based on a combination of global and local features, characterized in that, Includes the following steps: Step 1: Collect data by collecting satellite remote sensing images of airports from different geographical locations, types, and historical periods; Step 2: Label the collected data; Step 3: Data augmentation. Perform data augmentation operations on the labeled remote sensing images to form an airport remote sensing image dataset. Step 4: Construct a local feature extraction model, which is used to detect all runways, aprons, and control tower targets, including detection models for three types of targets: runways, aprons, and control towers; Step 5: Train the local feature extraction model. Using the airport remote sensing image dataset, train the detection model for three types of targets: runway, apron, and control tower. Step 6: Construct a global feature determination model, which is used to determine whether a target is an airport based on all runways, aprons and control towers. Step 7: Train the global feature determination network; Step 8: Model testing, using the trained local feature extraction model and global feature determination model to perform airport target detection on the image; The global feature determination model in step 6 includes a feature extraction network, a feature fusion network, and a classification network. The feature extraction network uses the EfficientNet network. For the three types of targets—runway, apron, and control tower—obtained by the local feature extraction model, the EfficientNet network is used for feature extraction. The output of the last convolutional layer is flattened to generate a feature vector. The feature fusion network is used to add and average the feature values ​​of targets of the same type to form the category features of that type of target; and to concatenate the category features of each category to form the overall features of the target. The classification network uses a ResNet152 network for classification. The overall features of the target are input into the ResNet152 network, and then the target is predicted to be an airport after passing through a fully connected layer. The loss function is the cross-entropy function.

2. The airport target detection method based on a combination of global and local features according to claim 1, characterized in that, The data annotation in step 2 uses a four-point slanted rectangular box for annotation, and the annotation categories include three types of data: airport runway, airport apron, and airport control tower.

3. The airport target detection method based on a combination of global and local features according to claim 2, characterized in that, Step 3 includes the following steps: Step 3-1: Rotate the labeled remote sensing image by 15 degrees, 30 degrees, 45 degrees, 60 degrees, 75 degrees and 90 degrees respectively. While rotating the image, calculate the coordinates in the corresponding label file to generate the label file corresponding to the rotated image. Adjust the label to fit the target. No fine-tuning is required for the 90-degree image. Step 3-2: Perform random scaling operations on the remote sensing images annotated in Step 2 and the remote sensing images generated by rotation in Step 3-1. The scaling ratio ranges from 70% to 130%. At the same time as each scaling operation, the coordinates of the corresponding annotation files are calculated synchronously to generate the corresponding annotation files. The annotation files generated by coordinate calculation after scaling operations do not require fine-tuning. Step 3-3: Randomly adjust the brightness of the remote sensing images annotated in Step 2, the remote sensing images rotated in Step 3-1, and the remote sensing images scaled in Step 3-2 to generate images with different brightness and contrast. At the same time, generate corresponding annotation files. When generating new annotation files, the coordinates of the target do not need to be recalculated and adjusted.

4. The airport target detection method based on a combination of global and local features according to claim 3, characterized in that, In step 4, the detection models for the three target types—runway, apron, and control tower—are built using the Oriented-RCNN tilted box detection model. The construction steps for the three detection models are as follows: Step 4-1: Construct a runway detection model using the Oriented-RCNN model. The initial aspect ratios of the candidate boxes are 1:5, 5:1, 1:7, and 7:1, for a total of 4 types. The initial sizes of the candidate boxes are 1000*200, 3000*600, and 2000*500. Step 4-2: Construct a tarmac detection model using the Oriented-RCNN model. The initial aspect ratios of the candidate boxes are 1:2, 1:1, and 2:1, for a total of 3 types. The initial sizes of the candidate boxes are 64*64, 128*128, 256*256, and 512*512, for a total of 4 types. Step 4-3: Construct a command tower detection model using the Oriented-RCNN model. The initial aspect ratios of the candidate boxes are 1:2, 1:1, and 2:1, for a total of 3 types. The initial sizes of the candidate boxes are 64*64, 128*128, and 256*256, for a total of 3 types.

5. The airport target detection method based on a combination of global and local features according to claim 4, characterized in that, In step 5, when training the detection models for the three types of targets—runway, apron, and control tower—the training epoch is 30, the batch is 1, the learning rate is 0.0001 to 0.001, and the loss function is the cross-entropy function.

6. The airport target detection method based on a combination of global and local features according to claim 5, characterized in that, In step 7, the global feature determination network is trained. The training epoch of the EfficientNet network is 60, the batch is 10, the learning rate is 0.005 to 0.01, and the loss function is the cross-entropy function; the training epoch of the ResNet152 network is 60, the batch is 1, the learning rate is 0.0001 to 0.005, and the loss function is the cross-entropy function.

7. The airport target detection method based on a combination of global and local features according to claim 6, characterized in that, Step 8 includes the following steps: Step 8-1: Detect the image to be detected using the runway detection model and output all runway targets; Step 8-2: Detect the image to be detected using the helipad detection model and output all helipad targets; Step 8-3: Detect the image to be detected using the command tower detection model and output all command tower targets; Step 8-4: Classify all runway targets, apron targets, and control tower targets using a global feature determination network to determine whether they are airport targets.

8. The airport target detection method based on a combination of global and local features according to claim 7, characterized in that, The data collection in step 1 includes finding and capturing remote sensing images of airports by entering the airport's latitude and longitude on Google Earth. These images include airports of different geographical locations and types. Images of the same airport are captured at different time periods, covering scenes from different seasons and periods. The different types of airports include military, civilian, and dual-use airports.

Images