An airport flight area target real-time detection method based on multi-scale feature decoupling

Abstract

The application discloses an airport flight area target real-time detection method based on multi-scale feature decoupling, and the steps include: 1, acquiring the airport flight area monitoring video, and constructing the airport flight area target detection data set; 2, constructing a multi-scale feature fusion module to realize the detection of targets of different scales; 3, decoupling the classification and positioning tasks in target detection, and constructing a learning network based on feature decoupling; 4, adding the learning network based on feature decoupling and the multi-scale feature fusion module to a YOLOv5 target detection network, and combining a loss function to train and optimize the target detection model. The application aggregates shallow detail information and deep semantic information by using the multi-scale feature fusion module, enhances the detection capability of targets of different scales, and decouples the classification and positioning tasks by using the feature decoupling learning network, learns the rotation-invariant and rotation-covariant features respectively, and improves the accuracy of target detection.

Description

TECHNICAL FIELD

[0001] The application relates to a target real-time detection method, in particular to an airport flight area target real-time detection method based on multi-scale feature decoupling. BACKGROUND

[0002] With the development of the economic society, the civil aviation transportation industry is booming, and the airport flight area undertakes important tasks such as aircraft berthing and passenger pickup. The target real-time detection of the airport flight area is the basis for ensuring the safe and efficient operation of aircraft, vehicles and staff on the airport scene, and the intelligent monitoring of the airport flight area plays an important role in the safe operation of the airport.

[0003] At present, the commonly used flight area monitoring system of the airport includes multiple methods such as staff visual observation, monitoring radar and video monitoring. Among them, the monitoring radar can only detect and locate the target, and cannot identify the type of the detected target. Moreover, due to the shielding of aircrafts, terminal bridges and terminal buildings, there is a certain blind area in the detection range. In recent years, with the development of monitoring video and computer vision, image-based target detection technology is widely used in various aspects of human life such as transportation and sports events, and is also applied to airport monitoring to identify and locate typical targets in the airport and realize real-time perception of the operation of various targets in the airport.

[0004] However, due to the complex airport environment, weather changes, different sizes of various targets and other factors, the accurate detection of the airport flight area target faces great challenges. First, the airport environment is complex, most of the cameras are installed on the terminal building, which causes the ground target to occupy fewer pixels in the image, affecting the detection effect. Second, the airport flight area target detection needs all-weather real-time detection. Under different weather conditions such as rain, fog, snow, day and night, the images collected are quite different, and the generalization performance of the target detection model is required to be high. Moreover, the aircraft, towing vehicle, luggage vehicle, water vehicle, dining vehicle and staff in the airport flight area have different sizes, and the differences between the classes are large, so the detection of small targets is difficult. SUMMARY

[0005] The technical problem to be solved by the application is to provide an airport flight area target real-time detection method based on multi-scale feature decoupling to solve the problems of the prior art.

[0006] In order to solve the above technical problems, the application discloses an airport flight area target real-time detection method based on multi-scale feature decoupling, which comprises the following steps:

[0007] Step 1, constructing an airport flight area target detection data set;

[0008] Step 2, constructing a multi-scale feature fusion module: a multi-scale feature fusion module based on channel attention mechanism and spatial attention mechanism is designed;

[0009] Step 3, constructing a feature decoupling-based learning network: for decoupling classification task and regression task, the class and position of the target are obtained through different network branches;

[0010] Step 4, constructing a target detection model, that is, adding the multi-scale feature fusion module and the feature decoupling-based learning network to the YOLOv5 target detection architecture to obtain the target detection model;

[0011] Step 5, target detection model training, verification and application: based on the airport flight area target detection data set constructed in step 1, the target detection model constructed in step 4 is trained and verified, and the multi-scale feature decoupling-based airport flight area target real-time detection is completed.

[0012] Beneficial effects:

[0013] (1) The multi-scale feature fusion module is used to extract the feature information of the image, which is beneficial to improve the target detection ability of different scales.

[0014] (2) The multi-scale feature fusion module constructed in the application contains channel attention and spatial attention modules, which can highlight the feature information of key positions, and has small complexity, only a small amount of parameters are added, and can be efficiently embedded into the target detection model.

[0015] (3) The feature decoupling network is used to improve the sensitivity of the classification and positioning branches to feature translation and rotation, the asymmetric convolution and the hollow convolution are used to improve the robustness of the classification branch to target rotation change, and the deformable convolution is used to improve the accuracy of target frame positioning. BRIEF DESCRIPTION OF DRAWINGS

[0016] The above and / or other aspects of the present application will become apparent from the following detailed description of the application taken in conjunction with the accompanying drawings.

[0017] Figure 1 is a schematic diagram of the overall process of the application.

[0018] Figure 2 is a schematic diagram of the multi-scale feature fusion module.

[0019] Figure 3 is a schematic diagram of the feature decoupling network in the target detection prediction network.

[0020] Figure 4 is a schematic diagram of the result example of the video data and the feature decoupling airport flight area target real-time detection model obtained in the implementation process of the application. Detailed Implementation

[0021] This invention proposes a real-time target detection method for airport flight areas based on multi-scale feature decoupling, such as... Figure 1 As shown, the specific process is as follows:

[0022] Step 1: Construct an airport flight area target detection dataset: Deploy surveillance cameras on the side of the aircraft to acquire raw airport flight area surveillance videos. Extract frames from the acquired videos to obtain single-frame images. Manually select typical images containing the targets to be detected. Use target detection annotation software to label each target in the image and generate an annotation text. The text file contains information such as the category of the target to be detected and the size and position of the detection box. After all images are annotated, an airport flight area target detection dataset is formed. Then, it is divided into training set, validation set and test set in a ratio of 8:1:1.

[0023] Step 2: Construct a multi-scale feature fusion module, such as... Figure 2 As shown, the steps are as follows:

[0024] Step 2.1: Select Cross Stage Partial Network (CSPNet) as the basic feature extraction network (Reference: CSPNet: A New Backbone that can Enhance Learning Capability of CNN[C] / / 2020IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW).IEEE,2020.). CSPNet consists of local dense blocks and local transition layers, which can reduce the computational cost of the network and improve the network running speed. After the input image is processed by CSPNet, feature maps C3, C4 and C5 of different sizes are obtained, with feature map sizes of 80×80, 40×40 and 20×20, respectively.

[0025] Step 2.2: Feature map C5 is upsampled to obtain feature map CS5, with a size of 40×40;

[0026] Step 2.3: Feature map C4 is processed by the spatial attention module, that is, C4 is processed by max pooling and mean pooling respectively, and the resulting feature maps are concatenated; the concatenated feature maps are processed by 2D convolution and sigmoid activation function to obtain the spatial weights of the feature maps; then multiplied with feature map C4 to obtain feature map CSA4 containing spatial weight information, with a size of 40×40;

[0027] Step 2.4: The feature map C4 passes through the channel attention module, that is, C4 respectively passes through the channel-level global maximum pooling and mean pooling, and the obtained feature maps are aggregated together after 1D convolution; the aggregated feature map is multiplied by the feature map C4 to obtain a feature map CCA4 containing channel weight information, with a size of 40x40;

[0028] Step 2.5: The feature map CS5 obtained in step 2.2, the feature map CSA4 obtained in step 2.3 and the feature map CCA4 obtained in step 2.4 are added to obtain a fused feature map F1, as shown in Figure 2 , with a size of 40x40;

[0029] Step 2.6: The feature map C4 is processed by upsampling to obtain a feature map CS4, with a size of 80x80;

[0030] Step 2.7: The feature map C3 passes through the spatial attention module, that is, C3 respectively passes through the maximum value pooling and mean value pooling operations, and the obtained feature maps are spliced; the spliced feature map is processed by 2D convolution and Sigmoid activation function to obtain the spatial weight of the feature map; then multiplied by the feature map C3 to obtain a feature map CSA3 containing spatial weight information, with a size of 80x80;

[0031] Step 2.8: The feature map C3 passes through the channel attention module, that is, C3 respectively passes through the channel-level global maximum pooling and mean pooling, and the obtained feature maps are aggregated together after 1D convolution; the aggregated feature map is multiplied by the feature map C3 to obtain a feature map CCA3 containing channel weight information, with a size of 80x80;

[0032] Step 2.9: The feature map CS4 obtained in step 2.6, the feature map CSA3 obtained in step 2.7 and the feature map CCA3 obtained in step 2.8 are added to obtain a fused feature map F2, with a size of 80x80;

[0033] Step 2.10: The feature map C5 is processed by 2D convolution to obtain a feature map F3, with a size of 20x20;

[0034] Step three, construct a learning network based on feature decoupling, as shown in Figure 3 , in the prediction module of the target detection model, decouple the classification task and the regression task, obtain the category and position of the target through different network branches, the steps are as follows:

[0035] Step 3.1: The feature maps F1, F2 and F3 obtained in step 2 are input into the target detection head, and then a classification branch and a regression branch are added;

[0036] Step 3.2: Add a convolutional layer behind the feature map F3 to obtain the feature map P3;

[0037] Step 3.3: In the classification network branch, add an asymmetric convolution and a dilated convolution module behind the feature map P3 respectively; then add the feature maps after the asymmetric convolution and the dilated convolution; and then output the class information of the target through the convolutional layer;

[0038] Step 3.4: In the regression network branch, add a deformable convolution behind the feature map P3; and then output the position information of the target through the convolutional layer;

[0039] Step 3.5: As in steps 3.2, 3.3, and 3.4, add the classification network branch and the regression network branch behind the feature maps F2 and F1 respectively to obtain the class information and the position information of the target;

[0040] Step Four, add the multi-scale feature fusion module and the learning network based on feature decoupling to the YOLOv5 target detection architecture, and the steps are as follows:

[0041] Step 4.1: Select the target detection architecture of YOLOv5, and add the multi-scale feature fusion module to the basic feature extraction network;

[0042] Step 4.2: In the multi-scale prediction branch, add the learning network based on feature decoupling;

[0043] Step 4.3: Set the parameters of the target detection model, apply the k-means clustering algorithm to the training samples of the airport flight area target detection dataset to obtain 9 kinds of different size prior boxes; set the initial learning rate of the network to 0.01; use the stochastic gradient descent method (SGD) as the optimizer; set the maximum number of iterations to 200; set the size of the batch training to 16; and set the model saving type to pth format;

[0044] Step 4.4: Train the target detection model: load the pre-trained model and adjust the model parameters using the airport flight area target detection dataset constructed in step one;

[0045] Step 4.5: Optimize the model: according to the results of the validation set, adjust the training parameters and optimize the target detection model.

[0046] Step 4.6: Deploy the model: apply the optimized target detection model to the monitoring video to obtain the class and position information of the target in the video.

[0047] Step five, target detection model training, verification and application: according to the parameters set in step 4.3, the target detection model is trained based on the constructed airport flight area target detection data set, and the test and verification are carried out on the test set image. The verified model is applied to the airport flight area monitoring video, the position of the target category in the video to be detected is recognized, and the corresponding target category information and target frame information are output. The detection result is shown in the following figure. Figure 4

[0048] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and the computer program can run the invention content of a kind of airport flight area target real-time detection method based on multi-scale feature decoupling and part or all steps in each embodiment provided by the present application when executed by a data processing unit. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0049] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present application can be realized by means of a computer program and its corresponding general hardware platform. Based on such understanding, the technical solutions in the embodiments of the present application can be embodied in the form of a computer program, i.e. a software product, which can be stored in a storage medium, including a number of instructions for causing a device (which can be a personal computer, a server, a single-chip microcomputer, a MUU or a network device, etc.) containing a data processing unit to execute the method described in each embodiment or some parts of the embodiments of the present application.

[0050] The present application provides a kind of airport flight area target real-time detection method based on the idea and method of multi-scale feature decoupling, there are many methods and ways to realize the technical solutions, the above described are only preferred embodiments of the present application, it should be pointed out, for the ordinary skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements should be regarded as the protection scope of the present application. The components not explicitly described in the embodiments can be realized by existing technology.​

Claims

1. A method for real-time detection of targets in an airport flight zone based on multi-scale feature decoupling, characterized in that, Comprising the following steps: Step 1, constructing an airport flight area target detection data set; Step 2, constructing a multi-scale feature fusion module: designing a multi-scale feature fusion module based on channel attention mechanism and spatial attention mechanism; Step 3, constructing a feature decoupling-based learning network: for decoupling classification task and regression task, obtaining the category and position of the target through different network branches; Step 4, constructing a target detection model, that is, adding the multi-scale feature fusion module and the feature decoupling-based learning network to the YOLOv5 target detection architecture to obtain the target detection model; Step 5, target detection model training, verification and application: based on the airport flight area target detection data set constructed in step 1, the target detection model constructed in step 4 is trained and verified, and the real-time detection of the airport flight area target based on multi-scale feature decoupling is completed; In step 2, the multi-scale feature fusion module is constructed in the following way: Step 2.1, select CSPNet as the basic feature extraction network, input the image to be detected to obtain feature maps C3, C4 and C5 of different sizes, the size of the feature map is 80×80, 40×40 and 20×20 respectively; Step 2.2, the feature map C5 is processed by upsampling to obtain the feature map CS5, the size is 40×40; Step 2.3, the feature map C4 passes through the spatial attention module, that is, C4 respectively passes through the maximum value pooling and mean value pooling operation, and the obtained feature maps are spliced; the spliced feature map is processed by 2D convolution and Sigmoid activation function to obtain the spatial weight of the feature map C4; then the spatial weight is multiplied by the feature map C4 to obtain the feature map CSA4 containing the spatial weight information, the size is 40×40; Step 2.4, the feature map C4 passes through the channel attention module, that is, C4 respectively passes through the channel-level global maximum value pooling and mean value pooling, and the obtained feature maps are aggregated together after 1D convolution; the aggregated feature map is multiplied by the feature map C4 to obtain the feature map CCA4 containing the channel weight information, the size is 40×40; Step 2.5, add the feature map CS5 obtained in step 2.2, the feature map CSA4 obtained in step 2.3 and the feature map CCA4 obtained in step 2.4 to obtain the fused feature map F1; Step 2.6, the feature map C4 is processed by upsampling to obtain the feature map CS4, the size is 80×80; Step 2.7, the feature map C3 passes through the spatial attention module, that is, C3 respectively passes through the maximum value pooling and mean value pooling operation, and the obtained feature maps are spliced; the spliced feature map is processed by 2D convolution and Sigmoid activation function to obtain the spatial weight of the feature map C3; then the spatial weight is multiplied by the feature map C3 to obtain the feature map CSA3 containing the spatial weight information, the size is 80×80; Step 2.8, the feature map C3 passes through the channel attention module, that is, the feature map C3 passes through the channel-level global maximum pooling and mean pooling respectively, and the obtained feature maps are aggregated together after 1D convolution; the aggregated feature map is multiplied by the feature map C3 to obtain a feature map CCA3 containing channel weight information, and the size is 80*80; Step 2.9: add the feature map CS4 obtained in step 2.6, the feature map CSA3 obtained in step 2.7 and the feature map CCA3 obtained in step 2.8 to obtain a fused feature map F2; Step 2.10: the feature map C5 is processed by 2D convolution to obtain a feature map F3.

2. The method of claim 1, wherein, The method for constructing the airport flight area target detection data set in step 1 comprises: An original airport flight area monitoring video is obtained by arranging a monitoring camera on the side of the aircraft, and a single frame image is obtained by frame extraction processing on the obtained video. A typical image containing a target to be detected in the single frame image is manually selected, and each target to be detected in the typical image is labeled. A label text file is generated, which contains the category, size and position information of the target to be detected. After all the typical images are labeled, an airport flight area target detection data set is formed.

3. The method of claim 2, wherein the method comprises: The airport flight area target detection data set in step 1 is divided into a training set, a verification set and a test set according to a ratio of 8:1:

1.

4. The method of claim 3, wherein the method comprises: The learning network based on feature decoupling in step 3 comprises a classification network branch and a regression network branch, and the specific method is as follows: Step 3.1, the feature maps F1, F2 and F3 obtained in step 2 are input into the target detection head of the learning network, and then the classification network branch and the regression network branch are added respectively; Step 3.2, a convolution layer is added after the output of the target detection head corresponding to the feature map F3 to obtain a feature map P3; Step 3.3, in the classification network branch, an asymmetric convolution and a hole convolution module are added after the feature map P3 respectively; then the feature maps after the asymmetric convolution and the hole convolution are added; and the category information of the target is output after the convolution layer; Step 3.4, in the regression network branch, a deformable convolution is added after the feature map P3; and the position information of the target is output after the convolution layer; Step 3.5, steps 3.2 to 3.4 are added to the classification network branch and the regression network branch after the feature maps F2 and F1 respectively to obtain the category information and the position information of the target.

5. The method of claim 4, wherein, The target detection model in step 4 is constructed by selecting a target detection architecture of YOLOv5, adding a multi-scale feature fusion module to a basic feature extraction network in the target detection architecture, and adding a learning network based on feature decoupling to a multi-scale prediction branch in the target detection architecture.

6. The method of claim 5, wherein the method comprises: The method for constructing the target detection model in step 4 is as follows: Step 4.1, a target detection architecture of YOLOv5 is selected, and a multi-scale feature fusion module is added to a basic feature extraction network in the target detection architecture; Step 4.2, in the multi-scale prediction branch in the target detection architecture, a learning network based on feature decoupling is added to obtain a target detection model; Step 4.3, parameters of the target detection model are set: prior box size, network learning rate, optimization function, maximum iteration number, size of batch training and model saving location.

7. The method of claim 6, wherein the method comprises: The target detection model training, verification and application in step 5 include: Step 5.1, training the target detection model: loading the target detection model obtained in step 4, pre-training and adjusting the model parameters by using the airport flight area target detection data set constructed in step 1; Step 5.2, optimizing the model: verifying by using the verification set, and adjusting the training parameters and optimizing the target detection model according to the verification result; Step 5.3, deploying the model: applying the optimized target detection model to the monitoring video to obtain the target category and location information in the video.

8. The method of claim 7, wherein the method comprises: The verification by using the verification set in step 5.2 is verifying in the verification set image, identifying the target category position in the video to be detected, and outputting the corresponding target category information and target box information.

9. The method of claim 8, wherein the method comprises: The calibration of each target to be detected in the typical image in step 1 is calibrated by using a target detection labeling software.

Images