Airport runway intrusion identification method based on esnb algorithm

Abstract

The application relates to the technical field of airport safety monitoring, in particular to an airport runway intrusion identification method based on an ESNB algorithm, which comprises the following steps: constructing an airport runway intrusion data set; constructing an improved Inception U-Net network, obtaining a teacher network based on the improved Inception U-Net network; obtaining an optimal student network based on the teacher network through a multi-objective evolutionary algorithm; performing knowledge distillation on the optimal student network based on the teacher network and the airport runway intrusion data set, and obtaining an ESNB network; and deploying the ESNB network in an airport for airport runway intrusion identification. The scheme can effectively overcome the problems of low airport runway intrusion identification efficiency and high computing power requirement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of airport safety monitoring, and particularly relates to an airport runway intrusion identification method based on an ESNB algorithm. BACKGROUND

[0002] Airport runway intrusion is defined by the International Civil Aviation Organization (ICAO) as any intrusion involving an aircraft, vehicle, or pedestrian that occurs in error within the protected area used for aircraft takeoff and landing in an airport. The occurrence of such events not only poses a serious threat to aviation safety, but is also a key link in the prevention of accidents on the airport. At present, in order to improve the monitoring capability of the airport runway and timely warn of intrusion risks, the following types of solutions are mainly used:

[0003] The first type is based on radar and cooperative surveillance technology. This type of technology can achieve all-weather detection of airport surface targets by transmitting and receiving electromagnetic waves through ASDE-X ground surveillance radar systems. Or it uses ADS-B cooperative surveillance technology, which relies on cooperative targets such as aircraft or vehicles to actively transmit their own position information, which is monitored by ground receiving stations. However, this type of technology has a detection blind area, which is prone to missed reports and poses a major safety hazard. In addition, the deployment and maintenance of radar systems are costly and difficult to popularize in small and medium-sized airports.

[0004] The second type is based on traditional video monitoring technology. This type of technology deploys cameras at key locations in the airport to obtain real-time video streams of the airport runway and surrounding areas. Monitoring personnel manually interpret the video images, but manual interpretation is prone to fatigue and oversight, and the response speed is slow, which cannot meet the stringent requirements of airport runway intrusion warning for immediacy, resulting in serious delays in response to warnings.

[0005] The third type is emerging visual recognition technology based on deep learning, such as models based on the YOLO (You Only Look Once) algorithm, which aims to directly identify and locate the bounding box of targets such as aircraft and vehicles from images. Or models based on network architectures such as U-Net or Inception U-Net, which aim to classify images at the pixel level and accurately segment the outlines of different areas such as aircraft, airport runway, and background. Then, by analyzing the relative positional relationship between the segmented aircraft mask and the airport runway mask, it is determined whether an intrusion event has occurred. Although visual recognition technology based on deep learning has lower deployment and maintenance costs, is easy to popularize, and can avoid the fatigue and oversight risks of manual interpretation based on traditional video monitoring technology, the complex structure and large number of parameters of deep learning models result in high computational costs and slow inference speeds, which can easily cause information processing delays and prevent accidents from occurring.

[0006] Therefore, how to provide an airport runway intrusion identification method capable of overcoming the low identification efficiency and high calculation cost of the existing airport runway intrusion identification method is a problem that needs to be solved by those skilled in the art. SUMMARY

[0007] Therefore, the present application provides an airport runway intrusion identification method based on an ESNB algorithm, which solves the problems involved in the background art, and the technical solution is as follows:

[0008] S1, constructing an airport runway intrusion data set;

[0009] S2, constructing an improved Inception U-Net network, and obtaining a teacher network based on the improved Inception U-Net network;

[0010] S3, obtaining an optimal student network based on the teacher network through a multi-objective evolutionary algorithm;

[0011] S4, performing knowledge distillation on the optimal student network based on the teacher network and the airport runway intrusion data set, and obtaining an ESNB network;

[0012] S5, deploying the ESNB network in an airport for airport runway intrusion identification.

[0013] In the above technical solution, through the dual optimization mechanism of the multi-objective evolutionary algorithm and the knowledge distillation, the core problems of target missed detection, model delay, etc. in the existing airport runway intrusion identification technology are solved, and an intelligent solution with high precision, high efficiency, and strong robustness is provided, which is suitable for real-time deployment requirements of various airport safety monitoring scenes.

[0014] Further, the improved Inception U-Net network specifically comprises:

[0015] an encoder, a bottleneck network, and a decoder;

[0016] The encoder comprises a first Inception attention module, a first max-pooling layer, a second Inception attention module, a second max-pooling layer, a third Inception attention module, a third max-pooling layer, a fourth Inception attention module, and a fourth max-pooling layer connected in sequence;

[0017] The decoder comprises a first reverse convolution concatenation layer, a sixth Inception attention module, a second reverse convolution concatenation layer, a seventh Inception attention module, a third reverse convolution concatenation layer, an eighth Inception attention module, a fourth reverse convolution concatenation layer, a ninth Inception attention module, and a softmax activation layer connected in sequence;

[0018] The bottleneck network comprises a fifth Inception attention module, a first cropping layer, a second cropping layer, a third cropping layer and a fourth cropping layer, an input end of the fifth Inception attention module is connected with an output end of the fourth maximum pooling layer, and an output end of the fifth Inception attention module is connected to an input end of the first reverse convolution layer;

[0019] The output of the first Inception attention module is connected to an input end of the fourth reverse convolution splicing layer after the first cropping layer; the output of the second Inception attention module is connected to an input end of the third reverse convolution splicing layer after the second cropping layer; the output of the third Inception attention module is connected to an input end of the second reverse convolution splicing layer after the third cropping layer; and the output of the fourth Inception attention module is connected to an input end of the first reverse convolution splicing layer after the fourth cropping layer.

[0020] The first Inception attention module, the second Inception attention module, the third Inception attention module, the fourth Inception attention module, the fifth Inception attention module, the sixth Inception attention module, the seventh Inception attention module, the eighth Inception attention module and the ninth Inception attention module all adopt the same module structure, comprising an Inception layer and a CBAM attention layer connected in sequence.

[0021] In the above technical scheme, multi-level features are gradually extracted through the four-level downsampling process in the encoder, and the airport runway and target information from local details to global semantics are effectively captured. The decoder gradually restores the spatial resolution through the reverse convolution and the alignment of the cropping layer, ensuring the accuracy of the segmentation boundary. The introduced Inception layer fuses the features of different receptive fields through a multi-branch parallel structure, enhancing the adaptability to targets with significant scale differences in the airport scene. The CBAM attention mechanism at the end of each module further focuses the model on key channels and spatial regions, effectively suppressing complex background interference and improving the robustness to challenging scenes such as occlusion and small targets. The overall structure significantly improves the feature expression ability and recognition accuracy of the model in the specific task of airport monitoring on the basis of maintaining the excellent segmentation performance of U-Net through modular design and attention enhancement.

[0022] Further, each Inception layer comprises a branch unit, a feature splicing layer, a full connection layer and a dimension control convolution layer, the four branches of the branch unit are respectively:

[0023] The first branch comprises a 1x1 convolution unit;

[0024] a second branch comprising a 1x1 convolutional unit and a 3x3 convolutional unit connected in sequence;

[0025] a third branch comprising a 1x1 convolutional unit and a 5x5 convolutional unit connected in sequence;

[0026] a fourth branch comprising a 3x3 max-pooling unit and a 1x1 convolutional unit connected in sequence;

[0027] The first branch, the second branch, the third branch and the fourth branch are connected to an input end of a feature concatenation layer, an output end of the feature concatenation layer is connected to an input end of a full connection layer, an output end of the full connection layer is connected to a dimension control convolutional layer, the dimension control convolutional layer is a 1x1 convolutional unit, and an output end of the dimension control convolutional layer is connected to an input end of a CBAM attention layer.

[0028] In the above scheme, the Inception layer realizes excellent balance between multi-scale feature perception and computational efficiency through its four-branch parallel structure. The four branches use 1x1, 3x3, 5x5 convolutional kernels and pooling operations, which can simultaneously capture features of different receptive fields at the same network level, thereby effectively dealing with the problem of variable aircraft target size in the airport scene. The design also has efficient dimension reduction capability, using 1x1 convolution to compress the number of channels before 3x3 and 5x5 convolution, which significantly reduces the model complexity. Finally, the feature maps output by each branch are fused through concatenation and 1x1 convolution to form a "feature map" with rich information, providing high-quality input for the subsequent attention mechanism and enhancing the model's feature extraction ability and robustness for key targets in complex environments.

[0029] Further, the CBAM attention layer comprises a channel attention network layer and a spatial attention network layer connected in sequence;

[0030] The channel attention network layer comprises two parallel first average pooling layers and fifth max-pooling layers, output ends of the first average pooling layers and the fifth max-pooling layers are connected to a weight sharing layer, an output end of the weight sharing layer is connected to a feature summation layer, an output end of the feature summation layer is connected to a first sigmoid activation layer, an output end of the first sigmoid activation layer is connected to a first data weighting layer, the first data weighting layer also receives data input into the channel attention network layer, and an output end of the first data weighting layer is connected to an input end of the spatial attention network layer;

[0031] The spatial attention network layer includes two parallel second average pooling layers and sixth maximum pooling layers, the output ends of the second average pooling layers and the sixth maximum pooling layers are connected to a Concat layer, the output end of the Concat layer is connected to a weight learning convolution layer, the output end of the weight learning convolution layer is connected to a second sigmoid activation layer, the output end of the second sigmoid activation layer is connected to a second data weighting layer, and the second data weighting layer also receives data output by the first data weighting layer.

[0032] In the technical solution, the CBAM attention layer realizes adaptive enhancement of key features through the channel and spatial attention mechanism in series. The channel attention network accurately evaluates and strengthens the importance of different feature channels by using maximum pooling and average pooling in parallel and integrating them through a shared weight layer, so that the model can focus on more discriminative information dimensions. Subsequently, the spatial attention network locates and highlights the key regions of the target in the spatial level through similar double-pooling operations and convolution layers, effectively suppressing the interference of complex backgrounds and irrelevant noise. This sequential "channel-spatial" attention mechanism, working together, enables the model to dynamically and accurately concentrate computing resources on the key target in the image, significantly improving the robustness and recognition accuracy of the model in complex airport environments (such as occlusion and light changes).

[0033] Further, the improved Inception U-Net network is used as a teacher network, specifically:

[0034] An airport runway intrusion data set X={x i is input into the improved Inception U-Net network to obtain a prediction x i Each pixel in x is the probability of an aircraft in the airport Based on x A first loss function L1 is calculated:

[0035] The first loss function L1 includes a binary cross-entropy loss and a Dice loss, where the binary cross-entropy loss L BCE is represented as:

[0036]

[0037] Where N represents the total number of pixels of a single x i y j represents the label corresponding to the jth pixel of x i x i The jth pixel is the predicted probability of an aircraft in the airport.

[0038] The Dice loss L Diceis expressed as:

[0039]

[0040] wherein, is a smoothing constant.

[0041] The first loss function L1 is expressed as:

[0042]

[0043] wherein, represents a binary cross-entropy loss weight and , represents a Dice loss weight.

[0044] Based on the first loss value L1, the improved Inception U-Net is trained by using a back propagation method, and the improved Inception U-Net network at this time is saved as a teacher network after a training stop condition is reached.

[0045] In the above scheme, in view of the serious class imbalance problem caused by the extremely small proportion of the intrusion target pixels in the airport runway intrusion recognition, the technical scheme designs a compound loss function L1 which fuses the binary cross-entropy loss (BCE Loss) and the Dice loss, and achieves significant technical effects. The design makes the BCE Loss focus on the pixel-level classification accuracy, effectively constrains the "false alarm" of misjudging the background as an airplane; at the same time, the Dice Loss is not sensitive to class imbalance, and focuses on optimizing the overlap degree of the predicted area and the real area, and strongly suppresses the "miss alarm" of missing the airplane. The two are weighted and fused, and work together in the training process, which guarantees the high performance of the segmentation result in the pixel-level accuracy and the overall contour integrity, and reduces the false alarm and miss alarm risk of the airport runway intrusion event.

[0046] Further, the optimal student network obtained by the multi-objective evolutionary algorithm comprises:

[0047] S3-1, evaluating the importance of the branches in each Inception attention module in the teacher network, obtaining an importance vector V={v b}, and the calculation method of the importance score of the bth branch is as follows:

[0048]

[0049] wherein, represents the parameter set of the bth branch; ||.||1 represents the L1 norm; b=1,...,36.

[0050] S3-2, obtaining the optimal student network by using a genetic algorithm based on the importance vector.

[0051] Generate m student control vectors s based on the importance vector V x ={u b} Denoted as student control vector set S, construct student network set based on S, and perform fitness evaluation on student networks in the student network set:

[0052] (1) Input the verification set into the student network, and for each student network, the following formula is used:

[0053]

[0054] Where E represents the predicted pixel set belonging to the aircraft in the airport obtained by inputting the verification set into the student network, T represents the true pixel set belonging to the aircraft in the airport, and Fitness represents the fitness.

[0055] (2) Based on the fitness, a new population is constructed using the strategy of combining elite reservation and tournament selection.

[0056] Sq1, according to the fitness, extract E student networks with the highest fitness from the initial population as elite individuals, and the rest are ordinary individuals. The elite individuals are directly put into the new population and do not need subsequent tournament selection, crossover and mutation operations.

[0057] Sq2, based on the ordinary individuals of Sq1, select m-E student networks using tournament selection based on the fitness. It should be noted that the tournament selection method is implemented in the prior art and is not described here.

[0058] Sq3, based on the m-E student networks extracted, perform crossover and mutation operations based on the corresponding student control vectors, combine the student networks after crossover and mutation with the elite individuals to obtain a new population;

[0059] (3) Perform (1)-(3) on the new population, and when the maximum number of iterations is reached, the student network with the highest fitness in the new population at this time (3) is denoted as the optimal student network.

[0060] In the above scheme, the teacher network is pruned by a multi-objective evolutionary algorithm to realize the unification of model lightweight and high precision. Firstly, the importance of each branch of the teacher network is evaluated based on the L1 norm, and the population is initialized with this prior knowledge, which significantly improves the search efficiency. Then, taking the segmentation accuracy (IoU) as the core fitness index, combined with the elite reservation and tournament selection strategy, the genetic algorithm is guided to search efficiently in the multi-objective space composed of model complexity, inference speed and recognition accuracy. This method can automatically explore and retain the most critical network branches for airport runway intrusion identification task, eliminate redundant structures, and finally generate an optimal student network that significantly reduces the parameter quantity and greatly shortens the inference delay while still inheriting the high precision characteristics of the teacher network, successfully solving the contradiction between lightweight deployment and performance guarantee.

[0061] Further, the S4, in particular:

[0062] S4-1, calculate the soft label loss L soft ;

[0063] When the training data in the training set is input into the teacher network, the following is calculated:

[0064]

[0065] where p L is the teacher soft label that the Lth pixel of the training data is an airplane in the airport, v L represents the output data of the ninth Inception attention module of the teacher network when the Lth pixel of the training data is input into the teacher network; and T is a temperature coefficient.

[0066] When the training data in the training set is input into the optimal student network, the following is calculated:

[0067]

[0068] where q L is the student soft label that the Lth pixel of the training data is an airplane in the airport, z L represents the output data of the ninth Inception attention module of the optimal student network when the Lth pixel of the training data is input into the optimal student network.

[0069] The soft label loss L soft is represented as:

[0070]

[0071] where α represents a weight coefficient, and α=0.5.

[0072] S4-2, calculate the hard label loss L hard .

[0073]

[0074] wherein, is the predicted label of the Lth pixel input into the optimal student network, is the real label corresponding to the Lth pixel.

[0075] S4-3, based on L hard and L soft The distillation loss L2 is constructed, and the optimal student network is trained to obtain the ESNB network.

[0076]

[0077] The optimal student network is trained based on the distillation loss L2 using a back propagation algorithm, and the training is stopped when a training stop condition is reached, and the optimal student network at this time is saved and recorded as the ESNB network.

[0078] In the above scheme, the generalization ability of the teacher network is efficiently migrated to the lightweight optimal student network through the knowledge distillation method. The scheme uses the temperature coefficient T to soften the output of the teacher network, generates soft labels rich in inter-class relationship information, and guides the student network to learn its smooth probability distribution. At the same time, the hard label loss of the real label is combined to ensure the basic classification accuracy. The distillation loss function is constructed by weighted fusion of soft and hard label losses, so that the student network can effectively make up for the performance loss caused by structural pruning while inheriting the strong feature expression ability of the teacher network. The final ESNB network realizes a substantial reduction in parameter quantity and inference delay while maintaining high accuracy.

[0079] Compared with the prior art, the present application has at least the following advantages: the present application extracts images of two scenes of "aircraft intrusion on airport runway" and "no aircraft intrusion on airport runway" from the airport overhead image database, labels the images and makes them into an airport runway intrusion dataset, uses the improved Inception U-Net network to obtain a teacher network based on the airport runway intrusion dataset, uses a multi-objective evolutionary algorithm to prune the teacher network to obtain an optimal student network, and uses a knowledge distillation method to optimize the optimal student network, records the optimized optimal student network as an ESNB network, and finally deploys the ESNB network in an airport for airport runway intrusion recognition. The present method prunes a part of the neural network branches by pruning the teacher network using a multi-objective evolutionary algorithm, thereby achieving the purpose of reducing the amount of calculation. Since the optimal student network is calculated by a multi-objective evolutionary algorithm, the superior performance of the optimal student network in the identification task can be guaranteed. The optimal student network is only optimized by distillation, which further ensures that the optimal student network inherits the excellent identification performance of the teacher network with a smaller number of parameters. Attached Figure Description

[0080] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0081] Figure 1 This is a flowchart of an airport runway intrusion detection method based on the ESNB algorithm.

[0082] Figure 2 This is a diagram of the improved Inception U-Net network structure for an airport runway intrusion detection method based on the ESNB algorithm.

[0083] Figure 3 The structure of the Inception layer in the airport runway intrusion detection method based on the ESNB algorithm is shown.

[0084] Figure 4 This is a structural diagram of the channel attention network layer in the airport runway intrusion detection method based on the ESNB algorithm.

[0085] Figure 5 This is a structural diagram of the spatial attention network layer in an airport runway intrusion detection method based on the ESNB algorithm. Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0087] See Figure 1 This embodiment provides an airport runway intrusion identification method based on the ESNB algorithm, specifically including S1 to S5:

[0088] S1. Construct an airport runway intrusion dataset.

[0089] From the airport overhead image database, images covering two scenarios of "aircraft runway invasion" and "aircraft runway non-invasion" are extracted. For each image, if the aircraft runway has been invaded, the pixels in the overlapping area of the aircraft and the airport runway are assigned a label of 1, and the pixels in other areas are assigned a label of 0. For images without runway invasion, all pixels are assigned a label of 0. Twenty percent of the labeled images are used as verification data and are made into a verification set, and the remaining eighty percent are used as training data and are made into an airport runway invasion data set X = {x i}, wherein i = 1,..., n.

[0090] S2, constructing an improved Inception U-Net network, and obtaining a teacher network based on the improved Inception U-Net network.

[0091] As shown in Figure 2 , the improved Inception U-Net network includes an encoder, a bottleneck network, and a decoder. The encoder includes a first Inception attention module, a first max-pooling layer, a second Inception attention module, a second max-pooling layer, a third Inception attention module, a third max-pooling layer, a fourth Inception attention module, and a fourth max-pooling layer connected in sequence.

[0092] The decoder includes a first reverse convolution concatenation layer, a sixth Inception attention module, a second reverse convolution concatenation layer, a seventh Inception attention module, a third reverse convolution concatenation layer, an eighth Inception attention module, a fourth reverse convolution concatenation layer, a ninth Inception attention module, and a softmax activation layer connected in sequence.

[0093] The bottleneck network includes a fifth Inception attention module, a first cropping layer, a second cropping layer, a third cropping layer, and a fourth cropping layer. The input end of the fifth Inception attention module is connected to the output end of the fourth max-pooling layer, and the output end of the fifth Inception attention module is connected to the input end of the first reverse convolution layer.

[0094] The output of the first Inception attention module is connected to the input end of the fourth reverse convolution concatenation layer after passing through the first cropping layer; the output of the second Inception attention module is connected to the input end of the third reverse convolution concatenation layer after passing through the second cropping layer; the output of the third Inception attention module is connected to the input end of the second reverse convolution concatenation layer after passing through the third cropping layer; and the output of the fourth Inception attention module is connected to the input end of the first reverse convolution concatenation layer after passing through the fourth cropping layer.

[0095] The first Inception attention module, the second Inception attention module, the third Inception attention module, the fourth Inception attention module, the fifth Inception attention module, the sixth Inception attention module, the seventh Inception attention module, the eighth Inception attention module and the ninth Inception attention module all adopt the same module structure, each Inception attention module comprising an Inception layer and a CBAM attention layer connected in sequence.

[0096] As shown in Figure 3 each Inception layer comprises a branch unit, a feature splicing layer, a full connection layer, a dimension control convolution layer, the four branches of the branch unit being respectively:

[0097] a first branch, the first branch comprising a 1x1 convolution unit;

[0098] a second branch, the second branch comprising a 1x1 convolution unit and a 3x3 convolution unit connected in sequence;

[0099] a third branch, the third branch comprising a 1x1 convolution unit and a 5x5 convolution unit connected in sequence;

[0100] a fourth branch, the fourth branch comprising a 3x3 max-pooling unit and a 1x1 convolution unit connected in sequence.

[0101] The first branch, the second branch, the third branch and the fourth branch are all connected to the input end of the feature splicing layer, the output end of the feature splicing layer is connected to the input end of the full connection layer, the output end of the full connection layer is connected to the dimension control convolution layer, the dimension control convolution layer is a 1x1 convolution unit, and the output end of the dimension control convolution layer is connected to the input end of the CBAM attention layer.

[0102] The CBAM attention layer comprises a channel attention network layer and a spatial attention network layer connected in sequence;

[0103] As shown in Figure 4 the channel attention network layer comprises two parallel fifth max-pooling layers and first average-pooling layers, the output ends of the fifth max-pooling layers and the first average-pooling layers are connected to a weight sharing layer, the output end of the weight sharing layer is connected to a feature summation layer, the output end of the feature summation layer is connected to a first sigmoid activation layer, the output end of the first sigmoid activation layer is connected to a first data weighting layer, the first data weighting layer also receives data input into the channel attention network layer, and the output end of the first data weighting layer is connected to the input end of the spatial attention network layer.

[0104] As shown in Figure 5 , the spatial attention network layer includes two parallel sixth max-pooling layers and a second average-pooling layer, the output ends of the sixth max-pooling layers and the second average-pooling layer are connected to a Concat layer, the output end of the Concat layer is connected to a weight learning convolution layer, the output end of the weight learning convolution layer is connected to a second sigmoid activation layer, the output end of the second sigmoid activation layer is connected to a second data weighting layer, and the second data weighting layer also receives data output by the first data weighting layer.

[0105] S2-2, obtaining a teacher network based on the improved Inception U-Net network.

[0106] The airport runway intrusion data set X is input into the improved Inception U-Net network, and for each pixel x i , the probability that the pixel is an airplane in the airport is obtained , and a first loss function L1 is calculated based on . The first loss function L1 includes a binary cross-entropy loss and a Dice loss, where the binary cross-entropy loss L BCE is expressed as:

[0107]

[0108] where N represents the total number of pixels of a single x i , y j represents the label of the jth pixel of the airport runway intrusion data x i , and represents the predicted probability that the jth pixel of the airport runway intrusion data x i is an airplane in the airport.

[0109] The Dice loss L Dice is expressed as:

[0110]

[0111] where is a smoothing constant, .

[0112] The first loss function L1 is expressed as:

[0113]

[0114] where represents the weight of the binary cross-entropy loss, and , represents the weight of the Dice loss, and , In this embodiment, the optimal and .

[0115] Based on the first loss value L1, the improved Inception U-Net network is trained for 500 rounds by using the back propagation method, and the obtained Inception U-Net network is denoted as a teacher network.

[0116] S3, based on the teacher network, the optimal student network is obtained by a multi-objective evolutionary algorithm.

[0117] S3-1, the importance of each branch in the Inception attention module in the teacher network is evaluated, and an importance vector V={v b} is obtained.

[0118] The importance of each branch in the Inception attention module in the teacher network is calculated. It should be noted that the teacher network contains 9 Inception attention modules, and each Inception attention module contains 4 branches. Therefore, from the first branch of the first Inception attention module to the fourth branch of the ninth Inception attention module, there are a total of 36 branches, i.e. the importance scores of the branches of the first Inception attention module are v1, v2, v3, v4, and the importance scores of the branches of the ninth Inception attention module are v 33 , v 34 , v 35 , v 36 .

[0119] The importance score of the bth branch is calculated as follows:

[0120]

[0121] wherein, represents the parameter set of the bth branch; ||.||1 represents the L1 norm; b=1,...,36.

[0122] S3-2, based on the importance vector, the optimal student network is obtained by using a genetic algorithm.

[0123] S3-2-1, based on the importance vector V, a student control vector s x ={u b} is generated, and an initial population is constructed based on the student control vector set.

[0124] The student control vector s x ={u bis a vector composed of 0 and 1 with a length of 36 bits, where b = 1,..., 36, each bit of the student control vector represents the reserved or pruned state of the corresponding branch of the student network, if the corresponding value in the student control vector is 0, it represents that the corresponding branch of the student network is in the pruned state, if the corresponding value in the student control vector is 1, it represents that the corresponding branch of the student network is in the reserved state. The generation method of the student control vector is as follows:

[0125] Sp1, calculate the student control vector s x the b-th element u b of the probability P:

[0126]

[0127] where, represents the maximum importance score in the importance vector V;

[0128] Sp2, determine the value of u b :

[0129] Generate a random number r, where 0 ≤ r < 1, if r < P(u b = 1), then u b = 1, otherwise u b = 0;

[0130] Generate m student control vectors using the method of Sq1-Sq2, denoted as the student control vector set S = {s x}, where x = 1,..., m.

[0131] Based on the student control vector set S, a student network set is constructed, for each student control vector in the student control vector set, the following method is used:

[0132] Take the teacher network as the base network, the base network inherits the network structure and network parameters of the teacher network, traverse the student control vector, and prune the base network according to the student control vector, that is, when the value in the student control vector is 0, set the branch at the corresponding position of the base network to the pruned state, when the value in the student control vector is 1, set the branch at the corresponding position of the base network to the reserved state, after pruning the base network, a student network is obtained, then continue to construct student networks in the same way until m student control networks are constructed, the constructed student network set is denoted as the initial population.

[0133] S3-2-2, based on the initial population, use genetic algorithm to obtain the optimal student network.

[0134] (1) input the verification set into the student network, for each student network, the following formula is used:

[0135]

[0136] wherein, E represents the predicted pixel set belonging to the aircraft in the airport obtained by inputting the verification set into the student network, T represents the pixel set actually belonging to the aircraft in the airport in the verification set, Fitness represents the fitness, since there are m student networks in the initial population, m fitness values will be obtained.

[0137] (2) Based on the fitness, a new population is constructed by using the strategy of combining elite reservation and tournament selection.

[0138] Sq1, according to the fitness, E student networks with the highest fitness values are extracted from the initial population as elite individuals, and the rest are ordinary individuals, the elite individuals are directly put into the new population, and do not need subsequent tournament selection, crossover and mutation operations.

[0139] Sq2, based on the ordinary individuals of Sq1, m-E student networks are selected by using the tournament selection method according to the fitness, it should be noted that the tournament selection method is realized by the prior art, and will not be described here.

[0140] Sq3, based on the m-E student networks, crossover and mutation operations are performed according to the corresponding student control vectors, the student networks after crossover and mutation are combined with the elite individuals to obtain a new population; it should be noted that the crossover and mutation operation method is realized by the prior art, and will not be described here.

[0141] (3) The new population is executed (1)-(3), when the maximum number of iterations is reached, the student network with the highest fitness in the new population of (3) at this time is recorded as the optimal student network.

[0142] S4, based on the teacher network and the airport runway intrusion data set, knowledge distillation is performed on the optimal student network to obtain an ESNB network.

[0143] S4-1, the soft label loss L is calculated soft .

[0144] The training data in the airport runway intrusion data set is input into the teacher network, and the following is calculated:

[0145]

[0146] wherein, p L is the teacher soft label of the Lth pixel of the training data being an aircraft in the airport, v L represents the output data of the ninth Inception attention module of the teacher network when the Lth pixel of the training data is input into the teacher network, and T is a temperature coefficient, T = 10.

[0147] The training data in the airport runway intrusion dataset is input into the optimal student network, and the calculation is as follows:

[0148]

[0149] Wherein, q L is the student soft label of the Lth pixel of the training data that is an airplane in the airport, z L represents the output data of the ninth Inception attention module of the optimal student network input into the optimal student network by the Lth pixel of the training data; T represents the temperature coefficient, T = 10.

[0150] The soft label loss L soft is expressed as:

[0151]

[0152] Wherein, a represents the weight coefficient, a = 0.5.

[0153] S4-2, calculate the hard label loss L hard .

[0154]

[0155] Wherein, is the predicted label obtained by inputting the Lth pixel into the optimal student network, is the true label corresponding to the Lth pixel.

[0156] S4-3, based on L hard and L soft , construct the distillation loss L2, and train the optimal student network to obtain the ESNB network.

[0157]

[0158] The optimal student network is trained based on the distillation loss L2 using the back propagation algorithm, and the training is stopped when the number of training reaches 500 rounds, and the optimal student network at this time is saved and recorded as the ESNB network.

[0159] S5, deploy the ESNB network in the airport for airport runway intrusion identification.

[0160] The ESNB network is converted into a format suitable for deployment in the airport monitoring system, integrated into the existing monitoring system in the airport, and used for airport runway intrusion detection by real-time processing of camera video streams. Once an intrusion event is identified, an audible and visual alarm is triggered and the tower controller is notified, and the event details are recorded for post-event analysis and system optimization.

[0161] The above detailed description of the specific embodiments of the present application has been given to understand the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An airport runway intrusion identification method based on ESNB algorithm, characterized in that, The method comprises the following steps: S1, constructing an airport runway intrusion dataset; S2, constructing an improved Inception U-Net network, and obtaining a teacher network based on the improved Inception U-Net network; The teacher network based on the improved Inception U-Net network is specifically: The airport runway intrusion dataset X = {x i} is input into the improved Inception U-Net network, and a prediction x i is obtained, wherein each pixel of x i is the probability of an airplane in the airport , and a first loss function L1 is calculated based on The first loss function L1 includes a binary cross-entropy loss and a Dice loss, where the binary cross-entropy loss L BCE is represented as: where N represents a single x i total number of pixels, y j represents x i the label corresponding to the jthpixel; represents the airport runway intrusion data x i the jthpixel is the predicted probability of an aircraft within the airport. Dice loss L Dice is represented as: wherein is a smoothing constant; The first loss function L1 is represented as: wherein, represents a binary cross-entropy loss weight and , represents a Dice loss weight and ; Based on the first loss value L1, the improved Inception U-Net is trained by using a back propagation method, and the improved Inception U-Net network at this time is saved as a teacher network after a training stop condition is reached; S3, obtaining an optimal student network based on the teacher network by using a multi-objective evolutionary algorithm; S4, performing knowledge distillation on the optimal student network based on the teacher network and the airport runway intrusion dataset, and obtaining an ESNB network; The S4 is specifically: S4-1, calculate the soft label loss L soft ; When training data in a training set is input into the teacher network, the following is calculated: wherein p L is the teacher soft label of the Lth pixel of the training data being an airplane within the airport, v L represents the output data of the ninth Inception attention module of the teacher network into which the Lth pixel of the training data is input; and T is a temperature coefficient. When the training data in the training set is input into the optimal student network, the following is calculated: wherein q L is the student soft label that the Lth pixel of the training data is an airplane within the airport, z L represents the output data of the Lth pixel of the training data input to the optimal student network, the output data of the ninth Inception attention module of the optimal student network Soft label loss L soft is represented as: Wherein, α represents a weight coefficient, and α = 0.5; S4-2, calculating the hard label loss L hard ; wherein, is the predicted label obtained by inputting the Lth pixel into the optimal student network, is the true label corresponding to the Lth pixel; S4-3, based on L hard and L soft Construct the distillation loss L2, and train the optimal student network to obtain the ESNB network; Based on the distillation loss L2, the optimal student network is trained by using a back propagation algorithm, and the training is stopped when a training stop condition is reached, and the optimal student network at this time is saved as an ESNB network; S5, deploying the ESNB network in an airport for airport runway intrusion recognition.

2. The ESNB algorithm based method for identifying runway incursion at an airport as claimed in claim 1 wherein, The improved Inception U-Net network specifically comprises: An encoder, a bottleneck network and a decoder; The encoder comprises a first Inception attention module, a first maximum pooling layer, a second Inception attention module, a second maximum pooling layer, a third Inception attention module, a third maximum pooling layer, a fourth Inception attention module and a fourth maximum pooling layer connected in sequence; The decoder comprises a first reverse convolution concatenation layer, a sixth Inception attention module, a second reverse convolution concatenation layer, a seventh Inception attention module, a third reverse convolution concatenation layer, an eighth Inception attention module, a fourth reverse convolution concatenation layer, a ninth Inception attention module and a softmax activation layer connected in sequence; The bottleneck network comprises a fifth Inception attention module, a first cropping layer, a second cropping layer, a third cropping layer and a fourth cropping layer, an input end of the fifth Inception attention module is connected with an output end of the fourth maximum pooling layer, and an output end of the fifth Inception attention module is connected with an input end of the first reverse convolution layer; The output of the first Inception attention module is connected to the input end of the fourth reverse convolution splicing layer after the first pruning layer; the output of the second Inception attention module is connected to the input end of the third reverse convolution splicing layer after the second pruning layer; the output of the third Inception attention module is connected to the input end of the second reverse convolution splicing layer after the third pruning layer; the output of the fourth Inception attention module is connected to the input end of the first reverse convolution splicing layer after the fourth pruning layer; The first Inception attention module, the second Inception attention module, the third Inception attention module, the fourth Inception attention module, the fifth Inception attention module, the sixth Inception attention module, the seventh Inception attention module, the eighth Inception attention module and the ninth Inception attention module all adopt the same module structure, which includes an Inception layer and a CBAM attention layer connected in sequence.

3. The ESNB algorithm based method for identifying runway incursion at an airport as claimed in claim 2, wherein, Each Inception layer includes a branch unit, a feature splicing layer, a full connection layer, and a dimension control convolution layer, the four branches of the branch unit are respectively: The first branch includes a 1*1 convolution unit; The second branch includes a 1*1 convolution unit and a 3*3 convolution unit connected in sequence; The third branch includes a 1*1 convolution unit and a 5*5 convolution unit connected in sequence; The fourth branch includes a 3*3 max pooling unit and a 1*1 convolution unit connected in sequence; The first branch, the second branch, the third branch and the fourth branch are all connected to the input end of the feature splicing layer, the output end of the feature splicing layer is connected to the input end of the full connection layer, the output end of the full connection layer is connected to the dimension control convolution layer, the dimension control convolution layer is a 1*1 convolution unit, and the output end of the dimension control convolution layer is connected to the input end of the CBAM attention layer.

4. The ESNB algorithm based method for identifying runway incursion at an airport as claimed in claim 2, wherein, The CBAM attention layer includes a channel attention network layer and a spatial attention network layer connected in sequence; The channel attention network layer includes two parallel fifth max pooling layers and first average pooling layers, the output ends of the fifth max pooling layers and the first average pooling layers are connected to a weight sharing layer, the output end of the weight sharing layer is connected to a feature summation layer, the output end of the feature summation layer is connected to a first sigmoid activation layer, the output end of the first sigmoid activation layer is connected to a first data weighting layer, the first data weighting layer also receives data input into the channel attention network layer, and the output end of the first data weighting layer is connected to the input end of the spatial attention network layer; The spatial attention network layer comprises two juxtaposed sixth max-pooling layers and a second average-pooling layer, output ends of the sixth max-pooling layers and the second average-pooling layer are connected to a Concat layer, an output end of the Concat layer is connected to a weight learning convolution layer, an output end of the weight learning convolution layer is connected to a second sigmoid activation layer, an output end of the second sigmoid activation layer is connected to a second data weighting layer, and the second data weighting layer also receives data output by the first data weighting layer.

5. The ESNB algorithm based method for identifying runway incursion at an airport as claimed in claim 4, wherein, The optimal student network obtained through the multi-objective evolutionary algorithm comprises: S3-1, evaluate the importance of each branch in the Inception attention module in the teacher network, obtain the importance vector V={v b}, the importance score of the bth branch is calculated as follows: wherein a set of parameters representing the b-th branch; ||. ||1represents the L1 norm; b = 1,..., 36; S3-2, obtaining the optimal student network through a genetic algorithm based on the importance vector; Generate m student control vectors s based on the importance vector V x = {u b}, denoted as a student control vector set S, construct a student network set based on S, and perform fitness evaluation on the student network in the student network set: (1) inputting the verification set into the student network, and having the following formula for each student network: wherein E represents a predicted pixel set belonging to an aircraft in an airport obtained by inputting the verification set into the student network, T represents a pixel set belonging to an aircraft in an airport in the verification set, and Fitness represents fitness; (2) constructing a new population based on fitness by using an elite reservation and tournament selection combined strategy; Sq1, extracting E student networks with the highest fitness values from the initial population as elite individuals according to fitness, and the rest as common individuals, and putting the elite individuals into a new population; Sq2, selecting m-E student networks from the common individuals of Sq1 by using a tournament selection method according to fitness; Sq3, performing crossover and mutation operations on the m-E student networks according to corresponding student control vectors, and combining the student networks after the crossover and mutation operations with the elite individuals to obtain a new population; (3) performing (1)-(3) on the new population, and recording a student network with the highest fitness in the new population of (3) at this time as the optimal student network when a maximum number of iterations is reached.

Images