An airport runway line detection method based on attention guidance

Abstract

The application discloses an airport runway line detection method based on attention guidance and belongs to the technical field of target detection in computer vision, and comprises the following steps: acquiring airport runway image data; performing adaptive contrast enhancement processing on the airport runway image data; performing multi-scale edge detection on the enhanced airport runway image data to obtain fused local features; performing multi-dimensional attention fusion on the fused local features to obtain fused features; performing classification regression calculation on the fused features to predict airport runway line confidence and geometric parameters; and performing airport runway line detection based on the airport runway line confidence. Through adaptive image enhancement and deep learning detection, the application comprehensively adopts adaptive contrast enhancement technology, multi-scale edge detection, attention mechanism feature extraction and other technologies, realizes efficient airport runway line detection, and ensures high-precision detection of the airport runway under complex environmental conditions.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target detection in computer vision, and particularly relates to an airport runway line detection method based on attention guidance. BACKGROUND

[0002] With the increasing popularity of automatic driving, its application range is not limited to the automatic driving of vehicles, and is also increasingly widely applied to fixed-wing unmanned driving. However, the existing large fixed-wing still needs to be manually driven by a towing vehicle to be towed when entering and exiting the hangar, and the scheduling process from the hangar to the designated take-off position cannot be completely automated, which leads to an increase in the number of fixed-wing aircrafts in the airport, the consumption of manpower and material resources for the scheduling and parking of fixed-wing aircrafts on the ground, and a decrease in the operation efficiency of the aircrafts in the airport. Therefore, the fixed-wing aircrafts need to be able to autonomously travel from the hangar to the take-off position on the airport road to realize autonomous driving on the airport road, which can greatly reduce the required manpower and material resources and improve the operation efficiency of the airport.

[0003] The runway detection of the airport is the basis for realizing the autonomous driving of the fixed-wing aircrafts on the airport runway. Although the existing similar lane line detection methods have achieved good results, there are still limitations in dealing with the airport environment. The airport runway and taxiway are huge, and a single field of view is difficult to cover the complete scene. The runway surface is usually gray, and the contrast with the marking line is low. Compared with urban roads, there are fewer identifiable feature points in the airport environment. The open environment is easily affected by weather and lighting conditions. In some existing solutions, there is no algorithm model improved for the autonomous driving of the fixed-wing aircrafts in the airport. Therefore, the stability of most models is not high. Moreover, the model not only needs to be robust but also needs to be efficient. In many application scenarios, the airport runway detection must be completed in real time or even faster to save processing capacity for other systems of the aircraft, which is also a requirement that many models cannot meet. SUMMARY

[0004] The purpose of the present application is to solve the problem that the traditional detection method is difficult to cope with the complex input of the wide, low-contrast and sparse feature environment of the airport by providing an airport runway line detection method based on attention guidance.

[0005] To achieve the above purpose, the technical solution adopted by the present application is:

[0006] An airport runway line detection method based on attention guidance, comprising the following steps:

[0007] S1, acquiring airport runway image data;

[0008] S2, performing adaptive contrast enhancement processing on the airport runway image data;

[0009] S3. Perform multi-scale edge detection on the enhanced airport runway image data to obtain fused local features;

[0010] S4. Perform multi-dimensional attention fusion on the fused local features to obtain the fused features;

[0011] S5. Perform classification regression calculations on the fused features to predict airport runway line confidence and geometric parameters;

[0012] S6. Airport runway line detection based on airport runway line confidence level.

[0013] Furthermore, step S2 specifically includes the following sub-steps:

[0014] S21. Calculate the image contrast score and perform airport environment segmentation based on the image contrast score:

[0015]

[0016] In the formula, Represents the image contrast score; The standard deviation of the image gradient; The average brightness of the image. The gradient operator for an image;

[0017] When C S If the value is less than 20, it indicates an extreme environment;

[0018] When 20 ≤ C S A value less than 40 indicates a medium-quality environment.

[0019] When C S A value of ≥ 40 indicates a good environment;

[0020] S22. Adaptive image enhancement based on image contrast score, expressed as:

[0021]

[0022] In the formula, This represents the image after adaptive enhancement. This represents the input airport runway image data; clahe This represents the adaptive CLAHE enhancement function;

[0023] S23. Perform directional feature enhancement processing on the adaptively enhanced image:

[0024]

[0025] In the formula, This represents the image after directional feature enhancement. 、 represents a corresponding weight; represents a convolution operation; represents a horizontal direction detection kernel; represents a vertical direction detection kernel.

[0026] Further, the S3 specifically comprises the following steps:

[0027] S31, the directional feature enhanced image is input into the backbone feature extraction network, and a plurality of convolution layers are used to extract feature maps of different scales;

[0028] S32, the feature maps of different scales are input into the feature pyramid network for multi-scale feature fusion, which is represented as:

[0029]

[0030] In the formula, represents the fusion local feature vector of the i-th layer; represents a 1x1 convolution operation; represents the feature map of the i-th layer; represents an up-sampling operation; represents the fusion local feature of the i-th layer.

[0031] Further, the S4 specifically comprises the following steps:

[0032] S41, the channel attention mechanism, the spatial attention mechanism and the linear attention mechanism are used to process the fusion local feature respectively, and the corresponding multi-dimensional attention weight is calculated;

[0033] S42, multi-dimensional attention fusion is performed based on the multi-dimensional attention weight to obtain the fusion feature.

[0034] Further, in the S41, the channel attention mechanism is used to process the fusion local feature, which is represented as:

[0035]

[0036] In the formula, represents a channel attention weight; is an activation function; represents a multi-layer perception; represents a global average pooling; is a fusion local feature; represents a global maximum pooling; ​​​​

[0037] Based on the channel attention weight, and introducing the residual connection, the channel attention enhanced feature is calculated:

[0038]

[0039] In the formula, The channel attention enhanced feature is represented by; The element-wise multiplication is represented by;

[0040] The spatial attention mechanism is used to process the fused local features, which is represented by:

[0041]

[0042] In the formula, The spatial attention weight is represented by; The feature concatenation operation is represented by; ChannelAvg The cross-channel average operation is represented by; ChannelMax The cross-channel maximum operation is represented by;

[0043] The linear attention mechanism is used to process the fused local features, which is represented by:

[0044]

[0045] In the formula, The linear attention weight is represented by; The normalized exponential function is represented by; The full connection layer is represented by; And Both are the index of anchor point.

[0046] Further, the S42 specifically includes:

[0047] The channel attention mechanism, the spatial attention mechanism and the linear attention mechanism are respectively strengthened and measured, which is represented by:

[0048]

[0049]

[0050]

[0051] In the formula, The intensity index of channel attention is represented by; The intensity index of spatial attention is represented by; The intensity index of linear attention is represented by; The batch of feature map is represented by; The channel number of feature map is represented by; The height of feature map is represented by; denotes a width of a feature map; denotes a number of anchors; denotes a feature map batch index; denotes a channel dimension index; h denotes a height dimension index; denotes a width dimension index; denotes a channel attention matrix; denotes a spatial attention matrix; denotes a linear attention matrix;

[0052] based on the intensity indicator of the channel attention, the intensity indicator of the spatial attention, and the intensity indicator of the linear attention, a fusion weight vector is calculated:

[0053]

[0054]

[0055]

[0056]

[0057] wherein, is the fusion weight vector, denotes a channel attention fusion weight vector, denotes a spatial attention fusion weight vector, denotes a linear attention fusion weight vector; denotes a 3-dimensional vector composed of the intensity indicators of the three attentions; denotes a transpose; is a hidden layer output feature; denotes an activation function; denotes a weight matrix of the first layer network; denotes a bias term of the first layer network; denotes a weight matrix of the second layer network; denotes a bias term of the second layer network;

[0058] based on the fusion weight vector, the channel attention mechanism, the spatial attention mechanism, and the linear attention mechanism are fused:

[0059]

[0060] wherein, denotes a fused feature.

[0061] Further, in the S5, the fused feature is input into a double-branch prediction module and a classification regression calculation is performed to predict an airport runway line confidence and a geometric parameter, which specifically includes the following steps:

[0062] S51, adopt the classification branch prediction airport runway line confidence in the double branch prediction module:

[0063]

[0064] In the formula, indicates the airport runway line confidence; indicates the classification weight matrix; indicates the bias parameter of the classification branch;

[0065] S52, adopt the regression branch prediction airport runway line geometric parameters in the double branch prediction module:

[0066]

[0067]

[0068] In the formula, indicates the predicted parameter offset; indicates the regression weight matrix; is the bias parameter of the regression; indicates the final decoded airport runway line parameters; indicates the predefined anchor point parameters;

[0069] S53, calculate the loss function of the airport runway line detection in the double branch prediction module.

[0070] Further, the S53 specifically includes:

[0071] Adopt the focal loss function to calculate the classification loss of the double branch prediction module, which is represented as:

[0072]

[0073] In the formula, indicates the classification loss; , is the focal loss parameter; is the prediction probability of the airport runway line;

[0074] Adopt the smooth L1 loss function to calculate the regression loss of the double branch prediction module, which is represented as:

[0075]

[0076] In the formula, indicates the regression loss; SmoothL1 indicates the smooth loss function; indicates the predicted parameters, corresponding to the final decoded airport runway line parameters ; represents the real parameter, which is derived from artificial labeling;

[0077] The geometric constraint loss of the double-branch prediction module is represented as:

[0078]

[0079] wherein:

[0080]

[0081]

[0082]

[0083] in the formula, represents the geometric constraint loss of the airport runway line; 、 、 is a constraint weight coefficient; is the parallelism constraint loss; is the length consistency constraint loss; is the angle constraint loss; , is the angle of different airport runway lines; is the predicted length, is the standard length of the airport runway line; is the reference angle of the airport runway;

[0084] The total loss of the double-branch prediction module is represented as:

[0085]

[0086] in the formula, represents the total loss; is a classification loss weight; is a regression loss weight; is a geometric constraint loss weight.

[0087] Further, the S6 specifically comprises the following steps:

[0088] S61, based on the airport runway line confidence, using attention guidance to screen the candidate airport runway line;

[0089] wherein, the attention guidance is represented as:

[0090]

[0091] in the formula, is the attention-enhanced screening score of the th candidate; is the attention-enhanced screening score of the Each candidate confidence level; This is the attention enhancement coefficient; The weights for combining the three attention methods are expressed as follows:

[0092]

[0093] In the formula, 、 、 For attention fusion weights; The first Each corresponding channel, spatial, and linear attention weight;

[0094] The candidate airport runway lines selected are:

[0095]

[0096] In the formula, For the candidate set of airport runway lines, This represents the m-th candidate detection box; The filtering threshold;

[0097] S62. The distance between the filtered airport runway lines is measured based on the common valid index, and it is expressed as follows:

[0098]

[0099] In the formula, Indicates two airport runway lines and The distance between them; This represents the minimum value of the starting point of the anchor points of the two airport runway lines; This represents the maximum value of the endpoints of the anchor points of the two airport runway lines. Indicates airport runway lines The Middle i The x-coordinate of each anchor point; Indicates airport runway lines The Middle i The x-coordinate of each anchor point;

[0100] S63. Based on the airport runway line distance metric, a non-maximum suppression algorithm is used to identify and suppress redundant airport runway lines.

[0101] Furthermore, S63 specifically includes:

[0102] Define the distance threshold as When the distance between two airport runway lines meets the following conditions: When the distance threshold, the direction consistency and the confidence degree of the two airport runway lines satisfy the suppression mechanism, it is determined that the two airport runway lines overlap, and a suppression mechanism is set to further remove the duplicate airport runway lines.

[0103] The suppression mechanism is set according to the distance threshold, the direction consistency and the confidence degree of the two airport runway lines.

[0104]

[0105] In the formula, is the degree of direction consistency of the two airport runway lines. and are the confidence degrees of the two airport runway lines, respectively. is the direction consistency threshold.

[0106] When the two airport runway lines satisfy the suppression mechanism, the airport runway line with a low confidence degree is suppressed to realize the removal of the duplicate airport runway lines.

[0107] The airport runway line detection method based on attention guidance provided by the application has the following beneficial effects:

[0108] The application realizes efficient airport runway line detection by adaptive image enhancement and deep learning detection, and comprehensively uses adaptive contrast enhancement technology, multi-scale edge detection and attention mechanism feature extraction technology, to ensure high-precision detection of airport runways under complex environmental conditions.

[0109] The application significantly improves the precision and robustness of airport runway line detection through a multi-stage processing method.

[0110] The first stage is adaptive enhancement optimization, which automatically adjusts the CLAHE parameter according to the image contrast feature, effectively improves the low-contrast image quality, reduces the noise interference of the airport environment, and ensures the consistency of the input data.

[0111] In the second stage, the cross-layer refinement mechanism is combined with multi-scale feature fusion, and a convolutional neural network is used as the backbone network, which overcomes the problem of few feature points caused by extremely low contrast in the airport scene, and greatly improves the efficiency of feature extraction.

[0112] In the third stage, the improved deep learning network is combined with the attention mechanism, which first unifies the channel, spatial and linear three kinds of attention mechanisms into the airport runway line detection, and integrates local features and global features, so that the long-distance small runway line can be detected in the large-scale space span of the airport.

[0113] The fourth stage executes a classification regression algorithm through a double-branch prediction module and executes a non-maximum suppression algorithm through a post-processing module, so that the detection process is real-time and efficient, meets the real-time requirement of aircraft navigation, and is fully compatible with the safety standard of the airport. BRIEF DESCRIPTION OF DRAWINGS

[0114] Figure 1 A flowchart of the airport runway line detection method based on attention guidance according to an embodiment of the present application.

[0115] Figure 2 A schematic diagram of the cross-layer refinement network architecture according to an embodiment of the present application.

[0116] Figure 3 A schematic diagram of the multi-dimensional attention fusion architecture according to an embodiment of the present application. DETAILED DESCRIPTION

[0117] The specific embodiments of the present application are described below to facilitate the understanding of the present application by those skilled in the art, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

[0118] The airport runway line detection method based on attention guidance according to the present embodiment is suitable for various weather conditions, lighting environments and airport scales, supports different types of runway line detection, has a wide application prospect, in addition, is robust to environmental changes, can be extended to complex airport scenes, and provides an efficient technical solution for intelligent airport construction, for reference Figure 1 , which specifically includes the following contents:

[0119] S1, acquiring airport runway image data;

[0120] S2, performing adaptive contrast enhancement processing on the airport runway image data, which specifically includes the following steps:

[0121] S21, calculating an image contrast score and dividing the airport environment based on the image contrast score;

[0122] Specifically, the key contrast features of the airport runway image are acquired, the contrast distribution data of the image is acquired by using gradient statistics, variance analysis and other methods, and the image contrast score is specifically represented as:

[0123] Wherein, ;

[0124] In the formula, Contrast score of image, used to measure the relative contrast intensity of image, high value represents clear image edge, low value means image is fuzzy and edge is weak; Standard deviation of image gradient, as a normalization factor, reflects the intensity of image edge change; Mean intensity of image, Gradient operator of image; this step can accurately evaluate the quality of airport image under different light and weather conditions;

[0125] The specific division process is as follows:

[0126] When C S <20, it means extreme environment, such as thick fog, night;

[0127] When 20 ≤C S <40, it means medium quality environment, such as cloudy, dusk;

[0128] When CS≥ 40, it means good environment, such as sunny day;

[0129] This division covers the main scene types of airport and can adapt most medium quality images to obtain moderate enhancement, which meets the principle of "efficiency and effect balance" in engineering practice.

[0130] S22, adaptive CLAHE enhancement of image according to image contrast score, to convert low contrast image into high quality detection input, which is represented as:

[0131]

[0132] In the formula, Indicates the image after adaptive enhancement; Indicates the input airport runway image data; clahe Indicates the adaptive CLAHE enhancement function;

[0133] This embodiment performs three-layer hierarchical processing on adaptive parameter selection:

[0134] When the image contrast score is less than 20, clip_limit = 6.0, tile_size = (8, 8), and extreme low contrast enhancement is performed; when the image contrast score is greater than 20, clip_limit = 4.0, tile_size = (12, 12), and low contrast medium enhancement is performed; otherwise, clip_limit = 2.0, tile_size = (16, 16), and normal contrast light enhancement is performed.

[0135] wherein clip limit is a contrast limit factor, tile size is a local window size, the effect of the CLAHE algorithm is determined by clip limit and tile size, and the selection of both follows the principle of adapting to image quality: clip limit balances the enhancement effect and noise control, this parameter limits the maximum enhancement amplitude of histogram equalization, avoiding noise amplification caused by excessive processing. tile size balances local fineness and global consistency, this parameter controls the degree of localization of CLAHE, the smaller the window, the more focused the enhancement of local details; the larger the window, the more uniform the global tone.

[0136] S23, performing directional feature enhancement processing on the image after adaptive enhancement;

[0137] Specifically, the embodiment performs directional enhancement on the horizontal and vertical kernels of the runway marking, establishes continuity enhancement through morphological operation, and finally generates an image after directional feature enhancement.

[0138] Directional feature enhancement:

[0139] A special convolution kernel in the horizontal and vertical directions is used for edge detection, which specially captures the directional features of the runway marking, and the directional feature enhancement objective function is:

[0140]

[0141] In the formula, denotes the image after directional feature enhancement, , denotes the corresponding weight; Conv denotes convolution operation; denotes a horizontal direction detection kernel; denotes a vertical direction detection kernel.

[0142] The enhanced airport runway image of the embodiment is obviously better than the original airport runway image.

[0143] S3, constructing a cross-layer refinement network architecture, performing multi-scale edge detection on the enhanced airport runway image data to obtain fused local features, referring to Figure 2 , which specifically includes the following steps:

[0144] S31, inputting the image after directional feature enhancement into the backbone feature extraction network, and using multiple convolution layers to extract feature maps of different scales, which are represented as:

[0145]

[0146] In the formula, denotes the first feature map of the layer; the first layer output.

[0147] S32, input the feature maps of different scales into the feature pyramid network for multi-scale feature fusion, which is represented as:

[0148]

[0149] wherein, the fusion local feature of the first layer; represents a 1x1 convolution operation; the fusion local feature of the first layer; represents an up-sampling operation; the fusion local feature of the first layer.

[0150] S4, construct a multi-dimensional attention fusion architecture to perform multi-dimensional attention fusion on the fusion local feature to obtain a fusion feature, as shown in Figure 3 , which specifically includes the following steps:

[0151] S41, use a channel attention mechanism, a spatial attention mechanism and a linear attention mechanism to process the fusion local feature respectively to calculate corresponding multi-dimensional attention weights;

[0152] Among them, in view of the characteristics of sparse airport environment features, the channel attention mechanism is used to process the fusion local feature to enhance the key feature channel, which is represented as:

[0153]

[0154] wherein, the channel attention weight, is a real set; B represents a batch of feature maps, and C represents a channel number of the feature map; is an activation function; represents a multi-layer perception containing a bottleneck structure with a dimension reduction ratio represents a global average pooling; the fusion local feature; represents a global maximum pooling;

[0155] In view of the sparse feature of the runway line in the airport, as shown in Figure 3 , a double-branch pooling strategy is adopted, wherein captures the overall distribution, ​​The prominent features are highlighted, and a residual connection is introduced to calculate the enhanced features after channel attention:

[0156]

[0157] In the formula, represents the enhanced features after channel attention; represents the weighting on different channels by element-wise multiplication;

[0158] For the spatial positioning needs of large-area scenes in airports, a spatial attention mechanism is used to process the fused local features, which is represented as:

[0159]

[0160] In the formula, represents the spatial attention weight; represents the feature concatenation operation; ChannelAvg represents the cross-channel average operation; ChannelMax represents the cross-channel maximum operation;

[0161] For the adaptive optimization of the runway line size in the airport, the specific implementation is as follows:

[0162] 7x7 convolution kernel design: considering that the width of the runway line in the airport is usually 0.15-0.3 meters, which corresponds to 7-15 pixels in the image, a 7x7 convolution kernel can effectively capture the edge features of the runway line;

[0163] Dual-channel input: average pooling captures continuity features, and maximum pooling highlights boundary features;

[0164] A linear attention mechanism is used to process the fused local features, which is represented as:

[0165]

[0166] In the formula, represents the linear attention weight; represents the normalized exponential function, which converts the input vector into a probability distribution; represents the fully connected layer; and are the indices of the table anchor points; is the index of the row anchor point dimension, are the indices of the column anchor point dimension, used to distinguish anchor points at different positions.

[0167] wherein, The specific representation is:

[0168]

[0169] In the formula, Indicates the first One input value; Indicates the first One input value; Indicates the dimension of the input vector; The function ensures that the output value is within Within a certain range and with the sum of all output values ​​equal to 1, the attention score is converted into a probability weight, which is used to normalize the original attention scores between anchor points.

[0170] S42. Perform multi-dimensional attention fusion based on multi-dimensional attention weights to obtain fused features;

[0171] The reinforcement measures for channel attention, spatial attention, and linear attention mechanisms are respectively expressed as follows:

[0172]

[0173]

[0174]

[0175] In the formula, An indicator of the intensity of channel attention; Indicators representing the intensity of spatial attention; Indicators representing the intensity of linear attention; Indicates a batch of feature maps; Indicates the number of channels in the feature map; Indicates the height of the feature map; Indicates the width of the feature map; Indicates the number of anchor points; Indicates the batch index of the feature map; Indicates the channel dimension index; h Indicates the height dimension index; Indicates the width dimension index; Represents the channel attention matrix; Represents the spatial attention matrix; Represents a linear attention matrix;

[0176] The fusion weight vector is calculated based on the intensity indices of channel attention, spatial attention, and linear attention:

[0177]

[0178]

[0179]

[0180]

[0181] wherein, is a fusion weight vector, denotes a channel attention fusion weight vector, denotes a spatial attention fusion weight vector, denotes a linear attention fusion weight vector; denotes a 3-dimensional vector composed of the strength indicators of the three kinds of attention; denotes a transpose; is a hidden layer output feature; denotes an activation function; denotes a weight matrix of the first layer network; denotes a bias term of the first layer network; denotes a weight matrix of the second layer network; denotes a bias term of the second layer network.

[0182] Based on the fusion weight vector, the channel attention mechanism, the spatial attention mechanism and the linear attention mechanism are fused:

[0183]

[0184] wherein, denotes a fusion feature.

[0185] S5, a dual-branch prediction module is used to perform classification regression calculation on the fusion feature to predict the airport runway line confidence and the geometric parameters, which specifically includes the following steps:

[0186] S51, the classification branch in the dual-branch prediction module is used to predict the airport runway line confidence. In this embodiment, the airport environment characteristics are combined with the multi-dimensional attention in S4 to guide the prediction:

[0187] wherein, the airport runway line confidence is expressed as:

[0188]

[0189] wherein, denotes the airport runway line confidence; denotes a classification weight matrix; denotes a bias parameter of the classification branch;

[0190] S52, the regression branch in the dual-branch prediction network is used to predict the airport runway line geometric parameters:

[0191]

[0192] wherein, denotes a predicted parameter offset; denotes a regression weight matrix; a bias parameter for regression;

[0193] Parameter decoding:

[0194]

[0195] wherein, represents the final decoded airport runway line parameter; represents an anchor point, specifically a preset runway line candidate template, containing a starting coordinate, an angle, a length, and n a sampling point; by parameter decoding, the predicted offset is combined with the anchor point to obtain the final decoded airport runway line parameter .

[0196] S53, based on the classification loss, the regression loss, and the airport runway line geometric constraint loss, calculate the total loss of the double-branch prediction module;

[0197] The classification loss of the double-branch prediction module is calculated using the focal loss function, which is represented as:

[0198]

[0199] wherein, represents the classification loss; , is a focal loss parameter; is the predicted probability of the airport runway line category; = 0.25, = 2.0.

[0200] The regression loss of the double-branch prediction module is calculated using the smooth L1 loss function, which is represented as:

[0201]

[0202] wherein, represents the regression loss; SmoothL1 represents the smooth loss function; represents the predicted parameter, corresponding to the final decoded airport runway line parameter ; represents the true parameter, derived from artificial high-precision labeling;

[0203] The airport runway line geometric constraint loss in the double-branch prediction module is represented as:

[0204]

[0205] wherein:

[0206]

[0207]

[0208]

[0209] wherein, denotes the loss of the geometric constraint of the airport runway line; 、 、 is a constraint weight coefficient; is the loss of parallelism constraint; is the loss of length consistency constraint; is the loss of angle constraint; , is the angle of different airport runway lines; is the predicted length, is the standard length of the airport runway line; is the reference angle of the airport runway; = 0.1, = 0.05, =0.15 is a constraint weight coefficient.

[0210] The total loss is denoted as:

[0211]

[0212] wherein, denotes the total loss; is a classification loss weight; is a regression loss weight; is a geometric constraint loss weight; specifically, 、 =1、 =0.5.

[0213] S6, airport runway line detection based on airport runway line confidence;

[0214] This step preliminarily screens the original prediction results output by the double-branch prediction module by setting a dynamic confidence threshold, eliminates low-quality detection, and retains high-confidence candidate airport runway lines, and specifically includes the following steps:

[0215] S61, based on the confidence of the airport runway line, using attention guidance to screen the candidate airport runway lines;

[0216] Taking the confidence predicted by S5 as the main screening basis, combined with the attention weight information of S4 for enhanced screening, more accurate candidate filtering is realized;

[0217] wherein, the attention guidance screening formula is:

[0218]

[0219] wherein, is the th candidate attention enhanced screening score; is the th candidate confidence; is the attention enhanced coefficient; is the weight of the three attentions, denoted as:

[0220]

[0221] wherein, =0.4, =0.3, =0.3 is the attention fusion weight; are the corresponding channel, spatial, linear attention weights, respectively.

[0222] The screened candidate airport runway lines are:

[0223]

[0224] wherein, is the candidate set of airport runway lines, is the th candidate bounding box, from the prediction output of S5, containing the bounding box coordinates, prediction confidence, etc. = 0.3 is the screening threshold, based on statistical analysis of training data, the confidence of the real airport runway line is distributed in the range of 0.4-1.0, and the false detection is mainly distributed in the range of 0.1-0.3.

[0225] S62, based on the common effective index, the distance of the screened runway lines is measured, which includes:

[0226] Based on the characteristics of the anchor point detection method, the lane line distance measurement method based on the common effective index is adopted. For two candidate runway lines and , the common effective index or y coordinate based on it is calculated.

[0227] Effective interval determination:

[0228] Let and be the start and end indexes of the two runway lines, respectively, then the common effective interval is defined as:

[0229] Lane line distance measurement:

[0230]

[0231]

[0232] The runway line distance metric is defined as:

[0233]

[0234] wherein, denotes the distance between two runway lines and ; and denotes the minimum value of the starting points of the anchor points of two airport runway lines; denotes the maximum value of the ending points of the anchor points of two airport runway lines; denotes the horizontal coordinate of the i-th anchor point in the runway line ; and i denotes the horizontal coordinate of the i-th anchor point in the runway line . i

[0235] S63, based on the airport runway line distance metric, a non-maximum suppression algorithm is used for deduplication processing, which specifically includes:

[0236] Distance threshold determination:

[0237] The distance threshold is defined as , and when the distance between two airport runway lines satisfies: , it is determined that the two airport runway lines overlap, and a suppression mechanism is set to further deduplicate the predicted airport runway lines, which includes the following contents:

[0238] Direction consistency verification;

[0239] Considering the directionality of the runway line, the direction consistency of the two airport runway lines is calculated:

[0240]

[0241] wherein, denotes the degree of direction consistency of the two airport runway lines, is the included angle between the two airport runway lines, which is calculated by fitting a straight line:

[0242]

[0243] wherein, and are the main direction vectors of the two airport runway lines, respectively;

[0244] wherein, according to the distance threshold, the direction consistency and the confidence of the two airport runway lines, the suppression mechanism is set as:

[0245] ​​

[0246] In the formula, and respectively are the confidence of two airport runway lines; is a direction consistency threshold, usually set to 0.7, corresponding to a 45-degree angle.

[0247] When the two airport runway lines meet the suppression mechanism, the airport runway line with low confidence is suppressed, thereby realizing the deduplication processing of the predicted airport runway line.

[0248] Although the specific embodiments of the application are described in detail with reference to the accompanying drawings, it should not be understood as limiting the scope of protection of the patent. Various modifications and variations made by those skilled in the art within the scope described in the claims are still within the scope of protection of the patent.

Claims

1. An attention-guided airport runway line detection method, characterized in that, Includes the following steps: S1. Acquire airport runway image data; S2. Perform adaptive contrast enhancement processing on airport runway image data; S3. Perform multi-scale edge detection on the enhanced airport runway image data to obtain fused local features; S4. Perform multi-dimensional attention fusion on the fused local features to obtain fused features; S4 specifically includes the following sub-steps: S41. Channel attention, spatial attention, and linear attention mechanisms are used to process the fused local features respectively, and the corresponding multidimensional attention weights are calculated. In S41, the channel attention mechanism is used to process the fused local features, which is expressed as follows: In the formula, Indicates channel attention weights; for Activation function; This represents a multilayer perceptron; Indicates global average pooling; To integrate local features; Indicates global max pooling; Based on channel attention weights and by introducing residual connections, the features enhanced by channel attention are calculated: In the formula, This indicates the features after channel attention enhancement; This indicates element-wise multiplication; Spatial attention mechanism is used to process the fused local features, which is represented as follows: In the formula, Indicates spatial attention weights; Indicates feature concatenation operation; ChannelAvg Indicates cross-channel averaging operation; ChannelMax Indicates the maximum number of operations across channels; A linear attention mechanism is used to process the fused local features, which is expressed as follows: In the formula, Represents linear attention weights; This represents the normalized exponential function; Indicates a fully connected layer; and All are indexes of anchor points; S42. Perform multi-dimensional attention fusion based on multi-dimensional attention weights to obtain fused features; S42 specifically includes: The reinforcement measures for channel attention, spatial attention, and linear attention mechanisms are respectively expressed as follows: In the formula, An indicator of the intensity of channel attention; Indicators representing the intensity of spatial attention; Indicators representing the intensity of linear attention; Indicates a batch of feature maps; Indicates the number of channels in the feature map; Indicates the height of the feature map; Indicates the width of the feature map; Indicates the number of anchor points; Indicates the batch index of the feature map; Indicates the channel dimension index; h Indicates the height dimension index; Indicates the width dimension index; Represents the channel attention matrix; Represents the spatial attention matrix; Represents a linear attention matrix; The fusion weight vector is calculated based on the intensity indices of channel attention, spatial attention, and linear attention: In the formula, To fuse the weight vectors, This represents the channel attention fusion weight vector. This represents the spatial attention fusion weight vector. Represents the linear attention fusion weight vector; This represents a 3-dimensional vector composed of three intensity indices of attention. Indicates transpose; The hidden layer outputs features; Indicates the activation function; This represents the weight matrix of the first layer of the network; This represents the bias term of the first layer network; This represents the weight matrix of the second layer network; This represents the bias term of the second-layer network; Based on the fusion weight vector, the channel attention mechanism, spatial attention mechanism, and linear attention mechanism are fused: In the formula, Indicates fusion characteristics; S5. Perform classification regression calculations on the fused features to predict airport runway line confidence and geometric parameters; S6. Airport runway line detection based on airport runway line confidence level.

2. The airport runway line detection method based on attention guidance according to claim 1, characterized in that, S2 specifically includes the following steps: S21. Calculate the image contrast score and perform airport environment segmentation based on the image contrast score: In the formula, Represents the image contrast score; The standard deviation of the image gradient; The average brightness of the image. The gradient operator for an image; When C S A value less than 20 indicates an extreme environment; When 20 ≤ C S A value less than 40 indicates a medium-quality environment. When C S A value of ≥ 40 indicates a good environment; S22. Adaptive image enhancement based on image contrast score, expressed as: In the formula, This represents the image after adaptive enhancement. This represents the input airport runway image data; clahe This represents the adaptive CLAHE enhancement function; S23. Perform directional feature enhancement processing on the adaptively enhanced image: In the formula, This represents the image after directional feature enhancement. 、 Indicates the corresponding weight; Indicates the convolution operation; Indicates the detection kernel in the horizontal direction; This indicates a vertical detection kernel.

3. The airport runway line detection method based on attention guidance according to claim 2, characterized in that, S3 specifically includes the following steps: S31. Image with enhanced directional features The input is incorporated into the backbone feature extraction network, and multiple convolutional layers are used to extract feature maps of different scales. S32. Input feature maps of different scales into the feature pyramid network for multi-scale feature fusion, which is represented as follows: In the formula, Indicates the first The fusion of local feature vectors of the layers; This represents a 1×1 convolution operation; Indicates the first Feature map of the layer; Indicates an upsampling operation; Indicates the first Local features of layer fusion.

4. The airport runway line detection method based on attention guidance according to claim 3, characterized in that, In step S5, the fused features are input into the dual-branch prediction module and classification regression calculation is performed to predict the confidence level and geometric parameters of the airport runway lines. This specifically includes the following sub-steps: S51. Use the classification branch in the dual-branch prediction module to predict the confidence level of airport runway lines: In the formula, Indicates the confidence level of the airport runway line; Represented as a classification weight matrix; The bias parameter representing the classification branch; S52. Use the regression branch in the dual-branch prediction module to predict the geometric parameters of the airport runway lines: In the formula, This indicates the predicted parameter offset; Represents the regression weight matrix; These are the bias parameters for the regression; This represents the final decoded airport runway line parameters; This represents predefined anchor point parameters; S53. Calculate the loss function for airport runway line detection in the bi-branch prediction module.

5. The airport runway line detection method based on attention guidance according to claim 4, characterized in that, Specifically, S53 includes: The classification loss of the bi-branch prediction module is calculated using the focus loss function, which is expressed as follows: In the formula, Indicates classification loss; , For focus loss parameters; The predicted probability of airport runway lines; The regression loss of the bi-branch prediction module is calculated using the smoothed L1 loss function, and is expressed as follows: In the formula, Indicates regression loss; SmoothL1 Represents the smoothing loss function; These represent the predicted parameters, corresponding to the airport runway line parameters obtained from the final decoding. ; These represent actual parameters, derived from manual annotation. The geometric constraint loss of the bi-branch prediction module is expressed as: in: In the formula, This represents the geometric constraint loss of the airport runway lines; , , These are the constraint weighting coefficients; For parallelism constraint loss; The loss is due to length consistency constraints; For angle constraint loss; , For the angles of runway lines at different airports; To predict the length, This refers to the standard length of an airport runway. This is the reference angle for the airport runway; The total loss of the dual-branch prediction module is expressed as: In the formula, Indicates the total loss; For classification loss weights; For regression loss weights; The weights are for geometric constraint loss.

6. The airport runway line detection method based on attention guidance according to claim 1, characterized in that, S6 specifically includes the following steps: S61. Based on the confidence level of airport runway lines, attention-guided screening is used to select candidate airport runway lines. Attention guidance is represented as follows: In the formula, For the first Attention enhancement screening score for each candidate; For the first Each candidate confidence level; This is the attention enhancement coefficient; The weights for combining the three attention methods are expressed as follows: In the formula, 、 、 For attention fusion weights; The first Each corresponding channel, spatial, and linear attention weight; The candidate airport runway lines selected are: In the formula, For the candidate set of airport runway lines, This represents the m-th candidate detection box; The filtering threshold; S62. The distance between the filtered airport runway lines is measured based on the common valid index, and it is expressed as follows: In the formula, Indicates two airport runway lines and The distance between them; This represents the minimum value of the starting point of the anchor points of the two airport runway lines; This represents the maximum value of the endpoints of the anchor points of the two airport runway lines. Indicates airport runway lines The Middle i The x-coordinate of each anchor point; Indicates airport runway lines The Middle i The x-coordinate of each anchor point; S63. Based on the airport runway line distance metric, a non-maximum suppression algorithm is used to identify and suppress redundant airport runway lines.

7. The airport runway line detection method based on attention guidance according to claim 6, characterized in that, Specifically, S63 includes: Define the distance threshold as When the distance between two airport runway lines meets the following conditions: If the two airport runway lines are found to overlap, a suppression mechanism is set up to remove duplicates from the predicted airport runway lines. Among them, a suppression mechanism is set based on the distance threshold, directional consistency, and confidence level of the two airport runway lines: In the formula, This indicates the degree of directional consistency between two airport runway lines. and The confidence levels for the two airport runway lines are respectively; This is the threshold for directional consistency. When two airport runway lines satisfy the suppression mechanism, the airport runway line with the lower confidence level is suppressed, thereby achieving the deduplication of the predicted airport runway lines.

Images