Airplane runway segmentation method based on attention mechanism

Abstract

The invention belongs to the technical field of image processing and deep learning, and discloses a runway segmentation method based on an attention mechanism, and the method comprises the steps: S1, carrying out the marking of a runway and an obstacle through employing a manually-marked runway image, and constructing a runway image semantic segmentation data set; s2, training the model by using the airline runway image semantic segmentation data set to obtain an airline runway image segmentation model; and S3, carrying out online deployment and real-time model prediction based on the airplane runway image semantic segmentation model. According to the method, the attention mechanism is introduced, so that the model can more accurately position and identify the runway area, and high-precision runway segmentation is realized. The method has the advantages of being high in robustness, good in real-time performance and the like, runway images under the conditions of different illumination, shadows, noise and the like can be processed, and effective technical support is provided for automatic management of airplane runways.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and deep learning, and specifically relates to an aircraft runway segmentation method based on an attention mechanism. Background Art

[0002] With the development of the aviation industry, the automated management and safety guarantee of aircraft runways have become crucial. The segmentation of aircraft runways is a key step in automated management, and its accuracy directly affects the effectiveness of subsequent runway status analysis, flight safety assessment, etc. Traditional runway segmentation methods are susceptible to factors such as illumination, shadows, and noise when processing complex runway images, resulting in low segmentation accuracy. In recent years, deep learning technology has made remarkable progress in the field of image processing, and the attention mechanism, as a technology that can enhance the model's attention to key information, has been widely applied to various image processing tasks. Therefore, the present invention proposes an aircraft runway segmentation method based on an attention mechanism to improve the accuracy and robustness of runway segmentation. Summary of the Invention

[0003] To overcome the above technical problems, the present invention provides an aircraft runway segmentation method based on an attention mechanism, which enhances the network's feature learning of the runway area by introducing an attention module, thereby improving the accuracy and robustness of segmentation.

[0004] The present invention adopts the following technical solutions:

[0005] An aircraft runway segmentation method based on an attention mechanism, comprising:

[0006] S1. Using manually labeled aircraft runway images, performing annotations on aircraft runways and obstacles, and constructing an aircraft runway image semantic segmentation dataset;

[0007] The model is improved based on the semantic segmentation network DeepLabv3+. The model includes multiple convolutional layers, activation layers, and pooling layers. An attention mechanism module is introduced on the basis of DeepLabv3+. The attention mechanism module is composed of multiple fully connected layers and non-linear activation functions, and is used to enhance the model's attention to the runway area; the introduced attention mechanism modules are a position attention mechanism and a channel attention mechanism;

[0008] S2. Using the aircraft runway image semantic segmentation dataset to train the model to obtain an aircraft runway image segmentation model;

[0009] S3. Based on the aircraft runway image semantic segmentation model, perform online deployment and real-time model prediction.

[0010] Preferably, the steps executed by the aircraft runway semantic segmentation model training module include:

[0011] (1) Dataset construction:

[0012] Collect aircraft runway images under different airports, weather, and lighting conditions to construct a dataset containing at least 1000 images; annotate the dataset to clarify areas such as runways, marking lines, and obstacles for subsequent model training and validation;

[0013] (2) Image preprocessing:

[0014] Normalize the size of the acquired images first;

[0015] (3) Aircraft runway semantic segmentation model;

[0016] (4) Model training:

[0017] Use the above dataset to train the model. During the training process, adopt the cross-entropy loss function and the Adam optimizer, and regularize the model during the training process to prevent overfitting; judge whether the specified number of iterations is met; when the training of the model does not reach the set number of iterations, continue to input images to adjust the model weight hyperparameters, and stop training if the set number of iteration rounds is reached; output the best model weight parameters.

[0018] Preferably, the execution steps of the aircraft runway semantic segmentation model prediction module include:

[0019] (1) Obtain camera images:

[0020] Obtain the real-time images of the camera installed on the aircraft in real time;

[0021] (2) Image preprocessing:

[0022] Normalize the size of the acquired images first;

[0023] (3) Aircraft runway semantic segmentation model:

[0024] Load the best weight parameters into the model to predict the semantic segmentation results of the input images.

[0025] Preferably, deploy the aircraft runway semantic segmentation model to the algorithm computing hardware platform on the aircraft; output the prediction results of the aircraft runway and obstacles: after the input images are inferred by the network model, output the results predicted by the aircraft runway semantic segmentation model.

[0026] Compared with the prior art, the beneficial effects of the present invention are:

[0027] By introducing the attention mechanism, this method enables the model to more accurately locate and identify the runway area, thus achieving high-precision runway segmentation;

[0028] Automatically identify and segment the runway area in aircraft runway images through a deep learning model to support the automated management and safety guarantee of aircraft runways;

[0029] This method has the advantages of strong robustness and good real-time performance, and can process runway images under different lighting, shadow, noise and other conditions, providing effective technical support for the automated management of aircraft runways. Brief Description of the Drawings

[0030] Figure 1 It is the system architecture diagram of the present invention.

[0031] Figure 2 It is the structural schematic diagram of the position attention mechanism module.

[0032] Figure 3 It is the structural schematic diagram of the channel attention mechanism module.

[0033] Figure 4 It is the network structure schematic diagram of the semantic segmentation network.

[0034] Figure 5 It is the visualization display diagram of the segmentation result. Detailed Embodiment

[0035] The embodiments of the present invention are described in detail below. The examples of the embodiments are shown in the drawings. Unless otherwise specified, the raw materials and equipment used can be purchased from the market or are commonly used in the art. The methods in the embodiments, unless otherwise specified, are conventional methods in the art. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation of the present invention.

[0036] The aircraft runway segmentation method based on the attention mechanism includes the following steps:

[0037] S1. Use the manually labeled aircraft runway images to perform annotations on the aircraft runway and obstacles, and construct an aircraft runway image semantic segmentation dataset;

[0038] S2. Use the aircraft runway image semantic segmentation dataset to train the model to obtain an aircraft runway image segmentation model;

[0039] S3. Perform online deployment based on the aircraft runway image semantic segmentation model and perform real-time model prediction.

[0040] Among them,

[0041] The steps executed by the aircraft runway semantic segmentation model training module include:

[0042] Dataset construction:

[0043] Collect aircraft runway images under different airports, weather, and lighting conditions to construct a dataset containing at least 1000 images.

[0044] Annotate the dataset to clarify areas such as runways, marking lines, and obstacles for subsequent model training and validation.

[0045] Image preprocessing:

[0046] Normalize the size of the acquired images first; uniformly transform the sizes of all acquired images to 2048 * 1024 pixels.

[0047] Aircraft runway semantic segmentation model:

[0048] The model is improved based on the semantic segmentation network DeepLabv3+. The model includes multiple convolutional layers, activation layers, and pooling layers.

[0049] Introduce an attention mechanism module based on DeepLabv3+. This module consists of multiple fully connected layers and non-linear activation functions, and is used to enhance the model's attention to the runway area.

[0050] The introduced attention mechanism modules are the position attention mechanism and the channel attention mechanism;

[0051] Position attention module: It can simulate rich global feature context information, so as to enhance the same features at different positions and improve the semantic segmentation ability. The structure diagram of the position attention module is as Figure 2 shown:

[0052] Through the backbone network, the local feature A ∈ R C×H×W matrix can be obtained. The feature A is convolved to obtain matrices B, C, and D, {B, C, D} ∈ R C×H×W , then matrices B and C are reshaped into R C×N , where N = H × W. Matrices B and C are multiplied, and finally the softmax layer is used to calculate the position attention S ∈ R N×N .

[0053]

[0054] Furthermore, matrices A and D are multiplied, and finally, by multiplying by a scaling factor α and summing with matrix A, the final output E is obtained, E ∈ R C×H×W .

[0055] Channel attention mechanism module: The high-level semantic feature maps of different channels extracted by the semantic segmentation model are predictions of a specific type, and there are specific connections between different types of semantics, as Figure 3 shown:

[0056] The channel attention module directly passes through \(A\in\mathbb{R}\) C×H×W to calculate the channel attention map \(X\in\mathbb{R}\) C×C . First, the shape of the feature matrix \(A\) is changed to \(\mathbb{R}\) C×N , where \(N = H\times W\). Then, the matrix multiplication of \(A\) and the transpose of \(A\) is performed and \(X\) is obtained through the softmax layer;

[0057] Furthermore, the matrices of \(X\) and \(A\) are multiplied, and finally, by multiplying a scaling factor \(\beta\) and summing with the matrix \(A\), the final output \(E\) is obtained, \(E\in\mathbb{R}\) C×H×W .

[0058] Finally, the two attention mechanism modules are added to the DeepLabv3+ model. The feature map extracted by the backbone network is subjected to a \(3\times3\) convolution operation, and then it is respectively sent into the channel attention module and the position attention module for processing, and the processed feature maps are summed, as Figure 4 shown:

[0059] In the figure, the lower branch uses the ASPP operation in DeepLabv3+. First, the feature map is sent into ASPP for processing, and then the feature map processed by ASPP is fused with the feature map processed by the dual attention module. Finally, the fused feature map is reduced to 256 channels. The network decoding module follows the operation of the DeepLabv3+ decoding module, and finally the image segmentation map is obtained.

[0060] Model training:

[0061] The above dataset is used to train the model. During the training process, the cross-entropy loss function and the Adam optimizer are adopted. Regularization is performed on the model during the training process to prevent overfitting.

[0062] Judge whether the specified number of iterations is satisfied:

[0063] The specified number of iterations is 400 epochs; when the training of the model does not reach the set number of iterations, continue to input images to adjust the hyperparameters of the model weights. If the set number of iteration rounds is reached, the training stops.

[0064] Output the best model weight parameters:

[0065] After the model stops training, the best weight parameters will be selected according to the calculation results of the recognition accuracy rate of each round of training, and the weight parameters will be saved to a file;

[0066] The trained aircraft runway semantic segmentation model can be used for online aircraft runway semantic segmentation recognition, obtain camera images, and predict the aircraft runway semantic segmentation results.

[0067] The steps executed by the prediction module of the aircraft runway semantic segmentation model include:

[0068] Obtain camera image:

[0069] Obtain the real-time image of the camera installed on the aircraft in real time;

[0070] Image preprocessing:

[0071] First, normalize the size of the obtained image; in this embodiment, the sizes of all obtained images are uniformly transformed into 2048*1024 pixel sizes.

[0072] Aircraft runway semantic segmentation model:

[0073] The model adopted in this step is the same as the model in the training stage described above. However, the weight parameters of the model do not need to be adjusted again through training, but by loading the best weight parameters output above into the model to predict the semantic segmentation result of the input image.

[0074] Model deployment:

[0075] Deploy the aircraft runway semantic segmentation model to the algorithm computing hardware platform on the aircraft;

[0076] Output the prediction results of the aircraft runway and obstacles:

[0077] After the input image is inferred through the network model, output the prediction results of the aircraft runway semantic segmentation model, as Figure 5 shown.

[0078] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and deformations can be made to the above embodiments without departing from the principles and purposes of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. An aircraft runway segmentation method based on an attention mechanism, characterized in that, Including: S1. Using manually labeled airplane runway images, perform runway and obstacle annotation to construct an airplane runway image semantic segmentation dataset; The model is improved based on the semantic segmentation network DeepLabv3+. The model includes multiple convolutional layers, activation layers, and pooling layers. An attention mechanism module is introduced on the basis of DeepLabv3+. The attention mechanism module consists of multiple fully connected layers and non-linear activation functions, which are used to enhance the model's attention to the runway area; the introduced attention mechanism modules are the position attention mechanism and the channel attention mechanism respectively; S2. Use the airplane runway image semantic segmentation dataset to train the model to obtain an airplane runway image segmentation model; S3. Based on the airplane runway image semantic segmentation model, perform online deployment and real-time model prediction.

2. A runway segmentation method based on the attention mechanism according to claim 1, characterized in that The steps executed by the airplane runway semantic segmentation model training module include: (1) Dataset construction: Collect airplane runway images under different airports, different weather, and lighting conditions to construct a dataset containing at least 1000 images; annotate the dataset to clarify areas such as runways, marking lines, and obstacles for subsequent model training and verification; (2) Image preprocessing: First perform size normalization on the obtained images; (3) Airplane runway semantic segmentation model; (4) Model training: Use the above dataset to train the model. During the training process, use the cross-entropy loss function and the Adam optimizer, and regularize the model during the training process to prevent overfitting; determine whether the specified number of iterations is met; when the training of the model does not reach the set number of iterations, continue to input images to adjust the model weight hyperparameters, and stop training if the set number of iteration rounds is reached; output the best model weight parameters.

3. A method for aircraft runway segmentation based on an attention mechanism according to claim 2, characterized in that The steps executed by the airplane runway semantic segmentation model prediction module include: (1) Obtain camera images: Real-time obtain the real-time images of the camera installed on the airplane; (2) Image preprocessing: First perform size normalization on the obtained images; (3) Airplane runway semantic segmentation model: By loading the best weight parameters into the model, it is used to predict the semantic segmentation result of the input image.

4. According to an attention mechanism-based airplane runway segmentation method as claimed in claim 3, wherein Deploy the airplane runway semantic segmentation model to the algorithm computing hardware platform on the airplane; output the prediction results of the airplane runway and obstacles: after the input image is inferred by the network model, output the prediction results of the airplane runway semantic segmentation model.

Images