An aircraft type identification method and an aircraft parking guidance method

Abstract

This invention provides an aircraft type identification and berth guidance method, comprising: acquiring raw optical image information of a taxiing aircraft waiting to berth through an image acquisition unit; inputting the raw optical image information into a feature extraction network to extract the most effective features for classification from the input raw data; using a region nomination network to output candidate regions of the most effective features for classification; inputting the candidate region images into an image classification network to output aircraft type classification. This invention can effectively, accurately, and quickly identify the aircraft type preparing to park on the apron.

Description

Technical Field

[0001] This invention relates to the field of civil aviation, and in particular to an aircraft type identification method and an aircraft parking guidance method. Background Technology

[0002] Aircraft guidance at tarmac berths is divided into two methods: manual guidance and automated guidance. Manual guidance wastes manpower and resources, and human negligence can lead to serious accidents. Automated guidance improves the efficiency and safety of aircraft berth guidance and accelerates the informatization of airports. With the expansion of airports and fleets and the improvement of automation levels, automated guidance has become the main method of aircraft berth guidance.

[0003] An Airport Docking Auto-guide System (ADGS) is an automated control system that provides pilots with information on the distance between the aircraft and its parking position, its taxiing speed, and whether the aircraft has deviated from the taxiway centerline during the journey from the taxiway to its parking position on the apron. This allows the pilots to control the aircraft to park accurately in its designated parking position. From a principle perspective, there are currently two methods for achieving aircraft docking guidance: one based on image recognition technology, and the other based on laser ranging principles.

[0004] Different aircraft models correspond to different stop line positions on the apron, making aircraft model identification one of the key tasks of automated parking guidance systems. However, currently, neither image recognition nor laser rangefinder-based parking guidance systems can effectively, accurately, and quickly identify the aircraft model preparing to park on the apron.

[0005] This invention relates to an image-based deep learning-based aircraft identification model for aircraft parking guidance. Trained on a large amount of data, it achieves high stability in aircraft identification and detection, thus solving a key task in parking guidance systems. Summary of the Invention

[0006] To address the problem of the inability to effectively, accurately, and quickly identify the aircraft type preparing to park on the apron, this invention provides an aircraft type identification method and an aircraft parking guidance method using the aircraft type identification method.

[0007] On one hand, the present invention provides an aircraft type identification method, characterized in that: S1, the original optical image information of a taxiing aircraft waiting to be parked is acquired by an image acquisition unit; S2, the original optical image information is input into a feature extraction network to extract the most effective features for classification from the input original data, the effective features including one or more of the following: engine, cabin door, landing gear, and wing; S3, the original optical image information and the most effective features are input into a region nomination network to output the candidate region of the most effective features for classification; S4, the candidate region image is input into an image classification network to output the aircraft type classification.

[0008] Furthermore, before inputting the original optical image information into the feature extraction network, the original optical image is resized to a uniform preset size.

[0009] Further, the feature extraction network consists of a first residual network, a second residual network, a third residual network, a fourth residual network, a fifth residual network, and a first activation layer connected in sequence; the first residual network consists of a 3*3 convolutional layer and a ReLU activation layer connected in sequence; the second residual network consists of a 1*1, 3*3, and 5*5 convolutional layers and a 5*5 pooling layer connected in sequence; the third residual network consists of a 3*3, 5*5 convolutional layer and a 5*5 pooling layer connected in sequence; the fourth residual network consists of a 1*1, 3*3, 5*5, and 7*7 convolutional layers and a 5*5 pooling layer connected in sequence; the fifth residual network consists of a 1*1, 3*3, and 5*5 convolutional layer and a 5*5 pooling layer connected in sequence; and the first activation layer is activated by ReLU.

[0010] Furthermore, the specific structure of the region nomination network is as follows: the sixth residual network is connected to the first convolutional layer and the second convolutional layer respectively; the first convolutional layer is connected in sequence to the first adjustment dimension module, the first IOU calculation module, the second adjustment dimension module, and the first region nomination module; the second convolutional layer is connected to the first adjustment coordinate module and the second adjustment coordinate module respectively; the output of the first adjustment coordinate module is input to the first region nomination module; the first region nomination module is connected in sequence to the third adjustment dimension module, the second IOU calculation module, the fourth adjustment dimension module, and the second region nomination module; the output of the second adjustment coordinate module is connected to the second region nomination module.

[0011] Furthermore, the sixth residual network consists of a 3*3 convolution and a ReLU activation layer; the first convolutional layer is a 3*3 convolution and the second convolutional layer is a 5*5 convolution; the first, second, third, and fourth dimension adjustment modules are all composed of 2*2 max pooling layers; the first and second IOU calculation modules calculate the IOU value after dimension adjustment, retaining only the nominated regions with an IOU greater than a preset value; the first coordinate adjustment module adjusts the region coordinates of the second convolutional layer to the coordinates output by the first region nomination module; the second coordinate adjustment module adjusts the region coordinates of the second convolutional layer to the coordinates output by the second region nomination module; the second region nomination module outputs the effective feature candidate regions.

[0012] Furthermore, the image classification network is specifically composed of a 5*5 pooling layer, a first fully connected layer, a 3*3 convolutional layer, a second fully connected layer, a 5*5 convolutional layer, and a third fully connected layer connected sequentially.

[0013] Furthermore, after classifying the output aircraft models, the output image is resized to a second preset size, and the aircraft model is labeled in the image.

[0014] Furthermore, the original apron image, aircraft image, and aircraft component image are respectively fed into the image classification model for training, resulting in three classification sub-networks. The original apron image classification sub-network learns the overall features of the image, the aircraft image classification sub-network learns the overall features of the aircraft, and the aircraft component image classification sub-network learns the detailed features of the aircraft components. By combining the advantages of the different levels of features of the three classification sub-networks through the classification module, the acquisition of differential information is increased, and the final aircraft model classification result is obtained.

[0015] On the other hand, this application also provides an aircraft parking guidance method, characterized by the following steps: after the airport tower detects an aircraft, the airport tower computer system issues an instruction to the aircraft to enter the corresponding parking area; the aforementioned method is used to identify the aircraft type; based on the aircraft type identification result, the parking information corresponding to the aircraft type identification result is read from the airport tower computer system; the parking information is sent to the aircraft; and the airport tower computer system controls the parking area display screen to guide the aircraft into the parking space.

[0016] The technical solution provided by this invention, through the design of a special feature extraction network, a region nomination network, and a classification network, can effectively, accurately, and quickly identify the aircraft type preparing to park on the apron. Attached Figure Description

[0017] Figure 1 This is a network structure diagram used in the model identification method of the present invention;

[0018] Figure 2 This is a schematic diagram of the model identification result of the present invention;

[0019] Figure 3 This describes the process for guiding aircraft to park at designated berths. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments provided by this invention without inventive effort are within the scope of protection of this invention.

[0021] Obviously, the accompanying drawings described below are merely some examples or embodiments of the present invention. Those skilled in the art can apply the present invention to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this invention, modifications to design, manufacturing, or production based on the technical content disclosed in this invention are merely conventional technical means and should not be construed as insufficient disclosure of the present invention.

[0022] In this invention, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this invention may be combined with other embodiments without conflict.

[0023] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "a," "an," "an," "the," and similar words used in this invention do not indicate quantity limitation and may indicate singular or plural. The terms "comprising," "including," "having," and any variations thereof used in this invention are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms "connected," "linked," "coupled," and similar words used in this invention are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "A plurality" in this invention refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships may exist; for example, "A and / or B" can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects have an "or" relationship. The terms "first," "second," and "third" used in this invention are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0024] like Figure 1 As shown, in one aspect, the present invention discloses a model identification method, comprising the following steps:

[0025] S1 acquires raw optical image information of the taxiing aircraft waiting to be parked through the image acquisition unit.

[0026] In order to classify aircraft types, it is necessary to first collect raw optical image information. This can be done by setting up cameras in areas such as airport runways and parking berths to photograph the aircraft. This application does not limit the specific method of acquisition, as long as aircraft images can be obtained.

[0027] S2, input the raw optical image information into the feature extraction network, and extract the most effective features for classification from the input raw data. The effective features include one or more of the following: engine, cabin door, landing gear and wing.

[0028] Since the differences between different aircraft models are very small, the error would be very large if the original images were used directly for classification. In order to reduce the error, this application first needs to extract the difference features between different aircraft models. Specifically, features such as engines, cabin doors, landing gear, and wings can be used as identification features between different aircraft models.

[0029] Furthermore, for convenience, the original optical image is resized before being input into the feature extraction network to a uniform preset size; the specific size is not specifically limited in this application, as long as it facilitates subsequent processing, and those skilled in the art can choose according to actual needs.

[0030] Specifically, through extensive experimental comparisons, this invention provides a superior feature extraction network model structure. The feature extraction network consists of a first residual network, a second residual network, a third residual network, a fourth residual network, a fifth residual network, and a first activation layer, connected sequentially. The first residual network consists of a 3x3 convolutional layer and a ReLU activation layer connected sequentially. The second residual network consists of a 1x1, 3x3, and 5x5 convolutional layers and a 5x5 pooling layer connected sequentially. The third residual network consists of a 3x3, 5x5 convolutional layer and a 5x5 pooling layer connected sequentially. The fourth residual network consists of a 1x1, 3x3, 5x5, and 7x7 convolutional layers and a 5x5 pooling layer connected sequentially. The fifth residual network consists of a 1x1, 3x3, and 5x5 convolutional layer and a 5x5 pooling layer connected sequentially. The first activation layer is activated by ReLU.

[0031] Due to the differences between different models, in order to reduce the gradient vanishing problem when training the feature extraction network, this invention first uses ReLU activation, and then uses different combinations of convolutional layers to highlight details in different dimensions. Based on the pre-labeled effective feature region samples, the effective features are extracted for use in subsequent steps.

[0032] S3, input the original optical image information and the most effective feature into the region nomination network, and output the candidate region of the most effective feature for classification;

[0033] To accelerate the classification process, this invention further employs a region nomination network to nominate regions of interest in the original image. The region nomination network can evaluate image regions with effective features and extract the regions containing effective features. The subsequent classification network only needs to further classify these extracted regions, thus greatly improving the classification speed.

[0034] Specifically, based on extensive experimental verification and comparison, this embodiment presents a superior structure for a region nomination network. The specific structure of the region nomination network is as follows: a sixth residual network is connected to the first convolutional layer and the second convolutional layer; the first convolutional layer is sequentially connected to the first adjustment dimension module, the first IOU calculation module, the second adjustment dimension module, and the first region nomination module; the second convolutional layer is connected to the first adjustment coordinate module and the second adjustment coordinate module; the output of the first adjustment coordinate module is input to the first region nomination module; the first region nomination module is sequentially connected to the third adjustment dimension module, the second IOU calculation module, the fourth adjustment dimension module, and the second region nomination module; the output of the second adjustment coordinate module is connected to the second region nomination module.

[0035] Furthermore, the sixth residual network consists of a 3*3 convolution and a ReLU activation layer; the first convolutional layer is a 3*3 convolution and the second convolutional layer is a 7*7 convolution; the first, second, third, and fourth dimension adjustment modules are all composed of 2*2 max pooling layers; the first and second IOU calculation modules calculate the IOU value after dimension adjustment, retaining only the nominated regions with an IOU greater than a preset value; the first coordinate adjustment module adjusts the region coordinates of the second convolutional layer to the coordinates output by the first region nomination module; the second coordinate adjustment module adjusts the region coordinates of the second convolutional layer to the coordinates output by the second region nomination module; the second region nomination module outputs the effective feature candidate regions.

[0036] In the region nomination network provided in this invention, two different scales are used: a first convolutional layer of 3*3 and a second convolutional layer of 7*7 as the basis for feature recognition. By recognizing at two scales sequentially, excessive details are avoided from being included in the recognition results, while effective features are highlighted. Furthermore, through several modules that adjust dimensions (2*2 max pooling layers), the data is reduced by two dimensions each time, further filtering out unnecessary details. Candidate regions with too little overlap with effective features are eliminated through two IOU calculations, avoiding excessive invalid calculations. After the effective features in the image are identified, the specific feature coordinates are marked by a coordinate adjustment module for subsequent processing by the classification module.

[0037] S4, input the candidate region image into the image classification network, and output the aircraft type classification.

[0038] The image classification network is specifically composed of a 5*5 pooling layer, a first fully connected layer, a 3*3 convolutional layer, a second fully connected layer, a 5*5 convolutional layer, and a third fully connected layer connected sequentially.

[0039] Furthermore, after classifying the output aircraft types, the output image is resized to a second preset size, and the aircraft type is labeled in the image. The recognition result is as follows: Figure 2 As shown.

[0040] Furthermore, to further improve the accuracy of recognition, the original apron image, aircraft image, and aircraft component image are trained in the image classification model to obtain three classification sub-networks. The original apron image classification sub-network learns the overall features of the image, the aircraft image classification sub-network learns the overall features of the aircraft, and the aircraft component image classification sub-network learns the detailed features of the aircraft components. By combining the advantages of the different levels of features of the three classification sub-networks through the classification module, the acquisition of differential information is increased, and the final aircraft model classification result is obtained.

[0041] like Figure 3 As shown, in another embodiment, an aircraft parking guidance method is also disclosed.

[0042] Once the airport tower detects an aircraft, its computer system sends a command to the aircraft to move into the designated parking area.

[0043] The aircraft type is identified using the method described above.

[0044] Based on the aircraft type identification result, the corresponding berth information is read from the control tower computer system;

[0045] Send the berth information to the aircraft.

[0046] The airport tower computer system controls the display screen in the parking area to guide the aircraft into the parking space.

[0047] In this invention, the term "multiple" refers to two or more unless otherwise explicitly defined. The terms "install," "connect," "link," and "fix" should be interpreted broadly. For example, "connect" can be a fixed connection, a detachable connection, or an integral connection; "link" can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0048] In the description of this specification, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0049] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A model identification method, characterized in that: S1, acquiring original optical image information of a taxiing aircraft at a parking position through an image acquisition unit; S2, inputting the original optical image information into a feature extraction network to extract the most effective features for classification from the input original data, the effective features including one or a combination of more than one of an engine, a cabin door, a landing gear and a wing; S3, inputting the original optical image information and the most effective features into a region nomination network to output the most effective feature candidate regions for classification; and S4, inputting the candidate region image into an image classification network to output an aircraft model classification; the feature extraction network is composed of a first residual network, a second residual network, a third residual network, a fourth residual network, a fifth residual network and a first activation layer connected in sequence; the first residual network is composed of a 3*3 convolution layer and a ReLU activation layer connected in sequence; the second residual network is composed of a 1*1, 3*3, 5*5 convolution layer and a 5*5 pooling layer connected in sequence; the third residual network is composed of a 3*3, 5*5 convolution layer and a 5*5 pooling layer connected in sequence; the fourth residual network is composed of a 1*1, 3*3, 5*5, 7*7 convolution layer and a 5*5 pooling layer connected in sequence; the fifth residual network is composed of a 1*1, 3*3, 5*5 convolution layer and a 5*5 pooling layer connected in sequence; and the first activation layer is a ReLU activation; the region nomination network has the following specific structure: a sixth residual network is connected with a first convolution layer and a second convolution layer; the first convolution layer is connected with a first dimension adjustment module, a first IOU calculation module, a second dimension adjustment module and a first region nomination module in sequence; the second convolution layer is connected with a first coordinate adjustment module and a second coordinate adjustment module; the output of the first coordinate adjustment module is input into the first region nomination module; the first region nomination module is connected with a third dimension adjustment module, a second IOU calculation module, a fourth dimension adjustment module and a second region nomination module in sequence; and the output of the second coordinate adjustment module is connected to the second region nomination module; the sixth residual network is composed of a 3*3 convolution and a ReLU activation layer; the first convolution layer is a 3*3 convolution, and the second convolution layer is a 5*5 convolution; the first dimension adjustment module, the second dimension adjustment module, the third dimension adjustment module and the fourth dimension adjustment module are all composed of a 2*2 maximum value pooling layer; the first IOU calculation module and the second IOU calculation module calculate the IOU values after dimension adjustment, and only the nominated regions with an IOU greater than a preset value are retained; the first coordinate adjustment module adjusts the region coordinates of the second convolution layer to the coordinates output by the first region nomination module; the second coordinate adjustment module adjusts the region coordinates of the second convolution layer to the coordinates output by the second region nomination module; and the second region nomination module outputs the effective feature candidate regions. ​ ​ ​ ​ ​ ​ 2. The machine type identification method of claim 1, wherein: Before inputting the original optical image information into the feature extraction network, the original optical image is resized to a uniform preset size.

3. The machine type identification method of claim 1, wherein: The image classification network is specifically connected by a 5*5 pooling, a first full connection layer, a 3*3 convolution, a second full connection layer, a 5*5 convolution and a third full connection layer in sequence.

4. The machine type identification method of claim 1, wherein: After outputting the aircraft model classification, the output image is resized to a second preset size, and the aircraft model is labeled in the image.

5. The machine type identification method of claim 1, wherein: The original apron image, the aircraft image and the aircraft component image are respectively put into the image classification model for training to obtain three classification sub-networks; the original apron image classification sub-network learns the overall features of the image, the aircraft image classification sub-network learns the overall features of the aircraft, and the aircraft component image classification sub-network learns the detailed features of the aircraft component; through the classification module, the advantages of different levels of features of the three classification sub-networks are combined to increase the acquisition of difference information, and the final aircraft model classification result is obtained.

6. An aircraft docking guidance method, characterized in that The method comprises the following steps: After the airport tower detects the aircraft, the airport tower computer system sends an instruction to the aircraft to enter the corresponding parking area; The aircraft is identified by the method of any one of claims 1-5; the parking information corresponding to the aircraft model identification result is read from the tower computer system according to the aircraft model identification result; The parking information is sent to the aircraft; The airport tower computer system controls the parking area display screen to guide the aircraft to enter the parking space.

Images