Method and system for intelligent identification and positioning of aircraft parking in wide-area scenario

Abstract

The present application pertains to the field of aircraft model identification and positioning, and particularly relates to a method and system for intelligent identification and positioning of aircraft parking in a wide-area scenario. The method comprises: constructing an aircraft model image database; constructing an initial aircraft model identification and positioning model, and training the initial aircraft model identification and positioning model by means of the aircraft model image database, and obtaining a first aircraft model identification and positioning model; performing model compression on the first aircraft model identification and positioning model, and obtaining a second aircraft model identification and positioning model; and acquiring an aircraft parking image, using the second aircraft model identification and positioning model to perform classification and position identification on a target in the aircraft parking image, and labeling a target type and target location information in the aircraft parking image, and obtaining a labeled aircraft parking image. According to the present application, rapid and reliable aircraft model identification and positioning of parked aircrafts in aircraft parking images can be achieved, and the detection effects of real-time processing and reliable identification on multiple video streams can be attained, thus greatly improving the level of safety and the production management efficiency of a parking apron.

Description

A method and system for intelligent identification and positioning of aircraft parking in wide-area scenarios Technical Field

[0001] The present application relates to the field of aircraft model identification and positioning, and in particular to a method and system for intelligent identification and positioning of parked aircraft in a wide-area scenario. Background Art

[0002] Intelligent aviation manufacturing has become a trend in the industry, and the parking and scheduling of aircraft on the apron has become a key factor affecting test efficiency. Traditional daily management of parked aircraft relies on manual monitoring and scheduling, which is inefficient and poses safety risks.

[0003] Therefore, it is desired to have a technical solution to overcome or at least alleviate at least one of the above-mentioned deficiencies of the prior art.

[0004] Summary of the Invention

[0005] The purpose of this application is to provide a method and system for intelligent identification and positioning of parked aircraft in a wide-area scenario, so as to solve the problems of low efficiency and safety hazards in the management of parked aircraft in the existing technology.

[0006] The technical solution of this application is:

[0007] The first aspect of the present application provides a method for intelligent identification and positioning of parked aircraft in a wide-area scenario, comprising:

[0008] Step 1: Build a model image database;

[0009] Step 2: constructing an initial aircraft model recognition and positioning model, and training the initial aircraft model recognition and positioning model using the aircraft model image database to obtain a first aircraft model recognition and positioning model;

[0010] Step 3: compressing the first aircraft type identification and positioning model to obtain a second aircraft type identification and positioning model;

[0011] Step 4: Acquire an image of a parked aircraft, classify and locate targets in the image of the parked aircraft using the second aircraft type recognition and positioning model, mark the target type and target location information in the image of the parked aircraft, and obtain a marked image of the parked aircraft.

[0012] In at least one embodiment of the present application, in step 1, constructing a model image database includes:

[0013] Collecting a predetermined number of images of parked aircraft;

[0014] Classifying and positioning the objects in the parked aircraft image to generate a label document, wherein each parked aircraft image corresponds to a label document;

[0015] An aircraft model image database is generated based on the parked aircraft image and the label document.

[0016] In at least one embodiment of the present application, the label document includes the name, target type, and target location information of the aircraft parking image.

[0017] In at least one embodiment of the present application, in step 2, the initial model identification and positioning model adopts a convolutional neural network YOLOv8 architecture, and a multi-scale feature fusion module is provided in the initial model identification and positioning model, wherein:

[0018] The multi-scale feature fusion module processes data in the following way:

[0019] Given input feature F in , input feature F in After one convolution operation, the feature F1 is obtained: F1 = F in *W in ;

[0020] Feature F1 performs four-way feature operations of different scales in parallel. The calculation formulas for each feature extracted are:

[0021] The obtained four-way features are fused to obtain the output feature F out The calculation formula is:

[0022] Among them, F in is the input feature, F out are output features, F1, F 01 、F 11 、F 12 、F 21 、F 22 、F 31 、F 32 are the corresponding intermediate feature quantities, W in 、W 11 、W 12 、W 21 、W 22 、W 31 、W 32 、W out are the corresponding weights respectively, * represents the convolution operation, It represents the feature point-by-point element addition operation, and Concat represents the cascade function.

[0023] In at least one embodiment of the present application, in step 2, before training the initial model recognition and positioning model through the model image database, a data enhancement algorithm is used to expand the data in the model image database, and the data enhancement algorithm processes the data in a manner including hue transformation and affine transformation.

[0024] In at least one embodiment of the present application, in step 2, when the initial model recognition and positioning model is trained using the model image database, the convolutional neural network weight parameters of the initial model recognition and positioning model are corrected using a stochastic gradient descent method, so that the loss value of the initial model recognition and positioning model decreases and converges to a stable value.

[0025] In at least one embodiment of the present application, in step three, compressing the first aircraft type identification and positioning model to obtain a second aircraft type identification and positioning model includes:

[0026] A channel score model is added to the first aircraft model identification and positioning model. The channel score tensor S in the channel score model is:

[0027] Among them, σ1 is the ReLU activation function, σ2 is the sigmoid activation function, W1 and W2 are the corresponding fully connected layer weights, S1, S2…S C are the corresponding channel score tensors respectively;

[0028] The channel score tensors are sorted by size, and several channel score tensors with channel scores lower than the threshold are introduced into the model training loss function. The channel score penalty term of the model training loss function is:

[0029] Among them, C r is the total number of feature channels filtered from each layer of the model, ||S c ||1 is the L1 norm of the score tensor of the cth channel in each layer;

[0030] The channel score model is removed from the first aircraft type identification and positioning model, and several redundant parameters with low channel scores in the first aircraft type identification and positioning model are trimmed to obtain a second aircraft type identification and positioning model.

[0031] In at least one embodiment of the present application, the step 5 is also included.

[0032] Get the parking type and location information of each parking stand in the apron;

[0033] Determining whether the target position information in the aircraft parking image matches the parking stand position information, and determining whether the target type in the aircraft parking image matches the parking stand type, and counting the matching results;

[0034] The matching results and marked aircraft parking images are stored, managed and displayed.

[0035] The second aspect of the present application provides a system for intelligent identification and positioning of parked aircraft in a wide-area scenario, based on the above-mentioned method for intelligent identification and positioning of parked aircraft in a wide-area scenario, comprising: a high-definition network surveillance camera, a computing power analysis server, a network switch, and an image management system, wherein:

[0036] The high-definition network surveillance camera is used to collect images of parked aircraft and transmit the collected data to the computing power analysis server through a network switch;

[0037] The computing power analysis server includes an image enhancement module, a model compression module, and an aircraft type identification and positioning model module. The image enhancement module is used to expand the data in the aircraft type image database, the model compression module is used to compress the aircraft type identification and positioning model, and the aircraft type identification and positioning model module is used to classify and locate targets in the aircraft parking image.

[0038] The image management system is used to record and manage data of classification and positioning recognition results.

[0039] The invention has at least the following beneficial technical effects:

[0040] The intelligent identification and positioning method for parking aircraft in wide-area scenarios of this application can realize rapid and reliable model identification and positioning of parked aircraft in aircraft parking images, and can achieve real-time processing and reliable identification of multiple video streams, greatly improving the safety level of the apron and production management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] FIG1 is a flow chart of a method for intelligent identification and positioning of parked aircraft in a wide-area scenario according to one embodiment of the present application;

[0042] FIG2 is a schematic diagram of a multi-scale feature fusion module according to an embodiment of the present application;

[0043] FIG3 is a schematic diagram of an intelligent identification and positioning system for parking aircraft in a wide-area scenario according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the implementation of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below in conjunction with the drawings in the embodiments of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of this application, not all of the embodiments. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain this application, and should not be understood as limitations on this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The embodiments of this application are described in detail below in conjunction with the drawings.

[0045] The present application will be further described in detail below with reference to Figures 1 to 3 .

[0046] The first aspect of the present application provides a method for intelligent identification and positioning of parked aircraft in a wide-area scenario, as shown in FIG1 , comprising the following steps:

[0047] Step 1: Build a model image database;

[0048] Step 2: construct an initial aircraft model recognition and positioning model, and train the initial aircraft model recognition and positioning model through the aircraft model image database to obtain a first aircraft model recognition and positioning model;

[0049] Step 3: compress the first aircraft type identification and positioning model to obtain a second aircraft type identification and positioning model;

[0050] Step 4: Obtain an image of a parked aircraft, classify and locate the targets in the image using the second aircraft type recognition and positioning model, mark the target type and target location information in the image of the parked aircraft, and obtain a marked image of the parked aircraft.

[0051] Step 5:

[0052] Get the parking type and location information of each parking stand in the apron;

[0053] Determine whether the target location information in the aircraft parking image matches the parking stand location information, and also determine whether the target type in the aircraft parking image matches the parking stand type, and count the matching results;

[0054] The matching results and marked aircraft parking images are stored, managed and displayed.

[0055] In a preferred embodiment of the present application, first, in step 1, the process of constructing the model image database is as follows:

[0056] Collecting a predetermined number of images of parked aircraft;

[0057] Classify and locate the objects in the parked aircraft images and generate label documents, where each parked aircraft image corresponds to a label document;

[0058] Generate an aircraft model image database based on aircraft parking images and label documents.

[0059] The present invention's intelligent identification and location method for parked aircraft in wide-area scenarios utilizes a high-definition network surveillance camera to capture a predetermined number of parked aircraft images. The captured images are then categorized and labeled for each aircraft type, and an aircraft model image database is established based on the labeled information. After categorization and location labeling are completed for each parked aircraft image, an XML-formatted label document is generated. The label document contains the image name, the object type in the image, and the object's location. The aircraft model image database includes the parked aircraft images and the corresponding XML-formatted label documents.

[0060] In a preferred embodiment of the present application, in step 2, a convolutional neural network model for aircraft model recognition and positioning is built based on the aircraft model image database. The convolutional neural network model can classify various aircraft models in the aircraft parking image and identify the positioning feature information.

[0061] Specifically, in this embodiment, the initial aircraft model recognition and positioning model adopts the convolutional neural network YOLOv8 architecture, which can identify and locate multiple targets in the image in a relatively short time, and is more suitable for the scenario of this application. Furthermore, since the distance of the target varies greatly in the actual shooting of high-definition network surveillance cameras, how to accurately identify the characteristics of each aircraft model is the key to the real-time intelligent recognition and positioning of the aircraft parking image. Based on this problem, a multi-scale feature fusion module is set in the initial aircraft model recognition and positioning model, where, as shown in Figure 2, the multi-scale feature fusion module processes the data in the following way:

[0062] Given input feature F in , input feature F in After one convolution operation, the feature F1 is obtained: F1 = F in *W in ;

[0063] Feature F1 performs four-way feature operations of different scales in parallel. The calculation formulas for each feature extracted are:

[0064] The obtained four-way features are fused to obtain the output feature F out The calculation formula is:

[0065] Among them, F inis the input feature, F out are output features, F1, F 01 、F 11 、F 12 、F 21 、F 22 、F 31 、F 32 are the corresponding intermediate feature quantities, W in 、W 11 、W 12 、W 21 、W 22 、W 31 、W 32 、W out are the corresponding weights respectively, * represents the convolution operation, Indicates the point-by-point element addition operation of the feature, Concat represents the cascade function; in Figure 2, C is the feature depth, H is the feature height, and W is the feature width.

[0066] Advantageously, in this embodiment, before training the initial model recognition and positioning model using the model image database, a data augmentation algorithm is used to augment the data in the model image database. The data augmentation algorithm processes the data using tonal and affine transformations. Furthermore, during training of the initial model recognition and positioning model using the model image database, the convolutional neural network weight parameters of the initial model recognition and positioning model are modified using a stochastic gradient descent method, causing the initial model recognition and positioning model's loss value to decrease and converge to a stable value, thus achieving deep neural network training.

[0067] The present invention relates to a method for intelligently identifying and locating parked aircraft in a wide-area scenario. When identifying each aircraft type in an image of parked aircraft, the redundant parameters in the aircraft type identification and positioning model need to be analyzed and detected in real time across multiple video streams. This affects the efficiency of detection, and therefore requires model compression of the aircraft type identification and positioning model. In a preferred embodiment of the present invention, in step three, the first aircraft type identification and positioning model is compressed to obtain a second aircraft type identification and positioning model, including the following steps:

[0068] A channel scoring model for judging the importance of feature channels is added to the first aircraft identification and positioning model. The channel score tensor S in the channel scoring model is:

[0069] Among them, σ1 is the ReLU activation function, σ2 is the sigmoid activation function, W1 and W2 are the corresponding fully connected layer weights, S1, S2…S c are the corresponding channel score tensors respectively;

[0070] The channel score tensors are sorted by size, and several channel score tensors with channel scores lower than the threshold are introduced into the model training loss function. The redundant channels are sparsed through model fine-tuning training. The channel score penalty term of the model training loss function is:

[0071] Among them, C r is the total number of feature channels filtered from each layer of the model, ||S c ||1 is the L1 norm of the score tensor of the cth channel in each layer;

[0072] The channel score model is removed from the first aircraft type identification and positioning model, and several redundant parameters with low channel scores in the first aircraft type identification and positioning model are trimmed to obtain a second aircraft type identification and positioning model.

[0073] The present application discloses an intelligent identification and positioning method for parking aircraft in a wide-area scenario. By adding a channel scoring model for judging the importance of feature channels to the aircraft type identification and positioning model, the channel scoring module is removed after fine-tuning training is completed, and redundant parameters with low channel scores in the aircraft type identification and positioning model are trimmed, thereby reducing the number of model parameters and the amount of calculation to obtain the final aircraft type identification and positioning model.

[0074] In the method for intelligent identification and positioning of parked aircraft in a wide-area scenario of the present application, in step four, by real-time acquisition of parked aircraft images taken by various high-definition network surveillance cameras, the second aircraft model identification and positioning model is used to quickly classify and locate various aircraft models in the parked aircraft images, and different colored boxes are used to mark various aircraft models on the parked aircraft images to obtain a marked parked aircraft image.

[0075] In the present application's method for intelligently identifying and locating parked aircraft in wide-area scenarios, in step five, before performing video analysis of the apron, each parking space on the apron needs to be manually marked, with its parking space type and location information marked. After classifying and locating each parked aircraft type on the apron, the method uses the known parking space marking information to determine whether the aircraft is parked within the designated parking space area and whether the parked aircraft type matches the parking space type. Finally, the matching results of the type and location of each parked aircraft are tallied, and the matching results and the marked aircraft parking images are fed back to the image management system for storage, management, and display.

[0076] The present application discloses an intelligent identification and positioning method for parking aircraft in a wide-area scenario. First, a high-definition network surveillance camera is used to collect a certain amount of parking aircraft images. After labeling various aircraft models in the collected parking aircraft images, an aircraft model image database is established. A convolutional neural network model for aircraft model identification and positioning is built. The convolutional neural network model is trained using the aircraft model image database. The weight parameters of the convolutional neural network are corrected using the stochastic gradient descent method until the training loss value of the convolutional neural network model decreases and converges to a stable value, thereby obtaining an aircraft model identification and positioning model. The aircraft model identification and positioning model is compressed to reduce the number of model parameters and the amount of calculation, thereby obtaining the final aircraft model identification and positioning model. During the working process, by collecting the aircraft parking images taken by various cameras in real time, using the aircraft type recognition and positioning model, the various aircraft types in the aircraft parking images are quickly classified and positioned, and different color boxes are used to mark various aircraft types on the aircraft parking images. After the marked aircraft parking images are obtained, the number of various aircraft types is counted, and it is determined whether the aircraft is parked within the set parking area. At the same time, it is determined whether the parked aircraft type matches the parking type. This can achieve real-time processing and reliable identification of multiple video streams, greatly improving the apron safety level and production management efficiency.

[0077] Based on the above-mentioned method for intelligent identification and positioning of parked aircraft in wide-area scenarios, the second aspect of the present application also provides an intelligent identification and positioning system for parked aircraft in wide-area scenarios, as shown in Figure 3, including: a high-definition network surveillance camera, a computing power analysis server, a network switch, and an image management system, wherein the high-definition network surveillance camera is used to collect images of parked aircraft and transmit the collected data to the computing power analysis server through the network switch; the computing power analysis server includes an image enhancement module, a model compression module, and an aircraft type identification and positioning model module, the image enhancement module is used to expand the data in the aircraft type image database, the model compression module is used to compress the aircraft type identification and positioning model, and the aircraft type identification and positioning model module is used to classify and locate targets in the aircraft parking image; the image management system is used to record and manage data of the classification and positioning identification results.

[0078] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for intelligent identification and positioning of parked aircraft in a wide-area scenario, characterized in that: include: Step 1: Build a model image database; Step 2: constructing an initial aircraft model recognition and positioning model, and training the initial aircraft model recognition and positioning model using the aircraft model image database to obtain a first aircraft model recognition and positioning model; Step 3: compressing the first aircraft type identification and positioning model to obtain a second aircraft type identification and positioning model; Step 4: Acquire an image of a parked aircraft, classify and locate targets in the image of the parked aircraft using the second aircraft type recognition and positioning model, mark the target type and target location information in the image of the parked aircraft, and obtain a marked image of the parked aircraft.

2. The method for intelligent identification and positioning of parked aircraft in a wide-area scenario according to claim 1, characterized in that: In step 1, the construction of the model image database includes: Collecting a predetermined number of images of parked aircraft; Classifying and positioning the objects in the parked aircraft image to generate a label document, wherein each parked aircraft image corresponds to a label document; An aircraft model image database is generated based on the parked aircraft image and the label document.

3. The method for intelligent identification and positioning of parked aircraft in a wide-area scenario according to claim 2, characterized in that: The label document includes the name, target type and target location information of the aircraft parking image.

4. The method for intelligent identification and positioning of parked aircraft in a wide-area scenario according to claim 1, characterized in that: In step 2, the initial model identification and positioning model adopts the convolutional neural network YOLOv8 architecture, and a multi-scale feature fusion module is set in the initial model identification and positioning model, wherein: The multi-scale feature fusion module processes data in the following way: Given input feature F in , input feature F in After one convolution operation, feature F1 is obtained: F1=F in *W in ; Feature F1 performs four-way feature operations of different scales in parallel. The calculation formulas for each feature extracted are: F 01 =F1 F 12 =F 11 *W 12 F 22 =F 21 *W 22 ; F 32 =F 31 *W 32 The obtained four-way features are fused to obtain the output feature F out The calculation formula is: Among them, F in is the input feature, F out are output features, F1, F 01 、F 11 、F 12 、F 21 、F 22 、F 31 、F 32 are the corresponding intermediate feature quantities, W in 、W 11 、W 12 、W 21 、W 22 、W 31 、W 32 、W out are the corresponding weights respectively, * represents the convolution operation, It represents the feature point-by-point element addition operation, and Concat represents the cascade function.

5. The method for intelligent identification and positioning of parked aircraft in a wide-area scenario according to claim 4, characterized in that: In step 2, before the initial aircraft model recognition and positioning model is trained using the aircraft model image database, a data enhancement algorithm is used to expand the data in the aircraft model image database. The data enhancement algorithm processes the data in a manner including hue transformation and affine transformation.

6. The method for intelligent identification and positioning of parked aircraft in a wide-area scenario according to claim 5, characterized in that: In step 2, when the initial model recognition and positioning model is trained using the model image database, the convolutional neural network weight parameters of the initial model recognition and positioning model are corrected using the stochastic gradient descent method, so that the loss value of the initial model recognition and positioning model decreases and converges to a stable value.

7. The method for intelligent identification and positioning of parked aircraft in a wide-area scenario according to claim 1, characterized in that: In step three, the first aircraft type identification and positioning model is compressed to obtain a second aircraft type identification and positioning model, including: A channel score model is added to the first aircraft model identification and positioning model. The channel score tensor S in the channel score model is: S=σ2(σ1(F1*W1)*W2) S=[S1,S2,...S C ]; Among them, σ1 is the ReLU activation function, σ2 is the sigmoid activation function, W1 and W2 are The corresponding fully connected layer weights, S1, S2…S C are the corresponding channel score tensors respectively; The channel score tensors are sorted by size, and several channel score tensors with channel scores lower than the threshold are introduced into the model training loss function. The channel score penalty term of the model training loss function is: Among them, C r is the total number of feature channels filtered from each layer of the model, ‖S c ‖1 is the L1 norm of the score tensor of the cth channel in each layer; The channel score model is removed from the first aircraft type identification and positioning model, and several redundant parameters with low channel scores in the first aircraft type identification and positioning model are trimmed to obtain a second aircraft type identification and positioning model.

8. The method for intelligent identification and positioning of parked aircraft in a wide-area scenario according to claim 1, characterized in that: Also includes step five, Get the parking type and location information of each parking stand in the apron; Determining whether the target position information in the aircraft parking image matches the parking stand position information, and determining whether the target type in the aircraft parking image matches the parking stand type, and counting the matching results; The matching results and marked aircraft parking images are stored, managed and displayed.

9. A system for intelligent identification and positioning of parked aircraft in a wide-area scenario, based on the method for intelligent identification and positioning of parked aircraft in a wide-area scenario according to claim 8, characterized in that: include: High-definition network surveillance cameras, computing power analysis servers, network switches, and image management systems, among which: The high-definition network surveillance camera is used to collect images of parked aircraft and transmit the collected data to the computing power analysis server through a network switch; The computing power analysis server includes an image enhancement module, a model compression module, and an aircraft type identification and positioning model module. The image enhancement module is used to expand the data in the aircraft type image database, the model compression module is used to compress the aircraft type identification and positioning model, and the aircraft type identification and positioning model module is used to classify and locate targets in the aircraft parking image. The image management system is used to record and manage data of classification and positioning recognition results.

Images