Aircraft positioning system for a ramp

Abstract

The application discloses a kind of apron aircraft positioning systems, comprising: multiple image acquisition devices are fixedly arranged in apron, each image acquisition device is collected in apron Image;Multiple video stream processing units are connected with each image acquisition device respectively, the image collected by each image acquisition device is identified;Aircraft detection unit for determining the type of aircraft in identified image and marking the area of aircraft in identified image;Aircraft front wheel detection unit for identifying the complete front wheel of aircraft in the saliency area of aircraft;Position information calculation unit for calculating the position information of aircraft in apron based on the position information of identified complete front wheel and the image acquisition device corresponding to the identified image and the identified image.The present application can avoid the problems of false detection, missed detection and repeated detection, while accelerating the calculation of aircraft position information, it can also improve the accuracy of position calculation.

Description

Technical Field

[0001] The invention belongs to the technical field of apron aircraft positioning, and in particular relates to an apron aircraft positioning system. Background Art

[0002] In the daily operations of airports, the management of aircraft on the apron has always been a key task in maintaining normal commercial operations. With the rapid development of China's economy, the transportation and logistics industries have ushered in unprecedented development opportunities, and the number of passenger and cargo flights at airports has experienced explosive growth. As the primary activity area for various types of aircraft, the apron has become the most important operating area for passenger and cargo transportation at civil airports. In recent years, aircraft collisions on the apron have frequently occurred, seriously impacting the normal commercial operations of airports and causing significant economic losses to airlines. Therefore, effective management of various aircraft on the apron has become a crucial means of ensuring the safe and orderly operation of airports.

[0003] Aircraft management in complex apron environments requires real-time aircraft location information to avoid accidents such as collisions and improve aircraft management efficiency. However, obtaining real-time aircraft location information still faces numerous challenges. First, the extensive apron monitoring equipment deployed within airports cannot accurately and quickly provide aircraft location information, requiring manual observation to determine aircraft locations and manage them. Second, while three-dimensional point cloud positioning devices such as LiDAR can locate aircraft, they are expensive, and the deployment and installation of new equipment is subject to strict restrictions imposed by airport regulations.

[0004] Therefore, there is a particular need for an apron aircraft positioning system that is inexpensive and can accurately position aircraft. Summary of the Invention

[0005] The purpose of the present invention is to provide an apron aircraft positioning system which is cheap and can accurately position aircraft.

[0006] To achieve the above-mentioned purpose, the present invention provides an apron aircraft positioning system, comprising: a plurality of image acquisition devices, wherein the plurality of image acquisition devices are fixedly arranged in the apron, and each of the image acquisition devices acquires images of a fixed area in the apron; a multi-channel video stream processing unit, wherein the multi-channel video stream processing unit is connected to each image acquisition device and identifies the images acquired by each image acquisition device, wherein the identification corresponds to the image acquisition device that acquires the image; an aircraft detection unit, wherein the aircraft detection unit is used to determine the type of aircraft in the identification image and mark the prominent area of ​​the aircraft in the identification image; an aircraft front wheel detection unit, wherein the aircraft front wheel detection unit is used to identify the complete front wheel of the aircraft in the prominent area of ​​the aircraft; and a position information calculation unit, wherein the position information calculation unit calculates the position of the aircraft in the apron based on the identified complete front wheel and the identification image in which it is located, and the position of the image acquisition device corresponding to the identification image.

[0007] Optionally, the aircraft detection unit determines the type of aircraft in the identification image through a first convolutional neural network model and marks a salient area of ​​the aircraft in the image; the aircraft front wheel detection unit identifies the complete front wheel of the aircraft in the salient area of ​​the aircraft through a second convolutional neural network model.

[0008] Optionally, the first convolutional neural network model is obtained by the following steps: obtaining multiple images of each type of aircraft in an actual scene; for each image, manually labeling the type of aircraft and calibrating the area where the aircraft is located in the image; using the images with the labeled aircraft type and the calibrated aircraft area as training sets; training the first initial convolutional neural network model based on the training set to obtain the first convolutional neural network model.

[0009] Optionally, a second convolutional neural network model is obtained by the following steps: obtaining multiple images of each type of aircraft in an actual scene; for each image, manually calibrating the area where the complete front wheel of the aircraft is located in the image; using the image of the area where the complete front wheel of the aircraft is located as a training set; and training the second initial convolutional neural network model based on the training set to obtain a second convolutional neural network model.

[0010] Optionally, the calculation of the position of the aircraft on the apron based on the identified complete front wheel and the identification image where it is located, and the position of the image acquisition device corresponding to the identification image includes: obtaining the size of the complete front wheel in the identification image; comparing the size with the standard imaging size of the front wheel, and obtaining the distance d between the actual front wheel of the aircraft in the identification image and the image acquisition device corresponding to the identification image based on the optical parameters of the image acquisition device corresponding to the identification image; calculating the position of the actual front wheel of the aircraft in the first coordinate system based on the distance d, the first coordinate system being the world coordinate system; calculating the position of the aircraft on the apron based on the position of the actual front wheel of the aircraft in the first coordinate system and the position of the image acquisition device corresponding to the identification image on the apron.

[0011] Optionally, the first coordinate system takes the projection of the image acquisition device corresponding to the identification image on the ground as the origin, the north direction as the y-axis, and the east direction as the x-axis. The calculation of the position of the actual front wheel of the aircraft in the first coordinate system based on the distance includes: calculating the distance L between the projection of the image acquisition device corresponding to the identification image on the ground and the actual front wheel of the aircraft based on the distance d and the height H of the image acquisition device corresponding to the identification image from the ground; and calculating the y-axis coordinate and x-axis coordinate of the actual front wheel of the aircraft in the first coordinate system based on the distance L and the angle between the image acquisition device corresponding to the identification image and the x-axis in the first coordinate.

[0012] Optionally, the distance L is calculated using the following formula:

[0013]

[0014] Optionally, the position of the aircraft within the apron is calculated by the following steps: based on the position of the image acquisition device corresponding to the identification image within the apron, the position of the projection of the image acquisition device corresponding to the identification image on the ground is obtained, wherein the projected position includes the x-axis coordinate within the apron and the y-axis coordinate within the apron; the x-axis coordinate projected within the apron is added to the x-axis coordinate of the actual front wheel of the aircraft in the first coordinate system to obtain the x-axis coordinate information of the aircraft within the apron; the y-axis coordinate projected within the apron is added to the y-axis coordinate of the actual front wheel of the aircraft in the first coordinate system to obtain the y-axis coordinate information of the aircraft within the apron.

[0015] Optionally, the apron aircraft positioning system further includes: a display terminal, which is connected to the position information calculation unit and is used to display the position information of each aircraft in the apron.

[0016] Optionally, the image acquisition device is a fixed monocular camera.

[0017] The beneficial effects of the present invention are as follows: the apron aircraft positioning system of the present invention can avoid the problems of false detection, missed detection and repeated detection based on the rapid detection of the aircraft's front wheels in the significant area. While accelerating the calculation of aircraft position information, it can also improve the accuracy of position calculation. At the same time, it is inexpensive and easy to use widely.

[0018] The present invention has other features and advantages that will be apparent from or will be described in detail in the accompanying drawings and the following specific examples incorporated herein, which together serve to explain the specific principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

[0020] Figure 1 A structural block diagram of an apron aircraft positioning system according to an embodiment of the present invention is shown.

[0021] Figure 2 A schematic diagram illustrating a system for positioning aircraft on an apron according to an embodiment of the present invention for calculating the distance between a front wheel of an aircraft in a marked image and an image acquisition device corresponding to the marked image.

[0022] Description of reference numerals:

[0023] 102. Image acquisition device; 104. Multi-channel video stream processing unit; 106. Aircraft detection unit; 108. Aircraft front wheel detection unit; 110. Position information calculation unit; 202. Aircraft front wheel. DETAILED DESCRIPTION

[0024] The preferred embodiments of the present invention will be described in more detail below. Although the preferred embodiments of the present invention are described below, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Instead, these embodiments are provided to make the present invention more thorough and complete and to fully convey the scope of the present invention to those skilled in the art.

[0025] According to the present invention, an apron aircraft positioning system includes: multiple image acquisition devices, which are all fixedly arranged in the apron, and each image acquisition device acquires images of a fixed area in the apron; a multi-channel video stream processing unit, which is connected to each image acquisition device and identifies the images acquired by each image acquisition device, and the identification corresponds to the image acquisition device that acquires the image; an aircraft detection unit, which is used to determine the type of aircraft in the identification image and mark the prominent area of ​​the aircraft in the identification image; an aircraft front wheel detection unit, which is used to identify the complete front wheel of the aircraft in the prominent area of ​​the aircraft; and a position information calculation unit, which calculates the position of the aircraft in the apron based on the identification of the complete front wheel and the identification image where it is located and the position of the image acquisition device corresponding to the identification image.

[0026] Specifically, the video images captured in real time by the image acquisition device are encoded and compressed, then uploaded to the management and control center's multi-channel video stream processing unit via the real-time streaming protocol for processing, storage, and display. The multi-channel video stream processing unit primarily decodes the uploaded video images and annotates the decoded videos according to the corresponding camera's deployment location on the apron, allowing the management and control center's display terminal to reconstruct a complete monitoring of the apron space.

[0027] One of the keys to aircraft positioning is the rapid detection of the aircraft's front wheels, but a simple aircraft front wheel detection unit will have problems such as false detection, missed detection, and repeated detection in actual applications. The main reason for false detection is that the tires of other operating equipment on the apron interfere with the target detection algorithm of the aircraft's front wheels. The reason for missed detection is that the complex apron environment often has obstacles blocking the aircraft's front wheels. The large number of monocular cameras deployed on the apron will lead to repeated detection of the same aircraft's front wheels, and due to factors such as different angles and environmental interference, a lot of computing time and server resources are required to determine whether the detection targets of multiple cameras are the same object. In response to the above difficulties, this application uses two independent target detection units to achieve rapid detection of the aircraft and the aircraft's front wheels in two stages.

[0028] The aircraft detection unit employs a convolutional neural network-based object detection method, enabling real-time aircraft detection and classification, marking the aircraft's salient area within the image. Detection results are displayed in real time on the management and control center's display terminal, marking the aircraft and generating its salient area. The detection unit can also determine the aircraft model based on the classification results and search for relevant information in the database.

[0029] After detecting an aircraft and obtaining its position calibration and model information in the image, the aircraft's front wheel detection unit quickly detects the aircraft's front wheels within the aircraft's prominent areas. Aircraft detection and annotation of the aircraft's prominent areas are performed on each image. Front wheel detection is then performed on the images with the annotated aircraft's prominent areas. If multiple monocular cameras can simultaneously detect the same aircraft's front wheels, the aircraft detection unit uniformly annotates them and utilizes the video images from multiple cameras in the position information calculation unit to accelerate position information calculation and improve position accuracy. Based on the aircraft's relevant parameter information, the position information calculation unit compares the actual size of the aircraft's front wheels in the image with their size within a standard defining surface. Using the camera's perspective principle, the unit rapidly measures distance. A first coordinate system is established with the fixed monocular camera's projection on the ground as the origin. The aircraft's position is then immediately calculated based on the distance between the aircraft's front wheels and the monocular camera.

[0030] The accurate position information of the aircraft on the apron and the detection frame of the aircraft in the video image are displayed on the display terminal of the management and control center, and the display terminal provides the position information of each aircraft on the apron in real time.

[0031] According to an exemplary embodiment, the apron aircraft positioning system can avoid the problems of false detection, missed detection and repeated detection based on the rapid detection of the aircraft's front wheels in the aircraft's prominent area. While accelerating the calculation of aircraft position information, it can also improve the accuracy of position calculation. At the same time, it is inexpensive and easy to use widely.

[0032] As an optional solution, the aircraft detection unit determines the type of aircraft in the identification image and marks the salient area of ​​the aircraft in the image through a first convolutional neural network model; the aircraft front wheel detection unit identifies the complete front wheel of the aircraft in the salient area of ​​the aircraft through a second convolutional neural network model.

[0033] The identification image is used as the input data of the first convolutional neural network model. Through the classification and detection of the first convolutional neural network model, the type of aircraft in the identification image is determined and the salient area of ​​the aircraft is marked in the image; the image with the salient area of ​​the aircraft marked is used as the input data of the second convolutional neural network model. Through the recognition of the second convolutional neural network model, the complete front wheel of the aircraft is identified.

[0034] As an optional solution, a first convolutional neural network model is obtained by the following steps: obtaining multiple images of each type of aircraft in an actual scenario; for each image, manually labeling the type of aircraft and calibrating the area where the aircraft is located in the image; using the images with the labeled aircraft type and the calibrated aircraft area as a first training set; and training a first initial convolutional neural network model based on the first training set to obtain a first convolutional neural network model.

[0035] The primary purpose of the aircraft detection unit is to detect aircraft in real-time within video images, classify and identify aircraft types, and label prominent areas within them. Within the apron operating area, aircraft are large and possess various distinctive features, such as a long fuselage, mid-section wings, and tail wings, which clearly distinguish them from other equipment on the apron. Therefore, when capturing images of aircraft in real-world scenarios, the aircraft type is manually labeled and the area within the image where the aircraft is located is calibrated. Each image of a known aircraft type and the calibrated area serves as the first training set. Based on this first training set, a first initial convolutional neural network model is trained to identify the aircraft type and area within the image using the aircraft's characteristics. This results in extremely low false positives and missed detections. Compared to real-time detection of the aircraft's front wheels, real-time aircraft detection is simpler and faster, significantly improving the performance of the aircraft positioning system.

[0036] By classifying and identifying aircraft types, duplicate detection can be avoided. The characteristics of individual aircraft aid the object detection algorithm in identification and classification. While obtaining relevant aircraft parameters, the system can also quickly determine whether aircraft in different images are homologous. Based on the annotation results, salient regions can be generated in the video image, allowing for rapid detection of aircraft front wheels.

[0037] As an optional solution, a second convolutional neural network model is obtained by the following steps: obtaining multiple images of each type of aircraft in an actual scenario; for each image, manually calibrating the area where the aircraft is located in the image, and manually calibrating the area where the complete front wheel of the aircraft is located in the area where the aircraft is located; using the images of the area where the complete front wheel of the aircraft is calibrated as a second training set; training the second initial convolutional neural network model based on the second training set to obtain a second convolutional neural network model.

[0038] Specifically, images of aircraft in actual scenarios are collected, the area where the aircraft is located is manually calibrated in the image, and the area where the complete front wheel of the aircraft is located is manually calibrated in the area where the aircraft is located. The images of the area where the complete front wheel of the calibrated aircraft is located are used as the second training set. Based on the second training set, a second initial convolutional neural network model is trained, and the complete front wheel of the aircraft in the image is identified through the features of the complete front wheel of the aircraft to obtain a second convolutional neural network model.

[0039] Rapid detection of an aircraft's front wheels has two main purposes: first, to determine the position of the aircraft's front wheels in the image. Subsequent calculations will determine the aircraft's spatial position based on the two-dimensional coordinates in the image; second, to obtain the size of the aircraft's front wheels in the image based on the aircraft's salient area, which will be used by the subsequent position calculation unit to measure the distance between the aircraft's front wheels and the camera.

[0040] As an optional solution, calculating the position of the aircraft on the apron based on the identified complete front wheel and the identification image where it is located, as well as the position of the image acquisition device corresponding to the identification image includes: obtaining the size of the complete front wheel in the identification image; comparing the size with the standard imaging size of the front wheel, and obtaining the distance d between the actual front wheel of the aircraft in the identification image and the image acquisition device corresponding to the identification image based on the optical parameters of the image acquisition device corresponding to the identification image; calculating the position information of the actual front wheel of the aircraft in the first coordinate system based on the distance d, which is the world coordinate system; calculating the position of the aircraft on the apron based on the position of the actual front wheel of the aircraft in the first coordinate system and the position of the image acquisition device corresponding to the identification image on the apron.

[0041] Specifically, an apron coordinate system covering the entire airport is established based on the airport's latitude and longitude information, and the latitude and longitude coordinates of each image acquisition device in the airport coordinate system are confirmed. Then, taking the image of the complete front wheel of the aircraft as an example, the position information of the aircraft on the apron is calculated, and the projection of the image acquisition device corresponding to the image of the complete front wheel of the aircraft on the ground is used as the origin to establish a first coordinate system. The first coordinate system uses the north direction as the y-axis and the east direction as the x-axis. Fixed-model aircraft will be equipped with front wheels of uniform specifications, and the aircraft tires have strict shape and size standards.

[0042] Based on the aircraft detection unit's aircraft type determination function, the aircraft model and its front wheel parameters are obtained. The dimensions of the aircraft's front wheel imaged on the reference surface are used as the detection benchmark. Based on the accurate optical parameters of the image acquisition equipment deployed on the apron, a direct regression calculation is performed using the perspective principle of the image acquisition equipment, which states that objects appear larger near and smaller far away. The distance between the aircraft's front wheel and the image acquisition equipment is then calculated using the Pythagorean theorem. The distance between the aircraft's front wheel and the farthest point in the first coordinate system is then calculated based on the image acquisition equipment's rotation angle. The y- and x-axis coordinates of the aircraft's front wheel in the first coordinate system are then obtained. The aircraft's position within the apron is then determined using a coordinate system conversion formula.

[0043] As an optional scheme, the first coordinate system takes the projection of the image acquisition device corresponding to the identification image on the ground as the origin, the north direction as the y-axis, and the east direction as the x-axis. Based on the distance, the position of the actual front wheel of the aircraft in the first coordinate system is calculated, including: based on the distance d and the height H of the image acquisition device corresponding to the identification image from the ground, calculating the distance L between the projection of the image acquisition device corresponding to the identification image on the ground and the actual front wheel of the aircraft; based on the distance L and the angle between the image acquisition device corresponding to the identification image and the x-axis in the first coordinate, calculating the y-axis coordinate and x-axis coordinate of the actual front wheel of the aircraft in the first coordinate system.

[0044] Specifically, the distance information d between the image acquisition device and the front wheel of the aircraft can be obtained through the perspective principle of the image acquisition device and its accurate optical parameters, while the height H of the fixed image acquisition device and the angle of rotation of the camera are known parameter information, and the y-axis coordinate and x-axis coordinate of the front wheel of the aircraft in the first coordinate system are then calculated.

[0045] Alternatively, the distance L can be calculated using the following formula:

[0046]

[0047] Specifically, through a simple Pythagorean theorem, the distance L between the front wheel of the aircraft and the coordinates of the origin in the first coordinate system can be obtained.

[0048] As an optional solution, the position of the aircraft on the apron is calculated by the following steps: based on the position of the image acquisition device corresponding to the identification image on the apron, the position of the projection of the image acquisition device corresponding to the identification image on the ground is obtained, wherein the projected position includes the x-axis coordinate on the apron and the y-axis coordinate on the apron; the x-axis coordinate projected on the apron is added to the x-axis coordinate of the actual front wheel of the aircraft in the first coordinate system to obtain the x-axis coordinate information of the aircraft on the apron; the y-axis coordinate projected on the apron is added to the y-axis coordinate of the actual front wheel of the aircraft in the first coordinate system to obtain the y-axis coordinate information of the aircraft on the apron.

[0049] Specifically, the airport's global coordinate system is converted based on the position information of the image acquisition device corresponding to the identification image within the apron, and the coordinate information of the aircraft in the first coordinate system is directly converted into the coordinate information within the apron range. After the coordinates of all aircraft on the apron are integrated, they are displayed on the display terminal of the control and management center.

[0050] As an optional solution, the apron aircraft positioning system further includes: a display terminal connected to the position information calculation unit, and configured to display the position information of each aircraft within the apron.

[0051] Specifically, the above method is used to calculate the position information of each aircraft in the apron, and the display terminal displays the position information of each aircraft in the apron for the convenience of staff to view.

[0052] As an optional solution, the image acquisition device is a fixed monocular camera.

[0053] Specifically, fixed monocular cameras are widely deployed within the apron to collect video images of aircraft moving on the apron and related operating equipment and personnel around the clock. The fixed monocular cameras are inexpensive, making the apron aircraft positioning system easy to promote widely.

[0054] Example

[0055] Figure 1 A structural block diagram of an apron aircraft positioning system according to an embodiment of the present invention is shown. Figure 2 A schematic diagram illustrating a system for positioning aircraft on an apron according to an embodiment of the present invention for calculating the distance between a front wheel of an aircraft in a marking image and an image acquisition device corresponding to the marking image is shown.

[0056] Combine Figure 1 and Figure 2As shown, the apron aircraft positioning system includes: multiple image acquisition devices 102, which are fixedly arranged in the apron, and each image acquisition device 102 acquires images of a fixed area in the apron; a multi-channel video stream processing unit 104, which is connected to each image acquisition device 102 and identifies the images acquired by each image acquisition device 102, and the identification corresponds to the image acquisition device that acquires the image; an aircraft detection unit 106, which is used to determine the type of aircraft in the identification image and mark the prominent area of ​​the aircraft in the identification image; an aircraft front wheel detection unit 108, which is used to identify the complete front wheel of the aircraft in the prominent area of ​​the aircraft; a position information calculation unit 110, which calculates the position of the aircraft in the apron based on the identified complete front wheel and the identification image where it is located, and the position of the image acquisition device 102 corresponding to the identification image.

[0057] Among them, the aircraft detection unit determines the type of aircraft in the identification image through the first convolutional neural network model and marks the salient area of ​​the aircraft in the image; the aircraft front wheel detection unit identifies the complete front wheel of the aircraft in the salient area of ​​the aircraft through the second convolutional neural network model.

[0058] The first convolutional neural network model is obtained by the following steps: obtaining multiple images of each type of aircraft in an actual scene; for each image, manually labeling the type of aircraft and calibrating the area where the aircraft is located in the image; using the images with the labeled aircraft type and the calibrated aircraft area as the first training set; and training the first initial convolutional neural network model based on the first training set to obtain the first convolutional neural network model.

[0059] Among them, the second convolutional neural network model is obtained by the following steps: obtaining multiple images of each type of aircraft in the actual scene; for each image, manually calibrating the area where the complete front wheel of the aircraft is located in the image; using the image of the area where the complete front wheel of the aircraft is located as the second training set; training the second initial convolutional neural network model based on the second training set to obtain the second convolutional neural network model.

[0060] Among them, based on the identified complete front wheel and the identification image where it is located and the position of the image acquisition device corresponding to the identification image, calculating the position of the aircraft on the apron includes: obtaining the size of the complete front wheel in the identification image; comparing the size with the standard imaging size of the front wheel, and based on the optical parameters of the image acquisition device 102 corresponding to the identification image, obtaining the distance d between the actual front wheel 202 of the aircraft in the identification image and the image acquisition device 102 corresponding to the identification image; based on the distance d, calculating the position information of the actual front wheel 202 of the aircraft in the first coordinate system; based on the position of the actual front wheel 202 of the aircraft in the first coordinate system and the position of the image acquisition device 102 corresponding to the identification image on the apron, calculating the position of the aircraft on the apron.

[0061] Among them, the first coordinate system takes the projection of the image acquisition device 102 corresponding to the identification image on the ground as the origin, the north direction as the y-axis, and the east direction as the x-axis. Based on the distance, the position of the actual front wheel 202 of the aircraft in the first coordinate system is calculated, including: based on the distance and the height H of the image acquisition device 102 corresponding to the identification image from the ground, calculating the distance L between the projection of the image acquisition device 102 corresponding to the identification image on the ground and the actual front wheel 202 of the aircraft; based on the distance L and the angle between the image acquisition device corresponding to the identification image and the x-axis in the first coordinate, calculating the y-axis coordinate and the x-axis coordinate of the front wheel 202 of the aircraft in the first coordinate system.

[0062] The distance L is calculated using the following formula:

[0063]

[0064] The position of the aircraft within the apron is calculated by the following steps: based on the position of the image acquisition device 102 corresponding to the identification image within the apron, the position of the projection of the image acquisition device 102 corresponding to the identification image on the ground is obtained, wherein the projected position includes the x-axis coordinate within the apron and the y-axis coordinate within the apron; the x-axis coordinate projected within the apron is added to the x-axis coordinate of the actual front wheel 202 of the aircraft in the first coordinate system to obtain the x-axis coordinate information of the aircraft within the apron; the y-axis coordinate projected within the apron is added to the y-axis coordinate of the actual front wheel 202 of the aircraft in the first coordinate system to obtain the y-axis coordinate information of the aircraft within the apron.

[0065] The apron aircraft positioning system further includes: a display terminal connected to the position information calculation unit, and used to display the position information of each aircraft in the apron.

[0066] The image acquisition device 102 is a fixed monocular camera.

[0067] While various embodiments of the present invention have been described above, the above description is intended to be illustrative, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. An apron aircraft positioning system, characterized in that: include: A plurality of image acquisition devices, each of which is fixedly arranged in the apron, and each of which acquires an image of a fixed area in the apron; a multi-channel video stream processing unit, the multi-channel video stream processing unit being connected to each image acquisition device and identifying an image acquired by each image acquisition device, wherein the identification corresponds to the image acquisition device that acquired the image; an aircraft detection unit, the aircraft detection unit being configured to determine the type of aircraft in the identification image and mark a significant area of ​​the aircraft in the identification image; an aircraft nose wheel detection unit, the aircraft nose wheel detection unit being configured to identify an intact nose wheel of the aircraft in a prominent area of ​​the aircraft; a position information calculation unit, the position information calculation unit calculating the position of the aircraft on the apron based on the identified complete front wheel and the identification image where it is located, and the position of the image acquisition device corresponding to the identification image; Calculating the position of the aircraft on the apron based on the identified complete front wheel and the identification image where the front wheel is located, and the position of the image acquisition device corresponding to the identification image includes: Obtain the size of the complete front wheel in the identification image; Comparing the size with a standard imaging size of the front wheel, and obtaining a distance d between the actual front wheel of the aircraft in the identification image and the image acquisition device corresponding to the identification image based on optical parameters of the image acquisition device corresponding to the identification image; Calculating the actual position of the front wheel of the aircraft in a first coordinate system based on the distance d, where the first coordinate system is a world coordinate system; Calculating the position of the aircraft within the apron based on the actual position of the front wheel of the aircraft in the first coordinate system and the position of the image acquisition device corresponding to the identification image within the apron; The first coordinate system takes the projection of the image acquisition device corresponding to the identification image on the ground as the origin and the north direction as the Axis, with due east as axis, calculating the actual position of the front wheel of the aircraft in the first coordinate system based on the distance includes: Calculate, based on the distance d and a height H between the image acquisition device corresponding to the identification image and the ground, a distance L between a projection of the image acquisition device corresponding to the identification image on the ground and an actual front wheel of the aircraft; Based on the distance L and the image acquisition device corresponding to the identification image relative to the first coordinate The angle between the axes is calculated to calculate the actual front wheel of the aircraft in the first coordinate system. Axis coordinates and axis coordinates; The distance L is calculated by the following formula: 。 2. The apron aircraft positioning system according to claim 1, characterized in that: The aircraft detection unit determines the type of aircraft in the identification image and marks the salient area of ​​the aircraft in the image through a first convolutional neural network model; the aircraft front wheel detection unit identifies the complete front wheel of the aircraft in the salient area of ​​the aircraft through a second convolutional neural network model.

3. The apron aircraft positioning system according to claim 2, characterized in that: The first convolutional neural network model is obtained by the following steps: Acquire multiple images of each type of aircraft in realistic scenarios; For each image, manually label the type of aircraft and demarcate the area where the aircraft is located in the image; The images labeled with the aircraft type and the area where the aircraft is located are used as the first training set; A first initial convolutional neural network model is trained based on the first training set to obtain a first convolutional neural network model.

4. The apron aircraft positioning system according to claim 3, characterized in that: The second convolutional neural network model is obtained by the following steps: Acquire multiple images of each type of aircraft in realistic scenarios; For each image, manually calibrate the area where the complete front wheel of the aircraft is located in the image; The images of the area where the complete front wheel of the calibrated aircraft is located are used as the second training set; A second initial convolutional neural network model is trained based on the second training set to obtain a second convolutional neural network model.

5. The apron aircraft positioning system according to claim 1, characterized in that: The position of the aircraft on the ramp is calculated by the following steps: Based on the position of the image acquisition device corresponding to the identification image in the apron, the position of the projection of the image acquisition device corresponding to the identification image on the ground is obtained, wherein the projection position includes the position of the image acquisition device in the apron. Axis coordinates and apron axis coordinates; Project the above information on the apron The axis coordinates are the same as the actual front wheel coordinates of the aircraft in the first coordinate system. The axis coordinates are added to obtain the position of the aircraft within the apron. Axis coordinate information; project the The axis coordinates are the same as the actual front wheel coordinates of the aircraft in the first coordinate system. The axis coordinates are added to obtain the position of the aircraft within the apron. Axis coordinate information.

6. The apron aircraft positioning system according to claim 5, characterized in that: Also includes: A display terminal is connected to the position information calculation unit and is used to display the position information of each aircraft in the apron.

7. The apron aircraft positioning system according to claim 1, characterized in that: The image acquisition device is a fixed monocular camera.

Images