A Real-time Detection Method and System for Airport Runway Targets in Deep Learning

Abstract

The present application provides a real-time detection method and system for airport runway targets in deep learning, belonging to the technical field of intelligent computing. Specifically, it includes: receiving an input image, adjusting the resolution of the image to meet the requirements of the preliminary detection model; using an embedded device to perform real-time detection of the deep learning model on the image after the line resolution adjustment to determine the position information of the runway, where the position information is a rectangular frame containing the runway; establishing a deep learning model for runway edge detection; for the image of the runway position information, restoring it to the original image resolution, inputting the restored image into the learning model, and outputting encoded information, where the encoded information represents the runway edge data; performing inverse decoding on the encoded information output by the deep learning model to obtain the lines of the runway edge, making marks on the original image, and outputting a visualization result. Through the processing solution of the present application, the problem of low processing efficiency of the embedded computing platform is effectively solved.

Description

Technical Field

[0001] This application relates to the field of intelligent computing, and in particular, to a real-time detection method and system for airport runway targets based on deep learning. Background Art

[0002] With the continuous development of intelligent aviation technology, the need for real-time detection of airport runways has become increasingly necessary. However, real-time detection of airport runways requires high real-time performance and accuracy. Due to the influence of space and heat dissipation on the airborne platform, the computing performance is limited.

[0003] In order to improve the processing efficiency of embedded deep learning algorithms, hardware manufacturers have successively launched multiple AI chips to improve the processing speed by optimizing the computing structure at the hardware level. However, it is difficult to achieve the required real-time detection only by hardware optimization. Summary of the Invention

[0004] In view of this, this application provides a real-time detection method and system for airport runway targets based on deep learning, which solves the problems in the prior art and effectively solves the problem of low processing efficiency of the embedded computing platform.

[0005] A real-time detection method for airport runway targets based on deep learning provided by this application adopts the following technical solutions:

[0006] A real-time detection method for airport runway targets based on deep learning includes:

[0007] Step 1: Receive the input image and adjust the image resolution to meet the requirements of the preliminary detection model.

[0008] Step 2: Use an embedded device to perform real-time detection of the deep learning model on the image after row resolution adjustment to determine the position information of the runway. The position information is a rectangular frame containing the runway.

[0009] Step 3: Establish a deep learning model for runway edge detection.

[0010] Step 4: For the image of the runway position information, restore it to the original image resolution, input the restored image into the learning model, and output encoded information, which represents the runway edge data through the encoded information.

[0011] Step 5: Decode the encoded information output by the deep learning model to obtain the runway edge lines, make marks on the original image, and output the visualization result.

[0012] Optionally, the step 4 outputs multiple groups of encoded information, and the multiple groups of encoded information are respectively the left edge line of the runway, the right edge line of the runway, the runway center line, and the starting line of the crosswalk.

[0013] Optionally, the encoded information includes a slope, an intercept, start point information, and end point information.

[0014] Optionally, a straight line is obtained based on the slope and intercept in a set of codes. The straight line forms two intersection points with the rectangular frame. Among them, the start point information includes the distance between the intersection point with a smaller ordinate value and the start point, and the end point information includes the distance between the intersection point with a smaller ordinate value and the end point.

[0015] Optionally, step 1 is specifically to adjust the original image according to the number of pixels of the preset width W and height H. The width W and height H are determined according to the computing power of the processor to meet the frame rate requirements of real-time detection.

[0016] Optionally, step 2 is specifically to perform an initial detection of runway recognition on the image with pixels of W×H based on the convolutional neural network algorithm and output runway position information.

[0017] Optionally, step 2 uses FPGA to perform hardware acceleration on the deep learning model to meet the requirements of better real-time performance.

[0018] On the other hand, a real-time airport runway target detection system for deep learning provided by the present application adopts the following technical solutions:

[0019] A real-time airport runway target detection system for deep learning, comprising:

[0020] An image preprocessing unit, configured to receive an input image and adjust the image resolution to meet the model input requirements for real-time detection of the runway position

[0021] A real-time runway position detection unit, which performs target detection on the preprocessed image to obtain runway position-related information;

[0022] A real-time runway edge detection unit, which outputs encoded information according to the runway position image and represents the runway edge data through the encoded information.

[0023] In summary, the present application includes the following beneficial technical effects:

[0024] To meet the real-time detection requirements of airport runway targets, in view of the limitations of running deep neural networks on embedded devices, a real-time airport runway target detection method for embedded deep learning is proposed. First, the general runway range is determined based on the initial detection of the runway position, and then the runway edge is detected, effectively solving the problem of low processing efficiency of the embedded computing platform. Description of the Drawings

[0025] To more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0026] Figure 1 This is the network structure diagram of the roadwayNet model of the present application;

[0027] Figure 2 This is the flowchart of the present application. Detailed implementation manners

[0028] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0029] The following uses specific specific examples to illustrate the implementation manners of the present application. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. The present application can also be implemented or applied through other different specific implementation manners. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts belong to the scope of protection of the present application.

[0030] It should be noted that the following describes various aspects of the embodiments within the scope of the appended claims. It should be obvious that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on the present application, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, this device and / or this method can be implemented using other structures and / or functions in addition to one or more of the aspects described herein.

[0031] It also needs to be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner. The diagrams only show the components related to the present application, rather than being drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in its actual implementation can be an arbitrary change, and the component layout type may also be more complex.

[0032] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0033] An embodiment of the present application provides a real-time detection method for airport runway targets using deep learning.

[0034] As Figure 1 and Figure 2 shown, a real-time detection method for airport runway targets using deep learning includes:

[0035] Step 1: Receive the input image and adjust the resolution of the image to meet the requirements of the preliminary detection model.

[0036] Step 2: Use an embedded device to perform real-time detection of the deep learning model on the image after row resolution adjustment to determine the position information of the runway. The position information is a rectangular frame containing the runway.

[0037] Step 3: Establish a deep learning model for runway edge detection.

[0038] Step 4: For the image of the runway position information, restore it to the original image resolution. Input the restored image into the learning model, and output encoded information, which represents the runway edge data through the encoded information.

[0039] Step 5: Decode the encoded information output by the deep learning model to obtain the lines of the runway edge, make marks on the original image, and output the visualization result.

[0040] Specifically, the step 4 outputs multiple groups of encoded information. The multiple groups of encoded information are respectively the left edge line, right edge line, runway center line, and starting line of the crosswalk of the runway. The encoded information includes slope, intercept, starting point information, and ending point information.

[0041] According to the slope and intercept in a group of codes, a straight line is obtained. The straight line forms two intersection points with the rectangular frame. The straight line obtained by the slope and intercept is more accurate and faster than the traditional method of connecting multiple points. Among them, the starting point information includes the distance between the intersection point with a smaller ordinate value and the starting point. When the ordinates of the two intersection points are the same, select the intersection point with a smaller abscissa; the ending point information includes the distance between the intersection point with a smaller ordinate value and the ending point. When the ordinates of the two intersection points are the same, select the intersection point with a smaller abscissa value. Selecting the intersection point with a smaller coordinate value makes it easier for the distance between the starting point and the intersection point to be zero, and the deep learning model can more easily learn the position of the starting point. This representation method can better represent the characteristics of the runway edge.

[0042] As Figure 1As shown in the figure, specifically, the roadwayNet model is trained using pre-annotated airport runway data. The input image will pass through the convolutional layer and the fully connected layer to complete the output of runway edge information. In particular, we encode the edge information using the characteristics of the airport runway.

[0043] Let (a1, b1, s1, e1), (a2, b2, s2, e2), (a3, b3, s3, e3), (a4, b4, s4, e4) be used to represent the left edge line, right edge line, runway center line, and starting line of the crosswalk of the runway respectively. Among them, for each line, a and b are used to represent the slope and intercept of the line respectively, s represents the starting point information of the line, and e represents the ending point information of the line.

[0044] The specific step 1 is to adjust the original image according to the number of pixels of the preset width W and height H. The width W and height H are determined according to the computing power of the processor to meet the frame rate requirements of real-time detection.

[0045] The specific step 2 is to perform an initial detection of runway recognition on the image with pixels of W×H based on the convolutional neural network algorithm, and output the runway position information. The FPGA is used to perform hardware acceleration on the deep learning model to meet the requirements of better real-time performance.

[0046] This application also discloses a real-time detection system for airport runway targets using deep learning.

[0047] A real-time detection system for airport runway targets using deep learning includes:

[0048] An image preprocessing unit, which is used to receive the input image and adjust the image resolution to meet the model input requirements for real-time detection of the runway position.

[0049] A real-time runway position detection unit, which performs target detection on the preprocessed image to obtain information related to the runway position.

[0050] A real-time runway edge detection unit, which outputs encoded information according to the runway position image, and represents the runway edge data through the encoded information.

[0051] The image first performs an initial detection of the runway position and then performs a detection of the runway edge. Specifically: the image is adjusted and then the initial detection is used to quickly find the approximate position of the runway, and the image of the approximate position of the runway is restored and the target detection result of the runway edge is completed.

[0052] As Figure 2 shown, taking the YOLO target detection algorithm as an example, the specific working process is as Figure 1 shown. The specific process is as follows:

[0053] First, the input image is adjusted to a resolution of 416*416 by the image preprocessing unit.

[0054] Then, the images enter the target detection unit in batches, and the preliminary detection of the runway position is performed in parallel based on the YOLO algorithm.

[0055] Finally, the results of the preliminary detection are restored to the original image, and edge detection of the runway is performed. Using techniques such as Hough transform, image enhancement, and line segment edge detection, the edge position of the runway is found, and the visualization process is completed to output the image.

[0056] For the target detection of each sub-image above, different conventional algorithms (usually based on convolutional neural network algorithms) can be used according to actual needs. For example, in addition to the YOLO algorithm (focusing on operation speed), the Faster R-CNN algorithm (focusing on positioning accuracy) can also be used.

[0057] As described above, this is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A real-time detection method for airport runway targets based on deep learning, characterized in that , including: Step 1: Receive the input image and adjust the image resolution to meet the requirements of the preliminary detection model. Step 2: Use an embedded device to perform real-time detection of the deep learning model on the image after row resolution adjustment to determine the position information of the runway. The position information is a rectangular frame containing the runway. Step 3: Establish a deep learning model for runway edge detection. Step 4: For the image with runway position information, restore it to the original image resolution. Input the restored image into the learning model, and output encoded information. The runway edge data is represented by the encoded information, which includes slope, intercept, start point information, and end point information. A straight line is obtained based on the slope and intercept in a set of encoded information, and the straight line forms two intersection points with the rectangular frame. Among them, the start point information includes the distance between the intersection point with a smaller ordinate value and the start point, and the end point information includes the distance between the intersection point with a smaller ordinate value and the end point. Step 5: Decode the encoded information output by the deep learning model to obtain the lines of the runway edge, make marks on the original image, and output the visualization result.

2. The real-time airport runway target detection method for deep learning according to claim 1, characterized in that, The Step 4 outputs multiple sets of encoded information, and the multiple sets of encoded information are respectively the left edge line, right edge line, middle line of the runway, and the starting line of the crosswalk.

3. The real-time airport runway target detection method for deep learning according to claim 1, wherein Specifically, in Step 1, the original image is adjusted according to the number of pixels of the preset width W and height H, and the width W and height H are determined according to the computing power of the processor to meet the frame rate requirements of real-time detection.

4. The real-time airport runway target detection method for deep learning according to claim 1, characterized in that: Specifically, in Step 2, based on the convolutional neural network algorithm, the image with pixels of W×H is initially detected for runway recognition, and the runway position information is output.

5. The real-time airport runway target detection method for deep learning according to claim 4, characterized in that: In Step 2, the FPGA is used to perform hardware acceleration on the deep learning model to meet the requirements of better real-time performance.

6. A real-time airport runway target detection system based on deep learning, characterized in that, An airport runway target real-time detection method for deep learning for performing any one of the above claims 1-5. The real-time detection system includes: An image preprocessing unit for receiving the input image and adjusting the image resolution to meet the model input requirements for real-time detection of the runway position. A runway position real-time detection unit for performing target detection on the preprocessed image to obtain runway position-related information. A runway edge real-time detection unit for outputting encoded information according to the runway position image, and representing the runway edge data by the encoded information.

Images