Airport runway corner point detection method, device, computer equipment and storage medium

Abstract

The present invention discloses an airport runway corner detection method, device, computer device and storage medium, which are used to improve the detection accuracy of airport runway corners. The main technical solution is as follows: input the airborne image outside the cabin to be detected into the airport runway detection model, and predict the coordinates of the airport runway center relative to the upper left point of the grid, the values of the width and height of the airport runway relative to the airborne image outside the cabin, and the deviation amounts of the four airport runway corners; according to the coordinates of the airport runway center relative to the upper left point of the grid, the values of the runway width and height relative to the airborne image outside the cabin, and the deviation amounts of the four runway corners, calculate the pixel coordinates of the four corners of the airport runway in the airborne image outside the cabin.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and particularly to a method, device, computer device and storage medium for detecting corner points of an airport runway. Background Art

[0002] With the development of airborne avionics technology, many flight assistance systems have emerged. Among them, the enhanced vision system can improve the pilot's perception of the external environment and enhance flight safety. Detecting corner points of the runway plays an important role in the enhanced vision system.

[0003] Currently, the corner points of the airport runway are mainly detected by the irregular quadrilateral detection technology, that is, while using the object detection algorithm based on deep learning to predict the position and size of the target box, the corner point offset amounts of four rectangular boxes are also predicted to adjust their corner positions to make them into irregular quadrilateral boxes.

[0004] The four corner points predicted by the above technology need to be on the four sides of the square (as Figure 1 shown), but during the aircraft's near landing process, the corner points of the airport runway observed by the camera are not necessarily all on the four sides of the square. There may be two corner points on one side, or the corner point may not fall on the side. Therefore, the accuracy of detecting the corner points of the airport runway by the existing technology is relatively low. Summary of the Invention

[0005] The present invention provides a method, device, computer device and storage medium for detecting corner points of an airport runway, which is used to improve the detection accuracy of the corner points of the airport runway.

[0006] An embodiment of the present invention provides a method for detecting corner points of an airport runway, and the method includes:

[0007] Inputting the external airborne image to be detected into the airport runway detection model, and predicting the coordinates of the airport runway center relative to the upper left point of the grid, the values of the airport runway width and height relative to the external airborne image, and the deviation amounts of the four airport runway corner points;

[0008] According to the coordinates of the airport runway center relative to the upper left point of the grid, the values of the runway width and height relative to the external airborne image, and the deviation amounts of the four runway corner points, calculating the pixel coordinates of the four airport runway corner points in the external airborne image.

[0009] An embodiment of the present invention provides an apparatus for detecting corner points of an airport runway, and the apparatus includes:

[0010] A prediction module, configured to input the external airborne image to be detected into the airport runway detection model, and predict the coordinates of the airport runway center relative to the upper left point of the grid, the values of the airport runway width and height relative to the external airborne image, and the deviation amounts of the four airport runway corner points;

[0011] A calculation module, configured to calculate the pixel coordinates of the four corner points of the airport runway in the external aircraft-borne image according to the coordinates of the center of the airport runway relative to the upper left point of the grid, the values of the runway width and height relative to the external aircraft-borne image, and the deviation amounts of the four runway corner points.

[0012] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned airport runway corner point detection method is implemented.

[0013] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned airport runway corner point detection method is implemented.

[0014] An airport runway corner point detection method, device, computer device, and storage medium provided by the present invention first input an external aircraft-borne image to be detected into an airport runway detection model, and predict the coordinates of the center of the airport runway relative to the upper left point of the grid, the values of the airport runway width and height relative to the external aircraft-borne image, and the deviation amounts of the four airport runway corner points; according to the coordinates of the center of the airport runway relative to the upper left point of the grid, the values of the runway width and height relative to the external aircraft-borne image, and the deviation amounts of the four runway corner points, calculate the pixel coordinates of the four corner points of the airport runway in the external aircraft-borne image. Since the airport runway detection model trained with corner offsets in the horizontal and vertical directions is used in this application, the airport runway detection model can predict the positions of the four corner points of any irregular quadrilateral in the image. And the application divides the image into SxS grids and selects the grid with the highest confidence to predict the airport runway, which can effectively reduce background interference and false detection rate. Therefore, the detection accuracy of the airport runway corner points can be improved through this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0016] Figure 1 It is a schematic diagram of an irregular quadrilateral box in the prior art;

[0017] Figure 2 It is a flowchart of the airport runway corner point detection method in an embodiment of the present invention;

[0018] Figure 3It is the architecture diagram of the airport runway corner detection method in an embodiment of the present invention;

[0019] Figure 4 It is the training flow chart of the airport runway detection model in an embodiment of the present invention;

[0020] Figure 5 It is the architecture diagram of the airport runway detection model in an embodiment of the present invention;

[0021] Figure 6 It is the schematic diagram of the corner deviation amount in an embodiment of the present invention;

[0022] Figure 7 It is a principle block diagram of the airport runway corner detection device in an embodiment of the present invention;

[0023] Figure 8 It is a schematic diagram of a computer device in an embodiment of the present invention. Specific embodiments

[0024] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0025] In one embodiment, as Figure 2 and Figure 3 shown, a method for detecting airport runway corners is provided, and the method includes the following steps:

[0026] S201, input the external aircraft-mounted image to be detected into the airport runway detection model, and predict the coordinates of the airport runway center relative to the upper left point of the grid, the values of the airport runway width and height relative to the external aircraft-mounted image, and the deviation amounts of the four airport runway corners.

[0027] In this embodiment, after the external aircraft-mounted image is input into the airport runway detection model, the external aircraft-mounted image will be divided into multiple grids, and then the coordinates of the airport runway center relative to the upper left point of each grid, the values of the airport runway width and height relative to the external aircraft-mounted image, and the deviation amounts of the four airport runway corners are predicted.

[0028] The output dimension of the airport runway detection model is SxSxN, where SxS represents the number of divided grids, N is the prediction result corresponding to each grid, and the grid with the largest confidence in the prediction result is taken as the output of the airport runway detection model. Suppose the grid in the i-th row and j-th column has the largest confidence, and the predicted output of the airport runway detection model is:

[0029]

[0030] Among them, is the grid confidence of the i-th row and j-th column, is the coordinate of the airport runway center relative to the upper left point of the grid, The width and height of the airport runway relative to the value of the outboard aircraft-mounted image.

[0031] are the deviation amounts of the four airport runway corner points. Specifically, are the horizontal deviation amounts respectively corresponding to the four airport runway corner points, are the vertical deviation amounts respectively corresponding to the four airport runway corner points.

[0032] It should be noted that the horizontal deviation amount in this embodiment is calculated from the predicted width and height of the runway bounding box, the four coordinates of the largest box, and the coordinates of the four airport runway corner points. The calculation method for the horizontal deviation amount is: the abscissa of each corner point of the largest box minus the abscissa of the airport runway corner point divided by the predicted width of the runway bounding box; the calculation method for the vertical deviation amount is: the ordinate of each corner point of the largest box minus the ordinate of the airport runway corner point divided by the predicted height of the runway bounding box.

[0033] In this embodiment, the airport runway detection model can refer to YOLOv3, uses the darknet53 network, and finally obtains image grids of 13x13, 26x26, and 52x52. Different from YOLOv3, this model does not use anchor boxes, but directly predicts the position, size, and corner offset of the target. At the same time, during the prediction process, non-maximum suppression is not used, but directly the grid data with the largest confidence in the image grid is taken as the prediction result for output.

[0034] S202. According to the coordinate of the airport runway center relative to the upper left point of the grid, the value of the runway width and height relative to the outboard aircraft-mounted image, and the deviation amounts of the four runway corner points, calculate the pixel coordinates of the four airport runway corner points in the outboard aircraft-mounted image.

[0035] Specifically, step S202 for calculating the pixel coordinates of the four airport runway corner points in the outboard aircraft-mounted image includes:

[0036] S2021. According to the coordinate of the airport runway center relative to the upper left point of the grid and the value of the runway width and height relative to the outboard aircraft-mounted image, obtain the center point and width and height of the largest bounding rectangle of the runway.

[0037] For example, if the grid confidence of the i-th row and j-th column is the largest, the predicted output of the airport runway detection model is:

[0038] Among them, are the respective corresponding horizontal deviation amounts of the four airport runway corner points, are the respective corresponding vertical deviation amounts of the four airport runway corner points. is the grid confidence of the i-th row and j-th column, is the coordinate of the airport runway center relative to the upper left point of the grid, The width and height of the airport runway relative to the value of the external aircraft-mounted image.

[0039] Calculate the center point, width and height of the maximum bounding rectangle of the runway according to the following formula:

[0040]

[0041] Among them, is the abscissa of the center point of the maximum bounding rectangle, is the ordinate of the center point of the maximum bounding rectangle; is the width of the maximum bounding rectangle of the runway, is the height of the maximum bounding rectangle of the runway. g w represents the width of each grid, g h represents the height of each grid, W represents the width of the external aircraft-mounted image, and H represents the height of the external aircraft-mounted image.

[0042] S2022. According to the deviation amounts of the four runway corner points and the center point, width and height of the maximum bounding rectangle of the runway, calculate the pixel coordinates of the four corner points of the airport runway in the external aircraft-mounted image.

[0043] In this embodiment, according to the deviation amounts of the four runway corner points and the center point, width and height of the maximum bounding rectangle of the runway the predicted coordinates of the four corner points of the runway can be obtained: the upper left corner point the upper right corner point the lower left corner point the lower right corner point The calculation formulas for the coordinates of each corner point are as follows:

[0044]

[0045] As Figure 3 shown in the flowchart of the airport runway detection model, after collecting the external aircraft-mounted data (i.e., the external aircraft-mounted image), the external aircraft-mounted data is input into the trained model (airport runway detection model), and the center position of the airport runway, the size of the airport runway and the offset amount of the airport runway corner points are predicted. Then, according to the predicted results, the coordinates of the four corner points of the airport runway are calculated, so that the prediction of the angle coordinates of the airport runway is realized through the airport runway detection model, thereby improving the detection accuracy of the airport runway corner points.

[0046] A method for detecting corner points of an airport runway provided by the present invention first inputs an external cabin airborne image to be detected into an airport runway detection model, and predicts the coordinates of the center of the airport runway relative to the upper left point of the grid, the width and height of the airport runway relative to the external cabin airborne image, and the deviation amounts of the four corner points of the airport runway; according to the coordinates of the center of the airport runway relative to the upper left point of the grid, the width and height of the runway relative to the external cabin airborne image, and the deviation amounts of the four corner points of the runway, the pixel coordinates of the four corner points of the airport runway in the external cabin airborne image are calculated. Since the airport runway detection model trained with the corner point offset amounts in the horizontal and vertical directions is used in this application, the airport runway detection model can predict the positions of the four corner points of any irregular quadrilateral in the image, and by dividing the image into SxS grids and taking the grid with the highest confidence to predict the airport runway, the background interference and false detection rate can be effectively reduced. Therefore, the detection accuracy of the corner points of the airport runway can be improved through this application.

[0047] In one embodiment, as Figure 4 and Figure 5 shown, a training process of an airport runway detection model is provided, and the training process of this model is as follows:

[0048] S401, obtain an external cabin airborne sample image and divide the external cabin airborne sample image into multiple grids.

[0049] Among them, the coordinates of the four corner points of the airport runway are marked in the external cabin airborne sample image.

[0050] S402, calculate the true values of the external cabin airborne sample image according to the coordinates of the four corner points of the airport runway.

[0051] Among them, the true values include the grid confidence of each grid, the coordinates of the center of the airport runway of the grid relative to the upper left point of the grid, the width and height of the airport runway relative to the external cabin airborne image, and the deviation amounts of the four corner points of the airport runway. If the center point of the runway box falls in a certain grid, the grid confidence p of this grid is set to 1, the grid confidence p of the grids that do not contain the runway center point is set to 0, and the other label values are also 0, that is, for the grid confidence of the grids that do not contain the runway center, the coordinates of the center of the airport runway of the grid relative to the upper left point of the grid, the width and height of the airport runway relative to the external cabin airborne image, and the deviation amounts of the four corner points of the runway, there is no need to calculate and they are all set to 0.

[0052] Specifically, the calculating the true values of the external cabin airborne sample image according to the coordinates of the four corner points of the airport runway includes:

[0053] S4021, calculate the width w b ,h b and the center point position xb , y b ;

[0054] In this embodiment, the abscissas of the four corner points of the runway in pixels can be represented by x1, x2, x3, x4, and the ordinates of the four corner points of the runway in pixels can be represented by y1, y2, y3, y4.

[0055] After marking the coordinates of the four corner points of the airport runway, it is necessary to obtain the largest rectangle enclosing the runway, including the width w b , h b and the center point position x b , y b , as shown in the following formula:

[0056] x b = [max(x1, x2, x3, x4) + min(x1, x2, x3, x4)] / 2

[0057] y b = [max(y1, y2, y3, y4) + min(y1, y2, y3, y4)] / 2

[0058] w b = max(x1, x2, x3, x4) - min(x1, x2, x3, x4)

[0059] h b = max(y1, y2, y3, y4) - min(y1, y2, y3, y4)

[0060] S4022. According to the width w b , h b of the largest rectangle enclosing the runway and the center point position x b , y b , as well as the width and height of the external aircraft-borne sample image and the width and height of each grid, calculate the coordinates of the center of the airport runway in each grid relative to the upper left point of the grid, and the values of the width and height of the airport runway relative to the external aircraft-borne image.

[0061] Specifically, calculate the coordinates of the center of the airport runway in each grid relative to the upper left point of the grid and the values of the width and height of the airport runway relative to the external aircraft-borne image through the following formula:

[0062] x = (x b % g w ) / g w

[0063] y = (y b % g h ) / g h

[0064] w = wb / W

[0065] h = h b / H

[0066] Wherein, x is the abscissa of the center of the airport runway of each grid relative to the upper left point of the grid, y is the ordinate of the center of the airport runway of each grid relative to the upper left point of the grid, w is the width of the airport runway relative to the value of the out-of-cabin airborne image, and h is the height of the airport runway relative to the value of the out-of-cabin airborne image; g h is the height of each grid, W is the width of the out-of-cabin airborne image, and H is the height of the out-of-cabin airborne image.

[0067] S4023. According to the width w b , h b of the runway surrounded, the four coordinates of the largest square and the coordinates of the four airport runway corner points, calculate the deviation amounts of the four airport runway corner points.

[0068] Specifically, calculate the deviation amounts of the four airport runway corner points through the following formula:

[0069] α x n = s x n / w b

[0070] α y n = s y n / h b

[0071] s x n = |X n - x n |

[0072] s y n = |Y n - y n |

[0073] Wherein, α x n is the horizontal deviation amount, a y n the vertical deviation amount; X n is the abscissa of the nth corner point of the largest square, x n is the abscissa of the nth airport runway corner point, Y n is the ordinate of the nth corner point of the largest square, y n is the ordinate of the nth airport runway corner point. s y n and sy n The meaning in the box is shown in Figure 6 as follows. For example, in Figure 6 , the deviation amount s x 1 of the first airport runway corner point is |X1 - x1|, where X1 is the abscissa of the corner point at the upper left corner of the largest box, and x1 is the abscissa of the corner point at the upper left corner of the runway (the quadrilateral inside the box).

[0074] Furthermore, in this embodiment, after calculating the true quantity of the out-of-cabin airborne sample image according to the coordinates of the four airport runway corner points, data augmentation methods can be used to perform processing methods such as translation, zooming in and out, rotation, and brightness adjustment on the image and the irregular quadrilateral box simultaneously, for increasing the diversity of the dataset. It should be noted that during the processes of translation, zooming in and out, and rotation, if any runway corner point exceeds the image area, the confidence of the grid is set to 0.

[0075] S403, input the out-of-cabin airborne sample image into the airport runway detection model to obtain the prediction quantity of the out-of-cabin airborne sample image.

[0076] Among them, the prediction quantity includes the grid confidence of each grid, the predicted coordinates of the airport runway center of the grid relative to the upper left point of the grid, the predicted values of the airport runway width and height relative to the out-of-cabin airborne image, and the predicted deviation amounts of the four airport runway corner points.

[0077] S404, calculate the loss function value of the prediction quantity and the true quantity, and update the parameters of the airport runway detection model according to the loss function value through backpropagation.

[0078] In an alternative embodiment provided by the present invention, calculating the loss function value of the prediction quantity and the true quantity includes: calculating the loss of the grid confidence through the binary cross-entropy loss function l obj = BCE(pre, truth); calculating the loss of the position and size through the loss function l box = 1 - iou; calculating the loss of the runway corner point deviation amount through the smoothL1 loss function l bias = smoothL1(pre, truth). Among them, iou is the ratio of the intersection value and the union value of the predicted runway area and the true runway area.

[0079] If the grid confidence in the true quantity is 1, determine the weighted values of the l obj , the l box and the l bias as the loss value of the corresponding grid; if the grid confidence in the true quantity is 0, the l objDetermine the loss value corresponding to the grid; add up the loss values of all grids to obtain the loss function value.

[0080] Since there is only one airport runway in the scene, only one target detection box is designed. By dividing the image into SxS grids and taking the grid with the highest confidence as the predicted grid of the airport runway, the situation of background misdetection can be effectively reduced. And because the airport runway detection model trained with the corner offset in the horizontal and vertical directions is used in this application, the airport runway detection model can predict the positions of the four corners of any irregular quadrilateral in the image.

[0081] During the aircraft's approach and landing process, by inputting the external airborne image into the airport runway detection model based on the improved YOLO, the target box of the airport runway is obtained, and at the same time, the horizontal and vertical sliding offsets of the four corners of the airport runway relative to the box are obtained, and then the pixel coordinates of the four corners of the airport runway in the image are obtained. The invention can be used to assist the pilot in the approach and landing task and provide a visual reference for the pilot.

[0082] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0083] In one embodiment, an airport runway corner detection device is provided, and the airport runway corner detection device corresponds one-to-one to the airport runway corner detection method in the above embodiment. As Figure 7 shown, the airport runway corner detection device includes: a prediction module 10 and a calculation module 20. The detailed descriptions of each functional module are as follows:

[0084] The prediction module 10 is configured to input the external airborne image to be detected into the airport runway detection model, and predict the coordinates of the center of the airport runway relative to the upper left point of the grid, the values of the width and height of the airport runway relative to the external airborne image, and the deviation amounts of the four airport runway corners;

[0085] The calculation module 20 is configured to calculate the pixel coordinates of the four corners of the airport runway in the external airborne image according to the coordinates of the center of the airport runway relative to the upper left point of the grid, the values of the runway width and height relative to the external airborne image, and the deviation amounts of the four runway corners.

[0086] In an alternative embodiment, the device further includes:

[0087] An acquisition module 30 is configured to acquire an external airborne sample image, divide the external airborne sample image into multiple grids, and the coordinates of the four airport runway corners are marked in the external airborne sample image;

[0088] The calculation module 20 is configured to calculate the true values of the external-cabin airborne sample images according to the coordinates of the four airport runway corner points, where the true values include the grid confidence of each grid, the coordinates of the airport runway center of the grid relative to the upper left point of the grid, the values of the airport runway width and height relative to the external-cabin airborne image, and the deviation amounts of the four airport runway corner points;

[0089] The prediction module 10 is configured to input the external-cabin airborne sample images into the airport runway detection model to obtain the predicted values of the external-cabin airborne sample images, where the predicted values include the grid confidence of each grid, the predicted coordinates of the airport runway center of the grid relative to the upper left point of the grid, the predicted values of the airport runway width and height relative to the external-cabin airborne image, and the predicted deviation amounts of the four airport runway corner points;

[0090] The training module 40 is configured to calculate the loss function values of the predicted values and the true values, and update the parameters of the airport runway detection model by backpropagation according to the loss function values.

[0091] In an alternative embodiment, the calculation module 20 is specifically configured to:

[0092] Calculate the width w b , h b and the center point position x b , y b ;

[0093] According to the width w b , w b and the center point position x b , y b , as well as the width and height of the external-cabin airborne sample image and the width and height of each grid, calculate the coordinates of the airport runway center of each grid relative to the upper left point of the grid, and the values of the airport runway width and height relative to the external-cabin airborne image;

[0094] According to the width w b , h b of the runway, the four coordinates of the maximum bounding box, and the coordinates of the four airport runway corner points, calculate the deviation amounts of the four airport runway corner points.

[0095] In an alternative embodiment, the calculation module 20 is specifically configured to:

[0096] Calculate the coordinates of the airport runway center of each grid relative to the upper left point of the grid, and the values of the airport runway width and height relative to the external-cabin airborne image through the following formula:

[0097] x = (x b % g w) / g w

[0098] y = (y b %g h ) / g h

[0099] w = w b / W

[0100] h = h b / H

[0101] Wherein, x is the abscissa of the center of the airport runway of each grid relative to the upper left point of the grid, y is the ordinate of the center of the airport runway of each grid relative to the upper left point of the grid, w is the width of the airport runway relative to the value of the outboard airborne image, and h is the height of the airport runway relative to the value of the outboard airborne image;

[0102] g w is the width of each grid, g h is the height of each grid, W is the width of the outboard airborne image, and H is the height of the outboard airborne image.

[0103] In an alternative embodiment, the calculation module 20 is specifically configured to:

[0104] α x n = s x n / w b

[0105] α y n = s y n / h b

[0106] s x n = |X n -x n |

[0107] s y n = |Y n -y n |

[0108] Wherein, α x n is the horizontal deviation amount, α y n vertical deviation amount; X n is the abscissa of the nth corner point of the maximum box, x n is the abscissa of the nth airport runway corner point, Y n is the ordinate of the nth corner point of the maximum box, y nis the ordinate of the nth airport runway corner point.

[0109] In an optional embodiment, the training module 40 is specifically configured to:

[0110] Calculate the loss of grid confidence through the binary cross-entropy loss function l obj = BCE(pre, truth), where BCE(pre, truth) calculates the loss of grid confidence:

[0111] Calculate the loss of position and size through the loss function l box = 1 - iou, where iou is the ratio of the intersection value to the union value of the predicted runway area and the true runway area;

[0112] Calculate the loss of the runway corner point deviation through the smoothL1 loss function l bias = smoothL1(pre, truth);

[0113] If the grid confidence in the true quantity is 1, determine the weighted values of the l obj the l box and the l bias as the loss value of the corresponding grid; if the grid confidence in the true quantity is 0, determine the l obj as the loss value of the corresponding grid;

[0114] Sum the loss values of all grids to obtain the loss function value.

[0115] In an optional embodiment, the calculation module 20 is specifically configured to:

[0116] Obtain the center point and width and height of the runway's largest bounding rectangle according to the coordinates of the airport runway center relative to the upper left point of the grid and the values of the runway width and height relative to the out-of-cabin airborne image;

[0117] Calculate the pixel coordinates of the four airport runway corner points in the out-of-cabin airborne image according to the deviation amounts of the four runway corner points and the center point and width and height of the runway's largest bounding rectangle.

[0118] For the specific limitations of the airport runway corner point detection device, reference can be made to the limitations on the airport runway corner point detection calculation method described above, which will not be elaborated here. Each module in the above airport runway corner point detection device can be implemented in whole or in part through software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0119] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in Figure 8 . The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements an airport runway corner detection method.

[0120] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0121] Input the external airborne image to be detected into the airport runway detection model, and predict the coordinates of the airport runway center relative to the upper left point of the grid, the values of the airport runway width and height relative to the external airborne image, and the deviation amounts of the four airport runway corner points;

[0122] According to the coordinates of the airport runway center relative to the upper left point of the grid, the values of the runway width and height relative to the external airborne image, and the deviation amounts of the four runway corner points, calculate the pixel coordinates of the four corner points of the airport runway in the external airborne image.

[0123] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the following steps are implemented:

[0124] Input the external airborne image to be detected into the airport runway detection model, and predict the coordinates of the airport runway center relative to the upper left point of the grid, the values of the airport runway width and height relative to the external airborne image, and the deviation amounts of the four airport runway corner points;

[0125] According to the coordinates of the airport runway center relative to the upper left point of the grid, the values of the runway width and height relative to the external airborne image, and the deviation amounts of the four runway corner points, calculate the pixel coordinates of the four corner points of the airport runway in the external airborne image.

[0126] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0127] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0128] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention and should all be included in the protection scope of the present invention.

Claims

1. An airport runway corner point detection method, characterized in that The method includes: Inputting the outboard aircraft-mounted image to be detected into the airport runway detection model, and predicting the coordinates of the airport runway center relative to the upper left point of the grid, the values of the width and height of the airport runway relative to the outboard aircraft-mounted image, and the deviation amounts of the four airport runway corner points; the deviation amounts include the horizontal deviation amount obtained by dividing the difference between the abscissa of each corner point of the maximum bounding box and the abscissa of the airport runway corner point by the predicted width of the runway bounding box, and the vertical deviation amount obtained by dividing the difference between the ordinate of each corner point of the maximum bounding box and the ordinate of the airport runway corner point by the predicted height of the runway bounding box; Calculating the pixel coordinates of the four airport runway corner points in the outboard aircraft-mounted image according to the coordinates of the airport runway center relative to the upper left point of the grid, the values of the runway width and height relative to the outboard aircraft-mounted image, and the deviation amounts of the four runway corner points; The calculating the pixel coordinates of the four airport runway corner points in the outboard aircraft-mounted image according to the coordinates of the airport runway center relative to the upper left point of the grid, the values of the runway width and height relative to the outboard aircraft-mounted image, and the deviation amounts of the four runway corner points includes: Obtaining the center point, width, and height of the maximum bounding rectangle of the runway according to the coordinates of the airport runway center relative to the upper left point of the grid and the values of the runway width and height relative to the outboard aircraft-mounted image; Calculating the pixel coordinates of the four airport runway corner points in the outboard aircraft-mounted image according to the deviation amounts of the four runway corner points and the center point, width, and height of the maximum bounding rectangle of the runway.

2. The method according to claim 1, wherein The method further includes: Obtaining an outboard aircraft-mounted sample image, and dividing the outboard aircraft-mounted sample image into multiple grids, where the coordinates of the four airport runway corner points are marked in the outboard aircraft-mounted sample image; Calculating the true quantities of the outboard aircraft-mounted sample image according to the coordinates of the four airport runway corner points, where the true quantities include the grid confidence of each grid, the coordinates of the airport runway center relative to the upper left point of the grid, the values of the runway width and height relative to the outboard aircraft-mounted image, and the deviation amounts of the four airport runway corner points; Inputting the outboard aircraft-mounted sample image into the airport runway detection model to obtain the predicted quantities of the outboard aircraft-mounted sample image, where the predicted quantities include the grid confidence of each grid, the predicted coordinates of the airport runway center relative to the upper left point of the grid, the predicted values of the runway width and height relative to the outboard aircraft-mounted image, and the predicted deviation amounts of the four airport runway corner points; Calculating the loss function value of the predicted quantities and the true quantities, and updating the parameters of the airport runway detection model by backpropagation according to the loss function value.

3. The method according to claim 2, wherein, The calculating the true quantities of the outboard aircraft-mounted sample image according to the coordinates of the four airport runway corner points includes: Calculate the width w b , h b and the center point position x b , y b ; According to the width and height w of the maximum rectangle surrounding the runway b , w b and the center point positions x b , y b , as well as the width and height of the external aircraft-borne sample image and the width and height of each grid, calculate the coordinates of the center of the airport runway relative to the upper left point of each grid, and the values of the width and height of the airport runway relative to the external aircraft-borne image; According to the width and height w of the surrounding runway b , h b , calculate the deviation amounts of the four airport runway corner points based on the coordinates of the four corners of the maximum rectangle and the coordinates of the four airport runway corner points.

4. The method according to claim 3, wherein The width w of the largest rectangle enclosing the runway b , h b and the central point position x b , y b , and the width and height of the external aircraft-mounted sample image, the width and height of each grid, calculate the coordinates of the center of the airport runway of each grid relative to the upper left point of the grid, and the values of the width and height of the airport runway relative to the external aircraft-mounted image, including: Calculating the coordinates of the airport runway center relative to the upper left point of each grid and the values of the runway width and height relative to the outboard aircraft-mounted image through the following formula: x = (x b %g w ) / g w y = (y b %g h ) / g h w = w b / W h = h b / H where x is the abscissa of the airport runway center relative to the upper left point of each grid, y is the ordinate of the airport runway center relative to the upper left point of each grid, w is the value of the runway width relative to the outboard aircraft-mounted image, and h is the value of the runway height relative to the outboard aircraft-mounted image; g w is the width of each grid, g h is the height of each grid, W is the width of the outboard airborne image, and H is the height of the outboard airborne image.

5. The method according to claim 3, wherein According to the width and height w b , h b of the surrounding runway, and the coordinates of the four corners of the maximum square and the coordinates of the four airport runway corner points, calculate the deviation amounts of the four airport runway corner points, including: Calculate the deviation amounts of the four airport runway corner points through the following formula: α x n = s x n / w b α y n = s y n / h b s x n = |X n - x n | s y n = |Y n -y n | Among them, α x n is the horizontal deviation amount, and α y n is the vertical deviation amount; X n is the abscissa of the nth corner point of the largest rectangle, x n is the abscissa of the nth airport runway corner point, Y n is the ordinate of the nth corner point of the largest rectangle, y n is the ordinate of the nth airport runway corner point.

6. The method according to claim 3, wherein The loss function value for calculating the predicted amount and the actual amount includes: Calculate the loss of grid confidence through the binary cross-entropy loss function l obj = BCE(pre, truth), Through the loss function l box = 1 - iou, calculate the loss of position and size, where iou is the ratio of the intersection value to the union value of the predicted runway area and the true runway area; Calculate the loss of the runway corner point deviation through the smoothL1 loss function l bias = smoothL1(pre, truth); If the grid confidence in the true quantity is 1, determine the weighted values of the l obj , the l box and the l bias as the loss value of the corresponding grid; if the grid confidence in the true quantity is 0, determine the l obj as the loss value of the corresponding grid; Add the loss values of all grids to obtain the loss function value.

7. An airport runway corner detection device, characterized in that, The device includes: A prediction module, configured to input the airborne image of the aircraft outside the cabin to be detected into the airport runway detection model, and predict the coordinates of the airport runway center relative to the upper left point of the grid, the width and height of the airport runway relative to the airborne image of the aircraft outside the cabin, and the deviation amounts of the four airport runway corner points; the deviation amounts include the horizontal deviation amount obtained by dividing the difference between the abscissa of each corner point of the maximum bounding box and the abscissa of the airport runway corner point by the predicted width of the bounding runway, and the vertical deviation amount obtained by dividing the difference between the ordinate of each corner point of the maximum bounding box and the ordinate of the airport runway corner point by the predicted height of the bounding runway A calculation module, configured to obtain the center point, width, and height of the maximum bounding rectangle of the runway according to the coordinates of the airport runway center relative to the upper left point of the grid and the values of the runway width and height relative to the airborne image of the aircraft outside the cabin; and calculate the pixel coordinates of the four airport runway corner points in the airborne image of the aircraft outside the cabin according to the deviation amounts of the four runway corner points and the center point, width, and height of the maximum bounding rectangle of the runway.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the airport runway corner point detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the airport runway corner point detection method according to any one of claims 1 to 6.

Images