A runway contour line acquisition method based on maximum area

Abstract

The present application belongs to the field of image processing, and particularly relates to a runway contour line acquisition method based on maximum area. First, the runway position information is acquired through target detection; then the runway contour is segmented through a semantic segmentation neural network, the maximum area connected region in the segmented runway contour is extracted, and other wrong connected regions are excluded; finally, the obtained runway contour information is combined with a least square and a centerline fitting algorithm to obtain runway contour line information.

Description

Technical Field

[0001] This invention belongs to the field of image processing, specifically relating to a method for obtaining runway contour lines based on the maximum area. Background Technology

[0002] With the continuous advancement of image acquisition and processing technologies, the resolution of images that can be acquired is increasing, and the processing speed is accelerating. Extracting target objects from acquired images has become a crucial application. Airports, as an important type of artificial landform, serve as vital reference points for drone takeoffs and landings. Currently, most research focuses on airport identification itself, primarily aiming to locate airports. However, research on the automated extraction of airport outlines is insufficient. Summary of the Invention

[0003] Purpose of the invention: To provide a method for obtaining runway contour lines based on the maximum area.

[0004] Technical solution:

[0005] A method for obtaining runway contour lines based on maximum area includes:

[0006] A single frame of image is read and fed into the object detection neural network;

[0007] Obtain the results of the object detection neural network and check whether the results are valid;

[0008] When the target detection neural network yields a valid result, an image containing the target is cropped and fed into the semantic segmentation neural network.

[0009] The results of the semantic segmentation neural network are obtained. The left, right, and bottom edge regions of the runway region in the results of the semantic segmentation neural network are binarized. The binarization process will assign 1 to gray values ​​greater than the threshold and 0 to gray values ​​less than the threshold.

[0010] Set label values ​​for the points with a pixel of 1 in the left, right, and bottom edge regions of the binarized image, and determine the maximum area connected region of the left, right, and bottom edges based on the label values;

[0011] For the largest connected region in the left, right, and bottom edge regions respectively, traverse the pixels in the largest connected region of the left edge region and find the first pixel with a value of 1 on both sides as the elements of the point set on both sides.

[0012] The slope and intercept of the line corresponding to the elements in the two point sets are obtained by performing least squares fitting on the elements in the two point sets respectively, as well as the coordinates of the vertex and the base point;

[0013] Add the vertex and bottom pixel coordinates of the lines corresponding to the elements in the point sets on both sides, and take the average value to obtain the vertex and bottom pixel coordinates of the current edge contour.

[0014] Furthermore, label values ​​are assigned to the pixels with a pixel value of 1 in the left, right, and bottom edge regions of the binarized image, and the maximum area connected region of the left, right, and bottom edges is determined based on the label values, specifically including:

[0015] Scan sequentially until the first pixel with a value of 1 is found, and set the label of that pixel to 1;

[0016] Continue scanning the pixels, find the next pixel with a value of 1, and check the four pixels above, below, left, and right of that pixel;

[0017] When the values ​​of the pixels above, below, left, and right of a pixel are all 0, increment the label value of that pixel by one.

[0018] If there is a pixel with a value of 1 in the pixels above, below, left, or right of the current pixel, assign the label of that pixel with a value of 1 to the current pixel.

[0019] When multiple adjacent pixels have a value of 1, the smaller label value is selected and assigned to the current pixel.

[0020] After one scan is completed, a second scan is performed to update the label of each point with the smallest label value in the connected region;

[0021] Determine each connected region based on the updated label values;

[0022] Calculate the area of ​​each connected component and find the connected component with the largest area.

[0023] Furthermore, each connected region is determined based on the updated label values, specifically as follows:

[0024] If any two adjacent points have different labels, then these two points are considered to be in the same connected region.

[0025] Furthermore, for the largest connected region in the left-side line region, traverse the pixels in the largest connected region of the left-side line region, and find the first pixel with a value of 1 on both sides as the elements of the point sets on both sides, specifically including:

[0026] Traverse the pixels in the largest connected region of the left line area from left to right and top to bottom, and find the first pixel with a value of 1 in each row as the element of the left point set;

[0027] Traverse the pixels in the largest connected region of the left line area from right to left and top to bottom, and find the first pixel with a value of 1 in each row as the element of the right point set.

[0028] Furthermore, for the largest connected region in the right-side line region, traverse the pixels in the largest connected region in the left-side line region, and find the first pixel with a value of 1 on both sides as elements in the point sets on both sides. Specifically, this includes:

[0029] Traverse the pixel values ​​of the largest connected region of the right line from left to right and top to bottom, and find the points with a pixel value of 1 in each row as elements of the left point set;

[0030] Traverse the pixel values ​​of the largest connected region of the right line from right to left and top to bottom, and find the points with a pixel value of 1 in each row as elements of the right point set.

[0031] Furthermore, for the largest connected region in the edge region, traverse the pixels in the largest connected region of the left edge region, and find the first pixel with a value of 1 on both sides as the elements of the point sets on both sides, specifically including:

[0032] Traverse the pixels in the largest connected region of the bottom edge area from top to bottom and from left to right, and find the first pixel with a value of 1 in each column as the element of the top point set.

[0033] Traverse the pixels in the largest connected region of the bottom edge area from bottom to top and from left to right, and find the first pixel with a value of 1 in each column as the element of the bottom point set.

[0034] Furthermore, the method also includes:

[0035] Based on the vertical pixel coordinates of the left vertex of the bottom edge outline, correct the vertical coordinates of the vertex of the left edge outline closest to the bottom edge, and update the horizontal coordinates based on the vertical coordinates.

[0036] Furthermore, the method also includes:

[0037] Based on the vertical pixel coordinates of the right vertex of the bottom edge outline, correct the vertical coordinates of the vertex of the right edge outline closest to the bottom edge, and update the horizontal coordinates based on the vertical coordinates.

[0038] Beneficial effects:

[0039] First, runway location information is obtained through object detection; then, the runway contour is segmented using a semantic segmentation neural network, and the largest connected region in the segmented runway contour is extracted to exclude other erroneous connected regions; finally, the obtained runway contour information is combined with least squares and midline fitting algorithms to obtain runway contour line information. Attached Figure Description

[0040] Figure 1 This is a flowchart of a method for obtaining the runway outline based on the maximum area. Detailed Implementation

[0041] This invention relates to image processing technology, target recognition technology, and line fitting technology, and more specifically, to a method for obtaining runway contour lines based on the maximum area.

[0042] This method establishes a runway contour acquisition method based on the maximum area. First, runway location information is obtained through object detection; then, the runway contour is segmented using a semantic segmentation neural network, and the connected regions with the largest area in the segmented runway contour are extracted, while other erroneous connected regions are excluded; finally, the obtained runway contour information is combined with least squares and midline fitting algorithms to obtain the runway contour line information.

[0043] The technical problem to be solved by this invention is to obtain the maximum area connected region in the detected runway area and obtain the runway outline, and provides a method for obtaining the runway outline based on the maximum area.

[0044] According to the present invention, a method for obtaining runway contour lines based on maximum area is provided, the technical solution of which includes:

[0045] The runway location information was detected using an object detection neural network. An image was then extracted based on the object detection results and fed into a semantic segmentation neural network. After obtaining the semantic segmentation results, the results were first binarized to separate the various categories. Then, the binarized images of each category were scanned sequentially. The first pixel with a value of 1 was identified and its label was set to 1. The scanning continued, and if the left and upper neighbor pixels of a pixel were invalid, a new label was assigned, incrementing the label by 1. If either the left or upper neighbor pixel had a valid value, that pixel's label was assigned to the pixel's label value. If both were valid values, the smaller label value was assigned to the pixel. After one scan, a second scan was performed to update the label of each point with the smallest label in the set. Finally, the area of ​​each connected component was calculated, and the connected component with the largest area among all categories was selected.

[0046] To obtain the largest connected component of the left and right edge categories, the pixel values ​​of the edge category are traversed from top to bottom along the y-axis, finding points with a pixel value of 1 in both left and right directions. The pixel coordinates of these points are then stored in containers for the corresponding left and right point sets. The least squares algorithm is then used to fit the slope and intercept of the lines for the largest connected component of the edge category to these point sets. The maximum and minimum y-axis pixel coordinates of the left and right point sets are then substituted into the left and right lines obtained by the least squares algorithm to obtain the left x-axis pixel coordinates of each line. Finally, the maximum and minimum y-axis and x-axis pixel coordinates of the left and right lines are summed and their average is taken to obtain the left vertex and bottom pixel coordinates of the center line of the left and right edge categories.

[0047] The method for obtaining the maximum connected component of the bottom edge is the same as that for the left and right edges, except that the traversal is changed from along the y-axis to along the x-axis, and the point set is changed from left and right to up and down. The point set obtained by clicking is the straight line formed by the maximum and minimum x-axis pixel coordinates of the bottom edge. I will not go into details here.

[0048] Finally, the obtained y-axis pixel coordinates of the left and right endpoints of the bottom edge are substituted into the equations of the left and right edge lines respectively to update the coordinates of the bottom points of the left and right edge lines. This yields the complete runway outline. Figure 1 The specific steps of the method for obtaining the runway outline based on the maximum area are as follows:

[0049] Step 1: Input a frame of image into the object detection neural network;

[0050] Step 2: Obtain the results of the object detection neural network and check whether the results are valid.

[0051] Step 3: When the result is valid, crop the image based on the target detection result and feed the cropped image into the semantic segmentation neural network;

[0052] Step 4: Obtain the result of the semantic segmentation neural network. Perform binarization on the left, right, and bottom edges of the runway region in the result of the semantic segmentation neural network. The binarization process will assign 1 to gray values ​​greater than the threshold and 0 to gray values ​​less than the threshold.

[0053] Perform steps 5-12 on the left, right, and bottom edges of the binarized image respectively to obtain the maximum area of ​​the connected components of the left, right, and bottom edges.

[0054] Step 5: Scan the binarized images of each category (left, right, bottom edge) in sequence, find the first pixel with a value of 1, and set the label of that pixel to 1;

[0055] Step 6: Continue scanning pixels to find the next pixel with a value of 1, and check the four pixels above, below, left, and right of that pixel;

[0056] Step 7: When the values ​​of the pixels above, below, left, and right of a pixel are all 0, assign a new label to that pixel and increment the label by 1; the initial value of the label is 0.

[0057] Step 8: When there is a pixel with a value of 1 above, below, left, or right of a given pixel, assign the label of that pixel with a value of 1 to that pixel. If multiple adjacent pixels have a value of 1, select the smallest (non-zero) label value and assign it to that pixel. If the pixel is on an edge, consider all surrounding pixels.

[0058] Step 9: Check all points. If two adjacent points have different label values, then consider these two pixels as the same connected region.

[0059] Step 10: After one scan is completed, perform a second scan on the current image (repeat steps 5-10). Update the label of each point to the smallest label in the connected region, and determine each connected region based on the smallest label (specifically determined according to the definition in step 9).

[0060] Step 11: Calculate the area of ​​each connected component and obtain the connected component with the largest area;

[0061] Step 12: Repeat steps 5-11 to obtain the maximum area of ​​the 5-connected regions of the left, right, and edge lines respectively;

[0062] Step 13: Traverse the pixels in the largest connected region of the left line from left to right and from top to bottom, and from right to left and from top to bottom. Stop when you find the point with a pixel value of 1 in each row, and store it in the corresponding left and right point set containers.

[0063] Step 14: Obtain the slope and intercept of the straight line for the left and right point sets of the left side profile using the least squares fitting algorithm;

[0064] Step 15: After obtaining the maximum and minimum y-axis pixel coordinates of the left and right point sets, substitute them into the respective line equations of the left and right point sets to obtain the corresponding x-axis pixel coordinates, and use them as the vertex and bottom pixel coordinates of the left and right point sets of the left line, respectively; (where the positive y-axis is the image...)

[0065] The vertical direction is downward, and the positive x-axis is the horizontal direction to the right of the image.

[0066] Step 16: Add the vertex and base pixel coordinates of the line generated from the left and right point sets, and take the result.

[0067] By taking their average, we can obtain the pixel coordinates of the vertices and the bottom point of the left outline.

[0068] Step 17: Obtain the outline of the right line in the same way as the left line. Traverse the pixel values ​​of the largest connected component of the right line from left to right and top to bottom, and from right to left and top to bottom, to find...

[0069] Stop at each point where the pixel value is 1 and store it in the corresponding left and right point set containers; Step 18: Obtain the slope and intercept of the straight line of the left and right point sets of the right line contour respectively by using the least squares fitting algorithm.

[0070] Step 19: After obtaining the maximum and minimum y-axis pixel coordinates in the left and right point sets, substitute them into the respective line equations of the left and right point sets to obtain the corresponding x-axis pixel coordinates, and then proceed accordingly.

[0071] The vertex and basal pixel coordinates of the left and right point sets of the right side outline;

[0072] Step 20: The bottom edge is traversed differently from the left and right edges. It traverses the pixel values ​​in the largest connected component of the bottom edge from top to bottom and from left to right and from bottom to top and from left to right. It stops after finding the point with a pixel value of 1 in each column and stores it in the corresponding upper and lower point set containers.

[0073] Step 21: Obtain the slope and intercept of the straight line of the upper and lower point sets of the bottom edge contour using the least squares fitting algorithm;

[0074] Step 22: After obtaining the maximum and minimum x-axis pixel coordinates of the upper and lower point sets, substitute them into the respective line equations of the upper and lower point sets to obtain the corresponding y-axis pixel coordinates, and use them as the vertex and bottom pixel coordinates of the left and right point sets of the bottom edge contour line in turn.

[0075] Step 23: Since the left and right sidelines intersect with the bottom sideline, the maximum y-axis pixel coordinates of the corresponding left and right track outlines are corrected by using the obtained y-axis pixel coordinates of the left and right endpoints of the bottom sideline, and their bottom point coordinates are updated.

Claims

1. A method for obtaining runway contour lines based on maximum area, characterized in that, include: A single frame of image is read and fed into the object detection neural network; Obtain the results of the object detection neural network and check whether the results are valid; When the target detection neural network yields a valid result, an image containing the target is cropped and fed into the semantic segmentation neural network. The results of the semantic segmentation neural network are obtained. The left, right, and bottom edge regions of the runway region in the results of the semantic segmentation neural network are binarized. The binarization process will assign 1 to gray values ​​greater than the threshold and 0 to gray values ​​less than the threshold. Label values ​​are assigned to pixels with a value of 1 in the left, right, and bottom edge regions of the binarized image. Based on these label values, the largest connected region in each region is determined. Specifically, the process involves: sequentially scanning to find the first pixel with a value of 1 and setting its label to 1; continuing to scan for the next pixel with a value of 1 and checking its four adjacent pixels; incrementing the label of a pixel when all four adjacent pixels have a value of 0; assigning the label of a pixel with a value of 1 to the current pixel when one of its adjacent pixels has a value of 1; assigning the smallest label value to the current pixel when multiple adjacent pixels have a value of 1; updating the label of each pixel to the smallest label value in the connected region after one scan; determining the connected regions based on the updated label values; calculating the area of ​​each connected region and obtaining the largest connected region. For the largest connected region in the left, right, and bottom edge regions respectively, traverse the pixels in the largest connected region of the left edge region and find the first pixel with a value of 1 on both sides as the elements of the point set on both sides. The slope and intercept of the line corresponding to the elements in the two point sets are obtained by performing least squares fitting on the elements in the two point sets respectively, as well as the coordinates of the vertex and the base point; Add the vertex and bottom pixel coordinates of the lines corresponding to the elements in the point sets on both sides, and take the average value to obtain the vertex and bottom pixel coordinates of the current edge contour.

2. The method for obtaining runway contour lines based on maximum area according to claim 1, characterized in that, The connected regions are determined based on the updated label values, specifically: If any two adjacent points have different labels, then these two points are considered to be in the same connected region.

3. The method for obtaining runway contour lines based on maximum area according to claim 2, characterized in that, For the largest connected region in the left line region, traverse the pixels in the largest connected region of the left line region, and find the first pixel with a value of 1 on both sides as the elements of the point sets on both sides. Specifically, this includes: Traverse the pixels in the largest connected region of the left line area from left to right and top to bottom, and find the first pixel with a value of 1 in each row as the element of the left point set; Traverse the pixels in the largest connected region of the left line area from right to left and top to bottom, and find the first pixel with a value of 1 in each row as the element of the right point set.

4. The method for obtaining runway contour lines based on maximum area according to claim 2, characterized in that, For the largest connected region in the right-side region, traverse the pixels in the largest connected region in the left-side region, and find the first pixel with a value of 1 on both sides as elements in the point sets on both sides. Specifically, this includes: Traverse the pixel values ​​of the largest connected region of the right line from left to right and top to bottom, and find the points with a pixel value of 1 in each row as elements of the left point set; Traverse the pixel values ​​of the largest connected region of the right line from right to left and top to bottom, and find the points with a pixel value of 1 in each row as elements of the right point set.

5. The method for obtaining runway contour lines based on maximum area according to claim 2, characterized in that, For the largest connected region in the edge region, traverse the pixels in the largest connected region of the left edge region, and find the first pixel with a value of 1 on both sides as the elements of the point sets on both sides. Specifically, this includes: Traverse the pixels in the largest connected region of the bottom edge area from top to bottom and from left to right, and find the first pixel with a value of 1 in each column as the element of the top point set. Traverse the pixels in the largest connected region of the bottom edge area from bottom to top and from left to right, and find the first pixel with a value of 1 in each column as the element of the bottom point set.

6. The method for obtaining runway contour lines based on maximum area according to claim 5, characterized in that, The method further includes: Based on the vertical pixel coordinates of the left vertex of the bottom edge outline, correct the vertical coordinates of the vertex of the left edge outline closest to the bottom edge, and update the horizontal coordinates based on the vertical coordinates.

7. The method for obtaining runway contour lines based on maximum area according to claim 1, characterized in that, The method further includes: Based on the vertical pixel coordinates of the right vertex of the bottom edge outline, correct the vertical coordinates of the vertex of the right edge outline closest to the bottom edge, and update the horizontal coordinates based on the vertical coordinates.

Images