Airplane runway automatic labeling method based on semantic segmentation

Abstract

The invention relates to an airplane runway automatic labeling method based on semantic segmentation. The method is specially used for labeling an airplane runway area in a landing scene. According to the method, on the basis of simple geometric characteristics of a runway, after a runway area is recognized through a pre-training model, a contour with the maximum area is reserved. And through extraction of the contour vertex, key points marked by the runway contour are determined, and then final marking information is generated. Comprising the following steps: S1, acquiring an input image needing semantic segmentation and annotation; s2, obtaining a binary segmentation image of the airport runway; s3, obtaining all contours in the binary segmentation image; s4, acquiring a contour with the maximum area; and step S5, calculating vertex coordinates of the contour to obtain labeling information. According to the implementation of the method, the accuracy of automatic labeling is remarkably improved, and meanwhile, the workload of manual fine adjustment of the labeling file is reduced, so that the efficiency and accuracy of model fitting are remarkably improved.

Description

Technical Field

[0001] The invention relates to an image processing technology, to an automatic labeling technology of a supervised learning data set, and in particular to an automatic labeling method for an airplane runway based on semantic segmentation. Background Art

[0002] The Chinese patent "Semantic segmentation automatic annotation method, system, storage medium and electronic device" (patent publication number: CN116630633A, patent publication date: 20230822) discloses a semantic segmentation automatic annotation method. Figure 1 As shown in FIG, the method first obtains an input image, and then obtains the text features and image features of the target in the image respectively. These features are combined to obtain the classification label of the target and the corresponding mask, thereby obtaining the annotation information of the image.

[0003] In the automatic annotation of airport runway contours in landing scenarios, since the geometric shape of the runway is relatively regular and simple, usually only four key points are needed to accurately depict its trapezoidal contour. However, if the above method is directly used for annotation, the operation may be more complicated.

[0004] In semi-supervised learning, in order to improve the labeling efficiency, a limited data set is usually used to initially train a model, and then the model is used to batch recognize and label unlabeled images. The labelers then review and screen these automatically generated labeling results, exclude poor quality data, and fine-tune images with inaccurate labels.

[0005] When applying the aforementioned patent to automatically label airport runways, a large number of labeling points may be generated, which will bring many inconveniences to the subsequent manual fine-tuning stage, thereby affecting the overall labeling efficiency. In addition, in semi-supervised learning, the labeling model has not yet achieved ideal generalization capabilities and has an underfitting problem. When the classification effect of some pixels on the runway is poor, the generated labeling results will be too scattered and unable to form a coherent whole, which not only affects the accuracy of the labeling, but also has a negative impact on the subsequent model training. Summary of the invention

[0006] The present invention aims to propose a semantic segmentation automatic labeling method, which is specifically used to label the aircraft runway area in the landing scene. Based on the simple geometric characteristics of the runway, the method identifies the runway area through a pre-trained model and retains the contour with the largest area. By extracting the vertices of this contour, the key points of the runway contour labeling are determined, and then the final labeling information is generated.

[0007] The present invention is achieved by the following technical solutions:

[0008] A method for automatic labeling of an aircraft runway based on semantic segmentation comprises the following steps:

[0009] Step S1, obtaining an input image that needs semantic segmentation and annotation;

[0010] Step S2, obtaining a binary segmentation image of the airport runway;

[0011] Step S3, obtaining all contours in the binary segmentation image;

[0012] Step S4, obtaining the contour with the largest area;

[0013] Step S5, obtaining the vertex coordinates of the contour to obtain annotation information.

[0014] In step S1, an input image is generated by an image generating device.

[0015] The specific steps of step S2 include:

[0016] 1) Obtain pixel classification of input images based on the Deeplabv3+ model;

[0017] Input the image to be annotated into the semantic segmentation model to obtain the classification of each pixel in the image;

[0018] 2) Generate a binary image based on pixel classification;

[0019] According to the image pixel category information output by the semantic segmentation model, the image is re-represented, where the airport runway part is displayed with white pixels and the background part is displayed with black pixels.

[0020] In step S3, all contours in the binary image are obtained through the cv2.findContours() function.

[0021] The specific steps of step S4 include:

[0022] 1) Obtain the minimum / maximum coordinate values ​​of all contours in the horizontal / vertical directions of the image respectively;

[0023] 2) Calculate the difference between the minimum and maximum coordinate values ​​of all contours in the vertical direction of the image;

[0024] 3) Calculate the difference between the minimum and maximum coordinate values ​​of all contours in the horizontal direction of the image;

[0025] 4) Calculate the average value of the difference between the minimum and maximum coordinate values ​​of all contours in the horizontal and vertical directions of the image;

[0026] 5) Obtain the contour with the largest area based on the above average value.

[0027] The specific steps of step S2 include:

[0028] 1) Get the contour length through cv2.arcLength() function;

[0029] 2) According to the contour length, the contour is approximated to a polygon through the cv2.approxPolyDP() function and the vertex coordinates are output.

[0030] The implementation of this method will significantly improve the accuracy of automatic annotation, while reducing the workload of manual fine-tuning of annotation files, thereby significantly improving the efficiency and accuracy of model fitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of the patent method related to the background technology of the present invention;

[0032] Figure 2 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0033] The invention provides an automatic marking method for an airplane runway based on semantic segmentation, the method comprising the following steps: obtaining an input image; obtaining a binary segmentation image of an airport runway; obtaining all contours in the binary segmentation image; obtaining a contour with the largest area; obtaining vertex coordinates of the contour to obtain marking information.

[0034] 1. Obtaining the binary segmentation image of the airport runway includes the following steps:

[0035] Obtain pixel classification of input images based on the Deeplabv3+ model;

[0036] Generate a binary image based on pixel classification, with the runway portion represented in white and the rest in black.

[0037] 2. Obtaining all contours in the binary segmentation image includes the following steps:

[0038] Get all the contours in the binary image through cv2.findContours() function.

[0039] 3. Obtaining the contour with the largest area includes the following steps:

[0040] Get the minimum / maximum coordinate values ​​of all contours in the horizontal / vertical directions of the image respectively;

[0041] Calculate the difference between the minimum and maximum coordinate values ​​of all contours in the vertical direction of the image;

[0042] Calculate the difference between the minimum and maximum coordinate values ​​of all contours in the horizontal direction of the image;

[0043] Calculate the average value of the difference between the minimum and maximum coordinate values ​​of all contours in the horizontal and vertical directions of the image;

[0044] ·According to the above average values, the contour with the largest area is obtained.

[0045] 4. Obtaining the vertex coordinates of the contour includes the following steps:

[0046] Get the contour length through cv2.arcLength() function;

[0047] According to the contour length, the contour is approximated to a polygon through the cv2.approxPolyDP() function,

[0048] Output vertex coordinates.

[0049] The technical solution in the embodiment of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiment of the present invention. Figure 2 As shown, the method for automatic marking of aircraft runways based on semantic segmentation described in the present invention includes steps S1 to S5.

[0050] Step S1, obtaining an input image.

[0051] Specifically, firstly, an input image that needs semantic segmentation annotation is obtained. The input image may be generated by a generating device, such as a camera.

[0052] Step S2: Obtain a binary segmented image of the airport runway.

[0053] Specifically, obtaining a binary segmentation image of an airport runway includes the following steps.

[0054] 1) Obtain pixel classification of input images based on the Deeplabv3+ model.

[0055] The Deeplabv3+ model has been pre-trained using a small amount of manually annotated data sets combined with data augmentation technology. It is an underfitting model whose performance still needs to be improved.

[0056] The image to be annotated is input into the semantic segmentation model to obtain the classification of each pixel in the image.

[0057] 2) Generate a binary image based on pixel classification.

[0058] According to the image pixel category information output by the semantic segmentation model, the image is re-represented, where the airport runway part is displayed with white pixels and the background part is displayed with black pixels.

[0059] Step S3: Obtain all contours in the binary segmentation image.

[0060] Specifically, all contours in the binary image are obtained through the cv2.findContours() function.

[0061] The findContours() function is used in OpenCV to find contours in an image. It accepts three parameters: image, mode, and method; and returns two values: contours and hierarchy.

[0062] The input value image is a binary image, usually a grayscale image or an image after thresholding.

[0063] The input value mode is the contour detection mode, which is set to cv2.RETR_LIST, which means detecting all contours and saving them in a list.

[0064] The input value method is the contour approximation method, which is set to cv2.CHAIN_APPROX_NONE, that is, all contour points are stored, and the pixel position difference between two adjacent points does not exceed 1.

[0065] The return value contours is the detected contour, which is a Numpy array.

[0066] The return value hierarchy is the hierarchical information of the contour, which can be used to analyze the relationship between contours.

[0067] Step S4: Obtain the contour with the largest area.

[0068] Specifically, obtaining the contour with the largest area includes the following steps.

[0069] 1) Use np.min() and np.max() functions to obtain the maximum coordinate X of each contour along the X axis in the image. max , the minimum coordinate X along the X axis min , the maximum coordinate Y along the Y axis max , the minimum coordinate Y along the Y axis min .

[0070] 2) Calculate the maximum coordinate Y of each contour along the Y axis in the image max With the minimum coordinate Y min The difference D Y .

[0071] D Y =Y max -Y min

[0072] 3) Calculate the maximum coordinate X of each contour along the X-axis in the image max With the minimum coordinate X minThe difference D X .

[0073] D X =X max -X min

[0074] 4) Calculate the above two differences D for each contour respectively Y With D X The average value A.

[0075]

[0076] 5) By comparing the above average value A, the contour with the largest area is obtained.

[0077] Step S5, obtaining the vertex coordinates of the contour to obtain annotation information.

[0078] Specifically, obtaining the vertex coordinates of the contour includes the following steps.

[0079] 1) Get the length of the contour through the cv2.arcLength() function;

[0080] The arcLength() function is a function in OpenCV that is used to calculate the perimeter of a contour or the length of a closed curve. It accepts two parameters, curve and closed; and returns a parameter, length.

[0081] The input value curve is the input curve or contour, usually a Numpy array consisting of points.

[0082] The input value closed is a Boolean value that specifies whether the curve is closed. If it is set to True, the curve is considered closed and the perimeter of the closed curve is calculated. Otherwise, the length of the open curve is calculated.

[0083] The return value length indicates the calculated length of the curve.

[0084] 2) According to the contour length, the contour is approximated to a polygon through the cv2.approxPolyDP() function and the vertex coordinates are output.

[0085] The cv2.approxPolyDP() function is a function in OpenCV that is used to perform polygonal approximation on contours or curves. It accepts three parameters: curve, epsilon, and closed; and returns a parameter: approx.

[0086] The input value curve is the input curve or contour, usually a Numpy array consisting of points.

[0087] The input value epsilon is a parameter that controls the approximation accuracy. It is a distance threshold that indicates the distance from the original curve at which points are omitted.

[0088] The input value closed is a Boolean value that specifies whether the curve is closed. If it is set to True, the curve is considered closed, otherwise the curve is open.

[0089] The return value approx is a Numpy array representing the vertices of the approximation polygon. This array is the key points required for automatic runway target marking.

[0090] The method for automatic labeling of runways based on semantic segmentation described in the present invention can automatically identify and generate high-precision coordinates of key vertices of the runway contour for runway objects, thereby realizing automatic labeling of semantic segmentation. In the automatic labeling task for runway targets, this method greatly reduces the number of labeling points in the image mask, simplifies the operational complexity during manual fine-tuning, effectively improves the labeling efficiency of the data set, reduces the cost of manual labeling, and speeds up the training process of the model. In addition, this method obtains the labeling information by obtaining the contour with the largest area, effectively avoiding the problem of overly dispersed labeling results caused by poor model fitting, thereby improving the accuracy of the labeling results.

[0091] The protection scope of the present invention is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, replacing or adjusting the steps of the prior art based on the principles of the present invention is included in the protection scope of the present invention. This includes but is not limited to the following aspects: using different semantic segmentation models; using similar binary image display methods; using other methods to obtain contour information in images; using other methods to calculate the contour with the largest area; using other methods to obtain the coordinates of contour vertices; and other adjustments made by adding or subtracting steps.

Claims

1. A method for automatic labeling of aircraft runways based on semantic segmentation, characterized by: The following steps are involved: Step S1, obtaining an input image that needs semantic segmentation and annotation; Step S2, obtaining a binary segmentation image of the airport runway; Step S3, obtaining all contours in the binary segmentation image; Step S4, obtaining the contour with the largest area; Step S5, obtaining the vertex coordinates of the contour to obtain annotation information.

2. The method for automatic marking of aircraft runways based on semantic segmentation according to claim 1, characterized in that: In step S1, an input image is generated by an image generating device.

3. The method for automatic marking of aircraft runways based on semantic segmentation according to claim 1, characterized in that: The specific steps of step S2 include: 1) Obtain pixel classification of input images based on the Deeplabv3+ model; Input the image to be annotated into the semantic segmentation model to obtain the classification of each pixel in the image; 2) Generate a binary image based on pixel classification; According to the image pixel category information output by the semantic segmentation model, the image is re-represented, where the airport runway part is displayed with white pixels and the background part is displayed with black pixels.

4. The method for automatic marking of aircraft runways based on semantic segmentation according to claim 1, characterized in that: In step S3, all contours in the binary image are obtained through the cv2.findContours() function.

5. The method for automatic marking of aircraft runways based on semantic segmentation according to claim 1, characterized in that: The specific steps of step S4 include: 1) Obtain the minimum / maximum coordinate values ​​of all contours in the horizontal / vertical directions of the image respectively; 2) Calculate the difference between the minimum and maximum coordinate values ​​of all contours in the vertical direction of the image; 3) Calculate the difference between the minimum and maximum coordinate values ​​of all contours in the horizontal direction of the image; 4) Calculate the average value of the difference between the minimum and maximum coordinate values ​​of all contours in the horizontal and vertical directions of the image; 5) Obtain the contour with the largest area based on the above average value.

6. The method for automatic marking of aircraft runways based on semantic segmentation according to claim 1, characterized in that: The specific steps of step S2 include: 1) Get the contour length through cv2.arcLength() function; 2) According to the contour length, the contour is approximated to a polygon through the cv2.approxPolyDP() function and the vertex coordinates are output.

Images