Airport runway light recognition and location method based on contour space feature analysis

Abstract

The present invention discloses a method for identifying and locating airport runway lights by contour space feature analysis, comprising the following steps: obtaining image samples of multiple target runway lights in advance, extracting contours from the image samples to create offline runway light contour templates, and merging the various runway light contour templates into a template library; obtaining real-time images of target runway lights online; performing preprocessing operations on the real-time images to generate corresponding corrected images; performing pixel gradient and distribution law analysis on the corrected images to generate a binary geometric feature map; performing similarity measurement on the runway light contour template and the geometric feature map, and identifying whether the real-time image contains runway lights based on the measurement results, and locating the pixel position and orientation of the real-time image. The present invention effectively retains the stable structure of the image by mapping the template matching process from the unstable RGB space to the stable contour space, and effectively resists interference from factors such as changes in ambient lighting and contamination of the runway light appearance.

Description

Technical Field

[0001] The present invention relates to the technical field of airport runway light visual recognition and positioning, and in particular to an airport runway light recognition and positioning method based on contour space feature analysis. Background Art

[0002] Airport runway lights, also known as navigation lights, are one of the important equipment to ensure pilots can fly at night and in complex weather conditions. Airport runway guidance lights provide visual guidance signals for aircraft takeoff and landing at night, in low visibility or in bad weather conditions, ensuring safe takeoff and landing of aircraft. When the aircraft takes off and lands, the tires generate high temperatures due to friction, causing the tread to melt. The melted rubber will adhere to the runway lights with the airflow. The high frequency of takeoffs and landings causes the clay to quickly cover the surface of the runway light translucent mirror, making the surface of the runway light dirty, resulting in a decrease in the light intensity of the emitted light components, affecting flight safety. In order to ensure flight safety, it is necessary to clean the lamps regularly, especially the translucent mirror components, but the high-density takeoffs and landings leave less and less time for cleaning the runway lights.

[0003] At present, the cleaning and maintenance of airport runway lights mainly take place during the idle period of the runway at night. General airports still use the traditional cleaning method, that is, manual towing equipment for regular manual cleaning and inspection. The runway lights are mainly cleaned by applying detergent on the surface of the light-transmitting mirror, and then high-pressure washing and manual mechanical scraping. This method has a low degree of automation, low efficiency, and high labor costs. A small number of large airports currently use semi-autonomous cleaning equipment to clean runway lights. It is necessary to manually drive above the runway light, manually aim at the light-transmitting mirror part to start cleaning, and judge the cleaning completion based on experience. Then drive to the next runway light until the cleaning is completed. This type of cleaning mechanism generally uses high-pressure water flow, high-temperature steam, dry powder spraying or dry ice blasting. To a certain extent, the cleaning efficiency has been improved, but labor costs are still required.

[0004] Therefore, realizing autonomous cleaning and maintenance of runway lights is of great significance to reducing costs and increasing efficiency of airport operations and ensuring flight safety. Among them, the robot automatic cleaning system is the most ideal solution to realize autonomous cleaning and maintenance of runway lights, but the premise of this solution is to accurately identify and locate the images of airport runway lights.

[0005] In the relevant field of visual detection technology, target recognition and positioning based on template matching is the most common technical solution. This solution first generates an image template library of the target object offline, then traversally calculates the similarity between the image template and the online detection image during the detection process, determines whether the online image contains the target object, and finally determines the position and orientation information of the target object in the online image if it contains the target object. However, the existing template matching method has its own limitations. It generally measures the similarity between the image template and the online image in the RGB space, and is heavily dependent on matching conditions such as image texture features, lighting conditions, appearance integrity and neatness. It is difficult to deal with the situation where airport runway lights are easily contaminated during long-term use, resulting in inconsistency between the appearance and the template image, which is manifested in many matching restrictions and poor robustness. Summary of the invention

[0006] In view of the above problems, the present invention proposes a method for identifying and locating airport runway lights based on contour space feature analysis, which mainly solves the problem that the template matching method is not suitable for the detection task of runway lights.

[0007] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0008] A method for identifying and locating airport runway lights based on contour space feature analysis comprises the following steps:

[0009] Acquire image samples of a plurality of target runway lights in advance, extract outlines from the image samples to create offline runway light outline templates, and merge the runway light outline templates into a template library;

[0010] Acquire a real-time image of the target runway light online;

[0011] Performing a preprocessing operation on the real-time image to generate a corresponding corrected image;

[0012] Analyzing pixel gradient and distribution rules of the corrected image to generate a binary geometric feature map;

[0013] The runway light outline template and the geometric feature map are measured for similarity, and whether the real-time image contains runway lights is identified based on the measurement result, and the pixel position and orientation of the real-time image are located.

[0014] In some embodiments, the image sample and the real-time image are derived from a visual acquisition device of the same configuration.

[0015] In some embodiments, the process of creating the runway light outline template includes:

[0016] The image samples are preprocessed and contours are extracted. The extracted contours are screened, contour point sets are extracted from the screened contours, a shape model is created using the contour point sets as input, a rotation amount is applied to the shape model according to a preset rotation resolution and the model is saved, thereby obtaining the runway light contour template with multiple rotation angles and binarization.

[0017] In some embodiments, the real-time image acquisition process includes:

[0018] The target runway light is illuminated by a line scanning light source, the line scanning light source is placed above and to the side of the target runway light, and the incident angle of the line scanning light source forms an obtuse angle with the front face of the target runway light;

[0019] The image acquisition device obtains calibration parameters after calibration, and the image acquisition device acquires an RGB image;

[0020] The RGB image is converted into the real-time image in grayscale.

[0021] In some embodiments, the preprocessing operation of the real-time image includes:

[0022] Performing nonlinear distortion correction on pixel coordinate values ​​of the real-time image according to the calibration parameters;

[0023] The real-time image after distortion correction adopts bilateral filtering to reduce image sharp noise to generate the corrected image.

[0024] In some embodiments, the geometric feature map generation process includes:

[0025] Scaling the corrected image according to a preset ratio to obtain a reduced image;

[0026] Calculating the horizontal angles and gradient values ​​of all pixels in the reduced image, and calculating the horizontal line field of the corrected image according to the horizontal angles and the gradient values;

[0027] The pixels of the reduced image are sorted and retrieved according to the gradient value, and then the gradient value corresponding to the pixel is used as a seed to perform contour edge search and clustering operations, wherein the rule of the search and clustering operation is to determine whether the detection pixels adjacent to the current pixel belong to the same contour edge area according to the similarity of the horizontal angle value of the seed point, and iterate until the growth and expansion of the seed are completed, and the line segment extraction is completed; finally, all the extracted line segments are merged and enlarged and restored to form a contour edge;

[0028] An all-zero matrix is ​​introduced, the number of rows and columns of the all-zero matrix is ​​equal to the real-time image, and then all the line segments are traversed, the coordinate values ​​of the starting point and the end point of the line segment are enlarged and restored according to the proportional factor, and according to the pixel coordinates and pixel occupancy of the line segment after each enlarged size, the value of the corresponding pixel on the all-zero matrix is ​​set to 1 to obtain the binary geometric feature map.

[0029] In some embodiments, the similarity measurement step includes:

[0030] Extracting the runway light outline template from the template library, traversing the runway light outline template in sequence according to a preset step length, intercepting each of the runway light outline templates into multiple matrices and saving them in a sub-matrix, traversing the geometric feature map in sequence according to a preset step length, intercepting an area with the same size and position as the sub-matrix from the geometric feature map as a to-be-matched area, calculating a point-to-point distance between the to-be-matched area and the sub-matrix, and using the distance as the image similarity;

[0031] In some embodiments, after calculating the image similarities between all the runway light outline templates and the geometric feature map, all the image similarities are threshold screened and consistency verified to determine whether the geometric feature map contains the target runway light. If so, the position and orientation of the target runway light in the geometric feature map are output based on the step size and the number corresponding to the runway light outline template.

[0032] The beneficial effect of the present invention is that by mapping the template matching process from the unstable RGB space to the stable contour space, this space mapping effectively retains the stable structure of the image, thereby effectively resisting the interference of factors such as changes in ambient lighting and contamination on the appearance of runway lights on the recognition and positioning of runway lights. Therefore, the method has the advantages of high robustness and strong adaptability, and can be promoted and applied in the application fields of automated detection and operations in scenarios where the target appearance is changeable and the target is easily contaminated. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart of a method for identifying and locating airport runway lights using contour space feature analysis disclosed in an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the runway light outline template;

[0035] Figure 3 Schematic diagram for comparing the corrected image and the binary geometric feature map;

[0036] Figure 4 Schematic diagram of the recognition and positioning results of real-time images. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the content of the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It is understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for the convenience of description, only the parts related to the present invention are shown in the accompanying drawings, rather than all the contents.

[0038] This embodiment proposes a method for identifying and locating airport runway lights based on contour space feature analysis. The template matching process is mapped from the unstable RGB space to the stable contour space. This spatial mapping effectively retains the stable structure of the image, thereby effectively resisting the interference of factors such as changes in ambient lighting and damage to the appearance of runway lights on the identification and positioning of runway lights. Therefore, this method has the advantages of high robustness and strong adaptability, and can be promoted and applied in the application fields of automated detection and operations in scenarios where the target appearance is changeable and the target is easily contaminated.

[0039] like Figure 1 As shown, the method comprises the following steps 1-5:

[0040] Step 1: pre-acquire multiple target runway light image samples, such as Figure 2 As shown, contours are extracted from image samples to create offline runway light contour templates, and the various runway light contour templates are merged into a template library.

[0041] In step 1, the collected image samples and the real-time images collected in the subsequent step 2 are all from a visual acquisition device with the same configuration, and the visual acquisition device can be an RGB monocular camera or a high-definition camera.

[0042] Specifically, the process of creating a runway light outline template includes:

[0043] Step 101, image preprocessing and contour extraction: preprocess the image samples and extract the contours, wherein the preprocessing operation needs to include at least grayscale and bilateral filtering, and the preprocessing step helps to reduce noise and enhance contour information. For the contour extraction operation, a contour extraction algorithm such as edge detection or connected region analysis can be selected to extract the contours of the target runway lights from the preprocessed image samples. These contours are composed of a series of coordinate points and describe the boundary shape of the target runway lights.

[0044] Step 102, contour screening, shape model creation and template multi-parameter construction: The extracted contours are screened according to the contour area, length, shape and other features, contours that do not meet the conditions are excluded, and contour point sets are extracted from the screened contours. Finally, the contour point set is used as input to create a shape model, which is a data structure that describes the shape features of the runway lights and is used for subsequent matching operations. The shape model is rotated according to the preset rotation resolution and saved, thereby obtaining a runway light contour template with multiple rotation angles and binarization.

[0045] Step 2: Obtain a real-time image of the target runway light online.

[0046] Specifically, the real-time image acquisition process includes:

[0047] Step 201, light source construction: Use a line scanning light source to illuminate the target runway light. The line scanning light source is placed above the side of the target runway light, and the incident angle of the line scanning light source forms an obtuse angle with the front of the target runway light. In this lighting mode, the smooth surface of the runway light will be displayed brighter in the real-time image, while the complex area of ​​the surface structure will be displayed darker in the real-time image, so the contour features of the runway light can be highlighted.

[0048] Step 202, camera calibration: the image acquisition device is calibrated to obtain calibration parameters, and the image acquisition device acquires the RGB image. Optionally, a checkerboard calibration plate of known size is placed in the camera field of view at different distances and postures, and the camera's intrinsic parameter matrix is ​​obtained by the Zhang Zhengyou calibration method. And 5 distortion coefficients k1, k2, k3, p1, p2.

[0049] Step 203, image reading: converting the RGB image into a real-time image in grayscale form. Optionally, the camera is connected to a computer via a communication cable, the RGB image fed back by the camera in real time is read, and converted into a grayscale image. For an image with n×m pixels, its matrix form can be expressed as

[0050] Step 3: preprocess the real-time image to generate a corresponding corrected image.

[0051] Specifically, the preprocessing operations of real-time images include:

[0052] Step 301, distortion correction: perform nonlinear distortion correction on the pixel coordinate values ​​of the real-time image according to the calibration parameters. Optionally, the pixel coordinate values ​​can be corrected nonlinearly using the calibration parameters of the camera to remove pixel coordinate distortion caused by the processing error and installation error of the camera lens. For the real-time image I' obtained in step 203, record each point x uv'The pixel coordinate value is (u',v'), then point x uv 'Dedistortion coordinates The five distortion parameters k1, k2, k3, p1, and p2 calibrated in step 203 can be transformed by the following formula (1):

[0053]

[0054] In the formula, r = u' 2 +v' 2 , u' is the horizontal coordinate of the real-time image, v' is the vertical coordinate of the real-time image, is the horizontal coordinate of the real-time image after dedistortion, is the ordinate of the real-time image after dedistortion, k1, k2, k3, p1, p2 are distortion parameters, and the dedistorted image is obtained by traversing all pixels.

[0055] Step 302, noise filtering: The real-time image after distortion correction is subjected to bilateral filtering to reduce image sharp noise and generate a corrected image. The above-mentioned bilateral filtering is used to reduce image sharp noise, achieve image smoothing and blurring while maintaining image edge characteristics, and improve the accuracy of feature extraction in the boundary transition area. Remember every pixel on it The grayscale pixel value is Then the pixel value weight coefficient w of each pixel is uv The pixel points in its 3×3 image block are Coordinate values ​​and pixel values The calculation results are:

[0056]

[0057] In the formula, for The horizontal axis value of for The vertical coordinate value of i is the pixel point in the neighborhood image block. The horizontal coordinate value of j is The horizontal axis value The horizontal coordinate value of v is the vertical coordinate value of σ s and σ r These are all artificially set parameters, which are used as the variance of the Gaussian filter kernel to reflect the weighting of different distances of neighboring pixels. Image after traversing the distorted edge According to the weighted weight of each pixel in formula (2), the calculated weight value is multiplied by the original pixel value to obtain the new pixel value The corrected image I is obtained.

[0058] Step 4: Analyze the pixel gradient and distribution of the corrected image to generate a binary geometric feature map, such as Figure 3 shown.

[0059] Specifically, the geometric feature map generation process includes:

[0060] Step 401, image scaling: scaling the corrected image by a preset ratio to obtain a reduced image. Optionally, scaling the x-axis and y-axis of the corrected image I by a scale factor of 0.8 to obtain a reduced-size image I s , in order to solve the jagged edge problem of the image.

[0061] Step 402, gradient calculation: calculate the horizontal angles and gradient values ​​of all pixels in the reduced image, and calculate the horizontal line field of the corrected image based on the horizontal angles and gradient values.

[0062] In step 402, the reduced image I s Any pixel x uv , if its pixel coordinates are marked as (u, v), then the three points with coordinates (u+1, v), (u, v+1), (u+1, v+1) are used to calculate the point x uv The gradient values ​​on the x-axis and y-axis are:

[0063]

[0064] Then the horizontal angle θ corresponding to this point is uv and the gradient value G uv Size:

[0065]

[0066] Traverse and reduce image I s For all the pixels in , the corresponding horizontal angles and gradient values ​​are calculated according to the above formula to obtain the horizontal line field of the corrected image I.

[0067] Step 403, contour edge search: sort and search the pixels of the reduced image according to the gradient value, and then use the gradient value corresponding to the pixel as the seed to perform contour edge search and clustering operations, wherein the search and clustering operation rule is to determine whether the detection pixels adjacent to the current pixel belong to the same contour edge area according to the similarity of the horizontal angle value of the seed point, and iterate until the growth and expansion of the seed are completed, and the line segment extraction is completed; finally, all the extracted line segments are merged and enlarged and restored to form a contour edge;

[0068] Step 404, binarize the contour image: introduce an all-zero matrix, the number of rows and columns of the all-zero matrix is ​​equal to that of the real-time image, then traverse all the line segments, and according to the pixel coordinates and pixel occupancy of each line segment, set the value of the corresponding pixel on the all-zero matrix to 1 to obtain a binary geometric feature map.

[0069] After the line segments in the reduced image are extracted in step 403, each line segment l i can be described by the coordinates of its two end points, that is, In the formula, the subscript i represents the i-th line segment extracted, and the superscripts s and e indicate the starting point and end point of the line segment corresponding to the point, respectively. In the generation of binary contour images, first, initialize a matrix I with all values ​​​​0 c , whose number of rows and columns is equal to that of the original image I; secondly, traverse all the extracted line segments {l i}, the coordinate values ​​of the starting point and the end point of the line segment are enlarged and restored according to the scale factor 01.25, and the pixel coordinates and pixel occupancy of the line segment after each enlargement are calculated in I c Set the value of the corresponding pixel to 1; finally, obtain a geometric feature map I with only 0 or 1 values c , this matrix reflects the contour information of the image.

[0070] Step 5: perform similarity measurement between the runway light outline template and the geometric feature map, and identify whether the real-time image contains runway lights based on the measurement results, and locate the pixel position and orientation of the real-time image, such as Figure 4 shown.

[0071] Specifically, the steps of similarity measurement include:

[0072] Step 501, matching area interception and similarity measurement: extract the runway light outline template from the template library, traverse the runway light outline template in sequence according to the preset step length, intercept each runway light outline template into multiple matrices, and save them into a sub-matrix, traverse the geometric feature map in sequence according to the preset step length, intercept the area with the same size and position as the sub-matrix from the geometric feature map as the area to be matched, calculate the point-to-point distance between the area to be matched and the sub-matrix, and use the distance as the image similarity.

[0073] Optionally, in step 501, the template library is traversed in sequence, the runway light outline template is read and saved in a matrix, and the matrix is ​​used as the geometric feature map I c The sub-matrix is ​​used for measurement calculation. In the traversal process, the window sliding step size s is set first, and the sub-matrix is ​​adjusted along the x-axis and then along the y-axis in the geometric feature map I c position on the geometric feature map I cThe area with the same size and position as the submatrix is ​​intercepted as the area to be matched; the area to be matched will be used to match the geometric feature map I c A similarity measurement is performed to confirm whether the area contains the target object.

[0074] Due to the geometric feature map I c The runway light outline template is a binary image and can be regarded as sparse, discrete pixel occupancy data. The point-to-point distance calculation method is used to measure the similarity between the matching area image intercepted in the previous step and the outline template. First, read the point set {q i |q i =(u i ,v i )} and the point set {t i |t i =(x i ,y i )}, where (u i ,v i ) and (x i ,y i ) is the pixel coordinate value of the corresponding pixel point. Secondly, the similarity between the matching area image Q and the target contour template T is calculated as follows:

[0075]

[0076] In the formula, |{t i}| represents the total number of points on the target contour template T, min(||tq i || 2 ) represents a point t∈{t i}The minimum distance value to the point on the matching area image Q.

[0077] Step 502, after calculating the image similarity between all runway light outline templates and the geometric feature map, threshold screening and consistency verification are performed on all image similarities to determine whether the geometric feature map contains the target runway light. If so, the position and orientation of the target runway light in the geometric feature map are output based on the step size and the number corresponding to the runway light outline template.

[0078] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying and locating airport runway lights based on contour space feature analysis, characterized in that: The following steps are involved: Acquire image samples of a plurality of target runway lights in advance, extract outlines from the image samples to create offline runway light outline templates, and merge the runway light outline templates into a template library; Acquire a real-time image of the target runway light online; Performing a preprocessing operation on the real-time image to generate a corresponding corrected image; Analyzing pixel gradient and distribution rules of the corrected image to generate a binary geometric feature map; The geometric feature map generation process includes: scaling the corrected image according to a preset ratio to obtain a reduced image; calculating the horizontal angles and gradient values ​​of all pixels in the reduced image, and calculating the horizontal line field of the corrected image according to the horizontal angles and the gradient values; sorting and retrieving the pixels of the reduced image according to the gradient value, and then using the gradient values ​​corresponding to the pixels as seeds to perform contour edge search and clustering operations, wherein the search and clustering operation rule is to determine whether the current pixel belongs to the same contour edge region according to the similarity of the horizontal angle values ​​of the detection pixels adjacent to the current pixel and the seed point, and iterate until the growth and expansion of the seeds are completed, and the line segment extraction is completed; finally, all the extracted line segments are merged and enlarged and restored to form contour edges; an all-zero matrix is ​​introduced, the number of rows and columns of the all-zero matrix is ​​equal to that of the real-time image, and then all the line segments are traversed, the coordinate values ​​of the starting point and the end point of the line segment are enlarged and restored according to the proportional factor, and according to the pixel coordinates and pixel occupancy of the line segment after each enlargement, the value of the corresponding pixel on the all-zero matrix is ​​set to 1 to obtain the binary geometric feature map; The runway light outline template and the geometric feature map are measured for similarity, and whether the real-time image contains runway lights is identified based on the measurement result, and the pixel position and orientation of the real-time image are located.

2. The method for identifying and locating airport runway lights by using contour space feature analysis as claimed in claim 1, characterized in that: The image sample and the real-time image are derived from a visual acquisition device with the same configuration.

3. The method for identifying and locating airport runway lights by using contour space feature analysis as claimed in claim 1, characterized in that: The process of creating the runway light outline template includes: The image samples are preprocessed and contours are extracted. The extracted contours are screened, contour point sets are extracted from the screened contours, a shape model is created using the contour point sets as input, a rotation amount is applied to the shape model according to a preset rotation resolution and the model is saved, thereby obtaining the runway light contour template with multiple rotation angles and binarization.

4. The method for identifying and locating airport runway lights by using contour space feature analysis as claimed in claim 1, characterized in that: The real-time image acquisition process includes: The target runway light is illuminated by a line scanning light source, the line scanning light source is placed above and to the side of the target runway light, and the incident angle of the line scanning light source forms an obtuse angle with the front face of the target runway light; The image acquisition device obtains calibration parameters after calibration, and the image acquisition device acquires an RGB image; The RGB image is converted into the real-time image in grayscale.

5. The method for identifying and locating airport runway lights by using contour space feature analysis as claimed in claim 4, characterized in that: The preprocessing operation of the real-time image includes: Performing nonlinear distortion correction on pixel coordinate values ​​of the real-time image according to the calibration parameters; The real-time image after distortion correction adopts bilateral filtering to reduce image sharp noise to generate the corrected image.

6. The method for identifying and locating airport runway lights by using contour space feature analysis as claimed in claim 1, characterized in that: The steps of similarity measurement include: Extracting the runway light outline template from the template library, traversing the runway light outline template in sequence according to a preset step length, intercepting each of the runway light outline templates into multiple matrices and saving them in a sub-matrix, traversing the geometric feature map in sequence according to a preset step length, intercepting an area with the same size and position as the sub-matrix from the geometric feature map as a to-be-matched area, calculating a point-to-point distance between the to-be-matched area and the sub-matrix, and using the distance as the image similarity; After calculating the image similarities between all the runway light outline templates and the geometric feature map, threshold screening and consistency verification are performed on all the image similarities to determine whether the geometric feature map contains the target runway light. If so, the position and orientation of the target runway light in the geometric feature map are output based on the step size and the number corresponding to the runway light outline template.

Images