A real-time detection method for infrared image horizon line

Abstract

The application provides a real-time detection method for the horizon line of an infrared image, comprising the following steps: image preprocessing: performing Gaussian filtering on the infrared image to obtain a filtered image; edge detection: performing edge detection on the filtered image to eliminate non-edge pixel points; straight line detection: detecting straight line segments in the filtered image from which the non-edge pixel points are eliminated; filtering oblique lines and vertical lines: filtering straight line segments with slopes not within a preset range to obtain n (n >= 1) candidate horizon lines; determining the horizon line: extracting the row values at both ends of all the candidate horizon lines, constructing 2n horizontal boundary lines located on the row values, and dividing the image into a sky region and a ground region; the gray mean value of the sky region is greater than that of the ground region, and the boundary line with the minimum sum of the gray variances of the sky region and the ground region is set as the horizon line. The application has good robustness even for complex conditions such as clouds and power lines in the sky, and meets the real-time processing requirement.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a real-time detection method for the sky-ground line of an infrared image. BACKGROUND

[0002] In recent years, sky-ground line detection technology has been widely used in the fields of infrared early warning and target search, and is an indispensable key technology. In the application scenario of infrared early warning, the observed scene is a composite image containing the sky and the ground. Due to the large difference between the sky background and the ground background, the sky background is more pure and clean, while the ground background is more complex and diverse. Generally, infrared target detection is more suitable for the sky background, while the complexity and diversity of the ground background often lead to an increase in the false detection rate. In order to avoid the influence of ground clutter on detection accuracy, the sky-ground line detection technology is of great significance. Using the real-time sky-ground line detection technology, the target detection range is effectively reduced, thereby eliminating the influence of ground objects. This provides a strong guarantee for the accuracy and detection efficiency of target detection of the infrared early warning system.

[0003] At present, there are few domestic studies on infrared image sky-ground line detection, and most of them are based on the sea-sky line detection method. The traditional sea-sky line detection method mainly includes the region segmentation method and the Hough transform method. However, these methods only consider the imaging characteristics of the sea-sky line, while the imaging characteristics of the sky-ground line of an infrared image are different from those of the sea-sky line of a visible image, and the texture information of the ground object is more complex, making it difficult to apply the sea-sky line detection method to sky-ground line detection.

[0004] In 2018, Dai Yongshou et al. published "Sea-sky line detection based on local Otsu segmentation and Hough transform" in Opto-Electronic Engineering. A sea-sky line detection method based on local Otsu segmentation and Hough transform was proposed. First, the median filter was used to suppress high-frequency noise such as light spots in the grayscale image. Then, the image was divided into blocks to compensate for the unevenness of the light, and the local Otsu segmentation was performed on each block to obtain a binary image and extract the edge pixels. Finally, the Hough transform was used to fit the edge pixels to obtain the sea-sky line. This method can effectively suppress the interference of factors such as sea waves, reflections, and ships. However, when this method is used for sky-ground line detection, it is easily affected by complex ground environments and sky clouds, leading to failure of sky-ground line detection.

[0005] In 2020, Sun Yuxin et al. published "Research on infrared image sky-ground line detection algorithm based on semantic segmentation" in Opto-Electronic Technology and Applications. A sky-ground line detection algorithm based on semantic segmentation was proposed. This algorithm introduced a deep semantic segmentation model into the sky-ground line detection task, and used the Deeplab-v3+ algorithm as the semantic segmentation algorithm to effectively segment the sky and the ground. However, the real-time performance of this method needs to be improved.

[0006] In 2021, Liang Lei et al. published "Unmanned aerial vehicle sky-ground line detection based on sky segmentation" in Modern Computer, which proposes an unmanned aerial vehicle sky-ground line detection method based on sky segmentation. The method first converts the image to be detected into a color space; performs sky segmentation on the converted image; then uses the Canny operator to detect the edges of the segmented image; finally, the Hough transform is used to identify and detect the sky-ground line. This method is only suitable for color image sky-ground line detection, so its applicability is limited.

[0007] In summary, there is little research on infrared image sky-ground line detection methods in China, and the real-time performance of existing infrared image sky-ground line detection methods based on semantic segmentation needs to be further improved and researched. The unmanned aerial vehicle sky-ground line detection method based on sky segmentation is based on color space extraction, so it is not suitable for infrared images. In addition, most of them are based on sea-sky line detection methods. When they are used for sky-ground line detection tasks, the sea-sky line detection method only considers the imaging characteristics of the sea-sky line, while the imaging characteristics of the infrared image sky-ground line have some differences, and the texture information of the ground object is more complex, making the sea-sky line detection method directly used for sky-ground line detection have limited applicability.

[0008] Therefore, it is urgent to propose a real-time sky-ground line detection method for infrared images. SUMMARY

[0009] To solve the above problems, the present application proposes a real-time detection method for infrared image sky-ground line with high precision and simple calculation process. In order to achieve the above purpose, the present application adopts the following specific technical scheme:

[0010] A real-time detection method for infrared image sky-ground line, comprising the following steps:

[0011] S1, image preprocessing: Gaussian filtering of the infrared image to obtain a filtered image;

[0012] S2, edge detection: edge detection of the filtered image to remove non-edge pixels;

[0013] S3, straight line detection: detecting straight line segments in the filtered image after removing non-edge pixels;

[0014] S4, filtering oblique lines and vertical lines: filtering straight line segments with slopes not in the preset range after straight line detection of the filtered image, to obtain n≥1 candidate sky-ground lines;

[0015] S5, determine the sky-ground line: extract the row values of both ends of all candidate sky-ground lines, construct 2n horizontal boundary lines located at the row value r, and divide the image after filtering oblique lines and vertical lines into sky area and ground object area by the boundary lines;

[0016] Among all the demarcation lines, the demarcation line that makes the average gray value of the sky region greater than the average gray value of the ground object region and makes the sum of the gray variances of the sky region and the ground object region minimum is set as the horizon.

[0017] Further, in step S2, the Canny operator is used to perform edge detection on the filtered image.

[0018] Further, in step S3, the Hough transform method is used to detect straight line segments.

[0019] Further, in step S4, the preset range is (5°, +5°).

[0020] Further, step S5 specifically includes the following steps:

[0021] S51, n candidate horizons are obtained through step S4 and are represented as L n {(r n1 ,c n1 ),(r n2 ,c n2 )};

[0022] wherein r is a row value, c is a column value, (r n1 ,c n1 ) is row position information and column position information of a starting point of the candidate horizon, and (r n2 ,c n2 ) is row position information and column position information of an ending point of the candidate horizon.

[0023] S52, when r∈{r 11 ,r 12 ,Λr n1 ,r n2}, the image is divided into a sky region and a ground object region using a horizontal demarcation line located at the row value r.

[0024] S53, the optimal demarcation line is found with the average gray values of the sky region and the ground object region as constraint conditions to make the energy of the sky region and the ground object region minimum, the optimal demarcation line is expressed as:

[0025]

[0026] wherein J(B r ) represents the sum of the energy of the sky region and the ground object region when the demarcation line is B r , and J(B r ) is expressed as:

[0027]

[0028] wherein I U and I D respectively represent the pixel gray value of the sky region and the ground object region when the demarcation line is B r H and W are respectively the height and width of the image; and are respectively the gray mean value of the sky region and the ground object region when the demarcation line is B r and are respectively expressed by the following formulae:

[0029]

[0030]

[0031] wherein N U and N D are respectively the total number of the pixel points of the sky region and the ground object region;

[0032] S54, the optimal demarcation line is solved according to formulae (1)-(4) and is taken as the horizon.

[0033] The present application can achieve the following technical effects:

[0034] 1) Compared with the traditional sea-sky line detection method, the present application is improved on the basis of the traditional algorithm, and the straight line segments detected by the edge image are not all in line with the characteristics of the candidate horizon, so the detected straight line segments are filtered to filter out the influence of the non-horizontal line on the candidate horizon. Secondly, the imaging characteristics of the sky-ground composite background infrared image after the horizon detection are fully considered, the energy function is established, the mean values of the upper and lower regions are taken as the constraint conditions, and the energy optimal line, i.e. the horizon, which makes the sky and ground object region variance minimum is found.

[0035] 2) Compared with the existing infrared image horizon detection method, the present application has good robustness and real-time performance. Through experimental test, it can be known that the method can effectively detect the horizon in most scenes, and can maintain good robustness even in the presence of complex conditions such as clouds and power lines in the sky, and can better meet the real-time processing requirements of common horizon detection tasks. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is the flow chart of the whole detection method disclosed by the present application;

[0037] Figure 2 is the flow chart of another detection method disclosed by the present application;

[0038] ​Figure 3 This is a diagram showing the results of the line detection disclosed in this invention;

[0039] Figure 4 This is a diagram showing the results of candidate top and bottom lines detection for filtering out non-horizontal lines as disclosed in this invention.

[0040] Figure 5 This is a diagram showing the results of the geodetic datum detection disclosed in this invention;

[0041] Figure 6 This is a result image of the detection of the horizon and ground line in an infrared image under a complex background as disclosed in this invention.

[0042] Figure 7 This is a result image of the detection of the horizon and ground line in an infrared image under another complex background disclosed in this invention;

[0043] Figure 8 This is a result image of the detection of the horizon and ground plane in an infrared image under another complex background as disclosed in this invention.

[0044] Attached image label: Skyline 1. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0046] This invention provides a real-time detection method for the horizon and ground plane in infrared images. Figure 1 A flowchart of the detection method is shown, such as Figure 1 As shown, the real-time detection method includes the following steps:

[0047] S1. Image preprocessing: Gaussian filtering is applied to the infrared image to obtain the filtered image.

[0048] S2, Edge Detection: Perform edge detection on the filtered image and remove non-edge pixels.

[0049] S3, Line Detection: Detects line segments in a filtered image after removing non-edge pixels.

[0050] S4. Filtering diagonal and vertical lines: For the filtered image after line detection, line segments with slopes outside the preset range are filtered to obtain n≥1 candidate top and bottom lines.

[0051] S5. Determine the sky-ground line: Extract the row values ​​at both ends of all candidate sky-ground lines, construct 2n horizontal dividing lines located on the row value r, and divide the image after filtering diagonal and vertical lines into sky region and ground region by the dividing lines; among all dividing lines, the dividing line that makes the gray mean of the sky region greater than the gray mean of the ground region, and minimizes the sum of the gray variances of the sky region and the ground region is set as the sky-ground line.

[0052] Specifically, extract the row values ​​at both ends of all candidate top and bottom lines to obtain 2n horizontal lines B located at row value r. r Horizontal line B r The image is divided into a sky region and a ground feature region using this as a boundary line. The determination is made based on whether the average grayscale value of the sky region is greater than that of the ground feature region.

[0053] If the result of the judgment is negative, then discard the current row value;

[0054] If the judgment result is yes, then calculate and record the sum of the grayscale variances of the sky area and the ground area;

[0055] By comparing the sum of the grayscale variances of all records, the boundary line corresponding to the minimum sum of grayscale variances is set as the top-to-bottom line 1.

[0056] The following example uses an infrared image of the horizon and skylines to be detected with a resolution of 640*512, and refers to... Figure 2 The procedure for the detection method shown will be explained in detail.

[0057] S1. Image preprocessing: Considering the influence of the complex texture structure of ground features on the extraction of the horizon line, the input infrared image is preprocessed with Gaussian filtering to remove Gaussian noise and smooth the small structures in the infrared image.

[0058] S2. Edge detection: The Canny operator is used to perform edge detection on the filtered image obtained in step S1. Compared with other edge detection operators such as Sobel and Prewitt, the Canny operator has better edge detection advantages. It makes full use of the edge gradient direction to suppress the non-maximum value of pixels, thereby eliminating most non-edge pixels.

[0059] S3. Line Detection: The horizon line in infrared images is extracted from the edge contours of the ground and sky backgrounds, exhibiting certain straight-line characteristics. Therefore, the Hough transform method is used to detect straight-line segments in the edge image. The detection results are as follows: Figure 3 As shown.

[0060] S4: Filtering Diagonal and Vertical Lines. Due to the complexity and diversity of terrain features, the detection results include various straight line segments, such as vertical and diagonal lines. These straight line segments are considered not to meet the characteristics of candidate top and bottom lines. To improve algorithm efficiency and eliminate their influence on top and bottom line detection results, the straight line segments detected in step S3 are filtered for non-horizontal lines. Therefore, straight line segments with slopes outside the range of [-5°, +5°] are filtered out. The filtering result is as follows: Figure 4 As shown.

[0061] In filtering diagonal and vertical lines, the preset range of the slope can be changed depending on the situation and is not limited to the range of this embodiment.

[0062] S5: Determine the horizon: Considering that the composite background infrared image of the horizon used for horizon detection has two imaging characteristics:

[0063] 1) In most cases, the texture information of the ground area is more than that of the sky area;

[0064] 2) The sky area appears brighter in the image than the ground area.

[0065] Therefore, the grayscale characteristics of infrared images with composite sky and earth backgrounds can be summarized as follows:

[0066] 1) The sum of the gray variance of the sky area and the gray variance of the ground features is the smallest.

[0067] 2) The average gray value of the sky area is greater than the average gray value of the ground area.

[0068] S51. Assume that after filtering in step S4, n candidate top-to-bottom lines are obtained, i.e., candidate top-to-bottom line segments, and denoted as L. n {(r n1 ,c n1 ),(r n2 ,c n2 )};

[0069] Where r is the row value and c is the column value, (r n1 ,c n1 (r) represents the row and column position information of the starting point of the candidate vertical line segment. n2 ,c n2 () represents the row and column position information of the endpoint of the candidate vertical line segment;

[0070] S52, when r∈{r 11 ,r 12 ,Λr n1 ,r n2 When}, the horizontal line B located on row value r will be... rAs a dividing line, the dividing line divides the image into two regions, one above and one below the sky region and the other below the ground feature region.

[0071] Specifically, multiple candidate horizon lines can be obtained from the infrared image, including those representing buildings and other scenes. These lines are composed of multiple small line segments. In step S4, straight lines that are clearly not horizontal are eliminated. For the remaining n≥1 candidate horizon lines after the elimination process, the position of the endpoint of each candidate horizon line can be obtained from the image. A horizontal line is drawn at each endpoint to segment the image. The optimal dividing line is then determined from the 2n horizontal lines using the following step S53.

[0072] S53. Find the optimal boundary line using the average grayscale values ​​of the sky region and the ground feature region as constraints. Minimize the energy in the sky region and the ground region; optimal boundary line. It can be expressed as follows:

[0073]

[0074] Among them, J(B r (Based on the current boundary line being B) r The sum of the energy of the sky region and the ground region at that time, that is, the sum of the gray variances of the two regions, J(B r It can be expressed as follows:

[0075]

[0076]

[0077] Among them, I U and I D They represent B respectively r The grayscale values ​​of the pixels in the upper and lower regions of the image are used as the dividing line. H and W represent the height and width of the image, respectively. and Indicates B r The grayscale average of the two regions above and below the dividing line is given by the following formula:

[0078]

[0079]

[0080] Where, N U and N D These represent the total number of pixels in the sky region and the ground region, respectively.

[0081] S54. Solve for the optimal boundary line according to formulas (1) to (4). Optimal dividing line As the celestial-terrestrial line, the optimal energy boundary line can be solved. The location, the detected horizon, such as Figure 5 As shown.

[0082] To verify the robustness and accuracy of this invention, experiments were conducted on infrared images of different scenes to detect the horizon and ground line. The detection steps were as described above, and some horizon and ground line detection results are shown below. Figures 6-8 As shown. The detection results indicate that, regardless of cloud cover or power line interference present in the infrared image, this invention can accurately detect the corresponding skyline and ground line while meeting real-time requirements.

[0083] This invention first preprocesses the infrared image using Gaussian filtering; then extracts the edge contours of the infrared image using the Canny edge detection algorithm; finally, it uses the straight line detection characteristic of Hough transform to detect straight line segments in the edge image, filtering out those with slopes outside the range of -5° to +5°, and using the remaining horizontal straight line segments as candidate skylines; considering the imaging characteristics of the composite background infrared image used for skyline detection, it is assumed that, generally, the texture information of the ground area is more abundant than that of the sky area, and the sky area is brighter than the ground area in terms of imaging effect. Therefore, an energy function is established, with the mean of the upper and lower regions when the candidate skyline is used as the boundary as a constraint condition, to find the energy-optimal line (i.e., the skyline) that minimizes the sum of the variances of the upper and lower regions. This invention can effectively detect skylines in most scenarios and has low computational complexity. Therefore, this invention has good robustness and real-time performance.

[0084] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0085] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

[0086] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for real-time detection of the horizon and ground plane in infrared images, characterized in that, Includes the following steps: S1. Image preprocessing: Perform Gaussian filtering on the infrared image to obtain the filtered image; S2. Edge detection: Perform edge detection on the filtered image and remove non-edge pixels; S3, Line Detection: Detects line segments in a filtered image after removing non-edge pixels; In step S3, the Hough transform method is used to detect straight line segments; S4. Filtering diagonal and vertical lines: For the filtered image after line detection, filter line segments whose slope is not within the preset range to obtain n≥1 candidate top and bottom lines; In step S4, the preset range is (5°, +5°). S5. Determine the sky-ground line: Extract the row values ​​at both ends of all candidate sky-ground lines, construct 2n horizontal dividing lines located on the row values, and divide the image after filtering diagonal and vertical lines into sky region and ground feature region by the dividing lines. Among all the boundary lines, the boundary line that makes the average gray value of the sky region greater than the average gray value of the ground feature region, and minimizes the sum of the gray value variances of the sky region and the ground feature region, is set as the sky-ground line. Extract the row values ​​at both ends of all candidate top and bottom lines to obtain 2n horizontal lines located at row value r. Horizontal line The image is divided into a sky region and a ground feature region using this as a boundary line. The determination is made based on whether the average grayscale value of the sky region is greater than that of the ground feature region. If the result of the judgment is negative, then discard the current row value; If the judgment result is yes, then calculate and record the sum of the grayscale variances of the sky area and the ground area; By comparing the sum of the grayscale variances of all records, the boundary line corresponding to the minimum sum of grayscale variances is set as the top and bottom line.

2. The real-time detection method for infrared image horizon and ground line according to claim 1, characterized in that, In step S2, the Canny operator is used to perform edge detection on the filtered image.

3. The real-time detection method for infrared image horizons according to claim 1, characterized in that, Step S5 specifically includes the following steps: S51. After filtering in step S4, n candidate top-to-bottom lines are obtained and represented as follows: ; in, For row values, For column values, This refers to the row and column position information of the starting point of the candidate horizon line. The row and column position information of the endpoint of the candidate horizon line; S52, when At that time, the image is divided into the sky region and the ground feature region using a horizontal dividing line located on the row value r; S53. Using the average grayscale values ​​of the sky region and the ground feature region as constraints, find the optimal boundary line. The optimal boundary line minimizes the sum of energy in the sky region and the ground feature region. It can be expressed as follows: （1）； in, The boundary line is indicated as At that time, the sum of the energy of the sky region and the ground feature region, the It can be expressed as follows: （2）； in, and The dividing lines are respectively represented as At that time, the grayscale values ​​of the pixels in the sky region and the ground feature region; and These are the height and width of the image, respectively; and The dividing lines are respectively represented as At that time, the average grayscale values ​​of the sky area and the ground feature area are expressed by the following formulas respectively: （3）； （4）； in, and These represent the total number of pixels in the sky region and the ground feature region, respectively. S54. Solve for the optimal dividing line according to the formulas (1) to (4). The optimal dividing line As the aforementioned celestial and terrestrial lines.

Images