A multi-scene robust sky detection method

Abstract

The application relates to a multi-scene robust sea-sky line detection method, and aims to solve the problems of large scene limitation and poor robustness of various sea-sky line detection algorithms. The process is as follows: I. detecting an input image; if the input image is an RGB image, extracting an R channel image and a B channel image of the RGB image, separately storing the R channel image and the B channel image, and executing II; if the input image is a gray image, executing II; II. obtaining a low-frequency image IM; III. obtaining an edge image EM; IV. removing a connected domain from the edge image EM, and expanding the image after the connected domain removal; V. obtaining a candidate straight line set and a value of an accumulator HN corresponding to each straight line in the candidate straight line set; VI. judging the straight lines in the candidate straight line set through an HGS method, and obtaining a real sea-sky line. The application is used in the field of sea-sky line detection.

Description

TECHNICAL FIELD

[0001] The present application relates to a multi-scene robust sea-sky line detection method. BACKGROUND

[0002] Sea-sky line detection plays an important role in the field of ocean engineering. The literature [1] By analyzing the sea-sky line, the attitude information of the ship is calculated, and an attitude measurement method is provided. The literature [2] The sea-sky region is obtained through the sea-sky line, and the speed and accuracy of the target detection algorithm are improved by limiting the search range. The literature [3] The images of adjacent frames are registered by detecting the sea-sky line. The literature [4] The attitude of a micro air vehicle is estimated by detecting the sea-sky line. However, the existing methods have the problem of large scene limitation and lack of robustness when estimating the sea-sky line. SUMMARY

[0003] The present application aims to solve the problem of large scene limitation and lack of robustness of the mainstream sea-sky line detection algorithm, and proposes a multi-scene robust sea-sky line detection method.

[0004] The specific process of a multi-scene robust sea-sky line detection method is as follows:

[0005] Step one, detecting the input image;

[0006] If it is an RGB image, the R channel image and the B channel image of the RGB image are extracted, and the R channel image and the B channel image are stored separately; Step two is executed;

[0007] If it is a grayscale image, step two is executed;

[0008] Step two, NSCT decomposition is performed on the R channel image or the grayscale image in step one to obtain a low-frequency image IM;

[0009] Step three, the edges of the low-frequency image IM are detected by a Canny operator to obtain an edge image EM;

[0010] Step four, the edge image EM is subjected to connected domain removal, and the image after connected domain removal is dilated;

[0011] Step five, Hough transformation is performed on the dilated edge image in step four to obtain a set of candidate straight lines and the value of the accumulator HN corresponding to each straight line in the set of candidate straight lines;

[0012] Step six, the straight lines in the set of candidate straight lines are judged by the HGS method to obtain the true sea-sky line.

[0013] The beneficial effects of the present application are:

[0014] The present application proposes a robust algorithm with wide application range, firstly analyzes the physical characteristics carried by light waves of different wavelengths, decomposes the R channel image through NSCT, and then extracts the low-frequency part of the edge by Canny operator, because the Canny operator has high sensitivity and less interference information in the low-frequency signal, the fuzzy edge can be extracted better. Then remove the interference through morphological processing and use Hough transform to extract the to-be-measured straight line, and then discriminate the to-be-measured straight line through GHS (gray value, gradient, entropy value) method, and get the final horizon. Finally, through experiment and analysis, the current popular methods are compared, and it is proved that the algorithm has excellent robustness and wide applicability. However, since the entropy value and gray value relied on by the method are easy to be destroyed by complex texture, and the low-frequency interference is too sensitive, in the above-mentioned occasions, the result will deviate, resulting in the reduction of the accuracy of the algorithm.

[0015] Horizon detection has important application in target detection, motion control and perception technology of unmanned equipment. The current horizon detection algorithm generally has the limitations of poor detection effect for weak horizon, easy to be disturbed by obstacles and narrow application range. This paper proposes a robust (NC-HGS) horizon detection algorithm based on image low-frequency information and physical characteristics. Firstly, the image is decomposed by non-downsampling counterlet transform, and then the to-be-determined line is screened out through Canny edge detection, morphological processing and Hough transform. Finally, the straight line is evaluated by considering five evaluation parameters, and the line with the highest evaluation score is the true visual horizon. This algorithm not only can accurately identify the horizon in various scenes, but also can detect the horizon in the case of horizon virtualization and strong interference in the scene. The performance of the algorithm is verified by 6000 images of various scenes. Compared with several current mainstream algorithms, the algorithm has wider application occasions, higher accuracy and stronger robustness. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 The flowchart of the present application is shown in the figure;

[0017] Figure 2a The original image is shown in the figure;

[0018] Figure 2b The R channel gray scale diagram of Figure 2a

[0019] Figure 2c The B channel gray scale diagram of Figure 2a

[0020] Figure 3a The edge diagram of the original image is shown in the figure; ​​

[0021] Figure 3b for Figure 3a Schematic diagram of the edge of the R channel;

[0022] Figure 3c for Figure 3a A schematic diagram of the edge of channel B;

[0023] Figure 4a This is a schematic diagram of the low-frequency edge of the R channel;

[0024] Figure 4b This is a schematic diagram of the low-frequency edges of the original image;

[0025] Figure 4c This is a schematic diagram of the edges of the original image;

[0026] Figure 4d This is a schematic diagram of edge extraction using the LSD method (original image).

[0027] Figure 5a This is a schematic diagram of the original edges;

[0028] Figure 5b This is a schematic diagram of the edge after morphological processing.

[0029] Figure 6a for Figure 2a A schematic diagram of the average grayscale curve in the middle;

[0030] Figure 6b for Figure 2a A schematic diagram of the horizontal average gradient curve in the image;

[0031] Figure 7 for Figure 9b A schematic diagram illustrating the entropy characteristics;

[0032] Figure 8a for Figure 9b Medium entropy distribution diagram;

[0033] Figure 8b for Figure 9b Gradient distribution diagram of intermediate entropy;

[0034] Figure 9a Figure 1 illustrates the detection results;

[0035] Figure 9b Two figures illustrate the detection results;

[0036] Figure 9c Three figures illustrate the detection results;

[0037] Figure 9d Four figures illustrate the detection results;

[0038] Figure 9e Figure 5 illustrates the detection results;

[0039] Figure 9f Fig. 6 is a schematic diagram for detecting the effect;

[0040] Figure 9g Fig. 7 is a schematic diagram for detecting the effect;

[0041] Figure 9h Fig. 8 is a schematic diagram for detecting the effect;

[0042] Figure 10a Fig. 9 is a schematic diagram for default parameters;

[0043] Figure 10b Fig. 10 is a schematic diagram after adjusting parameters; DETAILED DESCRIPTION

[0044] Embodiment 1: combination Figure 1 In this embodiment, the specific process of the multi-scene robust skyline detection method is as follows:

[0045] Skyline detection: according to the Rayleigh scattering law, the longer the wavelength of light, the stronger the penetration. When the time is dawn or dusk, because the position of the sun is almost parallel to the observation, the light is scattered by a large number of particles in the air during the process of penetrating the air, so the sky at this time shows orange. For the RGB image, at this time, the edges of the object are more prominent in the red channel. For the blue channel image, the gray scale of the image changes relatively gently. In the sea-skyline area, there are more water molecules in the air, and the observer is far away, so almost all wavelengths of light are scattered into the air, so the sea-skyline area will show a gray uniform white strip pattern.

[0046] The NC method refers to detecting the low-frequency components of the image through the NSCT transform and the Canny operator to obtain the edge features with gentle changes in the image. Because the sea-skyline area is gray uniform and the line direction changes slowly, the sea-skyline area belongs to the low-frequency part in the whole image. For an image, the part with complex contour and prominent edge texture belongs to the high-frequency part, and in the process of sea-skyline detection, such signals belong to interference and noise. Based on the wavelet transform theory, the image signal can be decomposed in the frequency domain. Then, by processing the low-frequency image with slow changes, the weak edge can be enhanced, and the real sea-skyline can be detected among various interferences.

[0047] The entropy value in the image can represent how much information content in the image is, generally, the texture of the sky area above the horizon is relatively single, and the entropy value is low, while the sea surface below the horizon contains more abundant information, and the entropy value is high. When there are ships, islands and other objects on the sea surface, or thick clouds appear in the sky, the information carried by them is often more intense, and the entropy value is high. The gradient of the entropy value is also more obvious. By calculating the entropy value in the neighborhood of the image, the foreground information contained therein can be identified, and the interference caused thereby can be excluded.

[0048] Based on the above principle, the algorithm obtains a low-frequency image by performing multi-scale decomposition on the R channel image, and then obtains a candidate straight line set by combining edge detection and morphological processing with Hough transformation. Finally, the HGS (gray value, gradient, and entropy value) method proposed in the application is used to judge the straight lines, and the real horizon is obtained.

[0049] Step one, detecting the input image;

[0050] If it is an RGB image, the R channel image and the B channel image of the RGB image are extracted, and the R channel image and the B channel image are stored separately; step two is executed;

[0051] If it is a gray image, step two is executed;

[0052] Step two, performing NSCT (multi-scale) decomposition on the R channel image or the gray image in step one to obtain a low-frequency image IM;

[0053] Step three, detecting the edge of the low-frequency image IM by a Canny operator to obtain an edge image EM;

[0054] Step four, performing connected domain removal on the edge image EM, and dilating the image after the connected domain removal;

[0055] Step five, performing Hough transformation on the edge image after the dilation in step four to obtain a candidate straight line set and the value of the accumulator HN corresponding to each straight line in the candidate straight line set;

[0056] The application removes the connected domain of the edge image to suppress interference. Then, Hough transformation is used to obtain all possible lines in the edge image. All possible straight lines form a candidate straight line set. For a straight line in the candidate straight line set, the stronger the linear feature is, the higher the value in the accumulator HN corresponding to the straight line in the Hough transformation is. The linear feature of the horizon is very strong. Therefore, the straight line with a higher value in the accumulator HN is more likely to be the horizon.

[0057] Hough transform is to detect straight line by counting the number of points on the edge of the line. The advantage is that the robustness of the voting results, through the accumulator HN value of Hough transform statistics can get the most likely straight line segment.

[0058] Step six, through the HGS (gray, gradient, and entropy) method for judging the straight line in the candidate straight line set, get the real sea sky line.

[0059] Specific implementation two: the embodiment is different from the specific implementation one, the step six is judged by HGS (gray, gradient, and entropy) method for judging the straight line in the candidate straight line set, get the real sea sky line; the specific process is:

[0060] Step six one, along the straight line in the candidate straight line set to search for the cumulative gray R1, R2, and calculate the cumulative gray difference;

[0061] Step six two, along the straight line in the candidate straight line set to search for the cumulative gradient;

[0062] Step six three, along the straight line in the candidate straight line set to search for the cumulative entropy value in the neighborhood M;

[0063] Step six four, statistics along the straight line in the candidate straight line set to search for the cumulative entropy value in the neighborhood L;

[0064] Step six five, based on the value of the accumulator HN in the Hough transform, the cumulative gray difference, the cumulative gradient, the cumulative entropy value, the cumulative entropy value gradient, calculate the value of GHS, the highest value is the real sea sky line.

[0065] Image multi-scale decomposition and edge detection

[0066] Because the sea water is better for long wave absorption, the sea water is generally blue, the B channel is sensitive to the wave target, because the sea sky line is far away, the red wave has strong penetration, the R channel can better identify it. By edge detection on the low frequency information of R channel image, the potential sea sky line target is screened out. The contrast in the image of R channel is high, the gray scale change amplitude is large, and the high frequency signal is more. There are still more low frequency signals in the B channel image with gentle color change. After NSCT decomposition, Canny edge detection is carried out, the edge of R channel is more gentle and sparse, and contains less noise.

[0067] Contourlet transform (non-adaptive direction multi-scale analysis method) is a kind of image sparse representation method with direction, through Laplace pyramid and direction filter bank to get different scale direction sub-band. But Contourlet transform contains the sampling step in the process of image decomposition and reconstruction, so the decomposed image does not have the shift invariance. Non-subsampled Contourlet transform, correspondingly, uses non-subsampled pyramid and direction filter bank to realize multi-scale decomposition, to get the sub-band image with the same size as the source image, which ensures the shift invariance. There is no subsampling operation in NSCT transform, and the image is up-sampled by a method similar to the hole algorithm. Since the Canny operator can conveniently locate the extracted edge, it can be used to extract the fuzzy sea-sky line in the strong interference image.

[0068] Figure 3a 、 3b , 3c shows the edge detection effect of each channel. As can be seen from the above comparison, in the case of fuzzy sea-sky line and a large amount of interference, the low-frequency information of R channel can better represent the position of sea level. However, the sea-sky line extracted or simulated in documents [1] to [7] is not decomposed in R channel image, which cannot detect the fuzzy sea-sky line in Figure 9a .

[0069] Morphological processing and Hough transform: Hough transform is to detect straight lines by counting the number of points on the edge. The advantage of this method is that the voting result is robust, and the most likely straight line segment can be obtained by counting the H value. In real image set, ships, islands and reefs will cause strong interference to the detection of sea-sky line, especially in fuzzy images, the intensity of sea-sky line is not good, and there may be a phenomenon of distortion, which increases the difficulty of Hough transform to detect straight lines. By removing the connected domain of the binary edge image, the small noise interference of sea waves or ships can be effectively eliminated, and the "misvote" phenomenon can be avoided. The final H value is stable and effective. Then the processed image is dilated and Hough transformed to get the candidate line set.

[0070] Figure 5a 、 5b is the result of edge detection, after connected domain processing, the right edge becomes smooth and clearer, which is conducive to the identification of sea-sky line.

[0071] Table 1 Pseudocode block of sea level detection

[0072]

[0073] Sealine Line Estimation Based on HGS: By analyzing the physical characteristics of the sealine line, this invention proposes a new voting method, the HGS method. By analyzing the gray level, local entropy value and corresponding gradient changes near the candidate line segment, and combining the H value obtained by Hough transform, the candidate images are evaluated, and the line segment with the best evaluation is selected as the true sealine line.

[0074] The other steps and parameters are the same as in Specific Implementation Method 1.

[0075] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that, in step six-one, the cumulative gray levels R1 and R2 are obtained by searching the upper and lower neighboring regions along the selected straight line set, and the cumulative gray level difference H is calculated; the specific process is as follows:

[0076] Grayscale index: Under clear weather conditions and with a clear view, the grayscale level of the sea-line changes gradually from bottom to top when observed from the coast or a ship. However, in low light conditions and when there are thick clouds in the sky, the grayscale differences near the sea-line are significant. Figure 2a , 2b As shown in Figure 2c, the grayscale changes are gradual in the blue channel (B), which can effectively remove interference from sunlight. In most cases, there is a significant difference in grayscale above and below the sea-line. Since the sea-line itself is relatively long, distant obstacles or thin clouds are unlikely to alter its overall grayscale distribution. Comparing the statistical row mean gradient and row mean grayscale can help the algorithm determine the region where the sea-line is located.

[0077] Figure 6a In the image, the black line represents the location of the sea-line horizon. As can be seen from the image, in... Figure 2a In the sky region, the grayscale gradient is gentle, while near the sea horizon, the grayscale changes drastically when close to the ocean. Furthermore, the grayscale above the sea horizon is significantly higher than below. Therefore, the method of defining a region and statistically analyzing the cumulative grayscale difference and cumulative gradient difference near the line segment can be used to determine whether a straight line segment is a sea horizon.

[0078] If the straight line is expressed using the intercept form, the cumulative gray level difference is calculated as follows:

[0079]

[0080] In the formula, f B This represents the B channel image or grayscale image in the RGB image from step one, q represents the q-th column in the B channel image or grayscale image, z represents the vertical coordinate (up or down) of the line, · represents multiplication, K and b represent the slope and intercept of the sea-line, respectively, and V represents f. B The width of the image, N, is f BThe height of the image; R1 and R2 distributions represent the accumulated gray values ​​of searching the lower and upper neighborhoods along the candidate line; H is the gray value difference between the two ends, and the larger the value of H, the greater the probability that it is a sea line.

[0081] Image f is segmented from left to right using the region segmentation method. B The system segments the data and obtains the cumulative grayscale difference H for each block. It then removes the maximum, second-largest, minimum, and second-smallest cumulative grayscale differences H from all blocks. Finally, it calculates the mean of the remaining cumulative grayscale differences H. This filtering method does not introduce unnecessary calculations and can save computing power while improving accuracy.

[0082]

[0083] In the formula, H′ is the mean of the cumulative grayscale difference H, SUM represents the cumulative grayscale difference of each region after removing the second largest, second smallest, largest and smallest values, W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks.

[0084] Other steps and parameters are the same as in specific implementation method one or two.

[0085] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that, in step six-two, the cumulative gradient is obtained by searching the upper and lower neighboring regions along the selected straight line set; the specific process is as follows:

[0086] Figure 6b The image shows the gradient information in the vertical direction along the sea-line antenna. It can be seen that the gradient abruptly changes upwards along the sea-line antenna, and the derivatives of the gradient on both sides of the sea-line antenna differ significantly. If we define f... B Let (z, q) be the pixel value at a point perpendicular to the sea-line, then f B The magnitude of the gradient value at point (z, q) is:

[0087] G B (z, q) = |f B (z,q)-f B (z-1,q)|, z=(1,2,…,N), q=(1,2,…,V) (3)

[0088] In the formula, V represents f B The width of the image, N, is f B Image height;

[0089] Figure 6b In the diagram, the black line represents the manually marked sea-line contour; it is not always located at the point of greatest gradient and grayscale change. Since this invention does not require locating the point of greatest change, the algorithm uses accumulated gradients to measure the line.

[0090]

[0091] In the formula, HT is the cumulative gradient value of the image in the neighborhood with a radius of N;

[0092] However, the original shape of the sea-sky line is often destroyed by objects such as ships, which reduces the accuracy of sea-sky line detection. The gray value in the region of the obstacle on the sea-sky line is relatively uniform, and when the region changes, the gray value changes significantly, and the gradient value is large.

[0093] The image f B (z, q) is segmented from left to right by the region segmentation method to obtain the cumulative gradient value HT of each block, and the maximum and second maximum values and the minimum and second minimum values of the cumulative gradient value HT corresponding to all blocks are removed; the mean value of the remaining cumulative gradient value HT is calculated; this filtering method does not introduce redundant calculations, and can improve accuracy while saving computing power;

[0094]

[0095] In the formula, HT' is the mean value of the cumulative gradient value HT, SUM HT represents the cumulative gradient value of each region after removing the second maximum value, the second minimum value, the maximum value, and the minimum value, W represents the second maximum value, M represents the second minimum value, and m represents the total number of blocks.

[0096] The other steps and parameters are the same as one of the first to third embodiments.

[0097] Embodiment Five: The difference between this embodiment and one of the first to fourth embodiments is that in step six three, the cumulative entropy value in the neighborhood L along the alternative straight line is searched M to obtain the cumulative entropy value in the neighborhood; the specific process is as follows:

[0098] Entropy value index: when there is a strong interference object on the sea-sky line, the gray characteristic will be destroyed, and sometimes when the interference is too strong, the gray value and the gradient will be opposite. Therefore, the entropy value index is used for judgment. The entropy value of the image can express the degree of information disorder, and generally speaking, the texture information of the interference object and the sea surface region is more abundant than the sky, and the entropy value is higher. By performing local entropy value filtering on the image, the regional entropy value of each region can be obtained, and the position of the sea-sky line can be determined by the entropy value feature;

[0099] The B channel image or the gray image in the RGB image in step one is subjected to local entropy value filtering to obtain an entropy value image after local entropy value filtering;

[0100] The calculation formula is as follows:

[0101]

[0102] pi = num(i) / (N E ) 2 (7)

[0103] f E (q, z) = H E , z = (1, 2, …, N), q = (1, 2, …, V) (8)

[0104] where H E (q, z) is the entropy value of the point (q, z), N E is the size of the filtering neighborhood, p i is the probability of the i-th gray level appearing in the region N E , num(i) is the total number of the i-th gray level appearing in the filtering region, f E is the entropy value image after local entropy value filtering; V represents the width of the f B image, and N is the height of the f B image; the point (q, z) is a point in the B channel image of the RGB image or a point in the gray image in the first aspect;

[0105] Figure 9b The texture feature under the horizon is stronger and significantly higher than that above the horizon. The long mark of the submarine passing through has no obvious difference in gray level with the surrounding gray levels. Under the two indicators of the cumulative difference of the gray level and the gray level gradient, the straight line segment that runs through the entire image and has strong gray level and gradient changes will cause strong interference to the two indicators.

[0106] At the same time, the entropy value gradient feature of the candidate line in a neighborhood is counted by using a method similar to the cumulative gray level gradient; the cumulative entropy value is small, and the cumulative entropy value gradient is large, which is judged as the real horizon;

[0107]

[0108] where f E is the entropy value image after local entropy value filtering, S M is the final entropy value indicator, K and b represent the slope and intercept of the straight line respectively, V is the width of the image, and N S is the size of the window searched upward along the vertical direction of the image;

[0109] The image f E (z, q) is segmented from left to right by the region segmentation method to obtain the entropy value indicator S M of each block. The maximum and second maximum values and the minimum and second minimum values in the entropy value indicator S M corresponding to all blocks are removed. The remaining cumulative entropy value indicator S MThe mean value of the filtered values; this filtering method does not introduce redundant calculation, and can improve accuracy while saving computing power;

[0110]

[0111] In the formula, S is the cumulative entropy value, SUM s The cumulative entropy value of each region after removing the second largest value, the second smallest value, the maximum value, and the minimum value, W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks.

[0112] The other steps and parameters are the same as one of the first to third embodiments.

[0113] Embodiment six: The difference between this embodiment and one of the first to fifth embodiments is that the cumulative entropy value gradient of the upper and lower neighborhoods of the candidate straight line is calculated in step six four; the specific process is as follows:

[0114] The entropy value image f E is obtained in step six three. E The entropy graph gradient graph is obtained by formula 8.

[0115] If a point in f E is defined as f E (z, q), then the point G E (z, q) of the entropy graph gradient graph is:

[0116] G E (z, q) = |f E (z, q) - f E (z-1, q)|, z = (1, 2, …, N), q = (1, 2, …, V) (11)

[0117] After obtaining the entropy graph gradient graph G E , the cumulative entropy value gradient HE of the upper and lower neighborhoods of a candidate straight line is calculated.

[0118] The calculation process is shown in formula 9, wherein the width of the upper and lower neighborhoods is N S2 .

[0119]

[0120] In the formula, HE is the cumulative entropy value gradient value of the image in the neighborhood with a radius of N;

[0121] The image G E (z, q) is segmented from left to right by the region segmentation method, and the cumulative entropy value gradient HE of each block is obtained. The maximum value and the second largest value, and the minimum value and the second smallest value of the cumulative entropy value gradient value HE corresponding to all blocks are removed. The mean value of the remaining cumulative entropy value gradient HE is calculated.

[0122]

[0123] In the formula, HE' is the average value of cumulative entropy gradient HE, SUM HE represent the cumulative entropy gradient of each region after removing the second largest value, the second smallest value, the maximum value, and the minimum value, W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks;

[0124] When there are ships on the sea surface, thick clouds in the sky, or obstructions above the photographic equipment, the original entropy value information of the sky and the sea will be destroyed, and the distinction between them is no longer clear. Combining with the gray scale features of the image can play a good complementary role and improve the accuracy of the algorithm.

[0125] When there are ships on the sea surface, thick clouds in the sky, or obstructions above the photographic equipment, the original entropy value information of the sky and the sea will be destroyed, and the distinction between them is no longer clear. Combining with the gray scale features of the image can play a good complementary role and improve the accuracy of the algorithm.

[0126] The other steps and parameters are the same as one of the first to fifth embodiments.

[0127] Embodiment seven: The difference between this embodiment and one of the first to sixth embodiments is that the value of GHS is calculated based on the value of the accumulator HN in the Hough transform, the cumulative gray scale difference, the cumulative gradient, the cumulative entropy value, and the cumulative entropy gradient in step six five. The highest value is the real sea-sky line. The specific process is:

[0128] Sea-sky line estimation under GHS index: Gray scale value index (H) and gray scale gradient (HG) are suitable for situations where the sea-sky line is long and the interference information is relatively short. Gray scale gradient (HG) is also suitable for infrared images. Entropy value information (S) and entropy gradient information (SG) have stronger anti-interference ability, but are sensitive to interference information in the sky (such as clouds). (SG) information is also suitable for distinguishing infrared images. At the same time, Hough transform is sensitive to edges in binary images and has good robustness, so the value of the counting matrix H in Hough is considered, denoted as N. Considering the different characteristics of sea-sky lines, the evaluation method GHS index (gray scale, gradient, entropy value) designed by the application is as follows

[0129]

[0130] In the formula, GHS is the final evaluation index for straight lines, a-e are the weights of the five types of indexes, respectively, (z1) HN is the HN power of z1, (z2) H′ is the H' power of z2, (z3) HT′ is the HT' power of z3, (z4)1 / S is the 1 / S power of z4, (z5) HE′ is the HE' power of z5, abs() represents taking the absolute value.

[0131] In the algorithm, a = 1, b = 1.2, c = 0.5, d = 1.3, and e = 1. The five weights are determined according to the requirement of the maximum applicable range. If it is necessary to detect a certain type of image individually, the values of a-e can be recalculated. For example, in an infrared image, the values of a and c can be appropriately increased and the value of d can be decreased to make the algorithm more stable.

[0132] Table 2 Pseudocode block for sea horizon estimation

[0133]

[0134] The beneficial effects of the present application are verified by the following embodiments:

[0135] Embodiment one:

[0136] The present application proposes a method for detecting the sea-sky line based on NSCT decomposition and Canny edge detection (NC method). Not only can most edge noises in the image be filtered out to make the image smooth, but also the position of the sea-sky line in the image can be accurately located in various interferences. In order to prove the stability of the algorithm, the present application is directly compared with the methods in the literature [5][6][7] through code comparison.

[0137] Regarding Figure 9a , 9b , 9c, 9d, 9e, 9f, 9g, 9h, line 1 is the sea-sky line obtained by the present method, and lines 2, 3, and 4 are the sea-sky lines obtained by the methods in the literature [5][7][6] . As can be seen from Figure 9a , 9b , 9c, 9d, 9e, 9f, 9g, 9h, the present method has strong robustness and can be applied to various scenes. Figure 9a In Fig. 9a, the gray level difference of the sea-sky line is not obvious, and there is serious interference below the sea-sky line. Such an image cannot be directly detected by the Canny operator or the LSD operator to detect the sea-sky line, so the methods including the literature [5][7][6] are difficult to detect. Figure 9b In Fig. 9b, the tail trace of the submarine below penetrates the entire image, and it is difficult for the methods based on gradient and edge to detect the real edge. Figure 9c In Fig. 9c, the thick cloud layer in the sky destroys the texture details of the sky, and it is difficult for the methods based on the gray level co-occurrence matrix and the OSTU-based method to divide the real sea-sky line. Figure 9c In Fig. 9d, the left side of the sea-sky line is incomplete, and the mountain destroys the original entropy value feature, so the accuracy of various methods based on entropy value and line fitting will also decrease.Figure 9d , Figure 9f The blurred horizon and irregular waves in the image disrupt the original texture details, so edge-based methods fail here. Figure 9e In the case of severe wave interference, the other comparison methods could not achieve good results. Figure 9g and Figure 9h In the middle, because the obstacles cut off the original texture of the horizon and occupied most of the space, it was impossible to accurately count the texture information of the sky and the ocean.

[0138] Since the true sea level always appears in the first three terms of N, in order to speed up the algorithm's operation, this algorithm searches for the first three terms with the highest number of votes, thereby achieving faster running efficiency.

[0139] However, the parameters used in this paper are only empirical and not applicable to all situations. For example, when detecting targets such as coastlines and docks, the H (grayscale)S (entropy) index will have the opposite effect, interfering with the original evaluation system. Furthermore, when there is a large-scale interference directly in front of the horizon, the original entropy characteristics will be destroyed, and the gradient index will point in the opposite direction. Therefore, adjustments must be made in practical applications.

[0140] However, because this method is highly sensitive to low-frequency information when extracting edges, it extracts multiple straight lines near the horizon, causing interference with detection. Therefore, the detection accuracy is not high when the sea surface ripple signal is complex. Furthermore, when the texture near the horizon is complex and there is a lot of low-frequency interference, the Hough transform is only sensitive to edges, which can cause the detected straight lines to shift, thus also causing the detection result to shift.

[0141] References:

[0142] [1] Feng Hongkui, Fu Minhui, Zhong De'an, et al. Ship attitude algorithm based on sea surface [J]. Ordnance Automation, 2019, 38(5):37-40.

[0143] [2] Zhang Xun, Ma Haobo, Li Yuncheng. Sea surface antenna detection in UUV sea infrared images by joint optimization of row mean gradient and straight line fitting [J]. Applied Science and Technology, 2018, 45(4):6-12.

[0144] [3] S. Fefilatyev, D. Goldgof, M. Shreve, and C. Lembke, ‘Detection and tracking of ships in open sea with rapidly moving buoy-mounted camera system’, Ocean Engineering, vol. 54, pp. 1-12, Nov. 2012, doi: 10.1016 / j.oceaneng.2012.06.028

[0145] [4] S. M. Ettinger, M. C. Nechyba, P. G. Ifju, and M. Waszak, ‘Vision-guided flight stability and control for micro air vehicles’, Advanced Robotics, vol. 17, no. 7, pp. 617-640, Jan. 2003, doi: 10.1163 / 156855303769156983.

[0146] [5] M. Schwendeman and J. Thomson, ‘A Horizon-Tracking Method for Shipboard Video Stabilization and Rectification’, Journal of Atmospheric and Oceanic Technology, vol. 32, no. 1, pp. 164-176, Jan. 2015, doi: 10.1175 / JTECH-D-14-00047.1.

[0147] [6] LIANG D, LIANG Y. Horizon detection from electrooptical sensors under maritime environment [J]. IEEE Transactions on Instrumentation and Measurement, 2019, 69(1): 45-53.

[0148] [7] J. Hou and B. Li, ‘An Improved Algorithm for Horizon Detection Based on OSTU’, p. 4.

Claims

1. A multi-scene robust sky detection method, characterized in that: The method specifically comprises the following steps: Step 1: detecting an input image; If the input image is an RGB image, extracting an R channel image and a B channel image of the RGB image, separately storing the R channel image and the B channel image, and performing step 2; If the input image is a gray image, performing step 2; Step 2: performing NSCT decomposition on the R channel image or the gray image in step 1 to obtain a low-frequency image IM; Step 3: detecting edges of the low-frequency image IM by using a Canny operator to obtain an edge image EM; Step 4: performing connected domain removal on the edge image EM, and performing inflation on the image after the connected domain removal; Step 5: performing Hough transformation on the edge image after the inflation in step 4 to obtain a set of candidate straight lines and a value of an accumulator HN corresponding to each straight line in the set of candidate straight lines; Step 6: judging the straight lines in the set of candidate straight lines by using an HGS method to obtain a real horizon line.

2. The method of claim 1, wherein: The step 6 comprises the following steps: Step 6.1: searching upward and downward two neighboring areas along the straight lines in the set of candidate straight lines to obtain accumulated gray values R1 and R2, and calculating an accumulated gray difference; Step 6.2: searching upward and downward two neighboring areas along the straight lines in the set of candidate straight lines to obtain accumulated gradient values; Step 6.3: searching a neighboring area L upward along the straight lines in the set of candidate straight lines to obtain accumulated entropy values in the neighboring area L; Step 6.4: counting accumulated entropy gradient values of the upward and downward neighboring areas L along the straight lines in the set of candidate straight lines; Step 6.5: calculating a value of GHS based on the value of the accumulator HN, the accumulated gray difference, the accumulated gradient values, the accumulated entropy values and the accumulated entropy gradient values, and the straight line with the highest value is the real horizon line. The step 6.1 comprises the following steps:

3. The method of claim 2, wherein: The accumulated gray difference H is calculated according to the following formula: In the formula, H' is an average value of the accumulated gray difference H, SUM represents accumulated gray differences of each area after removing a second largest value, a second smallest value, a maximum value and a minimum value, W represents the second largest value, M represents the second smallest value, and m represents a total number of blocks. where f B represents the image of the B channel in the RGB image or the gray image in step one, q represents the qth column in the image of the B channel or the gray image, z represents the vertical coordinate or the vertical coordinate downward in the vertical direction of the straight line, · represents multiplication, K and b represent the slope and the intercept of the horizon line respectively, V represents the width of the image, N represents the height of the image, R1 and R2 represent the accumulated gray values searched in the downward neighborhood and the upward neighborhood along the candidate straight line, and H is the gray difference at both ends, and the greater the value of H is, the greater the possibility that it is the horizon line is. B represents the image of the B channel in the RGB image or the gray image in step one, q represents the qth column in the image of the B channel or the gray image, z represents the vertical coordinate or the vertical coordinate downward in the vertical direction of the straight line, · represents multiplication, K and b represent the slope and the intercept of the horizon line respectively, V represents the width of the image, N represents the height of the image, R1 and R2 represent the accumulated gray values searched in the downward neighborhood and the upward neighborhood along the candidate straight line, and H is the gray difference at both ends, and the greater the value of H is, the greater the possibility that it is the horizon line is. B represents the image of the B channel in the RGB image or the gray image in step one, q represents the qth column in the image of the B channel or The image f is divided from left to right by a region division method B The cumulative gray difference H of each block is obtained by division, the maximum and the second maximum, and the minimum and the second minimum of the cumulative gray difference H corresponding to all blocks are removed, and the mean of the remaining cumulative gray difference H is calculated. The step 6.2 comprises the following steps:

4. The method of claim 3, wherein: The accumulated gradient is used to measure the straight lines according to the following formula: Definition f B (z, q) is the pixel value of a certain point along the vertical direction of the horizon antenna, then f B The size of the gradient value of the (z, q) point is: G B (z, q) = |f B (z, q) - f B (z - 1, q) |, z = (1, 2,..., N), q = (1, 2,..., V) (3) where V represents f B the width of the image, N is f B the height of the image; In the formula, HT is an accumulated gradient value of the image in a neighboring area with a radius of N; The step 6.3 comprises the following steps: The image f is segmented from left to right by a region segmentation method B (z,q) is segmented to obtain the cumulative gradient value HT of each block, the maximum and second maximum values and the minimum and second minimum values in the cumulative gradient value HT corresponding to all blocks are removed; the mean value of the remaining cumulative gradient value HT is calculated; where HT' is the average of the accumulated gradient values HT, SUM HT where HT' is the average of the accumulated gradient values HT, SUM where HT' is the average of the accumulated gradient values HT, SUM 5. The method of claim 4, wherein: A local entropy filtering is performed on the B channel image or the gray image in step 1 to obtain an entropy image after the local entropy filtering; The calculation formula is as follows: The step 6.4 comprises the following steps: p i = num(i) / (N E ) 2 (7) f E (q,z) = H E , z = (1,2,..., N), q = (1,2,..., V) (8) where H(q,z) is the local entropy value at the point (q,z) in the image, N is the size of the filtering neighborhood, p is the probability of the i-th gray level occurring within the region N E (q,z) is the entropy value at the point (q,z), N E is the size of the filtering neighborhood, p i is the probability of the i-th gray level occurring within the region N E num(i) is the total number of occurrences of the i-th gray level within the filtering region, f E is the entropy image after local entropy filtering; V represents the width of the f B image, N is the height of the f B image; (q,z) is the point in the B channel image or the gray image in the RGB image in the first step. In the formula, f E is the entropy image after local entropy filtering, S M is the final entropy index, K and b represent the slope and intercept of the straight line, respectively, N S is the window size of the search in the vertical direction of the image; Image f is segmented from left to right using the region segmentation method. E The region (z, q) is divided to obtain the entropy index S of each block. M Remove the entropy index S corresponding to all blocks. M Find the maximum and second-largest, as well as the minimum and second-smallest values ​​in the range; calculate the residual cumulative entropy index S. M The mean; where S is the cumulative entropy value, SUM s where S is the cumulative entropy value, SUM s where S is the cumulative entropy value, SUM 6. The method of claim 5, wherein: In the formula, HE is an accumulated entropy gradient value of the image in a neighboring area with a radius of N; The entropy image f is obtained in step six three E The entropy image f is obtained in step six three E The entropy image f is obtained in step six three If a point in f E is defined as f E (z, q), then the entropy map gradient map at a point G E (z, q) is: G E (z, q) = |f E (z, q) - f E (z - 1, q) |, z = (1, 2,..., N), q = (1, 2,..., V) (11) obtaining an entropy map gradient map G E Then, the cumulative entropy value gradient HE of the upper and lower neighborhoods of the candidate straight line is calculated. The calculation process is shown in equation (9), where the width of the upper and lower neighborhoods is N S2 ; The step 6.5 comprises the following steps: The image G is divided from left to right by a region division method E (z,q) is divided to obtain the cumulative entropy gradient HE of each block, the maximum and the second maximum value and the minimum and the second minimum value in the cumulative entropy gradient HE corresponding to all blocks are removed; the mean value of the remaining cumulative entropy gradient HE is calculated; where HE' is the mean of the cumulative entropy gradient HE, SUM HE where HE' is the mean of the cumulative entropy gradient HE, SUM HE where HE' is the mean of the cumulative entropy gradient HE, SUM 7. The method of claim 6, wherein: The GHS index is as follows ​ In the formula, GHS is the final evaluation index of the straight line, a-e are the weight values of the five types of indexes respectively;(z1) HN HN power of z1(z2) H′ H' power of z2(z3) HT′ HT' power of z3(z4) 1 / S 1 / S power of z4(z5) HE′ HE' power of z5, and abs() represents taking the absolute value.

Images