Multi-scene robustness sea-sky-line detection method

Abstract

The invention relates to a multi-scene robustness sea-sky-line detection method, in particular to a multi-scene robustness sea-sky-line detection method. The objective of the invention is to solve the problems of large use scene limitation and poor robustness of various mainstream sea-sky-line detection algorithms at present. The method comprises the following steps of: 1, detecting an input image; if the image is the RGB image, extracting an R channel image and a B channel image of the RGB image, and independently storing the R channel image and the B channel image; executing the second step; if the image is a grayscale image, executing the step 2; 2, obtaining a low-frequency image IM; 3, obtaining an edge image EM; 4, performing connected domain removal on the edge image EM, and performing expansion on the image after the connected domain removal; 5, obtaining an alternative straight line set and a numerical value of an accumulator HN in Hough transform corresponding to each straight line in the alternative straight line set; and 6, judging the straight lines in the alternative straight line set through an HGS method to obtain a real sea-sky-line. The method is applied to the field of sea-sky-line detection.

Description

Technical Field

[0001] The invention relates to a multi-scenario robust sea-sky-line detection method. Background Art

[0002] Sea-sky-line detection plays a wide and important role in the field of marine engineering. [1] By analyzing the sea-sky line, the attitude information of the ship is calculated and an attitude measurement method is provided. [2] The sea-sky line is used to obtain the sea-sky area, and the speed and accuracy of the target detection algorithm are improved by limiting the search range. [3] The images of adjacent frames are registered by detecting the sea-sky line. [4] The attitude of the micro-aircraft is estimated by the sea-sky line. However, the existing methods have the problems of large usage scenarios and lack of robustness when estimating the sea-sky line. Summary of the invention

[0003] The purpose of the present invention is to solve the problem that the current mainstream sea-sky-line detection algorithms have large limitations in usage scenarios and lack robustness, and thus propose a multi-scenario robust sea-sky-line detection method.

[0004] A multi-scenario robust sea-sky-line detection method has the following specific processes:

[0005] Step 1: Detect the input image;

[0006] If it is an RGB image, extract the R channel image and the B channel image of the RGB image, and store the R channel image and the B channel image separately; execute step 2;

[0007] If it is a grayscale image, go to step 2;

[0008] Step 2: Perform NSCT decomposition on the R channel image or grayscale image in step 1 to obtain a low-frequency image IM;

[0009] Step 3: Detect the edge of the low-frequency image IM using the Canny operator to obtain an edge image EM;

[0010] Step 4: remove the connected domain from the edge image EM, and dilate the image after the connected domain is removed;

[0011] Step 5: Perform Hough transform on the edge image expanded in step 4 to obtain a candidate straight line set and the value of the accumulator HN in the Hough transform corresponding to each straight line in the candidate straight line set;

[0012] Step 6: Use the HGS method to judge the candidate straight line concentration straight line to obtain the real sea-sky line.

[0013] The beneficial effects of the present invention are:

[0014] The present invention proposes a robust algorithm with a wide range of applications. First, the physical characteristics carried by light waves of different wavelengths are analyzed, and the R channel image is decomposed by NSCT, and then the Canny operator is used to extract the edge of the low-frequency part. Since the Canny operator has high sensitivity and less interference information in the low-frequency signal, the fuzzy edge can be extracted better. Then, the interference is removed by morphological processing and the straight line to be measured is extracted using Hough transform, and then the straight line to be measured is judged by GHS (grayscale, gradient, entropy value) method to obtain the final sea-sky line. Finally, various popular methods are compared through experiments and analysis, and it is proved that the algorithm has excellent robustness and wide applicability. However, since the entropy value and grayscale information relied on by the method are easily destroyed by complex textures, and the method is too sensitive to low-frequency interference, in the above occasions, the result will be offset, resulting in reduced accuracy of the algorithm.

[0015] Sea-sky-line detection has important applications in target detection, motion control and perception technology of unmanned equipment. Current sea-sky-line detection algorithms generally have the limitations of poor detection of weak sea-sky-line, susceptibility to interference from obstacles and narrow application scope. This paper proposes a sea-sky-line detection algorithm based on the robustness of low-frequency information and physical features of images (NC-HGS). First, the image is decomposed into multiple scales using non-downsampled counterlet transform, and then the undetermined lines are screened out through Canny edge detection, morphological processing and Hough transform. Finally, the obtained straight lines are evaluated by comprehensively considering five evaluation parameters, and the line with the highest evaluation score is the true horizon. This algorithm can not only accurately identify the sea-sky-line in various scenes, but also detect it when the sea-sky-line is blurred and there is strong interference in the scene. The performance of this algorithm is verified by using 6,000 images of various scenes. Compared with the most mainstream algorithms at present, this algorithm has a wider range of applications, higher accuracy and stronger robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the present invention;

[0017] Figure 2a is the original picture;

[0018] Figure 2b for Figure 2a R channel grayscale schematic diagram;

[0019] Figure 2c for Figure 2a Schematic diagram of the B channel grayscale;

[0020] Figure 3a This is a schematic diagram of the edge of the original image;

[0021] Figure 3b for Figure 3a Schematic diagram of the R channel edge;

[0022] Figure 3c for Figure 3a Schematic diagram of the B channel edge;

[0023] Figure 4a This is a schematic diagram of the low-frequency edge of the R channel;

[0024] Figure 4b This is a schematic diagram of the low-frequency edge of the original image;

[0025] Figure 4c Schematic diagram of the edge of the original image;

[0026] Figure 4d Schematic diagram of edge extraction (original image) using the LSD method;

[0027] Figure 5a is the original edge diagram;

[0028] Figure 5b This is a schematic diagram of the edge after morphological processing;

[0029] Figure 6a for Figure 2a Schematic diagram of the row grayscale mean curve in ;

[0030] Figure 6b for Figure 2a Schematic diagram of the average gradient curve of the sea level;

[0031] Figure 7 for Figure 9b Schematic diagram of entropy characteristics;

[0032] Figure 8a for Figure 9b Medium entropy distribution graph;

[0033] Figure 8b for Figure 9b Medium entropy gradient distribution diagram;

[0034] Figure 9a Figure 1 is a schematic diagram of the detection effect;

[0035] Figure 9b Figure 2 shows the detection effect;

[0036] Fig.9c Figure 3 is a schematic diagram of the detection effect;

[0037] Figure 9d Figure 4 is a schematic diagram of the detection effect;

[0038] Fig.9e Figure 5 is a schematic diagram of the detection effect;

[0039] Figure 9f Figure 6 is a schematic diagram of the detection effect;

[0040] Figure 9g Figure 7 is a schematic diagram of the detection effect;

[0041] Figure 9h Figure 8 is a schematic diagram of the detection effect;

[0042] Fig.10a This is a schematic diagram of default parameters;

[0043] Fig.10b This is the schematic diagram after adjusting the parameters; DETAILED DESCRIPTION

[0044] Specific implementation method 1: Combination Figure 1 This embodiment is described. The specific process of a multi-scenario robust sea-sky-line detection method in this embodiment is as follows:

[0045] Sea-sky-line detection: According to Rayleigh's scattering law, the longer the wavelength, the stronger the penetrating power of light waves. When it is dawn or dusk, because the position of the sun is almost parallel to the observation, a large number of light waves are scattered by particles in the air as the light penetrates the air, so the sky during this period appears orange. For RGB images, the edges of objects are more prominent for the red channel at this time. For images in the blue channel, the grayscale changes of the image are relatively gentle. In the area near the sea-sky line, there are more water molecule particles in the air, and the observer is very far away, so almost all wavelengths of light are scattered into the air, so a white strip pattern with uniform grayscale appears above the sea-sky line.

[0046] The NC method refers to detecting the low-frequency components of an image through NSCT transformation and Canny operator to obtain edge features that change slowly in the image. Since the grayscale above the sea-sky line is uniform and the direction of the lines changes slowly, the sea-sky line belongs to the low-frequency part in the entire image. For an image, parts with complex contours and prominent edge textures belong to the high-frequency part. In the process of sea-sky line detection, such signals are interference and noise. Based on wavelet transform theory, the image signal can be decomposed in the frequency domain. Then, by processing the slowly changing low-frequency image, the weak edges can be enhanced, and the real sea-sky line can be detected among various interferences.

[0047] The entropy value in an image can represent the amount of information in the image. Generally speaking, the sky area above the sea-sky line has a simpler texture and a lower entropy value, while the sea surface below the sea-sky line contains more abundant information and a higher entropy value. When there are objects such as ships and islands on the sea surface, or when there are heavy clouds in the sky, the information they carry is often stronger and has a higher entropy value. The gradient of the entropy value is also more obvious. By calculating the entropy value in the neighborhood of the image, the foreground information contained therein can be identified and the interference caused by it can be eliminated.

[0048] Based on the above principle, this algorithm obtains a low-frequency image by multi-scale decomposition of the R channel image, and then obtains a candidate line set through edge detection and morphological processing combined with Hough transform. Finally, the HGS (grayscale, gradient, and entropy) method proposed in this invention is used to judge the line and obtain the real sea-sky line.

[0049] Step 1: Detect the input image;

[0050] If it is an RGB image, extract the R channel image and the B channel image of the RGB image, and store the R channel image and the B channel image separately; execute step 2;

[0051] If it is a grayscale image, go to step 2;

[0052] Step 2: Perform NSCT (multi-scale) decomposition on the R channel image or grayscale image in step 1 to obtain a low-frequency image IM;

[0053] Step 3: Detect the edge of the low-frequency image IM using the Canny operator to obtain an edge image EM;

[0054] Step 4: remove the connected domain from the edge image EM, and dilate the image after the connected domain is removed;

[0055] Step 5: Perform Hough transform on the edge image expanded in step 4 to obtain a candidate straight line set and the value of the accumulator HN in the Hough transform corresponding to each straight line in the candidate straight line set;

[0056] The present invention removes the connected domain of the edge image to suppress interference. Then, all possible lines in the edge image are obtained by Hough transform. All possible straight lines constitute a candidate straight line set. For a certain straight line in the candidate straight line set, the stronger its linear feature is, the higher the value in the accumulator HN of the Hough transform corresponding straight line is. The linear feature of the sea-sky line is very strong. Therefore, the higher the value of the straight line in the accumulator HN is, the more likely it is the sea-sky line.

[0057] The Hough transform detects straight lines by counting the number of "votes" of points on the edge for the line. The advantage of this is that the voting results are highly robust, and the most likely straight line segment can be obtained by counting the HN value of the Hough transform accumulator.

[0058] Step 6: Use the HGS (grayscale, gradient, and entropy) method to judge the straight lines in the candidate straight line set and obtain the true sea-sky line.

[0059] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that in step 6, the straight line in the candidate straight line concentration is judged by the HGS (grayscale, gradient, and entropy value) method to obtain a true sea-sky line; the specific process is:

[0060] Step 61: Search the upper and lower neighborhoods along the candidate straight line concentration line to obtain the cumulative grayscale R1, R2, and calculate the cumulative grayscale difference;

[0061] Step 62: Search the upper and lower neighborhoods along the candidate straight line concentration line to obtain the cumulative gradient;

[0062] Step 6.3: Search M in the upward neighborhood L along the candidate straight line concentration line to obtain the cumulative entropy value in the neighborhood;

[0063] Step 64: Count the cumulative entropy gradients of the upper and lower L neighborhoods along the candidate straight line concentration line;

[0064] Step 65: Based on the value of the accumulator HN in the Hough transform, the accumulated grayscale difference, the accumulated gradient, the accumulated entropy value, and the accumulated entropy value gradient, calculate the value of GHS. The highest value is the real sea-sky line.

[0065] Image multi-scale decomposition and edge detection

[0066] Since seawater absorbs long waves better, seawater generally appears blue, and the B channel is sensitive to wave targets. Since the sea-sky line is far away and red waves have strong penetration, the R channel can better identify it. Potential sea-sky line targets are screened out by performing edge detection on the low-frequency information of the R channel of the image. The image in the R channel has a high contrast, a large grayscale change, and more high-frequency signals. The image with a gentle color change in the B channel also has more low-frequency signals. After NSCT decomposition and Canny edge detection, the edge of the R channel is smoother and sparser, and contains less noise.

[0067] Contourlet transform (non-adaptive directional multi-scale analysis method) is a directional image sparse representation method, which uses Laplacian pyramid and directional filter group to obtain directional sub-bands of different scales. However, Contourlet transform includes sampling steps in the process of image decomposition and reconstruction, so the decomposed image does not have translation invariance. Non-subsampled Contourlet transform uses non-subsampled pyramid and directional filter group to achieve multi-scale decomposition, and obtains sub-band images of the same size as the source image, ensuring translation invariance. There is no downsampling operation in NSCT transform, and the image is upsampled in a way similar to the hole algorithm. Since the Canny operator can easily locate the extracted edges. At the same time, since the Canny operator uses Gaussian filtering and double thresholds to detect edges, it can well extract the fuzzy sea-sky line in the strong interference image.

[0068] Figure 3a , 3b , 3c show the edge detection effect of each channel. From the above comparison, we can see that when the sea-sky line is blurred and there is a lot of interference, the low-frequency information of the R channel can better represent the position of the sea horizon. However, the sea-sky line extracted or simulated in references [1] to [7] did not decompose the R channel image and could not detect the sea horizon. Figure 9a Blurred sea horizon in the sky.

[0069] Morphological processing and Hough transform: Hough transform detects straight lines by counting the number of "votes" of points on the edge for the line. The advantage of this is that the voting results are highly robust, and the most likely straight line segments can be obtained by counting the H values. In real atlases, ships, islands, reefs, etc. will cause strong interference to the detection of the sea-sky line. Especially in blurred images, the strength of the sea-sky line itself is not good, and there may be distortion and deformation, which increases the difficulty of Hough transform detection of straight lines. By removing the connected domain of the binary edge image, the interference of small noise on the waves or ships can be effectively eliminated to avoid the occurrence of "miscasting". Ensure that the final statistical H value is stable and effective. Then, the processed image is expanded and Hough transformed to obtain a candidate line set.

[0070] Figure 5a , 5b This is the result of edge detection. After being processed by the connected domain, the right edge becomes smoother and clearer, which is conducive to the identification of the sea-sky line.

[0071] Table 1 Pseudo code block for sea level detection

[0072]

[0073] Sea-sky-line estimation based on HGS: By analyzing the physical characteristics of the sea-sky-line, the present invention proposes a new voting method, the HGS method, which evaluates the candidate images by analyzing the grayscale, local entropy value and corresponding gradient changes near the candidate line segments, combined with the H value obtained by the Hough transform, and selects the line segment with the best evaluation as the true sea-sky-line.

[0074] The other steps and parameters are the same as those in the first embodiment.

[0075] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that in step six one, the cumulative grayscale R1, R2 are searched in the upper and lower neighborhoods along the candidate straight line concentration line to obtain the cumulative grayscale difference H; the specific process is:

[0076] Grayscale index: In clear weather and with a clear view, when observed from the coast or ship, the grayscale of the sea-sky line changes smoothly from bottom to top. However, when the light is dark and there are heavy clouds in the sky, the grayscale difference near the sea-sky line is obvious. Figure 2a , 2b As shown in Figure 2c, the grayscale changes smoothly in the blue channel (B), which can effectively remove the interference caused by sunlight. In most cases, there will be obvious differences in the grayscale above and below the sea-sky line. The sea-sky line itself is long, and obstacles in the distance or thin clouds are unlikely to change the overall grayscale distribution of the sea-sky line. By comparing the statistical row mean gradient and row mean grayscale, the algorithm can determine the area where the sea-sky line is located.

[0077] Figure 6a In the figure, the black straight line represents the location of the sea-sky line. Figure 2a In the sky area, the grayscale gradient is gentle, while in the sea-sky line area, the grayscale changes dramatically when it is close to the ocean. At the same time, the grayscale above the sea-sky line is significantly higher than the grayscale below. Therefore, we can determine whether a straight line segment is a sea-sky line by defining an area and counting the cumulative grayscale difference and cumulative gradient difference near the line segment.

[0078] If the straight line is expressed in intercept form, the calculation method of the cumulative grayscale difference is:

[0079]

[0080] In the formula, f B represents the image or grayscale image of the B channel in the RGB image in step 1, q represents the qth column in the image or grayscale image of the B channel, z represents the vertical coordinate upward or downward in the vertical direction of the straight line, · represents multiplication, K and b represent the slope and intercept of the sea-sky line respectively, and V represents f B The width of the image, N is f BThe height of the image; R1 and R2 distributions represent the accumulated grayscale values ​​of searching the downward and upward neighborhoods along the candidate line; H is the grayscale difference between the two ends. The larger the value of H, the greater the possibility that it is a sea-sky line;

[0081] Image f is segmented from left to right using the region segmentation method. B Perform segmentation to obtain the cumulative grayscale difference H of each block, remove the maximum value and the second largest value and the minimum value and the second smallest value of the cumulative grayscale difference H corresponding to all blocks; calculate the mean of the remaining cumulative grayscale difference H; this filtering method does not introduce redundant calculations, and can improve accuracy while saving computing power;

[0082]

[0083] Where H′ is the mean of the cumulative grayscale difference H, SUM represents the cumulative grayscale difference of each area after removing the second largest value, second smallest value, maximum value, and minimum value, W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks.

[0084] The other steps and parameters are the same as those in the first or second embodiment.

[0085] Specific implementation method 4: This implementation method is different from one of the specific implementation methods 1 to 3 in that in step 62, the cumulative gradient is obtained by searching the upper and lower neighborhoods along the candidate straight line concentration line; the specific process is:

[0086] Figure 6b The vertical gradient information of the image along the sea-sky line is displayed. It can be seen that the gradient will suddenly change along the sea-sky line, and the derivatives of the gradient on both sides of the sea-sky line are very different. B (z, q) is the pixel value of a point along the vertical direction of the sea-sky line, then f B The magnitude of the gradient value at point (z, q) is:

[0087] G B (z, q) = |f B (z,q)-f B (z-1,q)|, z=(1,2,…,N), q=(1,2,…,V) (3)

[0088] Where V represents f B The width of the image, N is f B The height of the image;

[0089] Figure 6b In the figure, the position of the black line is the sea-sky line marked manually; it is not always at the position where the gradient and grayscale change is the largest. Since the present invention does not need to locate the point with the largest change, the algorithm uses the cumulative gradient to measure the straight line:

[0090]

[0091] Where HT is the cumulative gradient value of the image in a neighborhood with a radius of N;

[0092] However, objects such as ships often appear on the sea-sky line to destroy the original shape of the sea-sky line, reducing the accuracy of sea-sky line detection. The grayscale value of obstacles on the sea-sky line is relatively uniform within the area itself, but when the area changes, the grayscale changes significantly and the gradient value is large.

[0093] Image f is segmented from left to right using the region segmentation method. B (z, q) is segmented to obtain the cumulative gradient value HT of each block, and the maximum value, the second largest value, the minimum value, and the second smallest value of the cumulative gradient values ​​HT corresponding to all blocks are removed; the mean of the remaining cumulative gradient values ​​HT is calculated; this filtering method does not introduce redundant calculations, and can save computing power while improving accuracy;

[0094]

[0095] In the formula, HT′ is the mean of the cumulative gradient value HT, SUM HT Represents the cumulative gradient value of each area after removing the second largest value, second smallest value, maximum value, and minimum value. W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks.

[0096] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0097] Specific implementation method five: This implementation method is different from one of the specific implementation methods one to four in that in step six-three, a search is performed along the candidate straight line concentration straight line upward neighborhood L to obtain the cumulative entropy value in the neighborhood; the specific process is:

[0098] Entropy index: When there is a strongly interfering object on the sea-sky line, its grayscale characteristics will be destroyed. Sometimes, when the interference is too strong, the grayscale value and gradient will present opposite states. Therefore, the entropy index should be used to judge it. The entropy value of an image can express the degree of confusion of the information. Generally speaking, the texture information of interfering objects and sea surface areas is richer than that of the sky, and the entropy value is higher. By performing local entropy filtering on the image, the regional entropy value of each area can be obtained, and the location of the sea-sky line can be judged by the entropy value characteristics;

[0099] Performing local entropy filtering on the image of the B channel in the RGB image or the grayscale image in step 1 to obtain an entropy image after local entropy filtering;

[0100] The calculation formula is as follows:

[0101]

[0102] pi =num(i) / (N E ) 2 (7)

[0103] f E (q,z)=H E , z=(1,2,…,N), q=(1,2,…,V) (8)

[0104] Among them, H E (q,z) is the entropy value of point (q,z), N E is the size of the filtering neighborhood, p i For area N E The probability of the i-th gray level appearing in the filter area, num(i) is the total number of times the i-th gray level appears in the filter area, and f E is the entropy value image after local entropy filtering; V represents f B The width of the image, N is f B The height of the image; the point (q,z) is the image of the B channel in the RGB image in step 1 or the midpoint of the grayscale image;

[0105] Figure 9b The texture features below the sea-sky line are stronger, significantly higher than those above the sea-sky line. In the lower half of the entropy value, the long traces left by the submarine are not significantly different from the surrounding grayscale. Under the two indicators of grayscale and grayscale gradient cumulative difference, this straight line segment that runs through the entire image and has strong grayscale and gradient changes will cause strong interference to the two indicators.

[0106] At the same time, a method similar to cumulative grayscale gradient is used to count the entropy gradient characteristics of candidate lines in a certain neighborhood; the ones with small cumulative entropy values ​​and large cumulative entropy gradients are judged as real sea-sky lines;

[0107]

[0108] In the formula, f E is the entropy image after local entropy filtering, S M is the final entropy index, K and b represent the slope and intercept of the straight line respectively, V is the width of the image, and N S is the window size for searching vertically upwards along the image;

[0109] Image f is segmented from left to right using the region segmentation method. E (z,q) is split to obtain the entropy index S of each block M , remove the entropy index S corresponding to all blocks M The maximum and second largest values, as well as the minimum and second smallest values ​​in the equation; calculate the residual cumulative entropy index S MThis filtering method does not introduce redundant calculations and can save computing power while improving accuracy.

[0110]

[0111] Where S is the cumulative entropy value, SUM s It represents the cumulative entropy value of each area after removing the second largest value, second smallest value, maximum value, and minimum value. W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks.

[0112] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0113] Specific implementation method 6: This implementation method is different from any one of the specific implementation methods 1 to 5 in that the cumulative entropy value gradient of the upper and lower L neighborhoods along the candidate straight line concentration line is counted in step 64; the specific process is:

[0114] In step 63, we get the entropy image f E Then, we can get f by formula 8. E Entropy map gradient map of ;

[0115] If we define f E A point in is f E (z,q), then the entropy graph gradient graph for a certain point G E (z,q) is:

[0116] G E (z,q)=|f E (z,q)-f E (z-1,q)|, z=(1,2,…,N), q=(1,2,…,V)(11)

[0117] Get the entropy graph gradient graph G E After that, the cumulative entropy gradient HE of the upper and lower neighborhoods of a candidate straight line is calculated;

[0118] The calculation process is shown in Formula 9, where the width of the upper and lower neighborhoods is N S2 ;

[0119]

[0120] Where HE is the cumulative entropy gradient value of the image in a neighborhood with a radius of N;

[0121] Image G is segmented from left to right using the region segmentation method. E (z, q) is segmented to obtain the cumulative entropy gradient HE of each block, and the maximum value, the second largest value, the minimum value, and the second smallest value of the cumulative entropy gradient HE corresponding to all blocks are removed; the mean value of the remaining cumulative entropy gradient HE is calculated;

[0122]

[0123] Where HE′ is the mean of the cumulative entropy gradient HE, SUM HE Represents the cumulative entropy gradient of each region after removing the second largest value, second smallest value, maximum value, and minimum value. W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks.

[0124] When there are ships on the sea, thick clouds in the sky, or obstructions above the photographic equipment, the original entropy information of the sky and the ocean will be destroyed, and the distinction between them will no longer be clear. By combining with the grayscale features of the image, it can play a good complementary role and improve the accuracy of the algorithm.

[0125] When there are ships on the sea, thick clouds in the sky, or obstructions above the photographic equipment, the original entropy information of the sky and the ocean will be destroyed, and the distinction between them will no longer be clear. By combining with the grayscale features of the image, it can play a good complementary role and improve the accuracy of the algorithm.

[0126] The other steps and parameters are the same as those in Specific Implementation Methods 1 to 5.

[0127] Specific implementation method 7: This implementation method is different from the specific implementation methods 1 to 6 in that in step 65, the value of GHS is calculated based on the value of the accumulator HN in the Hough transform, the accumulated grayscale difference, the accumulated gradient, the accumulated entropy value, and the accumulated entropy value gradient, and the highest value is the real sea-sky line; the specific process is:

[0128] Sea-sky-line estimation under the GHS index: The gray value index (H) and gray gradient (HG) are suitable for occasions where the sea-sky-line is long and the interference information is relatively short. Among them, the gray gradient (HG) is also applicable to infrared images. Entropy information (S) and entropy gradient information (SG) have stronger anti-interference capabilities, but are sensitive to interference information in the sky (such as clouds). Among them, (SG) information is also suitable for the discrimination of infrared images. At the same time, the Hough transform is sensitive to the edges in binary images and has good robustness. Therefore, the value of the count matrix H in Hough is taken into account and denoted as N. Considering the different characteristics of sea-sky-line species, the evaluation method GHS index (grayscale, gradient, entropy) designed by the present invention is as follows

[0129]

[0130] In the formula, GHS is the final evaluation index for the straight line, and a~e are the weights of the five types of indicators respectively; (z1) HN is z1 raised to the power of HN, (z2) H′ is z2 to the power of H′, (z3) HT′ is z3 raised to the HT′ power, (z4)1 / S is z4 to the power of 1 / S, (z5) HE′ It is z5 raised to the power of HE′, and abs() represents the absolute value.

[0131] In this algorithm, a=1, b=1.2, c=0.5, d=1.3, e=1. These five weights are determined according to the requirements of the maximum applicable range. If a certain type of image needs to be detected separately, the values ​​of a~e can be recalculated. For example, in infrared images, the values ​​of a and c can be appropriately increased and the value of d can be reduced to make the algorithm more stable.

[0132] Table 2 Pseudo code block for sea level estimation

[0133]

[0134] The following examples are used to verify the beneficial effects of the present invention:

[0135] Embodiment 1:

[0136] The present invention proposes a method for detecting sea-sky line based on NSCT decomposition and Canny edge detection (NC method). It can not only filter out most edge noise in the image and make the image smooth, but also accurately locate the position of the sea-sky line in the image in various interferences. In order to prove the stability of the algorithm, the present invention is compared with the method on visible light image and infrared image. [5][6][7] Perform a direct comparison (directly compare through code).

[0137] about Figure 9a , 9b In 9c, 9d, 9e, 9f, 9g, and 9h, line 1 is the sea-sky line obtained by this method, and lines 2, 3, and 4 are respectively obtained from the literature. [5][7][6] The sea-sky line is obtained by the method in Figure 9a , 9b , 9c, 9d, 9e, 9f, 9g, and 9h show that this method is highly robust and can be applied to various scenarios. Figure 9a In the image, the grayscale difference between the upper and lower parts of the sea-sky line is not obvious, and there is serious interference at the bottom. For such images, it is impossible to directly detect the sea-sky line using the Canny operator or the LSD operator. [5][7][6] It is difficult to detect it by other methods. Figure 9b The wake of the submarine in the middle and lower part runs through the entire image, and it is difficult for gradient and edge-based methods to detect the true edge. Fig.9c In the image, the heavy clouds in the sky destroy the texture details of the sky. The gray-level co-occurrence matrix-based method and the OSTU-based method are difficult to distinguish the true sea-sky line. Fig.9c The left side of the sea-sky line is incomplete, and the hill destroys the original entropy characteristics. The accuracy of various methods based on entropy and straight-line fitting will also decrease. Figure 9d , Figure 9f The blurred sea horizon and irregular waves destroy the original texture details of the image, so edge-based methods are ineffective here. Fig.9e In the present study, due to the serious wave interference, the other comparison methods cannot play a good effect. Figure 9g and Figure 9h In the image, since obstacles cut the original texture of the sea-sky line and occupy most of the space, it is impossible to accurately count the texture information of the sky and the ocean.

[0138] Since the real sea-sky line always appears in the first three items of N, in order to speed up the operation of the algorithm, this algorithm searches the first three items with high votes, thereby achieving faster operation efficiency.

[0139] However, the parameters used in this paper are only empirical parameters and are not applicable to all occasions. For example, when detecting coastline and dock targets, the H (grayscale) S (entropy) index will play the opposite role and interfere with the original evaluation system. For example, when there is a large-scale interference in front of the sea-sky line, the original entropy value feature will be destroyed and the gradient value index will point in the opposite direction. Therefore, it needs to be adjusted in actual use.

[0140] However, since this method is very sensitive to low-frequency information when extracting edges, it will extract multiple straight line information near the sea-sky line, causing interference to the detection. Therefore, when the sea surface ripple signal is relatively complex, the detection accuracy is not high. When the texture near the sea-sky line is complex and there is a lot of low-frequency interference, since the Hough transform is only sensitive to the edge, it will cause the detected straight line to shift, which will also cause the detection result to shift.

[0141] References:

[0142] [1] Feng Hongkui, Fu Minhui, Zhong Dean, et al. Ship attitude algorithm based on sea-sky line[J]. Ordnance Automation, 2019, 38(5): 37-40.

[0143] [2] Zhang Xun, Ma Haobo, Li Yuncheng. Sea-sky-line detection of UUV sea surface infrared images based on joint optimization of row mean gradient and straight line fitting[J]. Applied Science and Technology, 2018, 45(4): 6-12.

[0144] [3]S.Fefilatyev,D.Goldgof,M.Shreve,and C.Lembke,‘Detection andtracking of ships in open sea with rapidly moving buoy-mounted camerasystem’,Ocean Engineering,vol.54,pp.1–12,Nov.2012,doi:10.1016 / j.oceaneng.2012.06.028

[0145] [4]S.M.Ettinger,M.C.Nechyba,P.G.Ifju,and M.Waszak,‘Vision-guidedflight stability and control for micro air vehicles’,Advanced Robotics,vol.17,no.7,pp.617–640,Jan.2003,doi:10.1163 / 156855303769156983.

[0146] [5]M.Schwendeman and J.Thomson,‘A Horizon-Tracking Method forShipboard Video Stabilization and Rectification’,Journal of Atmospheric andOceanic Technology,vol.32,no.1,pp.164–176,Jan.2015,doi:10.1175 / JTECH-D-14-00047.1.

[0147] [6]LIANG D,LIANG Y.Horizon detection from electrooptical sensorsunder maritime environment[J].IEEE Transactions on Instrumentation andMeasurement,2019,69(1):45-53.

[0148] [7]J.Hou and B.Li,‘An Improved Algorithm for Horizon Detection Basedon OSTU’,p.4。

Claims

1. A multi-scenario robust sea-sky-line detection method, characterized by: The specific process of the method is: Step 1: Detect the input image; If it is an RGB image, extract the R channel image and the B channel image of the RGB image, and store the R channel image and the B channel image separately; execute step 2; If it is a grayscale image, go to step 2; Step 2: Perform NSCT decomposition on the R channel image or grayscale image in step 1 to obtain a low-frequency image IM; Step 3: Detect the edge of the low-frequency image IM using the Canny operator to obtain an edge image EM; Step 4: remove the connected domain from the edge image EM, and dilate the image after the connected domain is removed; Step 5: Perform Hough transform on the edge image expanded in step 4 to obtain a candidate straight line set and the value of the accumulator HN in the Hough transform corresponding to each straight line in the candidate straight line set; Step 6: Use the HGS method to judge the candidate straight line concentration straight line to obtain the real sea-sky line.

2. The multi-scenario robust sea-sky-line detection method according to claim 1, characterized in that: In step 6, the candidate straight line concentration straight line is judged by the HGS method to obtain a true sea-sky line; The specific process is: Step 61: Search the upper and lower neighborhoods along the candidate straight line concentration line to obtain the cumulative grayscale R1 and R2, and calculate the cumulative grayscale difference; Step 62: Search the upper and lower neighborhoods along the candidate straight line concentration line to obtain the cumulative gradient; Step 6.3: Search M in the upward neighborhood L along the candidate straight line concentration line to obtain the cumulative entropy value in the neighborhood; Step 64: Count the cumulative entropy gradients of the upper and lower L neighborhoods along the candidate straight line concentration line; Step 65: Based on the value of the accumulator HN in the Hough transform, the accumulated grayscale difference, the accumulated gradient, the accumulated entropy value, and the accumulated entropy value gradient, calculate the value of GHS. The highest value is the real sea-sky line.

3. A multi-scenario robust sea-sky-line detection method according to claim 2, characterized in that: In step 61, the cumulative grayscale R1 and R2 are obtained by searching the upper and lower neighborhoods along the candidate straight line concentration line, and the cumulative grayscale difference H is obtained; the specific process is: The calculation method of the cumulative grayscale difference is: In the formula, f B represents the image or grayscale image of the B channel in the RGB image in step 1, q represents the qth column in the image or grayscale image of the B channel, z represents the vertical coordinate upward or downward in the vertical direction of the straight line, · represents multiplication, K and b represent the slope and intercept of the sea-sky line respectively, and V represents f B The width of the image, N represents f B The height of the image; R1 and R2 represent the accumulated grayscale values ​​of searching the downward and upward neighborhoods along the candidate line; H is the grayscale difference between the two ends. The larger the value of H, the greater the possibility that it is a sea-sky line; Image f is segmented from left to right using the region segmentation method. B Perform segmentation to obtain the cumulative grayscale difference H of each block, remove the maximum value and the second largest value as well as the minimum value and the second smallest value of the cumulative grayscale difference H corresponding to all blocks; and calculate the average value of the remaining cumulative grayscale difference H; Where H′ is the mean of the cumulative grayscale difference H, SUM represents the cumulative grayscale difference of each area after removing the second largest value, second smallest value, maximum value, and minimum value, W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks.

4. The multi-scenario robust sea-sky-line detection method according to claim 3, characterized in that: In the step 62, the cumulative gradient is obtained by searching the upper and lower neighborhoods along the candidate straight line concentration line; the specific process is: Define f B (z, q) is the pixel value of a point along the vertical direction of the sea-sky line, then f B The magnitude of the gradient value at point (z, q) is: G B (z,q)=|f B (z,q)-f B (z-1,q)|,z=(1,2,…,N),q=(1,2,…,V) (3) Where V represents f B The width of the image, N is f B The height of the image; Take the cumulative gradient to measure the straight line: Where HT is the cumulative gradient value of the image in a neighborhood with a radius of N; Image f is segmented from left to right using the region segmentation method. B (z, q) is segmented to obtain the cumulative gradient value HT of each block, and the maximum value, the second largest value, the minimum value, and the second smallest value of the cumulative gradient values ​​HT corresponding to all blocks are removed; and the average value of the remaining cumulative gradient values ​​HT is calculated; In the formula, HT′ is the mean of the cumulative gradient value HT, SUM HT Represents the cumulative gradient value of each area after removing the second largest value, second smallest value, maximum value, and minimum value. W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks.

5. A multi-scenario robust sea-sky-line detection method according to claim 4, characterized in that: In the step 63, a search M is performed along the candidate straight line concentration line upward neighborhood L to obtain the cumulative entropy value in the neighborhood; the specific process is: Performing local entropy filtering on the image of the B channel in the RGB image or the grayscale image in step 1 to obtain an entropy image after local entropy filtering; The calculation formula is as follows: p i =num(i) / (N E ) 2 (7) f E (q,z)=H E ,z=(1,2,…,N),q=(1,2,…,V) (8) Among them, H E (q,z) is the entropy value of point (q,z), N E is the size of the filtering neighborhood, p i For area N E The probability of the i-th gray level appearing in the filter area, num(i) is the total number of times the i-th gray level appears in the filter area, and f E is the entropy value image after local entropy filtering; V represents f B The width of the image, N is f B The height of the image; the point (q,z) is the image of the B channel in the RGB image in step 1 or the midpoint of the grayscale image; In the formula, f E is the entropy image after local entropy filtering, S M is the final entropy index, K and b represent the slope and intercept of the straight line respectively, N S is the window size for searching vertically upwards along the image; Image f is segmented from left to right using the region segmentation method. E (z,q) is split to obtain the entropy index S of each block M , remove the entropy index S corresponding to all blocks M The maximum and second largest values, as well as the minimum and second smallest values ​​in the equation; calculate the residual cumulative entropy index S M The mean of Where S is the cumulative entropy value, SUM s It represents the cumulative entropy value of each area after removing the second largest value, second smallest value, maximum value, and minimum value. W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks.

6. A multi-scenario robust sea-sky-line detection method according to claim 5, characterized in that: In step 64, the cumulative entropy value gradient of the upper and lower L neighborhoods along the candidate straight line concentration line is counted; the specific process is: In step 63, we get the entropy image f E Then, we can get f by formula (8) E Entropy map gradient map of ; If we define f E A point in is f E (z,q), then the entropy graph gradient graph for a certain point G E (z,q) is: G E (z,q)=|f E (z,q)-f E (z-1,q)|,z=(1,2,…,N),q=(1,2,…,V) (11) Get the entropy graph gradient graph G E After that, the cumulative entropy gradient HE of the upper and lower neighborhoods of a candidate straight line is calculated; The calculation process is shown in formula (9), where the width of the upper and lower neighborhoods is N S2 ; Where HE is the cumulative entropy gradient value of the image in a neighborhood with a radius of N; Image G is segmented from left to right using the region segmentation method. E (z, q) is segmented to obtain the cumulative entropy gradient HE of each block, and the maximum value, the second largest value, the minimum value, and the second smallest value of the cumulative entropy gradient HE corresponding to all blocks are removed; the mean value of the remaining cumulative entropy gradient HE is calculated; Where HE′ is the mean of the cumulative entropy gradient HE, SUM HE It represents the cumulative entropy gradient of each region after removing the second largest value, second smallest value, maximum value, and minimum value. W represents the second largest value, M represents the second smallest value, and m represents the total number of blocks.

7. The multi-scenario robust sea-sky-line detection method according to claim 6, characterized in that: In step 65, based on the value of the accumulator HN in the Hough transform, the accumulated grayscale difference, the accumulated gradient, the accumulated entropy value, and the accumulated entropy value gradient, the value of GHS is calculated, and the highest value is the real sea-sky line; the specific process is: The GHS indicators are as follows In the formula, GHS is the final evaluation index for the straight line, and a~e are the weights of the five types of indicators respectively; (z1) HN is z1 raised to the power of HN, (z2) H′ is z2 to the power of H′, (z3) HT′ is z3 raised to the HT′ power, (z4) 1 / S is z4 to the power of 1 / S, (z5) HE′ It is z5 raised to the power of HE′, and abs() represents the absolute value.

Images