Seascape real-time detection method and system

Abstract

The present application belongs to the field of image processing and computer vision technology, and particularly relates to a sea-sky line real-time detection method and system. The present application divides the sea-sky line detection into two parts of coarse calculation and precise calculation. The first part is coarse calculation, the method of which is to down-sample and calculate a longitudinal gray gradient graph for the collected visible light or infrared image, to carry out threshold segmentation on the longitudinal gray gradient graph to obtain a binary image, to calculate the boundary of the binary image, and to calculate the approximate intercept and slope of the sea-sky line through hough transformation. The second part is precise calculation, the method of which is to search for the maximum value position of the image longitudinal gray gradient graph in the sea-sky line vicinity area of the longitudinal gray gradient graph calculated in the first part according to the sea-sky line intercept and slope calculated in the first part, to carry out hough transformation on the obtained position points, and then to calculate the accurate position and slope of the sea-sky line. The scheme has small calculation amount, small memory occupation, simple algorithm principle and high implementation efficiency.

Description

Technical Field

[0001] This invention belongs to the field of image processing and computer vision technology, and specifically relates to a method and system for real-time detection of sea surface area. Background Technology

[0002] Shipborne optoelectronic detection systems are important equipment for the Coast Guard to carry out maritime rights protection, duty, and law enforcement activities; using optical sensors such as visible light, infrared and laser rangefinders on board the system, it can complete the search, detection and tracking of targets.

[0003] Current maritime target detection and tracking technologies face numerous challenges. First, the maritime environment is complex and variable, frequently experiencing severe weather such as rain and fog. Second, influenced by various natural factors such as sunlight, wind, and waves, the sea surface environment is constantly changing, resulting in video data collected at sea containing significant noise and disturbance information, such as chaotic fish-scale-like light patterns and irregularly moving, textured waves. Finally, maritime targets have limited visibility, experience significant video jitter, and exhibit severe positional changes, making real-time, accurate positioning difficult. Faced with these new challenges arising from complex maritime weather conditions, traditional target detection technologies can no longer meet the demands for real-time, efficient processing of maritime data. Furthermore, under sea-sky background conditions and at a long distance with a line-of-sight perspective, targets such as missiles, drones, ships, vessels, and boats always appear near the sea-sky line. Therefore, before detecting weak targets, it is necessary to first detect the sea-sky line and then process the area near it. The advantages of doing this are twofold: First, determining the sea-line can narrow down the target search area. After obtaining the location information of the sea-line, only the area near the sea-line can be searched, reducing the amount of computation and ensuring the real-time performance of the system. Second, after detecting the sea-line, the region of interest (ROI) can be divided, which can exclude unnecessary interference information outside the sea-line area, such as clouds, fog, waves, seabirds, and background on the coast, thus improving the accuracy of target detection and tracking.

[0004] In terms of sea-sky detection, most existing detection technologies have the following drawbacks: (1) The calculation process is complex and the practicality is poor; (2) The calculation efficiency is low and the real-time performance is poor; (3) They are only applicable to simple sea-sky backgrounds. In severe weather, when there are many clouds, large waves, or large target interference on the sea surface, they often fail to detect; (4) Most of them are for infrared target detection and are not suitable for visible light targets.

[0005] Patents CN106127728B ("A Method for Detecting Connected Components of Sea Lines in Infrared Images Against a Sea-Air Background"), CN104268877A ("An Adaptive Detection Method for Sea Lines in Infrared Images"), and CN105787950A ("An Algorithm for Detecting Sea Lines in Infrared Images Based on Row Gradient Accumulation") disclose an algorithm for detecting sea lines in infrared images based on row gradient accumulation. However, these methods lose a significant amount of information in the image, making it difficult to balance speed and accuracy. Furthermore, they cannot be directly applied to visible light scenes, greatly limiting their application scope. They also cannot detect tilted sea lines, hindering subsequent target detection, among other limitations. The invention, with publication number CN1O5931228A and titled "A Method for Detecting Sealine in Infrared Images Based on a Linear Search Matrix," discloses a method for detecting sealine in infrared images based on a linear search matrix. This method is still designed for infrared images and can detect tilted sealine, but it cannot resist interference from clouds, sea clutter, seabirds, ships, etc.

[0006] Current methods for detecting the sea-line of light in visible light images mainly utilize the grayscale characteristics of the sea surface and the sky. For example, the area with the largest grayscale gradient along the column direction is used as the sea-line. For instance, patent CN102279973A, entitled "Sea-line Detection Method Based on High-Gradient Key Points," discloses a method for detecting the sea-line based on high-gradient key points. However, this method has the drawback of failing when there are significant grayscale differences between clouds and the sky, or between waves and the sea surface, or when there is interference from ships, islands, or large waves. Patent CN108776778A, entitled "A Method for Detecting Sea-Line Elevation Based on Mathematical Morphology," discloses a method for detecting sea-line elevation based on mathematical morphology to remove interference points and fit straight lines. This method performs morphological processing on the image, then uses the Sobel operator to extract boundary points, and finally performs morphological processing on the boundary points before performing a Hough transform to calculate the sea-line elevation. This method has poor anti-interference capability and is easily affected by continuous and cluttered clouds in the sky and textured waves on the sea surface, resulting in pseudo-straight lines. Patent CN108447067A, entitled "A Method for Detecting Sea-Line Elevation in Visible Light Images Based on Energy Slit Clipping and RANSAC Fitting," discloses a method for detecting sea-line elevation in visible light images based on energy slit clipping and RANSAC fitting. This method performs well when the grayscale of the sea surface and sky is smooth, but it exhibits significant deviations when the color distribution in the sky is severely interfered with by the grayscale distribution of clouds. Patent CN 109493353... Invention A, titled "A Method for Detecting the Sky-Sea Boundary Against a Sea-Sky Background," proposes a method for detecting the sky-sea boundary against a sea-sky background. This method detects the sky-sea line by calculating the line-by-line gradient difference. However, this method is only effective for simple sea-sky images; it fails when there is interference from clouds, waves, seabirds, ships, etc. Furthermore, this method cannot accurately detect tilted sky-sea lines. Invention CN 109242870A, titled "A Method for Detecting Sky-Sea Lines Based on Image Segmentation and Texture Features," discloses a method for detecting sky-sea lines based on image segmentation and texture features. Publication number CN 110298855 A, entitled "A method for detecting sea-line features based on Gaussian mixture model and texture analysis," discloses a method for detecting sea-line features based on Gaussian mixture model and texture analysis. Publication number CN 109978869A, entitled "A method and system for detecting sea-line features based on gray-level co-occurrence matrix and Hough transform," discloses a method and system for detecting sea-line features based on gray-level co-occurrence matrix and Hough transform. All three methods are based on gray-level co-occurrence matrix for sea-line feature detection. Gray-level co-occurrence matrix data is large, especially for high-precision videos such as 10-bit, 12-bit, and 14-bit, where the data volume increases dramatically, resulting in very low implementation efficiency.The third method, in particular, requires image enhancement. It obtains image texture features by calculating the gray-level co-occurrence matrix, extracts image boundaries in areas with the greatest texture changes using the Candy algorithm, and finally calculates the sea-line area using Hough transform. This method has a very complex computational process, making it difficult to perform in real time and limiting its practicality. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] The technical problem to be solved by this invention is: how to propose a method and system with strong anti-interference ability that can effectively, accurately and in real time detect the sea surface line in complex marine environments, addressing the existing technical problems of sea surface line detection.

[0009] (II) Technical Solution

[0010] To solve the above technical problems, the present invention provides a method for real-time detection of the sea surface line, the method comprising:

[0011] Step 1: Upon receiving the sea-line detection command from the photoelectric detection system, acquire a frame of raw image through the corresponding sensor;

[0012] Step 2: Calculate the vertical grayscale gradient map of the downsampled original image;

[0013] Step 3: Calculate the segmentation threshold of the vertical grayscale gradient map;

[0014] Step 4: Perform threshold segmentation on the vertical grayscale gradient map according to the segmentation threshold to obtain a binarized image;

[0015] Step 5: Calculate the boundaries of the segmented binarized image and store the coordinates of the boundary points;

[0016] Step 6: Input the coordinates of the boundary points into the Hough transform function for calculation;

[0017] Step 7: Search for the location of the maximum value of the hough space output by the hough transform function, and calculate the approximate intercept and slope of the sea-line using the coordinates of that location;

[0018] Step 8: Using the approximate intercept and slope of the sea-line calculated in Step 7 as a reference, select a small area near the sea-line in the longitudinal gray-scale gradient map calculated in Step 2, search for the maximum value of each column of the longitudinal gray-scale gradient map, and store the coordinates of the maximum value.

[0019] Step 9: Input the coordinates of the maximum value in each column into the Hough transform function for calculation;

[0020] Step 10: Search for the location of the maximum value in the Hough space output by the Hough transform function, calculate the precise intercept and slope of the sea-line using the coordinates of this location, and thus obtain the position of the sea-line.

[0021] Step 11: Plot the sea line on the above image with a straight line, and report the precise intercept and slope values ​​of the sea line to the optoelectronic system.

[0022] In step 2, the formula for calculating the vertical grayscale gradient image is:

[0023]

[0024] in:

[0025]

[0026]

[0027] Gray represents the vertical grayscale gradient image after downsampling;

[0028] F represents the original image;

[0029] i and j are the row and column indices of the vertical grayscale gradient image;

[0030] sample_height is the row downsampling rate;

[0031] sample_width is the column downsampling rate;

[0032] S represents the number of rows for the vertical gradient accumulation.

[0033] In step 3, the process of calculating the segmentation threshold of the vertical grayscale gradient image is as follows:

[0034] First, calculate the mean of the vertical grayscale gradient image:

[0035]

[0036] Wherein: H Gray =H / sample_height, W Gray = W / sample_width;

[0037] H Gray W Gray These represent the height and width of the vertical grayscale gradient image, respectively.

[0038] H and W are the height and width of the original image, respectively;

[0039] Next, search for the maximum value of the vertical grayscale gradient image, Gray. max ;

[0040] Gray max =max(Gray)

[0041] The segmentation threshold Gray is derived from the mean and maximum values ​​of the weighted vertical grayscale gradient images. Threshold :

[0042] Gray Threshold =Gray average ·t+Gray max ·(1-t);

[0043] Where t and 1-t are the weights of the mean gray value and the gray value of the edge point, respectively.

[0044] In step 4, the vertical grayscale gradient map is thresholded to obtain a binarized image.

[0045] Determine if the value of the vertical grayscale gradient image is greater than the segmentation threshold. Points with a value greater than the segmentation threshold are assigned a value of 255, and points with a value less than or equal to the segmentation threshold are assigned a value of 0; thus, a binary segmented image Gray* is obtained.

[0046]

[0047] In step 5, the boundaries of the segmented binarized image are calculated and the coordinates of the boundary points are stored.

[0048] Specifically, the first column of the binarized image is detected from top to bottom. If there is a change in grayscale, it is considered a boundary point, and the position of the point is stored. The number of boundary points is also counted.

[0049] Using the same method, we iterate through each column of the binarized image to obtain the EdgeMapList1 set of boundary point positions and the EdgeLength1 set of boundary points for the entire binarized image.

[0050] EdgeMapList1 = {px 11 py 11 , px 12 py 12 , px 13 py 13 ....px 1n py 1n ...}

[0051] Among them, px 1n This represents the x-coordinate of the nth boundary point during coarse calculation, py 1n This represents the y-coordinate of the nth boundary point during coarse calculation.

[0052] In step 6, the coordinates of the boundary points are fed into the Hough transform function for calculation.

[0053] Specifically: Preset the Rho value and Theta sequence value required for the Hough transform; based on the characteristic that the sea-line is basically horizontal, take several values ​​that are evenly distributed between 70° and 110° for the Theta sequence value; put the boundary point position sequence EdgeMapList1 and the number of boundary points EdgeLength1 into the Hough transform function to calculate the Hough spatial value.

[0054] In step 7, the location of the maximum value of the hough space output by the hough transform function is searched, and the approximate intercept and slope of the sea line are calculated using the coordinates of this location.

[0055] Specifically: Since there is a one-to-one correspondence between lines in the Cartesian coordinate system and points in Hough space, i.e., ρ = x·cos(θ) + y·sin(θ), where x and y are the coordinates of a point in the Cartesian coordinate system, and ρ and θ are the coordinates of a point in Hough space; Hough space is a Rho column, N Theta A two-dimensional matrix with rows; the coordinates of the maximum point of this matrix (ρ max1 θ max1 ) represents the longest straight line in the binary boundary graph, which is the sea-line line;

[0056] Therefore, the approximate slope k1 and intercept b1 of the sea line can be calculated through coordinate transformation;

[0057] k1=cos(θ max1 ) / sin(θ max1 ) / sample_width·sample_height;

[0058] b1=(ρ max1 -Rho / 2)·2 / sin(θ max1 )·sample_height;

[0059] In step 8, taking the approximate intercept and slope of the sea-line calculated in step 7 as a reference, a small range near the sea-line is selected in the longitudinal gray-scale gradient map calculated in step 2, and the maximum value of each column of the longitudinal gray-scale gradient map is searched and the coordinates of the maximum value are stored.

[0060] Specifically: Since step 7 has already calculated the approximate location of the sea-line, and obtained the approximate slope and intercept of the sea-line; using this value as a reference, select the area near the sea-line in the longitudinal grayscale gradient map calculated in step 2 and re-detect the sea-line; the width of the selected area is the same as the original longitudinal grayscale gradient map, the upper boundary of the height is top, and the lower boundary is bottom; the upper boundary top and the lower boundary bottom are selected according to the following formula;

[0061] If k1>=0

[0062] top = b1 - k1·W - 2·S;

[0063] bottom = b1 + 2·S;

[0064] If k1 < 0, then

[0065] top = b1 - 2·S;

[0066] bottom = b1 + k1·W-2·S;

[0067] Where W is the width of the original image;

[0068] S is the number of rows for vertical gradient accumulation;

[0069] k1 and b1 are the approximate slope and intercept of the sea line calculated earlier, respectively;

[0070] Search for the maximum value in each column of the sub-vertical grayscale gradient map from top to bottom and from left to right to obtain the position sequence of the maximum value EdgeMapList2 and the length of the maximum value point EdgeLength2;

[0071] EdgeMapList2 = {px 21 py 21 , px 22 py 22 , px 23 py 23 ...px 2n py 2n ....}

[0072] Among them, px 2n py represents the x-coordinate of the nth maximum point during precise calculation. 2n This represents the y-coordinate of the nth maximum value point during precise calculation.

[0073] In step 9, the coordinates of the maximum value point in each column are fed into the Hough transform function for calculation.

[0074] Step 6 is used to preset the Rho and Theta sequence values ​​required for the Hough transform. The boundary point position sequence EdgeMapList2 and the number of boundary points EdgeLength2 are then fed into the Hough transform function to calculate the Hough space value.

[0075] In step 10, the location of the maximum value of the Hough space output by the Hough transform function is searched, and the precise intercept and slope of the sea-line are calculated using the coordinates of this location, thereby obtaining the position of the sea-line.

[0076] The calculation methods for steps 10 and 7 are the same; the coordinates of the maximum point in the Hough space are denoted as (ρ max2 θ max2 The accurate location information of the sea-line was obtained through calculation. The slope k2 and intercept b2 were calculated as follows:

[0077] k2=cos(θ max2 ) / sin(θ max2 ) / sample_width·sample_height

[0078] b2=(ρ max2 -Rho / 2)·2 / sin(θ max2 )·sample_height.

[0079] Furthermore, the present invention also provides a real-time sea-line detection system, which is used to execute the above-described real-time sea-line detection method. The system includes: a longitudinal grayscale gradient map calculation module, a segmentation threshold calculation module, an edge detection module, and a sea-line fitting module; wherein:

[0080] The vertical grayscale gradient map calculation module is used to downsample the original image and calculate the vertical grayscale gradient map of the downsampled image.

[0081] The segmentation threshold calculation module is used to calculate the mean and maximum values ​​of the vertical grayscale gradient map and then weight them to obtain the segmentation threshold.

[0082] The edge detection module is used to determine the vertical grayscale gradient based on a segmentation threshold. Figure 2 Binarize and calculate the edge information of the binarized image;

[0083] The sea-line fitting module is used to detect straight lines at the edges, and finds the longest straight line in the image based on the Hough transform method and fits it as the sea-line.

[0084] (III) Beneficial Effects

[0085] To address the technical problems of existing sea-line detection methods, this invention proposes a method and system with strong anti-interference capabilities, enabling effective, accurate, and real-time sea-line detection in complex marine environments. This invention divides sea-line detection into two parts: coarse calculation and fine calculation. The first part, coarse calculation, involves downsampling the acquired visible light or infrared image and calculating the longitudinal gray-level gradient map. The longitudinal gray-level gradient map is then thresholded to obtain a binarized image. The boundaries of the binarized image are calculated, and the approximate intercept and slope of the sea-line are calculated using a Hough transform. The second part, fine calculation, involves using the sea-line intercept and slope calculated in the first part to search for the location of the maximum value of the longitudinal gray-level gradient map in the vicinity of the sea-line in the region calculated in the first part. The obtained location point is then subjected to a Hough transform, and the precise location and slope of the sea-line are calculated.

[0086] The technical solution of this invention exhibits excellent robustness against weather changes, uneven lighting, cloud interference, sea waves, islands, large ships, seabirds, and coastal scenery interference. It can detect horizons of varying tilt angles. This solution requires minimal computation and memory, has a simple algorithm principle, and is highly efficient to implement.

[0087] Compared with the prior art, the present invention has the following beneficial effects:

[0088] (1) This invention performs two calculations on the sea-line area: coarse and fine. The coarse calculation sets a low segmentation threshold for the longitudinal grayscale gradient map, so that the sea-line area can be segmented under different weather conditions, ensuring the accuracy of sea-line detection. The fine calculation performs a precise search calculation based on the approximate location of the sea-line obtained from the coarse calculation, ensuring the accuracy of sea-line detection.

[0089] (2) This invention calculates the slope and intercept of the sea-line using the method of searching for the brightest point in the Hough space. Since the sea-line is a long straight line traversing the entire image, while the boundaries of clouds, sea clutter, waves, seabirds, obstructions, and maritime targets are often not straight lines, or even if they are, their length is shorter than the sea-line. Furthermore, considering the Hough transform's detection characteristics for straight lines, this invention can effectively eliminate interference from clouds, sea clutter, illumination, waves, seabirds, obstructions, and maritime targets. This invention can also detect tilted sea-line patterns, such as… Figures 1-3 As shown.

[0090] (3) In the fine calculation, the present invention uses the longitudinal gray-scale gradient map obtained by coarse calculation without recalculation, thus saving calculation time.

[0091] (4) The precise calculation of the present invention only calculates the maximum value of the longitudinal gradient when extracting boundary points, which makes the calculation simple and reduces the number of points to be transformed by Hough in the later stage, thus greatly saving the calculation time.

[0092] (5) The present invention detects the boundary points of the grayscale image through Hough transform, thus it can also accurately detect tilted sea-line.

[0093] (6) In this invention, downsampling is performed when calculating the vertical grayscale gradient map. Since the sea line runs horizontally through the entire image, even if the image is very large, the downsampling coefficient can be adjusted appropriately to improve the calculation speed, save time, and reduce memory usage without affecting the accuracy of the sea line calculation.

[0094] (7) This invention is not limited to infrared images or visible light images, but applies to both.

[0095] (8) This invention only calculates the vertical grayscale gradient of the original image, so it is only sensitive to changes in the vertical gradient of the original image, but not sensitive to changes in the horizontal gradient. However, the vertical grayscale changes are large near the horizon, while the horizontal grayscale remains basically unchanged. However, the grayscale of the interfering objects in the sky and on the sea surface are not much different in the horizontal and vertical directions. Therefore, the technical solution of this invention has a certain degree of anti-interference capability. Attached Figure Description

[0096] Figure 1 This is a diagram showing the detection results of the method of the present invention under conditions of heavy cloud cover and low contrast between the sea and sky.

[0097] Figure 2 The results are shown in the form of the method of the present invention under conditions of large waves at sea.

[0098] Figure 3 This is the detection result of the method of the present invention under conditions of significant interference near the sea-line.

[0099] Figure 4 This is a flowchart of the method of the present invention.

[0100] Figure 5 The image to be detected is an embodiment of the present invention.

[0101] Figure 6 This is a downsampling vertical grayscale gradient image of an embodiment of the present invention.

[0102] Figure 7 This is a boundary diagram of an embodiment of the present invention.

[0103] Figure 8 This refers to the point where the vertical grayscale gradient reaches its maximum value in an embodiment of the present invention.

[0104] Figure 9 This is a diagram showing the detection results of the sea surface line according to an embodiment of the present invention. Detailed Implementation

[0105] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0106] To solve the above technical problems, the present invention provides a method for real-time detection of the sea surface line, the method comprising:

[0107] Step 1: Upon receiving the sea-line detection command from the photoelectric detection system, acquire a frame of raw image with a resolution of W·H using the corresponding sensor (television or infrared thermal imager); the raw image is a visible light or infrared image.

[0108] Step 2: Calculate the vertical grayscale gradient map of the downsampled original image;

[0109] Step 3: Calculate the segmentation threshold of the vertical grayscale gradient map;

[0110] Step 4: Perform threshold segmentation on the vertical grayscale gradient map according to the segmentation threshold to obtain a binarized image;

[0111] Step 5: Calculate the boundaries of the segmented binarized image and store the coordinates of the boundary points;

[0112] Step 6: Input the coordinates of the boundary points into the Hough transform function for calculation;

[0113] Step 7: Search for the location of the maximum value of the hough space output by the hough transform function, and calculate the approximate intercept and slope of the sea-line using the coordinates of that location;

[0114] Step 8: Using the approximate intercept and slope of the sea-line calculated in Step 7 as a reference, select a small area near the sea-line in the longitudinal gray-scale gradient map calculated in Step 2, search for the maximum value of each column of the longitudinal gray-scale gradient map, and store the coordinates of the maximum value.

[0115] Step 9: Input the coordinates of the maximum value in each column into the Hough transform function for calculation;

[0116] Step 10: Search for the location of the maximum value in the Hough space output by the Hough transform function, calculate the precise intercept and slope of the sea-line using the coordinates of this location, and thus obtain the position of the sea-line.

[0117] Step 11: Plot the sea line on the above image with a straight line, and report the precise intercept and slope values ​​of the sea line to the optoelectronic system.

[0118] In step 2, the formula for calculating the vertical grayscale gradient image is:

[0119]

[0120] in:

[0121]

[0122]

[0123] Gray represents the vertical grayscale gradient image after downsampling;

[0124] F represents the original image;

[0125] i and j are the row and column indices of the vertical grayscale gradient image;

[0126] sample_height is the row downsampling rate;

[0127] sample_width is the column downsampling rate;

[0128] S is the number of rows for vertical gradient accumulation;

[0129] In step 3, the process of calculating the segmentation threshold of the vertical grayscale gradient image is as follows:

[0130] First, calculate the mean of the vertical grayscale gradient image:

[0131]

[0132] Wherein: H Gray =H / sample_height, W Gray = W / sample_width;

[0133] H Gray W Gray These represent the height and width of the vertical grayscale gradient image, respectively.

[0134] H and W are the height and width of the original image, respectively;

[0135] Next, search for the maximum value of the vertical grayscale gradient image, Gray. max

[0136] Gray max =max(Gray)

[0137] The segmentation threshold Gray is derived from the mean and maximum values ​​of the weighted vertical grayscale gradient images. Threshold :

[0138] Gray Threshold =Gray average ·t+Gray max ·(1-t)

[0139] Where t and 1-t are the weights of the mean gray value and the gray value of the edge point, respectively.

[0140] In step 4, the vertical grayscale gradient map is thresholded to obtain a binarized image.

[0141] Determine if the value of the vertical grayscale gradient image is greater than the segmentation threshold. Points with a value greater than the segmentation threshold are assigned a value of 255, and points with a value less than or equal to the segmentation threshold are assigned a value of 0; thus, a binary segmented image Gray* is obtained.

[0142]

[0143] In step 5, the boundaries of the segmented binarized image are calculated and the coordinates of the boundary points are stored.

[0144] Specifically, the first column of the binarized image is detected from top to bottom. If there is a change in grayscale, it is considered a boundary point, and the position of the point is stored. The number of boundary points is also counted.

[0145] Using the same method, we iterate through each column of the binarized image to obtain the EdgeMapList1 set of boundary point positions and the EdgeLength1 set of boundary points for the entire binarized image.

[0146] EdgeMapList1 = {px 11 py 11 , px 12 py 12 , px 13 py 13 ....px 1n py 1n ...}

[0147] Among them, px 1n This represents the x-coordinate of the nth boundary point during coarse calculation, py 1n This represents the y-coordinate of the nth boundary point during coarse calculation.

[0148] In step 6, the coordinates of the boundary points are fed into the Hough transform function for calculation.

[0149] Specifically: Preset the Rho value and Theta sequence value required for the Hough transform; based on the characteristic that the sea-line is basically horizontal, take several values ​​that are evenly distributed between 70° and 110° for the Theta sequence value; put the boundary point position sequence EdgeMapList1 and the number of boundary points EdgeLength1 into the Hough transform function to calculate the Hough spatial value.

[0150] In step 7, the location of the maximum value of the hough space output by the hough transform function is searched, and the approximate intercept and slope of the sea line are calculated using the coordinates of this location.

[0151] Specifically: Since there is a one-to-one correspondence between lines in the Cartesian coordinate system and points in Hough space, i.e., ρ = x·cos(θ) + y·sin(θ); Hough space is a Rho column, N Theta A two-dimensional matrix with rows; the coordinates of the maximum point of this matrix (ρ max1 θ max1 ) represents the longest straight line in the binary boundary graph, which is the sea-line line;

[0152] Therefore, the approximate slope k1 and intercept b1 of the sea line can be calculated through coordinate transformation;

[0153] k1=cos(θ max1 ) / sin(θ max1 ) / sample_width·sample_height;

[0154] b1=(ρ max1 -Rho / 2)·2 / sin(θ max1 )·sample_height;

[0155] In step 8, taking the approximate intercept and slope of the sea-line calculated in step 7 as a reference, a small range near the sea-line is selected in the longitudinal gray-scale gradient map calculated in step 2, and the maximum value of each column of the longitudinal gray-scale gradient map is searched and the coordinates of the maximum value are stored.

[0156] Specifically: Since step 7 has already calculated the approximate location of the sea-line, and obtained the approximate slope and intercept of the sea-line; using this value as a reference, select the area near the sea-line in the longitudinal grayscale gradient map calculated in step 2 and re-detect the sea-line; the width of the selected area is the same as the original longitudinal grayscale gradient map, the upper boundary of the height is top, and the lower boundary is bottom; the upper boundary top and the lower boundary bottom are selected according to the following formula;

[0157] If k1>=0

[0158] top = b1 - k1·W - 2·S;

[0159] bottom = b1 + 2·S;

[0160] If k1 < 0, then

[0161] top = b1 - 2·S;

[0162] bottom = b1 + k1·W-2·S;

[0163] Where W is the width of the original image;

[0164] S is the number of rows for vertical gradient accumulation;

[0165] k1 and b1 are the approximate slope and intercept of the sea line calculated earlier, respectively;

[0166] Search for the maximum value in each column of the sub-vertical grayscale gradient map from top to bottom and from left to right to obtain the position sequence of the maximum value EdgeMapList2 and the length of the maximum value point EdgeLength2;

[0167] EdgeMapList2 = {px 21 py 21 , px 22 py 22 , px 23 py 23 ...px 2n py 2n ....}

[0168] Among them, px 2n py represents the x-coordinate of the nth maximum point during precise calculation. 2n This represents the y-coordinate of the nth maximum value point during precise calculation.

[0169] In step 9, the coordinates of the maximum value point in each column are fed into the Hough transform function for calculation.

[0170] Step 6 is used to preset the Rho and Theta sequence values ​​required for the Hough transform. The boundary point position sequence EdgeMapList2 and the number of boundary points EdgeLength2 are then fed into the Hough transform function to calculate the Hough space value.

[0171] In step 10, the location of the maximum value of the Hough space output by the Hough transform function is searched, and the precise intercept and slope of the sea-line are calculated using the coordinates of this location, thereby obtaining the position of the sea-line.

[0172] The calculation methods for steps 10 and 7 are the same; the coordinates of the maximum point in the Hough space are denoted as (ρ max2 θ max2 The accurate location information of the sea-line was obtained through calculation. The slope k2 and intercept b2 were calculated as follows:

[0173] k1=cos(θ max2 ) / sin(θ max2 ) / sample_width·sample_height

[0174] b1=(ρ max2 -Rho / 2)·2 / sin(θ max2)·sample_height.

[0175] Furthermore, the present invention also provides a real-time sea-line detection system, which is used to execute the above-described real-time sea-line detection method. The system includes: a longitudinal grayscale gradient map calculation module, a segmentation threshold calculation module, an edge detection module, and a sea-line fitting module; wherein:

[0176] The vertical grayscale gradient map calculation module is used to downsample the original image and calculate the vertical grayscale gradient map of the downsampled image; it can reduce the interference of obstacles such as sea waves, clouds, and seabirds.

[0177] The segmentation threshold calculation module is used to calculate the mean and maximum values ​​of the vertical grayscale gradient map and then weight them to obtain the segmentation threshold.

[0178] The edge detection module is used to determine the vertical grayscale gradient based on a segmentation threshold. Figure 2 Binarize and calculate the edge information of the binarized image;

[0179] The sea-line fitting module is used to detect straight lines at the edges, and finds the longest straight line in the image based on the Hough transform method and fits it as the sea-line.

[0180] Example 1

[0181] The processing flow of this embodiment is as follows: Figure 4 As shown, this embodiment provides a method for real-time detection of the sea surface area, specifically including:

[0182] Step 1: Upon receiving the sea-line detection command from the photoelectric detection system, acquire a raw image F with a resolution of W·H using the corresponding sensor (TV or infrared thermal imager).

[0183] Step 2: Calculate the original image (e.g., Figure 5 The vertical grayscale gradient image Gray after downsampling is calculated as follows: Figure 6 ;

[0184] The formula for calculating the vertical grayscale gradient image is:

[0185]

[0186] in:

[0187]

[0188]

[0189] Gray represents the vertical grayscale gradient image after downsampling;

[0190] i and j are the row and column indices of the vertical grayscale gradient image;

[0191] sample_height is the row downsampling rate, which is set to 1 in this example;

[0192] sample_width is the column downsampling rate, which is set to 8 in this example;

[0193] S is the number of rows for vertical gradient accumulation, which is set to 8 in this example;

[0194] Step 3: Calculate the segmentation threshold of the vertical grayscale gradient map;

[0195] First, calculate the mean of the vertical grayscale gradient image:

[0196]

[0197] Where: H = 576, W = 720 are the height and width of the original image, respectively;

[0198] H Gray W Gray These represent the height and width of the vertical grayscale gradient image, respectively.

[0199] H Gray =H / sample_height=576,W Gray =W / sample_width=90;

[0200] Next, search for the maximum value of the vertical grayscale gradient image, Gray. max

[0201] Gray max =max(Gray)

[0202] The segmentation threshold Gray is derived from the mean and maximum values ​​of the weighted vertical grayscale gradient images. Threshold :

[0203] Gray Threshold =Gray average 0.5+Gray max 0.5;

[0204] Step 4: Threshold segmentation is performed on the vertical grayscale gradient map to obtain a binarized image;

[0205] Determine whether the value of the vertical grayscale gradient image is greater than the segmentation threshold. Points with a value greater than the segmentation threshold are assigned a value of 255, and points with a value less than or equal to the segmentation threshold are assigned a value of 0, thus obtaining a binary segmented image Gray*.

[0206]

[0207] Step 5: Calculate the boundaries of the segmented image and store the coordinates of the boundary points;

[0208] The first column of the binarized image is detected from top to bottom. If there is a change in grayscale, it is considered a boundary point, and its position is stored. The number of boundary points is also counted. The same method is used to calculate and iterate through each column of the binarized image to obtain the boundary point position sequence set EdgeMapList1 and the number of boundary points EdgeLength1 for the entire binarized image.

[0209] EdgeMapList1 = {px 11 py 11 , px 12 py 12 , px 13 py 13 ....px 1n py 1n ...}

[0210] Among them, px 1n This represents the x-coordinate of the nth boundary point during coarse calculation, py 1n This represents the y-coordinate of the nth boundary point during coarse calculation. The boundary point image is as follows. Figure 7 As shown.

[0211] Step 6: Input the boundary point coordinates into the Hough transform function for calculation;

[0212] Pre-set the Rho and Theta sequence values ​​required for the Hough transform; in this example, the Rho value is set to 1000. The Theta sequence values ​​are several values ​​evenly distributed between 60° and 120° (in this example, the length N of the Theta sequence is used). Theta =120). The boundary point position sequence EdgeMapList1 and the number of boundary points EdgeLength1 are fed into the Hough transform function to calculate the Hough space value.

[0213] Step 7: Search for the brightest point in the hough space and calculate the approximate location and slope of the sea horizon;

[0214] Specifically: Since there is a one-to-one correspondence between lines in the Cartesian coordinate system and points in Hough space, i.e., ρ = x·cos(θ) + y·sin(θ), where x and y are the coordinates of a point in the Cartesian coordinate system, and ρ and θ are the coordinates of a point in Hough space; Hough space is a two-dimensional matrix with 1000 columns and 120 rows; the coordinates of the maximum value point of this matrix (ρ... max1 θ max1 ) represents the longest straight line in the binary boundary graph, which is the sea-line line;

[0215] Therefore, the approximate slope k1 and intercept b1 of the sea line can be calculated through coordinate transformation;

[0216] k1=cos(θ max1 ) / sin(θ max1 ) / 8·1;

[0217] b1=(ρ max1 -1000 / 2)·2 / sin(θ max1 )·1;

[0218] Step 8: Using the sea-line position calculated in Step 7 as a reference, select a small area near the sea-line in the vertical grayscale gradient map calculated in Step 2, and vertically search for the maximum value in each column of the grayscale gradient map, storing the coordinates of the maximum value point. The result of the maximum value point search is as follows: Figure 8 ;

[0219] Since step 7 has already calculated the approximate location of the sea-line, and obtained the approximate slope and intercept of the sea-line, using this value as a reference, the area near the sea-line in the longitudinal grayscale gradient map calculated in step 2 is selected to re-detect the sea-line; the width of the selected area is the same as the original grayscale gradient map, the upper boundary of the height is top, and the lower boundary is bottom; the upper boundary top and the lower boundary bottom are selected according to the following formula;

[0220] If k1>=0

[0221] top = b1 - k1·720 - 2·8;

[0222] bottom = b1 + 2·8;

[0223] If k1 < 0, then

[0224] top = b1 - 2.8;

[0225] bottom = b1 + k1·720 - 2·8;

[0226] k1 and b1 are the approximate slope and intercept of the sea line calculated earlier, respectively;

[0227] Search for the maximum value in each column of the sub-vertical grayscale gradient map from top to bottom and from left to right to obtain the position sequence of the maximum value EdgeMapList2 and the length of the maximum value point EdgeLength2;

[0228] EdgeMapList2={px21,py21,px22,py 22 , px 23 py 23 ...px 2n py 2n ....}

[0229] Among them, px2n py represents the x-coordinate of the nth maximum point during precise calculation. 2n This represents the y-coordinate of the nth maximum value point during precise calculation;

[0230] Step 9: Input the coordinates of the maximum value point into the Hough transform function for calculation;

[0231] Step 6 is used to preset the Rho and Theta sequence values ​​required for the Hough transform. The boundary point position sequence EdgeMapList2 and the number of boundary points EdgeLength2 are then fed into the Hough transform function to calculate the Hough space value.

[0232] Step 10: Accurately locate the sea-line position using the Hough spatial values ​​output by the Hough transform;

[0233] This step is calculated using the same method as step 7; the coordinates of the maximum point in the Hough space are denoted as (ρ max2 θ max2 The accurate location information of the sea-line can be obtained through calculation. The calculation methods for slope k2 and intercept b2 are as follows:

[0234] k1=cos(θ max2 ) / sin(θ max2 ) / 8·1

[0235] b1=(ρ max2 -1000 / 2)·2 / sin(θ max2 )·1;

[0236] Step 11: Output the slope k2 and intercept b2 of the sea-line, and output an image superimposed with the sea-line, such as... Figure 9 .

[0237] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for real-time detection of the sea surface line, characterized in that, The method includes: Step 1: Upon receiving the sea-line detection command from the photoelectric detection system, acquire a frame of raw image through the corresponding sensor; Step 2: Calculate the vertical grayscale gradient map of the downsampled original image; Step 3: Calculate the segmentation threshold of the vertical grayscale gradient map; Step 4: Perform threshold segmentation on the vertical grayscale gradient map according to the segmentation threshold to obtain a binarized image; Step 5: Calculate the boundaries of the segmented binarized image and store the coordinates of the boundary points; Step 6: Input the coordinates of the boundary points into the Hough transform function for calculation; Step 7: Search for the location of the maximum value of the hough space output by the hough transform function, and calculate the approximate intercept and slope of the sea-line using the coordinates of that location; Step 8: Using the approximate intercept and slope of the sea-line calculated in Step 7 as a reference, select a small area near the sea-line in the longitudinal gray-scale gradient map calculated in Step 2, search for the maximum value of each column of the longitudinal gray-scale gradient map, and store the coordinates of the maximum value. Step 9: Input the coordinates of the maximum value in each column into the Hough transform function for calculation; Step 10: Search for the location of the maximum value in the Hough space output by the Hough transform function, calculate the precise intercept and slope of the sea-line using the coordinates of this location, and thus obtain the position of the sea-line. Step 11: Plot the sea line on the above image using straight lines, and report the precise intercept and slope values ​​of the sea line to the optoelectronic system; In step 2, the formula for calculating the vertical grayscale gradient image is: in: Gray This is the vertical grayscale gradient image after downsampling; F Original image; i , j The row and column indices for the vertical grayscale gradient image; S The number of rows accumulated for the vertical gradient; sample_height To reduce the sampling rate; sample_width To reduce the sampling rate; In step 3, the process of calculating the segmentation threshold of the vertical grayscale gradient image is as follows: First, calculate the mean of the vertical grayscale gradient image: in: H Gray =H / sample_height , W Gray =W / sample_width ; H Gray , W Gray These represent the height and width of the vertical grayscale gradient image, respectively. H , W These are the height and width of the original image, respectively. Gray average This represents the mean value of the vertical grayscale gradient image; Next, search for the maximum value of the vertical grayscale gradient. Gray max The segmentation threshold is derived from the mean and maximum values ​​of the weighted vertical grayscale gradient image. Gray Threshold : Gray Threshold = Gray average · t+ Gray max · （1 -t ）； in, t and 1 -t These are the weights of the grayscale mean and the grayscale value of the edge points, respectively, 0 < t <1; In step 4, the vertical grayscale gradient map is thresholded to obtain a binarized image. Determine if the value of the vertical grayscale gradient image is greater than the segmentation threshold. Points with a value greater than the segmentation threshold are assigned a value of 255, and points with a value less than or equal to the segmentation threshold are assigned a value of 0; this yields a binarized segmented image. ; In step 5, the boundaries of the segmented binarized image are calculated and the coordinates of the boundary points are stored. Specifically, the first column of the binarized image is detected from top to bottom. If there is a change in grayscale, it is considered a boundary point, and the position of the point is stored. The number of boundary points is also counted. Using the same method, we iterate through each column of the binarized image to obtain the sequence of boundary point positions for the entire binarized image. EdgeMapList1 Number of sets and boundary points EdgeLength1 ; EdgeMapList1={px 11 py 11 , px 12 py 12 , px 13 py 13 , ...,px 1n py 1n ，...} in, px 1n Indicates the first coarse calculation n boundary points x coordinate, py 1n Indicates the first coarse calculation n boundary points y coordinate; In step 6, the coordinates of the boundary points are fed into the Hough transform function for calculation. Specifically, this involves: pre-setting the Hough transform parameters. Rho Value and Theta Sequence value; based on the fact that the sea surface is basically horizontal, Theta The sequence values ​​are taken from several values ​​evenly distributed between 70° and 110°; the boundary point position sequence is... EdgeMapList1 and the number of boundary points EdgeLength1 Input the Hough transform function and calculate the Hough space value; In step 7, the location of the maximum value of the hough space output by the hough transform function is searched, and the approximate intercept and slope of the sea line are calculated using the coordinates of this location. Specifically: because there is a one-to-one correspondence between lines in a Cartesian coordinate system and points in Hough space, that is... ρ = x·cos ( θ ) + y·sin ( θ ), x , y These are the coordinates of a point in a rectangular coordinate system. ρ , θ These are the coordinates of a point in Hough space; Hough space is a two-dimensional matrix; the coordinates of the point with the maximum value of this matrix (ρ) max1 θ max1 The line ) represents the longest straight line in the binary boundary graph, which is the sea-line line; Therefore, the approximate slope of the sea horizon can be calculated through coordinate transformation. k 1 and intercept b 1; k 1 = cos(θ max1 ) / sin(θ) max1 ) / sample_width·sample_height ； b 1 =(ρ max1 - Rho / 2)·2 / sin(θ max1 )·sample_height 。 2. The real-time detection method for the sea-line area as described in claim 1, characterized in that, In step 8, with the approximate intercept and slope of the sea-line calculated in step 7 as a reference, a small range near the sea-line is selected in the longitudinal gray-scale gradient map calculated in step 2, and the maximum value of each column of the longitudinal gray-scale gradient map is searched and the coordinates of the maximum value are stored. Specifically: Since step 7 has already calculated the approximate location of the sea-line, and obtained the approximate slope and intercept of the sea-line; using this value as a reference, select the area near the sea-line in the longitudinal grayscale gradient map calculated in step 2 to re-detect the sea-line; The width of the selected region is the same as the original vertical grayscale gradient image, and the upper boundary of the height is... top The lower boundary is bottom Upper boundary top and lower boundary bottom Select according to the following formula; if k 1 >=0 top = b 1 - k 1 · W-2·S; bottom = b 1 + 2·S； if k 1 <0 ,but top = b 1 - 2·S ； bottom = b 1 + k 1 · W - 2·S ； in, W The width of the original image; S The number of rows accumulated for the vertical gradient; k 1, b 1 represents the approximate slope and intercept of the sea line calculated earlier; Search for the maximum value in each column of the sub-vertical grayscale gradient map sequentially from top to bottom and left to right to obtain the sequence of maximum value positions. EdgeMapList2 and the length of the maximum point EdgeLength2 ; EdgeMapList2={px 21 py 21 , px 22 py 22 , px 23 py 23 , ...,px 2n py 2n ，...} in, px 2n Indicates the first precise calculation n The maximum value point x coordinate, py 2n Indicates the first precise calculation n The maximum value point y coordinate.

3. The real-time detection method for the sea-line area as described in claim 2, characterized in that, In step 9, the coordinates of the maximum value point in each column are fed into the Hough transform function for calculation; Use step 6 to preset the Hough transform requirements. Rho Value and Theta Sequence value, the sequence of boundary point positions EdgeMapList2 and the number of boundary points EdgeLength2 Input the Hough transform function to calculate the Hough space value.

4. The real-time detection method for the sea-line area as described in claim 3, characterized in that, In step 10, the location of the maximum value of the Hough space output by the Hough transform function is searched, and the precise intercept and slope of the sea-line are calculated using the coordinates of this location, thereby obtaining the position of the sea-line. The calculation methods for steps 10 and 7 are the same; the coordinates of the maximum point in the Hough space are ( ρ max2 , θ max2 The accurate location and slope of the sea-line were calculated. k 2 and intercept b 2. The calculation method is as follows: k 2 = cos(θ max2 ) / sin(θ) max2 ) / sample_width·sample_height b 2 =(ρ max2 - Rho / 2)·2 / sin(θ max2 )·sample_height 。 5. A real-time sea level detection system, wherein the real-time sea level detection system is used to execute the real-time sea level detection method as described in any one of claims 1-4, characterized in that, The system includes: a vertical grayscale gradient map calculation module, a segmentation threshold calculation module, an edge detection module, and a sea-line fitting module; wherein: The vertical grayscale gradient map calculation module is used to downsample the original image and calculate the vertical grayscale gradient map of the downsampled image. The segmentation threshold calculation module is used to calculate the mean and maximum values ​​of the vertical grayscale gradient map and then weight them to obtain the segmentation threshold. The edge detection module is used to binarize the vertical gray-level gradient map according to the segmentation threshold and calculate the edge information of the binarized image; The sea-line fitting module is used to detect straight lines at the edges, and finds the longest straight line in the image based on the Hough transform method and fits it as the sea-line.

Images