A highly robust sea-sky-line extraction method based on photoelectric sensor

Abstract

The present invention relates to a highly robust sea-sky-line extraction method based on a photoelectric sensor. By enhancing the edge of an image under a vertical gradient, filtering some irrelevant vertical line segments, and then sampling points on the line, candidate points of the boundary are obtained, while retaining the direction information of the line, and image blocks around the candidate points of the boundary are extracted, and the boundary is judged to be a sea-sky-line or the edge of other objects by the image feature information around the boundary points. The convolutional neural network is trained using image blocks containing sea-sky-line and image blocks with only background information, so that the network can learn the sea-sky-line features, effectively distinguish the sea-sky-line image blocks from the background information image blocks in the operation of filtering the boundary candidate points, and in the RANSAC straight line fitting, the Euclidean distance between the fitted straight line and the fitted point and the straight line direction and the direction information retained by sampling the fitting point from the straight line are considered, so that the final fitted sea-sky-line has higher accuracy and is more robust.

Description

Technical Field

[0001] The present invention relates to the technical field of photoelectric sensors and unmanned ship automatic driving, and more specifically, to a highly robust sea-sky-line extraction method based on photoelectric sensors. Background Art

[0002] Chinese patent CN201710878146.X discloses a sea-sky line detection method based on image semantic segmentation, which mainly includes the following steps: (1) inputting an image to be detected; (2) using the SLIC algorithm to perform superpixel segmentation on the input image; (3) using superpixels as the basic unit, establishing a probability graph model for semantic segmentation of sea surface images, and using the graph model to segment the image from top to bottom into a sky area, a mixed area of ​​land and haze, and a seawater area; (4) extracting the seawater area to obtain a mask image of the seawater area; (5) extracting the boundary points of the seawater area on the mask image of the seawater area according to the gradient information in the column direction; (6) using the RANSAC algorithm to perform straight line fitting on the boundary points of the seawater area to determine the straight line parameters of the seasky line. This method can accurately detect the seasky line in complex environments, and can also effectively detect the coastline in the background of long coasts. However, in some scenes where the boundary between the sea and sky areas is not obvious, it is easily affected by noise interference such as occlusion by boats and river banks. For some objects with long straight line features such as river banks and boats, they will be considered as part of the seasky line, affecting the extraction accuracy. Summary of the invention

[0003] In order to overcome the defects in the prior art, the present invention provides a highly robust sea-sky-line extraction method based on a photoelectric sensor, which effectively improves the accuracy of the fitted sea-sky-line and improves the robustness.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: a highly robust sea-sky-line extraction method based on a photoelectric sensor, comprising the following steps:

[0005] S1. Extract visual image information through photoelectric sensor, calculate the Sobel vertical gradient of the image through Sobel operator, and sharpen the edge of the scene in the image according to the gradient;

[0006] S2. Perform LSD line detection on the edge-enhanced image data to find pixels with large gradient changes in the image, merge similar pixels, and obtain a series of straight line segments;

[0007] S3. Filter short straight lines from the extracted straight lines, sample candidate points, and save the candidate points with the direction information of the straight line;

[0008] S4. Divide the image into blocks of a set size to obtain the distribution and number of candidate points in different blocks, and record the blocks containing more than the set number of candidate points as regions of interest (ROIs);

[0009] S5. The convolutional neural network performs a binary classification task to divide all regions of interest on the image into sea-skyline regions and background regions, thereby filtering candidate points existing in the background region;

[0010] S6. The control points distributed near the sea-sky line are obtained through step S5, and the fitted straight line is the sea-sky line by using the random sampling verification method of RANSAC.

[0011] The present invention extracts visual image information through a photoelectric sensor, enhances the sea-sky line and the riverbank through edge enhancement, extracts long line segments in the image through straight line detection, extracts boundary candidate points through sampling, divides the image into blocks, classifies and judges the image blocks where the candidate points are located using a binary classification neural network, filters the extracted candidate points to obtain control points, and finally uses RANSAC straight line fitting to fit the control points, and fits the sea-sky line based on the Euclidean distance, the direction information of the control points, and the distribution of the control points.

[0012] The present invention proposes a highly robust sea-sky-line extraction method based on a photoelectric sensor, which can be applied not only to a wide sea surface, but also to the extraction of coastlines of long coasts or riverbanks; it provides important information for the unmanned ship to evaluate its own posture and the target detection range, and provides the necessary conditions for the image registration in the subsequent target detection and target tracking. The present invention calculates the Sobel vertical gradient of the image through the Sobel operator, sharpens the edge of the scene in the image according to the gradient, solves the characteristic of unclear edges in blurred images, and prevents missed detection caused by blurring of the sea-sky-line due to changes in illumination or camera shaking. The edge-enhanced image data is subjected to the LSD straight line detection operation to obtain straight line segments. Among all the extracted straight lines, short straight lines are filtered to solve some interference caused by the edges of waves, clouds, islands, etc. After that, all straight lines are sampled to obtain candidate points, reducing the computational complexity, while saving the direction information of the straight line where the candidate point is located, and maximally preserving the original information, that is, the directionality of the straight line. The entire visual image is divided into blocks of a certain size (32*32 pixels) to obtain the distribution and number of candidate points in different blocks. The blocks containing more than a certain number of candidate points are recorded as regions of interest (ROIs), and some regions are extracted to reduce the computational complexity. A convolutional neural network is used for binary classification tasks to divide all regions of interest on the image into sea-sky line regions and background regions, thereby filtering candidate points in the background region. Compared with other distinction methods using traditional features, neural networks can achieve very high classification effects. After denoising the candidate points, control points distributed near the sea-sky line are obtained. The sea-sky line is fitted using RANSAC using a random sampling verification method to improve detection accuracy and robustness.

[0013] Furthermore, the algorithm used by the Sobel operator edge operator is to first perform weighted averaging and then perform differential operation. The calculation method is as follows:

[0014] Δ x f(x,y)=[f(x-1,y+1)+2f(x,y+1)+f(x+1,y+1)]

[0015] -[f(x-1,y-1)+2f(x,y-1)+f(x+1,y-1)]

[0016] Δ y f(x,y)=[f(x-1,y-1)+2f(x-1,y)+f(x-1,y+1)]

[0017] -[f(x+1,y-1)+2f(x+1,y)+f(x+1,y+1)]

[0018] Where: x, y are the pixel values ​​of the original image f in the x and y directions, and the obtained Δ xf(x,y) is the edge operator of the original image in the x direction, Δ y f(x,y) is the edge operator of the original image in the y direction.

[0019] Furthermore, the LSD line detection includes the following steps: first, the gradient size and direction of all pixel points in the image are calculated, and then the points with small gradient direction changes and adjacent to each other are regarded as a connected domain. Then, according to the rectangularity of each domain, it is judged whether it is necessary to disconnect it according to the rules to form multiple domains with larger rectangularity. Finally, all the generated connected domains are improved and screened, and the domains that meet the conditions are retained, which are the final line detection results.

[0020] Furthermore, the LSD line detection method first calculates the horizontal line angle of each pixel to form a horizontal line field, and these horizontal line fields are divided into several connected parts, where the directions are similar and within the tolerance τ, and a series of regions are obtained, which are connected domains.

[0021] Furthermore, the connected domain is a group of connected pixels, which are candidates for straight line segments. For the entire line segment connected domain, the minimum circumscribed matrix of the group of pixels is calculated. If the angle difference between the horizontal line of the pixel in the rectangle and the main direction of the minimum circumscribed rectangle is within the tolerance τ, then this point is called a homogeneous point. By counting the number of all pixels n in the minimum circumscribed rectangle and the number of homogeneous points k therein, it is used to determine whether the connected domain, that is, the group of connected pixels, is a straight line segment.

[0022] Furthermore, the verification method in step S6 includes the following steps:

[0023] Randomly assume that some of the in-place points are initial values; then use these in-place points to fit a model, which is suitable for the assumed in-place points, and all unknown parameters can be calculated from the assumed in-place points;

[0024] Use the model obtained in the previous step to test all other data. If a point is suitable for the estimated model, it is considered to be an internal point and the internal point is expanded.

[0025] If enough points are classified as hypothesized inliers, then the estimated model is reasonable enough;

[0026] Then, use all the assumed in-game points to re-estimate the model, because this model is only estimated at the initial assumed in-game points, and needs to be updated after subsequent expansion;

[0027] Finally, the model is evaluated by estimating the error rate between the in-game points and the model;

[0028] The above process is repeated a certain number of times, and the model stops when it reaches a certain accuracy or stops when the accuracy requirement cannot be met after the number of iterations.

[0029] Furthermore, the step of fitting a straight line in step S6 includes:

[0030] Randomly select 2 points from the control points to fit a straight line;

[0031] Use this line to test other control points: If the control point (x, y) satisfies the distance d from the line l and the slope k of the original line segment and the slope k′ of the line are within a certain threshold, then it is an internal point w:

[0032]

[0033] Considering that more points in the horizontal position of the image should be fitted in the sea-sky line, the x-coordinates of the in-place points are subjected to one-dimensional gridding, thereby calculating the number of grids occupied by the fitted in-place points in the horizontal position. The number of effective fitting points = the number of occupied grids / the number of fitted in-place points;

[0034] Update strategy for straight lines:

[0035] a) If the number of fitting points of the new round of straight line is greater than the number of fitting points of the current optimal straight line, and the number of effective fitting points is less than the number of effective fitting points of the current optimal straight line, then compare whether the number of fitting points exceeds 1.5 times the number of fitting points of the optimal straight line;

[0036] b) If the valid fitting number is greater than the valid fitting number of the current best straight line, then compare whether the number of fitting points exceeds half of the number of fitting points of the best straight line;

[0037] If one of the above conditions is met, the current fitting straight line is used to update the optimal straight line.

[0038] Furthermore, the convolutional neural network in step S5 is a lightweight convolutional neural network. The size of the input image is 32x32 pixels. After the image is input into the convolutional neural network model, it passes through two convolutional layers and a pooling layer, and then outputs the classification label through a fully connected layer.

[0039] The present invention also provides an electronic device, comprising:

[0040] Memory for storing computer programs;

[0041] A processor is used to implement the steps of the above method when executing the computer program.

[0042] The present invention also provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0043] Compared with the prior art, the beneficial effects are:

[0044] 1. The method proposed in the present invention expands the scope of application scenarios. Compared with other detection methods, most of them can only be applied to relatively ideal water surface environments. The experimental scenarios of the present invention include wide waters, river banks with complicated ports, and coastal areas. It can also be used in scenes with complex background information, greatly expanding the scope of application scenarios.

[0045] 2. The proposed method for extracting sea-sky line has high accuracy and robustness. Directly using straight line detection and fitting methods on the sea surface is easily interfered by other edge information such as the hull, wake waves generated by the boat, and clouds. The present invention uses a binary classification neural network to filter noise points, greatly improving the accuracy of sea-sky line extraction and being applicable to more complex marine scenarios.

[0046] 3. The present invention solves the problems of slow speed and low efficiency caused by traditional methods for processing visual images, such as multi-scale image filtering, regional texture analysis and fusion algorithms, etc. By using neural network for denoising and RANSAC straight line fitting, the processing time of a frame of visual image is only 0.1s, which greatly shortens the time required for sea-sky line detection and provides real-time guarantee for subsequent image registration and target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic flow chart of the method of the present invention.

[0048] Figure 2 It is the Sobel operator template of the present invention.

[0049] Figure 3 It is a schematic diagram of the horizontal line in the present invention.

[0050] Figure 4 It is a connectivity diagram of the present invention.

[0051] Figure 5 This is a schematic diagram of obtaining candidate points by sampling points in step S3 of the present invention.

[0052] Figure 6 It is a schematic diagram of the candidate point related area obtained by image block interception in step S4 of the present invention.

[0053] Figure 7 It is a schematic diagram of the binary classification neural network structure selected by the present invention.

[0054] Figure 8 It is a schematic diagram of classifying the area where the candidate point is located using a binary classification model in the present invention.

[0055] Fig. 9It is a schematic diagram of denoising candidate points in a related area using classification results in the present invention.

[0056] Fig.10 The present invention uses the RANSAC algorithm to fit control points to generate a sea-sky line schematic diagram. DETAILED DESCRIPTION

[0057] The drawings are only for illustrative purposes and should not be construed as limiting the present invention. To better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the drawings. The positional relationships described in the drawings are only for illustrative purposes and should not be construed as limiting the present invention.

[0058] The invention uses photoelectric sensors to extract sea-sky lines in three steps: first, straight line detection, then noise filtering of candidate points, and finally fitting of candidate points. Figure 1 shown.

[0059] Step 1: Extract visual image information through the photoelectric sensor, calculate the Sobel vertical gradient of the image through the Sobel operator, and sharpen the edge of the scene in the image according to the gradient. The Sobel operator is an important processing method in the field of computer vision. It is mainly used to obtain the first-order gradient of digital images. Its common application and physical meaning is edge detection. The Sobel operator is to detect the edge by taking the weighted difference of the grayscale values ​​of the four upper, lower, left and right neighborhoods of each pixel in the image and reaching the extreme value at the edge. Using this operator at any point in the image will generate a corresponding gradient vector or its normal vector. The Sobel operator not only produces a better detection effect, but also has a smoothing suppression effect on noise, but the obtained edge is coarse and pseudo edges may appear.

[0060] The algorithm used by the Sobel operator edge operator is to first perform weighted averaging and then perform differential operations. The calculation method of the operator is as follows:

[0061] Δ x f(x,y)=[f(x-1,y+1)+2f(x,y+1)+f(x+1,y+1)]

[0062] -[f(x-1,y-1)+2f(x,y-1)+f(x+1,y-1)]

[0063] Δ y f(x,y)=[f(x-1,y-1)+2f(x-1,y)+f(x-1,y+1)]

[0064] -[f(x+1,y-1)+2f(x+1,y)+f(x+1,y+1)]

[0065] Where: x, y are the pixel values ​​of the original image f in the x and y directions, and the obtained Δ x f(x,y) is the edge operator of the original image in the x direction, Δ y f(x,y) is the edge operator of the original image in the y direction.

[0066] The templates of the Sobel operator in the vertical and horizontal directions are as follows Figure 2 As shown, the former can detect the horizontal edges in the image, while the latter can detect the vertical edges in the image.

[0067] Step 2: Perform LSD line detection on the edge-enhanced image data to find pixels with large gradient changes in the image, merge similar pixels, and obtain a series of straight line segments.

[0068] LSD is a line segment detection algorithm that can obtain high-precision straight line segment detection results in a short time. The algorithm first calculates the gradient size and direction of all pixels in the image, and then regards the points with small gradient direction changes and adjacent to each other as a connected domain. Then, according to the rectangularity of each domain, it determines whether it needs to be disconnected according to the rules to form multiple domains with larger rectangularity. Finally, all the generated connected domains are improved and screened, and the domains that meet the conditions are retained, which is the final line detection result. The advantages of this algorithm are fast detection speed, no parameter adjustment, and the error control method is used to improve the accuracy of line detection.

[0069] The LSD line detection method first calculates the horizontal line angle of each pixel point, thus forming a horizontal line field (Level-Line Field), that is, a unit vector field. Here, the horizontal line angle of the pixel point is the vertical angle of the gradient direction of the point, such as Figure 3 shown.

[0070] The LSD line detection method first calculates the horizontal line angle of each pixel to form a horizontal line field. The field is divided into several connected parts, which have approximately the same direction and are within the tolerance τ, so that a series of regions can be obtained. These regions are called connected domains (line support regions), such as Figure 4 shown.

[0071] A connected domain is a group of connected pixels, which are candidates for a straight line segment. For the entire connected domain of the line segment, the minimum circumscribed matrix of the group of pixels is calculated. If the angle difference between the horizontal line angle of the pixel in the rectangle and the main direction of the minimum circumscribed rectangle is within the tolerance τ, then this point is called a homogeneous point. By counting the number of all pixels n in the minimum circumscribed rectangle and the number of homogeneous points k in it, it is used to determine whether this connected domain, that is, the group of connected pixels, is a straight line segment.

[0072] Step 3: Among the extracted straight lines, short straight lines are filtered out and candidate points are sampled. The candidate points saved at the same time have the direction information of the straight line.

[0073] Step 4: Divide the image into blocks of a certain size (32*32 pixels) to obtain the distribution and number of candidate points in different blocks. Record the blocks containing more than a certain number of candidate points as regions of interest (ROIs), and intercept the relevant areas of the candidate points in blocks.

[0074] Step 5: Use a convolutional neural network for a binary classification task to divide all regions of interest on the image into sea-skyline regions and background regions, thereby filtering candidate points that exist in the background region.

[0075] Since the image size of the neural network input is small, 32x32, the network structure is not too complicated. It is an ordinary lightweight CNN network. The neural network model is as follows Figure 7 As shown in Figure 2. After the image is input into the CNN model, it passes through two convolutional layers and a pooling layer, and then outputs the classification label through a fully connected layer.

[0076] Step 6: Through step 5, we can get the control points distributed near the sea-sky line. Using RANSAC, we can use the random sampling verification method to fit the straight line as the sea-sky line.

[0077] After step 5, some noise caused by other edge information is removed by the binary classification neural network, but there are still some difficult to remove noise such as the edge straight line of the hull near the sea-sky line and the straight line information generated by the river bank. Therefore, the RANSAC algorithm can be used to further eliminate the interference of noise and fit the final sea-sky line, such as Fig.10 shown.

[0078] The input of the RANSAC algorithm is a set of observations, a parameterized model that can explain or adapt to the observations, and some credible parameters. RANSAC achieves its goal by repeatedly selecting a set of random subsets of the data. The selected subsets are assumed to be inliers and verified using the following method:

[0079] (1) First, randomly assume some in-place points as initial values. Then use these in-place points to fit a model. This model is suitable for the assumed in-place points, and all unknown parameters can be calculated from the assumed in-place points.

[0080] (2) Use the model obtained in (1) to test all other data. If a point is suitable for the estimated model, it is considered to be an internal point and the internal point is expanded.

[0081] (3) If enough points are classified as hypothesized inliers, then the estimated model is reasonable enough.

[0082] (4) Then, use all the assumed in-library points to re-estimate the model, because this model is only estimated based on the initial assumed in-library points and needs to be updated after subsequent expansion.

[0083] (5) Finally, the model is evaluated by estimating the error rate between the inliers and the model.

[0084] (6) Repeat the above process a certain number of times, and stop when the model reaches a certain accuracy or stop when the accuracy requirement cannot be met after the number of iterations.

[0085] The steps for fitting a sea-sky line are as follows:

[0086] (1) Randomly select two points from the control points to fit a straight line;

[0087] (2) Use this line to test other control points: If the control point (x, y) satisfies the distance d from the line l and the slope k of the original line segment and the slope k′ of the line are within a certain threshold, then it is an internal point w;

[0088]

[0089] (3) Considering that more points in the horizontal position of the image should be fitted in the sea-sky line, the x-coordinates of the in-place points are subjected to one-dimensional gridding processing, thereby calculating the number of grids occupied by the fitted in-place points in the horizontal position. The number of effective fitting points = the number of occupied grids / the number of fitted in-place points;

[0090] (4) Straight line update strategy:

[0091] a) If the number of fitting points of the new round of straight line is greater than the number of fitting points of the current optimal straight line, and the number of effective fitting points is less than the number of effective fitting points of the current optimal straight line, then compare whether the number of fitting points exceeds 1.5 times the number of fitting points of the optimal straight line;

[0092] b) If the valid fitting number is greater than the valid fitting number of the current best straight line, then compare whether the number of fitting points exceeds half of the number of fitting points of the best straight line.

[0093] If one of the above conditions is met, the current fitting straight line is used to update the optimal straight line.

[0094] In view of the complex and changeable water surface conditions, the present invention proposes a highly robust method for extracting sea-sky lines from unmanned boats based on photoelectric sensors, which can be applied not only to the vast sea surface, but also to the extraction of coastlines on long coasts or riverbanks. By enhancing the edge of the image under the vertical gradient, some irrelevant vertical line segments are first filtered out. Since the unmanned boat will encounter complex sea conditions when sailing on the sea, such as being affected by wind, waves, etc., it will cause bumps, resulting in blurred images obtained by the photoelectric sensor camera and unclear boundaries. The problem of the sea-sky line not being detected by the straight line detector is solved by edge enhancement. Then, the straight line is sampled to obtain candidate points of the boundary, while retaining the direction information of the straight line. According to the candidate points of the boundary, the image blocks around the candidate points are extracted. The boundary is judged to be the sea-sky line or the edge of other objects by the image feature information around the boundary points, thereby filtering the boundary candidate points. A deep learning network was used in this binary classification task. The network was trained using image blocks containing sea-sky lines and image blocks with only background information, so that the network could learn the sea-sky line features, effectively distinguish the sea-sky line image blocks from the background information image blocks in the operation of filtering the boundary candidate points, and eliminate the interference of other edge noises such as wake waves, clouds and hulls generated by boats. In order to eliminate the interference of some discontinuous points and some noises that have not been filtered, in the RANSAC straight line fitting, the Euclidean distance between the fitting line and the fitting point, as well as the direction of the line and the direction information retained by the sampling of the fitting point from the line are comprehensively considered, so that the final fitted sea-sky line is more accurate and more robust.

[0095] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

[0096] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A highly robust sea-sky-line extraction method based on photoelectric sensors, characterized in that: The following steps are involved: S1. Extract visual image information through photoelectric sensor, calculate the Sobel vertical gradient of the image through Sobel operator, and sharpen the edge of the scene in the image according to the gradient; S2. Perform LSD line detection on the edge-enhanced image data to find pixels with large gradient changes in the image, merge similar pixels, and obtain a series of straight line segments; S3. Filter short straight lines from the extracted straight lines, sample candidate points, and save the candidate points with the direction information of the straight line; S4. Divide the image into blocks of a set size to obtain the distribution and number of candidate points in different blocks, and record the blocks containing more than the set number of candidate points as regions of interest (ROIs); S5. The convolutional neural network performs a binary classification task to divide all regions of interest on the image into sea-skyline regions and background regions, thereby filtering candidate points existing in the background region; S6. The control points distributed near the sea-sky line are obtained through step S5, and the fitted straight line is the sea-sky line by using the random sampling verification method of RANSAC; The LSD line detection method first calculates the horizontal line angle of each pixel to form a horizontal line field, and these horizontal line fields are divided into several connected parts, where the directions are similar and within the tolerance τ, and a series of regions are obtained, which are connected domains; The connected domain is a group of connected pixels, which are candidates for straight line segments. For the entire line segment connected domain, the minimum circumscribed matrix of the group of pixels is calculated. If the angle difference between the horizontal line angle of the pixel in the rectangle and the main direction of the minimum circumscribed rectangle is within the tolerance τ, then this point is called a homogeneous point. By counting the number of all pixels n in the minimum circumscribed rectangle and the number of homogeneous points k in it, it is used to determine whether the connected domain, that is, the group of connected pixels, is a straight line segment. The verification method in step S6 comprises the following steps: Randomly assume that some of the in-place points are initial values; then use these in-place points to fit a model, which is suitable for the assumed in-place points, and all unknown parameters can be calculated from the assumed in-place points; Use the model obtained in the previous step to test all other data. If a point is suitable for the estimated model, it is considered to be an internal point and the internal point is expanded. Then, use all the assumed in-game points to re-estimate the model, because this model is only estimated at the initial assumed in-game points, and needs to be updated after subsequent expansion; Finally, the model is evaluated by estimating the error rate between the in-game points and the model; The above process is repeated a certain number of times, and the model stops when it reaches a certain accuracy or stops when the accuracy requirement cannot be met after the number of iterations.

2. The highly robust sea-sky-line extraction method based on photoelectric sensor according to claim 1, characterized in that: The algorithm used by the Sobel operator edge operator is to first perform weighted averaging and then perform differential operations. The calculation method is as follows: Δ x f(x, y) = [f(x-1, y+1)+2f(x, y+1)+f(x+1, y+1)] - [f(x-1, y-1)+2f(x, y-1)+f(x+1, y-1)] Δ y f(x, y) = [f(x-1, y-1) + 2f(x-1, y) + f(x-1, y+1)] - [f(x+1, y-1) + 2f(x+1, y) + f(x+1, y+1)] Where: x, y are the pixel values ​​of the original image f in the x and y directions, and the obtained Δ x f(x, y) is the edge operator of the original image in the x direction, Δ y f(x, y) is the edge operator of the original image in the y direction.

3. The highly robust sea-sky-line extraction method based on photoelectric sensor according to claim 1, characterized in that: The LSD line detection comprises the following steps: firstly, the gradient size and direction of all pixel points in the image are calculated, and then the points with small gradient direction changes and adjacent to each other are regarded as a connected domain. Then, according to the rectangularity of each domain, it is judged whether it is necessary to disconnect it according to the rules to form multiple domains with large rectangularity. Finally, all the generated connected domains are improved and screened, and the domains that meet the conditions are retained, which are the final line detection results.

4. The highly robust sea-sky-line extraction method based on photoelectric sensor according to claim 1, characterized in that: The step of fitting a straight line in step S6 includes: Randomly select 2 points from the control points to fit a straight line; Use this line to test other control points: If the control point (x, y) satisfies the distance d from the line l and the slope k of the original line segment and the slope k' of the line are within a certain threshold, then it is an internal point w: Considering that more points in the horizontal position of the image should be fitted in the sea-sky line, the x-coordinates of the in-place points are subjected to one-dimensional gridding, thereby calculating the number of grids occupied by the fitted in-place points in the horizontal position. The number of effective fitting points = the number of occupied grids / the number of fitted in-place points; Update strategy for straight lines: a) If the number of fitting points of the new round of straight line is greater than the number of fitting points of the current optimal straight line, and the number of effective fitting points is less than the number of effective fitting points of the current optimal straight line, then compare whether the number of fitting points exceeds 1.5 times the number of fitting points of the optimal straight line; b) If the valid fitting number is greater than the valid fitting number of the current best straight line, then compare whether the number of fitting points exceeds half of the number of fitting points of the best straight line; If one of the above conditions is met, the current fitting straight line is used to update the optimal straight line.

5. The highly robust sea-sky-line extraction method based on photoelectric sensor according to any one of claims 1 to 4, characterized in that: The convolutional neural network in step S5 is a lightweight convolutional neural network. The size of the input image is 32x32 pixels. After the image is input into the convolutional neural network model, it passes through two convolutional layers and a pooling layer, and then outputs the classification label through a fully connected layer.

6. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method according to any one of claims 1 to 5 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images