A sea-sky line detection method based on semantic segmentation and hough transform combination

Abstract

The application provides a sea-sky line detection method based on semantic segmentation and Hough transformation, and specific steps are as follows: S1: inputting a visible light image of a water traffic scene, and preparing a data set of the water traffic image; S2: establishing a Mo-SegNet network model of the water traffic image as a semantic segmentation model, and performing pixel-by-pixel segmentation on the input test set of the water traffic image to output a semantic segmentation image; S3: performing edge detection on the semantic segmentation image output by S2 by using a Canny detection algorithm, and outputting a binary image containing only the edge contour of the sea-sky line and a background; and S4: performing Hough transformation on the binary image, fitting the edge pixels of the edge contour of the sea-sky line, and obtaining a sea-sky line detection result. The Mo-SegNet network model is used as the semantic segmentation model, and the Hough transformation is used to fit the sea-sky line, so that accurate detection of the sea-sky line in a complex water traffic scene is realized.

Description

Technical Field

[0001] This invention relates to the field of sea-line detection technology, and in particular to a sea-line detection method based on a combination of semantic segmentation and Hough transform. Background Technology

[0002] The sea-line is the boundary line that divides maritime traffic images into sky and sea areas, and it is one of the important features of maritime traffic images. In maritime traffic images observed horizontally at long distances, most ship targets are located near the sea-line. Sea-line detection helps to narrow the search range for ship targets, providing safety assurance for intelligent navigation of ships, and is of great significance in fields such as maritime search and rescue, route planning, and sea area monitoring.

[0003] Numerous studies have proposed various algorithms for detecting the sea-line horizon, including line fitting, gradient saliency analysis, and image segmentation. However, these methods have not yielded satisfactory results in complex water traffic scenarios. The main reason for this is that water traffic images contain numerous factors that interfere with sea-line horizon detection, such as the similarity between striped waves and the horizon line, the similarity between the sky and the sea surface in foggy weather, and the disruption of the horizon line's continuity by numerous vessels. Therefore, accurate detection in complex water traffic scenarios has certain limitations. Summary of the Invention

[0004] To address the challenges of complex water traffic scenarios and numerous interference factors in sea-line detection, which result in poor performance of existing sea-line detection technologies, this invention proposes a sea-line detection method based on a combination of semantic segmentation and Hough transform. The method employs a Mo-SegNet network model as the semantic segmentation model and uses Hough transform to fit the sea-line, achieving accurate detection of sea-line in complex water traffic scenarios.

[0005] The technical solution is as follows:

[0006] A method for detecting sea-line terrain based on a combination of semantic segmentation and Hough transform includes the following steps:

[0007] S1: Input a visible light image of a water traffic scene, and create a dataset of water traffic images. The dataset includes a training set and a test set. The training set is expanded to obtain an expanded training set.

[0008] S2: Establish a Mo-SegNet network model for water traffic images as a semantic segmentation model. Use the expanded training set in S1 to train the Mo-SegNet network model to obtain semantic segmentation weights. Use the trained Mo-SegNet network model to perform pixel-by-pixel segmentation on the water traffic images in the test set in S1 and output semantic segmentation images.

[0009] S3: The Canny detection algorithm is used to perform edge detection on the semantically segmented image output by S2, and the output is a binary image containing only the outline of the sea-line edge and the background;

[0010] S4: Perform Hough transform on the binary image to fit the edge pixels of the sea-line contour and obtain the sea-line detection result.

[0011] Preferably, in S1, for the visible light image of the input water traffic scene, the image is labeled pixel by pixel by annotation software to create a dataset. The water traffic image is classified into sky region, sea surface region and other regions according to pixels. The dataset is divided into training set and test set in an 8:2 ratio. The training set is expanded by rotating and mirroring the image to finally obtain the expanded training set. The other regions include pixels of ships, floating objects and obstructions on the sea surface.

[0012] Preferably, in S2, the Mo-SegNet network model consists of an encoding network and a decoding network;

[0013] The encoding network consists of a standard convolution and three depthwise separable convolutions. Each convolution adds a batch normalization layer to normalize the output feature map. Through multi-layer convolution, pixel features such as sky, sea surface, and ships in water traffic images are extracted to obtain the corresponding feature layers.

[0014] The decoding network employs three repeated zero-padding, convolution, batch normalization, activation, and upsampling operations to assign a category label to each pixel in the water traffic image; the last decoder layer outputs a high-dimensional feature representation vector, which serves as the input to the function in a trainable Softmax classifier.

[0015] Preferably, the Softmax function in the Softmax classifier is:

[0016]

[0017] In the formula, i represents the output of the i-th node, and z i Let C be the output value of the i-th node, and C be the number of output nodes, i.e. the number of categories.

[0018] Preferably, the Softmax classifier uses cross-entropy as the loss function, as shown in the formula:

[0019]

[0020] In the formula, L ce The loss value is T, where M is the number of categories and T is T. i For the true probability, F i To predict probabilities.

[0021] Preferably, in S3, the specific process of using the Canny detection algorithm to perform edge detection on the semantic segmentation image is as follows: using operators representing the vertical and horizontal directions of edge detection, such as the horizontal G operator... x and the vertical direction G y The first derivative value is obtained, and the orientation of the pixel is inferred. The gradient G of a pixel is calculated using the following formula:

[0022]

[0023]

[0024] In the formula, G represents the gradient intensity, G x G represents the gradient magnitude in the x-direction. y This represents the gradient magnitude in the y-direction. Represents the direction of a pixel, and arctan is the arctangent function.

[0025] Preferably, in S4, the method for fitting the edge pixels of the sea-sky boundary using the Hough transform is as follows:

[0026] In the image space, the straight line within the pixel coordinates of the sea-line edge contour is defined as the predicted sea-line.

[0027] In the xy plane coordinate system, the pixel coordinates (x, y) of each point on any predicted sea line are transformed into Hough space through parameters to obtain a set of cells H(θ, ρ) in the ρθ plane coordinate system. The cells are voted and accumulated to obtain the accumulated value of the predicted sea line. The predicted sea line with the largest accumulated value is selected as the detection result of the sea line.

[0028] Preferably, the formula for the parameter transformation is: ρ = xcosθ + ysinθ

[0029] In the formula, ρ represents the distance from the line to the origin, and θ is the angle between the line and the positive x-axis.

[0030] This invention proposes a sea-sky boundary detection method based on a combination of semantic segmentation and Hough transform. It improves upon the classic semantic segmentation network SegNet by using Mobilenet as the encoding network and simplifies the decoding network to create Mo-SegNet. This Mo-SegNet performs pixel-by-pixel segmentation on the input test set of the water traffic image in S1, reducing network parameters. Simultaneously, the Mo-SegNet network is expanded by performing deep learning training on the training set to enhance its robustness. Third, in S3, the Canny detection algorithm is used to detect edges in the semantically segmented image, identifying the sea-sky boundary and ship edges, effectively eliminating the influence of linear interference factors such as clouds and waves. Fourth, this invention uses Hough transform to fit the sea-sky boundary, effectively solving the problem of the sea-sky boundary being disrupted by ships. In summary, this invention uses the Mo-SegNet network model as the semantic segmentation model and Hough transform to fit the sea-sky boundary, achieving accurate detection of the sea-sky boundary in complex water traffic scenarios. Attached Figure Description

[0031] Figure 1 This is a flowchart of a sea-line detection method based on a combination of semantic segmentation and Hough transform, as shown in the embodiment.

[0032] Figure 2 This is a visible light image of a water traffic scene in one of the embodiments.

[0033] Figure 3 The example image includes the outline of the sea surface and the background.

[0034] Figure 4 The image shows the detection effect of the sea-line detection method based on the combination of semantic segmentation and Hough transform in the embodiment. Detailed Implementation

[0035] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0036] Example 1:

[0037] like Figure 1 As shown, a sea-line detection method based on a combination of semantic segmentation and Hough transform includes the following steps:

[0038] S1: Input a visible light image of a water traffic scene, and create a dataset of water traffic images. The dataset includes a training set and a test set. The training set is expanded to obtain an expanded training set.

[0039] S2: Establish a Mo-SegNet network model for water traffic images as a semantic segmentation model. Use the expanded training set in S1 to train the Mo-SegNet network model to obtain semantic segmentation weights. Use the trained Mo-SegNet network model to perform pixel-by-pixel segmentation on the water traffic images in the test set in S1 and output semantic segmentation images.

[0040] S3: The Canny detection algorithm is used to perform edge detection on the semantically segmented image output by S2, and the output is a binary image containing only the outline of the sea-line edge and the background;

[0041] S4: Perform Hough transform on the binary image to fit the edge pixels of the sea-line contour and obtain the sea-line detection result.

[0042] In step S1, for the input water traffic images, the images are labeled pixel-by-pixel by annotation software to create a dataset. The water traffic images are classified according to pixels into sky regions, sea surface regions, and other regions. The other regions include pixels of ships, floating objects, and obstructions on the sea surface. The dataset is then divided into training and testing sets in an 8:2 ratio. Image processing operations such as rotation and mirroring are performed on the training set to expand the dataset and enhance the robustness of the network.

[0043] In step S2, a semantic segmentation model for water traffic images is established. Specifically, this includes:

[0044] (1) An improved version of the classic semantic segmentation network SegNet is proposed, namely the Mo-SegNet network model.

[0045] The encoding network consists of one standard convolution and three depthwise separable convolutions. Each convolution adds a Batch Normalization (BN) layer to normalize the output feature map. Multiple convolutions extract pixel features such as the sky, sea surface, and ships from the water traffic image, resulting in corresponding feature layers. The decoding network is a simplified modification of the classic SegNet network decoding structure. It employs three repeated zero-padding, convolution, batch normalization, activation, and upsampling operations to assign a class label to each pixel in the water traffic image. The final decoder layer outputs a high-dimensional feature representation vector, which serves as the input to a trainable Softmax classifier.

[0046] The softmax function is defined and calculated as follows:

[0047] In the formula, i represents the output of the i-th node, and z i Let C be the output value of the i-th node, and C be the number of output nodes, i.e. the number of categories.

[0048] Using cross-entropy as the loss function, its calculation formula is as follows:

[0049] In the formula, L ce The loss value is T, where M is the number of categories and T is T. i For the true probability, F i To predict probabilities.

[0050] (2) The training set created in S1 is fed into the proposed network model for training, and the weights obtained from the training are used to predict the images in the test set.

[0051] In step S3, the Canny detection algorithm is used to perform edge detection on the semantic segmentation image, which detects the sea-sky boundary and ship edges in the semantic segmentation image, effectively eliminating the influence of linear interference factors such as clouds in the sky and waves on the sea.

[0052] Images of water traffic exhibit significant differences between adjacent pixels with large gradient changes, indicating the presence of edges in these regions. The edge detection process involves operators representing the vertical and horizontal directions of edge detection, such as the horizontal G operator. x and the vertical direction G y The first derivative value is obtained, and the orientation of the pixel is inferred. The gradient G of the pixel is calculated using the following formula: and

[0053] In the formula, G represents the gradient intensity, G x G represents the gradient magnitude in the x-direction. y This represents the gradient magnitude in the y-direction. Represents the direction of a pixel, and arctan is the arctangent function.

[0054] After finding the intensity gradient of the water traffic image, non-maximum suppression technique is applied to suppress pixels with insufficient gradients, retaining only the largest gradient to prevent false edge detection.

[0055] In step S4, the Hough transform is used to fit the edge pixels of the sea-sky boundary to obtain the sea-sky boundary detection result. The basic idea of ​​the Hough transform is to transform the pixel coordinates (x, y) in the image space to a set of cells H(θ, ρ) in the Hough space through parameter transformation and then accumulate the results by voting. The parameter curve is defined by the following formula:

[0056] ρ=xcosθ+ysinθ

[0057] In the formula, ρ represents the distance from the line to the origin, and θ is the angle between the line and the positive x-axis.

[0058] By using the accumulation mechanism of the cumulative units in the ρθ plane coordinate system, the accumulated value of each straight line can be obtained. The larger the accumulated value of each straight line, the more edge pixels that the line passes through in the xy plane, and the greater the probability that it is a sea-line. Therefore, the Hough transform is used to obtain a point with the largest accumulated value in the ρθ plane coordinate system, and the straight line corresponding to this point in the xy plane is selected as the sea-line.

[0059] Example 2:

[0060] The following describes a sea-line detection method based on a combination of semantic segmentation and Hough transform disclosed in this invention. This embodiment is built on the Tensorflow deep learning framework and implemented using the Python programming language and OpenCV library. The specific implementation steps are as follows:

[0061] S1: Obtain visible light images of typical water traffic scenes and create a dataset.

[0062] This embodiment selects a typical water traffic scene including ocean waves, clouds, and ship interference. The original color image is a 24-bit RGB digital image with a resolution of 1920×1080. Figure 2 As shown.

[0063] S2: Establish a semantic segmentation model for water traffic images and perform pixel-by-pixel segmentation on the input water traffic images.

[0064] The training set was fed into the proposed Mo-SegNet network for feature learning. To ensure effective network operation, the original image resolution was reduced from 1920×1080 to 416×416. Features were extracted by the encoding network and parsed by the decoding network to segment the water traffic image into three categories: sky region, sea surface region, and other regions. Figure 3 As shown.

[0065] S3: The Canny detection algorithm is used to perform edge detection on the semantically segmented image, thereby obtaining a binary image containing only edge contours and background.

[0066] In semantic segmentation images, pixels such as clouds are classified as sky regions, pixels such as waves are classified as sponge regions, and pixels such as ships are classified as other regions, which effectively reduces the interference of linear noise such as clouds and waves for subsequent edge detection.

[0067] S4: Perform Hough transform on the binary image and fit the edge pixels of the sea-sky boundary to obtain the sea-sky boundary detection result.

[0068] The pixels at the horizon edge are fitted as the final horizon detection result, such as... Figure 4 As shown.

[0069] To quantitatively evaluate the sea-line detection method based on semantic segmentation and Hough transform proposed in this embodiment, experiments were conducted in a typical water traffic scenario, and the method was compared with PSPnet, Unet, and DeepLab v3+ networks. Evaluation metrics included Accuracy, Precision, F1-score, and IoU. The calculation formulas are as follows:

[0070]

[0071]

[0072]

[0073]

[0074]

[0075] In the formula: RB represents the number of pixels correctly classified as background; RH represents the number of pixels correctly classified as the sea-line region; LB represents the number of background pixels incorrectly classified as the sea-line region; LH represents the number of sea-line region pixels incorrectly classified as background.

[0076] The calculation results are shown in Table 1.

[0077] Table 1

[0078]

[0079] This embodiment uses surveillance video captured under ocean wave interference conditions, including numerous interfering factors such as clouds, waves, and ships. As shown in Table 1, the proposed sea-line detection method combining the Mo-SegNet semantic segmentation network and Hough transform demonstrates improvements across various evaluation metrics compared to other network algorithms, essentially achieving accurate detection of sea-line antennas in complex maritime traffic scenarios.

[0080] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for detecting sea-line terrain based on a combination of semantic segmentation and Hough transform, characterized in that, Includes the following steps: S1: Input a visible light image of a water traffic scene, and create a dataset of water traffic images. The dataset includes a training set and a test set. The training set is expanded to obtain an expanded training set. S2: Establish a Mo-SegNet network model for water traffic images as a semantic segmentation model. Use the expanded training set in S1 to train the Mo-SegNet network model to obtain semantic segmentation weights. Use the trained Mo-SegNet network model to perform pixel-by-pixel segmentation on the water traffic images in the test set in S1 and output semantic segmentation images. The Mo-SegNet network model consists of an encoding network and a decoding network; The encoding network consists of a standard convolution and three depthwise separable convolutions. Each convolution adds a batch normalization layer to normalize the output feature map. Through multi-layer convolution, pixel features such as sky, sea surface, and ships in water traffic images are extracted to obtain the corresponding feature layers. The decoding network employs three repeated zero-padding, convolution, batch normalization, activation, and upsampling operations to assign a category label to each pixel in the water traffic image; the last decoder layer outputs a high-dimensional feature representation vector, which serves as the input to the function in a trainable Softmax classifier. S3: The Canny detection algorithm is used to perform edge detection on the semantically segmented image output by S2, and the output is a binary image containing only the outline of the sea-line edge and the background; S4: Perform a Hough transform on the binary image to fit the edge pixels of the sea-line contour, and obtain the sea-line detection result: In the image space, the straight line within the pixel coordinates of the sea-line edge contour is defined as the predicted sea-line. exist In a planar coordinate system, the pixel coordinates of each point on any predicted sea-line are... By transforming the parameters into Hough space, we obtain A set of cells in a plane coordinate system The cells are voted on and accumulated to obtain the accumulated value of the predicted sea level. The predicted sea level with the largest accumulated value is selected as the detection result of the sea level.

2. The sea-line detection method based on semantic segmentation and Hough transform as described in claim 1, characterized in that, In S1, for the visible light image of the input water traffic scene, the image is labeled pixel by pixel by the annotation software to create a dataset. The water traffic image is classified into sky region, sea surface region and other regions according to the pixels. The dataset is divided into training set and test set in an 8:2 ratio. The training set is expanded by rotating and mirroring the image. The other regions include pixels of ships, floating objects and obstructions on the sea surface.

3. The sea-line detection method based on semantic segmentation and Hough transform as described in claim 1, characterized in that, The Softmax function in the Softmax classifier is: In the formula, i represents the output of the i-th node. For the first The output value of each node, This represents the number of output nodes, i.e., the number of categories.

4. The sea-line detection method based on semantic segmentation and Hough transform as described in claim 1, characterized in that, The Softmax classifier uses cross-entropy as the loss function, and the formula is as follows: In the formula, The loss value. For the number of categories, For true probability, To predict probabilities.

5. The sea-line detection method based on semantic segmentation and Hough transform as described in claim 1, characterized in that, In S3, the specific process of using the Canny detection algorithm to perform edge detection on the semantic segmentation image is as follows: Operators representing the vertical and horizontal directions of edge detection, such as the horizontal direction... and vertical direction The first derivative value is obtained, and the orientation of the pixel is inferred. and pixel gradient The calculation formula is: In the formula, Represents gradient strength. represent Gradient magnitude in the direction, represent Gradient magnitude in the direction, Represents the direction of a pixel, and arctan is the arctangent function.

6. The sea-line detection method based on semantic segmentation and Hough transform as described in claim 1, characterized in that, The formula for the parameter transformation is: In the formula, This represents the distance from the line to the origin. For the line and positive The included angle of the axis.

Images