A water-air antenna detection method for an unmanned ship perception system

Abstract

The application relates to a water antenna detection method for an unmanned ship sensing system, and the features include the following steps: S1) establishing a water antenna detection convolutional neural network structure, wherein the water antenna detection convolutional neural network structure comprises a backbone network for feature extraction and a branch prediction output head network; S2) training the water antenna detection convolutional neural network structure established in the step S1 by using a training data set; S3) when the training reaches an iteration number Q, fixing and saving all weight values w and bias values b in the network to obtain a trained convolutional neural network model W; S4) storing all (x, y, theta) in a container allpoints; and S5) obtaining a water antenna image coordinate straight line equation y=k*x+b. The water antenna detection method can avoid the low image segmentation precision of a traditional image algorithm, cannot obtain accurate water antenna detection results, has high running speed and high reliability.

Description

Technical Field

[0001] This invention belongs to the field of computer image processing technology, and specifically relates to a method for detecting underwater antennas in an unmanned surface vessel (USV) perception system. Background Technology

[0002] In recent years, unmanned surface vessels (USVs) have played an increasingly important role in civilian scientific research and military applications. USVs need to navigate autonomously and complete various tasks without human intervention, therefore they must possess the ability to perceive the surface environment and various targets within it. Current USVs are equipped with radar, cameras, photoelectric sensors, and other sensors to acquire environmental and target information within a certain range.

[0003] In visible light images captured by cameras or optoelectronic devices mounted on unmanned surface vessels (USVs) and other maritime vessels, ship targets typically appear near the horizon line, and the position of the horizon line in the image is related to the camera's attitude. Therefore, accurate detection of the horizon line is crucial for narrowing the target detection range, assisting image calibration, and facilitating optoelectronic target tracking. Furthermore, the horizon line can be used to segment the image into two parts, allowing for different processing methods for each part. Thus, the horizon line can be used for visual navigation and optoelectronic target tracking of USVs, serving as an important reference feature for autonomous navigation.

[0004] There is considerable research on water antenna detection, mostly applied to infrared images, and can be broadly categorized into two methods. The first is edge-point line detection, which uses line fitting or projection of high-gradient edge pixels to detect water antennas or candidate water antennas. The advantage of this method is its high accuracy when there is minimal interference from ocean waves. However, its disadvantage is that it cannot simultaneously extract water antenna edges and suppress wave edges, and it struggles to adapt to changes in water antenna gradients due to factors such as lighting. The second method is image segmentation-based, which segments the image through grayscale thresholding or background modeling, then extracts the upper boundary of the seawater region as the water antenna. This method is robust to edge interference from ocean waves, but its disadvantage is that it is limited by the accuracy of image segmentation, resulting in inaccurate water antenna detection. In addition, there are line scanning-based methods and shear wave transform-based methods. However, the former is difficult to adapt to images with nearby ships or tilted water antennas, while the latter is computationally complex and has poor real-time performance.

[0005] Visible light sea surface images captured by shipborne cameras are subject to interference factors such as waves, reflections, and ships, and the image clarity changes constantly with sea conditions and lighting conditions. This places high demands on the robustness and universality of the surface detection method.

[0006] Current methods for detecting water antennas utilize visual sensors to acquire images, followed by algorithmic processing. These algorithms include pixel-based saliency methods, transform-domain methods, and deep learning-based methods. These water antenna detection algorithms generally achieve good extraction results and high reliability; the differences lie in the real-time performance and the ease of hardware implementation. Ultimately, water antenna detection boils down to extracting straight-line features from the image scene. Therefore, most traditional image detection algorithms combine Hough transform or Randon transform to complete water antenna detection. Using traditional image processing algorithms for water antenna detection is a widely used approach. Dai Yongshou et al. from the School of Information and Control Engineering at China University of Petroleum (East China) proposed "Water Antenna Detection Based on Local Otsu Segmentation and Hough Transform." First, longitudinal median filtering is performed to suppress high-frequency noise. Then, edge pixels are extracted based on local Otsu image segmentation. Finally, the Hough transform is used to fit the edge pixels to obtain the straight-line equation of the water antenna.

[0007] Therefore, existing technologies for detecting water antenna lines, especially those for inland lakes and unmanned surface vessels (USVs) during near-shore patrols, have several drawbacks due to the abundance and complexity of background information: 1) Traditional image processing algorithms for water antenna detection suffer from numerous background interference elements and noise points, which are difficult to completely eliminate through filtering, resulting in low image segmentation accuracy and inability to obtain accurate water antenna detection results. Therefore, their practicality is limited; 2) Due to the inherent hardware characteristics of visual sensors (cameras), image quality is poor in conditions such as rain, snow, backlighting, strong light, and nighttime. This is particularly true for USVs, where external factors such as sea vapor and waves can cause water droplets and mist to interfere with the camera lens, leading to image blurring and affecting the extraction and recognition of water antenna features. Summary of the Invention

[0008] The purpose of this invention is to propose a method for detecting hydrofoil antennas in an unmanned surface vessel (USV) perception system. This method avoids the problem of low image segmentation accuracy in traditional image processing algorithms, which prevents accurate detection of hydrofoil antennas. It also avoids the difficulties in extracting hydrofoil antenna features and the low recognition accuracy caused by the hardware characteristics of the visual sensor (camera) itself under conditions such as rain, snow, backlight, strong light, and night. The final algorithm has better detection performance, faster operation speed, and higher reliability.

[0009] The specific technical solution of this invention is a method for detecting hydrofoil lines in an unmanned surface vessel (USV) sensing system, characterized by the following steps:

[0010] S1) Establish a convolutional neural network structure for water antenna detection, wherein the convolutional neural network structure for water antenna detection includes: a backbone network for feature extraction and a branch prediction output head network.

[0011] The backbone network used for feature extraction consists of 9 convolutional layers. The data input layer (Data) contains 16 images of size 192*512. After the CBR1 operation, the feature map size is 32*96*256. CBR1 consists of 32 5*5 convolutional operations with a stride of 2, plus Batch Normalization (BN) and ReLU activation. After the CBR2 operation, the feature map size is 64*48*128. CBR2 consists of 64 3*3 convolutional operations with a stride of 2, plus BN and ReLU activation. After the CBR3 operation, the feature map size is 64*48*128. CBR3 consists of 64 3*3 convolutional operations with a stride of 1, plus BN and ReLU activation. After the P1 operation, the feature map size is 64*24. *64, P1 is a 3*3 Maxpool operation with a stride of 2; after CBR4, the feature map size is 64*24*64, CBR4 consists of 64 3*3 convolution operations with a stride of 1 + BN operation + ReLU activation function; after CBR5, the feature map size is 128*12*32, CBR5 consists of 128 3*3 convolution operations with a stride of 2 + BN operation + ReLU activation function; then two operation branches are formed, after Conv1, the feature map size is 128*12*32, Conv1 consists of 128 3*3 convolution operations with a stride of 1, and after P2, the feature map size is 128*6*16, P2 is a 3*3 Maxpool operation with a stride of 2. Pooling operations are performed; after CBR6, the feature map size is 128*6*16. CBR6 consists of 128 3*3 convolutions with a stride of 1, plus BN operations and ReLU activation. After CBR7, the feature map size is 128*12*32. CBR7 consists of 128 1*1 convolutions with a stride of 1, plus BN operations and ReLU activation. Two more branches are then generated: Conv2, which results in a feature map size of 128*6*16, consisting of 128 3*3 convolutions with a stride of 1; and P3, which results in a feature map size of 128*3*8, consisting of 3*3 max convolutions with a stride of 2. Pooling operation; After CBR8, the feature map size is 128*3*8. CBR8 consists of 128 3*3 convolution operations with a stride of 1 + BN operation + ReLU activation function; After Deconv1, the feature map size is 128*6*16. Deconv1 consists of 128 3*3 deconvolution operations; After Eltwise1, the feature map size is 256*6*16. Eltwise1 is a summation operation; After Deconv2, the feature map size is 128*12*32. Deconv2 consists of 128 3*3 deconvolution operations; After Eltwise2, the feature map size is 256*12*32. Eltwise2 is a summation operation.After the CBR9 operation, the feature map size is 128*12*32. CBR9 consists of 128 3*3 convolution operations with a stride of 1, BN operations, and ReLU activation function.

[0012] The branch prediction output head network specifically includes three branches: a keypoint heatmap prediction branch, a keypoint offset prediction branch, and a water surface angle prediction branch. The keypoint heatmap prediction branch operates on CBR10-1 and Conv3-1 methods. After CBR10-1, the feature map size is 64*12*32, consisting of 64 3*3 convolutions with a stride of 1, followed by BN and ReLU activation. After Conv3-1, the feature map size is N*12*32, consisting of N 1*1 convolutions with a stride of 1, followed by BN and ReLU activation, where N is the number of target categories (N=1 here). The loss function is the keypoint heatmap loss L. hm The keypoint offset prediction branch network operates on CBR10-2 and Conv3-2 methods. After CBR10-2, the feature map size is 64*12*32, consisting of 64 3*3 convolutions with a stride of 1, followed by batch normalization (BN) and ReLU activation. After Conv3-2, the feature map size is 2*12*32, consisting of two 1*1 convolutions with a stride of 1, followed by BN and ReLU activation. The keypoint offset prediction branch has two output channels, and the loss function is the offset loss L from the predicted keypoint to the ground truth keypoint. offset The water antenna angle prediction branch network operates on CBR10-3 and Conv3-3. After CBR10-3, the feature map size is 64*12*32, consisting of 64 3*3 convolution operations with a stride of 1, followed by Batch Normalization (BN) and ReLU activation. After Conv3-3, the feature map size is 1*12*32, consisting of one 1*1 convolution operation with a stride of 1, followed by BN and ReLU activation. The water antenna angle prediction branch has one output channel, and its output is the angle θ between the water antenna and the positive x-axis of the image. The loss function is the angle θ loss L. θ ;

[0013] S2) Train the water antenna detection convolutional neural network structure established in step S1 using the training dataset. The total loss of the loss function during training is given by the following formula (I):

[0014] L loss =γ1L hm +γ2L offset +γ3L θ .................(I)

[0015] Among them, γ1, γ2, and γ3 are L respectively. hm L offset The weighting factors of Lθ respectively;

[0016] S3) When the training reaches the iteration number Q, all weight values ​​w and bias values ​​b in the network are fixed and saved to obtain the trained convolutional neural network model W. During water antenna detection, the image to be identified is input into the convolutional neural network model W. To limit the values ​​in feature map Conv3-1 to the range (0, 1), feature map Conv3-1 undergoes another sigmoid operation to obtain a heatmap, transforming the 1*12*32 three-dimensional matrix heatmap into a 12*32 matrix X, where each value in matrix X is the confidence of the key point at that location; transforming the 1*12*32 three-dimensional matrix feature map Conv3-2 into two 12*32 offset matrix matrices ΔX and ΔY; and transforming the 1*12*32 three-dimensional matrix feature map Conv3-3 into a 12*32 water antenna angle matrix Ω.

[0017] S4) By combining the coordinates (i, j) of the positions with a confidence level greater than 0.5 in matrix X with the corresponding ΔX(i, j) and ΔY(i, j) in matrices ΔX and ΔY, we can obtain the pixel coordinates (x, y) of the points on the horizon in the real image, as well as the corresponding θ value, using the following formula V. The detailed calculation formula is as follows:

[0018]

[0019] Store all (x, y, θ) values ​​into the container allpoints;

[0020] S5) Assuming there are m (x, y, θ) values, if m is less than 5, then it is assumed that no waterline exists in the image. If m is greater than 5, all (x, y, θ) values ​​in the allpoints container are sorted in ascending order according to the x-axis direction, where the x-axis direction is the direction of the unmanned surface vessel from left to right in the world coordinate system. The mean of all x and y values ​​in the allpoints container is then calculated. The mean of all θ values ​​in the container allpoints is obtained. This leads to the slope k of the waterline. Depend on Given k, calculate the parameter b of the linear equation to obtain the linear equation y = k × x + b for the water antenna image.

[0021] Furthermore, L in step S2 hm L offset Lθ The calculation formulas are as follows: (II), (III), (IV):

[0022]

[0023] Where N is the number of currently detected keypoints, and xyc represents the current category number c at position xy in the heatmap. Y is the predicted value for detecting key points. xyc The values ​​are the actual values ​​at key points on the horizon, where α and β are hyperparameters.

[0024]

[0025] Where p represents the coordinates of the key point. Let D be the keypoint deviation value we want to learn, and D be the downsampling factor. This represents the actual deviation caused by keypoint downsampling.

[0026]

[0027] Where k represents the key point being detected. It is the predicted angle information. This is the true angle information of the waterline. The beneficial effects of this invention are: 1) The waterline detection method for unmanned surface vessel perception systems of this invention utilizes deep learning algorithms to overcome interference from factors such as severe loss of sea-sky edges, large differences in different sea-sky scenes, and lighting, ripples, and reflections in complex sea-sky scenes, thereby improving the accuracy and efficiency of sea-sky image segmentation and detecting waterlines with clear and complete outlines; 2) The waterline detection method based on convolutional neural networks achieves all-weather operation despite image problems caused by weather, lighting, and other factors; 3) Compared with traditional machine learning algorithms for detecting waterlines, the waterline detection method based on convolutional neural networks is faster and more accurate.

[0028] Experimental results show that the proposed method for detecting water-line antennas in unmanned surface vessels (USVs) perception systems can detect relatively clear and complete water-line antennas in different sea and sky images. This algorithm overcomes interference from factors such as severe sea-sky edge loss, significant differences in different sea-sky scenes, and complex sea-sky scenes involving lighting, ripples, and reflections. It improves the accuracy and efficiency of sea-sky image segmentation, detects water-line antennas with clear and complete outlines, serves the situational awareness of USVs, provides a foundation for subsequent decision-making and control systems, and reduces the cost of USVs' perception systems and the power consumption of perception software equipment. In terms of data processing, the accuracy reaches 98.91%, and the image processing speed is around 15ms, both real-time performance and accuracy meeting the mission requirements of USVs. Attached Figure Description

[0029] Figure 1This is a schematic diagram of the process for detecting the hydrofoil antenna in an unmanned surface vessel's perception system.

[0030] Figure 2 A schematic diagram of the neural network training and testing structure used in the hydrofoil detection method for unmanned surface vessel perception systems;

[0031] Figure 3 A schematic diagram with annotations for training images of hydrofoil lines used in a hydrofoil detection method for an unmanned surface vessel (USV) perception system.

[0032] Figure 4 The image shows the detection results of the hydrofoil detection method used in the unmanned surface vessel perception system. Detailed Implementation

[0033] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0034] As attached Figure 1 As shown, a method for detecting an underwater antenna in an unmanned surface vessel (USV) sensing system according to the present invention includes the following steps:

[0035] 1) Construct a convolutional neural network (CNN) structure for water antenna detection and train it using a training dataset. The CNN structure includes a backbone network for feature extraction and a head network for branch prediction output. The model can be built using PyTorch software.

[0036] The backbone network used for feature extraction is attached. Figure 2The upper part of the left-hand image contains 9 convolutional layers. The data input layer (Data) consists of 16 images of size 192*512. After the CBR1 operation, the feature map size is 32*96*256. CBR1 consists of 32 5*5 convolutional operations with a stride of 2, plus BN operations and ReLU activation. After the CBR2 operation, the feature map size is 64*48*128. CBR2 consists of 64 3*3 convolutional operations with a stride of 2, plus BN operations and ReLU activation. After the CBR3 operation, the feature map size is 64*48*128. BR3 consists of 64 3x3 convolution operations with a stride of 1, plus batch normalization (BN) and ReLU activation. After P1, the feature map size is 64*24*64, and P1 is a 3x3 Maxpool operation with a stride of 2. After CBR4, the feature map size is also 64*24*64, consisting of 64 3x3 convolution operations with a stride of 1, plus BN and ReLU activation. After CBR5, the feature map size is 128*12*32, and CBR5 consists of 128 steps... The process begins with a 3x3 convolution operation of length 2, followed by batch normalization (BN) and ReLU activation. Two branches are then formed: Conv1, which performs 128 x 12 x 32 convolutions with a stride of 1, and P2, which performs a 3x3 Maxpool operation with a stride of 2, resulting in a feature map size of 128 x 12 x 32. CBR6, on the other hand, also performs 128 x 6 x 16 convolutions with a stride of 1. The CBR7 operation involves a product operation, a batch normalization (BN) operation, and a ReLU activation function. After the CBR7 operation, the feature map size is 128*12*32. CBR7 consists of 128 1x1 convolutions with a stride of 1, followed by a batch normalization (BN) operation and a ReLU activation function. Two more branches are then generated: Conv2, which results in a feature map size of 128*6*16, consisting of 128 3x3 convolutions with a stride of 1; and P3, which results in a feature map size of 128*3*8, consisting of 3x3 convolutions with a stride of 2. Pooling operation; After CBR8, the feature map size is 128*3*8. CBR8 consists of 128 3*3 convolution operations with a stride of 1 + BN operation + ReLU activation function; After Deconv1, the feature map size is 128*6*16. Deconv1 consists of 128 3*3 deconvolution operations; After Eltwise1, the feature map size is 256*6*16. Eltwise1 is a summation operation; After Deconv2, the feature map size is 128*12*32. Deconv2 consists of 128 3*3 deconvolution operations; After Eltwise2, the feature map size is 256*12*32. Eltwise2 is a summation operation.After the CBR9 operation, the feature map size is 128*12*32. CBR9 consists of 128 3*3 convolution operations with a stride of 1, plus batch normalization (BN) operations and ReLU activation function.

[0037] See attached branch prediction output head network. Figure 2 The lower part of the left-hand diagram includes three branches: keypoint heatmap prediction, keypoint offset prediction, and water surface angle prediction. The keypoint heatmap prediction branch uses CBR10-1 and Conv3-1 operations. After CBR10-1, the feature map size is 64*12*32, consisting of 64 3*3 convolutions with a stride of 1, followed by BN and ReLU activation. After Conv3-1, the feature map size is N*12*32, consisting of N 1*1 convolutions with a stride of 1, followed by BN and ReLU activation. N represents the number of target categories, here N=1. The loss function is the keypoint heatmap loss L. hm The keypoint offset prediction branch network operates on CBR10-2 and Conv3-2 methods. After CBR10-2, the feature map size is 64*12*32, consisting of 64 3*3 convolutions with a stride of 1, followed by batch normalization (BN) and ReLU activation. After Conv3-2, the feature map size is 2*12*32, consisting of two 1*1 convolutions with a stride of 1, followed by BN and ReLU activation. The keypoint offset prediction branch has two output channels, and the loss function is the offset loss L from the predicted keypoint to the ground truth keypoint. offset The water antenna angle prediction branch network operates on CBR10-3 and Conv3-3. After CBR10-3, the feature map size is 64*12*32, consisting of 64 3*3 convolution operations with a stride of 1, followed by Batch Normalization (BN) and ReLU activation. After Conv3-3, the feature map size is 1*12*32, consisting of one 1*1 convolution operation with a stride of 1, followed by BN and ReLU activation. The water antenna angle prediction branch has one output channel, and its output is the angle θ between the water antenna and the positive x-axis of the image. The loss function is the angle θ loss L. θ .

[0038] 2) After the network is built, it is trained using the training dataset. The training steps are as follows:

[0039] 2a) Randomly initialize all weight values ​​w and offset values ​​b in each layer of the water antenna detection convolutional neural network structure;

[0040] 2b) The preprocessed training samples are input into the data input layer of the neural network for water antenna detection convolutional neural network training. The total loss of the loss function during training is given by the following formula (I):

[0041] L loss =γ1L hm +γ2L offset +γ3L θ ......(I)

[0042] Where L hm L is the loss function for the keypoint heatmap. offset To predict the offset loss function from keypoints to true keypoints, L θ Let L be the angle θ loss function, and let γ1 = 1, γ2 = 0.9, and γ3 = 0.9 be L respectively. hm L offset L θ Each has its own weighting factor. The loss function is continuously reduced through iterative training of the network until the set number of iterations, Q = 200, is completed.

[0043] 2c) This method treats the water-line antenna as multiple coordinate points for detection. Specifically, it uses multiple keypoints on the water-line antenna to represent the target, predicts the keypoint offset, and the angle θ between the water-line antenna and the image x-axis, thus obtaining the actual linear equation of the water-line antenna. The heatmap, on the other hand, represents classification information. Each category has its own heatmap. If a keypoint exists at a certain coordinate on each heatmap, a keypoint (represented by a Gaussian circle) is generated at that coordinate.

[0044] Input image I∈R W×H×3 R represents the image training data set, W represents the image width, and H represents the image height. The network outputs a keypoint heatmap. Where D represents the step size of the output relative to the original image, and C represents the number of categories, let W = 192, H = 512, D = 16, C = 1. The keypoint heatmap loss function L in step 2b) hm Loss function L for predicting the offset from the actual keypoint offset and angle θ loss function L θ The calculation formulas are as follows:

[0045] Keypoint heatmap loss function L hm According to formula (II):

[0046]

[0047] Where N is the number of currently detected keypoints, and xyc represents the current category number c at position xy in the heatmap. Y is the predicted value for detecting key points. xyc Y represents the actual value of the key point on the waterline. α and β are hyperparameters, set to 2 and 4 respectively, to balance the difficulty of samples and the positive and negative samples. xyc When = 1, It then serves a corrective function, if If the value is close to 1, it means that this is a relatively easy point to detect. That's correspondingly lower. And when... When the value is close to 0, it indicates that this key point has not yet been learned, so the training emphasis should be increased. This will be very large. Otherwise, the training weight (loss) of other nearest neighbors of the actual keypoint is also adjusted here to prevent the predicted value from being too large. Too high, close to 1, so use To punish Loss; and (1-γ) xyc ) β The closer this parameter is to the center, the smaller its value; this weight is used to reduce the penalty. If the predicted value... If it is close to 0, then It will be very small, reducing its proportion of loss, for (1-Y) xyc ) β In this way, the weight of the loss from other negative samples around the keypoint is reduced, which can better improve the detection accuracy of the algorithm.

[0048] Because the backbone network for feature extraction downsamples the image by D=16, remapping this feature map onto the original image introduces accuracy errors. Therefore, for each keypoint, an additional loss function is used to eliminate the deviation in keypoint position when mapping the heatmap onto the original image. This deviation is calculated as L. offset According to formula (III):

[0049]

[0050] Where p represents the coordinates of the key point. Let D be the keypoint deviation value we want to learn, and D be the downsampling factor of 16. This represents the actual deviation caused by downsampling at key points.

[0051] To facilitate the calculation of the linear equation of the water antenna, a learning function for the angle θ between the water antenna and the image x-axis is introduced, with each keypoint corresponding to an angle θ. A loss function L is used to obtain the deflection angle θ of the water antenna. θ According to formula (IV):

[0052]

[0053] Where k represents the key point being detected. It is the predicted angle information. This is the actual angle information of the waterline.

[0054] 2d) Based on the loss function defined in step 2c), perform backpropagation to optimize the convolutional neural network;

[0055] 2e) In the backpropagation algorithm, gradient descent is used to adjust the weights (w) and biases (b) of each layer in the neural network, as well as the various changes Δw and Δb. The update formulas for Δw and Δb are as follows:

[0056]

[0057] Where L loss The total loss function is shown in formula (I).

[0058] 2f) Use the learning rate lr, weight change Δw, and bias change Δb to update the weights (w) and biases (b) of each layer in the network. The update formula for the weights and biases of the i-th layer is: w i =w i -lr*Δw i b i =b i -lr*Δb i An initial learning rate (lr) of 0.01 is appropriate. When training reaches 100 steps, the learning rate becomes lr = 0.001.

[0059] 3) When the training reaches the number of iterations Q = 200, fix and save all weight values ​​(w) and bias values ​​(b) in the network to obtain the trained convolutional neural network model W.

[0060] The trained network was evaluated using a test dataset. The specific process involved resizing and normalizing a 1920*1080*3 image to obtain a 192*512*3 image for detection. This image was then input into the trained convolutional neural network model W. (See appendix for details.) Figure 2As shown in the right figure, in order to restrict the values ​​in feature map Conv3-1 to the range of (0, 1), feature map Conv3-1 undergoes another sigmoid operation to obtain a heatmap, transforming the 1*12*32 three-dimensional matrix heatmap into a 12*32 matrix X, where each value in matrix X is the confidence of the key point at that location; transforming the 1*12*32 three-dimensional matrix feature map Conv3-2 into two 12*32 offset matrix matrices ΔX and ΔY; and transforming the 1*12*32 three-dimensional matrix feature map Conv3-3 into a 12*32 water antenna angle matrix Ω.

[0061] 4) To obtain the key points of the hydrothermal traverse line in the real image, the coordinates (i, j) of the positions with a confidence level greater than 0.5 in matrix X, and the corresponding ΔX(i, j) and ΔY(i, j) in matrices ΔX and ΔY, are processed by the following formula V to obtain the pixel coordinates (x, y) of the points on the hydrothermal traverse line in the real image, as well as the corresponding θ value. The calculation formula is as follows:

[0062]

[0063] Store all (x, y, θ) values ​​into the container allpoints.

[0064] 5) Assuming there are m (x, y, θ) values, if m is less than 5, then it is assumed that no waterline appears in the image; if m is greater than 5, all (x, y, θ) values ​​in the allpoints container are sorted in ascending order according to the x-axis direction, where the x-axis direction is the direction of the unmanned surface vessel from left to right in the world coordinate system; and the mean of all x and y values ​​in the allpoints container is calculated. The mean of all θ values ​​in the container allpoints is obtained. This leads to the slope of the waterline. Depend on Given k, calculate the parameter b of the linear equation to obtain the linear equation y = k × x + b for the water antenna image.

[0065] In one specific embodiment of the invention, a camera or photoelectric device is mounted on the top or bow of the unmanned surface vessel (USV) to collect image data including the waterline. To accommodate various complex water and sky scenes, the collected images of different sizes (1280*720, 1920*1080, etc.) need to be expanded, for example, by randomly flipping horizontally or vertically, scaling, or altering the lighting. This increases the amount of data and expands the image data from different angles.

[0066] The collected data was labeled using the data annotation software LabelMe to mark the water lines in the images, as shown in the attached image. Figure 3 As shown in the diagram. The dataset for training the water-sky antenna is then divided into a training sample set and a test sample set. Six thousand images of different sizes, angles, and types of water-sky scenes were collected, and 4,000 images were selected as training samples for the network. The training sample path is saved in the `train.txt` file. The training samples and the `train.txt` file constitute the training dataset. Similarly, the remaining 2,000 images and the `test.txt` file constitute the test dataset.

[0067] The water and sky images in the obtained training sample set were preprocessed. The preprocessing included: normalizing the image size to 192*512, randomly flipping the images horizontally and horizontally, and performing data augmentation using Mixup in PyTorch software, with the batch size set to 16.

[0068] Draw straight lines and corresponding points on the identified image to obtain a diagram of the waterline, as shown in the attached image. Figure 4 As shown.

[0069] While the present invention has been disclosed above with reference to preferred embodiments, these embodiments are not intended to limit the invention. Any equivalent changes or modifications made without departing from the spirit and scope of the invention are also within the scope of protection of the invention. Therefore, the scope of protection of the present invention should be determined by the claims of this application.

Claims

1. A method for detecting an underwater antenna in an unmanned surface vessel (USV) sensing system, characterized by the following steps: S1) Establish a convolutional neural network structure for water antenna detection, wherein the convolutional neural network structure for water antenna detection includes: a backbone network for feature extraction and a branch prediction output head network. The backbone network used for feature extraction consists of 9 convolutional layers. The data input layer, Data, contains 16 images of size 192*512. Data input layers sequentially pass through CBR1, CBR2, CBR3, P1, CBR4, and CBR5. After CBR1, the feature map size is 32*96*256. CBR1 consists of 32 five-dimensional convolutions with a stride of 2, followed by batch normalization (BN) and ReLU activation. After CBR2, the feature map size is 64*48*128. CBR2 consists of 64 three-dimensional convolutions with a stride of 2, followed by BN and ReLU activation. After CBR3, the feature map size is 64*48*128. CBR3 consists of 64 three-dimensional convolutions with a stride of 1, followed by BN and ReLU activation. After P1, the feature map size is 64*24*64. P1 is a three-dimensional convolution with a stride of 2 and a maximum step size of 2. Pooling operations are performed; after CBR4, the feature map size is 64*24*64. CBR4 consists of 64 3*3 convolutions with a stride of 1, plus BN operations and ReLU activation. After CBR5, the feature map size is 128*12*32. CBR5 consists of 128 3*3 convolutions with a stride of 2, plus BN operations and ReLU activation. Then, two branches are formed: one branch performs Conv1, and the other branch performs P2, CBR6, and CBR7 operations sequentially. After Conv1, the feature map size is 128*12*32. Conv1 consists of 128 3*3 convolutions with a stride of 1. After P2, the feature map size is 128*6*16. P2 is a 3*3 Max convolution with a stride of 2. Pooling operation; after CBR6, the feature map size is 128*6*16. CBR6 consists of 128 3*3 convolutions with a stride of 1, Batch Normalization (BN), and ReLU activation. After CBR7, the feature map size is 128*12*32. CBR7 consists of 128 1*1 convolutions with a stride of 1, Batch Normalization (BN), and ReLU activation. Two computation branches are then formed: one branch performs the Conv2 operation, and the other branch performs P3, CBR8, and Deconv1 operations sequentially. Conv2 operation completes... The final feature map size is 128*6*16. Conv2 consists of 128 3*3 convolution operations with a stride of 1. After P3, the feature map size is 128*3*8. P3 is a 3*3 Maxpool operation with a stride of 2. After CBR8, the feature map size is 128*3*8. CBR8 consists of 128 3*3 convolution operations with a stride of 1, plus BN operations and ReLU activation. After Deconv1, the feature map size is 128*6*16. Deconv1 consists of 128 3*3 deconvolution operations.The feature map after the Deconv1 operation and the feature map after the Conv2 operation are subjected to an Eltwise1 operation. After the Eltwise1 operation, the feature map size is 256*6*16. Eltwise1 is a summation operation. After the Eltwise1 operation, the Deconv2 operation is performed. After the Deconv2 operation, the feature map size is 128*12*32. Deconv2 is 128 3*3 deconvolution operations. The feature map after the Deconv2 operation and the feature map after the Conv1 operation are subjected to an Eltwise2 operation. After the Eltwise2 operation, the feature map size is 256*12*32. Eltwise2 is a summation operation. After the Eltwise2 operation, the CBR9 operation is performed. After the CBR9 operation, the feature map size is 128*12*32. CBR9 is 128 3*3 convolution operations with a stride of 1 + BN operation + ReLU activation function. The CBR9 operation is then input into the branch prediction output head network. The branch prediction output head network specifically includes three branches: a keypoint heatmap prediction branch, a keypoint offset prediction branch, and a water surface angle prediction branch. The keypoint heatmap prediction branch operates on CBR10-1 and Conv3-1 methods. After CBR10-1, the feature map size is 64*12*32. CBR10-1 consists of 64 3*3 convolutions with a stride of 1, plus BN operations and a ReLU activation function. After Conv3-1, the feature map size is N*12*32. Conv3-1 consists of N 1*1 convolutions with a stride of 1, plus BN operations and a ReLU activation function, where N is the number of target categories (N=1 here). The loss function is the keypoint heatmap loss function. The keypoint offset prediction branch network operates on CBR10-2 and Conv3-2 methods. After CBR10-2, the feature map size is 64*12*32, consisting of 64 3*3 convolutions with a stride of 1, followed by batch normalization (BN) and ReLU activation. After Conv3-2, the feature map size is 2*12*32, consisting of two 1*1 convolutions with a stride of 1, followed by BN and ReLU activation. The keypoint offset prediction branch has two output channels, and the loss function is the offset loss from the predicted keypoint to the ground truth keypoint. The water antenna angle prediction branch network operates on CBR10-3 and Conv3-3. After CBR10-3, the feature map size is 64*12*32, consisting of 64 3*3 convolutions with a stride of 1, followed by Batch Normalization (BN) and ReLU activation. After Conv3-3, the feature map size is 1*12*32, consisting of one 1*1 convolution with a stride of 1, followed by BN and ReLU activation. The water antenna angle prediction branch has one output channel, and its output is the water antenna and the image. Angle in the positive direction of the axis The loss function is angle. loss ; S2) Train the water antenna detection convolutional neural network structure established in step S1 using the training dataset. The total loss of the loss function during training is given by the following formula (I): ..................（I）； in, , , In order , , Their respective weighting factors; S3) When training reaches the iteration number Q, all weight values ​​in the network will be... and bias value After fixing and saving the trained convolutional neural network model W, the image to be identified is input into the convolutional neural network model W during water antenna detection. To limit the values ​​in the feature map Conv3-1 to the range of (0, 1), a sigmoid operation is performed on the feature map Conv3-1 to obtain a heatmap, transforming the 1*12*32 three-dimensional matrix heatmap into a 12*32 matrix. , where the matrix Each value in the matrix represents the confidence level of the key point at that location; this transforms the 1*12*32 three-dimensional matrix feature map Conv3-2 into two 12*32 offset matrixes. and an offset matrix of size 12*32 Transform the 1*12*32 three-dimensional matrix feature map Conv3-3 into a 12*32 hydrofoil angle matrix. ; S4) Find the coordinates of the positions in matrix X with a confidence level greater than 0.

5. and the corresponding matrix ,matrix In and The pixel coordinates of the points on the horizon in the real image are obtained by using the following formula V. There are also corresponding The value, and the detailed calculation formula are as follows: ........（V）； All Store in the container allpoints; S5) Assume that... indivual ,if If the value is less than 5, it is considered that there is no water line in the image; if it is... If there are more than 5, they will all be in the allpoints container. according to Pixels along the axis are sorted from smallest to largest, where... The axis direction is from left to right for the unmanned surface vessel in the world coordinate system; and it applies to all points in the container allpoints. , Calculate the mean to get , For all points in the container allpoints Calculate the mean to get Thus, the slope of the waterline is obtained. ,Depend on , and Calculate the parameters of the linear equation Thus, the coordinate equation of the waterline image line is obtained. .

2. The method for detecting an underwater antenna in an unmanned surface vessel (USV) sensing system according to claim 1, characterized in that, In step S2 , , The calculation formulas are as follows: (II), (III), (IV): ......(II); Where N is the number of currently detected keypoints, and xyc represents the current category number c at position xy in the heatmap. To detect the predicted values ​​of key points, The values ​​are the actual values ​​at key points on the horizon, where α and β are hyperparameters. ......(III); in, Represents the coordinates of key points. For the corresponding key point deviation value, This is the downsampling factor. This represents the actual deviation caused by keypoint downsampling. ......(IV); Where k represents the key point being detected. It is the predicted angle information. This is the actual angle information of the waterline.

Images