A method for sea ice recognition and ice area navigation path generation based on improved YOLOv5

Abstract

This paper constructs a remote sensing sea ice dataset and, based on the characteristics of sea ice images, improves the YOLOv5 model by adding the SE attention mechanism, improving the SPP pooling pyramid structure, and replacing the SiLU activation function with the FReLU. The model's confidence is verified using evaluation metrics such as accuracy, recall, F1-score, and mean average precision, ultimately resulting in an optimal sea ice target detection model. The test set is input into the target detection model to obtain recognition results, extracting information such as the location and size of the sea ice. Based on this extracted information, a grid map is created. The starting and ending points are determined based on the ship's own position, and a path is planned using the Theta* algorithm. The generated path is evaluated using metrics such as route distance, average offset distance, number of ship turns, and number of sea ice avoidances. This paper implements sea ice recognition at a remote sensing scale, constructs simulated navigation scenarios, and plans the optimal navigation path. This paper can improve the safety of ships sailing in ice areas and provide technical support for collision avoidance and navigation decision-making assistance in ice areas.

Description

Technical Field

[0001] The present invention relates to the field of sea ice parameter observation and ice zone navigation path planning in ship and ocean engineering, and in particular to a sea ice recognition and ice zone navigation path generation method based on improved YOLOv5. Background Art

[0002] Sea ice exists year-round in polar waters, and its extent fluctuates rapidly. Every winter, sea ice covers most of the polar waters, significantly increasing in both area and thickness. Ships navigating in ice-covered areas face numerous risks. When ships enter areas with ice or icebergs, factors such as swells, fog, and ice floes can affect navigation safety. Collisions between ships and sea ice can cause damage to the hull and equipment, including scratches and damage to rudders and propellers. When navigating in ice, identifying and acquiring sea ice parameter information is crucial, enabling the generation of real-time navigation paths and minimizing collisions between ships and ice, thereby improving safety.

[0003] Remote sensing sea ice imagery is a crucial tool for studying and monitoring sea ice. Remote sensing technology provides high-resolution imagery, real-time sea ice monitoring, and data on a wide range of areas. It's unrestricted by geographic location and can provide all-weather monitoring. Path planning can help ships avoid obstacles, reduce the risk of collisions with sea ice, and improve navigation safety. Based on acquired sea ice information, efficient and safe navigation routes can be planned, saving fuel costs and time, and improving navigation efficiency.

[0004] How to enable computers to quickly and accurately identify sea ice and plan routes based on real-time information is a key research challenge for many researchers. Numerous studies have been conducted both domestically and internationally on sea ice detection and route planning. These studies primarily focus on accurately identifying sea ice targets in images, extracting their precise locations, and planning safe and efficient routes. Key methods for extracting sea ice targets include threshold segmentation, object detection, and instance segmentation. Threshold segmentation converts an image into a binary image based on its grayscale values. Using the different grayscale values ​​of seawater and sea ice, a grayscale threshold is set to separate ice from water. Object detection utilizes a network framework based on object detection, employing convolutional neural networks. Through model training, the model is trained to identify sea ice and locate its location within the image. Path planning considers factors such as sea ice distribution and vessel characteristics, employing algorithms such as ant colonies and A* to generate safe and efficient routes, conduct risk assessments, and provide decision-making support.

[0005] In summary, the problems existing in the prior art are:

[0006] (1) Currently, sea ice recognition requires a large number of samples for model training. However, there are relatively few high-quality sea ice datasets available for model training. As a result, the model is only effective in recognizing sea ice of a specific shape. The training may be overfitted, and all sea ice in the image may not be recognized, resulting in a low recognition accuracy.

[0007] (2) Remote sensing images are large in size, while sea ice targets are small and often clustered together, making recognition difficult. Direct training on large-scale remote sensing images results in extended training time and slow inference time, which cannot meet the real-time requirements of ships. It is necessary to optimize the target recognition algorithm based on the characteristics of remote sensing sea ice images to enable more accurate and efficient recognition of sea ice.

[0008] (3) Currently, many studies focus on only one aspect, either target recognition or path planning. It is necessary to comprehensively utilize sea ice recognition data and ship path planning technology, and combine the methods of the two fields to improve the safety and efficiency of ice navigation. In domestic and foreign navigation technology, there is still a lack of a set of methods for generating ice obstacle avoidance paths for ice navigation based on deep learning. At present, no descriptions or reports of technologies similar to the present invention have been found, and similar materials at home and abroad have not yet been collected. Summary of the Invention

[0009] In response to the existing problems, the present invention provides a method for sea ice recognition and ice area navigation path generation based on improved YOLOv5.

[0010] The technical solutions of the present invention are as follows:

[0011] A multi-scale sea ice recognition method based on computer vision is characterized by comprising the following steps:

[0012] Step 1: Construct sea ice remote sensing dataset:

[0013] Step 1.1: Collect remote sensing sea ice images and preprocess them, including image normalization and data enhancement

[0014] Step 1.2: Use labelimg software to annotate the image and generate a corresponding XML annotation file. The XML annotation file includes the annotation box, the location information of the annotation box, and the actual category information of the content annotated by the annotation box;

[0015] Step 1.3: Convert the XML annotation file into a txt file that can be used for YOLOv5 model training, and divide it into training set and test set in a ratio of 8:2;

[0016] Step 2: Improve the YOLOv5 target detection model:

[0017] Step 2.1: Add SE attention mechanism to the YOLOv5 network backbone to capture the characteristics of sea ice images;

[0018] Step 2.2: Improve the SPP spatial pyramid pooling structure to process sea ice images of different scales and sizes while preserving the location information;

[0019] Step 2.3: Use FReLU activation function and train through backpropagation;

[0020] Step 3: Input the test set of the remote sensing sea ice dataset into the improved YOLOv5 for training to obtain the weight file and target recognition model;

[0021] Step 4: Input the weight file and sea ice image into the target recognition model, extract important sea ice information, and construct an ice area navigation scenario;

[0022] Step 5: Generate ice navigation path, including:

[0023] Step 5.1 Determine the starting point and end point based on the ship's position and the target location;

[0024] Step 5.2: Consider the sea ice area as an obstruction and quickly generate a path from the starting point to the end point;

[0025] Step 5.3: Evaluate the generated path based on evaluation indicators, where the evaluation indicators include at least one of the following: route distance D(L), average offset distance H(j), number of ship turns, and number of sea ice avoidances;

[0026]

[0027]

[0028] f n is the center point of the grid crossed by the nth navigation path,

[0029] d(f n ,f n+1 ) is f n to f n+1 The straight-line distance, l(f n ) point f n Distance to the connecting line of the starting point and the end point, N G The ratio of the distance between the starting point and the ending point of the flight segment to l is rounded to an integer;

[0030] Step 5.4: Select a path. Compare all generated path plans and select the path with shorter path length, smaller average offset distance, fewer ship turns, and fewer sea ice avoidance times as the recommended path.

[0031] Furthermore, step 2.1: adding the SE attention mechanism to the YOLOv5 network backbone to capture the features of the sea ice image. The SE attention mechanism is used to improve the target detection network structure to analyze the weights of features such as color, shape, and texture, and enhance the ability to extract features and suppress irrelevant information. Specifically:

[0032] Step 2.1.1 compresses the input feature map through the compression operation, then generates weights for each feature channel through the excitation operation, and finally performs the recalibration operation to normalize the weights and multiply them;

[0033] In step 2.1.2, the SE attention mechanism is embedded into the first layer of the spatial pyramid pooling structure (SPP) in Backbone. After further mining effective features, three maximum pooling and high-level feature extraction fusion operations are performed to improve the model's feature extraction capability for remote sensing sea ice images.

[0034] Furthermore, the step 2.2: improving the SPP spatial pyramid pooling structure is specifically:

[0035] The improved SPP spatial pyramid pooling structure adds 5 Conv layers, adds 1 feature input channel, changes the order of maximum pooling, and increases the depth of the pyramid pooling structure; the feature map is input into the improved SPP module, one path performs convolution operations to extract lower-level features, while the other path retains the original features. The fused features are further processed by subsequent convolution layers; while keeping the receptive field unchanged, the detection accuracy is improved.

[0036] Furthermore, the step three of inputting the test set of the remote sensing sea ice dataset into the improved YOLOv5 for training to obtain the target model specifically includes:

[0037] Step 3.1: Input the remote sensing sea ice training set into the improved YOLOv5 target detection model for network training. Save the model parameters with the highest accuracy on the test set during the training process and name the file as best.pt.

[0038] Step 3.2: Input the remote sensing sea ice test set into the improved YOLOv5 target detection model, and load the optimal weight file best.pt into the model to be detected for inference detection;

[0039] Step 3.3: Evaluate the model based on evaluation metrics, including at least one of the following: accuracy, recall, average precision (AP), F1-score, and mean average precision (MAP):

[0040]

[0041]

[0042]

[0043]

[0044]

[0045] Where TP is the number of correctly classified positive samples, FP is the number of incorrectly classified positive samples, TN is the number of correctly classified negative samples, FN is the number of incorrectly classified negative samples, and k is the number of target categories.

[0046] Furthermore, the step 4: inputting the weight file and the sea ice image into the target recognition model, extracting important sea ice information, and constructing an ice area navigation scenario specifically includes:

[0047] Step 4.1: Input the remote sensing sea ice image corresponding to the path planning scenario into the target recognition model for recognition and obtain the recognition result;

[0048] Step 4.2: Based on the identification results, extract important sea ice information, including sea ice location and size;

[0049] Step 4.3: Based on the extracted sea ice information, the navigation scenario is represented by a regional matrix and the corresponding grid map is constructed.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] This paper constructs a remote sensing sea ice dataset and, based on the characteristics of sea ice images, specifically improves the YOLOv5 model. This includes three improvements: adding a SE attention mechanism, improving the SPP pooling pyramid structure, and replacing the SiLU activation function with a FReLU activation function. A deep learning approach is used to train a neural network to generate a sea ice image recognition model. The trained model is validated for confidence using metrics such as accuracy, recall, F1-score, and mean average precision, ultimately yielding an optimal sea ice target detection model. The test dataset is input into the target detection model to obtain recognition results, extracting information such as the location and size of sea ice. Based on this extracted information, a grid map is created, and sea ice areas are designated as impassable areas. The starting and ending points are determined based on the vessel's own position. A path is planned using the Theta* algorithm, and the generated path is evaluated using metrics such as route distance, average offset distance, number of ship turns, and number of sea ice avoidances. All path plans are compared to select the optimal path. This paper achieves sea ice recognition at a remote sensing scale, extracts key sea ice information, constructs a simulated navigation scenario, and plans the optimal navigation path. The present invention can improve the safety of ships sailing in ice areas and provide technical support for collision avoidance and navigation auxiliary decision-making of ships in ice areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0053] Figure 1 It is the algorithm flow chart of the present invention;

[0054] Figure 2 Schematic diagram of data enhancement operation in the embodiment;

[0055] Figure 3 This is a schematic diagram of the network structure of the improved YOLOv5 in the embodiment;

[0056] Figure 4 Schematic diagram of the structure of the SE attention mechanism in the embodiment;

[0057] Figure 5 Schematic diagram of the improved SPP spatial pyramid pooling structure in the embodiment;

[0058] Figure 6 Schematic diagram of the FReLU activation function structure in the embodiment;

[0059] Figure 7 This is a schematic diagram of the recognition results of remote sensing sea ice by the improved YOLOv5 model in the embodiment;

[0060] Figure 8 A schematic diagram of ice area scene construction in the embodiment;

[0061] Figure 9 Schematic diagram of the path planned by the Theta* algorithm in the embodiment. DETAILED DESCRIPTION

[0062] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process. It should be noted that those skilled in the art may make various modifications and improvements without departing from the scope of the present invention, and these modifications and improvements fall within the scope of protection of the present invention.

[0063] This embodiment provides a method for sea ice recognition and ice navigation path generation based on improved YOLOv5, which is used to solve the problems of the existing technology. The present invention is described in detail below with reference to the accompanying drawings.

[0064] See also Figure 1 , Figure 1 The figure is a flow chart of a method for sea ice recognition and ice area navigation path generation based on improved YOLOv5 in the present invention.

[0065] The present invention provides a method for sea ice recognition and ice area navigation path generation based on improved YOLOv5, comprising steps S1 to S5.

[0066] Step S1: Collect sea ice remote sensing images and construct a dataset.

[0067] When constructing the dataset, the image should contain as few other objects as possible. For images without sea ice feature annotations, you can use image annotation tools to manually annotate the sea ice images.

[0068] In one achievable manner, the collecting of sea ice images and constructing of a data set includes:

[0069] The collected images were subjected to data enhancement processing, and manual image annotation was performed using LabelImg software. The annotated data set was divided into a training set and a test set.

[0070] Due to the extreme environment of sea ice, there are few high-quality sea ice image samples available for training. In this case, data enhancement technology can solve the problem of insufficient sea ice image data. Currently, effective data enhancement technology can be achieved through geometric transformations such as translation, shearing, rotation, and scaling. It can also be achieved by adding filters, adjusting image brightness, clarity, and sharpening pixel transformation methods. In the process of data processing, in order to retain more effective original information, the number of image processing times should be minimized. Examples of data enhancement are as follows: Figure 2 The data enhancement process includes at least one of the following:

[0071] Mosaic data enhancement, perspective transformation, left-right transformation, and rotation processing.

[0072] Rotation processing refers to the process of rotating a point on an image by a certain angle to produce a new image. The center of the image is typically used as the origin of rotation, meaning that the relative positions of all points in the image to the center remain consistent before and after rotation. In this embodiment, the rotation angles for sea ice images are set to 45°, 90°, 135°, and 180°.

[0073] Perspective transformation is the process of projecting an image from one viewing plane to another. Left-right transformation is the process of flipping the image without changing its size, position, or angle.

[0074] The main idea of ​​Mosaic data augmentation is to randomly crop four images and then stitch them together into a single image as training data. This has the advantage of enriching the background of the image and also indirectly increasing the batch_size by stitching the four images together.

[0075] When manually annotating sea ice images, we should try our best to ensure that every piece of sea ice in the image is labeled, and the resolution during annotation should be as high as possible. The quality of sea ice feature annotation will directly affect the quality of model training.

[0076] When dividing the data set, it is divided into 80% training set and 20% test set.

[0077] Step S2: Improve the YOLOv5 target detection model.

[0078] In one possible way, the YOLOv5 target detection model is improved, and the improved structure is as follows Figure 3 As shown in the figure, YOLOv5 can be divided into four modules in terms of structure: input end, backbone network Backbone, Neck and prediction end Prediction. Improvement methods include:

[0079] Add SE attention mechanism. Add SE attention mechanism to the YOLOv5 backbone network backbone. The structure of the attention mechanism is as follows Figure 4 As shown in the figure, the SE attention mechanism is a method used to enhance the performance of deep neural networks. It involves two main operations: squeeze and excitation. The SE attention mechanism considers the relationship between feature channels, automatically determines the importance of each feature channel, and reweights the feature map based on these channel weights. This enables the network to better capture the characteristics of sea ice images, improving model performance and generalization capabilities, and enhancing network performance without adding excessive additional computational burden.

[0080] Improve the SPP spatial pyramid pooling structure, such as Figure 5 As shown in Figure 2, the improved SPP uses pooling windows of different sizes to sample features at different levels, reducing information loss, improving detection accuracy, and increasing the speed of sea ice detection while maintaining the same receptive field. Furthermore, it enables the network to better handle objects of different scales, improving detection generalization capabilities.

[0081] Use FReLU activation function, the structure of the activation function is as follows Figure 6 As shown in Figure 2, the original SiLU activation function is computationally complex and can cause vanishing gradients when the input is large. The FReLU activation function can adaptively adjust its parameters to better capture sea ice characteristics, reduce information loss, and improve the network's representational capabilities. FReLU parameters can be trained through backpropagation, allowing the network to adapt to different data distributions and task requirements, reducing information loss and improving the network's representational capabilities.

[0082] In step S3, the sea ice dataset is input into the improved YOLOv5 for training to obtain the target model, and the model is verified.

[0083] The remote sensing sea ice dataset is fed into the improved YOLOv5 for network training. YOLOv5 first adaptively scales the sea ice image and then divides it into S×S grids. If the center of the target to be detected is within a grid, the grid is used to predict the target's category. The specific training parameters are shown in Table 1:

[0084] Table 1 Training parameters

[0085]

[0086] Save the model parameters of the improved YOLOv5 model with the highest accuracy during training and name it best.pt. Input the sea ice test set into the improved YOLOv5 model and load the optimal weight file best.pt into the model to be tested for inference. Evaluate the model using at least one of the following metrics: accuracy, recall, average precision (AP), F1 score, and mean average precision (MAP).

[0087]

[0088]

[0089]

[0090]

[0091]

[0092] Where TP is the number of correctly classified positive samples, FP is the number of incorrectly classified positive samples, TN is the number of correctly classified negative samples, and FN is the number of incorrectly classified negative samples. K is the number of target categories.

[0093] Step S4: constructing an ice navigation scenario.

[0094] S3 inputs the weight file and sea ice image into the improved YOLOv5 target recognition model to obtain the recognition results of the model in the test set. The recognition results of the YOLOv5 target detection model for remote sensing sea ice are as follows: Figure 7 shown.

[0095] Based on the recognition results, important sea ice information is extracted from the prediction frame, including the location and size of the sea ice, and the coordinate data of the four vertices on the x-axis and y-axis are read. The prediction frame represents the obstacle area of ​​the navigation scene. The navigation scene is represented by a region matrix, with the value of the obstacle area set to 1 and the value of other areas set to 0. A grid map is constructed to simulate the ice navigation scene. The ice scene construction process is as follows: Figure 8 shown.

[0096] Step S5: Generate ice navigation route

[0097] Determine the starting and ending points based on the vessel's position and navigational conditions. Use the Theta* algorithm to generate a path, treating ice areas as obstacles and rapidly generating a path from the starting point to the end point. Due to the winding ice channels, to minimize vessel maneuvering and ensure smooth paths, at least one of the following evaluation metrics is incorporated: route distance, average offset distance, number of turns, and number of ice avoidances.

[0098]

[0099]

[0100] In formula (6), D(L) is the route distance, and the grid center point crossed by the nth navigation path is f n ,d(f n ,f n+1 ) is f n to f n+1 In formula (7), l(f n ) is point f n Distance to the connecting line of the starting point and the end point, N G The ratio of the distance from the starting point to the ending point of the flight segment to l is rounded to an integer.

[0101] Select a route, compare all generated route plans, and select the one with shorter route distance, smaller average offset distance, fewer ship turns, and fewer sea ice avoidance times as the recommended route. The generated ice channel is as follows: Figure 9 shown.

[0102] In summary, the results show that the proposed solution can effectively identify small sea ice targets that are difficult to detect and distinguish in remote sensing sea ice imagery. By combining target recognition and path planning, and using the target recognition output as the path planning input, a sea ice recognition and ice navigation path generation method based on an improved YOLOv5 is proposed. This method identifies and locates sea ice, extracts important sea ice information, constructs a simulated navigation scenario, and uses the Theta* algorithm for path planning to evaluate the optimal navigation path. This method improves the safety of ships sailing in ice areas and provides technical support for collision avoidance and navigation decision-making in ice areas.

[0103] Matters not mentioned in the above embodiments of the present invention are well known in the art.

[0104] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A multi-scale sea ice recognition method based on computer vision, characterized in that: The steps include: Step 1: Construct sea ice remote sensing dataset: Step 1.1: Collect remote sensing sea ice images and preprocess them, including image normalization and data enhancement Step 1.2: Use labelimg software to annotate the image and generate a corresponding XML annotation file. The XML annotation file includes the annotation box, the location information of the annotation box, and the actual category information of the content annotated by the annotation box; Step 1.3: Convert the XML annotation file into a txt file that can be used for YOLOv5 model training, and divide it into training set and test set in a ratio of 8:2; Step 2: Improve the YOLOv5 target detection model: Step 2.1: Add SE attention mechanism to the YOLOv5 network backbone to capture the characteristics of sea ice images; Step 2.2: Improve the SPP spatial pyramid pooling structure to process sea ice images of different scales and sizes while preserving the location information; Step 2.3: Use FReLU activation function and train through backpropagation; Step 3: Input the test set of the remote sensing sea ice dataset into the improved YOLOv5 for training to obtain the weight file and target recognition model; Step 4: Input the weight file and sea ice image into the target recognition model, extract important sea ice information, and construct an ice area navigation scenario; Step 5: Generate ice navigation path, including: Step 5.1 Determine the starting point and end point based on the ship's position and the target location; Step 5.2: Consider the sea ice area as an obstruction and quickly generate a path from the starting point to the end point; Step 5.3: Evaluate the generated path based on evaluation indicators, where the evaluation indicators include at least one of the following: route distance D(L), average offset distance H(j), number of ship turns, and number of sea ice avoidances; f n is the center point of the grid crossed by the nth navigation path, d(f n ,f n+1 ) is f n to f n+1 The straight-line distance, l(f n ) point f n Distance to the connecting line of the starting point and the end point, N G The ratio of the distance between the starting point and the ending point of the flight segment to l is rounded to an integer; Step 5.4: Select a path. Compare all generated path plans and select the path with shorter path length, smaller average offset distance, fewer ship turns, and fewer sea ice avoidance times as the recommended path.

2. The multi-scale sea ice recognition method based on computer vision according to claim 1, characterized in that: Step 2.1: Add the SE attention mechanism to the YOLOv5 network backbone to capture the characteristics of sea ice images, specifically: The attention mechanism described in step 2.1.1 is used to improve the object detection network structure to analyze the weights of features such as color, shape, and texture, and enhance the ability to extract features and suppress irrelevant information; First, the input feature map is compressed through the compression operation, then the excitation operation generates weights for each feature channel, and finally the recalibration operation is performed to normalize the weights and multiply them; In step 2.1.2, the SE attention mechanism is embedded into the first layer of the spatial pyramid pooling structure (SPP) in Backbone. After further mining effective features, three maximum pooling and high-level feature extraction fusion operations are performed to improve the model's feature extraction capability for remote sensing sea ice images.

3. The multi-scale sea ice recognition method based on computer vision according to claim 1, characterized in that: Step 2.2: Improve the SPP spatial pyramid pooling structure, specifically: The improved SPP spatial pyramid pooling structure adds 5 Conv layers, adds 1 feature input channel, changes the order of maximum pooling, and increases the depth of the pyramid pooling structure; the feature map is input into the improved SPP module, one path performs convolution operations to extract lower-level features, while the other path retains the original features. The fused features are further processed by subsequent convolution layers; while keeping the receptive field unchanged, the detection accuracy is improved.

4. The multi-scale sea ice recognition method based on computer vision according to claim 1, characterized in that: Step 3: Input the test set of the remote sensing sea ice dataset into the improved YOLOv5 for training to obtain the target model, specifically including: Step 3.1: Input the remote sensing sea ice training set into the improved YOLOv5 target detection model for network training. Save the model parameters with the highest accuracy on the test set during the training process and name the file as best.pt. Step 3.2: Input the remote sensing sea ice test set into the improved YOLOv5 target detection model, and load the optimal weight file best.pt into the model to be detected for inference detection; Step 3.3: Evaluate the model based on evaluation metrics, including at least one of the following: accuracy, recall, average precision (AP), F1-score, and mean average precision (MAP): Where TP is the number of correctly classified positive samples, FP is the number of incorrectly classified positive samples, TN is the number of correctly classified negative samples, FN is the number of incorrectly classified negative samples, and k is the number of target categories.

5. The multi-scale sea ice recognition method based on computer vision according to claim 1, characterized in that: The fourth step is to input the weight file and the sea ice image into the target recognition model, extract important sea ice information, and construct an ice navigation scenario, which specifically includes: Step 4.1: Input the remote sensing sea ice image corresponding to the path planning scenario into the target recognition model for recognition and obtain the recognition result; Step 4.2: Based on the identification results, extract important sea ice information, including sea ice location and size; Step 4.3: Based on the extracted sea ice information, the navigation scenario is represented by a regional matrix and the corresponding grid map is constructed.

Images