Self-adaptive marine ship autonomous planning method

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a self-adaptive marine ship autonomous planning method, which can combine semantic segmentation and actual planning together, perform segmentation through an actually acquired image to determine an obstacle, obtain a preliminary optimal path from a starting point to an ending point through an Astar algorithm, and perform autonomous planning on a marine ship. And continuously inputting the acquired scene information into the semantic segmentation model for segmentation to finally obtain an optimal path arriving end point, so that the optimal path is ensured, the obstacle avoidance is ensured, the efficiency and the safety are improved, the self-adaptive autonomous planning is realized, and meanwhile, the self-adaptive autonomous planning is realized. And a richer data set and a semantic segmentation model with higher precision are obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and specifically to an adaptive autonomous planning method for marine vessels. Background Art

[0002] With the continuous development of the world economy and the increasing closeness of exchanges between countries, the status of maritime transportation has gradually risen and become a bridge for economic and trade exchanges between many countries. However, when a driver is navigating a vessel, problems such as deviation of the navigation route, inability to avoid maritime obstacles, and inability to select the optimal route are inevitable. This results in consequences such as extended navigation time and low safety factor.

[0003] With the continuous development of artificial intelligence technology, more and more high-performance neural network models have improved the accuracy of semantic segmentation tasks. During navigation, things in the image can be segmented through a semantic segmentation model, thereby achieving the function of distinguishing obstacles. Most of the existing autonomous planning technologies for marine vessels remain at the theoretical method level and are difficult to obtain obstacles and perform dynamic planning in practical applications.

[0004] Therefore, we propose an adaptive autonomous planning method for marine vessels to solve the above technical problems. Summary of the Invention

[0005] (1) Technical Problems to be Solved

[0006] Aiming at the deficiencies of the prior art, the present invention provides an adaptive autonomous planning method for marine vessels, which has the advantages of ensuring the optimal path while avoiding obstacles, improving efficiency and safety, and achieving adaptive autonomous planning, and solves the above technical problems.

[0007] (2) Technical Solutions

[0008] To achieve the purpose of ensuring the optimal path while avoiding obstacles, improving efficiency and safety, and achieving adaptive autonomous planning, the present invention provides the following technical solutions: An adaptive autonomous planning method for marine vessels, comprising the following steps:

[0009] Step 1: Prepare a marine channel scene data set and perform annotation for preliminary training of the model;

[0010] Step 2: Train a semantic segmentation model;

[0011] Step 3: Use a satellite navigation system to obtain the relative positions of the starting point and the ending point, and establish a grid map;

[0012] Step 4: Perform path planning using the Astar algorithm in the case of no obstacles to obtain an initial path;

[0013] Step 5: During navigation, use the equipped image acquisition device to collect images and input them into the pre-trained semantic segmentation model;

[0014] Step 6: Determine whether the segmentation is successful. If it is, mark the segmentation result as an obstacle in the grid map; if not, go to Step 7;

[0015] Step 7: If the segmentation fails, store the image in the image memory, annotate it, add it to the dataset, train the network model with the new dataset, and update the semantic segmentation model for image segmentation;

[0016] Step 8: When there are obstacles, re-use the Astar algorithm for path planning;

[0017] Step 9: Change the navigation direction and route, repeat the processes of collection, segmentation, planning, changing the course and route, and reach the end point with the safest and optimal path.

[0018] Preferably, the preparation of the maritime channel scene dataset and annotation in Step 1 includes the following steps:

[0019] A1. Obtain the image and video data of the maritime channel scene from the self-built maritime channel scene data, including islands, land, navigation aids, ships, and navigation marks;

[0020] A2. Preprocess the collected image or video data. The preprocessing includes image correction, denoising, brightness adjustment, and contrast adjustment;

[0021] A3. Divide the dataset into a training set, a validation set, and a test set;

[0022] A4. Annotate the image and video data. The annotation includes channel boundaries, port areas, navigation aid positions, and reef information;

[0023] A5. After annotation, perform rotation, translation, scaling, and flipping operations on the image to enhance the data;

[0024] A6. Save the annotated and enhanced data for subsequent model training.

[0025] Preferably, the self-built maritime channel scene data in Step A1 is a DMNet network model with an added ECA attention mechanism.

[0026] Preferably, Step 2 includes the following steps:

[0027] B1. Select a semantic segmentation model;

[0028] B2. Divide the annotated dataset into a training set and a test set according to a ratio of 7:3;

[0029] B3. Use a loss function to calculate the error between the prediction and the true label, and use the error to correct the model parameters;

[0030] B4. Configure network parameters, including the convolution kernel size, stride, and pooling layer size;

[0031] B5. After the parameter configuration is completed, train the semantic segmentation model.

[0032] Preferably, the third step includes the following steps:

[0033] C1. Use a satellite navigation system to receive satellite signals to obtain the longitude and latitude coordinates of the current location;

[0034] C2. Manually select the positions of the starting point and the ending point on the navigation path, either by manually inputting the longitude and latitude coordinates or by using a map tool to specify the positions;

[0035] C3. Based on the longitude and latitude coordinates of the starting point and the ending point, use the spherical triangle positioning method to obtain the relative position between the starting point and the ending point;

[0036] C4. Convert the obtained relative position of the starting point and the ending point into coordinates on a raster map;

[0037] C5. Connect the starting point and the ending point and draw a navigation path on the raster map;

[0038] C6. Convert the drawn navigation path into coordinates on a raster map by mapping the coordinates on the path to the corresponding raster squares;

[0039] C7. Integrate the path, obstacles, and roads to form the final raster map.

[0040] Preferably, the A* algorithm in the fourth step is carried out according to the following steps:

[0041] D1. Calculate the priority of each node through the following function:

[0042] f(n) = g(n) + h(n)

[0043] where f(n) is the comprehensive priority of node n. When traversing the planned path to find the next node, select the node with the highest comprehensive priority; g(n) is the cost of node n from the starting point; h(n) is the estimated cost of node n from the ending point;

[0044] D2. Put the unvisited nodes and the visited nodes into the open set and the closed set respectively, put the starting point into the open set, and set its priority to the highest;

[0045] D3. Detect and judge whether the open set is empty. If it is not empty, select the node n with the highest priority from this set;

[0046] D4. Judge whether the node n is the end point. If it is, start from this node and gradually trace its parent nodes until reaching the starting point. The returned path is the optimal path. If not, proceed to step D5;

[0047] D5. When n is not the end point, delete n from the open set and add it to the closed set, and then traverse all adjacent nodes of n;

[0048] D6. When one of the adjacent nodes m already exists in the closed set, skip m and select the next adjacent node. When the adjacent node m is not in the open set, set the parent node of m to n, calculate the priority of this node m, and add the node m to the open set;

[0049] D7. For g(n), take the g value of its parent node and add the cost between this node and its parent node on this basis;

[0050] D8. For the horizontal and vertical directions, directly add the cost. For the diagonal direction, approximate the cost by adding the square root of 2 times the cost;

[0051] D9. For h(n), use the Euclidean distance as the h(n) function, which is defined as follows:

[0052]

[0053] So far, the preliminary optimal path from the starting point to the end point is obtained.

[0054] Preferably, in step five, the image acquisition device dynamically obtains the scene information around the ship, and the image acquisition device is a 360° rotary acquisition device.

[0055] Preferably, in step six, when the segmentation fails, store this frame of image in the image memory, perform real-time annotation and add it to the data set, use the new data set to train the network model, and update the original network model to the newly trained model after training for adaptively updating the network model.

[0056] Preferably, in step six, when the segmentation is successful, mark the things in this image as obstacles, and use the lidar to determine their azimuth and mark it on the grid map.

[0057] Preferably, in step eight, directly add the node where the obstacle is located to the closed set, re-plan the path using the Astar algorithm, and adjust the heading.

[0058] Compared with the prior art, the present invention provides an adaptive autonomous planning method for maritime ships, which has the following beneficial effects:

[0059] The present invention obtains a preliminary optimal path from the starting point to the ending point through the Astar algorithm, continuously inputs the obtained scene information into the semantic segmentation model for segmentation, and finally obtains the optimal path to reach the ending point. While ensuring the optimal path, it also ensures avoiding obstacles, improves efficiency and safety, realizes adaptive autonomous planning. At the same time, a richer dataset and a semantic segmentation model with higher accuracy are also obtained. Brief Description of the Drawings

[0060] Figure 1 It is a schematic diagram of the process steps of the present invention. Detailed Embodiment

[0061] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0062] The present invention proposes an adaptive autonomous planning method for maritime ships, which can combine semantic segmentation and actual planning together, determine obstacles by segmenting actual collected images, and on this basis, perform a safer and better route planning, so as to realize the dynamic optimal route planning during the navigation process.

[0063] Please refer to Figure 1 , an adaptive autonomous planning method for maritime ships, including the following steps:

[0064] Step 1: Prepare a maritime channel scene dataset and perform annotation for preliminary training of the model;

[0065] Step 2: Train a semantic segmentation model;

[0066] Step 3: Use the satellite navigation system to obtain the relative positions of the starting point and the ending point, and establish a grid map;

[0067] Step 4: In the case of no obstacles, use the Astar algorithm for path planning to obtain an initial path;

[0068] Step 5: During the navigation process, use the equipped image acquisition device to collect images and input them into the pre-trained semantic segmentation model;

[0069] Step 6: Judge whether the segmentation is successful. If yes, mark the segmentation result as an obstacle in the grid map; if not, enter Step 7;

[0070] Step 7: If the segmentation fails, store the image in the image memory, annotate it, add it to the dataset, train the network model with the new dataset, and update the semantic segmentation model for image segmentation.

[0071] Step 8: Reuse the Astar algorithm for path planning in case of obstacles.

[0072] Step 9: Change the sailing direction and route, repeat the processes of acquisition, segmentation, planning, changing the heading and route, and reach the end point along the safest and optimal path.

[0073] Among them, the steps for preparing the offshore channel scene dataset and annotating it are as follows: First, obtain the image and video data of the offshore channel scene from the self-built offshore channel scene data, including islands, land, navigation aids, ships, and navigation marks, and perform preprocessing on the collected image or video data. The preprocessing includes image correction, denoising, brightness adjustment, and contrast adjustment. Then divide the dataset into a training set, a validation set, and a test set, and annotate the image and video data. The annotation includes channel boundaries, port areas, navigation aid positions, and reef information. After the annotation is completed, perform rotation, translation, scaling, and flipping operations on the image to enhance the data. Finally, save the annotated and enhanced data for subsequent model training.

[0074] When pre-training the semantic segmentation network model, the self-built offshore channel scene dataset is used, and a DMNet network model with an added ECA attention mechanism is applied. ResNet50 is used as the backbone network, and its unique DCM module and the added ECA attention mechanism are utilized to achieve the precise segmentation of the target image.

[0075] When training the semantic segmentation model, first select the semantic segmentation model, divide the annotated dataset into a training set and a test set according to a ratio of 7:3, then use the loss function to calculate the error between the prediction and the true label, and use the error to correct the model parameters. Then configure the network parameters, including the convolutional kernel size, stride, and pooling layer size. After the parameter configuration is completed, train the semantic segmentation model.

[0076] When establishing a grid map, first use the satellite navigation system to receive satellite signals to obtain the longitude and latitude coordinates of the current position. Manually select the positions of the starting point and the ending point on the navigation path, either by manually inputting the longitude and latitude coordinates or by using map tools to specify the positions. Based on the longitude and latitude coordinates of the starting point and the ending point, use the spherical triangle positioning method to obtain the relative position between the starting point and the ending point. Convert the obtained relative positions of the starting point and the ending point into coordinates on the grid map, connect the starting point and the ending point, and draw the navigation path on the grid map. Convert the drawn navigation path into coordinates on the grid map by mapping the coordinates on the path to the corresponding grid squares. Finally, fuse the path, obstacles, and roads to form the final grid map.

[0077] After determining the positions of the starting point and the ending point and establishing the grid map, the path planning algorithm Astar used is as follows:

[0078] The Astar algorithm has higher efficiency and lower computational complexity compared to the traditional Dijkstra algorithm. The Astar algorithm calculates the priority of each node through this function, and the expression is as follows:

[0079] f(n) = g(n) + h(n)

[0080] f(n) is the comprehensive priority of node n. When traversing the planned path to find the next node, the node with the highest comprehensive priority is always selected;

[0081] g(n) is the cost of node n from the starting point;

[0082] h(n) is the estimated cost of node n from the ending point;

[0083] The Astar algorithm will put the unvisited nodes and the visited nodes into the open set and the closed set respectively, put the starting point into the open set, and set its priority to the highest.

[0084] When executing the algorithm, Astar will first detect whether the open set is empty. If it is not empty, select the node n with the highest priority from this set. If n is the ending point, then gradually trace its parent nodes from this node until reaching the starting point, and the returned path is the optimal path; if n is not the ending point, then delete n from the open set and add it to the closed set, and then traverse all the adjacent nodes of n. If the adjacent node m already exists in the closed set, skip m and select the next adjacent node. If the adjacent node m is not in the open set, then set the parent node of m to n, calculate the priority of this node m, and add the node m to the open set.

[0085] For g(n), that is, take the g value of its parent node, and on this basis, add the cost between this node and its parent node. For the horizontal and vertical directions, directly add the cost. For the diagonal direction, the cost needs to be approximated by adding the square root of 2 times the cost.

[0086] For h(n), which is called the heuristic function of the A* algorithm. Since the ship can move in any direction, the Euclidean distance is used as the h(n) function in the present invention, and the definition is as follows:

[0087]

[0088] Thus, the preliminary optimal path from the starting point to the ending point can be obtained by using the A* algorithm.

[0089] When the preliminary optimal path is obtained for navigation, the image acquisition device will dynamically obtain the scene information around the ship. The image acquisition device mentioned in the present invention should be a 360° rotating acquisition device. After obtaining the scene information, it is input into the semantic segmentation model for segmentation, and the obtained results are divided into the following two cases:

[0090] 1. If the segmentation fails, store the frame image in the image memory, and perform real-time annotation and add it to the dataset. Use the new dataset to train the network model. After the training is completed, update the original network model to the newly trained model to achieve adaptive update of the network model.

[0091] 2. If the segmentation is successful, mark the things in the image as obstacles, use the lidar to determine their azimuth and mark it on the grid map, directly add the nodes where the obstacles are located to the close_set, use the A* algorithm to re-plan the path, and adjust the heading.

[0092] After repeating this process, finally, it will reach the ending point along the optimal path, ensuring the optimal path while avoiding obstacles, improving efficiency and safety, and achieving adaptive autonomous planning. At the same time, a richer dataset and a more accurate semantic segmentation model are also obtained.

Claims

1. An adaptive autonomous planning method for marine vessels, characterized in that, It includes the following steps: Step 1: Prepare a maritime channel scene dataset and annotate it for initial training of the model; Step 2: Train a semantic segmentation model; Step 3: Use the satellite navigation system to obtain the relative positions of the starting point and the ending point, and establish a grid map; Step 4: Use the Astar algorithm for path planning under obstacle-free conditions to obtain an initial path; Step 5: During navigation, use the equipped image acquisition device to collect images and input them into the pre-trained semantic segmentation model; Step 6: Judge whether the segmentation is successful. If so, mark the segmentation result as an obstacle in the grid map; If not, go to Step 7; Step 7: If the segmentation fails, store the image in the image memory, annotate it, add it to the dataset, train the network model with the new dataset, and update the semantic segmentation model for image segmentation; Step 8: Reuse the Astar algorithm for path planning in the presence of obstacles; Step 9: Change the navigation direction and route, repeat image acquisition, segmentation, path planning, and changing the course and route to reach the ending point along the safest and optimal path.

2. An adaptive autonomous planning method for marine vessels according to claim 1, wherein, The preparation of the maritime channel scene dataset and annotation in Step 1 includes the following steps: A1: Obtain the image and video data of the maritime channel scene from the self-built maritime channel scene data, including islands, land, navigation marks, ships, and navigation signs; A2: Preprocess the collected image or video data. The preprocessing includes image correction, denoising, brightness adjustment, and contrast adjustment; A3: Divide the dataset into a training set, a validation set, and a test set; A4: Annotate the image and video data. The annotation includes the channel boundary, port area, navigation mark position, and reef information; A5: After the annotation is completed, perform rotation, translation, scaling, and flipping operations on the image to enhance the data; A6: Save the annotated and enhanced data for subsequent model training.

3. An adaptive autonomous planning method for marine vessels according to claim 2, characterized in that, The self-built maritime channel scene data in Step A1 is a DMNet network model with an added ECA attention mechanism.

4. An adaptive autonomous planning method for offshore ships according to claim 3, characterized in that Step 2 includes the following steps: B1: Select a semantic segmentation model; B2: Divide the annotated dataset into a training set and a test set in a ratio of 7:3; B3: Use a loss function to calculate the error between the prediction and the true label, and use the error to correct the model parameters; B4: Configure the network parameters, including the convolutional kernel size, stride, and pooling layer size; B5: After the parameter configuration is completed, train the semantic segmentation model.

5. An adaptive autonomous planning method for marine vessels according to claim 4, wherein, Step 3 includes the following steps: C1: Use the satellite navigation system to receive satellite signals to obtain the longitude and latitude coordinates of the current position; C2: Manually select the positions of the starting point and the ending point on the navigation path by manually inputting the longitude and latitude coordinates or using a map tool to specify the position; C3: Based on the longitude and latitude coordinates of the starting point and the ending point, use the spherical triangle positioning method to obtain the relative positions of the starting point and the ending point; C4: Convert the obtained relative positions of the starting point and the ending point into coordinates on the grid map; C5: Connect the starting point and the ending point and draw a navigation path on the grid map; C6. Convert the drawn navigation path into coordinates on the grid map by mapping the coordinates on the path to the corresponding grid squares; C7. Integrate the path, obstacles, and roads to form the final grid map.

6. The self - adaptive autonomous planning method for marine vessels according to claim 5, wherein In step 4, the A* algorithm proceeds as follows: D1. Calculate the priority of each node through the following function: f(n) = g(n) + h(n) where f(n) is the comprehensive priority of node n. When planning the path to traverse and find the next node, select the node with the highest comprehensive priority; g(n) is the cost of node n from the starting point; h(n) is the estimated cost of node n to the ending point; D2. Put the unvisited nodes and the already visited nodes into the open set and the closed set respectively, put the starting point into the open set, and set its priority to the highest; D3. Detect and judge whether the open set is empty. If it is not empty, select the node n with the highest priority from this set; D4. Judge whether node n is the ending point. If it is, then trace its parent nodes step by step from this node until reaching the starting point. The returned path is the optimal path. If not, enter step D5; D5. When n is not the ending point, then delete n from the open set and add it to the closed set, and then traverse all the adjacent nodes of n; D6. When one of the adjacent nodes m already exists in the closed set, skip m and select the next adjacent node. When the adjacent node m is not in the open set, set the parent node of m to n, calculate the priority of this node m, and add node m to the open set; D7. For g(n), take the g value of its parent node and add the cost between this node and its parent node on this basis; D8. For the horizontal and vertical directions, directly add the cost. For the diagonal direction, add the square root of 2 times the cost to approximate the cost; D9. For h(n), use the Euclidean distance as the h(n) function, defined as follows: So far, the preliminary optimal path from the starting point to the ending point is obtained.

7. An adaptive autonomous planning method for offshore ships according to claim 6, characterized in that: In step 5, the image acquisition device dynamically obtains the scene information around the ship, and the image acquisition device is a 360° rotary acquisition device.

8. An adaptive autonomous planning method for marine ships according to claim 7, characterized in that: In step 6, when the segmentation fails, then store this frame of image in the image memory, perform real-time annotation and add it to the dataset, use the new dataset to train the network model, and update the original network model to the newly trained model after training for adaptive update of the network model.

9. An adaptive autonomous planning method for offshore ships according to claim 8, characterized in that: In step 6, when the segmentation is successful, then mark the things in this image as obstacles, determine their orientations using lidar and mark them on the grid map.

10. An adaptive autonomous planning method for marine vessels according to claim 9, characterized in that: In step 8, directly add the nodes where the obstacles are located to the closed set, re-plan the path using the A* algorithm, and adjust the heading.

Images