Seabed obstacle identification method and system based on deep learning

Abstract

The invention discloses a seabed obstacle identification method and system based on deep learning, and relates to the technical field of obstacle identification. The method comprises the following specific steps: acquiring underwater sonar image data by using sonar equipment; performing image segmentation on the underwater sonar image data, and separating an echo region, a shadow region and a seabed reverberation region to obtain a binary image; preprocessing the binary image, and constructing a standard data set for model training; building an improved Yov5 model, setting a loss function, and training the improved Yov5 model by using the standard data set to obtain a seabed obstacle recognition model; and inputting a to-be-detected image into the seabed obstacle recognition model to obtain a recognition result. According to the invention, through training the improved Yolov5 model, seabed obstacle identification is carried out based on deep learning, and seabed obstacle identification can be accurately carried out in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of obstacle recognition, and in particular to a method and system for recognizing submarine obstacles based on deep learning. Background Art

[0002] Obstacle recognition is the process of identifying obstacles in the environment through sensor and algorithm detection. It is usually used in fields such as autonomous driving and robot navigation to help the system avoid obstacles and ensure safe and effective operation. Submarine obstacle recognition is the application of obstacle recognition technology in the marine environment. This recognition usually relies on sensors such as sonar, radar, and cameras to help ships, unmanned submersibles and other equipment avoid underwater obstacles to ensure safe navigation or operation. However, due to the limitations of the special underwater imaging environment, underwater images often have many problems such as high noise interference, blurred texture features, and low contrast. The turbidity of the water and floating matter will affect visibility and increase the difficulty of obstacle detection. In addition, natural environmental factors such as water flow, waves, and sediments may also change the shape or position of obstacles, resulting in deviations in the identification and positioning of obstacles. Therefore, how to accurately, quickly, and stably detect and identify submarine obstacles in the case of poor image visibility is an urgent problem to be solved by technicians in this field. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for identifying submarine obstacles based on deep learning to solve the problems raised in the background technology.

[0004] To achieve the above object, the present invention provides the following solutions: On the one hand, a method for identifying submarine obstacles based on deep learning is provided, and the specific steps include the following:

[0005] Collect underwater sonar image data using sonar equipment;

[0006] Performing image segmentation on the underwater sonar image data to separate the echo area, the shadow area and the seabed reverberation area to obtain a binary image;

[0007] Preprocessing the binary image to construct a standard data set for model training;

[0008] An improved Yolov5 model is built, a loss function is set, and the improved Yolov5 model is trained using the standard data set to obtain a seabed obstacle recognition model;

[0009] The image to be detected is input into the seabed obstacle recognition model to obtain a recognition result.

[0010] Preferably, the underwater sonar image data is segmented using an improved Otsu segmentation algorithm.

[0011] Preferably, the specific steps of preprocessing the binary image are:

[0012] Performing NSCT decomposition on the binary image to obtain a high-pass sub-band component and a low-pass sub-band component;

[0013] After estimating the noise variance for the high-pass subband component, perform bivariate threshold denoising to obtain a first image;

[0014] The low-pass sub-band component is enhanced by improving the membership function and then inversely transformed to obtain a second image;

[0015] The first image and the second image are reconstructed to obtain a standard data set.

[0016] Preferably, it also includes performing intelligent control avoidance according to the recognition result of the seabed obstacle recognition model.

[0017] Preferably, the improved Yolov5 model is: using the GhostNet end-side neural network to replace the traditional convolutional neural network of layers 2 to 8 in the Yolov5 backbone network, building the backbone network, and adding an attention mechanism module to the backbone network; replacing the original neck network of Yolov5 with a weighted bidirectional pyramid network BiFPN for feature extraction.

[0018] Preferably, the specific steps of training the improved Yolov5 model using the standard data set are:

[0019] Dividing the standard data set into a training set and a test set;

[0020] The training set is passed into the improved Yolov5 model for training to obtain the predicted position, size and category of the obstacle;

[0021] The difference between the predicted obstacle and the actual situation is calculated through the loss function, including classification loss, positioning loss and confidence loss in the target detection task;

[0022] Through the gradient descent algorithm, the weight matrix and bias in the forward propagation process are continuously adjusted to narrow the gap between the predicted value and the true value, thereby optimizing the model parameters;

[0023] Repeat the above steps for multiple iterations until the model converges or reaches the predetermined number of training rounds;

[0024] After the training is completed, save the trained model weight parameters.

[0025] On the other hand, a deep learning-based submarine obstacle recognition system is provided, which is characterized by comprising a sonar image acquisition module, an image segmentation module, a preprocessing module, a training module, and an obstacle recognition module; wherein,

[0026] The sonar image acquisition module is used to collect underwater sonar image data using sonar equipment;

[0027] The image segmentation module is used to perform image segmentation on the underwater sonar image data, separate the echo area, the shadow area and the seabed reverberation area, and obtain a binary image;

[0028] The preprocessing module is used to preprocess the binary image and construct a standard data set for model training;

[0029] The training module is used to build an improved Yolov5 model, set a loss function, and use the standard data set to train the improved Yolov5 model to obtain a seabed obstacle recognition model;

[0030] The obstacle recognition module is used to input the image to be detected into the seabed obstacle recognition model to obtain a recognition result.

[0031] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0032] 1. Replace the 2-8 layers of traditional convolutional neural network in the backbone network with the GhostNet end-side neural network. Take advantage of the lightweight Ghost module to reduce model parameters and computing time. Replace the original neck network of Yolov5 with the weighted bidirectional pyramid network BiFPN for feature extraction. Introduce learnable weights for features of different scales to better balance feature information of different scales. Introduce the attention mechanism in the backbone network. The optimized neural network model has higher confidence and detection speed, and can accurately identify submarine obstacles in real time.

[0033] 2. The present invention can improve the automation and intelligence of obstacle detection by training the improved Yolov5 model and identifying submarine obstacles based on deep learning;

[0034] 3. Introducing the attention mechanism into the backbone network can better grasp the global and local features of the image. The optimized neural network model has higher confidence and detection speed to improve the robustness of obstacle detection and can accurately identify submarine obstacles in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 is a flow chart of the method of the present invention;

[0037] Figure 2 It is a flow chart of the improved Otsu segmentation algorithm of the present invention;

[0038] Figure 3 It is a system structure diagram of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The purpose of this invention is to provide a method for identifying submarine obstacles based on deep learning. Figure 1 As shown, the specific steps include the following:

[0041] S1. Collect underwater sonar image data using sonar equipment;

[0042] S2, performing image segmentation on the underwater sonar image data, separating the echo area, the shadow area and the seabed reverberation area, and obtaining a binary image;

[0043] S3, preprocessing the binary image to construct a standard dataset for model training;

[0044] S4. Build an improved Yolov5 model, set the loss function, and use the standard data set to train the improved Yolov5 model to obtain a submarine obstacle recognition model;

[0045] S5. Input the image to be detected into the seabed obstacle recognition model to obtain the recognition result.

[0046] Furthermore, in step S2, the underwater sonar image data is segmented using the improved Otsu segmentation algorithm. The specific steps of the improved Otsu segmentation algorithm are as follows:

[0047] S21. First, define three points (k0, k1, k2) to divide the entire grayscale into 4 parts, k0 is the average value of the entire gray histogram, k1 and k2 are the grayscale average values ​​of the upper and lower parts.

[0048] S22. Calculate the inter-class variance of the three points and get the maximum variance. Assuming that this point is gray level S, calculate and compare S, S-1 and S+1 to calculate the maximum inter-class variance.

[0049] S23. Assume that the sonar image is represented by L gray levels [0, 1, ..., L-1], A k The area of ​​the contour represented is less than or equal to the set value of 20 pixels for output, and the Open CV library function is used to find all contours in the binary image and calculate A k .

[0050] S24, set value comparison, if the area is greater than the set value and the number exceeds the set value, recalculate. The main process of the algorithm is as follows Figure 2 shown.

[0051] Furthermore, the specific steps of preprocessing the binary image in step S3 are:

[0052] S31, performing NSCT decomposition on the binary image to obtain a high-pass sub-band component and a low-pass sub-band component;

[0053] S32, estimating the noise variance of the high-pass subband component and performing bivariate threshold denoising to obtain a first image;

[0054] S33, performing inverse transformation on the low-pass subband component after enhancement by improving the membership function to obtain a second image;

[0055] S34, reconstructing the first image and the second image to obtain a standard data set.

[0056] Furthermore, the improved Yolov5 model in step S4 is: using the GhostNet end-side neural network to replace the 2nd to 8th layer of the traditional convolutional neural network in the Yolov5 backbone network, building the backbone network, and adding an attention mechanism module to the backbone network; replacing the original neck network of Yolov5 with a weighted bidirectional pyramid network BiFPN for feature extraction.

[0057] The specific steps for training the improved Yolov5 model using the standard dataset are:

[0058] S41, divide the standard data set into a training set and a test set;

[0059] S42, passing the training set into the improved Yolov5 model for training to obtain the predicted position, size and category of the obstacle;

[0060] S43, calculating the difference between the predicted obstacle and the actual situation through the loss function, including the classification loss, positioning loss and confidence loss in the target detection task;

[0061] S44, through the gradient descent algorithm, the weight matrix and bias in the forward propagation process are continuously adjusted to narrow the gap between the predicted value and the true value, thereby optimizing the model parameters;

[0062] S45, repeat the above steps for multiple iterations until the model converges or reaches a predetermined number of training rounds;

[0063] S46. After the training is completed, save the trained model weight parameters.

[0064] Furthermore, it also includes intelligent control and avoidance based on the recognition results of the seabed obstacle recognition model.

[0065] On the other hand, a deep learning-based submarine obstacle recognition system is provided. Figure 3 As shown, it includes a sonar image acquisition module, an image segmentation module, a preprocessing module, a training module, and an obstacle recognition module; wherein,

[0066] A sonar image acquisition module is used to collect underwater sonar image data using sonar equipment;

[0067] An image segmentation module is used to segment underwater sonar image data, separate the echo area, shadow area and seabed reverberation area, and obtain a binary image;

[0068] The preprocessing module is used to preprocess binary images and construct a standard data set for model training;

[0069] The training module is used to build an improved Yolov5 model, set the loss function, and use the standard data set to train the improved Yolov5 model to obtain a submarine obstacle recognition model;

[0070] The obstacle recognition module is used to input the image to be detected into the seabed obstacle recognition model to obtain the recognition result.

[0071] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for identifying submarine obstacles based on deep learning, characterized in that: The specific steps include the following: Collect underwater sonar image data using sonar equipment; Performing image segmentation on the underwater sonar image data to separate the echo area, the shadow area and the seabed reverberation area to obtain a binary image; Preprocessing the binary image to construct a standard data set for model training; An improved Yolov5 model is built, a loss function is set, and the improved Yolov5 model is trained using the standard data set to obtain a seabed obstacle recognition model; The image to be detected is input into the seabed obstacle recognition model to obtain a recognition result.

2. The method for identifying submarine obstacles based on deep learning according to claim 1, characterized in that: The underwater sonar image data is segmented using an improved Otsu segmentation algorithm.

3. The method for identifying submarine obstacles based on deep learning according to claim 1, characterized in that: The specific steps of preprocessing the binary image are: Performing NSCT decomposition on the binary image to obtain a high-pass sub-band component and a low-pass sub-band component; After estimating the noise variance for the high-pass subband component, perform bivariate threshold denoising to obtain a first image; The low-pass sub-band component is enhanced by improving the membership function and then inversely transformed to obtain a second image; The first image and the second image are reconstructed to obtain a standard data set.

4. The method for identifying submarine obstacles based on deep learning according to claim 1, characterized in that: It also includes intelligent control avoidance according to the recognition result of the seabed obstacle recognition model.

5. The method for identifying submarine obstacles based on deep learning according to claim 1, characterized in that: The improved Yolov5 model is as follows: the GhostNet end-side neural network is used to replace the traditional convolutional neural network of layers 2 to 8 in the Yolov5 backbone network, the backbone network is built, and an attention mechanism module is added to the backbone network; the original neck network of Yolov5 is replaced by a weighted bidirectional pyramid network BiFPN for feature extraction.

6. The method for identifying submarine obstacles based on deep learning according to claim 1, characterized in that: The specific steps of training the improved Yolov5 model using the standard data set are: Dividing the standard data set into a training set and a test set; The training set is passed into the improved Yolov5 model for training to obtain the predicted position, size and category of the obstacle; The difference between the predicted obstacle and the actual situation is calculated through the loss function, including classification loss, positioning loss and confidence loss in the target detection task; Through the gradient descent algorithm, the weight matrix and bias in the forward propagation process are continuously adjusted to narrow the gap between the predicted value and the true value, thereby optimizing the model parameters; Repeat the above steps for multiple iterations until the model converges or reaches the predetermined number of training rounds; After the training is completed, save the trained model weight parameters.

7. A deep learning-based submarine obstacle recognition system, characterized in that: It includes sonar image acquisition module, image segmentation module, preprocessing module, training module and obstacle recognition module; among them, The sonar image acquisition module is used to collect underwater sonar image data using sonar equipment; The image segmentation module is used to perform image segmentation on the underwater sonar image data, separate the echo area, the shadow area and the seabed reverberation area, and obtain a binary image; The preprocessing module is used to preprocess the binary image and construct a standard data set for model training; The training module is used to build an improved Yolov5 model, set a loss function, and use the standard data set to train the improved Yolov5 model to obtain a seabed obstacle recognition model; The obstacle recognition module is used to input the image to be detected into the seabed obstacle recognition model to obtain a recognition result.

Images