Underwater image recognition method based on Bayesian neural network

Abstract

The invention provides an underwater image recognition method based on a Bayesian neural network, and belongs to the technical field of underwater image recognition. The technical problem that a general target identification method is poor in robustness in an underwater environment with visual degradation is solved. The method comprises the following steps: S1, constructing an underwater submarine image data set; s2, constructing an underwater image recognition model based on a Bayesian neural network; s3, performing deep learning model training by using the constructed data set to obtain an underwater image recognition model; and S4, according to the trained prediction model, performing prediction analysis on an image acquired when the underwater vehicle executes the underwater task, and obtaining an image target classification result and uncertainty.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater image recognition, and in particular to an underwater image recognition method based on a Bayesian neural network in an uncertain environment. Background Art

[0002] With the rapid development of artificial intelligence technology, computer technology, etc., unmanned submarines play an increasingly important role in marine warfare. The new generation of unmanned underwater vehicles has a high enough degree of intelligence to interact with the environment so as to effectively detect and identify underwater objects when performing tasks in water. Modern submarines use the wake characteristics of submarines and high-sensitivity sensor technology, remote sensing technology, blue-green laser detection technology, signal processing technology, image recognition technology, etc. to conduct reconnaissance, surveillance, and tracking of submarines active in the ocean.

[0003] In recent years, underwater target detection has an important role in the field of marine exploration, and is particularly important for the execution of underwater computer vision tasks, such as target positioning, recognition, and tracking, especially providing technical support in the aspect of marine military. However, the underwater environment is often full of uncertainties, such as changes in light conditions, the influence of underwater particulate matter, complex terrain features, etc., which are difficult to meet the requirements of complex marine target detection. Summary of the Invention

[0004] The purpose of the present invention is to provide an underwater image recognition method based on a Bayesian neural network in an uncertain environment. This recognition method can quickly detect and identify submarines, unmanned underwater vehicles (UUVs), or other potential threats in a complex marine environment, and enhance its detection robustness in a complex underwater environment.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions: An underwater image recognition method based on a Bayesian neural network, and the steps of this method include:

[0006] S1. Construct an underwater submarine image dataset;

[0007] S2. Construct an underwater image recognition model based on a Bayesian neural network;

[0008] S3. Use the constructed dataset to train the deep learning model to obtain a Bayesian underwater image recognition model;

[0009] S4. According to the trained prediction model, perform prediction analysis on the images collected when the underwater vehicle performs underwater tasks to obtain the image target classification result and uncertainty.

[0010] Preferably, in the said step S1, constructing the underwater image dataset includes the following steps:

[0011] S11: Collect video data through an underwater camera and preprocess the video data to obtain datasets corresponding to three types: g_submarine (green submarine), b_submarine (blue submarine), and black_submarine (black submarine).

[0012] S12: Load the images and their annotation information in the dataset, including the target label y and the bounding box position u x , u y , u h , u w , Use 80% of the images in the dataset for training and validation, and 20% of the images for testing. Process the bounding boxes, labels, and additional information of the objects into Tensor format for convenient subsequent model training.

[0013] The specific method for collecting underwater target videos in the above step S11 is as follows: The underwater camera of the submarine is as close as possible to the object being photographed to reduce the impact of water particles on the image clarity. When collecting in deep water areas, additional light sources are supplemented by the submarine. The underwater camera pan-tilt is set with a self-stabilization model to reduce image blurring caused by the shaking of the submarine.

[0014] Preferably, in the underwater image recognition method based on a Bayesian neural network, the underwater image recognition model based on a Bayesian neural network in step S2 is composed of a convolutional layer, a batch normalization layer, a max pooling layer, a Bayesian linear layer, a fully connected layer, and an activation function Rule. In the classification and regression network layer of this network structure, we designed a Bayesian layer (Bnn_fc), and introduced uncertainty into the model with the output of the Bayesian layer, and then connected 2 fully connected layers for classification and boundary regression. The parameters of this Bayesian network layer introduce an underwater environment dynamic correction factor γ to assist in constructing a prior distribution, and the prior distribution follows a Gaussian distribution, satisfying The true posterior distribution q(ω) is approximated by a variational distribution q α (ω) parameterized by the parameter α, and the gap between this variational distribution and the true posterior is minimized by optimizing the KL divergence to enhance the generalization ability of the model. During training, the constructed model is optimized using a stochastic gradient descent optimizer with momentum and a step-type learning rate setting.

[0015] The goal is to minimize the loss function of the Bayesian neural network:

[0016]

[0017] In the formula, N represents the total number of samples, K represents the number of labels (including the background class), y i,c is an indicator variable, p i,c is the probability that sample i is predicted as class c. N pos is the number of positive samples, To predict the bounding box regression value, b i is the true bounding box regression value. q α (ω) is the variational distribution, and p(ω) is the prior probability of the parameters. KL is the divergence loss, and λ is a hyperparameter that adjusts the balance between the classification loss and the regression loss. This loss function includes the losses of classification and bounding box regression as well as the KL divergence.

[0018] Output a probability distribution for each class recognized in the image, that is, p = (p 0 ,..., p k ), and output the regression parameters of the corresponding class c predicted by the bounding box regressor These two constitute the overall output of the prediction model.

[0019] Preferably, the model training process in step S3 includes the following steps:

[0020] S31. Construct an underwater image recognition model based on a Bayesian neural network, including an initial convolutional layer, followed by a batch normalization layer and a ReLU activation function, and use a max-pooling layer for downsampling. Define a Bayesian linear layer to introduce uncertainty. This Bayesian linear layer fuses the prior of multimodal information and incorporates underwater depth data to assist in constructing the prior distribution.

[0021] S32. Load the pre-trained weights and perform initial parameter settings. The underwater environment dynamic correction factor is fine-tuned in real time according to the depth parameters collected in real time

[0022] S33. Perform a loop within the specified epoch. The specific steps are as follows:

[0023] (1) Sample from the approximate posterior distribution, perform forward propagation to obtain the predicted value and calculate the average likelihood term

[0024] (2) Calculate the loss function L e (θ). Use the stochastic gradient descent optimizer with momentum and the step-type learning rate setting. The optimization objective is to minimize L e (θ).

[0025] (3) Repeat the previous two steps until the model converges, and save the parameters.

[0026] S34. After each training round, in particular, construct an underwater target evaluation system, including image recognition accuracy, recall rate, and robustness indicators in complex underwater scenarios. Use the validation set to evaluate the model performance. After reaching the preset number of training rounds, save all model parameters and end the training.

[0027] Preferably, the prediction analysis according to the trained prediction model in step S4 includes the following steps:

[0028] S41. Collect underwater image videos as a dataset

[0029] S42. Input the image dataset, sequentially pass through the network layers, and output the result at the output layer

[0030] S43. Update the network parameters and calculate each loss value.

[0031] S44. Obtain the final prediction value, and acquire the target classification result and more accurate position information.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. The present invention collects images through the underwater camera of the submersible. When collecting, the self-stabilization model is set for the underwater camera pan-tilt to reduce the image blurring caused by the shaking of the submersible and additionally supplement the light source to initially enhance the details of the collected images and improve the detection accuracy.

[0034] 2. Since the underwater environment is often full of uncertainties, such as changes in light conditions, the influence of underwater particulate matter, complex terrain features, etc., it is difficult to meet the requirements of complex marine target detection. An underwater image recognition method based on Bayesian neural network proposed by the present invention can improve the performance of image recognition technology in complex and uncertain underwater environments.

[0035] 3. The present invention constructs an underwater image recognition model based on Bayesian neural network, which can not only improve the performance and accuracy of target detection, but also bring significant advantages in feature utilization, network structure and computational efficiency. This fusion enables the underwater image recognition model to more effectively perform complex recognition tasks.

[0036] 4. The present invention introduces uncertainty into the model by designing a Bayesian linear layer. This Bayesian linear layer integrates the prior of multimodal information. In addition to the pixel information of the image itself, it also incorporates underwater depth data to assist in constructing the prior distribution, making the model more adaptable when facing image recognition in different water environments.

[0037] 5. Construct an underwater target evaluation system, including image recognition accuracy rate, recall rate, and robustness indicators in complex underwater scenarios. Perform predictive analysis on the images collected when the submersible executes underwater tasks, and output a probability distribution for each category recognized in the image to obtain the image target classification result and more accurate position information. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is the overall flowchart of the present invention.

[0039] Figure 2 It is the schematic diagram of the model structure of the present invention.

[0040] Figure 3 This is the mAP curve graph on the validation set of the present invention.

[0041] Figure 4 This is the flow chart of the training model of the present invention.

[0042] Figure 5 This is the recognition effect diagram of each category in the first example of the present invention.

[0043] Figure 6 This is the recognition effect diagram of each category in the second example of the present invention. Detailed implementation manners

[0044] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0045] Embodiment 1

[0046] Refer to Figure 1 , an underwater image recognition method based on a neural network, comprising the following steps:

[0047] Step 1: Construct an underwater submarine image dataset;

[0048] Step 2: Construct an underwater image recognition model based on a Bayesian neural network;

[0049] Step 3: Use the constructed dataset to train a deep learning model to obtain an underwater image recognition model;

[0050] Step 4: According to the trained prediction model, perform prediction analysis on the images collected when the submersible performs underwater tasks to obtain the image target classification result and uncertainty.

[0051] Step 1: Constructing an underwater image dataset includes the following steps:

[0052] 1-1), Take screenshots of several key frames of the collected underwater target video to construct a dataset. It includes three categories: g_submarine (green submersible), b_submarine (blue submersible), and black_submarine (black submersible).

[0053] 1-2), Load the images and their annotation information in the dataset, including the target label y and the bounding box position u x ,u y ,u h ,u w , Use 80% of the images in this dataset for training and validation, and 20% of the images for testing. Process the bounding box, label, and additional information of the object into the Tensor format for convenient subsequent model training.

[0054] The specific method for collecting underwater target videos in the above step 1-1) is as follows: The underwater camera of the submersible is as close as possible to the object to be photographed to reduce the impact of water particles on the image clarity. When collecting in deep water areas, additional light sources are supplemented by the submersible. The underwater camera pan-tilt is set with a self-stabilization model to reduce image blurring caused by the shaking of the submersible.

[0055] Step 2: Construct an underwater image recognition model based on the Bayesian neural network. As Figure 2 shown, the underwater image recognition model based on the Bayesian neural network consists of a convolutional layer, a batch normalization layer, a max pooling layer, a Bayesian linear layer, a fully connected layer, and an activation function Rule. In the classification regression network layer of this network structure, we designed a Bayesian layer (Bnn_fc), and the output of the Bayesian layer was used to introduce uncertainty into the model, and then 2 fully connected layers were connected for classification and boundary regression. The parameters of this Bayesian network layer introduce an underwater environment dynamic correction factor γ to assist in constructing the prior distribution, and the prior distribution follows a Gaussian distribution, satisfying The true posterior distribution q(ω) is approximated by a variational distribution q α (ω) parameterized by the parameter α, and the generalization ability of the model is enhanced by optimizing the KL divergence to minimize the gap between this variational distribution and the true posterior. During training, the constructed model is optimized using a stochastic gradient descent optimizer with momentum and a step-type learning rate setting.

[0056] The goal is to minimize the loss function of the Bayesian neural network:

[0057]

[0058] In the formula, N represents the total number of samples, K represents the number of labels (including the background class), y i,c is an indicator variable, p i,c is the probability that sample i is predicted as class c. N pos is the number of positive samples, is the predicted bounding box regression value, b i is the true bounding box regression value. q α (ω) is the variational distribution, and p(ω) is the prior probability of the parameters. KL is the divergence loss, and λ is a hyperparameter that adjusts the balance between the classification loss and the regression loss. This loss function includes the losses of classification and bounding box regression as well as the KL divergence.

[0059] A probability distribution is output for each class recognized in the image, that is, p = (p 0 ,..., p k ), and the regression parameters corresponding to class c predicted by the bounding box regressor are output.

[0060] Step 3: The model training process includes the following steps:

[0061] 3-1), Construct an underwater image recognition model based on a Bayesian neural network, including an initial convolutional layer, followed by a batch normalization layer and a ReLU activation function, and use a max pooling layer for downsampling. Define a Bayesian linear layer to introduce uncertainty, which fuses the prior of multimodal information and incorporates underwater depth data to assist in constructing the prior distribution.

[0062] 3-2), Load the pre-trained weights and perform initial parameter settings. The underwater environment dynamic correction factor is fine-tuned in real time according to the depth parameters collected in real time

[0063] 3-3), Perform a loop within the specified number of epochs. The specific steps are as follows:

[0064] (1) Sample from the approximate posterior distribution, perform forward propagation to obtain the predicted value and calculate the average likelihood term

[0065] (2) Calculate the loss function L e (θ). Use the stochastic gradient descent optimizer with momentum and step-type learning rate settings, and the optimization goal is to minimize L e (θ).

[0066] (3) Repeat the previous two steps until the model converges and save the parameters.

[0067] 3-4), After each training round, in particular, construct an underwater target evaluation system, including image recognition accuracy, recall rate, and robustness indicators in complex underwater scenarios. Use the validation set to evaluate the model performance, and save all model parameters after reaching the preset number of training rounds to end the training.

[0068] Train the dataset, and finally the mAP curve on the validation set is as Figure 3 shown.

[0069] Step 4: The prediction analysis based on the trained prediction model includes the following steps:

[0070] 4-1), Collect underwater image videos as the dataset

[0071] 4-2), Input the image dataset, sequentially pass through the network layers, and output the result at the output layer

[0072] 4-3), Update the network parameters and calculate each loss value.

[0073] 4-4), Obtain the final predicted value, and obtain the target classification result and more accurate location information. The recognition effect is as Figure 5 shown.

[0074] Example 2

[0075] Based on Example 1, referring to Figure 1 , an underwater image recognition method based on a neural network includes the following steps:

[0076] Step 1: Construct an underwater submarine image dataset;

[0077] Step 2: Construct an underwater image recognition model based on a Bayesian neural network;

[0078] Step 3: Use the constructed dataset to train the deep learning model to obtain an underwater image recognition model;

[0079] Step 4: According to the trained prediction model, perform prediction analysis on the images collected when the submersible performs underwater tasks to obtain the image target classification results and uncertainties.

[0080] Step 1: Constructing an underwater image dataset includes the following steps:

[0081] 1-1), Take screenshots of several key frames of the collected underwater target video to construct a dataset. It includes three categories: g_submarine (green submersible), b_submarine (blue submersible), and black_submarine (black submersible).

[0082] 1-2), Load the images and their annotation information in the dataset, including the target label y and the bounding box position u x , u y , u h , u w , Use 80% of the images in this dataset for training and validation, and 20% of the images for testing. Process the bounding box, label, and additional information of the object into the Tensor format for convenient subsequent model training.

[0083] The specific method for collecting the underwater target video in the above step 1-1) is as follows: The underwater camera of the submersible is as close as possible to the object to be photographed to reduce the impact of water particles on the image clarity. When collecting in deep water areas, additional light sources are supplemented by the submersible. The underwater camera pan-tilt is set with a self-stabilization model to reduce the image blur caused by the shaking of the submersible.

[0084] Step 2: Construct an underwater image recognition model based on a Bayesian neural network. As Figure 2As shown in the figure, the underwater image recognition model based on the Bayesian neural network consists of a convolutional layer, a batch normalization layer, a max pooling layer, a Bayesian linear layer, a fully connected layer, and an activation function Rule. In the classification and regression network layer of this network structure, we designed a Bayesian layer (Bnn_fc), introduced uncertainty into the model with the output of the Bayesian layer, and then connected 2 fully connected layers for classification and boundary regression. The parameters of this Bayesian network layer introduce an underwater environment dynamic correction factor γ to assist in constructing the prior distribution, and the prior distribution follows a Gaussian distribution, satisfying The true posterior distribution q(ω) is approximated by a variational distribution q α (ω) parameterized by the parameter α, and the generalization ability of the model is enhanced by optimizing the KL divergence to minimize the gap between this variational distribution and the true posterior. During training, the constructed model is optimized using a stochastic gradient descent optimizer with momentum and a step-type learning rate setting.

[0085] The goal is to minimize the loss function of the Bayesian neural network:

[0086]

[0087] In the formula, N represents the total number of samples, K represents the number of labels (including the background class), y i,c is an indicator variable, p i,c is the probability that sample i is predicted as class c. N pos is the number of positive samples, is the predicted bounding box regression value, b i is the true bounding box regression value. q α (ω) is the variational distribution, and p(ω) is the prior probability of the parameters. KL is the divergence loss, and λ is a hyperparameter that adjusts the balance between the classification loss and the regression loss. This loss function includes the losses of classification and bounding box regression as well as the KL divergence.

[0088] Outputs a probability distribution for each class recognized in the image, that is, p = (p 0 ,..., p k ), and outputs the regression parameters corresponding to class c predicted by the bounding box regressor. The two constitute the overall output of this prediction model.

[0089] Step 3: The model training process includes the following steps:

[0090] 3-1), Construct an underwater image recognition model based on the Bayesian neural network, including an initial convolutional layer, followed by a batch normalization layer and a ReLU activation function, and use a max pooling layer for downsampling. Define a Bayesian linear layer to introduce uncertainty. This Bayesian linear layer fuses the prior of multimodal information and incorporates underwater depth data to assist in constructing the prior distribution.

[0091] 3-2), Load the pre-trained weights and perform initial parameter settings. The underwater environment dynamic correction factor is fine-tuned in real time according to the depth parameters collected in real time.

[0092] 3-3), Perform a loop within the specified epoch. The specific steps are as follows:

[0093] (1) Sample from the approximate posterior distribution, perform forward propagation to obtain the predicted value and calculate the average likelihood term.

[0094] (2) Calculate the loss function L e (θ). Use the stochastic gradient descent optimizer with momentum and the step-type learning rate setting. The optimization goal is to minimize L e (θ).

[0095] (3) Repeat the previous two steps until the model converges and save the parameters.

[0096] 3-4), After each training epoch, in particular, construct an underwater target evaluation system, including image recognition accuracy, recall rate, and robustness metrics in complex underwater scenarios. Use the validation set to evaluate the model performance. After reaching the preset number of training epochs, save all model parameters and end the training.

[0097] Train the dataset, and the mAP curve on the validation set is as Figure 3 shown.

[0098] Step 4: Perform prediction analysis based on the trained prediction model, including the following steps:

[0099] 4-1), Collect underwater image videos as the dataset.

[0100] 4-2), Input the image dataset, sequentially pass through the network layers, and output the results at the output layer.

[0101] 4-3), Update the network parameters and calculate each loss value.

[0102] 4-4), Obtain the final predicted value, acquire the target classification result and more accurate position information. The recognition effect is as Figure 6 .

[0103] Example 3

[0104] Based on Embodiment 2, the present invention compared the method of the present invention with two other relatively common underwater image recognition methods in constructing an underwater submarine dataset. The present invention adopted a series of evaluation indicators commonly used in object detection tasks, including mAP, mAP50-95, AR50-95 (maxdets = 10), AR50-95 (maxdets = 100), to objectively evaluate the proposed method. The larger the values of these indicators, the better the underwater target detection effect. The specific evaluation results are presented in Table 1 below:

[0105]

[0106] As can be seen from Table 1, the method of the present invention achieved 99.0% mAP, 61.2% mAP50-95, 68.2% AR50-95, and 68.2% AR50-95 on the self-constructed underwater submarine dataset, which is superior to other conventional image recognition algorithms in the field of underwater image recognition.

[0107] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An underwater image recognition method based on Bayesian neural network, characterized in that: The following steps are involved: S1. Build an underwater submarine image dataset; S2. Construct an underwater image recognition model based on Bayesian neural network; S3, using the constructed data set to train a deep learning model to obtain a Bayesian underwater image recognition model; S4. Based on the trained prediction model, perform prediction analysis on the images collected by the submersible when performing underwater missions to obtain image target classification results and uncertainties.

2. The underwater image recognition method based on Bayesian neural network according to claim 1 is characterized in that: The step S1 of constructing the underwater image dataset comprises the following steps: S11: Collect video data through an underwater camera and pre-process the video data to obtain data sets corresponding to three types: g_submarine green submarine, b_submarine blue submarine, and black_submarine black submarine; S12: Load the images in the dataset and their annotation information, including the target label y and the bounding box position u x ,u y ,u h ,u w ,80% of the images in this dataset are used for training and validation, and 20% of the images are used for testing. The bounding boxes, labels and additional information of the objects are processed into Tensor format.

3. The underwater image recognition method based on Bayesian neural network according to claim 1 is characterized in that: The underwater image recognition model based on the Bayesian neural network in step S2 is composed of a convolution layer, a batch normalization layer, a maximum pooling layer, a Bayesian linear layer, a fully connected layer and an activation function Rule. In the classification and regression network layer of the network structure, the output of the Bayesian layer is used to introduce uncertainty into the model, and then two fully connected layers are connected for classification and boundary regression. The Bayesian network layer parameters introduce the underwater environment dynamic correction factor γ to assist in constructing a priori distribution, and the prior distribution obeys a Gaussian distribution and satisfies The true posterior distribution q(ω) is parameterized by a variational distribution q α (ω) is used to approximate the variance distribution. The gap between the variational distribution and the true posterior is minimized by optimizing the KL divergence to enhance the generalization ability of the model. During training, the constructed model is optimized with a stochastic gradient descent optimizer with momentum and a step-type learning rate setting. The goal is to minimize the loss function of the Bayesian neural network: Where N is the total number of samples, K is the number of labels, including background classes, and y i,c is an indicator variable, p i,c is the probability that sample i is predicted to be category c, N pos is the number of positive samples, is the predicted bounding box regression value, b i is the true bounding box regression value, q α (ω) is the variational distribution, p(ω) is the prior probability of the parameter, KL is the divergence loss, λ is the hyperparameter for adjusting the balance between classification loss and regression loss. The loss function includes the loss of classification and bounding box regression as well as KL divergence; Output a probability distribution for each image category, that is, p = (p0, ..., p k ), output the regression parameters of the corresponding category c predicted by the bounding box regressor The two constitute the overall output of the prediction model.

4. The underwater image recognition method based on Bayesian neural network according to claim 1 is characterized in that: The model training process in step S3 includes the following steps: S31. Construct an underwater image recognition model based on a Bayesian neural network, including an initial convolutional layer, followed by a batch normalization layer and a ReLU activation function, use a maximum pooling layer for downsampling, define a Bayesian linear layer to introduce uncertainty, and the Bayesian linear layer fuses multimodal information priors and incorporates underwater depth data to assist in constructing a prior distribution; S32, load the pre-trained weights and set the initial parameters. The underwater environment dynamic correction factor is fine-tuned in real time based on the depth parameters collected in real time. S33, looping within the specified epoch, the specific steps are: (1) Sampling from the approximate posterior distribution, performing forward propagation to obtain the predicted value and calculating the average likelihood term; (2) Calculate the loss function L e (θ), using the stochastic gradient descent optimizer with momentum and a step-type learning rate setting, the optimization goal is to minimize L e (θ); (3) Repeat the first two steps until the model converges and save the parameters; S34. After each training round, an underwater target evaluation system is constructed, including image recognition accuracy, recall rate, and robustness indicators in complex underwater scenes. The model performance is evaluated using a validation set. After reaching the preset number of training rounds, all model parameters are saved and the training ends.

5. The underwater image recognition method based on Bayesian neural network according to claim 1 is characterized in that: The prediction analysis performed according to the trained prediction model in step S4 includes the following steps: S41, collecting underwater image videos as a data set; S42, input image data set, pass through network layers sequentially, and output the result in output layer; S43, updating network parameters and calculating each loss value; S44. Get the final prediction value, obtain the target classification result and more accurate location information.

Images