An underwater image recognition method based on a Bayesian neural network
By using an underwater image recognition method based on Bayesian neural networks, the problem of target recognition caused by the uncertainty of the underwater environment is solved, and efficient target detection in complex marine environments is achieved, improving recognition accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANTONG UNIV
- Filing Date
- 2025-02-06
- Publication Date
- 2026-06-26
AI Technical Summary
The uncertainties of the underwater environment, such as changes in lighting conditions and complex terrain features, make it difficult for existing technologies to effectively identify underwater targets, especially in meeting the complex marine target detection needs in the maritime military field.
An underwater image recognition method based on Bayesian neural networks is adopted. By constructing an underwater submarine image dataset, an uncertainty is introduced by designing a Bayesian layer, and a dynamic correction factor for the underwater environment is combined to optimize the model to improve recognition accuracy and robustness.
It improves the accuracy and robustness of target detection in complex underwater environments, effectively identifies submarines and unmanned underwater vehicles, and enhances detection capabilities in uncertain environments.
Smart Images
Figure CN120047808B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of underwater image recognition technology, and in particular to an underwater image recognition method based on Bayesian neural networks in uncertain environments. Background Technology
[0002] With the rapid development of artificial intelligence and computer technology, unmanned underwater vehicles (UUVs) are playing an increasingly important role in naval warfare. The new generation of UUVs possesses a high degree of intelligence, enabling them to interact with their environment and effectively detect and identify underwater objects while performing missions. Modern submarines utilize their wake characteristics, highly sensitive sensor technology, remote sensing technology, blue-green laser detection technology, signal processing technology, and image recognition technology to conduct reconnaissance, surveillance, and tracking of submarines operating in the ocean.
[0003] In recent years, underwater target detection has played a vital role in marine exploration, particularly in the execution of underwater computer vision tasks such as target localization, identification, and tracking, providing crucial technical support for marine military applications. However, the underwater environment is often fraught with uncertainty, including variations in lighting conditions, the influence of underwater particles, and complex terrain features, making it difficult to meet the demanding requirements of marine target detection. Summary of the Invention
[0004] The purpose of this invention is to provide an underwater image recognition method based on Bayesian neural networks in uncertain environments. This recognition method can quickly detect and identify submarines, unmanned underwater vehicles (UUVs) or other potential threats in complex marine environments, and enhance their detection robustness in complex underwater environments.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: an underwater image recognition method based on a Bayesian neural network, the method comprising the following steps:
[0006] S1. Construct an underwater submarine image dataset;
[0007] S2. Construct an underwater image recognition model based on a Bayesian neural network;
[0008] S3. Use the constructed dataset to train a deep learning model to obtain a Bayesian underwater image recognition model;
[0009] S4. Based on the trained prediction model, perform predictive analysis on the images collected by the submersible when performing underwater tasks, and obtain the image target classification results and uncertainty.
[0010] Preferably, in step S1, constructing the underwater image dataset includes the following steps:
[0011] S11: Collect video data using an underwater camera and preprocess the video data to obtain datasets for three types: g_submarine (green submersible), b_submarine (blue submersible), and black_submarine (black submersible).
[0012] S12: Load the images and their annotation information from the dataset, including the target label y and the bounding box position u. x ,u y ,u h ,u w 80% of the images in this dataset were used for training and validation, and 20% were used for testing. The bounding boxes, labels, and additional information of the objects were processed into Tensor format to facilitate subsequent model training.
[0013] The specific procedure for acquiring underwater target video in step S11 above is as follows: the underwater camera of the submersible is placed as close as possible to the object being filmed to reduce the impact of underwater particles on image clarity. When acquiring video in deep water, the submersible provides additional light. The underwater camera gimbal is equipped with a self-stabilizing model to reduce image blurring caused by the submersible's movement.
[0014] Preferably, the underwater image recognition method based on Bayesian neural networks is characterized in that the underwater image recognition model based on Bayesian neural networks in step S2 consists of convolutional layers, batch normalization layers, max pooling layers, Bayesian linear layers, fully connected layers, and activation function Rule. In the classification and regression network layer of this network structure, we designed a Bayesian layer (Bnn_fc) to introduce uncertainty into the model using the output of the Bayesian layer, followed by two fully connected layers for classification and boundary regression. The parameters of this Bayesian network layer introduce an underwater environment dynamic correction factor γ to assist in constructing a prior distribution, and the prior distribution follows a Gaussian distribution, satisfying… The true posterior distribution q(ω) is given by a variational distribution q parameterized by parameter α. α The model's generalization ability is enhanced by approximating the difference between the variational distribution and the true posterior by optimizing the KL divergence to minimize this difference. During training, the constructed model is optimized using a stochastic gradient descent optimizer with momentum and a step learning rate setting.
[0015] The objective is to minimize the loss function of the Bayesian neural network.
[0016]
[0017] In the formula, N represents the total number of samples, K represents the number of labels (including the background class), and y i,c p is an indicator variable. i,c Predict the probability that sample i belongs to class c. N pos The number of positive samples. To predict the bounding box regression value, b i q represents the true bounding box regression value. α (ω) represents the variational distribution, and p(ω) is the prior probability of the parameter. KL is the divergence loss, and λ is a hyperparameter that balances the classification loss and regression loss. This loss function includes the losses from classification and bounding box regression, as well as the KL divergence.
[0018] For each image category, output a probability distribution, i.e., p = (p0, ..., p...). k The output bounding box regressor predicts the regression parameters for the corresponding category c. Both constitute the overall output of the prediction model.
[0019] Preferably, the model training process in step S3 includes the following steps:
[0020] S31. Construct an underwater image recognition model based on a Bayesian neural network, including an initial convolutional layer, followed by a batch normalization layer and a ReLU activation function, and using a max pooling layer for downsampling. Define a Bayesian linear layer to introduce uncertainty. This Bayesian linear layer integrates multimodal prior information and incorporates underwater depth data to assist in constructing a prior distribution.
[0021] S32. Load the pre-trained weights and set the initial parameters. The underwater environment dynamic correction factor is fine-tuned in real time based on the depth parameters collected in real time.
[0022] S33. Loop within the specified epoch. The specific steps are as follows:
[0023] (1) Sample from the approximate posterior distribution, perform forward propagation to obtain the predicted value and calculate the average likelihood term.
[0024] (2) Calculate the loss function L e (θ). Using a stochastic gradient descent optimizer with momentum and a step learning rate setting, the optimization objective is to minimize L. e (θ).
[0025] (3) Repeat the first two steps until the model converges and save the parameters.
[0026] S34. After each training epoch, specifically, an underwater target evaluation system is constructed, including image recognition accuracy, recall, and robustness metrics in complex underwater scenarios. The model performance is evaluated using a validation set. After reaching the preset number of training epochs, all model parameters are saved, and training ends.
[0027] Preferably, step S4, which involves predictive analysis based on the trained prediction model, includes the following steps:
[0028] S41. Collect underwater images and videos as a dataset.
[0029] S42. Input image dataset, pass it sequentially through network layers, and output the results at the output layer.
[0030] S43. Update network parameters and calculate each loss value.
[0031] S44. Obtain the final predicted value, and get the target classification result and more accurate location information.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] 1. This invention uses an underwater camera on a submersible to acquire images. During acquisition, the underwater camera's gimbal is set to a self-stabilizing model to reduce image blurring caused by the submersible's shaking and to supplement the light source, thereby initially enhancing the details of the acquired images and improving the accuracy of detection.
[0034] 2. Due to the inherent uncertainties of underwater environments, such as variations in lighting conditions, the influence of underwater particles, and complex topographic features, it is difficult to meet the demands of complex marine target detection. This invention proposes an underwater image recognition method based on Bayesian neural networks, which can improve the performance of image recognition technology in complex and uncertain underwater environments.
[0035] 3. This invention constructs an underwater image recognition model based on Bayesian neural networks, which not only improves the performance and accuracy of target detection but also brings significant advantages in feature utilization, network structure, and computational efficiency. This fusion enables the underwater image recognition model to perform complex recognition tasks more effectively.
[0036] 4. This invention introduces uncertainty into the model by designing a Bayesian linear layer. This Bayesian linear layer integrates multimodal prior information. In addition to the pixel information of the image itself, it also incorporates underwater depth data to help construct the prior distribution, making the model more adaptable to image recognition in different aquatic environments.
[0037] 5. Construct an underwater target evaluation system, including image recognition accuracy, recall rate, and robustness indicators in complex underwater scenarios. Perform predictive analysis on images collected by the submersible during underwater missions, and output a probability distribution for each image's identified category to obtain image target classification results and more accurate location information. Attached Figure Description
[0038] Figure 1 This is the overall flowchart of the present invention.
[0039] Figure 2 This is a schematic diagram of the model structure of the present invention.
[0040] Figure 3 This is the mAP curve on the validation set of this invention.
[0041] Figure 4 This is a flowchart of the training model of the present invention.
[0042] Figure 5 The images show the recognition results for each category in Example 1 of this invention.
[0043] Figure 6 The images show the recognition results for each category in Example 2 of this invention. Detailed Implementation
[0044] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0045] Example 1
[0046] Reference Figure 1 A neural network-based underwater image recognition method includes the following steps:
[0047] Step 1: Construct an underwater submarine image dataset;
[0048] Step 2: Construct an underwater image recognition model based on a Bayesian neural network;
[0049] Step 3: Use the constructed dataset to train a deep learning model to obtain an underwater image recognition model;
[0050] Step 4: Based on the trained prediction model, perform predictive analysis on the images collected by the submersible during underwater missions to obtain the image target classification results and uncertainty.
[0051] Step 1: Constructing the underwater image dataset includes the following steps:
[0052] 1-1) Capture keyframes from the acquired underwater target videos to construct a dataset. This dataset includes three categories: g_submarine (green submersible), b_submarine (blue submersible), and black_submarine (black submersible).
[0053] 1-2) Load the images and their annotation information from the dataset, including the target label y and the bounding box position u. x ,u y ,u h ,u w 80% of the images in this dataset were used for training and validation, and 20% were used for testing. The bounding boxes, labels, and additional information of the objects were processed into Tensor format to facilitate subsequent model training.
[0054] The specific procedures for acquiring underwater target videos in step 1-1) above are as follows: the underwater camera of the submersible is placed as close as possible to the object being filmed to reduce the impact of underwater particles on image clarity. When acquiring videos in deep water, the submersible provides additional light. The underwater camera gimbal is equipped with a self-stabilizing model to reduce image blurring caused by the submersible's movement.
[0055] Step 2: Construct an underwater image recognition model based on a Bayesian neural network. For example... Figure 2 As shown, the underwater image recognition model based on a Bayesian neural network consists of convolutional layers, batch normalization layers, max pooling layers, Bayesian linear layers, fully connected layers, and activation functions (Rule). In the classification and regression network layers of this structure, we designed a Bayesian layer (Bnn_fc) to introduce uncertainty into the model using its output, followed by two fully connected layers for classification and boundary regression. The parameters of this Bayesian network layer incorporate an underwater environment dynamic correction factor γ to assist in constructing a prior distribution, and this prior distribution follows a Gaussian distribution, satisfying... The true posterior distribution q(ω) is given by a variational distribution q parameterized by parameter α. α The model's generalization ability is enhanced by approximating the difference between the variational distribution and the true posterior by optimizing the KL divergence to minimize this difference. During training, the constructed model is optimized using a stochastic gradient descent optimizer with momentum and a step learning rate setting.
[0056] The objective is to minimize the loss function of the Bayesian neural network.
[0057]
[0058] In the formula, N represents the total number of samples, K represents the number of labels (including the background class), and y i,c p is an indicator variable. i,c Predict the probability that sample i belongs to class c. N pos The number of positive samples. To predict the bounding box regression value, b i q represents the true bounding box regression value. α (ω) represents the variational distribution, and p(ω) is the prior probability of the parameter. KL is the divergence loss, and λ is a hyperparameter that balances the classification loss and regression loss. This loss function includes the losses from classification and bounding box regression, as well as the KL divergence.
[0059] For each image category, output a probability distribution, i.e., p = (p0, ..., p...). k The output bounding box regressor predicts the regression parameters for the corresponding category c. Both constitute the overall output of the prediction model.
[0060] Step 3: The model training process includes the following steps:
[0061] 3-1) Construct an underwater image recognition model based on a Bayesian neural network, including an initial convolutional layer, followed by a batch normalization layer and a ReLU activation function, and using a max pooling layer for downsampling. Define a Bayesian linear layer to introduce uncertainty. This Bayesian linear layer integrates multimodal prior information and incorporates underwater depth data to help construct the prior distribution.
[0062] 3-2) Load the pre-trained weights and set the initial parameters. The underwater environment dynamic correction factor is fine-tuned in real time based on the depth parameters collected in real time.
[0063] 3-3) Loop within the specified epoch. The specific steps are as follows:
[0064] (1) Sample from the approximate posterior distribution, perform forward propagation to obtain the predicted value and calculate the average likelihood term.
[0065] (2) Calculate the loss function L e (θ). Using a stochastic gradient descent optimizer with momentum and a step learning rate setting, the optimization objective is to minimize L. e (θ).
[0066] (3) Repeat the first two steps until the model converges and save the parameters.
[0067] 3-4) After each training epoch, specifically, an underwater target evaluation system is constructed, including image recognition accuracy, recall, and robustness metrics in complex underwater scenarios. The model performance is evaluated using a validation set. After reaching the preset number of training epochs, all model parameters are saved, and training ends.
[0068] The dataset was trained, and the final mAP curve on the validation set is as follows: Figure 3 As shown.
[0069] Step 4: Predictive analysis based on the trained prediction model includes the following steps:
[0070] 4-1) Collect underwater images and videos as a dataset
[0071] 4-2) Input image dataset, pass it sequentially through network layers, and output the results at the output layer.
[0072] 4-3) Update network parameters and calculate each loss value.
[0073] 4-4) Obtain the final predicted value, and acquire the target classification result and more accurate location information. The recognition effect is as follows: Figure 5 As shown.
[0074] Example 2
[0075] Based on Example 1, referring to Figure 1 A neural network-based underwater image recognition method includes the following steps:
[0076] Step 1: Construct an underwater submarine image dataset;
[0077] Step 2: Construct an underwater image recognition model based on a Bayesian neural network;
[0078] Step 3: Use the constructed dataset to train a deep learning model to obtain an underwater image recognition model;
[0079] Step 4: Based on the trained prediction model, perform predictive analysis on the images collected by the submersible during underwater missions to obtain the image target classification results and uncertainty.
[0080] Step 1: Constructing the underwater image dataset includes the following steps:
[0081] 1-1) Capture keyframes from the acquired underwater target videos to construct a dataset. This dataset includes three categories: g_submarine (green submersible), b_submarine (blue submersible), and black_submarine (black submersible).
[0082] 1-2) Load the images and their annotation information from the dataset, including the target label y and the bounding box position u. x ,u y ,u h ,u w 80% of the images in this dataset were used for training and validation, and 20% were used for testing. The bounding boxes, labels, and additional information of the objects were processed into Tensor format to facilitate subsequent model training.
[0083] The specific procedures for acquiring underwater target videos in step 1-1) above are as follows: the underwater camera of the submersible is placed as close as possible to the object being filmed to reduce the impact of underwater particles on image clarity. When acquiring videos in deep water, the submersible provides additional light. The underwater camera gimbal is equipped with a self-stabilizing model to reduce image blurring caused by the submersible's movement.
[0084] Step 2: Construct an underwater image recognition model based on a Bayesian neural network. For example... Figure 2As shown, the underwater image recognition model based on a Bayesian neural network consists of convolutional layers, batch normalization layers, max pooling layers, Bayesian linear layers, fully connected layers, and activation functions (Rule). In the classification and regression network layers of this structure, we designed a Bayesian layer (Bnn_fc) to introduce uncertainty into the model using its output, followed by two fully connected layers for classification and boundary regression. The parameters of this Bayesian network layer incorporate an underwater environment dynamic correction factor γ to assist in constructing a prior distribution, and this prior distribution follows a Gaussian distribution, satisfying... The true posterior distribution q(ω) is given by a variational distribution q parameterized by parameter α. α The model's generalization ability is enhanced by approximating the difference between the variational distribution and the true posterior by optimizing the KL divergence to minimize this difference. During training, the constructed model is optimized using a stochastic gradient descent optimizer with momentum and a step learning rate setting.
[0085] The objective is to minimize the loss function of the Bayesian neural network.
[0086]
[0087] In the formula, N represents the total number of samples, K represents the number of labels (including the background class), and y i,c p is an indicator variable. i,c Predict the probability that sample i belongs to class c. N pos The number of positive samples. To predict the bounding box regression value, b i q represents the true bounding box regression value. α (ω) represents the variational distribution, and p(ω) is the prior probability of the parameter. KL is the divergence loss, and λ is a hyperparameter that balances the classification loss and regression loss. This loss function includes the losses from classification and bounding box regression, as well as the KL divergence.
[0088] For each image category, output a probability distribution, i.e., p = (p0, ..., p...). k The output bounding box regressor predicts the regression parameters for the corresponding category c. Both constitute the overall output of the prediction model.
[0089] Step 3: The model training process includes the following steps:
[0090] 3-1) Construct an underwater image recognition model based on a Bayesian neural network, including an initial convolutional layer, followed by a batch normalization layer and a ReLU activation function, and using a max pooling layer for downsampling. Define a Bayesian linear layer to introduce uncertainty. This Bayesian linear layer integrates multimodal prior information and incorporates underwater depth data to help construct the prior distribution.
[0091] 3-2) Load the pre-trained weights and set the initial parameters. The underwater environment dynamic correction factor is fine-tuned in real time based on the depth parameters collected in real time.
[0092] 3-3) Loop within the specified epoch. The specific steps are as follows:
[0093] (1) Sample from the approximate posterior distribution, perform forward propagation to obtain the predicted value and calculate the average likelihood term.
[0094] (2) Calculate the loss function L e (θ). Using a stochastic gradient descent optimizer with momentum and a step learning rate setting, the optimization objective is to minimize L. e (θ).
[0095] (3) Repeat the first two steps until the model converges and save the parameters.
[0096] 3-4) After each training epoch, specifically, an underwater target evaluation system is constructed, including image recognition accuracy, recall, and robustness metrics in complex underwater scenarios. The model performance is evaluated using a validation set. After reaching the preset number of training epochs, all model parameters are saved, and training ends.
[0097] The dataset was trained, and the final mAP curve on the validation set is as follows: Figure 3 As shown.
[0098] Step 4: Predictive analysis based on the trained prediction model includes the following steps:
[0099] 4-1) Collect underwater images and videos as a dataset
[0100] 4-2) Input image dataset, pass it sequentially through network layers, and output the results at the output layer.
[0101] 4-3) Update network parameters and calculate each loss value.
[0102] 4-4) Obtain the final predicted value, and acquire the target classification result and more accurate location information. The recognition effect is as follows: Figure 6 .
[0103] Example 3
[0104] Building upon Example 2, this invention compares its method with two other commonly used underwater image recognition methods in constructing an underwater submarine dataset. This invention employs a series of evaluation metrics commonly used in target detection tasks, including mAP, mAP50-95, AR50-95 (maxdets = 10), and AR50-95 (maxdets = 100), to objectively evaluate the proposed method. Higher values for these metrics indicate better underwater target detection performance. Specific evaluation results are presented in Table 1 below:
[0105]
[0106] As can be seen from Table 1, the method of the present invention achieved 99.0% mAP, 61.2% mAP50-95, 68.2% AR50-95, and 68.2% AR50-95 on the self-constructed underwater submarine dataset, which is superior to other conventional image recognition algorithms in the field of underwater image recognition.
[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An underwater image recognition method based on a Bayesian neural network, characterized in that, Includes the following steps: S1. Construct an underwater submarine image dataset; S2. Construct an underwater image recognition model based on a Bayesian neural network; S3. Use the constructed dataset to train a deep learning model to obtain a Bayesian underwater image recognition model; The model training process in step S3 includes the following steps: S31. Construct an underwater image recognition model based on a Bayesian neural network, including an initial convolutional layer, followed by a batch normalization layer and a ReLU activation function. Use a max pooling layer for downsampling. Define a Bayesian linear layer to introduce uncertainty. This Bayesian linear layer integrates multimodal prior information and incorporates underwater depth data to assist in constructing a prior distribution. S32. Load the pre-trained weights and set the initial parameters. The underwater environment dynamic correction factor is fine-tuned in real time based on the depth parameters collected in real time. ; S33. Loop within the specified epoch, the specific steps of which are as follows: (1) Sample from the approximate posterior distribution, perform forward propagation to obtain the predicted value and calculate the average likelihood term; (2) Calculate the loss function Using a stochastic gradient descent optimizer with momentum and a step learning rate setting, the optimization objective is to minimize... ; (3) Repeat the first two steps until the model converges and save the parameters; S34. After each training round, construct an underwater target evaluation system, including image recognition accuracy, recall, and robustness indicators in complex underwater scenarios. Use the validation set to evaluate model performance. After reaching the preset number of training rounds, save all model parameters and end training. S4. Based on the trained prediction model, perform predictive analysis on the images collected by the submersible when performing underwater tasks, and obtain the image target classification results and uncertainty.
2. The underwater image recognition method based on Bayesian neural networks according to claim 1, characterized in that, The construction of the underwater image dataset in step S1 includes the following steps: S11: Collect video data using an underwater camera and preprocess the video data to obtain datasets corresponding to three types: g_submarine (green submersible), b_submarine (blue submersible), and black_submarine (black submersible). S12: Load the images and their annotation information from the dataset, including the target label y and the bounding box location. Using 80% of the images in this dataset for training and validation, and 20% for testing, we processed the bounding boxes, labels, and additional information of the objects into Tensor format.
3. The underwater image recognition method based on Bayesian neural networks according to claim 1, characterized in that, In step S2, the underwater image recognition model based on a Bayesian neural network consists of convolutional layers, batch normalization layers, max pooling layers, Bayesian linear layers, fully connected layers, and activation functions (Rule). In the classification and regression network layer of this network structure, the output of the Bayesian layer introduces uncertainty into the model. Two fully connected layers are then added for classification and boundary regression. The parameters of this Bayesian network layer incorporate dynamic correction factors related to the underwater environment. Auxiliary constructs a prior distribution, and the prior distribution follows a Gaussian distribution, satisfying the following conditions: True posterior distribution Use a parameter Parameterized variational distribution To approximate this, the model's generalization ability is enhanced by minimizing the difference between the variational distribution and the true posterior by optimizing the KL divergence. During training, the constructed model is optimized using a stochastic gradient descent optimizer with momentum and a step learning rate setting. The objective is to minimize the loss function of the Bayesian neural network. ； In the formula, N represents the total number of samples, and K represents the number of labels, including the background class. As an indicator variable, For the sample The probability of predicting it as category c. The number of positive samples. To predict the bounding box regression value, The true bounding box regression value, It is a variational distribution. Let be the prior probability of the parameter, and KL be the divergence loss. It is a hyperparameter that adjusts the balance between classification loss and regression loss. This loss function includes the loss for classification and bounding box regression, as well as the KL divergence. For each image, the identified category outputs a probability distribution, i.e. Output the regression parameters of the corresponding class c predicted by the bounding box regressor. The two together constitute the overall output of the prediction model.
4. The underwater image recognition method based on Bayesian neural networks according to claim 1, characterized in that, Step S4, which involves predictive analysis based on the trained prediction model, includes the following steps: S41. Collect underwater images and videos as a dataset; S42. Input image dataset, pass it sequentially through network layers, and output the results at the output layer; S43. Update network parameters and calculate each loss value; S44. Obtain the final predicted value and acquire the target classification result and location information.