Turbid underwater image enhancement method

Abstract

The invention relates to the technical field of image enhancement methods, provides a turbid underwater image enhancement method, and constructs a database updating method in a training process to solve the problem of lack of paired real underwater turbid data sets, and constructs an image enhancement algorithm based on an attention mechanism. The image is further enhanced through a detail restoration network guided by edge information, enhancement of a turbid underwater image with low contrast and definition is achieved, close-range observation can be achieved in a turbid water area, and the method comprises the following steps of S1, collecting turbid image data with labels, S2, making a data set, and establishing a training set and a test set. S3, establishing an image enhancement model based on an attention mechanism, S4, training by using the training set, and updating a training database, and S5, reasoning a turbid image by using the trained model.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement methods, and in particular to a turbid underwater image enhancement method. Background Art

[0002] Underwater close-range observation equipment based on optical visual imaging is widely used in marine engineering. However, due to the absorption and scattering of light in water, people's visible distance underwater is short, especially in turbid water, where the visible distance is about half a meter, which seriously limits the underwater detection performance of optical imaging equipment.

[0003] With the rapid development of deep models, data-driven methods are being used in more and more domains. Researchers have achieved good results through traditional CNN models. As the Transformer model has shown excellent performance in many computer vision tasks, the Transformer model has been introduced into the task and achieved good results. There are also some methods that are improved based on GAN to narrow the domain gap between underwater images and images in the air to improve the restoration performance. However, there is currently a lack of labeled turbid image data, which limits the ability of deep learning to enhance turbid images. Therefore, the present invention discloses a turbid underwater image enhancement method for solving the technical problem of difficult close-range observation of turbid water. A method for updating the database during training is constructed to solve the problem of lack of paired real underwater turbid data sets, and an image enhancement algorithm based on an attention mechanism is constructed. The image is further enhanced through a detail restoration network guided by edge information, thereby achieving turbid underwater image enhancement with low contrast and clarity, so that close-range observation can be achieved in turbid waters. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the shortcomings of the prior art, the present invention provides a method for enhancing turbid underwater images, which constructs a method for updating the database during training to solve the problem of lack of paired real underwater turbid data sets, and constructs an image enhancement algorithm based on an attention mechanism. The image is further enhanced through a detail restoration network guided by edge information, thereby achieving enhancement of turbid underwater images with low contrast and clarity, so that close-range observation can be achieved in turbid waters.

[0006] (II) Technical solution

[0007] To achieve the above object, the present invention provides the following technical solution: a turbid underwater image enhancement method, mainly comprising the following steps:

[0008] Step S1: collecting labeled turbid image data;

[0009] Step S2: Create a data set, establish a training set and a test set;

[0010] Step S3: Establish an image enhancement model based on attention mechanism;

[0011] Step S4: using the training set to perform training and update the training database;

[0012] Step S5: Use the trained model to infer the turbid image.

[0013] Preferably, the step S1 comprises:

[0014] Step S11: Fill a water tank placed in a dark room with a certain volume of water;

[0015] Step S12: Arranging a first physical scene on the loading platform of the water tank;

[0016] Step S13: The camera is placed directly in front of the water tank;

[0017] Step S14: adjusting the illumination intensity of the lighting device to the minimum;

[0018] Step S15: Move the stage from the closest position to the camera to the farthest position, taking pictures at 5 cm intervals;

[0019] Step S16: adjust the light intensity and repeat step S15;

[0020] Step S16: Change the scene and repeat steps S14 to S16;

[0021] Step S17: After all scenes are operated, the shooting is ended and the collection is completed.

[0022] Preferably, in step S13, the camera is connected to the computer via a network cable, and the computer controls the camera and saves the captured images.

[0023] Preferably, in step S12, the loading platform is composed of two U-shaped parts, an acrylic loading platform, four thick steel wires and connecting parts, wherein the U-shaped part is buckled on the upper edge of the water tank, and the thick steel wires and the connecting parts are used for rigid connection between the U-shaped part and the acrylic loading platform.

[0024] Preferably, the lighting device in step S14 includes 5 levels of brightness.

[0025] Preferably, the scene in step S16 includes stones, standard color cards, building blocks of different regular shapes, simulated water plants, small rockery, simulated corals and other common underwater objects.

[0026] Preferably, the step S2 comprises:

[0027] Step S21: using current advanced methods to infer the collected turbid images;

[0028] Step S22: recruit multiple volunteers to vote and select the inference result with the best enhancement effect as the original label of the turbid image;

[0029] Step S23: extracting data samples from a public labeled underwater image dataset. According to experience, the number of samples is twice the number of turbid underwater samples.

[0030] Step S24: The extracted samples and a portion of randomly extracted turbid images are mixed as a training set, and the remaining turbid images are used as a test set.

[0031] Preferably, the image enhancement model in step S3 is composed of an encoder network and a decoder network, and the output features of multiple different scales of the encoder are enhanced using an attention enhancement sub-network and then connected to the decoder;

[0032] The encoder consists of 6 downsampling layers and X attention layers. The first downsampling layer consists of a 5×5 convolution, an activation function, and a normalization layer in series. The other downsampling layers are residual modules, and the attention layer is a CBAM attention module. The decoder consists of 6 upsampling layers, each of which consists of an activation function, a normalization layer, and a 3×3 deconvolution in series.

[0033] Preferably, step S4 comprises:

[0034] Step S41: preprocessing the training set samples;

[0035] Step S42: After one round of training is completed, the loss function is calculated and the weights are updated;

[0036] Step S43: Calculate the image quality index value of each underwater image in the training set;

[0037] Step S44: Compare the calculated value with the image quality index value of the label;

[0038] Step S45: Replace the original label of the inference image with a large calculated value as a new label, and update the training data;

[0039] Step S46: Repeat steps S42 to S45 until the training is completed.

[0040] Preferably, the step S41 includes:

[0041] Step S411: scaling the size of the training sample;

[0042] Step S412: normalizing the training samples;

[0043] Step S413: Input the training samples into the network.

[0044] (III) Beneficial effects

[0045] Compared with the prior art, the present invention provides a method for enhancing turbid underwater images, which has the following beneficial effects:

[0046] 1. In the present invention, a method for updating the database during training is constructed to solve the problem of lack of paired real underwater turbidity data sets, and an image enhancement algorithm based on the attention mechanism is constructed.

[0047] 2. In the present invention, the image is further enhanced by a detail restoration network guided by edge information, thereby achieving enhancement of turbid underwater images with low contrast and clarity, so that close-range observation can be achieved in turbid waters. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of underwater image enhancement of the present invention;

[0049] Figure 2 It is a schematic diagram of the image enhancement model based on the attention mechanism of the present invention. DETAILED DESCRIPTION

[0050] In order to deepen the understanding of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings. The embodiments are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.

[0051] Example

[0052] See also Figure 1-Figure 2 , a turbid image enhancement method guided by image attenuation coefficient, comprising:

[0053] Step S1: Collect labeled turbid image data.

[0054] Specifically, step S1 includes:

[0055] Step S11: Fill a water tank placed in a dark room with a certain volume of water;

[0056] Step S12: Arranging a first physical scene on the loading platform of the water tank;

[0057] Step S13: The camera is placed directly in front of the water tank;

[0058] Step S14: adjusting the illumination intensity of the lighting device to the minimum;

[0059] Step S15: Move the stage from the closest position to the camera to the farthest position, taking pictures at 5 cm intervals;

[0060] Step S16: adjust the light intensity and repeat step S15;

[0061] Step S16: Change the scene and repeat steps S14 to S16;

[0062] Step S17: After all scenes are operated, the shooting is ended and the collection is completed.

[0063] In the embodiment of the present application, the camera in S13 is connected to the computer via a network cable, and the computer controls the camera and saves the captured images.

[0064] In an embodiment of the present application, the loading platform in step S12 is composed of two U-shaped parts, an acrylic loading platform, four thick steel wires and connecting parts, wherein the U-shaped part is buckled on the upper edge of the water tank, and the thick steel wires and the connecting parts are used for rigid connection between the U-shaped part and the acrylic loading platform.

[0065] In the embodiment of the present application, the lighting device in step S14 includes 5 levels of brightness.

[0066] In the embodiment of the present application, the scene in step S16 includes common underwater objects such as stones, standard color cards, building blocks of different regular shapes, simulated water plants, small rockery, simulated corals, etc.

[0067] Step S2: Create training set and test set;

[0068] Specifically, step S2 includes:

[0069] Step S21: using current advanced methods to infer the collected turbid images;

[0070] Step S22: recruit multiple volunteers to vote and select the inference result with the best enhancement effect as the original label of the turbid image;

[0071] Step S23: extracting data samples from a public labeled underwater image dataset. According to experience, the number of samples is twice the number of turbid underwater samples.

[0072] Step S24: Mix the extracted samples and a portion of randomly extracted turbid images as a training set, and use the remaining turbid images as a test set;

[0073] Step S3: Establish an image enhancement model based on attention mechanism;

[0074] In the embodiment of the present application, the image enhancement model is composed of an encoder network and a decoder network, and the output features of multiple scales of the encoder are enhanced by using an attention enhancement sub-network and then connected to the decoder;

[0075] In the embodiment of the present application, the encoder is composed of 6 downsampling layers and X attention layers. The first downsampling layer is composed of a 5×5 convolution, an activation function, and a normalization layer in series. The other downsampling layers are residual modules, and the attention layer is a CBAM attention module: the decoder is 6 upsampling layers, each of which is composed of an activation function, a normalization layer, and a 3×3 deconvolution in series. Figure 2 shown.

[0076] Step S4: using the training set to perform training and update the training database;

[0077] Specifically, step S4 includes:

[0078] Step S41: preprocessing the training set samples;

[0079] Step S42: After one round of training is completed, the loss function is calculated and the weights are updated;

[0080] Step S43: Calculate the image quality index value of each underwater image in the training set;

[0081] Step S44: Compare the calculated value with the image quality index value of the label;

[0082] Step S45: Replace the original label of the inference image with a large calculated value as a new label, and update the training data;

[0083] Step S46: Repeat steps S42 to S45 until the training is completed.

[0084] In the embodiment of the present application, the S41 includes:

[0085] Step S411: scaling the size of the training sample;

[0086] Step S412: normalizing the training samples;

[0087] Step S413: Input the training samples into the network.

[0088] In the embodiment of the present application, the loss function of step S42 is L pix =||BG(A)||1, B is the label of input A, G(A) is the model inference image.

[0089] Step S5: Use the trained model to infer the turbid image

[0090] The loss function of step S42 is L pix =||BG(A)||1, B is the label of input A, G(A) is the model inference image.

[0091] The above specific implementation methods are only for illustrating the technical concept and structural features of the present invention, and the purpose is to enable relevant persons familiar with this technology to implement it. However, the above content does not limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should fall within the protection scope of the present invention.

Claims

1. A turbid underwater image enhancement method, characterized in that: The main steps include: Step S1: collecting labeled turbid image data; Step S2: Create a data set, establish a training set and a test set; Step S3: Establish an image enhancement model based on attention mechanism; Step S4: using the training set to perform training and update the training database; Step S5: Use the trained model to infer the turbid image.

2. The method for enhancing turbid underwater images according to claim 1, characterized in that: The step S1 comprises: Step S11: Fill a water tank placed in a dark room with a certain volume of water; Step S12: Arranging a first physical scene on the loading platform of the water tank; Step S13: The camera is placed directly in front of the water tank; Step S14: adjusting the illumination intensity of the lighting device to the minimum; Step S15: Move the stage from the closest position to the camera to the farthest position, taking pictures at 5 cm intervals; Step S16: Adjust the light intensity and repeat step S15; Step S16: Change the scene and repeat steps S14 to S16; Step S17: After all scenes are operated, the shooting is ended and the collection is completed.

3. A turbid underwater image enhancement method according to claim 2, characterized in that: In step S13, the camera is connected to the computer via a network cable, and the computer controls the camera and saves the captured images.

4. The method for enhancing turbid underwater images according to claim 3, characterized in that: In step S12, the loading platform is composed of two U-shaped parts, an acrylic loading platform, four thick steel wires and connecting parts, wherein the U-shaped part is buckled on the upper edge of the water tank, and the thick steel wires and the connecting parts are used to rigidly connect the U-shaped part and the acrylic loading platform.

5. The method for enhancing turbid underwater images according to claim 4, characterized in that: In step S14, the lighting device includes 5 levels of brightness.

6. A turbid underwater image enhancement method according to claim 5, characterized in that: The scene in step S16 includes stones, standard color cards, building blocks of different regular shapes, simulated water plants, small rockery, simulated corals and other common underwater objects.

7. A turbid underwater image enhancement method according to claim 6, characterized in that: The step S2 comprises: Step S21: using current advanced methods to infer the collected turbid images; Step S22: recruit multiple volunteers to vote and select the inference result with the best enhancement effect as the original label of the turbid image; Step S23: extracting data samples from a public labeled underwater image dataset. According to experience, the number of samples is twice the number of turbid underwater samples. Step S24: The extracted samples and a portion of randomly extracted turbid images are mixed as a training set, and the remaining turbid images are used as a test set.

8. The method for enhancing turbid underwater images according to claim 7, characterized in that: The image enhancement model in step S3 is composed of an encoder network and a decoder network, and the output features of multiple different scales of the encoder are enhanced by using an attention enhancement sub-network and then connected to the decoder; The encoder consists of 6 downsampling layers and X attention layers. The first downsampling layer consists of a 5×5 convolution, an activation function, and a normalization layer in series. The other downsampling layers are residual modules, and the attention layer is a CBAM attention module. The decoder consists of 6 upsampling layers, each of which consists of an activation function, a normalization layer, and a 3×3 deconvolution in series.

9. The method for enhancing turbid underwater images according to claim 8, characterized in that: The step S4 comprises: Step S41: preprocessing the training set samples; Step S42: After one round of training is completed, the loss function is calculated and the weights are updated; Step S43: Calculate the image quality index value of each underwater image in the training set; Step S44: Compare the calculated value with the image quality index value of the label; Step S45: Replace the original label of the inference image with a large calculated value as a new label, and update the training data; Step S46: Repeat steps S42 to S45 until the training is completed.

10. The method for enhancing turbid underwater images according to claim 9, characterized in that: The step S41 comprises: Step S411: scaling the size of the training sample; Step S412: normalizing the training samples; Step S413: Input the training samples into the network.

Images