Underwater weak sample image enhancement and detection method and device for underwater vehicle

Abstract

The embodiment of the invention provides an underwater weak sample image enhancement and detection method and device for an underwater vehicle, and the method comprises the steps: carrying out the sampling of an optical camera, obtaining an underwater target image and an air original target image, and carrying out the data preprocessing of the underwater target image and the original target image; inputting the underwater image into a pre-trained feature extraction network model, and performing feature extraction on the underwater image; inputting the features of the underwater target image and the air original target image into an underwater image enhancement model based on a generative adversarial network, and performing dewatering processing through the underwater image enhancement model to obtain an enhanced pseudo underwater optical image; in a test stage, an underwater target image is sent into an underwater image enhancement model based on the generative adversarial network, dehydration processing is performed through the underwater image enhancement model, an enhanced high-quality underwater reconstruction sample is output, and the enhanced high-quality underwater reconstruction sample is input into a lightweight target detection model for detection.

Description

Technical Field

[0001] This document relates to the field of image processing technologies, and particularly to an underwater weak sample image enhancement and detection method and device for an underwater vehicle. Background Art

[0002] Underwater target detection technology is an important supporting technology for underwater equipment environmental perception, target recognition, and tracking, and is one of the key technologies to enhance the intelligence of underwater equipment. However, due to the complexity of the underwater environment, the development of intelligent underwater target recognition technology is relatively slow. The domestic and foreign underwater recognition technologies mainly achieve through two different imaging technologies, acoustic and optical, and identify and classify by extracting target characteristics from acoustic or optical images. Although extensive research has been carried out on the target recognition technology of underwater unmanned equipment at home and abroad, many problems and challenges still exist. Due to the complexity of the underwater environment, acoustic and optical characteristics, underwater image data are mostly weak samples, which limits the application of underwater unmanned equipment in military and civilian fields. Specifically, it is reflected in: (1) Low image quality. The underwater environment is complex, the lighting conditions are limited, and there is noise in the detection payload, making the underwater image data have defects such as uneven lighting, low contrast, and unclear contours. (2) High repeatability. Although a large amount of underwater sample data is obtained, the information repeatability is relatively high, and the truly distinguishable samples available for recognition are relatively few. (3) Insufficient samples. The complex sea conditions, underwater terrain, and various organisms in the ocean make the underwater data have a long-tail problem, that is, there are categories with only a small number of samples, resulting in extremely unbalanced samples between categories, making the generalization ability of the trained model poor and prone to overfitting.

[0003] The image recognition task of underwater unmanned equipment faces the problem of insufficient data. How to effectively improve the quality of underwater data and alleviate the impact of weak samples on the accuracy of target recognition is a key problem that needs to be solved urgently, which has important strategic significance for underwater target intelligent recognition technology and has great military and civilian value. Summary of the Invention

[0004] The purpose of the present invention is to provide an underwater weak sample image enhancement and detection method and device for an underwater vehicle, aiming to solve the above problems in the prior art.

[0005] The present invention provides an underwater weak sample image enhancement and detection method for an underwater vehicle, including:

[0006] Sampling an underwater target image and an original target image through an optical camera, and performing data preprocessing on the underwater target image and the original target image;

[0007] Input the underwater image into a pre-trained feature extraction network model, and extract features from the underwater image through the feature extraction network model; input the features of the underwater target image and the original target image into an underwater image enhancement model based on a generative adversarial network, and perform water treatment removal through the underwater image enhancement model to obtain an enhanced pseudo-underwater optical image;

[0008] In the test stage, input the underwater target image into an underwater image enhancement model based on a generative adversarial network, perform water treatment removal through the underwater image enhancement model, output an enhanced high-quality underwater reconstruction sample, and input the enhanced high-quality underwater reconstruction sample into a lightweight target detection model for detection.

[0009] The present invention provides an underwater weak sample image enhancement and detection device for an underwater vehicle, including:

[0010] An image acquisition module, configured to sample an underwater target image and an original target image through an optical camera, and perform data preprocessing on the underwater target image and the original target image;

[0011] An underwater image enhancement module, configured to input the underwater image into a pre-trained feature extraction network model, and extract features from the underwater image through the feature extraction network model; input the features of the underwater target image and the original target image into an underwater image enhancement model based on a generative adversarial network, and perform water treatment removal through the underwater image enhancement model to obtain an enhanced pseudo-underwater optical image;

[0012] An underwater image detection module, configured to, in the test stage, input the underwater target image into an underwater image enhancement model based on a generative adversarial network, perform water treatment removal through the underwater image enhancement model, output an enhanced high-quality underwater reconstruction sample, and input the enhanced high-quality underwater reconstruction sample into a lightweight target detection model for detection.

[0013] An embodiment of the present invention further provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the computer program is executed by the processor, the steps of the above-mentioned underwater weak sample image enhancement and detection method for an underwater vehicle are implemented.

[0014] An embodiment of the present invention further provides a computer-readable storage medium, on which an implementation program for information transmission is stored, and when the program is executed by a processor, the steps of the above-mentioned underwater weak sample image enhancement and detection method for an underwater vehicle are implemented.

[0015] By adopting the embodiment of the present invention, an underwater image enhancement model based on a generative adversarial network is used to generate pseudo-optical samples that conform to the distribution of real underwater image samples, which helps to alleviate the problem of weak samples caused by the complex underwater environment and improve the subsequent underwater target detection performance. And through the attenuation compensation attention mechanism and the brightness channel attention mechanism, the red and blue attenuation of the underwater image is compensated to enhance the image details and contrast. The generated pseudo-image conforms to the distribution of image samples in the air through the sub-discriminator. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in one or more embodiments of the present specification or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0017] Figure 1 is a flowchart of the underwater weak sample image enhancement and detection method for an underwater vehicle according to an embodiment of the present invention;

[0018] Figure 2 is a schematic diagram of the principle of the detailed processing of the underwater weak sample image enhancement and detection method for an underwater vehicle according to an embodiment of the present invention;

[0019] Figure 3 is a schematic diagram of the underwater weak sample image enhancement and detection device for an underwater vehicle according to an embodiment of the present invention;

[0020] Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present specification, the following will clearly and completely describe the technical solutions in one or more embodiments of the present specification with reference to the drawings in one or more embodiments of the present specification. Obviously, the described embodiments are only some embodiments of the present specification, rather than all embodiments. Based on one or more embodiments of the present specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this document.

[0022] Method Embodiment

[0023] According to an embodiment of the present invention, there is provided an underwater weak sample image enhancement and detection method for an underwater vehicle, Figure 1It is a flowchart of the underwater weak sample image enhancement and detection method for an underwater vehicle according to an embodiment of the present invention. As Figure 1 shown, the underwater weak sample image enhancement and detection method for an underwater vehicle according to an embodiment of the present invention specifically includes:

[0024] Step S101, sampling an underwater target image and an original target image through an optical camera, and performing data preprocessing on the underwater target image and the original target image; specifically including: screening the underwater target image and the original target image, annotating target detection data, and dividing the dataset.

[0025] Step S102, inputting the underwater image into a pre-trained feature extraction network model, and extracting features of the underwater image through the feature extraction network model; inputting the features of the underwater target image and the original target image into an underwater image enhancement model based on a generative adversarial network, and performing water removal processing through the underwater image enhancement model to obtain an enhanced pseudo-underwater optical image; specifically including:

[0026] Performing image enhancement processing on the underwater target image through the generative model G of the underwater image enhancement model to generate an enhanced pseudo-underwater image; distinguishing the pseudo-underwater image generated by the generative model G of the underwater image enhancement model from the underwater target image sample through the discriminant module D of the underwater image enhancement model to judge the authenticity of the input image; distinguishing the pseudo-underwater image generated by the generative model G of the underwater image enhancement model from the random region of the original target image through the sub-discriminant module Dsub of the underwater image enhancement model to judge whether it is a real original target image or an enhanced pseudo-underwater optical image output, and finally obtaining an enhanced pseudo-underwater optical image.

[0027] Among them, performing image enhancement processing on the underwater target image through the generative model G of the underwater image enhancement model to generate an enhanced pseudo-underwater image specifically includes:

[0028] Adopting an attenuation compensation attention mechanism A shown in Formula 1 - Formula 5 by the generative model G of the underwater image enhancement model c and a brightness channel attention mechanism A shown in Formula 6 - Formula 7 l , and adopting an attention-guided U-Net to extract multi-level features from different depth layers and synthesize an enhanced pseudo-underwater image by using multi-scale context information:

[0029] C g(x) =0 Formula 1;

[0030]

[0031] D c =I c -Gc *I c Formula 4;

[0032] A c(x) = f(C r，g，b(x) + D c ) Formula 5;

[0033] Wherein, I r , I g , I b represents the red, green, and blue channels of the image I, and the values of the channels are normalized to [0, 1]. represents I r , I g , I b 's average value. The attenuation compensation attention mechanism improves the restoration effect of red and blue information. C g(x) represents the green channel value, C r(x) represents the red channel value, C b(x) represents the blue channel value, D c represents the detail image, that is, the image G obtained by subtracting the image after Gaussian kernel blurring from the input image c *I c is obtained. * represents the convolution operation, f represents the fully connected layer, and then the normalization operation. C r，g，b(x) represents the red, green, and blue channel values;

[0034]

[0035] A l (i, j) = g(L B (i, j) - u B ) Formula 7;

[0036] Wherein, L B (i, j) represents the grayscale matrix of the image luminance channel L, W represents the width of the image, H represents the height of the image, u B represents the average value of the grayscale values of the luminance channel L, g represents the fully connected layer and the normalization operation, and A 1 is the attention weight map of the luminance channel.

[0037] Wherein, the sub-discriminant module Dsub of the underwater image enhancement model is used to distinguish the randomly selected regions of the pseudo-underwater image generated by the generation model G and the original target image, and determine whether it is the real original target image or the pseudo-underwater optical image output after enhancement. Finally, obtaining the enhanced pseudo-underwater optical image specifically includes:

[0038] The discriminant module D is used to distinguish the samples generated by the generation model G from the underwater image samples and determine the authenticity of the input image.

[0039] The sub-discriminator module Dsub of the underwater image enhancement model randomly crops a patches from the output image pseudo-underwater optical image I′ and the air image I S and distinguishes between the randomly cropped regions of the pseudo-underwater image generated by the generation model G and the original target image based on the loss functions shown in Equations 8, 9, 10, and 11 to determine whether it is the real original target image or the enhanced pseudo-underwater optical image, and finally obtains the enhanced pseudo-underwater optical image:

[0040]

[0041] where x r represents the sampling region of the real underwater image, x f represents the sampling region of the pseudo-underwater optical image I′, x s represents the sampling region of the air image I S L D represents the discriminator loss function, represents the sub-discriminator loss function, L G , represents the corresponding loss function of the generator, x r ~P raal represents the distribution conforming to the real underwater image, x f ~P fake represents the distribution of the pseudo-underwater optical image, x s ~P air represents the distribution of the air optical image.

[0042] Step S103, in the test phase, input the underwater target image into the underwater image enhancement model based on the generative adversarial network, perform water removal processing through the underwater image enhancement model, output the enhanced high-quality underwater reconstruction sample, and input the enhanced high-quality underwater reconstruction sample into the lightweight target detection model for detection. The lightweight target detection model is: the YOLO series deep learning detection model.

[0043] It can be seen from the above processing that the embodiment of the present invention obtains the underwater target image and the original target image in real time as the images to be converted, and improves the image quality through preprocessing; converts the image style of the images to be converted through the pre-trained generative adversarial model based on the graph method to obtain the reconstructed image of the underwater style target image. Finally, converts the image style of the images to be converted through the pre-trained generative adversarial model based on the graph method to obtain the reconstructed image of the underwater style target image.

[0044] The underwater weak sample image enhancement and detection method for an underwater vehicle according to an embodiment of the present invention is capable of enhancing the underwater target image and outputting high-quality underwater reconstruction samples. The underwater image enhancement model based on a generative adversarial network designed in the embodiment of the present invention generates pseudo-optical samples that conform to the distribution of real underwater image samples, which helps to alleviate the weak sample problem caused by the complex underwater environment and improve the subsequent underwater target detection performance. And through the attenuation compensation attention mechanism and the brightness channel attention mechanism, the red and blue attenuation of the underwater image is compensated to enhance the image details and contrast. The generated pseudo-image conforms to the distribution of air image samples through the sub-discriminator. The underwater weak sample image enhancement and detection method for an underwater vehicle according to an embodiment of the present invention can effectively alleviate the weak sample problem underwater and efficiently detect underwater targets.

[0045] The above technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0046] The purpose of the embodiment of the present invention is to provide an underwater weak sample image enhancement and detection method for an underwater vehicle, and design an underwater image enhancement model based on a generative adversarial network to generate pseudo-optical samples that conform to the distribution of real underwater image samples. By using unpaired underwater and air samples to train the model, the trained model is capable of outputting high-quality underwater samples after water treatment. The underwater weak sample image enhancement and detection method for an underwater vehicle includes the following steps:

[0047] Figure 2 The flowchart of an underwater weak sample image enhancement and detection method for an underwater vehicle according to an embodiment of the present invention is shown. As Figure 2 shown, the underwater weak sample image enhancement and detection method for an underwater vehicle includes the following steps:

[0048] 1) The underwater target image and the air target image (i.e., the above-mentioned original target image) are sampled by an optical camera, and the images suitable for subsequent processing are obtained through data preprocessing. The underwater and original target image datasets are sorted and labeled, and divided into the underwater target image training set D train , the validation set D val , the test set D test and the air target image training set S train .

[0049] 2) The image samples I and I train in the underwater target image training set D train and the air target image training set S S are sent into the underwater image enhancement model based on a generative adversarial network for water treatment to obtain the enhanced pseudo-underwater optical image I'; as Figure 2As shown, the graph model-based generative adversarial model includes a generative model G, a discriminant module D, and a sub-discriminant module Dsub. Among them,

[0050] The generative model G is used to perform image enhancement processing on the original underwater image to generate an enhanced pseudo-underwater image. The generative model G adopts an attention-guided U-Net. By extracting multi-level features from different depth layers and using multi-scale context information to synthesize high-quality images.

[0051] A decay compensation attention mechanism A c and a luminance channel attention mechanism A l are designed. In underwater images, the green channel is relatively better preserved underwater, while the red light with a longer wavelength is more likely to be lost, and the red and blue channels, especially the red channel, are severely attenuated. In order to effectively restore the information of the red and blue channels and obtain an underwater image closer to the true color, a decay compensation attention mechanism is introduced to enhance the compensation for the attenuation of the red and blue channels. The compensation is proportional to the difference between the green channel and the average value of the red channel. Under the gray world assumption (the average values of all channels are the same before attenuation), this difference reflects the difference / imbalance between the attenuation of the red and blue channels and the green channel. At the same time, the compensation of the red channel can only be performed in highly attenuated areas, that is, the red enhancement mainly affects the pixels with smaller red channel values and should not be transferred to the areas where the red channel information is still significant, avoiding the red appearance introduced by the gray world algorithm in overexposed areas. Taking the RGB color channels of the input image and normalizing them to [0,1], the red and blue decay compensation expressions are as follows:

[0052] C g(x) = 0 (1)

[0053]

[0054] where I r , I g , I b represent the red, green, and blue channels of image I, and the values of the channels are normalized to within [0,1]. represents the average value of I r , I g , I b , and the decay compensation attention mechanism improves the recovery effect of red and blue information.

[0055] Considering the detail attenuation loss caused by forward scattering, the expression of the decay compensation attention weight map A c(x) is as follows:

[0056] D c = I c - G c * I c (4)

[0057] A c(x) = f(C r,g,b(x) + D c )(5)

[0058] where D c represents the detail image, that is, the image G after Gaussian kernel blurring is subtracted from the input image c * I c is obtained, * represents the convolution operation, f represents the fully connected layer, and then the normalization operation.

[0059] A luminance channel attention mechanism A l , which uses the mean value to enhance the contrast of the L channel of the image. Specifically, the L channel gray matrix of the image luminance channel is L B (x, y), and the mean value is calculated. This can be expressed as:

[0060]

[0061] The L channel value in the input image block is subtracted by the mean value to obtain the high-frequency component, and the contrast of the input image is improved by appropriately enhancing the high-frequency component. An enhancement control factor is introduced for the high-frequency component. The enhancement process is defined as:

[0062] A l (i, j) = g(L B (i, j) - u B ), (7)

[0063] g represents the fully connected layer and the normalization operation. A l serves as the luminance channel attention weight map.

[0064] The attention map is adapted to each feature map size through the fully connected layer and multiplied by all U-Net intermediate feature maps and the output image.

[0065] The discriminant module D described above is used to distinguish the samples generated by the generation model G from the underwater image samples and judge the authenticity of the input image.

[0066] The sub-discriminant module Dsub described above is used to distinguish the samples generated by the generation model G from the random regions of the air image samples and judge whether it is a real air image sample or a pseudo-underwater image output after enhancement.

[0067] Randomly crop 5 patches from the output image pseudo-underwater optical image I' and the air image I S . The loss function is as follows:

[0068]

[0069] where, xr Denotes the sampling area of the real underwater image, x f Denotes the sampling area of the pseudo-underwater optical image I′, x s Denotes the air image I S Sampling area, L D Denotes the discriminator loss function, Denotes the sub-discriminator loss function, L G , Denotes the corresponding loss function of the generator, x r ~P real Denotes conforming to the real underwater image distribution, x f ~P fake Denotes the pseudo-underwater optical image distribution, x s ~P air Denotes the air optical image distribution.

[0070] The described sub-discriminator module Dsub is used to distinguish the samples generated by the generation model G from the random regions of the air image samples, and determine whether it is a real air image sample or a pseudo-underwater image output after enhancement.

[0071] 3) In the test stage, the original underwater image to be processed is sent into the underwater image enhancement model based on the generative adversarial network for water treatment removal, and a high-quality underwater reconstruction sample after enhancement is output.

[0072] 4) The high-quality underwater reconstruction sample after enhancement is sent into the lightweight object detection model for detection.

[0073] The described lightweight object detection model is a YOLO series deep learning detection model.

[0074] In summary, with the technical solution of the embodiment of the present invention, an underwater image enhancement model based on the generative adversarial network is proposed to generate pseudo-optical samples that conform to the distribution of real underwater image samples, effectively alleviating the impact of insufficient effective samples on the detection model. The generative adversarial network (GAN) is a generative model based on prior knowledge, composed of a generation network G and a discriminator network D. Among them, G takes prior knowledge or noise as input and generates pseudo-samples with the same distribution as the real samples. D is usually a binary classifier that determines the input real samples and pseudo-samples, and determines the generated pseudo-samples as false. So that the generation network G continuously learns parameters to make the generated pseudo-samples closer to the real samples in order to pass the discrimination of the discriminator network D, while the discriminator network D can more accurately judge the data source. The pseudo-samples generated by the generation network G are used to alleviate the decline in recognition performance caused by data loss.

[0075] Device Embodiment 1

[0076] According to the embodiment of the present invention, an underwater weak sample image enhancement and detection device for an underwater vehicle is provided,Figure 3 is a schematic diagram of an underwater weak sample image enhancement and detection device for an underwater vehicle according to an embodiment of the present invention. As Figure 3 shown, the underwater weak sample image enhancement and detection device for an underwater vehicle according to an embodiment of the present invention specifically includes:

[0077] An image acquisition module 30, configured to sample an underwater target image and an original target image through an optical camera, and perform data preprocessing on the underwater target image and the original target image; specifically for:

[0078] Perform image screening, target detection data annotation, and dataset division on the underwater target image and the original target image.

[0079] An underwater image enhancement module 32, configured to input the underwater image into a pre-trained feature extraction network model, extract features of the underwater image through the feature extraction network model; input the features of the underwater target image and the original target image into an underwater image enhancement model based on a generative adversarial network, and perform water removal processing through the underwater image enhancement model to obtain an enhanced pseudo-underwater optical image; specifically for:

[0080] Perform image enhancement processing on the underwater target image through the generative model G of the underwater image enhancement model to generate an enhanced pseudo-underwater image; distinguish the pseudo-underwater image generated by the generative model G from the underwater target image sample through the discriminant module D of the underwater image enhancement model to determine the authenticity of the input image; distinguish the pseudo-underwater image generated by the generative model G from the random region of the original target image through the sub-discriminant module Dsub of the underwater image enhancement model to determine whether it is a real original target image or an enhanced pseudo-underwater optical image output, and finally obtain an enhanced pseudo-underwater optical image;

[0081] Specifically, the generative model G of the underwater image enhancement model adopts an attenuation compensation attention mechanism A as shown in Formula 1 - Formula 5 c and a luminance channel attention mechanism A as shown in Formula 6 - Formula 7 l , and adopts an attention-guided U-Net to extract multi-level features from different depth layers and synthesize an enhanced pseudo-underwater image by using multi-scale context information:

[0082] C g(x) =0 Formula 1;

[0083]

[0084] D c =I c -G c *I c Formula 4;

[0085] A c(x) = f(C r,g,b(x) + D c ) Formula 5;

[0086] Wherein, I r , I g , I b represent the red, green, and blue channels of the image I, and the values of the channels are normalized to [0, 1]. represents I r , I g , I b 's average value. The attenuation compensation attention mechanism improves the recovery effect of red and blue information. C g(x) represents the green channel value, C r(x) represents the red channel value, C b(x) represents the blue channel value, D c represents the detail image, that is, the image G obtained by subtracting the image blurred by the Gaussian kernel from the input image c * I c is obtained. * represents the convolution operation, f represents the fully connected layer, and then the normalization operation. C r,g,b(x) represents the red, green, and blue channel values;

[0087]

[0088] A l (i, j) = g(L B (i, j) - u B ) Formula 7;

[0089] Wherein, L B (i, j) represents the grayscale matrix of the image luminance channel L, W represents the width of the image, H represents the height of the image, and u B represents the average grayscale value of the luminance channel L. g represents the fully connected layer and the normalization operation. A l is the attention weight map of the luminance channel.

[0090] The discriminant module D is used to distinguish the samples generated by the generation model G from the underwater image samples and judge the authenticity of the input image.

[0091] Through the sub-discriminant module Dsub of the underwater image enhancement model, a patches are randomly cropped from the output image pseudo-underwater optical image I' and the air image I S . Based on the loss functions shown in Formula 8, Formula 9, Formula 10, and Formula 11, the random regions of the pseudo-underwater image generated by the generation model G and the original target image are distinguished to judge whether it is the real original target image or the pseudo-underwater optical image output after enhancement, and finally the enhanced pseudo-underwater optical image is obtained:

[0092]

[0093] Among them, x r represents the sampling area of the real underwater image, x f represents the sampling area of the pseudo underwater optical image I′, x s represents the air image I S sampling area, L D represents the discriminator loss function, represents the sub - discriminator loss function, L G , represents the corresponding loss function of the generator, x r ~P real represents conforming to the real underwater image distribution, x f ~P fake represents the pseudo underwater optical image distribution, x s ~P air represents the air optical image distribution.

[0094] The underwater image detection module 34 is used in the test stage to send the underwater target image into the underwater image enhancement model based on the generative adversarial network, perform water removal processing through the underwater image enhancement model, output the enhanced high - quality underwater reconstruction sample, and input the enhanced high - quality underwater reconstruction sample into the lightweight target detection model for detection. The lightweight target detection model is: the YOLO series of deep learning detection models.

[0095] The embodiment of the present invention is a device embodiment corresponding to the above - mentioned method embodiment. The specific operations of each module can be understood with reference to the description of the method embodiment and will not be elaborated here.

[0096] Device Embodiment Two

[0097] The embodiment of the present invention provides an electronic device, as Figure 4 shown, including: a memory 40, a processor 42, and a computer program stored on the memory 40 and executable on the processor 42. When the computer program is executed by the processor 42, the steps described in the method embodiment are implemented.

[0098] Device Embodiment Three

[0099] The embodiment of the present invention provides a computer - readable storage medium. An implementation program for information transmission is stored on the computer - readable storage medium. When the program is executed by the processor 42, the steps described in the method embodiment are implemented.

[0100] The computer - readable storage medium described in this embodiment includes but is not limited to: ROM, RAM, magnetic disk, or optical disk, etc.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for underwater weak sample image enhancement and detection for underwater vehicles, characterized in that: include: An underwater target image and an original target image in the air are obtained by sampling with an optical camera, and data preprocessing is performed on the underwater target image and the original target image; The underwater image is input into a pre-trained feature extraction network model, and features of the underwater image are extracted by the feature extraction network model; the features of the underwater target image and the original target image are input into an underwater image enhancement model based on a generative adversarial network, and water removal is performed by the underwater image enhancement model to obtain an enhanced pseudo underwater optical image; During the testing phase, the underwater target image is sent to an underwater image enhancement model based on a generative adversarial network, dewatered by the underwater image enhancement model, and an enhanced high-quality underwater reconstruction sample is output, which is then input into a lightweight target detection model for detection.

2. The method according to claim 1, characterized in that The data preprocessing of the underwater target image and the original target image specifically includes: The underwater target image and the original target image are subjected to image screening, target detection data annotation, and data set division.

3. The method according to claim 1, characterized in that The underwater image enhancement model is used to perform water removal processing to obtain an enhanced pseudo underwater optical image, which specifically includes: The underwater target image is enhanced by the generation model G of the underwater image enhancement model to generate an enhanced pseudo underwater image; the pseudo underwater image generated by the generation model G is distinguished from the underwater target image sample by the discrimination module D of the underwater image enhancement model to judge the authenticity of the input image; the pseudo underwater image generated by the generation model G is distinguished from the random area of ​​the original target image in the air by the sub-discrimination module Dsub of the underwater image enhancement model to judge whether it is the real original target image or the enhanced pseudo underwater optical image output, and finally the enhanced pseudo underwater optical image is obtained.

4. The method according to claim 3, characterized in that Performing image enhancement processing on the underwater target image by using the generation model G of the underwater image enhancement model to generate an enhanced pseudo underwater image specifically includes: The generative model G of the underwater image enhancement model adopts the attenuation compensation attention mechanism A shown in Formula 1-Formula 5 c and the brightness channel attention mechanism A as shown in Formula 6-Formula 7 l , the attention-guided U-Net extracts multi-level features from different depth layers and uses multi-scale contextual information to synthesize enhanced pseudo underwater images: C g(x) =0 Formula 1; D c = I c - G c * I c Equation 4; A c(x) = f(C r,g,b(x) +D c ) Formula 5; Among them, I r ,I g ,I g Represents the red, green, and blue channels of image I, and the channel values ​​are normalized to [0,1]. Indicates I r ,I g ,I b The average value of the attenuation compensation attention mechanism improves the recovery effect of red and blue information. g(x) Indicates the green channel value, C r(x) Represents the red channel value, C b(x) Indicates the blue channel value, D c Represents the detail image, which is the image G minus the Gaussian kernel blurred image from the input image c *I c We get: * represents convolution operation, f represents the fully connected layer, and then normalization operation, C r,g,b(x) Represents red, green, and blue channel values; A l (i,j) = g(L B (i,j) - u B ) Equation 7; Among them, L B (i, j) represents the grayscale matrix of the image brightness channel L, W represents the width of the image, H represents the height of the image, and u B represents the mean gray value of the brightness channel L, g represents the fully connected layer and normalization operation, A l is the attention weight map of the brightness channel.

5. The method according to claim 3, characterized in that: The pseudo underwater image generated by the generation model G is distinguished from the random area of ​​the original target image by the sub-discrimination module Dsub of the underwater image enhancement model to determine whether it is the real original target image or the pseudo underwater optical image output after enhancement. The enhanced pseudo underwater optical image is finally obtained, which specifically includes: Through the sub-discrimination module Dsub of the underwater image enhancement model, the output image is a pseudo underwater optical image I ′ and air image I S A region is randomly cropped in the image. Based on the loss function shown in Formula 8 and Formula 9, the pseudo underwater image generated by the generative model G is distinguished from the random region of the original target image to determine whether it is the real original target image or the pseudo underwater optical image output after enhancement. Finally, the enhanced pseudo underwater optical image is obtained: Among them, x r represents the real underwater image sampling area, x f Represents the pseudo underwater optical image I ′ Sampling area, x s Represents the air image I S Sampling area, L D represents the discriminator loss function, represents the sub-discriminator loss function, L G , Represents the corresponding loss function of the generator, x r ~P raal Indicates that it conforms to the real underwater image distribution, x f ~P fake represents the pseudo underwater optical image distribution, x s ~P air Represents the air optical image distribution.

6. The method according to claim 1, characterized in that The lightweight target detection model is: YOLO series deep learning detection model.

7. An underwater weak sample image enhancement and detection device for underwater vehicles, characterized in that: include: An image acquisition module, used for obtaining an underwater target image and an original target image by sampling with an optical camera, and performing data preprocessing on the underwater target image and the original target image; An underwater image enhancement module is used to input the underwater image into a pre-trained feature extraction network model, and extract features of the underwater image through the feature extraction network model; input the features of the underwater target image and the original target image into an underwater image enhancement model based on a generative adversarial network, and perform water removal processing through the underwater image enhancement model to obtain an enhanced pseudo underwater optical image; The underwater image detection module is used to send the underwater target image to the underwater image enhancement model based on the generative adversarial network during the testing phase, perform dehydration processing through the underwater image enhancement model, output enhanced high-quality underwater reconstruction samples, and input the enhanced high-quality underwater reconstruction samples into the lightweight target detection model for detection.

8. The method according to claim 1, characterized in that The image acquisition module is specifically used for: Performing image screening, target detection data annotation, and data set division on the underwater target image and the original target image; The underwater image enhancement module is specifically used for: The underwater target image is enhanced by the generation model G of the underwater image enhancement model to generate an enhanced pseudo underwater image; the pseudo underwater image generated by the generation model G is distinguished from the underwater target image sample by the discrimination module D of the underwater image enhancement model to judge the authenticity of the input image; the pseudo underwater image generated by the generation model G is distinguished from the random area of ​​the original target image by the sub-discrimination module Dsub of the underwater image enhancement model to judge whether it is the real original target image or the pseudo underwater optical image output after enhancement, and finally the enhanced pseudo underwater optical image is obtained; The underwater image enhancement module is specifically used for: The generative model G of the underwater image enhancement model adopts the attenuation compensation attention mechanism A shown in Formula 1-Formula 5 c and the brightness channel attention mechanism A as shown in Formula 6-Formula 7 l , the attention-guided U-Net extracts multi-level features from different depth layers and uses multi-scale contextual information to synthesize enhanced pseudo underwater images: C g(x) =0 Formula 1; D c = I c - G c * I c Equation 4; A c(x) = f(C r,g,b(x) +D c ) Formula 5; Among them, I r ,I g ,I b Represents the red, green, and blue channels of image I, and the channel values ​​are normalized to [0,1]. Indicates I r ,I g ,I b The average value of the attenuation compensation attention mechanism improves the recovery effect of red and blue information. g(x) Indicates the green channel value, C r(x) Represents the red channel value, C b(x) Indicates the blue channel value, D c Represents the detail image, which is the image G minus the Gaussian kernel blurred image from the input image c *I c We get: * represents convolution operation, f represents the fully connected layer, and then normalization operation, C r,g,b(x) Represents red, green, and blue channel values; A l (i,j) = g(L B (i,j) - u B ) Formula 7; Among them, L B (i, j) represents the grayscale matrix of the image brightness channel L, W represents the width of the image, H represents the height of the image, and u B represents the mean gray value of the brightness channel L, g represents the fully connected layer and normalization operation, A l is the attention weight map of the brightness channel. The underwater image enhancement module is specifically used for: The discrimination module D is used to distinguish the samples generated by the generation model G from the underwater image samples and determine the authenticity of the input image. Through the sub-discrimination module Dsub of the underwater image enhancement model, the output image is a pseudo underwater optical image I ′ and air image I S A patches are randomly cropped in the image, and based on the loss functions shown in Formula 8, Formula 9, Formula 10, and Formula 11, the pseudo underwater image generated by the generative model G is distinguished from the random area of ​​the original target image to determine whether it is the real original target image or the enhanced pseudo underwater optical image output, and finally the enhanced pseudo underwater optical image is obtained: Among them, x r represents the real underwater image sampling area, x f Represents the pseudo underwater optical image I ′ Sampling area, x s Represents the air image I S Sampling area, L D represents the discriminator loss function, represents the sub-discriminator loss function, L G , Represents the corresponding loss function of the generator, x r ~P real Indicates that it is consistent with the real underwater image distribution, x f ~P fake represents the pseudo underwater optical image distribution, x s ~P air Represents the air optical image distribution. The lightweight target detection model is: YOLO series deep learning detection model.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the underwater weak sample image enhancement and detection method for underwater vehicles as described in any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an implementation program for information transmission, and when the program is executed by the processor, the steps of the underwater weak sample image enhancement and detection method for underwater vehicles according to any one of claims 1 to 6 are implemented.

Images