A joint optimization method for underwater multi-degraded image enhancement and target recognition based on genetic algorithm

Abstract

The present invention discloses a joint optimization method for underwater multi-degraded image enhancement and target recognition based on a genetic algorithm, which relates to the fields of underwater image enhancement and underwater target recognition. The method comprises the following steps: building an underwater robot target recognition system, acquiring underwater event camera data, and processing the underwater event camera data to obtain a sample data set; initializing an initial population, i.e., initial enhancement parameters, using a genetic algorithm; performing initial enhancement on environmental information using the initial enhancement parameters to obtain an initial enhanced image data set; constructing an underwater target recognition algorithm model using YOLOv5 as a basic framework to obtain the mean average precision (mAP) of underwater target detection; using the mean average precision (mAP) of underwater target detection as the fitness value of the genetic algorithm, optimizing the enhancement parameters of an underwater multi-degraded image enhancement module using the genetic algorithm, and then obtaining target enhancement parameters, enhancing environmental information to obtain a target enhanced image, thereby achieving the purposes of image enhancement and effective recognition at the same time.

Description

Technical Field

[0001] The present invention relates to the fields of underwater image enhancement and underwater target recognition, and in particular to a combined optimization method for underwater multi-degraded image enhancement and target recognition based on a genetic algorithm. Background Art

[0002] Underwater robots (UUVs), as widely used unmanned submersibles, are commonly used for ocean exploration, intelligence reconnaissance, and underwater search and rescue. Thanks to these features, UUVs can replace humans in underwater operations at depths beyond human capabilities and in highly dangerous environments, holding significant significance in scientific research and the military. UUVs are typically equipped with cameras to identify underwater targets.

[0003] On the one hand, due to the complex and ever-changing underwater environment, images captured underwater often suffer from various degradation issues, posing significant challenges to underwater robotic operations and target recognition. The absorption and scattering of light by the water medium reduces the contrast and visibility of underwater images. Furthermore, the underwater environment requires artificial fill lighting, which can result in uneven illumination and lead to under- or over-exposure in some images. Therefore, it is necessary to develop image enhancement methods tailored to underwater environments to mitigate the adverse effects of underwater image quality degradation on underwater robotic target recognition. Generally, there are two approaches to improving underwater image quality: the first is hardware-based approaches, which involve introducing new equipment to improve image quality by addressing the shortcomings of conventional cameras; the second is software-based approaches, which enhance image quality through underwater image processing methods.

[0004] On the other hand, underwater image processing and underwater object recognition are two different tasks. Underwater object recognition aims to improve image visibility for human perception, while object detection algorithms aim to improve target recognition accuracy. Directly using enhanced underwater images for target recognition may not yield good target recognition results. Therefore, it is essential to adjust underwater image processing algorithms in conjunction with underwater target recognition. Summary of the Invention

[0005] To address the above issues, the present invention proposes a joint optimization algorithm for underwater multi-degraded image enhancement and target recognition based on a genetic algorithm. The genetic algorithm is used to optimize the parameters of underwater multi-degraded image enhancement. The mean average precision (mAP) of target recognition is used as the fitness function to obtain a target underwater multi-degraded image enhancement solution for underwater target recognition, thereby improving the accuracy of underwater target recognition. The present invention provides a joint optimization method for underwater multi-degraded image enhancement and target recognition based on a genetic algorithm, comprising the following steps:

[0006] S1. Build an underwater robot target recognition system, wherein the underwater robot target recognition system includes an underwater robot and an underwater event camera, wherein the underwater event camera is mounted on the underwater robot;

[0007] S2. Acquire underwater event camera data and analyze the underwater event camera data to obtain environmental information of the underwater environment. The environmental information, i.e., an underwater multi-degraded image, includes an event stream E0 and an RGB image, i.e., a color frame I0 captured by the underwater event camera. The event stream E0 follows the RGB image, and the event stream E0 enhances the multi-degraded RGB image during the underwater multi-degraded image enhancement process.

[0008] S3. Use the target recognition data annotation tool to annotate the RGB image, and divide the environmental information according to the annotation results to obtain a sample data set;

[0009] S4, using genetic algorithm to initialize and obtain the initial enhancement parameters of the initial population, i.e., the underwater multi-degraded image enhancement algorithm;

[0010] S5. Performing initial enhancement on the environmental information using the initial enhancement parameters to obtain an initial enhanced image dataset;

[0011] S6. Using YOLOv5 as the basic framework, an underwater target recognition algorithm model is constructed. The initial enhanced image dataset is input into the underwater target recognition algorithm model to obtain the average precision (mAP) of underwater target detection.

[0012] S7. Take the mean average precision (mAP) of target detection as the fitness value of the genetic algorithm. The initial enhancement parameters are optimized through selection-crossover-mutation operations to obtain the next generation population, and then the original image is enhanced. The maximum evolutionary number of the genetic algorithm is preset. S5 and S6 are repeated until the genetic algorithm reaches the maximum evolutionary number to obtain the target enhancement parameters. Combined with the target enhancement parameters, the underwater multi-degraded image enhancement algorithm is used to enhance the environmental information to obtain the target enhanced image. The YOLOv5 model is used for target recognition to obtain the target recognition result.

[0013] Preferably, the initial enhancement parameters are used to perform initial enhancement on the environmental information to obtain the initial enhanced image dataset, specifically including:

[0014] S301: Environmental information enters a preprocessing stage to obtain preprocessing information, wherein the environmental information preprocessing stage includes an event stream reconstruction unit and an RGB image color correction unit, and the preprocessing information includes an event reconstruction image and an RGB color correction image;

[0015] S302: The pre-processed information enters the defogging stage to obtain enhanced environmental information data, wherein the enhanced environmental information data includes a fog-free event image and a fog-free RGB image. The defogging stage includes an event reconstruction image defogging unit and an RGB image defogging unit.

[0016] S303: The enhanced environmental information data enters the image fusion unit to obtain a fused image, ie, an initial enhanced image.

[0017] Preferably, the specific content of the event stream reconstruction unit is:

[0018] The event stream is the data sequence E i =(x i ,y i , t i , p i ), which contains information about the pixel position (x i ,y i ), sampling time t i and change symbol p i information;

[0019] Reconstruct the event stream E0 into an intensity image using the E2VID method ;

[0020] The specific content of the E2VID method includes using a convolutional recurrent network to learn intensity frame reconstruction from asynchronous event data;

[0021] Building an improved recursive network based on the UNet architecture;

[0022] The event simulator is used to generate 1000 simulated event sequences from the event stream to train and test the network, of which 950 are training simulated event sequences and 50 are verification simulated event sequences.

[0023] Divide the simulation event sequence into a series of non-overlapping space-time windows and convert them into event tensors;

[0024] The simulated event sequence in each spatiotemporal window is sequentially input into the trained network, and the intensity frames are reconstructed to obtain the reconstructed event image.

[0025] Preferably, the specific content of the RGB image color correction unit is:

[0026] According to the average intensity value of the three color channels R, G, and B in the underwater RGB image;

[0027] The expression for calculating the average intensity value is as follows:

[0028] ;

[0029] in and Represent the height and width of the input image respectively.

[0030] According to the average intensity value from high to low, the three color channels R, G, and B are divided into: high-quality color channels , medium quality color channels , low-quality color channels ;

[0031] High-quality color channels For reference, calculate the compensation factors for medium-quality color channel compensation respectively , and a compensation factor for compensating for low-quality color channels ;

[0032] Calculating the compensation factor and compensation factor The formula is as follows:

[0033] ;

[0034] ;

[0035] Use compensation factors and , to compensate for low and medium quality color channels;

[0036] The compensation expression is as follows:

[0037] ;

[0038] ;

[0039] The white balance algorithm based on the gray world hypothesis is used for further color correction to obtain the color-corrected RGB image. That is, RGB color corrected image.

[0040] Preferably, the specific contents of the event reconstruction image defogging unit are:

[0041] Event image reconstruction using a multi-scale fusion framework Atomization treatment;

[0042] Event reconstruction image Generate a sharpened image and gamma-corrected image sequences ;

[0043] Event reconstruction image Generate a sharpened image The calculation expression is:

[0044] ;

[0045] in, Represents the input image A blurred version of represents the linear normalization operator;

[0046] Event reconstruction image Generate gamma-corrected image sequence The expression is:

[0047] ;

[0048] in, is a positive constant, The value range is [1.0,3.0], is the number of different exposure images, . 、 、 There are three parameters to be adjusted;

[0049] Contrast weighted map and exposure weight graph As an indicator, four versions of the event reconstruction image are constructed using the Laplacian pyramid fusion method. Fusion: A sharpened version and three gamma-corrected versions , get the event image after defogging That is, the fog-free event map;

[0050] Contrast weight map and exposure weight graph The expressions are:

[0051] ;

[0052] ;

[0053] in is the Laplace operator, is the standard deviation parameter, usually set to 0.2.

[0054] Preferably, the specific content of the RGB image defogging unit is:

[0055] Color correction of RGB images using a multi-scale fusion framework Atomization treatment;

[0056] Color correct RGB image to produce sharpened image and gamma-corrected image sequences ;

[0057] Sharpen the image The calculation expression is:

[0058] ;

[0059] Gamma-corrected image sequence The calculation expression is:

[0060] ;

[0061] in, is a positive constant, The value range is [1.0,3.0], is the number of different exposure images, . 、 、 There are three parameters to be adjusted;

[0062] Contrast weighted map and exposure weight graph As an indicator, the Laplacian pyramid fusion method is used to color correct the RGB image Four versions of Fusion: A sharpened version and 3 gamma-corrected versions Fusion is performed to obtain the defogging RGB image That is, a fog-free RGB image;

[0063] Contrast weight map and saturation weight map The calculation expression is:

[0064] ;

[0065] ;

[0066] in, is the jth version of the color-corrected RGB image; 、 、 are the R, G, and B color channels of the j-th version of the RGB image, It is the average of the three RGB color channel values.

[0067] Preferably, the enhanced environmental information data in S303 enters the image fusion unit to obtain a fused image, i.e., an initial enhanced image. Specifically, the content is:

[0068] The illumination map is extracted from the enhanced RGB image using the RRDNET image decomposition method. ;

[0069] Use gamma correction to adjust the illumination map , the expression for adjusting the illumination map using the gamma correction method is as follows:

[0070] ;

[0071] in, is a positive constant, The value range is [0.1, 3.0], is the number of different exposure images, =3. 、 、 There are three parameters to be adjusted.

[0072] Enhanced event reconstruction using gamma correction exposure, resulting in Multiple exposure images, using gamma correction to enhance event reconstruction The expression of exposure is as follows:

[0073] ;

[0074] in, is a positive constant, The value range is [0.1, 3.0], is the number of different exposure images, =3, 、 、 There are three parameters to be adjusted;

[0075] The weight map of each artificially exposed image is set using three features: contrast, exposure, and average brightness;

[0076] Multi-scale fusion strategy to fuse multiple exposure image sequences and weight maps to generate fused images ;

[0077] Based on Retinex theory, the fused image Restore to RGB image and fuse the image The expression for restoring to RGB image is:

[0078] .

[0079] Preferably, YOLOv5 is used as the basic framework to build an underwater target recognition algorithm model, and the initial enhanced image data set is input into the underwater target recognition algorithm model respectively to obtain the specific content of the average precision mAP of multiple underwater target detections:

[0080] S501, image input: adjust the initial enhanced image to a fixed size (640 640) and normalized to ensure that the images input to the YOLOv5 target detection model maintain a consistent scale in diverse environments.

[0081] S502, Feature Extraction and Fusion: Using CSPDarknet53 as the backbone network, we extract underlying features from the enhanced underwater images and introduce a spatial attention mechanism to further improve the feature extraction capability of small and blurred targets in underwater environments. We use the feature pyramid FPN and path enhancement network PANet to fuse the extracted underlying features and generate multi-scale feature maps.

[0082] S504, Bounding Box and Classification Prediction: For each grid cell in the feature map, the model simultaneously outputs multiple predictions, including the target box, target confidence, and target category incidence. Maximum value suppression (NMS) is used to eliminate overlapping prediction boxes and retain the box with the highest confidence.

[0083] S506, model training: by calculating the gradient of the loss function, using the optimizer for backpropagation, and updating the target recognition network weights. The loss functions used include bounding box regression loss, classification loss, and confidence loss.

[0084] Total loss function The formula is as follows:

[0085] ;

[0086] in: is the bounding box regression loss, is the confidence loss, is the classification loss;

[0087] S507. After training is completed, the average precision (mAP) of target detection is calculated on the test set:

[0088] ;

[0089] in, is the number of target categories, For the The average precision of the categories.

[0090] Preferably, the target detection mean average precision mAP is used as the fitness value of the genetic algorithm, and the initial enhancement parameters are optimized through selection-crossover-mutation operations to obtain the next generation population, and then the original image (environmental information) is enhanced. The maximum evolutionary number of the genetic algorithm is preset, and S5 and S6 are repeated until the genetic algorithm reaches the maximum evolutionary number to obtain the target enhancement parameters. The underwater multi-degraded image enhancement algorithm is used to enhance the environmental information in combination with the target enhancement parameters to obtain a target enhanced image, and the YOLOv5 model is used for target recognition. The specific content of the target recognition result can be obtained as follows:

[0091] Genetic algorithm is used to optimize the initial enhancement parameters, which include 、 、 、 、 、 、 、 、 、 、 、 , the specific steps are:

[0092] S601, initialization: set the counter u=0, the maximum evolutionary generation U=20, and the population size t=30. The 12 parameters to be adjusted in the multi-degraded image enhancement method are coded into chromosome real numbers to generate a sample p containing 12 genetic genes. t , combined with the value range of the parameter to be adjusted, randomly initialize 30 samples As the initial population P 0；

[0093] S602, sample fitness value calculation: using the average detection precision (mAP) as the genetic fitness function, calculate P one by one u The fitness value of all samples in;

[0094] S603, sample selection: based on population P u The fitness value of the sample in the 30 samples is calculated, and the normalized probability of the fitness value of the 30 samples is calculated. Then, the roulette wheel method is used to select 30 samples to generate a new population P. u+1 ;

[0095] S604, crossover operation: select the seed from the population P according to the crossover probability u+1 Select two chromosomes, randomly determine the crossover position on the chromosome, and perform a crossover operation on the gene;

[0096] S605, mutation operation: according to the mutation probability, in the population P u+1Randomly select the individuals that need to mutate, then randomly select two mutation positions on the chromosome and exchange the genes at these two positions;

[0097] S606: Merge the selected parent chromosome with the offspring chromosome generated by crossover and mutation operations to form a new generation population P. u+1 , calculate the new generation population P u+1 The fitness value of the current generation is used to determine whether the optimal result of the current generation meets the conditions for terminating the iteration. If , the termination condition is not met, then S602–S606 are repeated;

[0098] if , the iteration is terminated, and the sample with the largest fitness in the last generation of population is output.

[0099] In summary, the genetic algorithm-based joint optimization method for underwater multi-degraded image enhancement and target recognition, compared to traditional techniques, uses an underwater robot equipped with an event camera to capture underwater targets, obtaining corresponding event information and RGB image information. In application, the underwater robot collects data in different water environments (e.g., clear water, turbid water, and uneven illumination), thereby constructing a diverse and comprehensive underwater target recognition dataset. This facilitates the training of more generalized target recognition models and improves recognition robustness. Images enhanced by the underwater multi-degraded image enhancement method based on event stream and RGB fusion can simultaneously achieve color correction, defogging, and restoration of uneven illumination details. This genetic algorithm-based joint optimization method for underwater multi-degraded image enhancement and target recognition can produce underwater multi-degraded image enhancement results that are beneficial for target recognition, improving underwater target recognition accuracy.

[0100] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] Figure 1 This is a flow chart of a joint optimization method for underwater multi-degraded image enhancement and target recognition based on genetic algorithm of the present invention;

[0102] Figure 2 This is a flow chart of a multi-degraded image enhancement method based on events and RGB fusion for a joint optimization method of underwater multi-degraded image enhancement and target recognition based on a genetic algorithm of the present invention;

[0103] Figure 3 This is a flow chart of a joint optimization method for underwater multi-degraded image enhancement and target recognition based on a genetic algorithm of the present invention, which uses a genetic algorithm to jointly optimize an underwater target recognition algorithm model and an underwater multi-degraded image enhancement method;

[0104] Figure 4This is a genetic algorithm flow chart of a genetic algorithm-based joint optimization method for underwater multi-degraded image enhancement and target recognition according to the present invention. DETAILED DESCRIPTION

[0105] The technical solutions of the present invention are further described below through the accompanying drawings and embodiments. It should be noted that unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions and values ​​described in these embodiments do not limit the scope of this application.

[0106] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0107] Technologies, systems, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0108] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0109] Unless otherwise defined, technical or scientific terms used in the present invention shall have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0110] like Figure 1 As shown, the present invention provides a joint optimization method for underwater multi-degraded image enhancement and target recognition based on genetic algorithm, S1, building an underwater robot target recognition system, the underwater robot target recognition system includes an underwater robot and an underwater event camera, and the underwater event camera is mounted on the underwater robot;

[0111] S2. Acquire underwater event camera data and analyze the underwater event camera data to obtain environmental information of the underwater environment. The environmental information, i.e., underwater multi-degraded images, includes an event stream E0 and an RGB image, i.e., a color frame I0 captured by the underwater event camera. The event stream E0 follows the RGB image and enhances the RGB image.

[0112] S3. Use the target recognition data annotation tool to annotate the RGB image, and divide the environmental information according to the annotation results to obtain a sample data set;

[0113] S4, using genetic algorithm to initialize and obtain initial enhancement parameters;

[0114] S5. Performing initial enhancement on the environmental information using the initial enhancement parameters to obtain an initial enhanced image dataset;

[0115] like Figure 2As shown, preferably, the initial enhancement parameters are used to perform initial enhancement on the environmental information to obtain the initial enhanced image data set, which specifically includes:

[0116] S301: Environmental information enters a preprocessing stage to obtain preprocessing information, wherein the environmental information preprocessing stage includes an event stream reconstruction unit and an RGB image color correction unit, and the preprocessing information includes an event reconstruction image and an RGB color correction image;

[0117] S302: The pre-processed information enters the defogging stage to obtain enhanced environmental information data, wherein the enhanced environmental information data includes a fog-free event image and a fog-free RGB image. The defogging stage includes an event reconstruction image defogging unit and an RGB image defogging unit.

[0118] S303: The enhanced environmental information data enters the image fusion unit to obtain a fused image, ie, an initial enhanced image.

[0119] Preferably, the specific content of the event stream reconstruction unit is:

[0120] The event stream is the data sequence E i =(x i ,y i , t i , p i ), which contains information about the pixel position (x i ,y i ), sampling time t i and change symbol p i Since the event stream and RGB frame have different representation forms, they cannot be directly fused and should be converted into intensity images through preprocessing.

[0121] Reconstruct the event stream E0 into an intensity image using the E2VID method ;

[0122] The specific content of the E2VID method includes using a convolutional recurrent network to learn intensity frame reconstruction from asynchronous event data;

[0123] Building an improved recursive network based on the UNet architecture;

[0124] The event simulator is used to generate 1000 simulated event sequences from the event stream to train and test the network, of which 950 are training simulated event sequences and 50 are verification simulated event sequences.

[0125] Divide the simulation event sequence into a series of non-overlapping space-time windows and convert them into event tensors;

[0126] The simulated event sequence in each spatiotemporal window is sequentially input into the trained network, and the intensity frames are reconstructed to obtain the reconstructed event image.

[0127] Preferably, the specific content of the RGB image color correction unit is:

[0128] According to the average intensity value of the three color channels R, G, and B in the underwater RGB image;

[0129] The expression for calculating the average intensity value is as follows:

[0130] ;

[0131] in and Represent the height and width of the input image respectively.

[0132] According to the average intensity value from high to low, the three color channels R, G, and B are divided into: high-quality color channels , medium quality color channels , low-quality color channels ;

[0133] High-quality color channels For reference, calculate the compensation factors for medium-quality color channel compensation respectively , and a compensation factor for compensating for low-quality color channels ;

[0134] Calculating the compensation factor and compensation factor The formula is as follows:

[0135] ;

[0136] ;

[0137] Using the Gain (Compensation) Factor and , to compensate for low and medium quality color channels;

[0138] The compensation expression is as follows:

[0139] ;

[0140] ;

[0141] The white balance algorithm based on the gray world hypothesis is used for further color correction to obtain the color-corrected RGB image. That is, RGB color corrected image.

[0142] Preferably, the specific contents of the event reconstruction image defogging unit are:

[0143] Event image reconstruction using a multi-scale fusion framework Atomization treatment;

[0144] Event reconstruction image Generate a sharpened image and gamma-corrected image sequences ;

[0145] Event reconstruction image Generate a sharpened image The calculation expression is:

[0146] ;

[0147] in, Represents the input image A blurred version of represents the linear normalization operator;

[0148] Event reconstruction image Generate gamma-corrected image sequence The expression is:

[0149] ;

[0150] in, is a positive constant, The value range is [1.0,3.0], is the number of different exposure images, . 、 、 There are three parameters to be adjusted;

[0151] Contrast weighted map and exposure weight graph As an indicator, four versions of the event reconstruction image are constructed using the Laplacian pyramid fusion method. Fusion: A sharpened version and three gamma-corrected versions , get the event image after defogging That is, the fog-free event map;

[0152] Contrast weight map and exposure weight graph The expressions are:

[0153] ;

[0154] ;

[0155] in is the Laplace operator, is the standard deviation parameter, usually set to 0.2.

[0156] Preferably, the specific content of the RGB image defogging unit is:

[0157] Color correction of RGB images using a multi-scale fusion framework Atomization treatment;

[0158] Color correct RGB image to produce sharpened image and gamma-corrected image sequences ;

[0159] Sharpen the image The calculation expression is:

[0160] ;

[0161] Gamma-corrected image sequence The calculation expression is:

[0162] ;

[0163] in, is a positive constant, The value range is [1.0,3.0], is the number of different exposure images, . 、 、 There are three parameters to be adjusted;

[0164] Contrast weighted map and exposure weight graph As an indicator, the Laplacian pyramid fusion method is used to color correct the RGB image Four versions of Fusion: A sharpened version and 3 gamma-corrected versions Fusion is performed to obtain the defogging RGB image That is, a fog-free RGB image;

[0165] Contrast weight map and saturation weight map The calculation expression is:

[0166] ;

[0167] ;

[0168] in It is the average of the three RGB color channel values.

[0169] Preferably, the enhanced environmental information data in S303 enters the image fusion unit to obtain the fused image, i.e., the initial enhanced image, with the following specific contents:

[0170] The illumination map is extracted from the enhanced RGB image using the RRDNET image decomposition method. ;

[0171] Use gamma correction to adjust the illumination map , the expression for adjusting the illumination map using the gamma correction method is as follows:

[0172] ;

[0173] in, is a positive constant, The value range is [0.1, 3.0], is the number of different exposure images, =3. 、 、 There are three parameters to be adjusted;

[0174] Enhanced event reconstruction using gamma correction exposure, resulting in Multiple exposure images, using gamma correction to enhance event reconstruction The expression of exposure is as follows:

[0175] ;

[0176] in, is a positive constant, The value range is [0.1, 3.0], is the number of different exposure images, =3, 、 、 There are three parameters to be adjusted;

[0177] To ensure that more scene details are preserved and the brightness is consistent during the image fusion process, the weight map of each artificially exposed image is set using three features: contrast, exposure, and average brightness;

[0178] Multi-scale fusion strategy to fuse multiple exposure image sequences and weight maps to generate fused images ;

[0179] Based on Retinex theory, the fused image Restore to RGB image and fuse the image The expression for restoring to RGB image is:

[0180] .

[0181] S6. Using YOLOv5 as the basic framework, an underwater target recognition algorithm model is constructed. The initial enhanced image dataset is input into the underwater target recognition algorithm model to obtain multiple underwater target detection mean average precisions (mAPs). The multiple underwater target detection mean average precisions (mAPs) are input into the genetic algorithm through selection, crossover, and mutation to generate the next generation population.

[0182] like Figure 3 As shown in the figure, using YOLOv5 as the basic framework, an underwater target recognition algorithm model is constructed, and the initial enhanced image dataset is input into the underwater target recognition algorithm model respectively. The specific contents of obtaining the average precision mAP of multiple underwater target detection are as follows:

[0183] S501, image input: adjust the initial enhanced image to a fixed size (640 640) and normalized to ensure that the images input to the YOLOv5 target detection model maintain a consistent scale in diverse environments.

[0184] S502, Feature Extraction and Fusion: Using CSPDarknet53 as the backbone network, we extract underlying features from the enhanced underwater images and introduce a spatial attention mechanism to further improve the feature extraction capability of small and blurred targets in underwater environments. We use the feature pyramid FPN and path enhancement network PANet to fuse the extracted underlying features and generate multi-scale feature maps.

[0185] S504, Bounding Box and Classification Prediction: For each grid cell in the feature map, the model simultaneously outputs multiple predictions, including the target box, target confidence, and target category incidence. Maximum value suppression (NMS) is used to eliminate overlapping prediction boxes and retain the box with the highest confidence.

[0186] S506, model training: by calculating the gradient of the loss function, using the optimizer for backpropagation, and updating the target recognition network weights. The loss functions used include bounding box regression loss, classification loss, and confidence loss.

[0187] Total loss function The formula is as follows:

[0188] ;

[0189] in: is the bounding box regression loss, is the confidence loss, is the classification loss;

[0190] S507. After training is completed, the average precision (mAP) of target detection is calculated on the test set:

[0191] ;

[0192] in, is the number of target categories, For the The average precision of the categories.

[0193] S7. Take the mean average precision (mAP) of target detection as the fitness value of the genetic algorithm. Optimize the initial enhancement parameters through selection-crossover-mutation operations to obtain the next generation population, and then enhance the original image (environmental information). The maximum evolutionary number of the genetic algorithm is preset. Repeat S5 and S6 until the genetic algorithm reaches the maximum evolutionary number to obtain the target enhancement parameters. Combined with the target enhancement parameters, the underwater multi-degraded image enhancement algorithm is used to enhance the environmental information to obtain a target enhanced image. The YOLOv5 model is used for target recognition to obtain the target recognition result.

[0194] like Figure 4 As shown in the figure, the underwater optimization recognition accuracy is used as the fitness value of the genetic algorithm. The genetic algorithm is used to optimize the initial enhancement parameters and then enhance the initial image. The maximum evolutionary number of the genetic algorithm is preset. S5 and S6 are repeated until the genetic algorithm reaches the maximum evolutionary number to obtain the target enhancement parameters. Combined with the target parameters, the event stream and RGB are fused to perform initial enhancement on the multi-degraded environment information to obtain the target enhanced image. The specific content is:

[0195] Genetic algorithm is used to optimize the initial enhancement parameters, which include 、 、 、 、 、 、 、 、 、 、 、 , the specific steps are:

[0196] S601, initialization: set the counter u=0, the maximum evolutionary generation U=20, and the population size t=30. The 12 parameters to be adjusted in the multi-degraded image enhancement method are coded into chromosome real numbers to generate a sample p containing 12 genetic genes. t , combined with the value range of the parameter to be adjusted, randomly initialize 30 samples As the initial population P 0；

[0197] S602, sample fitness value calculation: using the average detection precision (mAP) as the genetic fitness function, calculate P one by oneu The fitness value of all samples in;

[0198] S603, sample selection: based on population P u The fitness value of the sample in the 30 samples is calculated, and the normalized probability of the fitness value of the 30 samples is calculated. Then, the roulette wheel method is used to select 30 samples to generate a new population P. u+1 ;

[0199] S604, crossover operation: select the seed from the population P according to the crossover probability u+1 Select two chromosomes, randomly determine the crossover position on the chromosome, and perform a crossover operation on the gene;

[0200] S605, mutation operation: according to the mutation probability, in the population P u+1 Randomly select the individuals that need to mutate, then randomly select two mutation positions on the chromosome and exchange the genes at these two positions;

[0201] S606: Merge the selected parent chromosome with the offspring chromosome generated by crossover and mutation operations to form a new generation population P. u+1 , calculate the new generation population P u+1 The fitness value of the current generation is used to determine whether the optimal result of the current generation meets the conditions for terminating the iteration. If , the termination condition is not met, then S602–S606 are repeated;

[0202] if , the iteration is terminated, and the sample with the largest fitness in the last generation of population is output, that is, the target enhancement parameter.

[0203] The target enhancement parameters are applied to the underwater multi-degraded image enhancement algorithm to enhance the environmental information. The event stream is reconstructed into an intensity image through the event stream reconstruction module. The underwater image color is compensated through the RGB image color correction module. The image fogging and blurring are removed through the RGB image defogging module and the event reconstruction image defogging module to improve the image clarity. The enhanced RGB and event images are fused through the RGB / event image fusion module to integrate the texture detail features of the two and better restore the details of dark and bright areas.

[0204] The image enhanced by the underwater multi-degraded image enhancement method based on event stream and RGB fusion can simultaneously achieve color correction, haze removal and uneven illumination detail restoration to obtain the target enhanced image.

[0205] Then, the target enhanced image is input into the target recognition module, and the YOLOv5 model is used for target recognition to obtain the target recognition result.

[0206] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A joint optimization method for underwater multi-degraded image enhancement and target recognition based on genetic algorithm, characterized by: The following steps are involved: S1. Build an underwater robot target recognition system, wherein the underwater robot target recognition system includes an underwater robot and an underwater event camera, wherein the underwater event camera is mounted on the underwater robot; S2. Acquire underwater event camera data and use the event camera to record multi-degraded environmental information in a complex underwater environment. The multi-degraded environmental information includes a multi-degraded event stream E0 and a multi-degraded RGB image, i.e., a multi-degraded color frame I0 captured by the underwater event camera. The event stream E0 follows the RGB image, and the event stream E0 enhances the multi-degraded RGB image during the underwater multi-degraded image enhancement process. S3. Use the target recognition data annotation tool to annotate the RGB image, and divide the environmental information according to the annotation results to obtain a sample data set; S4, using genetic algorithm to initialize and obtain the initial population, i.e. the initial enhancement parameters; S5. Performing initial enhancement on the environmental information using the initial enhancement parameters to obtain an initial enhanced image dataset; S6. Using YOLOv5 as the basic framework, an underwater target recognition algorithm model is constructed. The initial enhanced image dataset is input into the underwater target recognition algorithm model to obtain the mean average precision (mAP) of underwater target detection. S7. Take the mean average precision (mAP) of target detection as the fitness value of the genetic algorithm. The initial enhancement parameters are optimized through selection-crossover-mutation operations to obtain the next generation population, and then the original image, i.e., the multi-degraded environmental information, is enhanced. The maximum evolutionary number of the genetic algorithm is preset. S5 and S6 are repeated until the genetic algorithm reaches the maximum evolutionary number to obtain the target enhancement parameters. Combined with the target enhancement parameters, the underwater multi-degraded image enhancement algorithm is used to enhance the environmental information to obtain a target enhanced image. The YOLOv5 model is used for target recognition to obtain the optimized target recognition result.

2. The method for joint optimization of underwater multi-degraded image enhancement and target recognition based on genetic algorithm according to claim 1, characterized in that: In S5, the initial enhancement parameters are used to perform initial enhancement on the environmental information to obtain the initial enhanced image dataset. The specific contents include: S301, the environmental information enters the preprocessing stage to obtain preprocessing information, the environmental information preprocessing stage includes an event stream reconstruction unit, an RGB image color correction unit, and the preprocessing information includes an event reconstruction image and RGB color corrected images ; S302: The pre-processed information enters the defogging stage to obtain enhanced environmental information data, wherein the enhanced environmental information data includes a fog-free event image and a fog-free RGB image. The defogging stage includes an event reconstruction image defogging unit and an RGB image defogging unit. S303: The enhanced environmental information data enters the image fusion unit to obtain a fused image, ie, an initial enhanced image.

3. The method for joint optimization of underwater multi-degraded image enhancement and target recognition based on genetic algorithm according to claim 2, characterized in that: The specific content of the event stream reconstruction unit in S301 is: The event stream is the data sequence E i =(x i ,y i , t i , p i ), which contains information about the pixel position (x i ,y i ), sampling time t i and change symbol p i information; Reconstruct the event stream E0 into an intensity image using the E2VID method ; The specific content of the E2VID method includes using a convolutional recurrent network to learn intensity frame reconstruction from asynchronous event data; Building an improved recursive network based on the UNet architecture; The event simulator is used to generate 1000 simulated event sequences from the event stream to train and test the network, of which 950 are training simulated event sequences and 50 are verification simulated event sequences. Divide the simulation event sequence into a series of non-overlapping space-time windows and convert them into event tensors; The simulated event sequence in each spatiotemporal window is sequentially input into the trained network, and the intensity frames are reconstructed to obtain the event reconstructed image.

4. The method for joint optimization of underwater multi-degraded image enhancement and target recognition based on genetic algorithm according to claim 3, characterized in that: The specific contents of the RGB image color correction unit in S301 are: According to the average intensity value of the three color channels R, G, and B in the underwater RGB image; The expression for calculating the average intensity value is as follows: ； in, and Represent the height and width of the input image respectively, is the pixel position, represents the average intensity value of the three color channels R, G, and B of the input image; According to the average intensity value from high to low, the three color channels of R, G, and B are divided into: high-quality color channel, medium-quality color channel, and low-quality color channel; Using the high-quality color channel as a reference, calculate the compensation factor for the medium-quality color channel compensation , and a compensation factor for compensating for low-quality color channels ; Calculating the compensation factor and compensation factor The formula is as follows: ； ； Use compensation factors and compensation factor , to compensate for low and medium quality color channels; The compensation expression is as follows: ； ； The white balance algorithm based on the gray world hypothesis is used for further color correction to obtain the color-corrected RGB image. That is, RGB color corrected image.

5. The method for joint optimization of underwater multi-degraded image enhancement and target recognition based on genetic algorithm according to claim 4, characterized in that: The specific contents of the event reconstruction image defogging unit in S302 are: Event image reconstruction using a multi-scale fusion framework Atomization treatment; Event reconstruction image Generate a sharpened image and gamma-corrected image sequences ; Event reconstruction image Generate a sharpened image The calculation expression is: ； in, Represents the input image A blurred version of represents the linear normalization operator; Event reconstruction image Generate gamma-corrected image sequence The expression is: ； in, is a positive constant, The value range is [1.0,3.0], is the number of different exposure images, ; 、 、 There are three parameters to be adjusted; Contrast weighted map and exposure weight graph As an indicator, the Laplace pyramid fusion method is used to reconstruct the image after four processed events The fusion of : A sharpened version and three gamma-corrected versions , get the event image after defogging That is, the fog-free event map; Contrast weight map and exposure weight graph The expressions are: ； in,( x , y ) is the pixel position; ； in Reconstruct an image for the processed events, is the Laplace operator, is the standard deviation parameter, set to 0.2, is an exponential function.

6. The method for joint optimization of underwater multi-degraded image enhancement and target recognition based on genetic algorithm according to claim 5, characterized in that: The specific content of the RGB image defogging unit in S302 is: Color correction of RGB images using a multi-scale fusion framework Atomization treatment; Color correct RGB image to produce sharpened image and gamma-corrected image sequences ; Sharpen the image The calculation expression is: ； Gamma-corrected image sequence The calculation expression is: ； in, is a positive constant, The value range is [1.0,3.0], is the number of different exposure images, ; 、 、 There are three parameters to be adjusted; Contrast weighted map and exposure weight graph As an indicator, the Laplacian pyramid fusion method is used to color correct the RGB image Four versions of The fusion of : A sharpened version and 3 gamma-corrected versions Fusion is performed to obtain the defogging RGB image That is, a fog-free RGB image; Contrast weight map and saturation weight map The calculation expression is: ； ； in, is the jth version of the color-corrected RGB image; 、 、 are the R, G, and B color channels of the j-th version of the RGB image, It is the average of the three RGB color channel values.

7. The method for joint optimization of underwater multi-degraded image enhancement and target recognition based on genetic algorithm according to claim 6, characterized in that: In step S303, the enhanced environmental information data enters the image fusion unit to obtain a fused image, i.e., the initial enhanced image. The specific content is: The illumination map is extracted from the enhanced RGB image using the RRDNET image decomposition method. ; Use gamma correction to adjust the illumination map , the expression for adjusting the illumination map using the gamma correction method is as follows: ； in, is a positive constant, The value range is [0.1, 3.0], is the number of different exposure images, =3, 、 、 There are three parameters to be adjusted; Enhanced event reconstruction using gamma correction exposure, resulting in Multiple exposure images, using gamma correction to enhance event reconstruction The expression of exposure is as follows: ； in, is a positive constant, The value range is [0.1, 3.0], is the number of different exposure images, =3, 、 、 There are three parameters to be adjusted; The weight map of each artificially exposed image is set using three features: contrast, exposure, and average brightness; Multi-scale fusion strategy to fuse multiple exposure image sequences and weight maps to generate fused images ; Based on Retinex theory, the fused image Restore to RGB image , the fused image The expression for restoring to RGB image is: ； in, is the RGB image after defogging; is the fused image.

8. The method for joint optimization of underwater multi-degraded image enhancement and target recognition based on genetic algorithm according to claim 7, characterized in that: Using YOLOv5 as the basic framework, an underwater target recognition algorithm model is constructed. The initial enhanced image dataset is input into the underwater target recognition algorithm model to obtain the mean average precision (mAP) of underwater target detection. The specific contents are as follows: S501, image input: adjust the initial enhanced image to a fixed size of 640 640, and normalized to ensure that the images input to the YOLOv5 target detection model maintain a consistent scale in diverse environments; S502, Feature Extraction and Fusion: Using CSPDarknet53 as the backbone network, we extract underlying features from the enhanced underwater images and introduce a spatial attention mechanism to further improve the feature extraction capability for small and blurred targets in underwater environments. We then fuse the extracted underlying features using the Feature Pyramid Network (FPN) and the Path Augmentation Network (PANet) to generate multi-scale feature maps. S504, Bounding Box and Classification Prediction: For each grid cell in the feature map, the model simultaneously outputs multiple predictions, including the target box, target confidence, and target category incidence. Maximum suppression (NMS) is used to eliminate overlapping prediction boxes and retain the box with the highest confidence. S506, model training: by calculating the gradient of the loss function, using the optimizer for backpropagation, and updating the target recognition network weights. The loss functions used include bounding box regression loss, classification loss, and confidence loss. Total loss function The formula is as follows: ； in: is the bounding box regression loss, is the confidence loss, is the classification loss; S507. After training is completed, the average precision (mAP) of target detection is calculated on the test set: ； in, is the number of target categories, For the The average precision of the categories.

9. The method for joint optimization of underwater multi-degraded image enhancement and target recognition based on genetic algorithm according to claim 8, characterized in that: The average precision mAP of underwater target detection is used as the fitness value of the genetic algorithm. The initial enhancement parameters are optimized through selection-crossover-mutation operations to obtain the next generation population, and then the original image is enhanced to obtain an enhanced image. The maximum evolutionary number of the genetic algorithm is preset. S5 and S6 are repeated until the genetic algorithm reaches the maximum evolutionary number to obtain the target enhancement parameters. Combined with the target enhancement parameters, the underwater multi-degraded image enhancement algorithm is used to enhance the environmental information to obtain the target enhanced image. The specific content is as follows: Genetic algorithm is used to optimize the initial enhancement parameters, which include 、 、 、 、 、 、 、 、 、 、 、 , the specific steps are: S601, initialization: set counter u=0, maximum evolutionary generation U=20, population size t=30; The 12 parameters to be adjusted in the multi-degraded image enhancement method are coded into chromosome real numbers to generate a sample p containing 12 genetic genes. t , combined with the value range of the parameter to be adjusted, randomly initialize 30 samples As the initial population P 0； S602, sample fitness value calculation: using the average detection accuracy as the genetic fitness function, calculate P one by one u The fitness value of all samples in; S603, sample selection: based on population P u The fitness value of the sample in the 30 samples is calculated, and the normalized probability of the fitness value of the 30 samples is calculated. Then, the roulette wheel method is used to select 30 samples to generate a new population P. u+1 ; S604, crossover operation: select the seed from the population P according to the crossover probability u+1 Select two chromosomes, randomly determine the crossover position on the chromosome, and perform a crossover operation on the gene; S605, mutation operation: according to the mutation probability, in the population P u+1 Randomly select the individuals that need to mutate, then randomly select two mutation positions on the chromosome and exchange the genes at these two positions; S606: Merge the selected parent chromosome with the offspring chromosome generated by crossover and mutation operations to form a new generation population P. u+1 , calculate the new generation population P u+1 The fitness value of the current generation is determined to determine whether the optimal result of the current generation meets the conditions for terminating the iteration; if , the termination condition is not met, then S602–S606 are repeated; if , the iteration is terminated, and the sample with the largest fitness in the last generation of population is output.

Images