Image processing method for heterogeneous imager array of underwater robot

Abstract

The invention relates to an image processing method for an underwater robot heterogeneous imager array, and the method comprises the steps: S1, collecting an original underwater image, and carrying out the brightness adjustment, and obtaining a first image with balanced brightness; s2, analyzing pixel distribution of the first image, and judging and executing de-scattering processing to obtain a second image; s3, performing multi-scale detail enhancement on the second image to obtain a third image with richer details; s4, evaluating color distribution of the third image, and performing color correction to obtain a fourth image; s5, performing adaptive denoising processing on the noise features of the fourth image to obtain a fifth image; s6, according to the contrast characteristic of the fifth image, contrast adjustment is carried out to ensure that a preset threshold value is met, and a sixth image is obtained; s7, edge enhancement processing is carried out on the sixth image, and a seventh image with the clear edge is obtained.By means of illumination compensation, scattering removal and multi-scale detail enhancement, the influence of complex optical conditions of the underwater environment on the image quality is effectively overcome.

Description

Technical Field

[0001] The present application relates to the technical field of control of non-electrical variables, and in particular to an image processing method for a heterogeneous imager array of an underwater robot. Background Art

[0002] Image acquisition and processing in underwater environments face a series of complex technical challenges. Although heterogeneous imager arrays can capture image data of different spectra and resolutions, the original images often have problems such as insufficient lighting, scattering, loss of details, color deviation, noise interference, and insufficient contrast.

[0003] The above problems jointly affect the quality and usability of underwater images. Insufficient lighting will cause the overall image to be dark, while the scattering effect will make the image blurred. The lack of detail information and the presence of noise further reduce the clarity and recognizability of the image. Color deviation will affect the authenticity of the image and the accuracy of subsequent analysis. Insufficient contrast makes it difficult to distinguish between the target and the background, and unclear edge features will affect the contour recognition of the target. The shortcomings of the above-mentioned existing technologies seriously restrict the application of underwater images in target detection, image segmentation and feature extraction and subsequent analysis tasks.

[0004] The present invention proposes a solution to the above-mentioned shortcomings of the prior art as follows: first, the original underwater image is acquired through a heterogeneous imager array, and the uneven brightness of the image is compensated by brightness adjustment. Next, the pixel distribution characteristics of the image are analyzed to determine whether there is scattering, and a deep learning algorithm is used to remove the influence of scattering. Then, the image is decomposed into multiple scales through wavelet transform, and sub-band images of different frequency bands are enhanced to achieve multi-scale detail enhancement. In addition, the color distribution of the image is analyzed to determine whether there is color deviation, and color balance is achieved through a color correction algorithm. At the same time, according to the noise characteristics of the image, a denoising algorithm is adaptively selected and denoising is performed. Finally, according to the contrast characteristics of the image, the image contrast is adjusted through a histogram equalization algorithm. Summary of the invention

[0005] The purpose of the present application is to provide an image processing method for a heterogeneous imager array of an underwater robot.

[0006] 1. The image processing method of the heterogeneous imager array of an underwater robot described in the present application comprises the following steps:

[0007] Step S1, obtaining an original underwater image collected by a heterogeneous imager array, wherein the heterogeneous imager array is used to capture image data of different spectra and resolutions, and adjusting the brightness of the image by using an illumination compensation algorithm to obtain a first image with balanced brightness in view of insufficient illumination;

[0008] Step S2, judging whether there is a scattering effect according to the pixel distribution characteristics of the first image in step S1, wherein the scattering effect may cause image blur and reduced contrast; if there is a scattering effect, a descattering algorithm is used to process the first image to obtain a descattered second image;

[0009] Step S3, using a multi-scale detail enhancement algorithm to extract high-frequency detail features in the image based on the detail information of the second image in step S2, to obtain a third image with enhanced details;

[0010] Step S4, judging whether there is color deviation according to the color distribution of the third image in step S3, and if there is color deviation, performing color balance processing on the third image using a color correction algorithm to obtain a color-corrected fourth image;

[0011] Step S5, according to the noise characteristics of the fourth image in step S4, adopt an adaptive denoising algorithm to suppress the noise of the image to obtain a denoised fifth image;

[0012] Step S6, judging whether a preset contrast threshold is met according to the contrast feature of the fifth image in step S5, and if not, adjusting the contrast of the fifth image by using a contrast enhancement algorithm to obtain a sixth image with enhanced contrast;

[0013] Step S7, using an edge enhancement algorithm to enhance the edge of the sixth image in step S6, thereby improving the clarity and detail recognizability of the image and obtaining a seventh image with clear edges;

[0014] Step S8, judging whether the overall quality of the seventh image in step S7 meets the requirements of subsequent image analysis tasks, including target detection, image segmentation and feature extraction, and if so, outputting the seventh image as the final enhancement result.

[0015] 2. Preferably, in the step S1, the original underwater image captured by the heterogeneous imager array is obtained, and the brightness is adjusted to obtain a first image with balanced brightness to address the problem of insufficient illumination, including: obtaining the original underwater image data captured by the heterogeneous imager array to obtain an initial image set; extracting the brightness histogram features of each image in the initial image set, and determining the average brightness value of the image according to the brightness histogram features; calculating the average brightness value of all images in the initial image set, and determining the value as the target brightness value; according to the difference between the average brightness value of each image and the target brightness value, using a piecewise linear transformation method to compensate for the pixel brightness value of the image to obtain a brightness-adjusted image; and splicing the brightness-adjusted images to obtain a brightness-balanced image.

[0016] 3. Preferably, in the step S2, whether there is a scattering effect is determined according to the pixel distribution characteristics of the first image, and if there is a scattering effect, descattering processing is performed to obtain a second image, including: obtaining pixel distribution feature data of the first image; inputting the pixel distribution feature data into a scattering effect judgment model, and outputting a scattering effect probability; if the scattering effect probability is greater than a preset scattering effect threshold, it is determined that there is a scattering effect on the first image; inputting the first image with a scattering effect into a descattering processing module, processing it with a descattering algorithm based on deep learning, extracting the scattering features of the first image through a convolutional neural network, and modeling the scattering features using a scattering degradation model, removing the scattering by optimizing the parameters of the scattering degradation model, and obtaining a descattered second image.

[0017] 4. Preferably, in the step S3, the detail information of the second image is subjected to multi-scale detail enhancement to obtain a third image with enhanced detail, including: using wavelet transform to perform multi-scale decomposition on the second image to obtain sub-band images of different frequency bands; for the high-frequency sub-band image, using a nonlinear mapping function to enhance the high-frequency details in the high-frequency sub-band image to obtain an enhanced high-frequency sub-band image; for the low-frequency sub-band image, using histogram equalization to enhance the low-frequency information in the low-frequency sub-band image to obtain an enhanced low-frequency sub-band image; based on the enhanced high-frequency sub-band image and the enhanced low-frequency sub-band image, using inverse wavelet transform to reconstruct a detail-enhanced image, and the detail-enhanced image is the third image.

[0018] 5. Preferably, in the step S4, whether there is a color deviation is determined based on the color distribution of the third image, and if there is a color deviation, color correction is performed to obtain a fourth image, including: obtaining a color distribution histogram of the third image, and respectively counting the distribution of pixel values ​​of the three color channels of red, green and blue to obtain a color distribution feature vector; comparing the color distribution feature vector with the third image according to a preset color balance standard model to calculate the color deviation degree; if the color deviation degree exceeds a preset threshold, it is determined that the third image has a color deviation; and using a gray world algorithm to perform color correction on the third image, and adjusting the gain coefficient of each channel so that the average values ​​of the three RGB channels of the corrected image are equal.

[0019] 6. Preferably, in the step S5, the noise characteristics of the fourth image are adaptively denoised to obtain the fifth image, including: obtaining noise characteristic information of the fourth image, the noise characteristic information including noise type and noise intensity; adaptively determining parameter settings of a denoising algorithm according to the noise characteristic information of the fourth image, the parameter settings including filter type, filter size and filter intensity; performing noise suppression processing on the fourth image using the adaptively determined denoising algorithm to obtain a preliminary denoised image; adopting different denoising strategies for different areas of the fourth image; dividing the fourth image into a plurality of sub-blocks, performing denoising processing on each of the sub-blocks respectively, to obtain denoising results of each of the sub-blocks; and splicing the denoising results of each of the sub-blocks to obtain a complete denoised image.

[0020] 7. Preferably, in step S6, the contrast feature of the fifth image is judged whether it meets a preset contrast threshold, and if not, the contrast is adjusted to obtain the sixth image, including: obtaining a contrast feature value of the fifth image, and comparing the contrast feature value with a preset contrast threshold; if the contrast feature value of the fifth image does not meet the preset threshold, determining a target interval for contrast enhancement according to the pixel value distribution of the fifth image; for pixels in the fifth image located in the target interval, using a histogram equalization algorithm to perform nonlinear stretching on their brightness values ​​to improve the contrast of the pixels in the target interval; for pixels in the fifth image that do not fall into the target interval, adjusting their brightness values ​​by a piecewise linear transformation according to their distance from the target interval.

[0021] 8. Preferably, in step S7, the edge features of the sixth image are subjected to edge enhancement processing to obtain the seventh image, including: obtaining edge feature information of the sixth image, and inputting the edge feature information into an edge enhancement algorithm for processing; the edge enhancement algorithm uses a Sobel operator to detect the edge of the image, and determines whether the pixel is an edge point by calculating the gradient value between the image pixel and the surrounding pixels; for pixels judged to be edge points, their grayscale values ​​are adjusted according to the size of their gradient values ​​using a nonlinear transformation function; for pixels that are not edge points, their original grayscale values ​​are maintained unchanged, and the original detail information of the image is retained; through the above processing, the edge enhancement result of the image is obtained.

[0022] The image processing method of an underwater robot heterogeneous imager array described in the present application has the advantage of addressing the problems of insufficient lighting, scattering effect, missing details, color deviation, noise interference and insufficient contrast in the original underwater images collected by the heterogeneous imager array.

[0023] The present invention gradually improves the image quality through a series of algorithms including illumination compensation, descattering processing, multi-scale detail enhancement, color correction, adaptive denoising and contrast enhancement. Finally, an edge enhancement algorithm is used to further improve the image clarity and detail recognizability.

[0024] The technical effect of the present invention is that it can effectively solve the impact of complex optical conditions in underwater environments on image quality, significantly improve the overall quality of underwater images, and provide high-quality image input for subsequent target detection, image segmentation, and feature extraction and analysis tasks, thereby improving the accuracy and efficiency of underwater image processing and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is the process of an image processing method for an underwater robot heterogeneous imager array described in this application Figure 1 ;

[0026] Figure 2 This is the process of an image processing method for an underwater robot heterogeneous imager array described in this application Figure 2 . DETAILED DESCRIPTION

[0027] like Figure 1-Figure 2 As shown, the image processing method of an underwater robot heterogeneous imager array described in the present application comprises the following steps:

[0028] Step S1, obtaining an original underwater image collected by a heterogeneous imager array, wherein the heterogeneous imager array is used to capture image data of different spectra and resolutions, and adjusting the brightness of the image by using an illumination compensation algorithm to obtain a first image with balanced brightness in view of insufficient illumination;

[0029] Step S2, judging whether there is a scattering effect according to the pixel distribution characteristics of the first image in step S1, wherein the scattering effect may cause image blur and reduced contrast; if there is a scattering effect, a descattering algorithm is used to process the first image to obtain a descattered second image;

[0030] Step S3, using a multi-scale detail enhancement algorithm to extract high-frequency detail features in the image based on the detail information of the second image in step S2, to obtain a third image with enhanced details;

[0031] Step S4, judging whether there is color deviation according to the color distribution of the third image in step S3, and if there is color deviation, performing color balance processing on the third image using a color correction algorithm to obtain a color-corrected fourth image;

[0032] Step S5, according to the noise characteristics of the fourth image in step S4, adopt an adaptive denoising algorithm to suppress the noise of the image to obtain a denoised fifth image;

[0033] Step S6, judging whether a preset contrast threshold is met according to the contrast feature of the fifth image in step S5, and if not, adjusting the contrast of the fifth image by using a contrast enhancement algorithm to obtain a sixth image with enhanced contrast;

[0034] Step S7, using an edge enhancement algorithm to enhance the edge of the sixth image in step S6, thereby improving the clarity and detail recognizability of the image and obtaining a seventh image with clear edges;

[0035] Step S8, judging whether the overall quality of the seventh image in step S7 meets the requirements of subsequent image analysis tasks, including target detection, image segmentation and feature extraction, and if so, outputting the seventh image as the final enhancement result.

[0036] like Figure 1-Figure 2 As shown, the working principle of the step S1 to step S8 in the embodiment is:

[0037] Step S1: Collect 100 underwater original images and calculate the average brightness of each image (the first image is 120); calculate the average brightness of the whole atlas 130 and set it as the target value; perform segmented brightness compensation on each image: +10 for pixels below 130 and -10 for pixels above 130; stitch the compensated images and use 5x5 median filter to remove noise; use Canny operator to detect edges and perform nonlinear enhancement on the grayscale value of edge pixels (y=0.02x 2 +1.5x+10);

[0038] Step S2: Detect image contrast (if < 0.6, it is determined that there is scattering); input the convolutional neural network de-scattering model and output a clear image; use the Sobel operator to detect edges (gradient > 50 is an edge point); perform y = 2x on the gray value of the edge point 2 Enhanced, non-edge points remain unchanged; use Brenner gradient function to evaluate clarity (>0.8 is qualified);

[0039] Step S3: Perform 3-level Haar wavelet decomposition on the image to obtain high-frequency (LH / HL / HH) and low-frequency (LL) subbands; for the high-frequency subband, use y=1.5x 0 · 9 Enhancement, low-frequency subband histogram equalization; inverse wavelet reconstruction, edge extraction using 5x5 Laplacian template; fusion of original image and edge map (weight 3:7), local contrast adaptive adjustment;

[0040] Step S4: Count the mean of RGB channels (R=100, G=120, B=180); calculate the Euclidean distance with the standard model (R=150, G=150, B=150) (>0.2 needs to be corrected); adjust the gain to make the RGB mean equal (R×1.5, G×1.25, B×0.83); score with the ResNet model (>4 points pass), otherwise reduce the correction intensity;

[0041] Step S5: Detect the noise type (Gaussian noise, σ=15); select 7x7 non-local mean filter (strength 0.8); segment the image into 64x64 sub-blocks for independent denoising; calculate SNR (>30dB) and PSNR (>36dB), and increase the filter strength if they do not meet the standards;

[0042] Step S6: Calculate image contrast (if < 0.5, enhance); equalize the histogram of pixels with grayscale 50-150; perform piecewise linear mapping on pixels with grayscale < 50 and > 150; detect edges with Sobel operator and enhance edge grayscale (f(x) = 2x 0 · 8 );

[0043] Step S7: Use the Canny operator to detect edges (gradient > 0.6 is an edge point); perform Gamma transformation on edge points (coefficient 1.5); calculate the gradient variance (> 5) and edge information entropy (> 2.8), and adjust the parameters if they do not meet the standards;

[0044] Step S8: Gaussian filter denoising (5x5, σ=1.5); Tenengrad score (>0.7 is qualified); use YOLOv3 to detect the target (confidence>0.5), FCN to segment the foreground, and VGG16 to extract features; if the comprehensive result meets the standard, it is output, otherwise return to S1 optimization.

[0045] like Figure 1-Figure 2 In the step S1, an original underwater image captured by a heterogeneous imager array is obtained. The heterogeneous imager array is used to capture image data of different spectra and resolutions. To address the problem of insufficient illumination, an illumination compensation algorithm is used to adjust the brightness of the image to obtain a first image with balanced brightness.

[0046] Further, in step S1, original underwater image data collected by the heterogeneous imager array is acquired to obtain an initial image set;

[0047] For each image in the initial image set, extract its brightness histogram feature, and determine the average brightness value of the image according to the brightness histogram feature;

[0048] Calculate the average brightness value of all images in the initial image set and determine the value as the target brightness value;

[0049] For each image in the initial image set, according to the difference between its average brightness value and the target brightness value, a piecewise linear transformation method is used to compensate the pixel brightness value of the image to obtain a brightness-adjusted image;

[0050] The brightness-adjusted images are stitched together to obtain a brightness-balanced image. The brightness-balanced image is smoothed and denoised using a median filter algorithm to obtain an underwater scene image.

[0051] Obtain edge feature information of the underwater scene image, and input the edge feature information into an edge enhancement algorithm for processing;

[0052] The edge enhancement algorithm uses the Sobel operator to detect the edge of the image and determines whether the pixel is an edge point by calculating the gradient value between the image pixel and the surrounding pixels.

[0053] For pixels judged as edge points, a nonlinear transformation function is used to adjust their grayscale values ​​according to the size of their gradient values ​​to achieve edge enhancement effects;

[0054] For pixels that are not edge points, their original grayscale values ​​are maintained unchanged, retaining the original detail information of the image;

[0055] The quality of the edge-enhanced image is evaluated by calculating the image clarity index and edge information entropy to determine whether the image quality meets the expected requirements. If not, the image is returned and edge enhancement is performed again until the quality requirements are met. Finally, an underwater target image with clear edge contours and preserved details is obtained.

[0056] Specifically, in step S1, the original underwater image data collected by the heterogeneous imager array is first acquired to obtain an initial image set including 100 images of different spectra and resolutions;

[0057] Then, for each image in the initial image set, extract its 256-level grayscale brightness histogram feature, and determine the average brightness value of the image according to the brightness histogram feature, including the first image whose average brightness value is 120;

[0058] Then the average brightness value of the 100 images in the initial image set is calculated, and the value 130 is determined as the target brightness value;

[0059] For each image in the initial image set, the pixel brightness value of the image is compensated by piecewise linear transformation method according to the difference between its average brightness value and the target brightness value, including that the difference between the average brightness value 120 of the first image and the target brightness value 130 is 10, then the brightness value of the pixels with brightness values ​​less than 120 in the first image is increased by 10;

[0060] For pixels with brightness values ​​greater than 120, the brightness value is reduced by 10, thus obtaining an image with brightness adjustment;

[0061] The 100 brightness-adjusted images are stitched together to obtain a brightness-balanced image with a size of 1000*1000;

[0062] For the brightness balanced image, a 5*5 median filter algorithm is used to smooth and reduce noise, remove speckle noise, and obtain an underwater scene image;

[0063] Obtain edge feature information of underwater scene images, detect edge pixels of the image using the Canny operator, and input the edge feature information into the edge enhancement algorithm for processing;

[0064] The edge enhancement algorithm uses the Sobel operator to detect the edge of the image. By calculating the gradient value between the image pixel and its 8 neighboring pixels, the pixel with a gradient value greater than the threshold of 50 is determined as an edge point.

[0065] For pixels judged as edge points, the grayscale value is adjusted according to the size of its gradient value using the nonlinear transformation function y=ax^2+bx+c, where a=0.02, b=1.5, c=10, to achieve edge enhancement effect;

[0066] For pixels that are not edge points, their original grayscale values ​​are maintained unchanged, retaining the original detail information of the image;

[0067] Finally, the quality of the edge-enhanced image is evaluated, and the clarity index of the image is calculated by the Tenengrad function. If the clarity index is greater than 0.85, it indicates that the image quality meets the expected requirements. If it is less than 0.85, it returns to the edge enhancement step and is reprocessed until the quality requirements are met. Finally, an underwater target image with clear edge contours and preserved details is obtained.

[0068] like Figure 1-Figure 2 As shown, in step S2, based on the pixel distribution characteristics of the first image, it is determined whether there is a scattering effect, which will cause image blur and reduced contrast. If there is a scattering effect, a descattering algorithm is used to process the first image to obtain a descattered second image.

[0069] Further, in step S2, it is determined whether there is a scattering effect according to the pixel distribution characteristics of the first image, and the scattering effect may cause image blur and reduced contrast;

[0070] If there is a scattering effect, the first image with the scattering effect is input into a de-scattering processing module, and the first image is processed by a de-scattering algorithm based on deep learning;

[0071] The descattering algorithm extracts the scattering features of the first image through a convolutional neural network, models the scattering features using a scattering degradation model, and removes the scattering of the first image by optimizing the parameters of the scattering degradation model;

[0072] A second image after descattering is obtained. Compared with the original first image, the scattering effect of the second image is removed, and the image clarity and contrast are improved;

[0073] Acquire edge feature information of the second image, and input the edge feature information into an edge enhancement algorithm for processing;

[0074] The edge enhancement algorithm uses the Sobel operator to detect the edge of the image and determines whether the pixel is an edge point by calculating the gradient value between the image pixel and the surrounding pixels.

[0075] For pixels judged as edge points, a nonlinear transformation function is used to adjust their grayscale values ​​according to the size of their gradient values ​​to achieve edge enhancement effects;

[0076] For pixels that are not edge points, their original grayscale values ​​are maintained unchanged, and the original detail information of the image is retained. Through the above processing, an edge enhancement result of the image is obtained, the edge contour is clearer and more obvious, and the image details are retained. The edge-enhanced image is output as the third image;

[0077] The quality of the third image is evaluated by calculating the image clarity index and edge information entropy to determine whether the image quality meets the expected requirements. If not, the image is returned and edge enhancement is performed again until the quality requirements are met. Finally, a high-quality image with scatter removal and edge enhancement is output.

[0078] Specifically, in step S2, according to the pixel distribution characteristics of the first image, by calculating the grayscale histogram and contrast index of the image, it is determined whether the image contrast is lower than a threshold value of 0.6. If so, it is considered that there is a scattering effect;

[0079] The scattering image is input into the descattering model based on the convolutional neural network. The model extracts the multi-scale features of the image, learns the mapping relationship between the scattering features and the non-scattering image, and uses the scattering degradation model to model the scattering features. By minimizing the reconstruction error between the scattering image and the non-scattering image, the model parameters are optimized to achieve scattering removal and obtain a second image after descattering.

[0080] The Sobel operator is used to perform edge detection on the second image. By setting a threshold of 0.2, pixels with gradient values ​​greater than the threshold are judged as edge points. The grayscale values ​​of edge point pixels are nonlinearly transformed, and the transformation function is y=2x^2. The grayscale values ​​of non-edge point pixels remain unchanged, and the edge-enhanced third image is obtained.

[0081] The clarity index of the third image is calculated using the Brenner gradient function. If the clarity index is greater than 0.8, the image quality is considered to meet the requirements. Otherwise, the image quality is returned to the edge enhancement step and processed until the quality requirements are met. The final high-quality scatter removal and edge enhancement image is output.

[0082] like Figure 1-Figure 2 As shown, in step S3, a multi-scale detail enhancement algorithm is used for the detail information of the second image to extract high-frequency detail features in the image to obtain a third image with enhanced details.

[0083] Further, in step S3, an image to be processed is obtained, the image to be processed is a second image, and wavelet transform is used to perform multi-scale decomposition on the second image to obtain sub-band images of different frequency bands, the sub-band images include high-frequency sub-band images and low-frequency sub-band images;

[0084] For the high-frequency sub-band image, a nonlinear mapping function is used to enhance the high-frequency details in the high-frequency sub-band image to obtain an enhanced high-frequency sub-band image;

[0085] For the low-frequency sub-band image, using histogram equalization to enhance the low-frequency information in the low-frequency sub-band image to obtain an enhanced low-frequency sub-band image;

[0086] Reconstructing a detail-enhanced image by using inverse wavelet transform according to the enhanced high-frequency sub-band image and the enhanced low-frequency sub-band image, wherein the detail-enhanced image is a third image, and extracting edge detail features of the third image by using a Laplace operator to obtain an edge-enhanced image;

[0087] According to the edge enhanced image, using morphological filtering to smooth edge details in the edge enhanced image to obtain a smoothed edge image;

[0088] Fusing the smoothed edge image with the third image to obtain a target image with enhanced details;

[0089] The target image for detail enhancement is subjected to local contrast adjustment, and the contrast value in the local area is calculated, the pixel grayscale is adaptively adjusted, and the local details are enhanced to obtain a final detail enhanced image.

[0090] Specifically, in step S3, firstly, the second image to be processed is obtained, and the image is decomposed into three levels using Haar wavelet to obtain a multi-scale representation including LL3, LH3, HL3, and HH3 subbands;

[0091] For the high-frequency subbands of LH3, HL3, and HH3, the power law function y=c*x^γ (c=1.5, γ=0.8) is used for nonlinear mapping to enhance the high-frequency details;

[0092] For the LL3 low-frequency subband, the cumulative probability density function is calculated and equalized to the range of 0-255 to highlight the low-frequency contrast;

[0093] Then, the enhanced high and low frequency sub-bands are subjected to a 3-level inverse wavelet transform to reconstruct a third image with enhanced details;

[0094] Then, the third image is convolved with a 5x5 Laplacian template to extract edge contour information, and the edge image is closed twice with a 3x3 structure element to smooth the edge;

[0095] Finally, the edge image and the third image are added and fused with a weight of 3:7, and the local standard deviation in the 8x8 neighborhood window is calculated as the local contrast adjustment coefficient to adaptively enhance local details, thereby obtaining the final detail-enhanced image.

[0096] like Figure 1-Figure 2 As shown, in step S4, whether there is color deviation is determined according to the color distribution of the third image. If there is color deviation, a color correction algorithm is used to perform color balance processing on the third image to obtain a color-corrected fourth image.

[0097] Furthermore, in step S4, a color distribution histogram of the third image is obtained, and pixel value distributions of the three color channels of red, green and blue are respectively counted to obtain a color distribution feature vector;

[0098] According to a preset color balance standard model, the color distribution feature vector of the third image is compared to calculate the color deviation degree. If the color deviation degree exceeds a preset threshold, it is determined that the third image has color deviation and needs to be corrected.

[0099] The gray world algorithm is used to perform color correction on the third image. The algorithm assumes that the average values ​​of the three RGB channels in the image should be equal. By adjusting the gain coefficient of each channel, the average values ​​of the three RGB channels of the corrected image are made equal, thereby achieving color balance.

[0100] Based on the gray world algorithm, the color correction result is optimized by combining the color saturation information of the image to avoid over-saturation or over-grayness and improve the authenticity of color restoration.

[0101] Perform edge detection on the color-corrected image and extract the texture features of the image. By comparing the changes in the texture features, determine the degree of detail retention of the image before and after color correction. If the loss of details is serious, reduce the intensity of color correction and perform correction again.

[0102] By training the color evaluation model, the color correction results are scored. The scoring results are used as feedback signals to dynamically adjust the parameters of the color correction algorithm to achieve adaptive optimization and obtain the color-corrected image with the best visual effect.

[0103] Output the color-corrected image as the fourth image, and attach key parameter information in the correction process to the image file as metadata to facilitate subsequent analysis and processing;

[0104] Performing a quality assessment on the fourth image, by calculating the clarity index and color information entropy of the image, to determine whether the image quality meets the expected requirements; if not, returning to the third step to perform color correction again until the quality requirements are met;

[0105] The fourth image is compared and analyzed with the third image. By calculating the difference in the color histograms of the two images, the changes before and after color correction are quantified, and a color correction effect evaluation report is generated to provide data support for the subsequent optimization of the color correction algorithm.

[0106] Specifically, in step S4, by counting the pixel value distribution of the three color channels of the image RGB, a color distribution feature vector is obtained, including [0.2, 0.3, 0.5];

[0107] Compare it with the preset color balance standard model [0.3, 0.4, 0.3], and calculate the Euclidean distance to be 0.24, which exceeds the threshold of 0.2, and it is judged that there is color deviation;

[0108] The gray world algorithm is used to correct the color. Assuming that the average value of the three RGB channels after correction is 150, white balance is achieved by multiplying the R, G, and B channels by gain coefficients of 150 / 100, 150 / 120, and 150 / 180 respectively.

[0109] The correction results are further optimized by combining the image color saturation information. If the average saturation exceeds 80%, the correction intensity is reduced;

[0110] The Gabor texture features are extracted from the corrected image, and its mean is calculated to be 0.6, which is 25% lower than the 0.8 before correction. If it exceeds the threshold of 20%, the correction intensity is reduced by 20% and reprocessed;

[0111] The correction results were scored by training the ResNet color evaluation model, with a score of 4.2. The model was used as feedback to dynamically adjust the parameters. After three iterations of optimization, the corrected image with the best visual effect was obtained.

[0112] Finally, the clarity index of the corrected image is calculated to be 0.95 and the color information entropy is 7.2, both of which meet the quality requirements, and the image is output as the fourth image.

[0113] like Figure 1-Figure 2 As shown, in step S5, an adaptive denoising algorithm is used to suppress the noise of the fourth image according to the noise characteristics of the fourth image to obtain a denoised fifth image.

[0114] Further, in step S5, noise characteristic information of the target image is obtained, wherein the noise characteristic information includes noise type and noise intensity;

[0115] Adaptively determining parameter settings of a denoising algorithm according to noise feature information of the target image, the parameter settings including filter type, filter size, and filter strength;

[0116] Using the adaptively determined denoising algorithm to perform noise suppression processing on the target image to obtain a preliminary denoised image, and using different denoising strategies for different areas of the target image;

[0117] Dividing the target image into a plurality of sub-blocks, and performing denoising processing on each of the sub-blocks respectively to obtain denoising results of each of the sub-blocks;

[0118] The denoising results of the sub-blocks are spliced ​​to obtain a complete denoised image, and the quality of the denoised image is evaluated to obtain a signal-to-noise ratio and a peak signal-to-noise ratio of the denoised image;

[0119] Determining whether the signal-to-noise ratio and the peak signal-to-noise ratio of the denoised image meet the preset threshold requirements; if the signal-to-noise ratio and the peak signal-to-noise ratio of the denoised image do not meet the preset threshold requirements, adjusting the parameter settings of the denoising algorithm, and returning to the step of performing noise suppression processing on the target image using the adaptively determined denoising algorithm;

[0120] If the signal-to-noise ratio and the peak signal-to-noise ratio of the denoised image meet the preset threshold requirements, the denoised image is output as a target denoised image.

[0121] Specifically, in step S5, first, by analyzing the grayscale histogram and local variance of the target image, it is determined that the image mainly contains Gaussian noise, and the standard deviation of the noise is estimated to be 15;

[0122] Then, according to the noise type of Gaussian noise and the noise intensity of 15, the non-local mean filtering algorithm is adaptively selected, the filter size is set to 7x7, and the filter intensity parameter is set to 0.8;

[0123] Then, the adaptive non-local mean filtering algorithm is used to perform preliminary denoising on the entire image, and different filtering strengths are used for the smooth area and texture area of ​​the image to better preserve the texture details.

[0124] Then, the image is divided into 64x64 sub-blocks, each sub-block is independently subjected to non-local mean filtering, and the filtering results of all sub-blocks are concatenated to obtain a complete denoised image;

[0125] The signal-to-noise ratio of the denoising result is calculated, and the SNR is 35dB, and the peak signal-to-noise ratio PSNR is 38dB. The SNR and PSNR are compared with the preset thresholds of 30dB and 36dB, and it is found that they meet the quality requirements. Then the denoised image is output as the final target denoising result;

[0126] If the quality is not satisfactory, the parameters of the non-local mean filter are adjusted, including increasing the filter size and filter strength, and the target image is denoised again until the SNR and PSNR of the denoised image meet the preset threshold requirements.

[0127] like Figure 1-Figure 2 As shown, in step S6, it is determined whether a preset contrast threshold is met based on the contrast characteristics of the fifth image. If not, a contrast enhancement algorithm is used to adjust the contrast of the fifth image to obtain a sixth image with enhanced contrast.

[0128] Further, in step S6, a contrast characteristic value of the fifth image is obtained, and the contrast characteristic value is compared with a preset contrast threshold to determine whether the threshold condition is met;

[0129] If the contrast characteristic value of the fifth image does not meet the preset threshold, determining a target interval for contrast enhancement according to the pixel value distribution of the fifth image;

[0130] For the pixels in the target interval in the fifth image, a histogram equalization algorithm is used to perform nonlinear stretching on the brightness values ​​thereof, so as to improve the contrast of the pixels in the target interval;

[0131] For pixels in the fifth image that do not fall within the target interval, the brightness values ​​are adjusted by piecewise linear transformation according to their distance from the target interval to keep the overall brightness distribution similar to that of the original image;

[0132] The pixels processed by histogram equalization are merged with the pixels processed by piecewise linear transformation to obtain a sixth image with enhanced contrast;

[0133] Calculating a contrast characteristic value of the sixth image, and comparing the contrast characteristic value with a preset threshold again to determine whether the threshold condition is met;

[0134] If the contrast characteristic value of the sixth image meets the preset threshold, outputting the sixth image as a result of contrast enhancement;

[0135] Otherwise, return to the second step, redetermine the target interval according to the pixel value distribution, and continue to adjust the contrast of the sixth image;

[0136] The resulting sixth image replaces the original fifth image as input for subsequent image analysis and processing;

[0137] According to the edge feature information of the sixth image, the Sobel operator is used for edge detection, and the grayscale value of the edge pixel is adjusted through a nonlinear transformation function to achieve edge enhancement, so as to obtain the seventh image with clearer edge contour for further image quality evaluation.

[0138] Specifically, in step S6, by calculating the average gray value and standard deviation of the fifth image, the contrast characteristic value is obtained to be 0.35, which is compared with the preset threshold value of 0.5, and it is found that the threshold condition is not met;

[0139] According to the grayscale histogram analysis of the fifth image, determining that the target interval for contrast enhancement is between grayscale values ​​50 and 150;

[0140] The histogram equalization algorithm is used to perform nonlinear stretching on the grayscale values ​​of pixels within the target interval to make the grayscale distribution more uniform and improve the contrast.

[0141] For pixels with grayscale values ​​less than 50, piecewise linear transformation is used to map their grayscale values ​​to between 20 and 49;

[0142] For pixels with grayscale values ​​greater than 150, they are mapped to between 151 and 220, keeping the overall brightness distribution similar to the original image;

[0143] The processed pixels are recombined to obtain a sixth image with enhanced contrast, and the contrast characteristic value of the sixth image is calculated to be 0.58, which meets a preset threshold, and the sixth image is output;

[0144] The Sobel operator is used to perform edge detection on the sixth image to extract edge pixels, and the grayscale value of the edge pixels is adjusted using a nonlinear transformation function f(x)=2x^0.8 according to the gradient amplitude of the edge pixels to highlight the edge contour;

[0145] The processed edge image is fused with the sixth image to obtain an edge-enhanced seventh image, in which edge clarity is improved and details are preserved, providing better input for subsequent image quality evaluation.

[0146] like Figure 1-Figure 2 As shown, in step S7, an edge enhancement algorithm is used to enhance the edge features of the sixth image to improve the clarity and detail recognizability of the image, thereby obtaining a seventh image with clear edges.

[0147] Further, in step S7, edge feature information of the sixth image is obtained, and the edge feature information is input into an edge enhancement algorithm for processing;

[0148] The edge enhancement algorithm uses the Sobel operator to detect the edge of the image and determines whether the pixel is an edge point by calculating the gradient value between the image pixel and the surrounding pixels.

[0149] For pixels judged as edge points, a nonlinear transformation function is used to adjust their grayscale values ​​according to the size of their gradient values ​​to achieve edge enhancement effects;

[0150] For pixels that are not edge points, their original grayscale values ​​are maintained unchanged, retaining the original detail information of the image;

[0151] Through the above processing, the edge enhancement result of the image is obtained, the edge contour is clearer and more obvious, and the image details are preserved;

[0152] The edge-enhanced image is output as the seventh image. Compared with the sixth image, the edge features of the seventh image are more prominent, and the overall clarity and recognizability are improved;

[0153] The seventh image is evaluated for quality, and the clarity index and edge information entropy of the image are calculated to determine whether the image quality meets the expected requirements;

[0154] If the image quality does not meet the requirements, return to perform edge enhancement again;

[0155] If the image quality meets the requirements, the final seventh image with clear edges is output, completing the image edge enhancement process.

[0156] Specifically, in step S7, edge feature information of the sixth image is first obtained, including detecting the edge of the image by using a Canny operator, and inputting the coordinates and gradient values ​​of the edge pixels as edge feature information into an edge enhancement algorithm;

[0157] In the edge enhancement algorithm, the Sobel operator is used to perform convolution operation on the image, and the gradient value between each pixel and its surrounding 8 pixels is calculated. Pixels with gradient values ​​exceeding the preset threshold, including 0.6, are judged as edge points;

[0158] For pixels judged as edge points, according to the size of their gradient values, nonlinear transformation functions including Gamma transformation are used to adjust their gray values. The transformation coefficient can be set to 1.5 to achieve edge enhancement effect.

[0159] For pixels that are not edge points, their original grayscale values ​​are maintained unchanged, and the original detail information of the image is retained. Through the above processing, the edge enhancement result of the image is obtained, the edge contour is clearer and the image details are retained;

[0160] The edge-enhanced image is output as the seventh image. Compared with the sixth image, the edge features of the seventh image are more prominent, and the overall clarity and recognizability are improved;

[0161] The seventh image is evaluated for quality by calculating the image clarity index including the gradient variance, which is 5.2, and the edge information entropy, which is 2.8, to determine whether the image quality meets the expected requirements;

[0162] If the image quality does not meet the requirements, return to the edge detection step to perform edge enhancement again, adjust the convolution kernel size of the Sobel operator to 5x5, and the nonlinear transformation coefficient to 1.8 until the image quality meets the requirements;

[0163] Finally, the seventh image with clear edges is output, completing the image edge enhancement process.

[0164] like Figure 1-Figure 2 As shown, in step S8, based on the overall quality of the seventh image, it is determined whether the requirements of subsequent image analysis tasks, including target detection, image segmentation and feature extraction, are met. If so, the seventh image is output as the final enhancement result.

[0165] Further, in step S8, a seventh image is acquired, and preprocessed on the seventh image, wherein the preprocessing includes denoising and correction to obtain a preprocessed image;

[0166] Use a preset image quality assessment model to score the quality of the preprocessed image to obtain a quality score;

[0167] The quality score is compared with a preset quality threshold. If the quality score is greater than or equal to the threshold, it is determined that the quality of the preprocessed image meets the requirement; otherwise, it is determined that the quality of the preprocessed image does not meet the requirement.

[0168] If the quality of the preprocessed image meets the requirements, the preprocessed image is determined as a candidate image;

[0169] For the candidate images, the target detection algorithm is used to perform target detection and obtain the target detection result;

[0170] For the candidate image, an image segmentation algorithm is used to perform image segmentation to obtain an image segmentation result;

[0171] For candidate images, feature extraction algorithms are used to extract image features and obtain feature vectors. The target detection results, image segmentation results and feature vectors are fused to obtain comprehensive analysis results.

[0172] Output comprehensive analysis results as the final result of image enhancement;

[0173] If the comprehensive analysis result meets the requirements of the subsequent image analysis task, the seventh image is output as the final enhancement result, otherwise it returns to the first step to re-perform image preprocessing and quality assessment until the requirements are met.

[0174] Specifically, in step S8, the seventh image is first subjected to Gaussian filtering for denoising, with a filter window size of 5x5 and a standard deviation of 1.5, and geometric correction is performed using affine transformation;

[0175] Then, the Tenengrad function based on gradient variance is used to score the image quality. If the quality score is higher than the threshold of 0.7, the image quality is considered qualified.

[0176] Then, the YOLOv3 algorithm is used to detect the target in the image, and the confidence threshold is set to 0.5; the FCN semantic segmentation algorithm is then used to segment the image into foreground and background;

[0177] At the same time, the VGG16 network is used to extract the 512-dimensional feature vector of the image;

[0178] Finally, the target detection results, segmentation masks and feature vectors are fused by weighted averaging to generate a comprehensive analysis result as the final image enhancement output;

[0179] If the comprehensive result can support the subsequent tasks of target detection and segmentation, the seventh image is directly output, otherwise it is iterated and optimized until the requirements are met.

[0180] The above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

[0181] For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of this application.

Claims

1. An image processing method for a heterogeneous imager array of an underwater robot, characterized in that: The following steps are involved: Step S1, obtaining an original underwater image collected by a heterogeneous imager array, wherein the heterogeneous imager array is used to capture image data of spectrum and resolution, and adjusting the brightness of the image data by using an illumination compensation algorithm to obtain a first image; Step S2, judging whether there is a scattering effect according to the pixel distribution characteristics in the first image in step S1, if there is a scattering effect, using a descattering algorithm to process the first image to obtain a descattered second image, if there is no scattering effect, no scattering removal is performed; Step S3, using a multi-scale detail enhancement algorithm to extract detail features in the second image in step S2 to obtain a third image; Step S4, judging whether there is color deviation according to the color distribution in the third image in step S3, if there is color deviation, performing color balancing processing on the third image using a color correction algorithm to obtain a fourth image, if there is no color deviation, no color balancing processing is performed; Step S5, according to the noise features in the fourth image of step S4, adopt an adaptive denoising algorithm to suppress the noise of the image to obtain a fifth image; Step S6, judging whether a preset contrast threshold is met according to the contrast feature in the fifth image in step S5, if so, no contrast adjustment is performed, if not, a contrast enhancement algorithm is used to adjust the contrast of the fifth image to obtain a sixth image; Step S7, processing the edge of the image using an edge enhancement algorithm based on the edge features in the sixth image in step S6 to obtain a seventh image; Step S8, based on the seventh image in step S7, determine whether the requirements of the image analysis task are met, the requirements include target detection, image segmentation and feature extraction. If the quality score is greater than or equal to a preset threshold, it is determined that the quality of the preprocessed image meets the requirements, otherwise it is determined that the quality of the preprocessed image does not meet the requirements.

2. The image processing method of the underwater robot heterogeneous imager array according to claim 1, characterized in that: The step S1 includes: Acquisition and initial processing 100 raw underwater images of different spectra and resolutions were acquired to form an initial image set; Brightness histogram analysis extracts the brightness histogram of 256 gray levels for each image and calculates the average brightness value; Calculate the average brightness value of all images as 130 and set it as the target brightness value; Illumination compensation adjusts the pixel brightness value of each image through a piecewise linear transformation method to match the target brightness value; Image stitching stitches the adjusted images into a brightness-balanced image of 1000*1000 pixels; The noise reduction process uses a 5*5 median filter algorithm to smooth the spliced ​​images; Edge detection uses Canny operator and Sobel operator to detect image edges, and adjusts the grayscale value of edge pixels through nonlinear transformation function; The quality assessment uses the Tenengrad function to evaluate the clarity of the image after edge enhancement, which is used to determine that the clarity index is greater than 0.

85.

3. The image processing method of the underwater robot heterogeneous imager array according to claim 1, characterized in that: The step S2 includes: analyzing the pixel distribution characteristics of the first image, and determining whether the image is affected by scattering by calculating the grayscale histogram and the contrast index, including the following steps: Contrast judgment: if the image contrast is lower than the threshold value of 0.6, the image is judged to be scattered; Descattering processing: The scattering image is input into the descattering model based on the convolutional neural network. The model learns the mapping relationship between the scattering features and the non-scattering image to minimize the reconstruction error and obtain the second image after descattering. Use the Sobel operator to perform edge detection on the second image, set a threshold of 0.2 to identify edge points, perform a nonlinear transformation y=2x^2 on the edge point pixels, and obtain a third image with edge enhancement; The clarity index of the third image is calculated using the Brenner gradient function. If the clarity index is greater than 0.8, it is considered that the image quality meets the requirements; Otherwise, return to the edge enhancement step to continue processing.

4. The image processing method of the underwater robot heterogeneous imager array according to claim 1, characterized in that: The step S3 includes: performing multi-scale wavelet transform and detail enhancement processing on the second image to be processed, including the following steps: Wavelet decomposition uses Haar wavelet to perform a three-level decomposition of the second image to obtain a multi-scale representation of the LL3, LH3, HL3, and HH3 subbands; Apply the power law function y=c*x^γ(c=1.5,γ=0.8) to the high frequency subbands of LH3, HL3, and HH3 for nonlinear mapping; Low-frequency contrast calculates the cumulative probability density function of the LL3 low-frequency subband and performs equalization processing; Inverse wavelet transform performs a three-level inverse wavelet transform on the enhanced high and low frequency sub-bands to reconstruct a third image with enhanced details; Convolve the third image with a 5x5 Laplacian mask to extract edge contour information, and then close the edge image twice with a 3x3 structure element; The edge image and the third image are added and fused with a weight of 3:7, and the local standard deviation in the 8x8 neighborhood window is calculated as the local contrast adjustment coefficient; Physical character meaning: y=c*x^γ:Power law function, used for nonlinear mapping to enhance high-frequency details, where: y: output value; c: constant, used to adjust the degree of enhancement; x: input value, original pixel value; γ: Exponent used to control the degree of nonlinear enhancement.

5. The image processing method of the underwater robot heterogeneous imager array according to claim 1, characterized in that: The step S4 includes: detecting, correcting and optimizing the color of the image, and outputting the corrected image, including the following steps: Color distribution feature extraction obtains the color distribution feature vector by analyzing the pixel value distribution of the RGB channel of the image; Color deviation judgment compares the feature vector with the preset color balance standard model and calculates the Euclidean distance. If it exceeds the preset threshold, it is judged that there is color deviation; Color correction uses the gray world algorithm to perform white balance correction by adjusting the gain coefficient of the RGB channels; Texture feature analysis extracts Gabor texture features and compares the changes before and after correction. If the changes exceed the preset threshold, the correction intensity is reduced and reprocessed. Color evaluation uses the ResNet color evaluation model to score the correction results.

6. The image processing method of the underwater robot heterogeneous imager array according to claim 1, characterized in that: The step S5 includes: noise analysis by analyzing the grayscale histogram and local variance of the image, identifying the image, and estimating the noise standard deviation as 15; Select the adaptive non-local mean filtering algorithm and set the filter size to 7x7 and the filter strength parameter to 0.8; Sub-block processing divides the image into 64x64 sub-blocks, performs non-local mean filtering on each sub-block independently, and concatenates the filtering results into a denoised image; Calculate the signal-to-noise ratio (SNR) and peak signal-to-noise ratio (PSNR) of the denoised image and compare them with the preset thresholds of 30 dB and 36 dB; If the SNR and PSNR of the denoised image meet the preset threshold requirements, the image is output; If not satisfied, adjust the filter parameters.

7. The image processing method of the underwater robot heterogeneous imager array according to claim 1, characterized in that: The step S6 includes: calculating the average gray value and standard deviation of the fifth image and comparing them with a preset threshold value of 0.5 to determine the contrast; Analyze the grayscale histogram and determine that the target interval for contrast enhancement is grayscale value 50 to 150; Apply the histogram equalization algorithm to the pixels in the target interval for nonlinear stretching; Perform piecewise linear transformation on pixels with grayscale values ​​less than 50 and greater than 150; The processed pixels are recombined to obtain a processed sixth image, and the ratio characteristic value is 0.58, which meets the preset threshold; The Sobel operator is used to perform edge detection on the sixth image, edge pixels are extracted, and the grayscale values ​​of the edge pixels are adjusted using a nonlinear transformation function f(x)=2x^0.

8.

8. The image processing method of the underwater robot heterogeneous imager array according to claim 1, characterized in that: The step S7 includes: edge feature extraction using a Canny operator to detect the edge of the sixth image, and obtaining the coordinates and gradient values ​​of edge pixels as edge feature information; Edge point judgment uses the Sobel operator to perform convolution operation on the image, calculate the gradient value, and determine the pixel points whose gradient value exceeds the preset threshold of 0.6 as edge points; For pixels determined as edge points, the nonlinear transformation function Gamma transformation is used to adjust the gray value, and the transformation coefficient is set to 1.5; The clarity index gradient variance 5.2 and edge information entropy 2.8 of the seventh image were calculated to evaluate the image quality.

9. The image processing method of the underwater robot heterogeneous imager array according to claim 1, characterized in that: The step S8 includes: applying a Gaussian filter with a window size of 5x5 and a standard deviation of 1.5 to the seventh image for denoising, and using an affine transformation for geometric correction; The Tenengrad function based on gradient variance is used to score the quality of the denoised image. If the score is higher than 0.7, the image quality is considered to be qualified. Apply the YOLOv3 algorithm to detect targets in the image and set the confidence threshold to 0.5; Use the FCN algorithm to segment the image into foreground and background; The VGG16 network is used to extract the 512-dimensional feature vector of the image.

Images