Underwater image quality intelligent evaluation method

Abstract

The invention discloses an underwater image quality intelligent evaluation method, relates to the technical field of computers, solves the technical problem that underwater image quality evaluation is not comprehensive enough, and is characterized in that firstly, an attribute feature map strongly related to five dimensions including brightness, color cast, definition, noisy points and naturalness is extracted; then carrying out statistical analysis on the attribute feature maps, and establishing reliable feature representation for each attribute; and finally, fusing and regressing the features through a constructed attribute feature fusion model and a proposed combined loss function to obtain a final quality score, thereby establishing an objective non-reference evaluation index. Compared with an existing method, the method has the advantages that image attributes considered by the method are more comprehensive and do not depend on any additional reference, the unprocessed underwater image and the enhanced underwater image can be effectively represented at the same time, and better effectiveness and robustness are achieved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to an intelligent underwater image quality evaluation method. Background Technique

[0002] Underwater visual perception tasks rely on underwater images as basic data. However, due to the light absorption and scattering in water bodies, the obtained underwater images often present problems such as insufficient brightness, color cast, blurriness, low contrast, etc. Low-quality images will seriously affect the implementation of subsequent tasks. Reliable underwater image quality evaluation can not only provide an objective evaluation standard for underwater image enhancement algorithms, but also guide the optimization direction of the algorithms, promoting the overall progress of underwater visual perception technology. Therefore, there is an urgent need to establish a practical and reliable underwater image quality evaluation method.

[0003] Existing underwater image quality evaluation methods are mainly divided into full-reference evaluation metrics and no-reference evaluation metrics according to whether they need to refer to the original distortion-free image. Among the full-reference evaluation metric methods, peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are the most commonly used metrics. However, these evaluation methods are often limited by the need to rely on the provided high-quality reference images. Existing no-reference underwater image evaluation metrics often quantify multiple attributes of the image and obtain the score of the image through weighted learning, such as methods like UCIQE, UIQM, CCF, and UIQI. However, previous image evaluation methods based on feature statistics often only study unprocessed underwater images and ignore the degradation characteristics of enhanced underwater images, and the feature representation ability of previous methods is limited and the generalization performance is not good. In view of this, by statistically analyzing the attribute characteristics of unprocessed underwater images and enhanced underwater images, this paper proposes a new underwater image quality evaluation metric to solve the above technical problems. Summary of the Invention

[0004] This application provides an intelligent underwater image quality evaluation method, and its technical purpose is to construct a reasonable and comprehensive image quality evaluation method to ensure effective representation of all types of underwater images.

[0005] The above technical purpose of this application is achieved through the following technical solutions:

[0006] An intelligent underwater image quality evaluation method, including:

[0007] Obtain an underwater image and preprocess the underwater image to a fixed size;

[0008] Output the preprocessed underwater image to an attribute feature extraction module to obtain an attribute feature map; wherein, the attribute feature map includes a brightness attribute feature map, a color cast attribute feature map, a sharpness attribute feature map, a noise attribute feature map, and a naturalness attribute feature map;

[0009] Measure the attribute feature map to obtain the attribute features of the underwater image; wherein, the attribute features include brightness feature, color deviation feature, clarity feature, noise feature and naturalness feature;

[0010] Train the attribute feature fusion model to obtain the optimal weight parameters;

[0011] Input the attribute features into the trained attribute feature fusion model to obtain the final underwater image quality score.

[0012] Further, the attribute feature extraction module includes a background light extraction module, a contrast extraction module, an edge information extraction module, a high-frequency signal extraction module, a grayscale conversion module and an MSCN coefficient extraction module;

[0013] Output the preprocessed underwater image to the attribute feature extraction module to obtain the attribute feature map, including:

[0014] Extract the brightness attribute feature map of the underwater image through the background light extraction module;

[0015] Extract the color deviation attribute feature map of the underwater image through the background light extraction module and the contrast extraction module;

[0016] Extract the clarity attribute feature map of the underwater image through the edge information extraction module and the high-frequency signal extraction module;

[0017] Extract the noise attribute feature map of the underwater image through the grayscale conversion module;

[0018] Extract the naturalness attribute feature map through the MSCN coefficient extraction module.

[0019] The beneficial effects of this application are as follows: The intelligent underwater image quality evaluation method described in this application proposes a comprehensive no-reference underwater image quality evaluation, which simultaneously considers the degradation characteristics of unprocessed images and enhanced images, extracts features from five aspects of brightness, color cast, clarity, noise and naturalness, and conducts a comprehensive evaluation. First, extract the attribute feature maps strongly related to the five dimensions of brightness, color cast, clarity, noise and naturalness; then conduct statistical analysis on these attribute feature maps to establish a reliable feature representation for each attribute; finally, fuse and regress these features through the constructed attribute feature fusion model and the proposed combined loss function to obtain the final quality score, thereby establishing an objective quantitative evaluation index. Compared with existing methods, the image attributes considered are more comprehensive and do not rely on any additional provided references, can effectively characterize all types of underwater images, and have better effectiveness and robustness. Description of the Drawings

[0020] Figure 1 This is a flow chart of the underwater image quality intelligent assessment method in an embodiment of the present application;

[0021] Figure 2 This is an example diagram of the intermediate attribute feature map for underwater image brightness representation in the embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solution of this application will be described in detail below with reference to the accompanying drawings.

[0023] like Figure 1 As shown, the underwater image quality intelligent assessment method described in this application includes:

[0024] 100: Acquire an underwater image and preprocess the underwater image into a fixed size;

[0025] 101: Output the preprocessed underwater image to an attribute feature extraction module to obtain an attribute feature map; wherein the attribute feature map includes a brightness attribute feature map, a color cast attribute feature map, a clarity attribute feature map, a noise attribute feature map, and a naturalness attribute feature map.

[0026] Preferably, the attribute feature extraction module includes a background light extraction module, a contrast extraction module, an edge information extraction module, a high-frequency signal extraction module, a grayscale conversion module and an MSCN coefficient extraction module.

[0027] Preferably, the pre-processed underwater image is output to an attribute feature extraction module to obtain an attribute feature map, including:

[0028] 1011: extracting a brightness attribute feature map of the underwater image through a background light extraction module;

[0029] 1012: extracting a color cast attribute feature map of the underwater image using a background light extraction module and a contrast extraction module;

[0030] 1013: extracting a clarity attribute feature map of the underwater image through an edge information extraction module and a high-frequency signal extraction module;

[0031] 1014: extracting the noise attribute feature map of the underwater image through a grayscale conversion module;

[0032] 1015: Extracting the naturalness attribute feature map through the MSCN coefficient extraction module.

[0033] Preferably, the brightness attribute feature map of the underwater image is extracted by a background light extraction module, including:

[0034] 10111: Input the pre-processed underwater image x into the background light extraction module to obtain the background light map X of the underwater image lum。

[0035] Among them, the process of obtaining the background light image X lum includes:

[0036] (1) Perform multi-scale Gaussian low-pass filtering on the preprocessed underwater image x to obtain

[0037] In the embodiments of the present application, according to experience, σ is respectively set to 30, 60, and 90, and an appropriate convolution kernel is selected through σ adaptive control.

[0038] (2) Considering that the Gaussian filter may still contain object details, perform logarithmic domain conversion and scaling on X blur to obtain the background light image X lum of the underwater image x, which is expressed as: X lum = Normalization(logX blur ).

[0039] 10112: Convert the background light image X lum to a grayscale image to obtain the global brightness attribute feature map X Global ;

[0040] 10113: Cut the underwater image x into m×n blocks and input them into the background light extraction module respectively to obtain m×n local background light images corresponding to the m×n blocks. Convert each local background light image into a local grayscale image to obtain a brightness uniformity attribute map X Uniformity composed of m×n local grayscale images;

[0041] 10114: The global brightness attribute feature map X Global and the brightness uniformity attribute map X Uniformity serve as the brightness attribute feature map of the underwater image, as Figure 2 shown.

[0042] In the present application, the color cast attribute feature map of the underwater image is extracted through the background light extraction module and the contrast extraction module. The background light image X lum is the color cast attribute feature map.

[0043] Preferably, the sharpness attribute feature map of the underwater image is extracted through the edge information extraction module and the high-frequency signal extraction module, including:

[0044] 10131: The edge information extraction module extracts the edge feature map X EDGE of the underwater image x through the sobel operator, which is expressed as: Among them, G x (x) and G y(x) represents the gradients of each pixel point on the underwater image x in the x-axis direction and y-axis direction respectively;

[0045] 10132: The high-frequency signal extraction module performs wavelet transform on the underwater image x, and then extracts the high-frequency information of the wavelet-transformed underwater image x in the vertical direction, horizontal direction, and diagonal direction through a high-pass filter, and obtains the high-frequency feature map X by adding pixels one by one DWT , which is expressed as: Among them, HL(·), LH(·), and HH(·) respectively represent the high-pass filters for extracting the high-frequency information of the image in the horizontal direction, vertical direction, and diagonal direction;

[0046] 10133: Characterize the clarity attribute feature map through the edge feature map X EDGE and the high-frequency feature map X DWT Characterize the clarity attribute feature map.

[0047] Preferably, extract the noise attribute feature map of the underwater image through the gray conversion module, including: converting the underwater image x into a grayscale image through the gray conversion module, and characterizing the noise attribute feature map through the grayscale image. The conversion formula is: X Gray =0.299×R + 0.587×G + 0.114×B.

[0048] Preferably, extract the naturalness attribute feature map through the MSCN coefficient extraction module, including: subtracting the ratio of the mean value to the standard deviation from the underwater image x as the naturalness attribute feature map, which is expressed as: Among them, x(i,j) represents the pixel block centered on (i,j), μ(i,j) represents the pixel mean of the pixel block, and σ(i,j) represents the pixel standard deviation of the pixel block.

[0049] 102: Measure the attribute feature map to obtain the attribute features of the underwater image; among them, the attribute features include brightness feature, color deviation feature, clarity feature, noise feature, and naturalness feature.

[0050] Preferably, measuring the brightness attribute feature map includes:

[0051] 10211: Calculate the average pixel value of the global brightness attribute feature map X Global , which is expressed as: X Global represents the global brightness map; F Luminance1 represents the global brightness characterization feature; N represents the number of pixels of the image;

[0052] 10212: The average brightness of the m×n image blocks included in the brightness uniformity attribute map X Uniformity Perform calculations; where j = 1, 2, …, m×n;

[0053] 10213: Calculate the standard deviation of each image block according to the average brightness, expressed as: where F represents the luminance uniformity characterization feature; Luminance2

[0054] 10214: Characterize the brightness feature of the underwater image through the global brightness characterization feature F Luminance1 and the luminance uniformity characterization feature F Luminance2 .

[0055] Preferably, measure the color cast attribute feature map, including:

[0056] 10221: Calculate the average RGB value of the background light map X lum , expressed as: [R, G, B] mean = mean(X lum );

[0057] 10222: Sort the three values in [R, G, B] mean , expressed as: a max , a mid , a min = rank([R, G, B] mean );

[0058] 10223: Calculate the background light color cast feature, expressed as: F Color_Cast1 = (a max + a mid ) / 2 - a min ;

[0059] 10224: Calculate the standard deviation of the foreground contrast map to obtain the foreground color cast feature, expressed as: F Color_Cast2 = Std(Max(x) - Max(X lum )); where Max(·) represents the maximum value of the channels extracted from the pixels of the input image; Std(·) represents the standard deviation of the image;

[0060] 10225: Characterize the color cast feature of the underwater image through the background light color cast feature F Color_Cast1 and the foreground color cast feature F Color_Cast2 .

[0061] Preferably, measure the sharpness attribute feature map, including:

[0062] 10231: Calculate the sum of pixel brightness ∑(X EDGE ) obtained by edge region detection;​

[0063] 10232: Calculate the sum of pixel brightnesses ∑(X DWT ) detected in the high-frequency region;

[0064] 10233: Calculate the sharpness feature, expressed as:

[0065] Preferably, measure the noise attribute feature map, and perform the following steps on the underwater image x and its grayscale image X Gray :

[0066] 10241: Divide the input image into image patches of size d to obtain a set where s = (M - d + 1)(N - d + 1); x t represents the t-th image patch; M represents the number of pixels in the vertical direction of the input image X; N represents the number of pixels in the horizontal direction of the input image X; C represents the number of channels of the input image X; d represents the size of the image patch; X represents the input image, i.e., the underwater image x or its grayscale image X Gray ;

[0067] 10242: Calculate the covariance matrix ∑ of the image patches, expressed as:

[0068] 10243: Perform eigenvalue decomposition on ∑ to obtain eigenvalues where r = d 2 ;

[0069] 10244: Sort the eigenvalues in descending order, expressed as: λ = {λ1 ≥ λ2 ≥ … ≥ λ r}, and determine the median of the set . The square root of this median is used as the noise level of the input image X; m represents the dimension of the low-dimensional subspace where the noise-free image patches are located, and m << r. Here, m is analyzed from the eigenvalues λ; λ1 ≥ λ2 ≥... ≥ λ r After sorting, check these eigenvalues from largest to smallest, and sequentially remove the largest ones until the average and median of the remaining eigenvalue set are almost equal. The number of removed eigenvalues is the estimated m;

[0070] 10245: Perform noise estimation on the underwater image x and its grayscale image X Gray respectively to obtain N RGB and N gray . Weight N gray and N RGB as the noise feature F Noise of a single image. Among them, N RGB represents the noise estimation of the underwater image x; NRGB The grayscale image X representing the underwater image x Gray for noise estimation.

[0071] Preferably, the measurement of the naturalness attribute feature map includes:

[0072] 10251: Calculate the average distribution of the MSCN coefficients of the reference image after enhancing the underwater image enhancement dataset UIEB, and use this average distribution as the ideal distribution P ideal ; where the underwater image enhancement dataset UIEB is an existing publicly available dataset;

[0073] 10252: Calculate the similarity F image between the MSCN coefficient distribution P ideal of the underwater image x and the ideal distribution P Naturalness1 , which is expressed as: F Naturalness1 = 1 - KL(P image ||P ideal ); where KL(·) represents the Kullback-Leibler divergence similarity between P image and P ideal ;

[0074] 10253: Calculate the kurtosis feature F Naturalness2 of the MSCN coefficient distribution curve of the underwater image x, which is expressed as: where kurtosis(·) represents the kurtosis of the distribution curve;

[0075] 10254: Calculate the skewness feature F Naturalness3 of the MSCN coefficient distribution curve of the underwater image x, which is expressed as: where skew(·) represents the skewness of the distribution curve;

[0076] 10255: Characterize the naturalness feature through F Naturalness1 , F Naturalness2 and F Naturalness3 .

[0077] 103: Train the attribute feature fusion model to obtain the optimal weight parameters.

[0078] Preferably, train the attribute feature fusion model through the image quality evaluation dataset UID2021 and the combined loss function to obtain the optimal weight parameters.

[0079] where the combined loss function is expressed as: Loss = a*L2 + b*L RANK , L RANK represents the grouped ranking loss, L2 represents the score loss,

[0080] When using the UID2021 dataset for training, each original image and its corresponding underwater image enhancement result UIE are taken as a group, and the given training set is grouped into {(X 11 , Y 11 ), (X 12 , Y 12 ), ··· (X k,q , Y k,q )}, where Xi,j represents the extracted attribute feature vector group, and Y i,j represents the corresponding MOS value; k represents the number of all unprocessed natural underwater images, that is, the number of groups of the dataset; q represents the number of UIE methods; rank(·) represents sorting the input vector to obtain the ranking corresponding to each vector value; i = 1, 2, ··· k, j = 1, 2, ··· q.

[0081] 104: Input the attribute features into the trained attribute feature fusion model to obtain the final underwater image quality score.

[0082] The above is an exemplary embodiment of the present application, and the protection scope of the present application is defined by the claims and their equivalents.

Claims

1. An intelligent underwater image quality assessment method, characterized in that Including: Obtain an underwater image and preprocess the underwater image to a fixed size; Output the preprocessed underwater image to an attribute feature extraction module to obtain an attribute feature map; wherein, the attribute feature map includes a brightness attribute feature map, a color cast attribute feature map, a sharpness attribute feature map, a noise attribute feature map, and a naturalness attribute feature map; Measure the attribute feature map to obtain the attribute features of the underwater image; wherein, the attribute features include brightness features, color cast features, sharpness features, noise features, and naturalness features; Train an attribute feature fusion model to obtain the optimal weight parameters; Input the attribute features into the trained attribute feature fusion model to obtain the final underwater image quality score.

2. The intelligent underwater image quality evaluation method according to claim 1, characterized in that The attribute feature extraction module includes a background light extraction module, a contrast extraction module, an edge information extraction module, a high-frequency signal extraction module, a grayscale conversion module, and an MSCN coefficient extraction module; Output the preprocessed underwater image to the attribute feature extraction module to obtain an attribute feature map, including: Extract the brightness attribute feature map of the underwater image through the background light extraction module; Extract the color cast attribute feature map of the underwater image through the background light extraction module and the contrast extraction module; Extract the sharpness attribute feature map of the underwater image through the edge information extraction module and the high-frequency signal extraction module; Extract the noise attribute feature map of the underwater image through the grayscale conversion module; Extract the naturalness attribute feature map through the MSCN coefficient extraction module.

3. The underwater image quality intelligent assessment method according to claim 2, wherein: The extraction of the brightness attribute feature map of the underwater image through the background light extraction module includes: Input the preprocessed underwater image x into the background light extraction module to obtain the background light map X of the underwater image lum ; Convert the background light image X lum to a grayscale image to obtain the global brightness attribute feature map X Global ; The underwater image x is sliced into m×n blocks and input into the background light extraction module respectively, obtaining m×n local background light maps corresponding to the m×n blocks. Each local background light map is respectively converted into a local grayscale map, obtaining a brightness uniformity attribute map X composed of m×n local grayscale maps Uniformity ; Global brightness attribute feature map X Global and brightness uniformity attribute map X Uniformity serve as the brightness attribute feature maps of underwater images.

4. The intelligent underwater image quality evaluation method according to claim 2, characterized in that The extraction of the naturalness attribute feature map through the MSCN coefficient extraction module includes: The ratio of subtracting the mean from the underwater image x to the standard deviation is used as the naturalness attribute feature map, expressed as: where x(i,j) represents the pixel block centered at (i,j), μ(i,j) represents the pixel mean of the pixel block, and σ(i,j) represents the pixel standard deviation of the pixel block.

5. The underwater image quality intelligent assessment method according to claim 3, wherein: Measuring the brightness attribute feature map includes: Calculate the average pixel value of the global brightness attribute feature map X Global , which is expressed as: X Global represents the global brightness map; F Luminance1 represents the global brightness characterization feature; N represents the number of pixels in the image; Brightness uniformity attribute graph X Uniformity The average brightness value of the m×n image blocks contained in Perform calculations; where j = 1, 2, ..., m × n; According to the average brightness The standard deviation of each image block is calculated as: Among them, F Luminance2 Indicates the brightness uniformity characterization feature; Characterize feature F by global brightness Luminance1 and characterize feature F by brightness uniformity Luminance2 Characterize the brightness feature.

6. The intelligent underwater image quality evaluation method according to claim 3, characterized in that Measuring the color cast attribute feature map includes: Background light image X lum The average RGB value is calculated and expressed as: [R,G,B] mean =mean(X lum ); Sort the three values in [R, G, B], denoted as: a mean , a max , a mid , a min = rank([R, G, B] mean ); Calculate the background light color deviation feature, expressed as: F Color_Cast1 =(a max +a mid ) / 2 - a min ; Calculate the standard deviation of the foreground contrast map to obtain the foreground color deviation feature, denoted as: F Color_Cast2 = Std(Max(x) - Max(X lum )); where, Max(·) represents the maximum value of the channels extracted from the pixel points of the input image; Std(·) represents the standard deviation of the image; Characterize the color cast feature of the underwater image through the background color cast feature F Color_Cast1 and the foreground color cast feature F Color_Cast2 to characterize the color cast feature of the underwater image.

7. The underwater image quality intelligent assessment method according to claim 2, wherein: Measuring the sharpness attribute feature map includes: Calculate the sum of pixel brightnesses ∑(X EDGE ) obtained from edge region detection; Calculate the sum of pixel brightness ∑(X DWT ) obtained from the detection of the high-frequency region; Calculate the clarity feature, expressed as:

8. The intelligent underwater image quality evaluation method according to claim 2, wherein Measure the noise attribute feature map, including the underwater image x and its grayscale image X Gray Perform the following steps: Divide the input image into image patches of size d to obtain a set where s = (M - d + 1)(N - d + 1); x t represents the t-th image patch; M represents the number of pixels in the vertical direction of the input image X; N represents the number of pixels in the horizontal direction of the input image X; C represents the number of channels of the input image X; d represents the size of the image patch; X represents the input image, i.e., the underwater image x or its grayscale image X Gray ; The covariance matrix ∑ of the image block is calculated and expressed as: Perform eigenvalue decomposition on ∑ to obtain the eigenvalues where r = d 2 ; Arrange the eigenvalues in descending order, denoted as: λ = {λ1 ≥ λ2 ≥ … ≥ λ r}, and determine the median of the set . The square root of this median is used as the noise level of the input image X; m represents the dimension of the low-dimensional subspace where the noise-free image patches are located, and m << r; Estimate the noise of the underwater image x and its grayscale image X respectively Gray to obtain N RGB and N gray . Weight N gray and N RGB as the noise feature F of a single image Noise .

9. The intelligent underwater image quality evaluation method according to claim 2, characterized in that Measuring the naturalness attribute feature map includes: Calculate the average distribution of the MSCN coefficients of the reference images after enhancing the underwater image enhancement dataset UIEB, and use this average distribution as the ideal distribution P ideal ; The MSCN coefficient distribution P of the underwater image x image and the ideal distribution P ideal similarity F Naturalness1 is calculated and expressed as: F Naturalness1 = 1 - KL(P image ||P ideal ); where KL(·) represents the Kullback-Leibler divergence similarity between P image and P ideal . The kurtosis feature F of the MSCN coefficient distribution curve of the underwater image x Naturalness2 is calculated and expressed as: where kurtosis(·) represents the kurtosis of the distribution curve; The skewness feature F of the MSCN coefficient distribution curve of the underwater image x Naturalness3 is calculated and expressed as: where skew(·) represents the skewness of the distribution curve; Characterize the naturalness feature through F Naturalness1 , F Naturalness2 and F Naturalness3 .

10. The intelligent underwater image quality evaluation method according to claim 1, characterized in that, The training of the attribute feature fusion model to obtain the optimal weight parameters includes: Train the attribute feature fusion model through an image quality evaluation dataset UID2021 and a combined loss function to obtain the optimal weight parameters; Among them, the combined loss function is expressed as: Loss = a * L2 + b * L RANK , L RANK represents the pairwise ranking loss, L2 represents the scoring loss, X i,j represents the extracted attribute feature vector group, and Y i,j represents the corresponding MOS value; k represents the number of all unprocessed natural underwater images, that is, the number of groups in the dataset; q represents the number of UIE methods; rank(·) represents sorting the input vector to obtain the ranking corresponding to each vector value; i = 1, 2, ··· k, j = 1, 2, ··· q.

Images