Quality evaluation method, device, equipment and medium for raindrop removal image

Abstract

The present invention discloses an image quality evaluation method, device, equipment and medium for raindrop removal images. The method comprises: establishing a raindrop image data set, wherein the data set is divided into a training set, a validation set and a test set, wherein an initial classification model can be trained according to the training set and the validation set to obtain a classification model for judging the distortion level of the image to be evaluated. According to the PSNR value, SSIM value and subjective evaluation score of each group of data in the training set and the validation set, a target evaluation model for evaluating the image quality at different distortion levels can be obtained. The test set can be used to test the target evaluation model to obtain a test result. According to the subjective evaluation score of each group of data in the test set, it can be judged whether the test result meets the standard. When the test result meets the standard, the image quality of the raindrop removal image can be evaluated according to the classification model and the target evaluation model. The above method can reduce the deviation between the objective calculation result and the subjective perception quality.

Description

Technical Field

[0001] The present invention relates to the technical field of image evaluation, and in particular to a quality evaluation method, device, equipment and medium for raindrop removal images. Background Art

[0002] With the advancement of science and technology, the technology of autonomous driving is booming. Autonomous driving software often needs to judge the road conditions based on the video images taken by the camera equipped with the vehicle. However, when the weather is bad, the video images taken by the camera equipped with the vehicle are very easily affected by raindrops, which will reduce the accuracy of the back-end vehicle recognition and road condition recognition, thereby affecting the decision-making of autonomous driving.

[0003] At present, there are many rain removal solutions for rainy image restoration, and the rain removal effect is constantly improving. With the rapid development of machine learning, there are more and more image rain removal algorithms based on deep learning. However, the image quality obtained by using different raindrop removal algorithms varies. In related technologies, the quality of rain removal images is often evaluated by PSNR (Peak Signal to Noise Ratio) and SSIM (Structural Similarity) indicators.

[0004] However, there are many situations in which the above evaluation method is inconsistent with subjective quality judgment, and the reliability of the evaluation results is poor. Summary of the invention

[0005] In view of the above problems, the present invention is proposed to provide an image quality evaluation method, device, equipment and medium for raindrop removal images that overcome the above problems or at least partially solve the above problems, which can evaluate the quality of raindrop removal images and improve the reliability of the evaluation results.

[0006] In a first aspect, a method for evaluating the quality of a raindrop removed image is provided, the method comprising:

[0007] Establishing a raindrop image data set, wherein the data set includes multiple groups of data, each group of data includes a data pair consisting of distorted data and true value data;

[0008] Obtaining a subjective evaluation score for each group of data in the data set;

[0009] Dividing the data set into a training set, a validation set, and a test set;

[0010] Inputting the training set and the validation set into an initial classification model for training, optimizing the parameters in the initial classification model to obtain a target classification model, wherein the target classification model is used to determine the distortion level of the image to be evaluated;

[0011] Calculating the peak signal-to-noise ratio and structural similarity of each group of data in the training set and the validation set;

[0012] Constructing a target evaluation model according to the peak signal-to-noise ratio, structural similarity and subjective evaluation score of each group of data in the training set and the validation set, wherein the target evaluation model is used to evaluate the image quality at different distortion levels;

[0013] Inputting multiple groups of data in the test set into the target evaluation model for testing to obtain test results;

[0014] Judging whether the test result meets the standard according to the subjective evaluation score of each group of data in the test set;

[0015] When the test result meets the standard, the image to be evaluated output by the raindrop removal algorithm and the corresponding raindrop-free true image are input into the target classification model and the target evaluation model in groups. The target classification model determines the distortion level of the image to be evaluated, and the target evaluation model outputs the corresponding image quality evaluation result according to the distortion level of the image to be evaluated.

[0016] Optionally, obtaining a subjective evaluation score for each group of data in the raindrop image dataset includes:

[0017] Obtaining scoring results of multiple observers on each set of data in the data set;

[0018] The scoring results of each group of data are normalized to obtain a subjective evaluation score for each group of data.

[0019] Optionally, dividing the data set into a training set, a validation set, and a test set comprises:

[0020] The multiple groups of data in the data set are randomly divided into training set, validation set and test set in a ratio of 3:1:1.

[0021] Optionally, according to the subjective evaluation scores, multiple groups of data in the data set are divided into multiple levels, and a corresponding level label is added to each group of data;

[0022] The multiple levels correspond to the distortion levels one by one.

[0023] Optionally,

[0024] The method of constructing multiple target evaluation models according to the peak signal-to-noise ratio, structural similarity and subjective evaluation score of each group of data in the training set and the validation set includes:

[0025] According to the level labels corresponding to the multiple groups of data in the training set and the validation set, the multiple groups of data in the training set and the validation set are divided into multiple subsets, and the level labels corresponding to the groups of data in each subset are the same;

[0026] For each subset, a binary nonlinear regression analysis is performed according to the peak signal-to-noise ratio, structural similarity, and subjective evaluation score of each group of data in the subset to obtain a nonlinear regression equation corresponding to the subset;

[0027] Obtaining multiple nonlinear regression equations corresponding to the multiple subsets to obtain the target evaluation model;

[0028] The number of the multiple nonlinear regression equations in the target evaluation model corresponds one-to-one to the number of the distortion levels, and each of the nonlinear regression equations is used to evaluate the image quality of the corresponding distortion level.

[0029] Optionally, the inputting the multiple groups of data in the test set into the target evaluation model for testing to obtain the test results includes:

[0030] The multiple groups of data in the test set are input into the target evaluation model. The target evaluation model calls the corresponding nonlinear regression equation according to the grade label corresponding to each group of data in the test set to calculate the test result of each group of data.

[0031] Optionally, determining whether the test result meets the standard according to the subjective evaluation score of each group of data in the test set includes:

[0032] When the difference between the test result of each group of data in the test set and the subjective evaluation score corresponding to each group of data is within a set range, it is determined that the test result meets the standard.

[0033] In a second aspect, a quality evaluation device for a raindrop removal image is provided, the device comprising:

[0034] A data set establishment module, used to establish a raindrop image data set, wherein the data set includes multiple groups of data, each group of data includes a data pair consisting of distorted data and true value data;

[0035] A scoring acquisition module, used to obtain a subjective evaluation score for each group of data in the data set;

[0036] A partitioning module, used for partitioning the data set into a training set, a validation set and a test set;

[0037] A classification model determination module inputs the training set and the verification set into an initial classification model for training, optimizes the parameters in the initial classification model, and obtains a target classification model, wherein the target classification model is used to determine the distortion level of the image to be evaluated;

[0038] A calculation module, used for calculating the peak signal-to-noise ratio and structural similarity of each group of data in the training set and the validation set;

[0039] An evaluation model determination module, used to construct a target evaluation model according to the peak signal-to-noise ratio, structural similarity and subjective evaluation score of each group of data in the training set and the validation set, wherein the target evaluation model is used to evaluate the image quality at different distortion levels;

[0040] A model testing module, inputting multiple groups of data in the test set into the target evaluation model for testing to obtain test results;

[0041] A judgment module, used to judge whether the test result meets the standard according to the subjective evaluation score of each group of data in the test set;

[0042] The evaluation module, when the test result meets the standard, inputs the image to be evaluated output by the raindrop removal algorithm and the corresponding true image without raindrops into the target classification model and the target evaluation model in groups, and the target classification model determines the distortion level of the image to be evaluated, and the target evaluation model outputs the corresponding image quality evaluation result according to the distortion level of the image to be evaluated.

[0043] In a third aspect, an electronic device is provided, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method described in the first aspect by executing the computer instructions.

[0044] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method according to the first aspect.

[0045] The technical solution provided in the embodiments of the present invention has at least the following technical effects or advantages:

[0046] The embodiment of the present invention provides an image quality evaluation method, device, equipment and medium for raindrop removal images. By establishing a raindrop image data set, a subjective evaluation score of each group of data in the data set is obtained. Then the data set is divided into a training set, a validation set and a test set. Among them, the training set and the validation set can be used to train the initial classification model to optimize the parameters in the initial classification model to obtain a target classification model for judging the distortion level of the image to be evaluated. According to the PSNR value, SSIM value and subjective evaluation score of each group of data in the training set and the validation set, a target evaluation model for evaluating the image quality of different distortion levels can be obtained. The test set can be used to test the target evaluation model to obtain a test result. According to the subjective evaluation score of each group of data in the test set, it can be judged whether the test result meets the standard to verify the reliability of the target evaluation model. Finally, when the test result meets the standard, the image to be evaluated output by the raindrop removal algorithm and the corresponding raindrop-free true value image can be grouped and input into the target classification model and the target evaluation model, and the target classification model determines the distortion level of the image to be evaluated, and the target evaluation model outputs the corresponding image quality evaluation result according to the distortion level of the image to be evaluated. The above method can reduce the deviation between the objective calculation results and the subjective perceived quality, further provide a reference for the selection and optimization of the image restoration algorithm model, and make the evaluation results more reliable.

[0047] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.

[0049] In the attached picture:

[0050] Figure 1 is a flow chart of an image quality evaluation method of a raindrop removal algorithm provided by an embodiment of the present invention;

[0051] Figure 2 It is a structural block diagram of a quality assessment device for a raindrop-removed image provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0053] Figure 1 FIG. 1 is a flow chart of an image quality evaluation method of a raindrop removal algorithm provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0054] Step S101: Collect raindrop image data set.

[0055] The data set includes multiple groups of data, each of which contains a data pair consisting of distorted data and true data. The distorted data is the data of rainy images, and the true data is the data of clear rainless images;

[0056] In this embodiment, raindrop image datasets of different scenes and different distortion degrees may be collected.

[0057] Specifically, you can park the car and take a rain-free photo as a reference value in different weather conditions such as evening, daytime, cloudy, sunny, etc., in various scenes such as outdoor and underground parking garages. Then use a spray bottle with adjustable flow rate and use the minimum flow rate to manually spray water droplets on the rearview lens of the vehicle, and take a photo as slightly distorted data; then wipe off the raindrops on the lens, increase the flow rate by one gear and continue to spray water droplets on the lens, and take the photo at this time as more seriously distorted data; continue to wipe off the raindrops on the lens, increase the flow rate again and spray water droplets on the lens, take the photo at this time as the most seriously distorted data, and then wipe off the raindrops on the lens; drive the vehicle to different locations and continue to repeat the above collection process until the data volume reaches the predetermined requirement. Pair the distorted data with the corresponding true value data one by one to obtain multiple sets of data pairs, thereby establishing a raindrop image dataset.

[0058] Step S102: Obtain the subjective evaluation score of each group of data in the data set.

[0059] Optionally, step S102 may include:

[0060] The first step is to obtain the scoring results of multiple observers for each set of data in the data set;

[0061] In this embodiment, 15 observers can perform subjective evaluation on each set of data in the data set one by one. The full score is 100, indicating no distortion, and the lowest score is 0, indicating the most serious distortion. The scoring result of each set of data is obtained by removing the maximum score and the minimum score in each set of data and calculating the average of the remaining scores.

[0062] The image quality evaluation method is mainly divided into two aspects: subjective evaluation and objective evaluation. Subjective evaluation takes people as observers and tries to truly reflect human visual perception by subjectively evaluating images; while objective evaluation methods reflect the subjective feelings of the human eye through some mathematical models and give results based on numerical calculations. In the present invention, by combining subjective evaluation with objective evaluation, the deviation between objective calculation results and subjective perceived quality can be reduced, which further provides a reference basis for the selection and optimization of image restoration algorithm models, and the evaluation results are more reliable.

[0063] The third step is to normalize the scoring results of each group of data to obtain the subjective evaluation score of each group of data.

[0064] In this embodiment, the scoring result of each set of data may be normalized to a value of [0, 1] as the final subjective evaluation score.

[0065] Exemplarily, the [0, 1] interval score can be divided into five levels: 0, 1, 2, 3, and 4. The intervals corresponding to the five levels are [0, 0.2), [0.2, 0.4), [0.4, 0.6), [0.6, 0.8), and [0.8, 1], respectively.

[0066] Optionally, the method may further include:

[0067] According to the subjective evaluation scores, multiple groups of data in the dataset are divided into multiple levels, and corresponding level labels are added to each group of data.

[0068] For example, the multiple groups of data in the data set can be divided into 5 levels from high to low according to the subjective evaluation scores, corresponding to the 5 hierarchical intervals one by one. The labels corresponding to the 5 levels can be "0", "1", "2", "3", and "4" respectively. In other implementations of this embodiment, other forms of level labels can also be used, and the present invention is not limited to this.

[0069] Step S103: divide the data set into a training set, a validation set and a test set.

[0070] Optionally, step S103 may include:

[0071] The multiple groups of data in the dataset are randomly divided into training set, validation set and test set in a ratio of 3:1:1.

[0072] Step S104: input the training set and the validation set into the initial classification model for training, optimize the parameters in the initial classification model, and obtain the target classification model.

[0073] Among them, the target classification model can be used to determine the distortion level of the image to be evaluated.

[0074] Specifically, each pair of data in the training set and validation set containing classification labels can be sent to a five-classification deep neural network. The network consists of two input layers, a Concatenate layer, a convolution layer, and a MobileNetV2 network. The preprocessed grouped true value and distorted data are respectively sent to two input layers with shapes of [1, 3, 224, 224]. Then, after the two pass through the Concatenate layer, the output shape is obtained. Tensor data of [1, 6, 224, 224] is obtained. After passing through a convolution layer, the output shape is obtained. Tensor data of [1, 3, 224, 224] is obtained. Finally, the output data passes through the MobileNetV2 network to obtain the training loss and verification accuracy, and the model parameters are updated through continuous iteration. After the training is completed, a five-classification model, that is, the target classification model, can be obtained.

[0075] That is to say, the image distortion level output by the target classification model in this embodiment can be divided into five levels from high to low.

[0076] Step S105: Calculate the peak signal-to-noise ratio and structural similarity of each set of data in the training set and the validation set.

[0077] Among them, Peak Signal to Noise Ratio (PSNR) is a measure used to describe the degree of distortion of an image after compression relative to the original image quality. The larger the PSNR, the lower the degree of distortion of the noise image and the better the image quality. Structural Similarity Index (SSIM) is an indicator to measure the similarity between two images. In the theory of structural similarity, the signals of natural images are highly structured, that is, there is a certain correlation between pixels. The calculation of SSIM is based on the structural information in the image. Compared with PSNR, SSIM can better reflect the difference in the overall information and the structural relationship between pixels between the two images, and is more in line with people's subjective feelings.

[0078] In this embodiment, the peak signal-to-noise ratio of each group of data can be calculated according to the following formula (1):

[0079]

[0080] in,

[0081]

[0082] Above, MAX I 2is the square value of the maximum pixel value in the true image I; m and n are the width and height of the image respectively, I(i, j) and K(i, j) are the pixel values ​​corresponding to the true image and the distorted image respectively.

[0083] The structural similarity of each group of data is calculated according to the following formula (2):

[0084] SSIM(x,y)=[l(x,y)] α [c(x,y)] β [s(x,y)] γ (2)

[0085] in,

[0086]

[0087]

[0088] The above x and y represent the true value image and distorted image of the input respectively, μ x and μ y are the average values ​​of x and y, respectively, x and σ y are the standard deviations of x and y, respectively, xy is the covariance of x and y; c1, c2 and c3 are constants to avoid system errors caused by the denominator being 0; in actual engineering calculations, α=β=γ=1 and c3=c2 / 2 are generally set.

[0089] Step S106: construct a target evaluation model according to the peak signal-to-noise ratio, structural similarity and subjective evaluation score of each group of data in the training set and the validation set.

[0090] Among them, the target evaluation model is used to evaluate the image quality at different distortion levels.

[0091] Optionally, step S106 may include:

[0092] The first step is to divide the multiple groups of data in the training set and the validation set into multiple subsets according to the level labels corresponding to the multiple groups of data in the training set and the validation set, and the level labels corresponding to the groups of data in each subset are the same;

[0093] In this embodiment, the training set and the validation set may be combined into a large data set, and the large data set may be divided into five subsets according to five level labels.

[0094] Step 2: For each subset, perform binary nonlinear regression analysis based on the peak signal-to-noise ratio, structural similarity, and subjective evaluation score of each group of data in the subset to obtain a nonlinear regression equation corresponding to the subset;

[0095] Step 3: Obtain multiple nonlinear regression equations corresponding to multiple subsets to obtain the target evaluation model;

[0096] The number of the multiple nonlinear regression equations in the target evaluation model corresponds to the number of distortion levels, and each nonlinear regression equation is used to evaluate the image quality of the corresponding distortion level.

[0097] In this embodiment, the nonlinear regression equation corresponding to each subset can be obtained according to the following formula (3):

[0098] y i =a i +b i PSNR+c i SSIM+d i PSNR 2 +e i SSIM 2 +f i PSNR·SSIM (3)

[0099] Wherein, i represents the i-th subset in the plurality of subsets. For example, when the number of the plurality of subsets is 5, i=1, 2, 3, 4, 5; a i 、b i 、c i d i 、e i and f i is the coefficient of the ith equation, which can be determined by the least square method and is a constant. PSNR and SSIM are the peak signal-to-noise ratio and structural similarity corresponding to each set of data.

[0100] Step S107: input multiple groups of data in the test set into the target evaluation model for testing to obtain test results.

[0101] Optionally, step S107 may include:

[0102] Multiple groups of data in the test set are input into the target evaluation model. The target evaluation model calls the corresponding nonlinear regression equation according to the grade label corresponding to each group of data in the test set to calculate the test results of each group of data.

[0103] The test result of each set of data is the image quality evaluation score corresponding to each set of data.

[0104] Step S108: Determine whether the test results meet the standards based on the subjective evaluation scores of each group of data in the test set.

[0105] In this embodiment, when the difference between the test result of each group of data in the test set and the subjective evaluation score corresponding to each group of data is within a set range, the test result is judged to be up to standard.

[0106] By comparing the test results of PSNR, SSIM and the target evaluation model of the present invention on the test set, it is found that the image quality evaluation score calculated by the method of this paper is closer to the subjective evaluation score. When the image distortion gradually increases or decreases, there is no situation where the PSNR and SSIM scores are relatively high or low, and the scores are relatively smooth, with few jumps. Therefore, according to the improved evaluation method of the present invention, different image restoration algorithms, as well as images of the same algorithm before and after optimization in different scenes can be more reasonably and accurately compared.

[0107] Step S109, when the test result meets the standard, the image to be evaluated output by the raindrop removal algorithm and the corresponding true image without raindrops are input into the target classification model and the target evaluation model in groups, the target classification model determines the distortion level of the image to be evaluated, and the target evaluation model outputs the corresponding image quality evaluation result according to the distortion level of the image to be evaluated.

[0108] Exemplarily, when the image to be evaluated output by the raindrop removal algorithm and the corresponding true image without raindrops are input into the target classification model and the target evaluation model in groups, if the target classification model determines that the distortion level of the image to be evaluated is the second level, the nonlinear regression equation corresponding to the second subset in the target evaluation model can be called to calculate the corresponding image quality evaluation result.

[0109] Based on the same inventive concept, an embodiment of the present invention further provides a quality assessment device for a raindrop-removed image. Figure 2 is a structural block diagram of a quality evaluation device for raindrop removal images provided by an embodiment of the present invention. Figure 2 As shown, the device 200 includes a data set establishment module 210, a score acquisition module 220, a division module 230, a classification model determination module 240, a calculation module 250, an evaluation model determination module 260, a model testing module 270, a judgment module 280 and an evaluation module 290.

[0110] A data set establishment module 210 is used to establish a raindrop image data set, wherein the data set includes multiple groups of data, each group of data includes a data pair consisting of distorted data and true value data;

[0111] The scoring acquisition module 220 is used to obtain the subjective evaluation score of each group of data in the data set;

[0112] A partitioning module 230, for partitioning the data set into a training set, a validation set, and a test set;

[0113] The classification model determination module 240 inputs the training set and the validation set into the initial classification model for training, optimizes the parameters in the initial classification model, and obtains a target classification model, which is used to determine the distortion level of the image to be evaluated;

[0114] A calculation module 250 is used to calculate the peak signal-to-noise ratio and structural similarity of each set of data in the training set and the validation set;

[0115] An evaluation model determination module 260 is used to construct a target evaluation model according to the peak signal-to-noise ratio, structural similarity and subjective evaluation score of each group of data in the training set and the validation set, and the target evaluation model is used to evaluate the image quality at different distortion levels;

[0116] Model testing module 270, inputting multiple groups of data in the test set into the target evaluation model for testing to obtain test results;

[0117] A judgment module 280 is used to judge whether the test result meets the standard according to the subjective evaluation score of each group of data in the test set;

[0118] Evaluation module 290, when the test result meets the standard, the image to be evaluated output by the raindrop removal algorithm and the corresponding true image without raindrops are input into the target classification model and the target evaluation model in groups, the target classification model determines the distortion level of the image to be evaluated, and the target evaluation model outputs the corresponding image quality evaluation result according to the distortion level of the image to be evaluated.

[0119] Optionally, the score acquisition module 220 is used to:

[0120] Obtain the scoring results of multiple observers for each set of data in the data set;

[0121] The scoring results of each group of data are normalized to obtain the subjective evaluation score of each group of data.

[0122] Optionally, the division module 230 is used to:

[0123] The multiple groups of data in the dataset are randomly divided into training set, validation set and test set in a ratio of 3:1:1.

[0124] Optionally, the device may further include:

[0125] The grading module is used to divide multiple groups of data in the dataset into multiple levels according to the subjective evaluation scores, and add corresponding level labels to each group of data.

[0126] Optionally, the evaluation model determination module 260 is used to:

[0127] According to the level labels corresponding to the multiple groups of data in the training set and the validation set, the multiple groups of data in the training set and the validation set are divided into multiple subsets, and the level labels corresponding to the groups of data in each subset are the same;

[0128] For each subset, binary nonlinear regression analysis was performed based on the peak signal-to-noise ratio, structural similarity, and subjective evaluation score of each group of data in the subset to obtain the nonlinear regression equation corresponding to the subset;

[0129] Obtain multiple nonlinear regression equations corresponding to multiple subsets to obtain a target evaluation model;

[0130] The number of the multiple nonlinear regression equations in the target evaluation model corresponds to the number of distortion levels, and each nonlinear regression equation is used to evaluate the image quality of the corresponding distortion level.

[0131] Optionally, the model testing module 270 is used to:

[0132] Multiple groups of data in the test set are input into the target evaluation model. The target evaluation model calls the corresponding nonlinear regression equation according to the grade label corresponding to each group of data in the test set to calculate the test results of each group of data.

[0133] Optionally, the determination module 280 is used to:

[0134] When the difference between the test result of each group of data in the test set and the subjective evaluation score corresponding to each group of data is within the set range, the test result is judged to be up to standard.

[0135] It can be understood that the quality evaluation device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the angle detection device can be divided into different functional modules to complete all or part of the functions described above.

[0136] The embodiment of the present invention also provides an electronic device, which may include a processor and a memory, wherein the processor and the memory may be connected via a bus or other means. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various types of chips. The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the quality evaluation method in the embodiment of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the quality evaluation method in the above method embodiment is realized.

[0137] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. The one or more modules are stored in the memory, and when executed by the processor, the execution is as follows: Figure 1 The quality evaluation method in the embodiment shown in the figure. Figure 1 The corresponding related descriptions and effects in the embodiments shown are understood and will not be repeated here. Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium may be a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memory.

[0138] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0139] The embodiment of the present invention provides an image quality evaluation method, device, equipment and medium for raindrop removal images. By establishing a raindrop image data set, a subjective evaluation score of each group of data in the data set is obtained. Then the data set is divided into a training set, a validation set and a test set. Among them, the training set and the validation set can be used to train the initial classification model to optimize the parameters in the initial classification model to obtain a target classification model for judging the distortion level of the image to be evaluated. According to the PSNR value, SSIM value and subjective evaluation score of each group of data in the training set and the validation set, a target evaluation model for evaluating the image quality of different distortion levels can be obtained. The test set can be used to test the target evaluation model to obtain a test result. According to the subjective evaluation score of each group of data in the test set, it can be judged whether the test result meets the standard to verify the reliability of the target evaluation model. Finally, when the test result meets the standard, the image to be evaluated output by the raindrop removal algorithm and the corresponding raindrop-free true value image can be grouped and input into the target classification model and the target evaluation model, and the target classification model determines the distortion level of the image to be evaluated, and the target evaluation model outputs the corresponding image quality evaluation result according to the distortion level of the image to be evaluated. The above method can reduce the deviation between the objective calculation results and the subjective perceived quality, further provide a reference for the selection and optimization of the image restoration algorithm model, and make the evaluation results more reliable.

[0140] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0141] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0142] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "one" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention may be implemented by means of hardware comprising a number of different components and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

Claims

1. A quality evaluation method for a raindrop-removed image, characterized in that: The method comprises: Establishing a raindrop image data set, wherein the data set includes multiple groups of data, each group of data includes a data pair consisting of distorted data and true value data; Obtaining a subjective evaluation score for each group of data in the data set; Dividing the data set into a training set, a validation set, and a test set; Inputting the training set and the validation set into an initial classification model for training, optimizing the parameters in the initial classification model to obtain a target classification model, wherein the target classification model is used to determine the distortion level of the image to be evaluated; Calculating the peak signal-to-noise ratio and structural similarity of each group of data in the training set and the validation set; Constructing a target evaluation model according to the peak signal-to-noise ratio, structural similarity and subjective evaluation score of each group of data in the training set and the validation set, wherein the target evaluation model is used to evaluate the image quality at different distortion levels; Inputting multiple groups of data in the test set into the target evaluation model for testing to obtain test results; Judging whether the test result meets the standard according to the subjective evaluation score of each group of data in the test set; When the test result meets the standard, the image to be evaluated output by the raindrop removal algorithm and the corresponding true value image without raindrops are input into the target classification model and the target evaluation model in groups, the target classification model determines the distortion level of the image to be evaluated, and the target evaluation model outputs the corresponding image quality evaluation result according to the distortion level of the image to be evaluated; The method of constructing multiple target evaluation models according to the peak signal-to-noise ratio, structural similarity and subjective evaluation score of each group of data in the training set and the validation set includes: According to the level labels corresponding to the multiple groups of data in the training set and the validation set, the multiple groups of data in the training set and the validation set are divided into multiple subsets, and the level labels corresponding to the groups of data in each subset are the same; For each subset, a binary nonlinear regression analysis is performed according to the peak signal-to-noise ratio, structural similarity, and subjective evaluation score of each group of data in the subset to obtain a nonlinear regression equation corresponding to the subset; Obtaining multiple nonlinear regression equations corresponding to the multiple subsets to obtain the target evaluation model; The number of the multiple nonlinear regression equations in the target evaluation model corresponds one-to-one to the number of the distortion levels, and each of the nonlinear regression equations is used to evaluate the image quality of the corresponding distortion level.

2. The method according to claim 1, characterized in that: The obtaining of a subjective evaluation score for each group of data in the raindrop image dataset comprises: Obtaining scoring results of multiple observers on each set of data in the data set; The scoring results of each group of data are normalized to obtain a subjective evaluation score for each group of data.

3. The method according to claim 1, characterized in that The step of dividing the data set into a training set, a validation set, and a test set comprises: The multiple groups of data in the data set are randomly divided into training set, validation set and test set in a ratio of 3:1:

1.

4. The method according to claim 1, characterized in that: The method further comprises: According to the subjective evaluation scores, multiple groups of data in the data set are divided into multiple levels, and corresponding level labels are added to each group of data; The multiple levels correspond to the distortion levels one by one.

5. The method according to claim 1, characterized in that The step of inputting the multiple groups of data in the test set into the target evaluation model for testing to obtain the test results includes: The multiple groups of data in the test set are input into the target evaluation model. The target evaluation model calls the corresponding nonlinear regression equation according to the grade label corresponding to each group of data in the test set to calculate the test result of each group of data.

6. The method according to claim 1, characterized in that Determining whether the test result meets the standard according to the subjective evaluation score of each group of data in the test set includes: When the difference between the test result of each group of data in the test set and the subjective evaluation score corresponding to each group of data is within a set range, it is determined that the test result meets the standard.

7. A quality evaluation device for a raindrop-removed image, characterized in that: The device comprises: A data set establishment module, used to establish a raindrop image data set, wherein the data set includes multiple groups of data, each group of data includes a data pair consisting of distorted data and true value data; A scoring acquisition module, used to obtain a subjective evaluation score for each group of data in the data set; A partitioning module, used for partitioning the data set into a training set, a validation set and a test set; A classification model determination module inputs the training set and the verification set into an initial classification model for training, optimizes the parameters in the initial classification model, and obtains a target classification model, wherein the target classification model is used to determine the distortion level of the image to be evaluated; A calculation module, used for calculating the peak signal-to-noise ratio and structural similarity of each group of data in the training set and the validation set; An evaluation model determination module, used to construct a target evaluation model according to the peak signal-to-noise ratio, structural similarity and subjective evaluation score of each group of data in the training set and the validation set, wherein the target evaluation model is used to evaluate the image quality at different distortion levels; A model testing module, inputting multiple groups of data in the test set into the target evaluation model for testing to obtain test results; A judgment module, used to judge whether the test result meets the standard according to the subjective evaluation score of each group of data in the test set; An evaluation module, when the test result meets the standard, inputs the image to be evaluated output by the raindrop removal algorithm and the corresponding raindrop-free true value image into the target classification model and the target evaluation model in groups, the target classification model determines the distortion level of the image to be evaluated, and the target evaluation model outputs the corresponding image quality evaluation result according to the distortion level of the image to be evaluated; The evaluation model determination module is used for: According to the level labels corresponding to the multiple groups of data in the training set and the validation set, the multiple groups of data in the training set and the validation set are divided into multiple subsets, and the level labels corresponding to the groups of data in each subset are the same; For each subset, a binary nonlinear regression analysis is performed according to the peak signal-to-noise ratio, structural similarity, and subjective evaluation score of each group of data in the subset to obtain a nonlinear regression equation corresponding to the subset; Obtaining multiple nonlinear regression equations corresponding to the multiple subsets to obtain the target evaluation model; The number of the multiple nonlinear regression equations in the target evaluation model corresponds one-to-one to the number of the distortion levels, and each of the nonlinear regression equations is used to evaluate the image quality of the corresponding distortion level.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

Images