Image processing apparatus, and image processing method

Abstract

To provide a technique to achieve an objective evaluation having high correlation with a subjective evaluation on an image.SOLUTION: An image processing apparatus acquires a first evaluation value for every corresponding image area in an image obtained by performing image processing on a first image from an image generator for which noise reduction is performed, and an image obtained by performing the image processing on a second image to be a reference of evaluation of the noise reduction for the first image. The image processing apparatus acquires a second evaluation value input according to a user operation for every corresponding image area in the first image and the second image, and acquires a parameter for calculating the first evaluation value correlated with the second evaluation value on the basis of the first and second evaluation values. The image processing apparatus acquires a third evaluation value on the basis of an image obtained by performing image processing on a third image from the image generator for which noise reduction is performed, and an image obtained by performing the image processing on a fourth image to be a reference of evaluation of the noise reduction for the third image, and acquires an evaluation value for the third image on the basis of the third evaluation value and the parameter.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an evaluation technique for images.

Background Art

[0002] In image generation technologies such as noise removal and super-resolution, in order to evaluate the quality of a tuned image generator, the quality of the generated and output images may be evaluated. There are roughly two types of evaluations of image quality, that is, image quality. Subjective evaluation in which a person directly observes each part of the image and can evaluate it in detail, but it takes time for the evaluation, and different evaluations may be made by different evaluators. And objective evaluation in which the image is analyzed mathematically and a quantitative evaluation is performed.

[0003] In subjective evaluation, a person can directly observe each part of the image and evaluate it in detail, but the evaluation takes time, and different evaluations may be made by different evaluators.

[0004] For objective evaluation, there are evaluation methods such as PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index Measure), and evaluation in a short time is possible by using a computer. In addition, since it is obtained by mathematical calculation, the reproducibility of the evaluation is also available. However, there may be a deviation between the result of the objective evaluation and the result of the subjective evaluation, and there are sporadic cases where the objective evaluation value is good but the evaluated image looks bad, or the objective evaluation value is bad but the evaluated image looks good.

[0005] Patent Document 1 discloses an objective image quality evaluation method highly correlated with subjective image quality evaluation. When comparing two images of the same scene, by using the upper limit value of the allowable deterioration of image quality by human vision based on brightness and evaluating the difference for each pixel, it is possible to perform an objective image quality evaluation close to the subjective image quality evaluation. In addition, by dividing the image into blocks (a collection of pixels) and appropriately weighting the evaluation values calculated for each block, it is possible to flexibly adjust the evaluation values.

Prior Art Documents

Patent Document

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] In the technology disclosed in Patent Document 1, when comparing and evaluating images of the same scene, a threshold is set according to the brightness of pixels, and an evaluation value is calculated using the comparison difference value and the threshold. Also, an objective evaluation approaching subjective evaluation is attempted by adjusting the weight assigned to each block with respect to the evaluation value. However, the technology disclosed in Patent Document 1 does not perform an evaluation focusing on the features and elements in the image that contribute to subjective evaluation. Also, Patent Document 1 does not describe details regarding an appropriate method for adjusting the weight. The present invention provides a technology for realizing an objective evaluation highly correlated with the subjective evaluation of an image.

Means for Solving the Problems

[0008] One aspect of the present invention is a first processed image obtained by performing image processing on a first image output by an image generator that performs noise reduction, and a second image that serves as a reference for evaluating the noise reduction of the first image. A first acquisition means for acquiring a first evaluation value by calculation for each corresponding image region with the second processed image obtained by performing the image processing on the second image; a second acquisition means for acquiring a second evaluation value input according to a user operation for each corresponding image region between the first image and the second image; a third acquisition means for acquiring a parameter for calculating the first evaluation value correlated with the second evaluation value based on the first evaluation value and the second evaluation value; a third processed image obtained by performing image processing on a third image output by an image generator that performs noise reduction, and a fourth processed image obtained by performing the image processing on a fourth image that serves as a reference for evaluating the noise reduction of the third image. A fourth acquisition means for acquiring a third evaluation value by calculation based on the fourth processed image; and a fifth acquisition means for acquiring an evaluation value for the third image based on the third evaluation value and the parameter.

Effects of the Invention

[0009] According to the present invention, it is possible to provide a technique for realizing an objective evaluation highly correlated with a subjective evaluation of an image.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiment for Carrying Out the Invention

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0012] [First Embodiment] In this embodiment, an image evaluation device that operates as an image processing device for calculating a value (integrated evaluation value) representing the result of an objective evaluation of a noise-reduced image (an image in which the noise of the input image has been reduced by an image generator) output by an image generator that performs noise reduction on an input image will be described.

[0013] An example of the functional configuration of a system including the image evaluation device according to this embodiment is shown in the block diagram of FIG. 1. As shown in FIG. 1, the system according to this embodiment includes a storage unit 101, an image evaluation device 100, and an image generation unit 102. In FIG. 1, the storage unit 101, the image evaluation device 100, and the image generation unit 102 are each shown as separate devices, but two or more of these functional units / devices may be combined into one functional unit / device, or one functional unit / device may be divided into a plurality of functional units / devices according to functions.

[0014] First, the storage unit 101 will be described. The storage unit 101 stores one or more sets of a restored image (learning restored image) output in advance by some image generator and a reference image (learning reference image) of the same scene that corresponds to the restored image and serves as a reference. The storage unit 101 may be a server device that can communicate with the image evaluation device 100 via a LAN or the Internet, or may be a functional unit included in the image evaluation device 100.

[0015] Next, the image generation unit 102 will be described. The image generation unit 102 has one or more image generators, generates a noise-reduced image (where the noise in the input image is reduced (removed or decreased)) as a restored image, and outputs the restored image.

[0016] Next, the image evaluation device 100 will be described. As shown in FIG. 1, the image evaluation device 100 is a functional unit for calculating a parameter for calculating an integrated evaluation value, which is an objective evaluation value approaching a subjective evaluation of an image. The image evaluation device 100 includes a creation unit 110 and a calculation unit 120 that calculates the above integrated evaluation value based on the parameter. Note that the subjective evaluation is not an evaluation based on preference, but an evaluation by a human being who is an evaluator and evaluates it as having high image quality.

[0017] The calculation process of the parameter by the creation unit 110 will be described according to the flowchart of FIG. 2. As shown in FIG. 2, the creation unit 110 calculates a parameter for realizing an objective evaluation highly correlated with the subjective evaluation based on a subjective evaluation value, which is a subjective evaluation value input by a user observing an image, and an objective evaluation value, which is an objective evaluation value obtained by performing a mathematical operation process on the image.

[0018] In step S201, the acquisition unit 111 acquires a set of a learning reference image (second image) and a learning restored image (first image) (learning image set) from the storage unit 101. In step S202, the division unit 112 divides each of the learning reference image (second image) and the learning restored image (first image) included in the set acquired in step S201 into a plurality of image regions (divided images). The image division method is not limited to a specific division method. For example, the division unit 112 may divide an image (learning reference image and learning restored image) into a plurality of rectangular regions (divided images), or may divide it into regions for each object (divided images) by segmentation such as semantic segmentation. The shape of the divided image is not limited to a rectangle, and it is divided so that the learning reference image (second image) and the learning restored image (first image) can be evaluated.

[0019] In step S203, the image display unit 113 selects one unselected divided image among the divided images in the learning reference image as a selected image (second selected divided image), and selects, as a selected image (first selected divided image), the divided image that corresponds positionally to the first selected divided image in the learning restored image. Then, the image display unit 113 causes the first selected divided image and the second selected divided image to be displayed on the display screen of the image evaluation device 100. The display screen is a liquid crystal screen or a touch panel screen.

[0020] The user observes the first selected divided image and the second selected divided image displayed on the display screen, and evaluates the image quality of the first selected divided image with respect to the second selected divided image (the subjective evaluation of the user). Then, the user operates the input unit 114, which is a user interface such as a keyboard, a mouse, or a touch panel screen, and inputs, as a subjective evaluation value, a value that quantitatively represents the result of the subjective evaluation.

[0021] The subjective evaluation value may be input using any scale. For example, it may be set such that the range is from 0 to 1, where 0 represents high image quality and 1 represents low image quality. In this case, as a result of the subjective evaluation, the better the image quality of the first selected divided image with respect to the second selected divided image, the closer to 0 the user inputs the subjective evaluation value. On the other hand, as a result of the subjective evaluation, the worse the image quality of the first selected divided image with respect to the second selected divided image, the closer to 1 the user inputs the subjective evaluation value. This "range" is not limited to 0 to 1, and may be, for example, 0 to 100. Thus, the relationship between the result of the subjective evaluation and the subjective evaluation value may be any relationship. The image display unit 113 acquires the subjective evaluation value input by the user in this way. Note that the method of inputting the subjective evaluation value by the user is not limited to a specific method. Also, the arbitrary scale determined here is unified until step S207 ends to make the evaluation consistent.

[0022] Next, in step S204, the image processing unit 115 applies a plurality of image processes to the second selected divided image to generate a plurality of processed divided images, such as a divided image obtained by applying the first image process to the second selected divided image, a divided image obtained by applying the second image process to the second selected divided image, and so on. Similarly, the image processing unit 115 applies a plurality of image processes to the first selected divided image to generate a plurality of processed divided images, such as a divided image obtained by applying the first image process to the first selected divided image, a divided image obtained by applying the second image process to the first selected divided image, and so on.

[0023] The plurality of image processes include image processes for making features that contribute to a subjective determination of the quality of an image stand out when a person visually observes the image. Such image processes include, for example, edge enhancement, texture enhancement, and low-frequency component enhancement. The image processes may be used sequentially, or different image processes may be further applied to the image output after the image process.

[0024] Then, for each image process applied by the image processing unit 115, the arithmetic unit 116 performs arithmetic processing based on the divided image generated by applying the image process to the first selected divided image and the divided image generated by applying the image process to the second selected divided image, and calculates a learning objective evaluation value, which is an objective evaluation value. The arithmetic processing is processing for calculating a value representing the difference between the divided images as the learning objective evaluation value, and various processes can be applied to the processing. For example, for the arithmetic processing, a process of calculating the sum of the pixel values in the difference image between the divided images as the learning objective evaluation value, and a process of calculating the sum of the results of the logical operation between the divided images as the learning objective evaluation value are applicable.

[0025] Note that the order of executing the process for obtaining the subjective evaluation value and the process for obtaining the learning objective evaluation value is not limited to the above order, and may be reversed, or these processes may be executed in parallel.

[0026] In step S205, the image display unit 113 determines whether all the divided images in the learning reference image (second image) and the learning restored image (first image) have been selected as the selected images. As a result of this determination, if all the divided images in the learning reference image and the learning restored image have been selected as the selected images, the process proceeds to step S206. On the other hand, if there are still divided images in the learning reference image and the learning restored image that have not yet been selected as the selected images, the process proceeds to step S203.

[0027] In step S206, the acquisition unit 111 determines whether all the sets to be used for parameter generation have been acquired. The "all sets to be used for parameter generation" may be, for example, all the sets stored in the storage unit 101, or may be sets preset as the "sets to be used for parameter generation".

[0028] As a result of this determination, if all the sets to be used for parameter generation have been acquired, the process proceeds to step S207. If there are still sets to be used for parameter generation that have not yet been acquired, the process proceeds to step S201.

[0029] In step S207, the learning unit 117 calculates (learns) parameters for realizing an objective evaluation that approaches the subjective evaluation by using the subjective evaluation value group acquired in step S203 and the learning objective evaluation value group acquired in step S204. Here, parameter calculation using multiple regression analysis will be described as an example. Multiple regression analysis requires one type of objective variable and two or more types of explanatory variables. For one divided region, the subjective evaluation value acquired in step S203 can be used as the objective variable, and a plurality of learning objective evaluation values acquired in step S204 for the divided region can be used as the explanatory variables. There will be as many datasets of this objective variable and explanatory variables as the number of divided images in the learning reference image (or learning restored image). The learning unit 117 performs multiple regression analysis using all the datasets of the objective variable and explanatory variables, and calculates, as parameters, a plurality of coefficients and one constant applied to each objective evaluation value such that an objective evaluation value approaching the subjective evaluation can be calculated.

[0030] Next, the objective evaluation by the calculation unit 120 will be described according to the flowchart of FIG. 3. As shown in FIG. 3, the calculation unit 120 calculates an integrated evaluation value by using the parameters calculated by the process according to the flowchart of FIG. 2.

[0031] In step S301, the acquisition unit 121 inputs an input image (for example, a degraded image including noise) to the image generation unit 102 (image generator), and acquires a set (evaluation image set) of "a restored image with reduced noise in the input image" (evaluation restored image) output from the image generation unit 102 and a reference image (evaluation reference image) of the same scene that corresponds to the restored image and serves as a reference. The method for acquiring the evaluation image set by the acquisition unit 121 is not limited to a specific acquisition method.

[0032] In step S302, the image processing unit 122 applies a plurality of image processes to the restored image for evaluation (third image), such as an image obtained by applying the first image process to the restored image for evaluation (third image), an image obtained by applying the second image process to the restored image for evaluation (third image), and so on, to generate a plurality of processed images. Similarly, the image processing unit 115 applies a plurality of image processes to the reference image for evaluation (fourth image), such as an image obtained by applying the first image process to the reference image for evaluation (fourth image), an image obtained by applying the second image process to the reference image for evaluation (fourth image), and so on, to generate a plurality of processed images. The plurality of image processes applied by the image processing unit 122 are the same as the plurality of image processes applied by the image processing unit 115 in step S204.

[0033] Then, for each image process applied by the image processing unit 122, the arithmetic unit 123 performs arithmetic processing based on the image generated by applying the image process to the restored image for evaluation (third image) and the image generated by applying the image process to the reference image for evaluation (fourth image), and calculates an objective evaluation value for evaluation (third evaluation value). The arithmetic processing performed by the arithmetic unit 123 is processing for calculating, as the objective evaluation value for evaluation, a value representing the difference between the image generated by applying the image process to the restored image for evaluation (third image) and the image generated by applying the image process to the reference image for evaluation (fourth image), and various processes can be applied to that processing. For example, applicable to the arithmetic processing are processing for calculating, as the objective evaluation value for evaluation, the sum of the pixel values in the difference image between the images, and processing for calculating, as the objective evaluation value for evaluation, the sum of the results of logical operations between the images. The arithmetic processing performed by the arithmetic unit 123 is the same as the arithmetic processing performed by the arithmetic unit 116 in step S204.

[0034] In step S303, the acquisition unit 124 acquires the parameters calculated by the creation unit 110 through the processing according to the flowchart of FIG. 2. In step S304, the integration unit 125 calculates an integrated evaluation value, which is an objective evaluation value for the restored image for evaluation, using the objective evaluation value for evaluation calculated for each image process in step S302 and the parameters acquired in step S303. Note that the method for calculating the integrated evaluation value by the integration unit 125 is not limited to a specific calculation method.

[0035] For example, the integration unit 125 may multiply each objective evaluation value for evaluation by the corresponding coefficient among the coefficient group included in the parameters, and calculate the linear sum of the objective evaluation value for evaluation multiplied by the coefficient and the constant included in the parameters as the integrated evaluation value. Also, for example, the integration unit 125 may obtain the integrated evaluation value by calculating the following formula.

[0036] Integrated evaluation value = α x √(gtA - evaA) 2 + β x √(gtB - evaB) 2 + ε This formula corresponds to the case where image processes A and B are applied to the restored image for evaluation (the third image) and the reference image for evaluation (the fourth image) in step S302. In this case, in step S302, the image processing unit 122 generates an image evaA obtained by applying image process A to the restored image for evaluation and an image evaB obtained by applying image process B to the restored image for evaluation. Also, the image processing unit 122 generates an image gtA obtained by applying image process A to the reference image for evaluation and an image gtB obtained by applying image process B to the reference image for evaluation.

[0037] Then, the arithmetic unit 123 obtains (gtA - evaA), which is a value based on the difference image A between the image evaA and the image gtA. 2 This formula may be, for example, the sum of the squares of the pixel values of each pixel in the difference image A. Also, the arithmetic unit 123 obtains (gtB - evaB), which is a value based on the difference image B between the image evaB and the image gtB. 2 This formula may be, for example, the sum of the squares of the pixel values of each pixel in the difference image B.

[0038] And the calculation unit 123 uses α, β, ε, (gtA - evaA) 2 , (gtB - evaB) 2 , to calculate the above formula to obtain an integrated evaluation value. Here, α and β are coefficients included in the parameters obtained in step S303, and ε is a constant included in the parameters obtained in step S303.

[0039] And the calculation unit 123 outputs the calculated integrated evaluation value. The output form of the integrated evaluation value is not limited to a specific output form. For example, the calculation unit 123 may display the integrated evaluation value on the above display screen, or may display an image representing the integrated evaluation value on the above display screen. In the latter case, for example, the calculation unit 123 may display an image or character that is larger as the integrated evaluation value is higher, or an image or character whose color is closer to a specified color (for example, red) as the integrated evaluation value is higher.

[0040] The integrated evaluation value depends on the superiority and inferiority relationship of the subjective evaluation values. For example, when setting the range from 0 to 1 with 0 being high image quality and 1 being low image quality, the closer the integrated evaluation value is to 0, the higher the image quality of the evaluation restoration image is for the evaluator who conducted the subjective evaluation.

[0041] By performing the evaluation of this embodiment on one or more images output by one image generator, an integrated evaluation value, which is an objective evaluation value considering the subjective evaluation during parameter creation, is output. This integrated evaluation value has less deviation from the feeling when visually checking the image and the superiority and inferiority of the objective evaluation value compared to the conventional method, and it becomes possible to judge the quality of the image more efficiently and accurately. Therefore, by inputting one or more high-noise images (images containing more noise / stronger noise) into a plurality of image generators, calculating the integrated evaluation value of this embodiment for all the output images, and displaying them as described above, the image quality of the images output by each image generator can be compared with relative numerical values. As a result, it becomes possible to objectively select an image generator that outputs an image with good image quality in subjective evaluation.

[0042] In this embodiment, a case where an integrated evaluation value is calculated using the parameters calculated by the creation unit 110 has been described, but it is also possible to calculate the integrated evaluation value using parameters prepared in advance.

[0043] Also, in this embodiment, a case where it is applied to an image generator related to noise reduction processing has been described, but the applicable cases are not limited to such cases. For example, it can also be applied to other degradation corrections, style conversions (such as monochrome conversion, etc.).

[0044] Note that in this embodiment, image processing is performed in units of divided images, but a plurality of first processed images and a plurality of second processed images may be obtained by performing a plurality of image processes on the restored image for learning and the reference image for learning. In this case, the image evaluation device 100 calculates a learning objective evaluation value by performing arithmetic processing for each corresponding image area between the first processed image and the second processed image. Also, the image evaluation device 100 acquires a subjective evaluation value input according to a user operation for each corresponding image area between the restored image for learning and the reference image for learning.

[0045] [Second Embodiment] In this embodiment, the differences from the first embodiment will be described, and unless otherwise specifically mentioned below, it is assumed to be the same as the first embodiment. A functional configuration example of the system according to this embodiment is shown in the block diagram of FIG. 4. In FIG. 4, the same reference numerals are assigned to the functional units similar to those shown in FIG. 1, and the description of the functional units is omitted. The parameter calculation process by the creation unit 110 will be described according to the flowchart of FIG. 5. In FIG. 5, the same step numbers are assigned to the processing steps similar to those shown in FIG. 2, and the description of the processing steps is omitted.

[0046] In step S505, the classification unit 118 classifies the first selected divided image into any one of a plurality of preset categories. For example, the classification unit 118 may obtain an index by performing a mathematical calculation on the first selected divided image, and classify the first selected divided image into any one of the plurality of categories according to the result of comparing the index with one or more threshold values. Alternatively, the classification unit 118 may classify the first selected divided image into any one of the plurality of categories by using a model that has been learned in advance by machine learning. For example, the classification unit 118 can extract edges in the first selected divided image and classify the first selected divided image into one of the plurality of categories according to the ratio of the edges in the first selected divided image.

[0047] Note that the order of executing the process for obtaining the subjective evaluation value, the process for obtaining the learning objective evaluation value, and the process in step S505 is not limited to the above order, and these processes may be executed in parallel.

[0048] In step S508, for each category, the learning unit 117 uses the subjective evaluation value input by the user for the first selected divided image classified into the category among the group of subjective evaluation values obtained in step S203 (the subjective evaluation value corresponding to the category), and the learning objective evaluation value calculated using the first selected divided image classified into the category among the group of learning objective evaluation values obtained in step S204 (the learning objective evaluation value corresponding to the category), and calculates (learns) the "parameters for realizing an objective evaluation approaching the subjective evaluation" corresponding to the category by using, for example, multiple regression analysis as in the first embodiment. That is, in this embodiment, the learning unit 117 calculates parameters for each category into which the first selected divided image is classified.

[0049] Next, the objective evaluation by the calculation unit 120 will be described according to the flowchart of FIG. 6. In FIG. 6, the same step numbers are assigned to the processing steps similar to those shown in FIG. 3, and the description of the processing steps is omitted.

[0050] In step S602, the acquisition unit 124 acquires the parameters for each category calculated by the creation unit 110 through the process according to the flowchart of FIG. 5. In step S603, the division unit 126 divides each of the reference image for evaluation and the restored image for evaluation included in the set acquired in step S301 into a plurality of image regions (divided images) by the same division method as the division method by the division unit 112.

[0051] In step S604, the image processing unit 122 performs the same process as step S302 above, using "the first selected divided image which is one of the unselected divided images in the divided image of the reference image for evaluation" instead of the "reference image for evaluation" and "the second selected divided image which is one of the unselected divided images in the divided image of the restored image for evaluation" instead of the "restored image for evaluation", to calculate the objective evaluation value for evaluation for each image process for the one unselected divided image.

[0052] In step S605, the classification unit 127 classifies the first selected divided image into any one of a plurality of preset categories, in the same manner as the classification unit 118. Note that the classification method and the number of classification categories are set in the same manner as in step S505. By matching the classification conditions of the region at the time of parameter creation and the divided region, the parameters created to approach the subjective evaluation are effectively applied, and it becomes possible to perform the evaluation.

[0053] In step S606, the integration unit 125 calculates the region integration evaluation value corresponding to the first selected divided image / second selected divided image, using the objective evaluation value for evaluation calculated in step S604 and the parameters acquired in step S602. Note that the calculation method of the region integration evaluation value by the integration unit 125 is not limited to a specific calculation method.

[0054] For example, the integration unit 125 may multiply each objective evaluation value for evaluation by the corresponding coefficient among the coefficient groups included in the parameter (parameters of the category to which the first selected divided image is classified), and calculate the linear sum of the objective evaluation value for evaluation multiplied by the coefficient and the constant included in the parameter as the region integration evaluation value. For another example, the integration unit 125 may obtain the region integration evaluation value by calculating the following formula.

[0055] Region integration evaluation value = α i x√(gtAR - evaAR) 2 + β i x√(gtBR - evaBR) 2 + ε i This formula corresponds to the case where image processing A and image processing B are applied to the first selected divided image and the second selected divided image in step S604. In this case, in step S604, the image processing unit 122 generates an image evaAR obtained by applying image processing A to the second selected divided image and an image evaBR obtained by applying image processing B to the second selected divided image. Further, the image processing unit 122 generates an image gtAR obtained by applying image processing A to the first selected divided image and an image gtBR obtained by applying image processing B to the first selected divided image.

[0056] Then, the calculation unit 123 obtains (gtAR - evaAR), which is a value based on the difference image AR between the image evaAR and the image gtAR. This formula may be, for example, the sum of the squares of the pixel values of each pixel in the difference image AR. Further, the calculation unit 123 obtains (gtBR - evaBR), which is a value based on the difference image BR between the image evaBR and the image gtBR. This formula may be, for example, the sum of the squares of the pixel values of each pixel in the difference image BR. 2 to be obtained. This formula may be, for example, the sum of the squares of the pixel values of each pixel in the difference image AR. Also, the calculation unit 123 obtains (gtBR - evaBR), which is a value based on the difference image BR between the image evaBR and the image gtBR. 2 to be obtained. This formula may be, for example, the sum of the squares of the pixel values of each pixel in the difference image BR.

[0057] Then the calculation unit 123 calculates α i , β i , ε i , (gtAR - evaAR) 2 , (gtBR - evaBR) 2By calculating the above formula using [parameters], the region integration evaluation value is obtained. Here, α i , β i are coefficients included in the parameters calculated for category i into which the first selected divided image is classified, and ε i is a constant included in the parameters calculated for category i into which the first selected divided image is classified. This region integration evaluation value is a quantitative evaluation value regarding the local image quality in the restored image for evaluation, and can be used for relative comparison regarding the image quality of local regions.

[0058] In step S607, the arithmetic unit 123 determines whether the region integration evaluation values have been calculated for all the divided images in the reference image for evaluation and the restored image for evaluation. As a result of this determination, if the region integration evaluation values have been calculated for all the divided images in the reference image for evaluation and the restored image for evaluation, the process proceeds to step S608. On the other hand, if there are still divided images for which the region integration evaluation values have not been calculated among the divided images in the reference image for evaluation and the restored image for evaluation, the process proceeds to step S604.

[0059] In step S608, the summing unit 128 calculates the sum of the region integration evaluation values calculated in step S606 as the integrated evaluation value, and outputs the calculated integrated evaluation value. The region integration evaluation values calculated in step S606 are local evaluation values, but by taking the sum, it is possible to represent an evaluation value of the entire image considering the local features in the image. Also, similar to the first embodiment, it can be an index to assist in determining an image generator that generates a high-quality image for the user.

[0060] In this embodiment, the case where the integrated evaluation value is calculated using the parameters calculated by the creation unit 110 has been described, but it is also possible to calculate the integrated evaluation value using parameters prepared in advance.

[0061] [Third Embodiment] In the image evaluation apparatus 100 shown in FIGS. 1 and 4, each functional unit may be implemented by hardware, or each functional unit except the input unit 114 may be implemented by software (computer program). In the latter case, a computer device capable of executing such a computer program is applicable to the image evaluation apparatus 100. A hardware configuration example of a computer device applicable to the image evaluation apparatus 100 will be described with reference to the block diagram of FIG. 7.

[0062] The CPU 701 executes various processes using the computer programs and data stored in the RAM 702. Thereby, the CPU 701 controls the operation of the entire computer device and executes or controls the various processes described as the processes performed by the image evaluation apparatus 100.

[0063] The RAM 702 has an area for storing computer programs and data loaded from the ROM 703 and the storage device 706, and an area for storing various information received from the outside via the I / F 707. Further, the RAM 702 has a work area used when the CPU 701 executes various processes. In this way, the RAM 702 can appropriately provide various areas.

[0064] The ROM 703 stores setting data of the computer device, computer programs and data related to the basic operation of the computer device, computer programs and data related to the startup of the computer device, and the like.

[0065] The operation unit 704 is a user interface such as a keyboard, a mouse, or a touch panel screen, and various instructions and information can be input to the computer device by the user's operation. The operation unit 704 can also be used as the input unit 114 described above.

[0066] The display unit 705 has a liquid crystal screen or a touch panel screen, and displays the processing result by the CPU 701 in the form of images, characters, etc. Note that the display unit 705 can also be used as the above-mentioned display screen. Note that the display unit 705 may be a projection device such as a projector that projects images and characters.

[0067] The storage device 706 is a large-capacity non-volatile memory such as a hard disk drive device. In the storage device 706, an operating system, computer programs and data for causing the CPU 701 to execute or control various processes described as the processes performed by the image evaluation device 100, etc. are stored.

[0068] The I / F 707 is a communication interface for performing data communication with an external device. For example, the above-mentioned storage unit 101 and image generation unit 102 can be connected to the I / F 707, and the computer device can perform data communication with the storage unit 101 and image generation unit 102 via the I / F 707.

[0069] The CPU 701, RAM 702, ROM 703, operation unit 704, display unit 705, storage device 706, and I / F 707 are all connected to the system bus 708. Note that the configuration shown in FIG. 7 is merely an example of the hardware configuration of a computer device applicable to the image evaluation device 100, and can be appropriately modified / changed.

[0070] The numerical values, processing timings, processing orders, processing entities, methods of acquiring / sending destinations / sending sources / storage locations of data (information), etc. used in the above-described embodiments and each modification example are given as examples for the purpose of specific explanation, and are not intended to be limited to such examples.

[0071] Also, some or all of the above-described embodiments and each modification example may be appropriately combined and used. Also, some or all of the above-described embodiments and each modification example may be selectively used.

[0072] (Other Embodiments) The present invention can also be realized by supplying a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. It can also be realized by a circuit (for example, ASIC) that implements one or more functions.

[0073] The invention described in this specification includes the following image processing apparatus, image processing method, and computer program. (Item 1) A first acquisition means for acquiring a first evaluation value by calculation for each corresponding image region between a first image-processed image obtained by performing image processing on a first image output by an image generator that performs noise reduction, and a second image-processed image obtained by performing the image processing on a second image that is a reference for evaluating the noise reduction of the first image; A second acquisition means for acquiring a second evaluation value input according to a user operation for each corresponding image region between the first image and the second image; A third acquisition means for acquiring a parameter for calculating the first evaluation value correlated with the second evaluation value based on the first evaluation value and the second evaluation value; A fourth acquisition means for acquiring a third evaluation value by calculation based on a third image-processed image obtained by performing image processing on a third image output by an image generator that performs noise reduction, and a fourth image-processed image obtained by performing the image processing on a fourth image that is a reference for evaluating the noise reduction of the third image; A fifth acquisition means for acquiring an evaluation value for the third image based on the third evaluation value and the parameter An image processing apparatus characterized by comprising the above. (Item 2) The image processing apparatus according to Item 1, wherein the first acquisition means acquires the first evaluation value based on a difference between corresponding image regions of the first image-processed image and the second image-processed image. (Item 3) The image processing apparatus according to item 1 or 2, wherein the first acquisition means acquires the first evaluation value for each image process. (Item 4) The image processing apparatus according to any one of items 1 to 3, wherein the second acquisition means displays the image area and acquires a second evaluation value input according to a user operation for the image area. (Item 5) The image processing apparatus according to any one of items 1 to 4, wherein the third acquisition means calculates the parameter by performing multiple regression analysis using the first evaluation value as an explanatory variable and the second evaluation value as an objective variable. (Item 6) The image processing apparatus according to any one of items 1 to 5, wherein the fourth acquisition means acquires the third evaluation value based on the difference between the third image-processed image and the fourth image-processed image. (Item 7) The image processing apparatus according to any one of items 1 to 6, wherein the fourth acquisition means acquires the third evaluation value for each image process. (Item 8) The image processing apparatus according to any one of items 1 to 7, wherein the fifth acquisition means outputs an evaluation value for the third image. (Item 9) The image processing apparatus according to item 8, wherein the fifth acquisition means outputs an evaluation value for the third image for each image generator. (Item 10) The image processing apparatus according to any one of items 1 to 9, wherein the first image is an image output by an image generator that performs noise reduction, and the second image is an image of the same scene corresponding to the image and serving as a reference. (Item 11) The image processing apparatus according to any one of items 1 to 10, wherein the third image is an image output by an image generator that performs noise reduction, and the fourth image is an image of the same scene corresponding to the image and serving as a reference. (Item 12) The image processing apparatus according to any one of Items 1 to 11, wherein the image area includes a rectangular area and an area for each object divided by segmentation. (Item 13) The third acquisition means acquires the parameter based on the first evaluation value and the second evaluation value corresponding to the category for each corresponding image area in the first processed image and the second processed image. The fifth acquisition means obtains an evaluation value for each corresponding image area in the third image and the fourth image, based on the third evaluation value corresponding to the image area and the parameter corresponding to the category of the image area, and obtains the sum of the evaluation values as the evaluation value for the third image. The image processing apparatus according to Item 1, characterized in that. (Item 14) An image processing method performed by an image processing apparatus, A first acquisition step of acquiring a first evaluation value by calculation for each corresponding image area of a first processed image obtained by performing image processing on a first image output by an image generator that performs noise reduction, and a second processed image obtained by performing the image processing on a second image that is a reference for evaluating the noise reduction of the first image; A second acquisition step of the second acquisition means of the image processing apparatus for acquiring a second evaluation value input in response to a user operation for each corresponding image area of the first image and the second image; A third acquisition step of the third acquisition means of the image processing apparatus for acquiring a parameter for calculating the first evaluation value correlated with the second evaluation value based on the first evaluation value and the second evaluation value; A fourth acquisition step of the fourth acquisition means of the image processing apparatus for acquiring a third evaluation value by calculation based on a third processed image obtained by performing image processing on a third image output by an image generator that performs noise reduction, and a fourth processed image obtained by performing the image processing on a fourth image that is a reference for evaluating the noise reduction of the third image; A fifth acquisition step in which a fifth acquisition means of the image processing apparatus acquires an evaluation value for the third image based on the third evaluation value and the parameter An image processing method characterized by comprising the above. (Item 15) A computer program for causing a computer to function as each means of the image processing apparatus according to any one of Items 1 to 13.

[0074] The invention is not limited to the above embodiments, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of Reference Numerals

[0075] 100: Image evaluation apparatus 101: Storage unit 102: Image generation unit 110: Creation unit 111: Acquisition unit 112: Division unit 113: Image display unit 114: Input unit 115: Image processing unit 116: Calculation unit 117: Learning unit 120: Calculation unit 121: Acquisition unit 122: Image processing unit 123: Calculation unit 124: Acquisition unit 125: Integration unit

Claims

1. First image processing means for performing image processing on a first image output by an image generator that reduces noise, and obtaining a first processed image; and second image processing means for performing the same image processing on a second image that serves as a reference for evaluating the noise reduction of the first image, and obtaining a second processed image. First acquisition means for obtaining a first evaluation value by calculation for each corresponding image area of the first processed image and the second processed image; Second acquisition means for obtaining a second evaluation value input according to a user operation for each corresponding image area of the first image and the second image; Third acquisition means for obtaining a parameter for calculating the first evaluation value correlated with the second evaluation value based on the first evaluation value and the second evaluation value; Fourth acquisition means for obtaining a third evaluation value by calculation based on a third processed image obtained by performing image processing on a third image output by an image generator that reduces noise, and a fourth processed image obtained by performing the same image processing on a fourth image that serves as a reference for evaluating the noise reduction of the third image; Fifth acquisition means for obtaining an evaluation value for the third image based on the third evaluation value and the parameter An image processing apparatus comprising the above.

2. The image processing apparatus according to claim 1, wherein the first acquisition means obtains the first evaluation value based on a difference between corresponding image areas of the first processed image and the second processed image.

3. The image processing apparatus according to claim 1, wherein the first acquisition means obtains the first evaluation value for each image process.

4. The image processing apparatus according to claim 1, wherein the second acquisition means displays the image area and obtains a second evaluation value input according to a user operation for the image area.

5. The image processing apparatus according to claim 1, wherein the third acquisition means calculates the parameter by performing multiple regression analysis using the first evaluation value as an explanatory variable and the second evaluation value as a target variable.

6. The image processing apparatus according to claim 1, wherein the fourth acquisition means obtains the third evaluation value based on a difference between the third processed image and the fourth processed image.

7. The image processing apparatus according to claim 1, wherein the fourth acquisition means obtains the third evaluation value for each image process.

8. The image processing apparatus according to claim 1, wherein the fifth acquisition means outputs an evaluation value for the third image.

9. The image processing apparatus according to claim 8, wherein the fifth acquisition means outputs an evaluation value for the third image for each image generator.

10. The image processing apparatus according to claim 1, wherein the first image is an image output by an image generator that performs noise reduction, and the second image is an image of the same scene corresponding to the first image and serving as a reference.

11. The image processing apparatus according to claim 1, wherein the third image is an image output by an image generator that performs noise reduction, and the fourth image is an image of the same scene corresponding to the third image and serving as a reference.

12. The image processing apparatus according to claim 1, wherein the image region includes a rectangular region and a region for each object divided by segmentation.

13. The third acquisition means acquires the parameter based on the first evaluation value and the second evaluation value corresponding to the category for each corresponding image region of the first processed image and the second processed image, The fifth acquisition means obtains an evaluation value for each corresponding image region of the third image and the fourth image based on the third evaluation value corresponding to the image region and the parameter corresponding to the category of the image region, and obtains the sum of the evaluation values as the evaluation value for the third image. The image processing apparatus according to claim 1, characterized in that.

14. An image processing method performed by an image processing apparatus, A first acquisition step in which a first acquisition means of the image processing apparatus acquires a first evaluation value by calculation for each corresponding image region of a first processed image obtained by performing image processing on a first image output by an image generator that performs noise reduction and a second processed image obtained by performing the image processing on a second image serving as a reference for evaluating the noise reduction of the first image; A second acquisition step in which a second acquisition means of the image processing apparatus acquires a second evaluation value input in response to a user operation for each corresponding image region of the first image and the second image; A third acquisition step in which a third acquisition means of the image processing apparatus acquires a parameter for calculating the first evaluation value correlated with the second evaluation value based on the first evaluation value and the second evaluation value; A fourth acquisition step of acquiring a third evaluation value by calculation based on a third image-processed image obtained by performing image processing on a third image output by an image generator that performs noise reduction, and a fourth image-processed image obtained by performing the image processing on a fourth image that is a reference for evaluating the noise reduction of the third image; A fifth acquisition step of acquiring an evaluation value for the third image by the fifth acquisition means of the image processing apparatus based on the third evaluation value and the parameter; An image processing method, characterized by comprising: **Claim 15** A computer program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 13.

Images