Image processing device, image apparatus, image processing method, and program

Abstract

To provide an image processing device, an image apparatus, an image processing method, and a program for obtaining an image having a resolution characteristic desired by a user (an image of a real object that is visually recognized as being changed relative to the real object according to the resolution characteristic).SOLUTION: An image processing device 100 includes a conversion unit 3 that converts an input image into an output image based on a goal MTF desired by a user, a target MTF obtained from the input image, and input spectral information obtained from the input image, and an output unit 5 that outputs the output image converted by the conversion unit 3.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus, an image device, an image processing method, and a program.

Background Art

[0002] In image devices such as display devices or imaging devices, in order to accurately convey information about real objects, technologies for acquiring, processing, or generating images based on optical characteristics such as imaging, display, or imaging are known.

[0003] For example, Patent Document 1 discloses an image processing apparatus that corrects the pixel value of a pixel at an image height to be corrected using a filter that performs blur correction set for each of a plurality of image heights. Further, Patent Document 2 discloses an image processing apparatus that performs unsharp masking processing on a captured image generated by imaging through an optical system using a filter generated based on information on a PSF corresponding to imaging conditions of the optical system.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the image processing apparatuses described in Patent Document 1 and Patent Document 2, processing is performed based on information regarding resolution characteristics such as the PSF (Point Spread Function) or MTF (Modulation Transfer Function) of the optical system preset by the manufacturer of the image processing apparatus. For this reason, users of the image processing apparatus or the like cannot operate the PSF or MTF, and there is room for improvement in obtaining an image having the resolution characteristics desired by the user. Further, in the image processing apparatus described in Patent Document 2, since it is an image sharpening process by software, the processed image is arbitrarily emphasized and unnatural, and noise and the like are likely to occur, and there is also room for improvement in that the generality of the image to be processed is lacking.

[0006] One aspect of the present disclosure aims to obtain an image having resolution characteristics desired by a user.

Means for Solving the Problems

[0007] An image processing apparatus according to one aspect of the present disclosure includes a conversion unit that converts the input image into an output image based on a target MTF, a target MTF acquired from the input image, and input spectrum information acquired from the input image, and an output unit that outputs the output image converted by the conversion unit.

[0008] An image processing method according to one aspect of the present disclosure is an image processing method by an image processing apparatus, in which the image processing apparatus converts the input image into an output image based on a target MTF, a target MTF acquired from the input image, and input spectrum information acquired from the input image by a conversion unit, and outputs the output image converted by the conversion unit by an output unit.

[0009] A program according to one aspect of the present disclosure causes an image processing apparatus to execute a process of converting an input image into an output image based on a target MTF, a target MTF obtained from the input image, and input spectrum information obtained from the input image by a conversion unit, and outputting the output image converted by the conversion unit by an output unit.

Advantages of the Invention

[0010] According to one aspect of the present disclosure, an image having resolution characteristics desired by a user can be obtained.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments for carrying out the present disclosure will be described in detail with reference to the drawings. However, the embodiments shown below are examples of an image processing apparatus, an image device, an image processing method, and a program for embodying the technical idea of the present disclosure, and are not limited thereto. In each drawing, the same reference numerals are given to the same components, and redundant explanations are omitted as appropriate.

[0013] In the embodiments shown below, the images shall include still images and moving images. Moving images can also be referred to as videos.

[0014] [First Embodiment] <Configuration of the Image Processing Apparatus According to the First Embodiment> With reference to FIGS. 1 and 2, the configuration of the image processing apparatus according to the first embodiment will be described. FIG. 1 is a block diagram showing an example of the hardware configuration of an image processing apparatus 100 according to the first embodiment. FIG. 2 is a block diagram showing an example of the functional configuration of the image processing apparatus 100.

[0015] (Hardware Configuration) As shown in FIG. 1, the image processing apparatus 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, and a RAM (Random Access Memory) 103. The image processing apparatus 100 also includes an HDD (Hard Disk Drive) 104 and a communication I / F (Interface) 105. These are communicably connected to each other via a system bus B.

[0016] The CPU 101 executes control processing including various arithmetic processes. The ROM 102 stores programs used for driving the CPU 101 such as an IPL (Initial Program Loader). The RAM 103 is used as a work area for the CPU 101. The HDD 104 is a non-volatile storage device that stores various data, programs, etc. The data includes images to be processed by the image processing apparatus 100, target MTFs, object MTFs, etc.

[0017] The communication I / F 105 is an interface for performing communication between the image processing apparatus 100 and devices or apparatuses other than the image processing apparatus 100. The communication I / F 105 may perform communication with devices or apparatuses other than the image processing apparatus 100 via a network or the like. Examples of devices other than the image processing apparatus 100 include display devices and imaging devices. Examples of apparatuses other than the image processing apparatus 100 include external PCs (Personal Computers) and external servers.

[0018] (Overall Functional Configuration) As shown in FIG. 2, the image processing apparatus 100 has a conversion unit 3 that converts an input image Si into an output image So based on a target MTF Mt, an object MTF Ms acquired from the input image Si, and input spectrum information Pi acquired from the input image Si. The image processing apparatus 100 also has an output unit 5 that outputs the output image So converted by the conversion unit 3.

[0019] The image processing apparatus 100 is an apparatus that converts an input image Si into an output image So and outputs it in order to obtain an image having resolution characteristics desired by a user. An image having resolution characteristics refers to an image of a real object that is visually recognized in a changed state with respect to the real object according to the resolution characteristics. The target MTF Mt means the MTF that is an index of the resolution characteristics desired by the user. The target MTF Ms means the MTF that is the target for adjustment with the target MTF Mt as an index so that the resolution characteristics in the input image Si become the resolution characteristics desired by the user. The input image Si means the image input to the image processing apparatus 100. The output image Si means the image output from the image processing apparatus 100. The input spectrum information Pi means information indicating the frequency characteristics of the input image Si. The input spectrum information Pi includes at least one of a power spectrum, an amplitude spectrum, and a phase spectrum.

[0020] For example, in an image device, in image acquisition, processing, and generation, it is important to handle the image clearly and accurately convey the optical information of a real object in consideration of the optical characteristics of imaging, display, and imaging. However, the influence of optical constraint conditions such as light scattering and aberration cannot be wiped out, and the image information is inferior to the real object, resulting in various quality degradations such as deterioration of image sharpness (i.e., blurring), false colors, and artifacts.

[0021] As an apparatus for reducing image degradation, Patent Document 1 discloses an image process that corrects the pixel value of a pixel at an image height to be corrected by using filters that perform blur correction respectively set for a plurality of image heights. Also, Patent Document 2 discloses an image processing apparatus that performs unsharp masking processing on a captured image using an optical system by using a filter generated based on information on a PSF corresponding to the shooting conditions of the optical system. However, in the image processing apparatuses described in Patent Document 1 and Patent Document 2, image processing is performed based on information regarding resolution characteristics such as the PSF or MTF of the optical system preset by the manufacturer of the image processing apparatus. Users of the image processing apparatus or imaging devices, etc. cannot know the information regarding the resolution characteristics preset by the manufacturer. For this reason, it may be difficult for the user to adjust the resolution characteristics of the image to the resolution characteristics desired by the user. Therefore, there is room for improvement in the image processing apparatuses described in Patent Document 1 and Patent Document 2 in terms of obtaining an image having the resolution characteristics desired by the user.

[0022] The image processing apparatus 100 according to this embodiment uses the MTF, which is an index of resolution characteristics, and converts the input image Si into the output image So based on the target MTF Mt desired by the user, the target MTF Ms acquired from the input image Si, and the input spectrum information Pi. For example, the image processing apparatus 100, by the conversion unit 3, converts the input image Ci(n) into the output image Co(n) so as to bring the target MTF Ms(n) closer to the target MTF Mt corresponding to the resolution characteristics desired by the user. Thereby, even if information regarding resolution characteristics such as the PSF or MTF is unknown, the user can adjust the resolution characteristics by operating the MTF characteristics and obtain an image having the resolution characteristics desired by the user. Note that the conversion of the input image Si to the output image So in the image processing apparatus 100 is not limited to conversion for correcting image blur, which is conversion for increasing the MTF, and may be various conversions such as conversion for generating image blur, which is conversion for decreasing the MTF.

[0023] In the image processing apparatus 100, the MTF characteristics can be converted into those having an arbitrary MTF shape. For example, by using the MTF information obtained from actual shooting, it becomes possible to sharpen a natural image. Even if the target MTF is set in an arbitrary form that is optically impossible, the conversion itself is possible.

[0024] Also, for example, Japanese Patent Application Laid-Open No. 2008-067415 discloses a method of obtaining target frequency characteristics by interpreting the content described in preset target frequency characteristic information. Compared with this method, the image processing apparatus 100 focuses on converting the MTF itself, rather than performing predetermined filter processing or filter processing classified by MTF information for various conditions such as the angle of view. Since the image processing apparatus 100 can process any input MTF, can set an arbitrary target MTF, and is capable of MTF conversion, it is advantageous in terms of versatility. Also, the image processing apparatus 100 is advantageous in that the accuracy of approaching the target MTF is high.

[0025] In the example shown in FIG. 2, the input image Si is a color image. The conversion unit 3 converts a plurality of input channel images Ci(n) into a corresponding plurality of output channel images Co(n) based on the target MTF Mt, a plurality of target MTFs Ms(n) corresponding to the plurality of input channel images Ci(n) divided from the input image Si, and a plurality of input spectrum information Pi(n) corresponding to the plurality of input channel images Ci(n). The output unit 5 outputs, as the output image So, an image obtained by synthesizing the plurality of output channel images Co(n) converted by the conversion unit 3.

[0026] The image number n is a natural number that displays the image number indicating the channel image included in the input image Si. In this specification, in the description of the processing by the image processing apparatus 100, the image number n may be used as a counter. For example, when the input image Si includes three channel images, the plurality of input channel images Ci(n) include the input channel image Ci(1), the input channel image Ci(2), and the input channel image Ci(3). The plurality of target MTFs Ms(n) include the target MTF Ms(1), the target MTF Ms(2), and the target MTF Ms(3) corresponding to the plurality of input channel images Ci(n). The plurality of input spectrum information Pi(n) include the input spectrum information Pi(1), the input spectrum information Pi(2), and the input spectrum information Pi(3) corresponding to the plurality of input channel images Ci(n). The plurality of output channel images Co(n) include the output channel image Co(1), the output channel image Co(2), and the output channel image Co(3). However, n is not limited to 3 and can be appropriately changed according to the specifications of the input image Si and the like. In this specification, the target MTF Ms not including the subscript (n) represents the target MTF of one input image Si. Also, in this specification, the input spectrum information Pi not including the subscript (n) represents the input spectrum information of one input image Si.

[0027] For example, in an image device that handles color images, generally, a 3-channel image expressed in the RGB color space system or the XYZ color space system, or a 4-channel image expressed by R (Red), G (Green), B (Blue), and W (White), etc. is used to generate a color image. Also, in a spectroscopic camera, a color image is generated from channel images corresponding to a number of wavelengths. That is, in an image device that handles color images, a plurality of channel images are used. Since wavelength characteristics or spatial frequency characteristics, etc. are different among the plurality of channel images, there are differences in resolution characteristics. Also, there are differences in resolution characteristics not only between a plurality of channel images in the same image device, but also between different image devices or between a plurality of channels in different image devices. Therefore, just adjusting the MTF of one channel image for a color image may not enable the user to obtain an image having the desired resolution characteristics.

[0028] In the image processing apparatus 100 shown in FIG. 2, the conversion unit 3 adjusts the target MTF Ms(n) for each of the plurality of input channel images Ci(n) divided from the input image Si by the division unit 2, and converts the input channel image Ci(n) into the output channel image Co(n). Thereby, even if there are differences in resolution characteristics among the plurality of input channel images Ci(n), the plurality of input channel images Ci(n) can be accurately converted into the corresponding plurality of output channel images Co(n), and an image having the desired resolution characteristics by the user can be obtained. Note that the image processing apparatus 100 is not limited to a 3-channel image expressed in the RGB color space system or the XYZ color space system, or a 4-channel image expressed by RGBW, and an image having the desired resolution characteristics by the user can be obtained in a color image expressed in various formats. Note that the processing target of the image processing apparatus 100 is not limited to a color image, and may be a monochrome image.

[0029] In the image processing apparatus 100 shown in FIG. 2, one target MTF Ms selected from a plurality of target MTFs Ms(n) corresponding to a plurality of input channel images Ci(n) divided from the input image Si of the same image device can be set as the target MTF Mt. Thereby, among the plurality of input channel images Ci(n), adjustment can be performed so that the MTF of the desired input channel image Ci is matched with the MTFs of the other input channel images Ci(n). As a result, the difference in resolution characteristics among the plurality of input channel images Ci(n) can be reduced, and an image having the resolution characteristics desired by the user can be obtained.

[0030] Also, in the image processing apparatus 100 shown in FIG. 2, the MTFs of images obtained by different image devices can be set as the target MTF Mt. Thereby, the difference in resolution characteristics of images among different image devices can be reduced, and an image having the resolution characteristics desired by the user can be obtained.

[0031] In the image processing apparatus 100 shown in FIG. 2, the conversion unit 3 converts the input image Si into the output image So so that each of the plurality of target MTFs Ms(n) corresponding to the plurality of channel images Ci(n) approaches the target MTF Mt. Thereby, the image processing apparatus 100 can obtain an image having the resolution characteristics desired by the user using the target MTF Mt as an index.

[0032] In the image processing apparatus 100 shown in FIG. 2, the conversion unit 3 (0) By performing each of the setting of the initial amplification factor r[i] and the acquisition of the effective coefficient c, either amplification or attenuation is performed on the MTF components for each of the plurality of frequencies in each of the plurality of target MTFs Ms(n). Thereby, the conversion unit 3 can bring each of the plurality of target MTFs Ms(n) corresponding to the plurality of input channel images Ci(n) closer to the target MTF Mt. Note that the initial amplification factor r[i] (0)This refers to the initial value of the increase / decrease rate used to obtain the effective coefficient c for each frequency. The effective coefficient c refers to the value used to bring each of the target MTF Ms(n) closer to the target MTF Mt in the frequency space. i means the label of the quantized frequency.

[0033] In the image processing apparatus 100 shown in FIG. 2, the conversion unit 3 calculates the effective coefficient c such that the target MTF Ms(n) of the output channel image Co(n) obtained from the result of integrating the spectrum in the input spectrum information Pi(n) of the input channel image Si(n) and the candidate of the effective coefficient c approaches the target MTF Mt in the frequency space. Thereby, the conversion unit 3 can bring each of the plurality of target MTFs Ms(n) corresponding to the plurality of input channel images Ci(n) closer to the target MTF Mt.

[0034] In the image processing apparatus 100 shown in FIG. 2, after the conversion unit 3 calculates the effective coefficient c corresponding to each of the plurality of quantization frequencies obtained by quantizing the entire frequency range into equal intervals, the effective coefficient c corresponding to the frequency between adjacent quantization frequencies is obtained by interpolation calculation. Thereby, the amount of calculation can be reduced as compared with the case where the effective coefficient c is calculated for each Nyquist frequency over the entire frequency range. Note that the quantization interval is not limited to an equal interval, and MTF conversion is possible even with an interval other than an equal interval such as an unequal interval. Also, various methods can be applied to the quantization method.

[0035] In the example shown in FIG. 2, the image processing apparatus 100 further includes an input unit 1, a division unit 2, and a synthesis unit 4. The input unit 1 inputs an input image Si and a target MTF Mt from a device or apparatus other than the image processing apparatus 100 by controlling communication with the device or apparatus other than the image processing apparatus 100. The division unit 2 divides the input image Si input via the input unit 1 into a plurality of input channel images Ci(n). The conversion unit 3 converts the plurality of input channel images Ci(n) divided by the division unit 2 into corresponding output channel images Co(n). The synthesis unit 4 obtains an output image So obtained by synthesizing the plurality of output channel images Co(n) converted by the conversion unit 3. The output unit 5 outputs the output image So obtained by the synthesis unit 4 to a device or apparatus other than the image processing apparatus 100 by controlling communication with the device or apparatus other than the image processing apparatus 100.

[0036] In the examples shown in FIGS. 1 and 2, each function of the input unit 1 and the output unit 5 is realized by a communication I / F 105 or the like. Each function of the division unit 2, the conversion unit 3, and the synthesis unit 4 is realized by the CPU 101 executing processing defined by a program stored in the ROM 102 or the like. Note that a part of each function of the input unit 1 and the output unit 5 may also be realized by the CPU 101 executing processing defined by a program stored in the ROM 102 or the like.

[0037] A part of the above functions provided in the image processing apparatus 100 may be realized by a device or apparatus other than the image processing apparatus 100. Further, a part of the above functions provided in the image processing apparatus 100 may be realized by distributed processing between the image processing apparatus 100 and a device or apparatus other than the image processing apparatus 100 or the like. Furthermore, a part of the above functions provided in the image processing apparatus 100 may be realized by one or a plurality of processing circuits. Examples of this processing circuit include an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), etc. designed to execute the above respective functions.

[0038] (Functional Configuration of Conversion Unit 3) Next, the functional configuration of the conversion unit 3 shown in FIG. 2 will be described in detail. The conversion unit 3 shown in FIG. 2 includes a target MTF acquisition unit 31, a target MTF acquisition unit 32, an initial increase / decrease rate setting unit 33, an effective coefficient acquisition unit 34, a DFT unit 35, an MTF conversion unit 36, and an inverse DFT unit 37.

[0039] The target MTF acquisition unit 31 acquires the target MTF Mt input from a device or apparatus other than the image processing apparatus 100 via the input unit 1 and passes it to the effective coefficient acquisition unit 34. Note that the target MTF acquisition unit 31 does not necessarily have to acquire the target MTF Mt via the input unit 1. For example, the target MTF Mt may be stored in advance in a storage unit or the like provided in the image processing apparatus 100, and the target MTF acquisition unit 31 may acquire the target MTF Mt by referring to this storage unit. The function of the storage unit can be realized by the ROM 102 or the HDD 104 or the like. Further, the target MTF acquisition unit 31 can also acquire the MTF of one channel image Ci(n) selected from a plurality of channel images Ci(n) divided from the input image Si as the target MTF Mt. In addition, the target MTF acquisition unit 31 can also acquire the target MTF Mt from an external PC or an external server via the network and the input unit 1.

[0040] The target MTF acquisition unit 32 acquires the target MTF Ms(n) of each of the plurality of input channel images Ci(n) divided from the input image Si by the division unit 2 through calculation. For example, the target MTF acquisition unit 32 receives a plurality of channel images Ci(n) of the input image Si obtained by photographing a slanted edge defined in ISO 12233 from the division unit 2. The target MTF acquisition unit 32 acquires the target MTF Ms(n) corresponding to each of the plurality of channel images Ci(n) received from the division unit 2 through calculation. The target MTF acquisition unit 32 passes the acquired target MTF Ms(n) to the effective coefficient acquisition unit 34.

[0041] The initial increase / decrease rate setting unit 33 sets the initial increase / decrease rate Ir and sets the set initial increase / decrease rate r[i] (0)It is passed to the effective coefficient acquisition unit 34. The effective coefficient acquisition unit 34 acquires the effective coefficient c and passes the acquired effective coefficient c to the MTF conversion unit 36. The effective coefficient acquisition unit 34 also includes an interpolation unit 340. The interpolation unit 340 performs an interpolation operation to calculate the effective coefficient c corresponding to the frequencies between adjacent quantization frequencies after calculating the effective coefficient c corresponding to each of a plurality of quantization frequencies obtained by quantizing the entire frequency range into equal intervals. Note that the initial increase / decrease rate setting unit 33 and the effective coefficient acquisition unit 34 will be described in detail with reference to FIGS. 4 to 6 separately.

[0042] The DFT unit 35 performs a two-dimensional discrete Fourier transform on each of the plurality of input channel images Ci(n) divided by the division unit 2, and obtains a plurality of input spectrum information Pi(n) corresponding to the plurality of input channel images Ci(n) through calculation. The DFT unit 35 passes the acquired input spectrum information Pi(n) to each of the effective coefficient acquisition unit 34 and the MTF conversion unit 36.

[0043] The MTF conversion unit 36 obtains the output spectrum information Po(n) by integrating the spectrum in the input spectrum information Pi(n) of the input channel image Ci(n) and the candidate of the effective coefficient c in the frequency space. The MTF conversion unit 36 passes the output spectrum information Po(n) to the inverse DFT unit 37.

[0044] The inverse DFT unit 37 obtains the output channel image Co(n) by performing a two-dimensional inverse discrete Fourier transform on the output spectrum information Po(n) received from the MTF conversion unit 36. When the target MTF Ms(n) of the output channel image Co(n) does not approach the target MTF Mt, the inverse DFT unit 37 passes the acquired output channel image Co(n) to the target MTF acquisition unit 32 via the effective coefficient acquisition unit 34. When the target MTF Ms(n) of the output channel image Co(n) approaches the target MTF Mt, the inverse DFT unit 37 passes the output channel image Co(n) to the synthesis unit 4.

[0045] The conversion unit 3 can repeatedly perform the processing by the effective coefficient acquisition unit 34, the processing by the MTF conversion unit 36, and the processing by the inverse DFT unit 37 until the effective coefficient c is optimized and each of the target MTFs Ms(n) approaches the target MTF Mt.

[0046] <Conversion processing by the image processing apparatus 100> FIG. 3 is a flowchart showing an example of the conversion processing by the image processing apparatus 100. The conversion processing by the image processing apparatus 100 is processing for converting the input image Si into the output image So. The conversion processing is processing for converting into an image having resolution characteristics desired by the user, and is processing for improving the blur of the input image Si, for example. The image processing apparatus 100 starts the processing shown in FIG. 3 with, for example, the input of the input image Si via the input unit 1 as a start condition.

[0047] First, in step S11, the image processing apparatus 100 divides the input image Si input via the input unit 1 into a plurality of input channel images Ci(n) by the division unit 2. The division unit 2 passes the plurality of input channel images Ci(n) obtained by the division to the target MTF acquisition unit 32.

[0048] Subsequently, in step S12, the image processing apparatus 100 acquires the target MTF Mt input from a device or apparatus other than the image processing apparatus 100 via the input unit 1 by the target MTF acquisition unit 31, and passes it to the effective coefficient acquisition unit 34.

[0049] Subsequently, in step S13, the image processing apparatus 100 sets the image number n to 1.

[0050] Subsequently, in step S14, the image processing apparatus 100 acquires the target MTF Ms(n) of the n-th input channel image Ci(n) by calculation by the target MTF acquisition unit 32. The target MTF acquisition unit 32 passes the acquired target MTF Ms(n) to the effective coefficient acquisition unit 34.

[0051] Subsequently, in step S15, the image processing apparatus 100 sets the initial increase / decrease rate Ir by the initial increase / decrease rate setting unit 33, and passes the set initial increase / decrease rate Ir to the effective coefficient acquisition unit 34.

[0052] Subsequently, in step S16, the image processing apparatus 100 acquires the effective coefficient c by the effective coefficient acquisition unit 34, and passes the acquired effective coefficient c to the MTF conversion unit 36.

[0053] Subsequently, in step S17, the image processing apparatus 100 performs a two-dimensional discrete Fourier transform on the n-th input channel image Ci(n) by the DFT unit 35, thereby obtaining input spectrum information Pi(n) corresponding to the input channel image Ci(n) through calculation. The DFT unit 35 passes the acquired input spectrum information Pi(n) to each of the effective coefficient acquisition unit 34 and the MTF conversion unit 36.

[0054] Subsequently, in step S18, the image processing apparatus 100 obtains the output spectrum information Po(n) by integrating the spectrum in the input spectrum information Pi(n) of the n-th input channel image Ci(n) and the candidates of the effective coefficient c in the frequency space by the MTF conversion unit 36. The MTF conversion unit 36 passes the output spectrum information Po(n) to the inverse DFT unit 37.

[0055] Subsequently, in step S19, the image processing apparatus 100 obtains the output channel image Co(n) by performing a two-dimensional inverse discrete Fourier transform on the output spectrum information Po(n) received from the MTF conversion unit 36 by the inverse DFT unit 37. The inverse DFT unit 37 passes the acquired output channel image Co(n) to the target MTF acquisition unit 32 via the effective coefficient acquisition unit 34.

[0056] Subsequently, in step S20, the image processing apparatus 100 obtains the target MTF Ms(n) of the output channel image Co(n) received from the inverse DFT unit 37 through calculation by the target MTF acquisition unit 32. The target MTF acquisition unit 32 passes the acquired target MTF Ms(n) to the effective coefficient acquisition unit 34.

[0057] Subsequently, in step S21, the image processing apparatus 100 determines whether the target MTF Ms(n) has approached the target MTF Mt. For example, the image processing apparatus 100 can determine whether the target MTF Ms(n) has approached the target MTF Mt by determining whether the sum of the squares of the differences between the MTF components for each frequency of the target MTF Ms(n) and the target MTF Mt has become minimum.

[0058] In step S21, if it is determined that the target MTF Ms(n) has not approached the target MTF Mt (step S21, NO), the image processing apparatus 100 repeats the processing after step S16 until the target MTF Ms(n) approaches the target MTF Mt. On the other hand, in step S21, if it is determined that the target MTF Ms(n) has approached the target MTF Mt (step S21, YES), the image processing apparatus 100 adds 1 to the image number n in step S22. When it is determined that the target MTF Ms(n) has approached the target MTF Mt, the image processing apparatus 100 can acquire the output channel image Co(n) obtained by converting the input channel image Ci(n).

[0059] Subsequently, in step S23, the image processing apparatus 100 determines whether the image number n is equal to Nm + 1. Note that Nm represents the total number of input channel images.

[0060] In step S23, if it is determined that the image number n is not equal to Nm + 1 (step S23, NO), the image processing apparatus 100 repeats the processes after step S14 until the image number n becomes equal to Nm + 1. On the other hand, in step S23, if it is determined that the image number n is equal to Nm + 1 (step S23, YES), the image processing apparatus 100, in step S24, acquires an output image So obtained by synthesizing a plurality of output channel images Co(n) converted by the conversion unit 3 by the synthesis unit 4. When it is determined that the image number n is equal to Nm + 1, the image processing apparatus 100 can acquire Nm output channel images Co(n) corresponding to the Nm input channel images Ci(n). The synthesis unit 4 passes the acquired output image So to the output unit 5.

[0061] Subsequently, in step S25, the image processing apparatus 100 outputs the output image So to a device or equipment other than the image processing apparatus 100 by the output unit 5.

[0062] As described above, the image processing apparatus 100 can acquire and output the output image So converted from the input image Si.

[0063] <Process for the image processing apparatus 100 to make the target MTF approach the target MTF> Next, with reference to FIGS. 4 to 6, the process for the image processing apparatus 100 to make the target MTF approach the target MTF will be described in detail. FIG. 4 is a first diagram showing an example of quantization processing by the image processing apparatus 100. FIG. 5 is a second diagram showing an example of quantization processing by the image processing apparatus 100.

[0064] In the image processing apparatus 100, an effective coefficient matrix having the same size as the number of vertical and horizontal pixels of the input channel image Ci(n) is created, and the target MTF Ms(n) is made closer to the target MTF Mt by integrating the spectrum of the input channel image Ci(n) and the effective coefficient c in the frequency domain. In this specification, the frequency of the MTF is expressed in cpp (cycles per pixel). That is, the Nyquist frequency of the cpp of the image is 0.5 cpp. First, the frequencies in the range from 0 cpp to 0.5 cpp are quantized into Q points. Let the i-th frequency counted from the DC component (i = 1) be f[i] (cpp), and its MTF value be m[i]. Next, the rate of increase or decrease r[i] of the effective coefficient c with respect to f[i - 1] at f[i] is defined by the following equation (1).

[0065] [Number]

[0066] Here, c[i] is the effective coefficient corresponding to the frequency f[i]. Also, the effective coefficient c[i] is the ratio of m T [i] corresponding to the target MTF Mt to m S [i] corresponding to the target MTF Ms.

[0067] [Number]

[0068] m T [i] and m S [i] are the same, the effective coefficient c[i] is 1. When c[i]>1, it is necessary to amplify the target MTF Ms(n). When c[i]<1, it is necessary to attenuate the target MTF Ms(n). The rate of increase or decrease r[i] means the rate of increase or decrease between the effective coefficient c[i - 1] and the effective coefficient c[i] at one frequency f[i - 1]. It is defined that r[1](0)=1.

[0069] (Quantization processing) In the acquisition process of the effective coefficient c[i] by the effective coefficient acquisition unit 34 in the image processing apparatus 100, the effective coefficient c[i] corresponding to each of a plurality of quantization frequencies obtained by equally quantizing the entire frequency range is calculated. Then, the image processing apparatus 100 obtains, by means of the interpolation unit 340, the effective coefficient c[i] corresponding to the frequency between adjacent quantization frequencies through interpolation calculation. Thereby, the computational amount in the acquisition process of the effective coefficient c[i] can be reduced.

[0070] In the quantization process by the image processing apparatus 100, it is assumed that the MTF decreases according to the sinc function because the pixels of the image are composed of rectangular apertures. As shown in FIG. 4, quantization is performed in equal intervals with respect to the total sum of the change amounts based on the absolute average value of the MTF change amount obtained from the 3rd to 30th power differentials of the sinc function of the pixels. In the examples shown in FIGS. 4 and 5, quantization is performed with Q = 21 (5% step). In this way, by quantizing so that the total sum of the change amounts is equal, the frequency band where the MTF changes rapidly can be preferentially quantized, and the target MTF Ms(n) can be efficiently approximated to the target MTF Mt even with a small number of quantization levels.

[0071] The quantization points of the frequencies for obtaining the increase / decrease rate r[i] are set to 21 points of f[i] = {0, 0.037, 0.056, 0.072, 0.085, 0.098, 0.110, 0.122, 0.134, 0.146, 0.158, 0.172, 0.185, 0.200, 0.217, 0.236, 0.258, 0.286, 0.323, 0.380, 0.500} (i = 1, 2, ···, 21) with respect to the interval [0, 0.5] (cpp) up to the Nyquist frequency. This is fixed regardless of the size of the input image Si and the size of the effective coefficient matrix.

[0072] (Setting of the initial increase / decrease rate and calculation of the effective coefficient) In the image processing apparatus 100, the effective coefficient c[i] at the frequency f[i] is calculated from the effective coefficient c[i - 1] and the increase / decrease rate r[i] according to the above formula (1). Therefore, it is necessary to estimate the increase / decrease rate r[i] at the frequency f[i]. In the image processing apparatus 100, r[i] is obtained by numerically iterative optimization. First, the initial increase / decrease rate r[i] (0) of r[i] is set by the following formula (3) as the ratio of the target MTF Mt to the target MTF Ms at adjacent quantization points. In formula (3), m T [i] and m T [i - 1] respectively correspond to the target MTF Mt, and m S [i] respectively corresponds to the target MTF Ms.

[0073]

Number

[0074] Next, in order to make the target MTF Ms approach the target MTF Mt, the increase / decrease rate r[i] is numerically optimized by sequential processing from low frequency to high frequency so that the MSE (Mean Squared Error) between the target MTF Mt and the target MTF Ms becomes small. The details of the MSE calculation method will be described later. In the optimization process, at adjacent quantization points, search candidates for the increase / decrease rate are obtained respectively, and the pair of increase / decrease rates that optimizes the MSE is calculated by all possible pairs of them. At frequencies where the relationship before and after a specific frequency increases or decreases significantly, the search change width becomes large. As a result, the amplitude width is flexibly determined according to the degree of change.

[0075] The effective coefficient c at frequencies other than the frequency f[i] can be obtained by linear interpolation from the obtained group of effective coefficients. Thereby, an effective coefficient matrix over the entire frequency range can be obtained. After performing a process of making the target MTF Ms approach the target MTF Mt in the frequency space using the created effective coefficient matrix, an inverse discrete Fourier transform of two dimensions is performed, and the real part is taken out. This image becomes the output channel image Co(n).

[0076] (Method for calculating MSE) As described above, the determination of the increase / decrease rate r[i] starts from the frequency f[2] and is determined in order from the low-frequency side to the high-frequency side. At this time, for all combinations of the increase / decrease rate r[i] of all search candidates from the frequency f[i] to the frequency f[i + 1], the MSE is calculated by the following formula (4). In formula (4), m T [f] corresponds to the target MTF Mt, and m S [f] corresponds to the target MTF Ms.

[0077]

Equation

[0078] Specifically, according to the relationship of the above formula (1), the effective coefficient c[i] is calculated from the increase / decrease rate r[i] and the effective coefficient c[i - 1], and the effective coefficient c[i + 1] is calculated from the increase / decrease rate r[i + 1] and the effective coefficient c[i]. From the effective coefficient c[i] to the effective coefficient c[i - 1], the optimized effective coefficient is used, and from c[i + 2] to c[N] which have not been optimized yet, the effective coefficient c calculated from the initial increase / decrease rate r[i] (0) is used respectively, and linear interpolation is performed at intervals of 0.001 cpp between adjacent effective coefficients c. In this way, the effective coefficient c can be obtained at intervals of 0.001 cpp. The reason for setting the interval to 0.001 cpp is that even if the MSE is calculated at intervals finer than 0.001 cpp, there is no significant change in the value and the value is stable.

[0079] Using the obtained effective coefficient c, in the frequency space, the spectrum of the input channel image Ci(n) and the effective coefficient c are integrated to obtain the output spectrum information Po(n). The target MTF Ms is calculated again from the output channel image Co(n) obtained by performing two-dimensional inverse discrete Fourier transform on the obtained output spectrum information Po(n). For the obtained target MTF Ms, in the interval of frequencies f[1] to f[N], the MSE between the target MTF Mt and the target MTF Ms is calculated by formula (4). The combination of the increase / decrease rate r[i] that minimizes this MSE is obtained and used as the effective coefficient between the frequencies f[i] and f[i + 1].

[0080] Similarly, the effective coefficient c from frequency f[i + 1] to frequency f[i + 3] is calculated at intervals of 0.001 cpp. By repeating this operation up to frequency f[N], the effective coefficient c can be calculated at intervals of 0.001 cpp in the range of 0 cpp to 0.5 cpp. Note that the rate of increase or decrease r[i] optimized at f[i] (i = 2, 3, ··· N) (t) is used as the initial value, and the optimal rate of increase or decrease r[i] (t+1) is repeatedly updated, and by obtaining the effective coefficient c[i] (t+1) the accuracy can be improved. At this time, by narrowing the search range as the number of iterations t increases, it is possible to efficiently converge to the optimal target MTF Ms.

[0081] <Processing Results by Image Processing Apparatus 100> Next, the processing results by the image processing apparatus 100 will be described.

[0082] (First Example) In the first example, among the three channel images in the XYZ colorimetric system, the MTF of the Y channel image was set as the target MTF Mt. FIGS. 6 to 11 show the results of performing processing to improve the blurring of the image so that the target MTF Ms becomes higher for each of the three input channels Ci(n) included in the color captured image, which was captured by the imaging device through the optical system for the object 70. For the three input channels Ci(n), three channel images of X, Y, and Z in the XYZ colorimetric system were used.

[0083] FIG. 6 is a diagram showing the first example of the input image Si. FIG. 7 is a diagram showing the target MTF Ms obtained from the input image Si of FIG. 6. FIG. 8 is a diagram showing the output image SoX converted from the input image Si of FIG. 6 by the image processing apparatus according to the comparative example. FIG. 9 is a diagram showing the target MTF obtained from the output image of FIG. 8. FIG. 10 is a diagram showing the first example of the output image So converted from the input image Si of FIG. 6 by the image processing apparatus 100. FIG. 11 is a diagram showing the target MTF Ms obtained from the output image So of FIG. 10.

[0084] In the image processing apparatus according to the comparative example shown in FIG. 8, processing using an unsharp masking filter widely used in the edge enhancement processing of an image was performed. In the processing other than the filter processing by the image processing apparatus according to the comparative example, the processing of bringing the target MTF Ms closer to the target MTF Mt was performed in the same manner as the processing by the image processing apparatus 100. The processing by the image processing apparatus according to this comparative example is the same also in the second example shown hereinafter.

[0085] Each of the enlarged image 71 in FIG. 6, the enlarged image 72 in FIG. 8, and the enlarged image 73 in FIG. 10 is an image obtained by enlarging a part of the edge region of the object 70.

[0086] The graph Xi in FIG. 7 shows the graph of the MTF corresponding to X in the XYZ colorimetric system in the input image Si. The graph Yi in FIG. 7 shows the graph of the MTF corresponding to Y in the XYZ colorimetric system in the input image Si. The graph Zi in FIG. 7 shows the graph of the MTF corresponding to Z in the XYZ colorimetric system in the input image Si. The graph Xx in FIG. 9 shows the graph of the MTF corresponding to X in the XYZ colorimetric system in the output image according to the comparative example. The graph Yx in FIG. 9 shows the graph of the MTF corresponding to Y in the XYZ colorimetric system in the output image according to the comparative example. The graph Zx in FIG. 9 shows the graph of the MTF corresponding to Z in the XYZ colorimetric system in the output image according to the comparative example. The graph Xo in FIG. 11 shows the graph of the MTF corresponding to X in the XYZ colorimetric system in the output image So. The graph Yo in FIG. 11 shows the graph of the MTF corresponding to Y in the XYZ colorimetric system in the output image So. The graph Zo in FIG. 11 shows the graph of the MTF corresponding to Z in the XYZ colorimetric system in the output image So.

[0087] As shown in FIG. 7, in the input image Si, a difference in MTF occurred among X, Y, and Z. According to this difference in MTF, in the input image Si shown in FIG. 6, false colors occurred in the edge region of the object 70 as shown in the enlarged image 71. This false color is an artifact that occurs due to the wavelength dependence of the resolution characteristics such as chromatic aberration of the optical system and does not exist in the actual object.

[0088] In the processing results according to the comparative example shown in FIGS. 8 and 9, the sharpening process is performed without considering the optical phenomenon. Therefore, as shown in FIG. 9, the MTF in the low-frequency region unnaturally bulges. When this bulge becomes excessive, in the output image SoX shown in FIG. 8, as shown in the enlarged image 72, artifacts of unnatural contour enhancement including false colors occur in the edge region of the object 70.

[0089] In the processing results by the image processing apparatus 100 shown in FIGS. 10 and 11, artifacts of unnatural contour enhancement do not occur and false colors are also reduced. False colors, artifacts of unnatural contour enhancement, etc. in the image are not the resolution characteristics desired by the user. Therefore, it has been found that the image processing apparatus 100 can acquire an image having the resolution characteristics desired by the user.

[0090] (Second Example) In the second example, the point where the MTF of the images captured by different imaging devices is set as the target MTF Mt is different from that in the first example. FIGS. 12 to 17 show the results of performing a process to improve the blurring of an image so that the target MTF Ms becomes high for each of the three input channels Ci(n) included in the color captured image by imaging the object 70 through the optical system by the imaging device. For the three input channels Ci(n), three channel images of X, Y, and Z in the XYZ color system are used.

[0091] FIG. 12 is a diagram showing a second example of the input image Si. FIG. 13 is a diagram showing the target MTF Ms obtained from the input image Si of FIG. 12. FIG. 14 is a diagram showing the output image SoX converted from the input image Si of FIG. 12 by the image processing apparatus according to the comparative example. FIG. 15 is a diagram showing the target MTF obtained from the output image of FIG. 14. FIG. 16 is a diagram showing a second example of the output image So converted from the input image Si of FIG. 12 by the image processing apparatus 100. FIG. 17 is a diagram showing the target MTF Ms obtained from the output image So of FIG. 16.

[0092] Each of the enlarged images 74 in FIG. 12, the enlarged image 75 in FIG. 14, and the enlarged image 76 in FIG. 16 is an image obtained by enlarging a part of the edge region of the object 70.

[0093] Graphs Tg in FIGS. 13, 15, and 17 respectively are the MTFs of images captured by different imaging devices and represent the target MTF Mt. The graph Xi in FIG. 13 shows the graph of the MTF corresponding to X in the XYZ colorimetric system in the input image Si. The graph Yi in FIG. 13 shows the graph of the MTF corresponding to Y in the XYZ colorimetric system in the input image Si. The graph Zi in FIG. 13 shows the graph of the MTF corresponding to Z in the XYZ colorimetric system in the input image Si. The graph Xx in FIG. 15 shows the graph of the MTF corresponding to X in the XYZ colorimetric system in the output image according to the comparative example. The graph Yx in FIG. 15 shows the graph of the MTF corresponding to Y in the XYZ colorimetric system in the output image according to the comparative example. The graph Zx in FIG. 15 shows the graph of the MTF corresponding to Z in the XYZ colorimetric system in the output image according to the comparative example. The graph Xo in FIG. 17 shows the graph of the MTF corresponding to X in the XYZ colorimetric system in the output image So. The graph Yo in FIG. 17 shows the graph of the MTF corresponding to Y in the XYZ colorimetric system in the output image So. The graph Zo in FIG. 17 shows the graph of the MTF corresponding to Z in the XYZ colorimetric system in the output image So. In FIG. 17, the graph Xo and the graph Yo almost overlap.

[0094] As shown in FIG. 13, in the input image Si, differences in MTF occurred among X, Y, and Z, and the MTF of each of X, Y, and Z differed from the target MTF Mt. According to the differences in MTF among X, Y, and Z, in the input image Si shown in FIG. 3, false colors occurred in the edge region of the object 70 as shown in the enlarged image 74. This false color is an artifact that results from the wavelength dependence of resolution characteristics such as chromatic aberration of the optical system and does not exist in the actual object.

[0095] In the processing result according to the comparative example, as shown in FIG. 15, compared with the MTF of the input image Si, the graph Xx, the graph Yx, and the graph Zx each approached the target MTF Mt. However, each of the graph Xx, the graph Yx, and the graph Zx had a difference from the target MTF Mt in the low-frequency region and the high-frequency region.

[0096] In the processing result by the image processing apparatus 100, as shown in FIG. 17, the graph Xx, the graph Yx, and the graph Zx each approached to such an extent that they substantially coincided with the target MTF Mt. As a result, as shown in FIG. 16, in the enlarged image 76 of the edge region of the object 70, the edge region generated a natural appearance. From the above, it was found that the image processing apparatus 100 can acquire an image having the resolution characteristics desired by the user.

[0097] [Second Embodiment] Next, the display device according to the second embodiment will be described. Note that the same names and reference numerals as those in the already described embodiments indicate the same or similar members or configurations, and the detailed description will be omitted as appropriate. This also applies to the embodiments shown hereinafter.

[0098] FIG. 18 is a block diagram showing an example of a display device 200 according to the second embodiment. As shown in FIG. 18, the display device 200 includes an image processing apparatus 100 and a display unit 250. The image processing apparatus 100 and the display unit 250 are connected to be communicable with each other by wire or wirelessly.

[0099] The display device 200 inputs the input image Si and the target MTF Mt to the image processing apparatus 100 from a device or apparatus other than the display device 200. Note that the device other than the display device 200 is a imaging device or a display device other than the display device 200, etc. The apparatus other than the display device 200 is an external PC or an external server, etc. The display device 200 passes the output image So obtained by converting the input image Si by the image processing apparatus 100 to the display unit 250. The display device 200 can display the output image So by the display unit 250.

[0100] As the display unit 250, a liquid crystal display, an organic EL (Electro Luminescence) display, a projector, or the like can be used. When the projector is used as the display unit 250, the display unit 250 can project the output image So onto the projection surface.

[0101] The display device 200 can display an image having the resolution characteristics desired by the user by displaying the output image So converted from the input image Si by the image processing device 100 on the display unit 250.

[0102] [Third Embodiment] Next, the imaging device according to the third embodiment will be described.

[0103] FIG. 19 is a block diagram showing an example of an imaging device 300 according to the third embodiment. As shown in FIG. 19, the imaging device 300 includes an imaging unit 350 and an image processing device 100. The imaging unit 350 and the image processing device 100 are communicably connected to each other by wire or wirelessly.

[0104] The imaging device 300 passes the input image Si captured by the imaging unit 350 to the image processing device 100. The imaging device 300 also inputs the target MTF Mt to the image processing device 100 from a device or apparatus other than the imaging device 300. Note that the device other than the imaging device 300 is a display device or an imaging device other than the imaging device 300, etc. The apparatus other than the imaging device 300 is an external PC or an external server, etc. The imaging device 300 can output the output image So obtained by converting the input image Si by the image processing device 100 to a device or apparatus other than the imaging device 300. In this case, the device other than the imaging device 300 is a display device or an imaging device other than the imaging device 300, etc. Also, the apparatus other than the imaging device 300 is an external PC, an external server, or an external storage device, etc.

[0105] As the imaging unit 350, a digital camera, a camera for mobile terminals such as a smartphone, a web camera, a video camera, or the like can be used. Also, for a multi-channel imaging unit 350, a multi-band camera or the like can be used.

[0106] The imaging device 300 can obtain an image having resolution characteristics desired by the user by converting the input image Si captured by the imaging unit 350 into the output image So by the image processing device 100.

[0107] Although the preferred embodiments have been described in detail above, the present invention is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments without departing from the scope described in the claims.

[0108] In the image processing device, image device, image processing method, and program according to the present disclosure, as long as the image device is the same, the effective coefficient calculated once can be used without calculating the effective coefficient every time from the input image. Also, the effective coefficient calculated once can be commonly used for any input image. From another perspective, in the image processing device, image device, image processing method, and program according to the present disclosure, the calculated effective coefficient can be made to function as an image processing filter corresponding to the effective coefficient.

[0109] The numbers such as ordinal numbers and quantities used in the description of the embodiments are all exemplified for specifically explaining the technology of the present disclosure, and the present disclosure is not limited to the exemplified numbers. Also, the connection relationship between the components is exemplified for specifically explaining the technology of the present disclosure, and the present disclosure is not limited to this connection relationship for realizing the functions of the present disclosure.

[0110] The division of the blocks in the functional block diagram is an example, and a plurality of blocks may be realized as one block, one block may be divided into a plurality, or some functions may be transferred to other blocks. Also, the functions of a plurality of blocks having similar functions may be processed by a single hardware or software in parallel or time-division. Also, some or all of the functions may be distributed among a plurality of computers.

[0111] The image processing apparatus, image device, image processing method, and program according to the present disclosure can obtain an image having resolution characteristics desired by a user of an image device or the like. Then, it is possible to reduce the influence of differences in resolution characteristics that occur between different channel images in one image device or between images of different image devices, and to acquire or generate a sharp image. For this reason, the image processing apparatus, image device, image processing method, and program according to the present disclosure can be suitably used for applications of image devices including display devices or imaging devices. Examples of display devices include liquid crystal displays, organic EL displays, projectors, and the like. Examples of imaging devices include digital cameras, video cameras, and the like. However, the applications of the image processing apparatus, image device, image processing method, and program are not limited to applications of display devices or imaging devices, and can be widely used for applications using images or applications using optical techniques.

[0112] In addition, since the image processing apparatus, image device, image processing method, and program according to the present disclosure can solve various problems of resolution characteristics, they can contribute to reducing the prototyping cost of image devices by applying them to image simulation technology in image device design, or improving the overall image reproduction technology between image devices.

[0113] Aspects of the present disclosure are as follows, for example. <1> An image processing apparatus having a conversion unit that converts an input image into an output image based on a target MTF, a target MTF acquired from the input image, and input spectrum information acquired from the input image, and an output unit that outputs the output image converted by the conversion unit. <2> The input image is a color image, and the conversion unit converts the plurality of input channel images into a corresponding plurality of output channel images based on the target MTF, a plurality of target MTFs corresponding to the plurality of input channel images divided from the input image, and a plurality of input spectrum information corresponding to the plurality of input channel images, and the output unit outputs, as the output image, an image obtained by synthesizing the plurality of output channel images converted by the conversion unit. The image processing apparatus according to <1>. <3> The image processing apparatus according to <2>, wherein the target MTF is one of the plurality of target MTFs corresponding to the plurality of input channel images divided from the input image of the same image device, and is selected therefrom. <4> The image processing apparatus according to <2> or <3>, wherein the target MTF is the MTF of an image obtained by a different image device. <5> The image processing apparatus according to any one of <1> to <4>, wherein the conversion unit converts the input image into the output image so as to bring the target MTF closer to the target MTF. <6> The image processing apparatus according to <5>, wherein the conversion unit amplifies or attenuates either one of the MTF components for each of a plurality of frequencies in the target MTF by performing each of setting an initial increase / decrease rate and obtaining an effective coefficient. <7> The image processing apparatus according to <6>, wherein the conversion unit calculates the effective coefficient so that the target MTF of the output image obtained from the result of integrating the spectrum in the input spectrum information of the input image and the candidate of the effective coefficient in the frequency space approaches the target MTF. <8> The image processing apparatus according to <6> or <7>, wherein the conversion unit calculates the effective coefficient corresponding to each of a plurality of quantization frequencies obtained by quantizing the entire frequency range into equal intervals, and then obtains the effective coefficient corresponding to the frequency between adjacent quantization frequencies by interpolation calculation. <9> An image device having the image processing apparatus according to any one of <1> to <8>. <10> An image processing method by an image processing apparatus, wherein the image processing apparatus converts the input image into an output image based on a target MTF, a target MTF obtained from the input image, and input spectrum information obtained from the input image by a conversion unit, and outputs the output image converted by the conversion unit by an output unit. <11> A program that causes an image processing apparatus to execute a process of converting an input image into an output image based on a target MTF, a target MTF obtained from the input image, and input spectrum information obtained from the input image, and outputting the output image converted by the conversion unit by an output unit.

Explanation of Signs

[0114] 1 Input unit 2 Division unit 3 Conversion unit 31 Target MTF acquisition unit 32 Target MTF acquisition unit 33 Initial increase / decrease rate setting unit 34 Effective coefficient acquisition unit 340 Interpolation unit 35 DFT unit 36 MTF conversion unit 37 Inverse DFT unit 4 Composition unit 5 Output unit 70 Object 71 - 76 Enlarged images 100 Image processing apparatus 101 CPU 102 ROM 103 RAM 104 HDD 105 Communication I / F B System bus c Effective coefficient Ci(n) Input channel image Co(n) Output channel image Mt Target MTF Ms, Ms(n) Target MTF Pi(n) Input spectrum information Po(n) Output spectrum information r[i] (0) Initial increase / decrease rate Si Input image So Output image Xi, Yi, Zi, Xo, Yo, Zo, Tg Graph

Claims

1. A conversion unit that converts the input image into an output image based on a target MTF, a target MTF obtained from the input image, and input spectrum information obtained from the input image; An output unit that outputs the output image converted by the conversion unit. An image processing apparatus.

2. The input image is a color image, The conversion unit converts the plurality of input channel images into a corresponding plurality of output channel images based on the target MTF, a plurality of target MTFs corresponding to the plurality of input channel images divided from the input image, and a plurality of input spectrum information corresponding to the plurality of input channel images. The output unit outputs, as the output image, an image obtained by synthesizing the plurality of output channel images converted by the conversion unit. The image processing apparatus according to claim 1.

3. The target MTF is one of the plurality of target MTFs selected from among the plurality of target MTFs corresponding to the plurality of input channel images divided from the input image of the same image device. The image processing apparatus according to claim 2.

4. The target MTF is an MTF of an image obtained by different image devices. The image processing apparatus according to claim 2.

5. The conversion unit converts the input image into the output image so as to bring the target MTF closer to the target MTF. The image processing apparatus according to claim 1.

6. The conversion unit amplifies or attenuates either one of the MTF components for each of a plurality of frequencies in the target MTF by performing each of setting an initial increase / decrease rate and obtaining an effective coefficient. The image processing apparatus according to claim 5.

7. In the frequency space, the conversion unit calculates the effective coefficient so that the target MTF of the output image obtained from the result of integrating the spectrum in the input spectrum information of the input image and the candidate of the effective coefficient approaches the target MTF. The image processing apparatus according to claim 6.

8. After calculating the effective coefficient corresponding to each of the plurality of quantization frequencies obtained by quantizing the entire frequency range into equal intervals, the conversion unit obtains the effective coefficient corresponding to the frequency between adjacent quantization frequencies by interpolation calculation. The image processing apparatus according to claim 6.

9. An image device having the image processing apparatus according to any one of claims 1 to 8.

10. An image processing method by an image processing apparatus, wherein the image processing apparatus converts the input image into an output image based on a target MTF, a target MTF obtained from the input image, and input spectrum information obtained from the input image by a conversion unit; An image processing method for outputting the output image converted by the conversion unit by an output unit. **Claim 11** converts the input image into an output image based on a target MTF, a target MTF obtained from the input image, and input spectrum information obtained from the input image by a conversion unit; outputs the output image converted by the conversion unit by an output unit; A program for causing an image processing apparatus to execute the process.

Images