Image processing apparatus, imaging apparatus, image processing method, and program

Abstract

To provide an image processing apparatus capable of determining optical correction information for accurately correcting images degraded by an optical system.SOLUTION: An image processing apparatus (500) that corrects an image degraded by an optical system (100) has subject determination means (112) for identifying a subject in the image, distance acquisition means (111) for obtaining subject distance information indicating the distance to the subject, position acquisition means (107) for obtaining lens position information indicating the position of a lens group in the optical system, and processing means (104) for determining optical correction information for correcting image degradation using the subject distance information and the lens position information.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an imaging device, an image processing method, and a program. [Background technology]

[0002] Conventionally, a method is known in which a subject distance is estimated using lens position information of an optical system, and an image degraded by the optical system is corrected based on the estimated subject distance. Patent Document 1 discloses an imaging device that corrects blur in a captured image based on the imaging characteristics of the optical system corresponding to the subject distance. Patent Document 2 discloses an image processing device that corrects degradation of a captured image caused by an optical lens system using a restoration filter according to the distance from the imaging device to a main subject area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-49773 [Patent Document 2] Japanese Patent Application Laid-Open No. 2011-44825 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the actual subject distance changes depending on back focus deviation due to manufacturing errors and environmental changes, the presence or absence of an optical filter, temperature changes, etc. Therefore, when the subject distance estimated from lens position information is used, it may not be possible to perform the blur correction disclosed in Patent Document 1 and the captured image correction disclosed in Patent Document 2 with high accuracy.

[0005] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an image processing device capable of determining optical correction information for correcting, with high precision, an image degraded by an optical system. [Means for solving the problem]

[0006] An image processing device according to one aspect of the present invention is an image processing device that corrects an image degraded by an optical system, and includes a determination means for determining a subject in the image, a distance acquisition means for acquiring subject distance information relating to the distance to the subject, a position acquisition means for acquiring lens position information relating to the position of a lens group in the optical system, and a processing means for determining optical correction information for correcting the image using the subject distance information and the lens position information.

[0007] Other objects and features of the present invention will be described in the following embodiments. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide an image processing device capable of determining optical correction information for correcting, with high precision, an image degraded by an optical system. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a cross-sectional view of the imaging optical system according to the present embodiment. [Figure 2] 3A to 3C are aberration diagrams of the imaging optical system according to the present embodiment. [Figure 3] FIG. 2 is a diagram showing a point image of the imaging optical system in the present embodiment. [Figure 4] FIG. 4 is a diagram showing a distortion map of the imaging optical system in the present embodiment. [Figure 5] FIG. 1 is a block diagram of an imaging apparatus according to an embodiment of the present invention. [Figure 6] 5 is a flowchart showing processing by an image processing unit in the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] First, an optical system (image pickup optical system) 100 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a cross-sectional view of the optical system 100. Note that the optical system 100 shown in Fig. 1 is an example of an image pickup optical system for acquiring an image, and various quantities of the optical system 100 will be described later as numerical examples.

[0012] The optical system 100 has a diaphragm (aperture stop) SP and multiple lens groups including a focus group (lens group) 101. In the optical system 100, the focus group 101 is the eighth optical element (eighth lens) counting from the object side, including the parallel plate. Note that in this embodiment, the configuration of the focus group 101 is not limited to this, and various focusing methods such as front focusing, rear focusing, full extension, and floating can be used. In this embodiment, when focusing from infinity to a close distance, the focus group 101 extends toward the object side. An image plane IP represents the paraxial image plane of the optical system 100, and the image sensor 102 (see FIG. 5) of the imaging device 500 is disposed thereon. However, in reality, the position of the image sensor 102 and the position of the paraxial image plane do not necessarily coincide.

[0013] Next, aberrations of the optical system 100 will be described with reference to FIGS. 2(a) to 2(c), 3(a) and 3(b), and 4(a) and 4(b). FIGS. 2(a) to 2(c) are aberration diagrams of the optical system 100. FIG. 2(a) shows an aberration diagram for an object distance of ∞. Aberrations change during focusing. FIG. 2(b) shows an aberration diagram for an object distance of −270 mm. The object distance is the distance from the image plane IP. The amount of movement of the focus group 101 to focus at an object distance of −270 mm is −0.33 mm. As described above, aberrations change during focusing. However, a method of performing image processing by applying an aberration correction value or an image restoration filter according to the object distance or the corresponding amount of movement of the focus group 101 is already known.

[0014] However, the length of the back focus, i.e., the distance from the final surface of the lenses constituting the optical system 100 (the lens surface of the optical system 100 on the image side) to the image sensor, varies due to various factors. For example, the back focus changes due to manufacturing errors in the mounts of the image pickup device and lens device in an interchangeable lens camera system, and sliding margins required to enable lens replacement. Furthermore, even in cameras that do not have interchangeable lenses, lens back focus shifts occur due to changes in environmental temperature.

[0015] Figure 2(b) shows an aberration diagram when the image sensor is shifted -0.1 mm from the paraxial image plane due to these factors. At this time, the object distance is ∞. Even though the object distance is ∞, the image needs to be formed closer to the object than the paraxial image plane, so the focus group 101 needs to be extended. Figure 2(b) shows an aberration diagram when the focus group 101 is moved -0.33 mm, corresponding to a back focus shift of -0.1 mm.

[0016] Generally, an imaging device equipped with a lens device having a focus group (or an imaging device equipped with a focus group) estimates the subject distance from the movement amount of the focus group. Therefore, as shown in the aberration diagram of FIG. 2(b), the imaging device determines that the subject distance is −270 mm from the movement amount of the focus group of −0.33 mm required to absorb the back focus shift of −0.1 mm by moving the focus group. The imaging device then selects optical correction information, such as an aberration correction value or an image restoration filter, that corresponds to the subject distance determined from the movement amount of the focus group.

[0017] However, as can be seen from the diagrams of point images of the optical system 100 at the d-line shown in Figures 3(a) and 3(b), the point images in Figure 2(b) and Figure 2(c) are different from each other. That is, even if the movement amount of the focus group 101 is the same, the point image changes if the subject distance is different. Note that Figures 3(a) and 3(b) show the point images at the d-line at image heights of 0 mm and 7 mm on a scale of ±0.05 mm.

[0018] Figures 4(a) and (b) show distortion maps of optical system 100 at the d-line. Figures 4(a) and (b) show distortion maps corresponding to Figures 2(a) and (b), respectively. Note that a distortion map is a diagram showing how a square lattice image on the object side changes due to distortion on the image side. The lower parts of Figures 4(a) and (b) show distortion maps in which the distortion is enhanced by 10 times. As shown, changes in subject distance not only change the point image, but also the distortion and the chromatic aberration of magnification that cause distortion in each RGB channel.

[0019] In this way, in the method of estimating the subject distance from the movement amount (or position information) of the focus group 101, the optical correction information such as the aberration correction amount or the image restoration filter contains errors due to back focus shift, so it is not possible to obtain highly accurate optical correction information.

[0020] Next, an imaging device 500 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram of the imaging device 500. The imaging device 500 is configured to include a camera body and a lens device (optical system 100) that is detachable from the camera body. However, this embodiment is not limited to this, and the optical system 100 and the camera body may be configured as an integrated unit.

[0021] The imaging device 500 is an image processing device that corrects an image (captured image) that has been degraded by the optical system (image capturing optical system) 100. Note that the imaging device 500 is a camera having one optical system 100 with multiple lenses arranged along an optical axis OA, and is different from a stereo camera having multiple optical systems arranged in parallel with each other.

[0022] The image sensor 102 is an image sensor (avalanche photodiode sensor) having an avalanche photodiode (APD), and is preferably a single-photon avalanche diode (SPAD) sensor. However, the present embodiment is not limited to this, and may be a complementary metal-oxide-semiconductor (CMOS) image sensor or a charge-coupled device (CCD) image sensor. The image sensor 102 photoelectrically converts an optical image formed by the imaging optical system 100 to obtain an electrical signal (analog signal). The electrical signal obtained by the image sensor 102 is converted into a digital signal and input to the image processing unit 104.

[0023] The system controller 110 is a control unit having a CPU, a ROM, a RAM, etc., and controls the imaging device 500 by executing a computer program stored in the ROM.

[0024] The image processing unit 104 acquires information about the state (image capturing state) of the image capturing device 500 from the state detection unit 107. The state detection unit 107 may acquire information about the image capturing state of the image capturing device 500 from the system controller 110, or may acquire the information from the optical system control unit 106.

[0025] Here, the information about the imaging state includes information about the type of optical system 100, the drive amount of the focus group, the position of the focus group, the drive amount of the zoom group, the position of the zoom group, the drive amount of the compensator group associated with zooming, the position of the compensator group, or the drive amount of the aperture stop SP. In particular, in this embodiment, the state detection unit 107 functions as a position acquisition unit that acquires lens position information about the position of the lens group (focus group 101) in the optical system 100. The information about the imaging state may also include a value (temperature information) from a temperature sensor (temperature detection unit) 113 built into the optical system 100, and information about a ROM (storage unit) built into the optical system 100. Note that in this embodiment, the information about the imaging state may include at least one of the above pieces of information, or may also include other information.

[0026] The distance acquisition unit (distance acquisition means) 111 has an avalanche photodiode sensor (light receiving means) such as a SPAD sensor, and acquires subject distance information relating to the distance to the subject (absolute distance). The subject distance information is the distance from the distance acquisition unit 111 (or the image sensor 102) to the subject, or the distance from the optical system 100 to the subject. In this embodiment, the function of the distance acquisition unit 111 and the function of the image sensor 102 may be realized by the same element (the distance acquisition unit 111 and the image sensor 102 may be shared). In other words, the image sensor 102 may be configured to realize the function of the distance acquisition unit 111.

[0027] In this embodiment, subject distance information regarding the distance to the subject is information obtained by actual measurement using the distance obtaining unit 111, and is different from information estimated from lens position information, such as the focus position and zoom position, of the optical system 100 obtained from the optical system control unit 106. The distance obtaining unit 111 may obtain distance information for the entire field of view of the optical system 100, or may obtain distance information only for the subject in focus. The image processing unit (processing means) 104 may perform region division (processing of dividing the captured image into multiple regions) according to the subject distance based on the distance information obtained by the distance obtaining unit 111.

[0028] The subject determination unit (determination means) 112 can determine and extract the area of ​​the subject (main subject) in the captured image (captured image data) or the subject area having the same focus (same focus state) as the main subject.

[0029] The image processing unit 104 acquires optical correction information, such as an optimal aberration correction amount or an image restoration filter, from the storage unit 108 using the subject distance information acquired by the distance acquisition unit 111 and information about the imaging state of the optical system 100 acquired by the state detection unit 107. That is, the image processing unit 104 determines optical correction information for correcting an image using the subject distance information and the information about the imaging state. Here, the information about the imaging state of the optical system 100 includes, for example, information about the drive amount or position of the focus group 101 (lens position information), but is not limited to this.

[0030] In this embodiment, the storage unit 108 is a storage means that stores a database in which optical correction information such as an optimal aberration correction amount and an image restoration filter for each drive amount and subject distance of the focus group 101 is registered. The storage unit 108 stores optical correction information for each back focus deviation information or optical filter information (information on the thickness of the optical filter), for example.

[0031] In this embodiment, the image processing unit 104 can acquire back focus deviation information using, for example, subject distance information and lens position information, and determine optical correction information corresponding to the back focus deviation information. Also, in this embodiment, the image processing unit 104 can acquire optical filter information using, for example, subject distance information and lens position information, and determine optical correction information corresponding to the optical filter information.

[0032] The image data corrected by the image processing unit 104 may be displayed on the display unit 105. In this embodiment, the restoration filter is created using design data of the optical system 100, and may also be created using fluctuation data due to manufacturing errors. In this embodiment, the storage unit 108 is provided in the camera body, but this is not limiting, and the storage unit 108 may be provided in the lens device. Also, in this embodiment, the restoration filter may be an inverse filter of a point spread function (PSF), a filter obtained by correcting the inverse filter, or a filter such as a Wiener filter.

[0033] Next, the processing of the image processing unit 104 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the processing of the image processing unit 104.

[0034] First, in step S101, the image processing unit 104 acquires captured image data from the image sensor 102. Next, in step S102, the subject determination unit 112 selects (determines) a main subject region based on the captured image data. Next, in step S103, the image processing unit 104 acquires distance information of the main subject region. Next, in step S104, the state detection unit 107 acquires position information of the lens group including the focus group 101.

[0035] Next, in step S105, the image processing unit 104 acquires (selects) optical correction information (correction coefficients) such as an aberration correction amount or an image restoration filter used for image correction from the storage unit 108 based on the distance information of the main subject region and the position information of the lens group. Note that in this embodiment, the storage unit 108 is provided in the imaging device 500, but is not limited to this. The storage unit 108 may be provided in, for example, an external device connected to the imaging device 500, or on the cloud.

[0036] Next, in step S106, the image processing unit 104 uses the optical correction information (correction coefficients) acquired in step S105 to correct an image (degraded image) that has been degraded by the optical system 100 and back focus deviation for a specific image component in the captured image. Here, this correction can be performed only on the main subject, or only on the subject area that has the same focus as the main subject. Alternatively, correction can be performed for the entire area of ​​the captured image for each distance and image height.

[0037] Next, conditions that the image processing unit 104 in this embodiment should preferably satisfy will be described.

[0038] The image sensor 102 is preferably an image sensor having an APD. By using an APD, the image sensor 102 can function as both the image sensor 102 and the distance acquisition unit 111. In this embodiment, the image sensor 102 preferably has an auxiliary light projection unit (light emitting unit) for acquiring distance information using the distance acquisition unit 111, the image sensor 102, or the APD. The distance acquisition unit 111 emits light toward the subject using the light emitting unit and receives light reflected from the subject using the light receiving unit.

[0039] In this embodiment, it is more preferable that the image sensor 102 is a SPAD sensor. By using a SPAD sensor, it is possible to capture high-sensitivity images and obtain clearer images even at night or when using a dark lens device with a large F-number.

[0040] In this embodiment, the optical correction information preferably includes correction information for distortion or chromatic aberration of magnification, has correction values ​​according to at least subject distance information and lens position information, and performs image correction processing that is rotationally symmetric about the optical center of the image.

[0041] 4(a) and 4(b), in this embodiment, it is possible to correct not only the subject distance but also distortion and chromatic aberration of magnification that fluctuate due to focus group drive associated with back focus shift. Furthermore, the optical correction information is correction information for axial chromatic aberration or field curvature aberration, and it is preferable to have a correction filter corresponding to at least subject distance information and lens position information, and to perform convolution processing (image restoration processing) on ​​the captured image.

[0042] As described with reference to FIGS. 3(a) and 3(b), in this embodiment, it is possible to perform aberration correction corresponding to not only the subject distance but also the PSF that fluctuates due to focus group drive associated with back focus deviation.

[0043] In this embodiment, the optical correction information preferably includes information corresponding to the variation of the focus position (focal point) due to frequency. Here, the frequency refers to the frequency on the image plane (line pairs / mm). The MTF (modulation transfer function) in the defocus direction of the optical system 100 changes depending on the variation of the subject distance and the focus group position. This changes the best focus position for each frequency. By providing this correction value to the optical system control unit 106, even higher precision focusing is possible, improving image quality.

[0044] The storage unit 108 preferably stores different optical correction information depending on optical filter information such as the thickness of an optical filter, such as an ND filter, attached to the lens device (optical system 100). The image processing unit 104 can select optimal optical correction information depending on the optical filter information input by the user or the optical filter information calculated by the system controller 110.

[0045] The thickness of the optical filter can be calculated as follows. First, the focus group position d1 without the optical filter at a certain subject distance is stored. Next, the optical filter is inserted and the focus group position d2 after focusing is stored. The system controller determines the amount of focus group drive by comparing d1 and d2. When an optical filter with a refractive index n is inserted, the back focus becomes longer by (n-1)ε, where ε is the thickness of the optical filter. Furthermore, the focus sensitivity, i.e., the amount of image plane movement when the focus group is driven 1 mm, is expressed as (1-β1^2)β2^2. Here, β1 is the lateral magnification of the focus group, and β2 is the lateral magnification of all lenses after the focus group.

[0046] The thickness of the optical filter can be calculated using ε = (d2 - d1) (1 - β1^2) β2^2 / (n-1). Differences in the thickness of the optical filter cause fluctuations in the field curvature, axial chromatic aberration, and spherical aberration of the optical system 100. Therefore, by storing optical correction information according to the subject distance and the movement amount of the focus group for each of various thicknesses ε, it is possible to correct image degradation caused by the optical filter. Here, it is not necessary to calculate the thickness ε; optical correction information corresponding to the difference in the optical filter may also be selected from the movement amount of the focus group 101 itself.

[0047] Furthermore, the optical correction information may differ for each temperature. The temperature (temperature information) may be acquired by a temperature sensor 113 built into the lens device or the image capturing device 500. Alternatively, the user may input the temperature into the image capturing device 500. This allows image correction processing to be performed that takes into account changes in the amount of back focus shift due to temperature.

[0048] Next, numerical examples of the optical system 100 will be shown. The numerical examples represent a state of focusing at infinity. In the numerical examples, the surface numbers indicate the numbers of the optical surfaces counted from the object side. r is the radius of curvature of the ith optical surface (i-th surface) counted from the object side (i is a natural number), and d is the distance (distance on the optical axis) between the ith surface and the (i+1)th surface. nd and vd are the refractive index and Abbe number for the d-line, respectively. Note that the Abbe number vd of a certain material is given by, when the refractive indices at the d-line (587.6 nm), F-line (486.1 nm), and C-line (656.3 nm) of the Fraunhofer lines are Nd, NF, and NC, respectively. νd=(Nd-1) / (NF-NC) It is expressed as:

[0049] In the numerical examples, aspherical lens surfaces are marked with an asterisk (*) after the surface number. ±PThe aspherical shape of an optical surface is expressed by the following formula (A), where x is the displacement from the vertex of the surface in the optical axis direction, h is the height from the optical axis in a direction perpendicular to the optical axis, R is the paraxial radius of curvature, k is the conic constant, and A4, A6, A8, and A10 are aspherical coefficients.

[0050] x=(h 2 / R) / [1+{1-(1+k)(h / R) 2} 1 / 2 ]+A4×h 4 +A6×h 6 +A8×h 8 +A10×h 10 [Numerical Example] Unit: mm Surface Data Surface number rd nd νd 1 29.003 1.75 2.00100 29.1 2 10.353 9.81 3 27.827 0.95 2.00100 29.1 4 10.911 5.58 5 -14.687 1.00 1.92250 36.0 6 20.939 0.65 7 29.084 2.26 1.89286 20.4 8 -16.390 3.71 9 ∞ 8.50 1.88300 40.8 10 ∞ 2.21 11 (Aperture) ∞ 0.60 12 571.710 1.69 1.52310 50.8 13 -20.256 2.00 14 ∞ 8.50 1.88300 40.8 15 ∞ 1.42 16* -56.169 1.71 1.43875 94.7 17 -12.502 0.98 18 -176.273 2.63 1.43875 94.9 19 -13.451 0.50 20 -31.094 1.20 2.00540 27.7 21 26.939 0.50 22 25.291 4.73 1.43875 94.9 23 -9.963 20.00 Image plane ∞ Aspheric data Page 16 K = 0.00000e+00 A 4=-3.70913e-04 A 6=-1.45620e-08 A 8=-7.58114e-08 Various data Focal length 4.40 F-number 4.00 Half angle of view (degrees) 57.85 Image height 7.00 Lens length 82.89 BF 20.00 Lens group data Group starting plane focal length 1 1 -6.77 2 9 ∞ 3 11 ∞ 4 12 37.43 5 14 ∞ 6 16 36.22 7 18 37.92 Focus group 6 groups (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0051] According to each embodiment, it is possible to provide an image processing device, an imaging device, an image processing method, and a program that are capable of determining optical correction information for correcting an image degraded by an optical system with high precision.

[0052] The disclosure of each embodiment includes the following configurations and methods. (Configuration 1) An image processing device that corrects an image degraded by an optical system, a determination means for determining a subject in the image; a distance acquisition means for acquiring subject distance information relating to the distance to the subject; a position acquisition unit for acquiring lens position information relating to the positions of the lens groups in the optical system; an image processing device comprising: a processing unit that determines optical correction information for correcting the image using the subject distance information and the lens position information; (Configuration 2) 2. The image processing device according to configuration 1, wherein the processing means corrects the image using the optical correction information. (Configuration 3) 3. The image processing device according to configuration 1 or 2, wherein the subject distance information is information relating to an absolute distance to the subject. (Configuration 4) 4. The image processing device according to any one of configurations 1 to 3, further comprising a storage means for storing the optical correction information for each piece of back focus deviation information or optical filter information. (Configuration 5) The processing means obtaining back focus deviation information using the subject distance information and the lens position information; 5. The image processing device according to any one of the first to fourth configurations, wherein the optical correction information corresponding to the back focus deviation information is determined. (Configuration 6) The processing means acquiring optical filter information using the subject distance information and the lens position information; 5. The image processing device according to any one of the first to fourth configurations, wherein the optical correction information corresponding to the optical filter information is determined. (Configuration 7) 7. The image processing device according to any one of configurations 1 to 6, wherein the distance acquisition means is an avalanche photodiode sensor. (Configuration 8) The distance acquisition means having a light emitting means and a light receiving means, the light emitting means emits light toward the subject, 8. The image processing device according to any one of configurations 1 to 7, wherein the light receiving means receives reflected light from the subject. (Configuration 9) the optical correction information is correction information related to distortion or chromatic aberration of magnification, 9. The image processing device according to any one of configurations 1 to 8, wherein the processing means performs image correction processing that is rotationally symmetric about the optical center of the image. (Configuration 10) the optical correction information is correction information related to axial chromatic aberration or field curvature aberration, 9. The image processing device according to any one of configurations 1 to 8, wherein the processing means performs image restoration processing on the image. (Configuration 11) 9. The image processing device according to any one of configurations 1 to 8, wherein the optical correction information is information relating to a variation in a focal position according to a frequency of the object in the image. (Configuration 12) Further comprising a temperature detection means for acquiring temperature information, 12. The image processing device according to any one of configurations 1 to 11, wherein the processing means determines the optical correction information using the subject distance information, the lens position information, and the temperature information. (Configuration 13) 13. An imaging device comprising: the image processing device according to any one of configurations 1 to 12; and an imaging element. (Configuration 14) 14. The imaging device according to configuration 13, wherein the imaging element and the distance acquisition means are integrated into one device. (Method 1) An image processing method for correcting an image degraded by an optical system, comprising: determining an object in the image; acquiring object distance information relating to the distance to the object; obtaining lens position information relating to positions of lens groups in the optical system; an image processing method comprising a step of determining optical correction information for correcting the image using the subject distance information and the lens position information. (Configuration 15) A program that causes a computer to execute the image processing method described in Method 1.

[0053] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0054] 100 Optical system 101 Focus group (lens group) 104 Image processing unit (processing means) 107 Status detection unit (position acquisition means) 111 Distance acquisition unit (distance acquisition means) 112 Subject determination section (determination means)

Claims

1. An image processing device that corrects an image degraded by an optical system, a determination means for determining a subject in the image; a distance acquisition means for acquiring subject distance information relating to the distance to the subject; a position acquisition unit for acquiring lens position information relating to the positions of the lens groups in the optical system; an image processing device comprising: a processing unit that determines optical correction information for correcting the image using the subject distance information and the lens position information;

2. 2. The image processing apparatus according to claim 1, wherein the processing means corrects the image using the optical correction information.

3. 2. The image processing apparatus according to claim 1, wherein the subject distance information is information relating to an absolute distance to the subject.

4. 2. The image processing apparatus according to claim 1, further comprising a storage unit for storing the optical correction information for each of back focus deviation information and optical filter information.

5. The processing means obtaining back focus deviation information using the subject distance information and the lens position information; 2. The image processing apparatus according to claim 1, wherein the optical correction information corresponding to the back focus deviation information is determined.

6. The processing means acquiring optical filter information using the subject distance information and the lens position information; 2. The image processing apparatus according to claim 1, wherein the optical correction information corresponding to the optical filter information is determined.

7. 7. The image processing device according to claim 1, wherein the distance acquisition means is an avalanche photodiode sensor.

8. The distance acquisition means having a light emitting means and a light receiving means, the light emitting means emits light toward the subject, 7. The image processing device according to claim 1, wherein the light receiving means receives light reflected from the subject.

9. the optical correction information is correction information related to distortion or chromatic aberration of magnification, 7. The image processing apparatus according to claim 1, wherein the processing means performs image correction processing that is rotationally symmetric about the optical center of the image.

10. the optical correction information is correction information related to axial chromatic aberration or field curvature aberration, 7. The image processing apparatus according to claim 1, wherein the processing means performs an image restoration process on the image.

11. 7. The image processing device according to claim 1, wherein the optical correction information is information relating to a variation in a focal position according to a frequency of the object in the image.

12. Further comprising a temperature detection means for acquiring temperature information, 7. The image processing apparatus according to claim 1, wherein the processing means determines the optical correction information using the subject distance information, the lens position information, and the temperature information.

13. An imaging device comprising: the image processing device according to claim 1; and an imaging element.

14. 14. The imaging device according to claim 13, wherein the imaging element and the distance acquisition means are integrated into one device.

15. An image processing method for correcting an image degraded by an optical system, comprising: determining an object in the image; acquiring object distance information relating to the distance to the object; obtaining lens position information relating to positions of lens groups in the optical system; an image processing method comprising a step of determining optical correction information for correcting the image using the subject distance information and the lens position information.

16. A program causing a computer to execute the image processing method according to claim 15.

Images