Distance measurement device, computer program, and distance measurement method

Abstract

To provide distance measurement device capable of making accurate distance measurement while enabling an anti-shake mechanism to operate.SOLUTION: A distance measurement device comprises: image information acquisition means for acquiring an image captured by an image capturing optical system having an anti-shake optical system; distance information acquisition means for acquiring distance information regarding a distance to an object based on the image; and distance correction means for correcting the distance information on the basis of aberration information corresponding to a driven amount of the anti-shake optical system.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a distance measurement device having an anti-vibration optical system. [Background technology]

[0002] Conventionally, a distance detection technique using a phase difference method has been known as a distance measurement device applicable to digital still cameras and digital video cameras and a distance detection technique using the same. One example of this method is to obtain two light images (hereinafter referred to as image A and image B, respectively) formed by light beams passing through two pupil regions offset in the horizontal direction, for example. The image shift amount (parallax, phase difference), which is the relative position shift amount between image A and image B, is calculated, and converted to a defocus amount via a conversion coefficient based on the distance between the two pupil regions on the lens pupil, thereby making it possible to calculate the distance to the subject.

[0003] In addition, in recent years, digital still cameras and digital video cameras have been equipped with an anti-shake function to prevent image blur caused by camera shake, etc. For example, in Patent Document 1, when image blur is suppressed by shifting a part of a lens system, the state of distortion aberration changes due to the shift of the lens system, so that the distortion aberration of the display image is corrected according to position information of the image blur correction optical system.

[0004] Furthermore, for example, Patent Document 2 describes a phase difference distance measurement method on the imaging plane using an imaging system with an anti-vibration function, and describes correcting distance information so as to reduce errors caused by positional shifts in the plane of two images that occur when the anti-vibration optical system is operated. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2012-231262 A [Patent Document 2] Patent No. 6628678 Summary of the Invention [Problem to be solved by the invention]

[0006] It is preferable that an imaging device equipped with a distance measuring device (such as a digital still camera or a digital video camera) is portable, and a distance map with an intended composition can be obtained by carrying the camera and taking pictures from an appropriate position. Also, a high-density, high-precision distance map can be obtained by taking pictures from a close distance.

[0007] However, when image correction is performed according to the light amount and image distortion that vary due to image vibration isolation using the method described in Patent Document 1, the variation in image brightness and distortion can be suppressed, but the distance information varies. In other words, there is a problem that an error occurs in the distance measurement. In Patent Document 2, although distance information is corrected by image stabilization, no consideration is given to correction caused by fluctuations in field curvature due to image stabilization, and there is a problem in that highly accurate correction cannot be performed.

[0008] The present invention has been made in consideration of the above problems, and has an object to provide a distance measurement device that is capable of performing accurate distance measurement while operating an anti-vibration mechanism. [Means for solving the problem]

[0009] In order to achieve the above object, a distance measurement device of the present invention comprises: an image information acquiring means for acquiring an image captured by an imaging optical system having an anti-vibration optical system; a distance information acquisition means for acquiring distance information relating to a distance to a subject based on the image; The vibration-proof optical system Drive phase and A distance correction unit corrects the distance information based on aberration information corresponding to a drive amount. death, The distance correction means includes a correction table based on the aberration information in the sagittal direction and a correction table based on the aberration information in the meridional direction. (Note: A limitation corresponding to claim 8) The distance correction means corrects the distance information using the data of the correction table in the sagittal direction and the data of the correction table in the meridional direction based on the drive phase and the drive amount of the vibration-proof optical system (Note: A limitation corresponding to claim 9). The present invention is characterized by the above. Effect of the Invention

[0010] According to the present invention, it is possible to provide a distance measurement device capable of performing accurate distance measurement while operating an anti-vibration mechanism. [Brief description of the drawings]

[0011] [Figure 1] 3 is a diagram showing an imaging relationship of lenses in Example 1 of the present invention. FIG. [Diagram 2] 5A to 5C are diagrams for explaining distance measurement errors caused by field curvature in the first embodiment. [Diagram 3] 4 is a diagram showing the relationship between the drive direction of the vibration isolation group and the parallax detection direction according to the first embodiment. FIG. [Figure 4] 1 is a block diagram showing a functional configuration of a digital camera 110 according to a first embodiment. [Diagram 5] 1 is a diagram for explaining an image sensor 11 of a digital camera 110 according to a first embodiment. [Figure 6] 3A to 3C are diagrams for explaining the distance measurement principle of the imaging plane phase difference distance measurement method according to the first embodiment. [Figure 7] 11 is a flowchart showing a defocus image generation process executed by a defocus image generation unit 141 according to the first embodiment. [Figure 8] 4 is a flowchart showing distance measurement-related processing according to the first embodiment. [Figure 9] 10 is a flowchart showing a distance measurement process according to the second embodiment of the present invention. [Figure 10] FIG. 11 is a diagram showing an example of an edge direction according to the second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In each drawing, the same members or elements are designated by the same reference numerals, and duplicated descriptions will be omitted or simplified. In the embodiments, the present invention will be described by way of example in which the imaging device is a digital camera, but the imaging device also includes electronic devices having an imaging function, such as digital video cameras, smartphones with cameras, tablet computers with cameras, vehicle-mounted cameras, and network cameras. EXAMPLES

[0013] FIG. 1 is a diagram showing the imaging relationship of lenses in the first embodiment of the present invention. In the embodiments, the direction parallel to the optical axis is defined as the z direction or defocus direction, the direction perpendicular to the optical axis and parallel to the horizontal direction of the imaging surface is defined as the x direction, and the direction parallel to the vertical direction of the imaging surface is defined as the y direction. Distance measurement methods, such as the imaging plane phase difference distance measurement method and the DFD (Depth from Defocus) method, are known in which the amount of parallax is calculated from an image formed through an imaging optical system and the amount of parallax is converted into an amount of defocus to measure distance.

[0014] In Figure 1, OBJ is the object plane, IMG is the image plane, H is the front principal point, H' is the rear principal point, S is the distance from the object plane to the front principal point, and S' is the distance from the rear principal point to the image plane. Additionally, ΔS' is the defocus amount, and ΔS is the relative distance on the object side according to the defocus amount. The dashed-dotted line is the optical axis, the dotted line is the imaging light beam, and the dashed line is the defocus light beam. It is known that the following equation holds true for lens imaging, where f is the focal length of the lens.

[0015]

number

[0016]

number

[0017] As described above, by simultaneously calculating Equation 1 and Equation 2, it is possible to generate subject distance information using shooting information and the defocus amount. Since the defocus amount can be calculated for a partial area or the entire area on the screen, subject distance information can also be obtained corresponding to the area where the defocus amount is calculated. <Distance measurement errors due to field curvature>

[0018] FIG. 2 is a diagram for explaining distance measurement errors caused by field curvature in the first embodiment. Distance measurement methods that measure distance from an image formed through an imaging optical system, such as the above-mentioned imaging plane phase difference distance measurement method and DFD method, are affected by the field curvature of the imaging optical system, especially near the periphery of the screen.

[0019] 2(A) is a schematic diagram illustrating the field curvature of the imaging optical system 10. Light emitted from an object point 102 existing on the optical axis 100 is ideally imaged at a point on the image plane IMG by the imaging optical system 101. Also, light emitted from an object point 113 existing on a plane perpendicular to the optical axis 100 is ideally imaged at a point on the plane of the image plane IMG.

[0020] However, the imaging optical system 101 generally has field curvature, and the position where the light emitted from the object point 113 forms an image is shifted in the optical axis direction, causing defocusing. In Fig. 2A, 104 denotes a light beam emitted from the object point 102, and 105 denotes a light beam emitted from the object point 113. Δd is the amount of field curvature of the image point corresponding to 105. Here, Fig. 2A shows the field curvature of the meridional image plane, and the field curvature of the sagittal image plane is omitted, but the field curvature of the sagittal image plane generally has a different value from that of the meridional image plane.

[0021] In this way, the amount of defocus on the image plane varies depending on the aberration of the imaging optical system, particularly the curvature of field at the periphery of the screen. Because the distance measurement method described above is dependent on the amount of defocus, the calculated distance will differ depending on the curvature of field even for subjects located at the same distance.

[0022] Normally, the curvature of field is a value that is almost fixed depending on the state of zoom, focus, and aperture. The deviation in the defocus amount including this curvature of field can be calculated and calibrated using a defocus correction map using the design value of the imaging optical system or the factory measurement value measured at the time of shipment from the factory, thereby eliminating distance measurement errors. <Distance measurement error when vibration is reduced>

[0023] However, during image stabilization, deviations occur between design values ​​and factory measured values ​​due to decentering aberrations, and if asymmetric field curvature remains, distance measurement errors occur. In FIG. 2B, 105 and 106 indicate light beams emitted from object points at symmetric positions with respect to the optical axis. As shown in the figure, by driving the image stabilization group (image stabilization optical system) 112, for example, in a direction perpendicular to the optical axis, image curvature Δdp occurs on one side of the image side, and image curvature Δdm occurs at the symmetric position. The figures show that the defocus amount increases or decreases with respect to Δd in FIG. 2A before the image stabilization group is driven.

[0024] Here, in Fig. 2, the drive of the vibration-isolating group is illustrated as drive in a cross section perpendicular to the optical axis, but the drive method is not limited to this. For example, a method of performing vibration isolation by tilting one side of a parallel plate filled with liquid, such as a variable angle prism (VAP), may be used. Also, a method of performing vibration isolation by tilting the vibration-isolating group may be used. That is, the vibration-proof optical system in this embodiment may perform vibration proofing by driving at least one optical element in a direction perpendicular to the optical axis or by tilting the optical element with respect to the optical axis, as described above.

[0025] In this way, when image stabilization is performed by refraction within an imaging optical system, asymmetric curvature of field occurs with respect to the center of the screen due to the influence of decentering aberration, and the amount of defocus varies, which becomes a cause of distance measurement errors.

[0026] A typical imaging optical system is designed so that decentering aberrations caused by image stabilization have no effect on image quality or have only a minimal effect. However, in the case of a distance measurement method that uses defocus for distance measurement, the field curvature component of decentering aberrations caused by image stabilization can cause significant distance measurement errors, especially in the peripheral areas of the screen. In particular, the greater the drive amount of the image stabilization group, the greater the amount of decentering aberrations that occur. <Creating a correction map based on the vibration isolation group drive phase>

[0027] For example, the defocus amount is calculated from the parallax amount using techniques such as the image plane phase difference distance measurement method or the DFD method, and distance measurement is performed. Also, in order to correct vibration, the above-mentioned vibration isolation group is driven by an arbitrary drive amount in an arbitrary direction (phase) on a plane perpendicular to the optical axis. In this case, since the direction in which parallax is detected is a predetermined direction that is mechanically or algorithmically restricted, the effect on parallax of the field curvature component caused by the movement of the vibration isolation group varies depending on the phase of the vibration isolation group, even if the drive amount of the vibration isolation group is the same. In other words, even with the same drive amount, the defocus correction map due to vibration isolation will differ depending on the phase of the vibration isolation group.

[0028] Therefore, in this embodiment, multiple defocus correction maps that do not depend on the phase (driving direction) of the vibration isolation group are held. These are correction maps that correspond to multiple first correction amounts. Furthermore, it is possible to generate a defocus correction map by mixing (e.g., weighted addition) the multiple first correction amounts with an amount determined by the drive amount and phase of the vibration isolation group. The mixed defocus correction map corresponds to the second correction amount. By correcting the distance information using the second correction amount, it is possible to perform high-precision distance measurement.

[0029] For example, the field curvature components of the sagittal and meridional image planes of the defocus amount generated by the image stabilization group are each linearly symmetric in the direction perpendicular to the drive direction. Therefore, by providing these two defocus correction maps and mixing them using the phase of the image stabilization group as a coefficient, it is possible to generate a defocus correction map in the parallax detection direction without having data for each phase.

[0030] FIG. 3 is a diagram showing the relationship between the drive direction of the vibration isolation group and the parallax detection direction according to the first embodiment of the present invention. When the parallax detection direction is the horizontal direction, the angle between the drive direction of the vibration isolation group and the horizontal is set as the drive phase θ. By mixing the field curvature component of the sagittal image plane and the field curvature component of the meridional image plane using the drive phase θ as an argument, it is possible to generate a defocus correction map according to the parallax detection direction. Note that, although the drive of the vibration isolation group is illustrated in FIG. 3 as drive in a cross section perpendicular to the optical axis, the drive method is not limited to this.

[0031] For example, as in the previously mentioned variable angle prism, a method of performing vibration reduction by tilting one side of a parallel plate filled with liquid may be used. Alternatively, a method of performing vibration reduction by tilting an optical element such as a lens in a vibration reduction group may be used. When a different method is used, the values ​​are converted to perform correction processing, for example, so that the drive amount is the tilt angle and the drive phase is the tilt direction. <Image capture information>

[0032] In this embodiment, the captured image information as input information is, for example, information that accompanies an image captured using a digital camera as metadata. FIG. 4 is a block diagram showing the functional configuration of the digital camera 110 according to the first embodiment. The imaging optical system 10 is a photographing lens built into a digital camera 110 or is an interchangeable lens type, and forms an optical image of a subject on an imaging element 11 .

[0033] The imaging optical system 10 is made up of a plurality of lenses arranged on an optical axis 100, and has an exit pupil 111 at a position a predetermined distance away from the imaging element 11, as well as a vibration-proof group (vibration-proof optical system) 112. In this embodiment, a digital camera serving as an imaging device functions as a distance measurement device, but the distance measurement device may also be configured to read and process an image from a recording medium in an image processing device such as a PC or tablet to measure distance.

[0034] The imaging element 11 is, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging element 11 photoelectrically converts a subject image formed on an imaging surface via the imaging optical system 10, and outputs an image signal corresponding to the subject image. The imaging element 11 functions as an imaging means for capturing an image of a subject. The control unit 12 is a control device including, for example, a CPU or a microprocessor as a computer, and functions as a control means that controls the operation of each block of the digital camera 110 based on a computer program stored in a memory.

[0035] The control unit 12 controls, for example, autofocus (AF: automatic focusing) during image capture, changing the focus position, changing the F-number (aperture), capturing images, and controlling each part inside the digital camera 110, such as the memory unit 15, input unit 16, display unit 17, and communication unit 18.

[0036] The measurement unit 13 calculates the distance to a focused subject. As shown in the figure, the measurement unit 13 includes a lens drive information acquisition unit 130, a vibration isolation group information acquisition unit 131, a correction information acquisition unit 132, and a correction unit 133.

[0037] The image processing unit 14 is a block that realizes various types of image processing in the digital camera 110. It performs various types of signal processing such as noise removal, demosaicing, luminance signal conversion, aberration correction, white balance adjustment, and color correction of the image signal output from the imaging element 11. As shown in the figure, the image processing unit 14 has a defocus generation unit 141, which generates a defocus signal from the two types of obtained image signals (A image signal, B image signal).

[0038] Image data (captured image) output from the image processing unit 14 is accumulated in a memory (not shown) and displayed on the display unit 17. The output image data is stored in the storage unit 15. The image processing unit 14 may be configured using a logic circuit, or may be configured from a central processing unit (CPU) and a memory that stores a processing program.

[0039] Next, the storage unit 15 is a non-volatile recording medium that records captured image data, intermediate data generated in the process of the operation of each block, parameters referenced in the operation of the image processing unit 14 and the digital camera 110, a correction table based on aberration information, etc. The storage unit 15 may be any type of recording medium that can be read and written at high speed and has a large capacity, and a flash memory, for example, is preferable.

[0040] Here, it functions as a memory that stores at least one correction table based on aberration information corresponding to the drive amount of the vibration-proof optical system. The measurement unit 13, image processing unit 14, storage unit 15, etc. constitute a distance correction means that corrects the distance information based on aberration information corresponding to the drive amount of the vibration-proof optical system.

[0041] The input unit 16 is a user interface that detects operation inputs such as dials, buttons, switches, and touch panels for inputting information and changing settings made to the digital camera 110. When the input unit 16 detects an operation input, it outputs a corresponding control signal to the control unit 12.

[0042] The display unit 17 is, for example, a display device such as a liquid crystal display or an organic electroluminescence (EL) display. The display unit 17 is used to check the composition when shooting by displaying the captured image through the screen, and to display various setting screens and message information. In addition, the display unit 17 can have both a display function and an input function by using a touch panel. The communication unit 18 is a communication interface for transmitting and receiving information to and from the outside, and is configured to be able to send the captured image, photographing information, and the like obtained to another device.

[0043] Sensor 19 is a type of sensor that monitors the state of the imaging device, and includes, for example, an acceleration sensor, a gyro sensor, a temperature sensor, etc. A gyro sensor and an acceleration sensor are generally used to prevent vibration of the imaging device, and the vibration of the imaging device is detected by the acceleration sensor, etc., and the vibration of the image is cancelled out by driving the vibration prevention optical system in response to the vibration.

[0044] In addition, the captured image information in this embodiment is not limited to information accompanying a captured image taken by a digital camera, but may be information accompanying an image or video that is simulated using a 3D model and ray tracing. <Example of imaging element configuration>

[0045] FIG. 5 is a diagram for explaining the image sensor 11 of the digital camera 110 of the first embodiment, and an example of the detailed configuration of the image sensor 11 will be described with reference to FIG. As shown in Fig. 5(A), the imaging element 11 is configured by connecting and arranging a plurality of pixel groups 210 in 2 rows x 2 columns, in which different color filters are arranged. As shown in the enlarged view, the pixel groups 210 are arranged with red (R), green (G), and blue (B) color filters, and each pixel (photoelectric conversion element) outputs an image signal indicating color information of either R, G, or B. Note that the color filter arrangement pattern is not limited to this.

[0046] Fig. 5(B) is a schematic diagram of the cross-sectional structure of one pixel, and in order to perform distance measurement using the imaging surface phase difference distance measurement method, one pixel (photoelectric conversion element) has multiple photoelectric conversion units 215, 216 arranged side by side in the horizontal cross section I-I' of Fig. 5(A). That is, as shown in Fig. 5(B), each pixel is composed of a light guide layer 213 including a microlens 211 and a color filter 212, and a light receiving layer 214 including a first photoelectric conversion unit 215 and a second photoelectric conversion unit 216.

[0047] In the light guide layer 213, the microlenses 211 are configured to guide the light beams incident on the pixels to the first photoelectric conversion unit 215 and the second photoelectric conversion unit 216. As a result, the photoelectric conversion unit 215 and the second photoelectric conversion unit 216 receive the light of the images passing through different exit pupil regions shifted in a predetermined direction. The color filter 212 passes light of a predetermined wavelength band, and passes only light of any one of the above-mentioned R, G, and B wavelength bands, and guides it to the first photoelectric conversion unit 215 and the second photoelectric conversion unit 216 in the subsequent stage.

[0048] The first photoelectric conversion unit 215 and the second photoelectric conversion unit 216 convert the received light into analog signals, and the two types of signals output from these two photoelectric conversion units are used for distance measurement. That is, each pixel of the image sensor 11 has two photoelectric conversion units arranged in the horizontal direction. Distance measurement is performed based on the correlation distance between a first image signal S1 (A image signal) composed of signals output from the first photoelectric conversion unit 215 group among all pixels, and a second image signal S2 (B image signal) composed of signals output from the second photoelectric conversion unit 216 group.

[0049] That is, the first photoelectric conversion unit 215 and the second photoelectric conversion unit 216 receive different parts of the light beam that enters the pixel through the microlens 211. Therefore, the two types of image signals S1 and S2 finally obtained are pupil-divided images related to the light beam that has passed through different regions of the exit pupil of the imaging optical system 10. Note that a signal obtained by combining the image signals photoelectrically converted by the first photoelectric conversion unit 215 and the second photoelectric conversion unit 216 at each pixel can be used as an image signal for display.

[0050] With such a structure, the imaging element 11 of this embodiment can not only obtain an image signal for display, but also separate and output image signals for distance measurement (A image signal and B image signal). Here, the imaging element 11 functions as an image information acquisition means that reads out and acquires an image (A image signal and B image signal) captured for a predetermined exposure period by an imaging optical system having an anti-vibration optical system from the imaging element. Note that the image information acquisition means is not limited to the imaging element, and includes, for example, a means that temporarily records an image captured by the imaging element (e.g., A image signal and B image signal) on a recording medium and reads out and acquires the image from the recording medium.

[0051] In this embodiment, all pixels of the image sensor 11 are described as having two photoelectric conversion units, but the implementation of this embodiment is not limited to this. For example, the structure shown in FIG. 5B may be arranged not only in the horizontal direction but also in the vertical direction. That is, each pixel may have four photoelectric conversion units, and the pixel configuration may enable pupil division not only in the horizontal direction but also in the vertical direction. By adopting a four-division configuration, it is possible to perform high-precision distance measurement even for an object having, for example, horizontal or vertical stripes. <Principle of distance measurement using phase difference distance measurement on the imaging surface>

[0052] FIG. 6 is a diagram for explaining the distance measurement principle of the image plane phase difference distance measurement method of the first embodiment. The principle of deriving the subject distance based on the above-mentioned pupil-divided image group (A image signal, B image signal) will be described with reference to FIG. 6. Fig. 6(A) is a schematic diagram showing the exit pupil 111 of the imaging optical system 10 and a light beam received by the first photoelectric conversion unit 215 of one pixel in the image sensor 11. Fig. 6(B) is a schematic diagram showing a light beam received by the second photoelectric conversion unit 216 in a similar manner.

[0053] The microlens 211 shown in Fig. 6(A) and (B) is disposed so that the exit pupil 111 and the light receiving layer 214 are optically conjugate with each other. A light beam passing through the exit pupil 111 of the imaging optical system 10 is collected by the microlens 211 and guided to the first photoelectric conversion unit 215 or the second photoelectric conversion unit 216. At this time, the first photoelectric conversion unit 215 and the second photoelectric conversion unit 216 mainly receive light beams passing through different pupil regions shifted in the horizontal direction, for example, as shown in Fig. 6(A) and (B), respectively. That is, the light beam passing through the first pupil region 510 is incident on the first photoelectric conversion unit 215, and the light beam passing through the second pupil region 520 is incident on the second photoelectric conversion unit 216.

[0054] The multiple first photoelectric conversion units 215 included in the image sensor 11 mainly receive the light beam that has passed through the first pupil region 510, and output a first image signal S1 (A image signal). At the same time, the multiple second photoelectric conversion units 216 included in the image sensor 11 mainly receive the light beam that has passed through the second pupil region 520, and output a second image signal S2 (B image signal). From the first image signal S1, it is possible to obtain the intensity distribution of the image formed on the image sensor 11 by the light beam that has passed through the first pupil region 510. From the second image signal S2, it is possible to obtain the intensity distribution of the image formed on the image sensor 11 by the light beam that has passed through the second pupil region 520.

[0055] The relative positional shift amount (so-called parallax amount or phase difference) between the first image signal S1 and the second image signal S2 is a value according to the defocus amount. The relationship between the parallax amount and the defocus amount will be described with reference to Figs. 6(C), (D), and (E). Figs. 6(C), (D), and (E) are schematic diagrams illustrating the image sensor 11 and the imaging optical system 10. Reference numeral 511 in the drawings denotes a first light flux passing through a first pupil region 510, and reference numeral 521 denotes a second light flux passing through a second pupil region 520.

[0056] Fig. 6C shows a state when in focus, where the first light beam 511 and the second light beam 521 are converged on the image sensor 11. At this time, the amount of parallax between the first image signal S1 formed by the first light beam 511 and the second image signal S2 formed by the second light beam 521 is zero. Fig. 5D shows a state where the image side is defocused in the negative direction of the z axis. At this time, the amount of parallax between the first image signal S1 formed by the first light beam and the second image signal S2 formed by the second light beam is not zero, but has a negative value.

[0057] FIG. 6(E) shows a state where the image side is defocused in the positive direction of the z-axis. At this time, the parallax amount between the first image signal S1 formed by the first light beam and the second image signal S2 formed by the second light beam has a positive value. Comparing FIG. 6(D) and FIG. 6(E), it can be seen that the direction of the positional deviation is reversed depending on whether the defocus amount is positive or negative. It can also be seen that the positional deviation occurs according to the imaging relationship (geometric relationship) of the imaging optical system depending on the defocus amount. The parallax amount, which is the positional deviation between the first image signal S1 and the second image signal S2, can be detected by a region-based matching method described later. <Defocused image generation process>

[0058] A defocus generating section 141 in the image processing section 14 generates a defocus image from the two types of image signals obtained. FIG. 7 is a flowchart showing a defocused image generation process executed by the defocused image generation unit 141 according to the first embodiment. The process relating to the defocused image generation will be described with reference to the flowchart of FIG.

[0059] In step S1401, the defocus generator 141 reads out and acquires from the image sensor an image (first image signal and second image signal S2) captured for a predetermined exposure period by an imaging optical system having an anti-vibration optical system. Then, light amount correction processing is performed on the first image signal and the second image signal S2. At the peripheral angle of view of the imaging optical system 10, the first pupil region 510 and the second pupil region 520 have different shapes due to vignetting, and the light amount balance is lost between the first image signal S1 and the second image signal S2. Therefore, in this step, the defocus generator 141 performs light amount correction on the first image signal S1 and the second image signal S2 using a light amount correction value stored in advance in the storage unit 15 as a memory, for example.

[0060] In step S1402, the defocus generator 141 performs processing to reduce noise generated during conversion in the image sensor 11. Specifically, the defocus generator 141 achieves noise reduction by applying filter processing to the first image signal S1 and the second image signal S2. In general, the higher the spatial frequency is, the lower the S / N ratio is, and the more noise components there are.

[0061] Therefore, the defocus generator 141 performs a process of applying a low-pass filter whose pass rate decreases as the spatial frequency increases. Note that the light amount correction in step S1401 may not be sufficient depending on manufacturing errors of the imaging optical system 10, so in step S1402, it is desirable to apply a band-pass filter that blocks DC components and has a low pass rate for high-frequency components.

[0062] In step S1403, the defocus generator 141 derives the amount of parallax between the first image signal S1 and the second image signal S2 based on these images. Specifically, the defocus generator 141 sets, in the first image signal S1, a point of interest corresponding to the representative pixel information and a matching region centered on the point of interest. The matching region is, for example, a rectangular region such as a square region centered on the point of interest and with one side having a predetermined length. Next, the defocus generator 141 sets a reference point in the second image signal S2, and sets a reference region centered on the reference point.

[0063] The reference area has the same size and shape as the above-mentioned collation area. The defocus generation unit 141 derives the degree of correlation between the image included in the collation area of ​​the first image signal S1 and the image included in the reference area of ​​the second image signal S2 while sequentially moving the reference point. Then, the reference point with the highest degree of correlation is specified as the corresponding point in the second image signal S2, which corresponds to the point of interest. The relative positional deviation amount between the corresponding point and the point of interest calculated in this way becomes the parallax amount at the point of interest.

[0064] The defocus generating unit 141 calculates the amount of parallax while sequentially changing the attention point according to the representative pixel information in this manner, thereby deriving the amount of parallax at a plurality of pixel positions determined by the representative pixel information. For simplicity, in this embodiment, in order to obtain defocus information at the same resolution as the image for display, the number of pixel positions for calculating the amount of parallax (the number of pixel groups included in the representative pixel information) is set to be the same as the number of pixels in the image for display. Note that, as a method for deriving the degree of correlation, methods such as NCC (Normalized Cross-Correlation), SSD (Sum of Squared Difference), and SAD (Sum of Absolute Difference) may be used.

[0065] Furthermore, by using a predetermined conversion coefficient, the derived parallax amount d can be converted into a defocus amount, which is distance information related to the distance from the image sensor 11 to the focal point of the imaging optical system 10. Here, the process of step S1403 functions as a distance information acquisition means for acquiring distance information related to the distance to the subject based on the image. If a given conversion coefficient K and a defocus amount ΔL are used, then

[0066]

number

[0067] The defocus generating unit 141 generates two-dimensional information in which the defocus amount for each pixel derived in this way is used as a pixel value, and stores the information in a memory (not shown) as a defocus image. That is, the distance information can be acquired as a two-dimensional distance map. The defocus image obtained in this way can be used for subject distance measurement, etc., as a map of the defocus amount.

[0068] FIG. 8 is a flowchart showing a process related to distance measurement according to the first embodiment. The flow of distance measurement according to the first embodiment will be described with reference to FIG. FIG. 8A is a flowchart of distance measurement in the first embodiment, which is processing carried out by the measurement unit 13 in FIG.

[0069] In the shooting information reading process in step S101, parameters such as the focal length, focus position, aperture value, shooting magnification, etc., which are added to the image as metadata, are read, and position information and phase information of the vibration isolation group are read. Each parameter is used in post-processing when calculating the object distance from the defocus amount, and is saved in a storage area.

[0070] The position information and phase information of the vibration-proof group are the drive amount and drive direction of the vibration-proof optical system during the period when the image is captured. That is, for example, the average value of the drive amount and drive direction of the vibration-proof optical system during the exposure period (image capture period) of the image captured to perform distance measurement using an image capture surface phase difference distance measurement method or the like is acquired. The same applies when the period when the image is captured is limited by a shutter or the like. In this manner, in this embodiment, the distance information is corrected based on aberration information corresponding to the amount of drive of the vibration-proof optical system during the period in which an image for distance measurement is captured.

[0071] In the pre-correction distance information generation process in step S102, distance information is generated that includes the fluctuation of the field curvature due to the image stabilization group. The defocus amount is calculated by the above-mentioned image plane phase difference distance measurement method or the like, and pre-correction object distance information is calculated using Equation 1, Equation 2, and the parameters obtained in step S101.

[0072] In step S103, a distance correction amount calculation process is performed. The details of the process in step S103 are shown in the flowchart of FIG. As shown in Fig. 8(B), position information of the vibration isolation group is extracted in step S1031. Here, position information is information regarding the drive amount and phase of the vibration isolation group, that is, the distance from the optical axis and the drive direction, as shown in Fig. 3.

[0073] In step S1032, data of a plurality of correction tables for the vibration isolation group is read, for example, from the storage unit 15. In this embodiment, the correction table is a table that includes data based on aberration information corresponding to the drive amount of the vibration isolation group, as well as data based on aberration information corresponding to at least one parameter such as focal length, focus position, aperture value, shooting magnification, etc. Also, in this embodiment, the data of the correction table is data on the defocus amount corresponding to the field curvature (aberration information) generated by the vibration isolation group.

[0074] Since the amount of data increases when each parameter is divided finely and a correction table is stored, each parameter is roughly divided and a correction table is stored, and a fine correction table is calculated by linear interpolation between the parameters. The defocus amount may be expressed by a polynomial with the optical axis as the center and the image height direction as a variable. The contents of the correction table stored in the memory are not limited to the defocus amount, and may be data related to aberration information, etc.

[0075] In this embodiment, a plurality of correction tables are prepared in advance for each combination of parameters. That is, the memory has, for example, a correction table based on the sagittal aberration and a correction table based on the meridional aberration for each combination of parameters. This reduces the amount of data in the correction tables, prevents the memory area for storing data from becoming bloated, and reduces the cost of the distance measuring device.

[0076] In step S1033, a process is performed to mix the data of the multiple correction tables extracted in step S1032. For example, a process is performed to mix the data of the correction table in the sagittal direction and the data of the correction table in the meridional direction according to a parameter corresponding to the phase (drive direction) of the image stabilization group.

[0077] Specifically, when the angle between the parallax detection direction and the drive direction of the vibration isolation group is taken as the drive phase θ as shown in Fig. 3, the sagittal and meridional tables are multiplied by sin θ and cos θ, respectively, and the data of the correction table is added to generate a corrected correction table. Note that, although the above-mentioned device is used in this embodiment to significantly reduce the amount of data, if there is sufficient memory capacity, multiple tables corresponding to the phase (drive direction) of the vibration isolation group may be stored in addition to the above parameters.

[0078] In step S1034, a process is performed to convert the corrected correction table calculated in step S1033 into distance. This process makes it possible to calculate the distance correction amount for correcting errors caused by the field curvature generated by image stabilization. In step S104, a correction process for distance information is performed. By adding the distance correction amount calculated in step S103 to the pre-correction object distance information calculated in step S102, it is possible to calculate corrected object distance information from which the field curvature component generated by the vibration reduction group has been removed. That is, in this embodiment, the object distance information is corrected based on aberration information corresponding to the drive amount of the vibration reduction optical system and the drive direction of the vibration reduction optical system.

[0079] In step S105, the corrected subject distance information calculated in step S104 is stored in a storage area. As described above, according to this embodiment, accurate distance measurement is possible even when shooting is performed while the vibration isolation mechanism is operating. EXAMPLES

[0080] Next, FIG. 9 is a flowchart showing distance measurement processing according to the second embodiment of the present invention. The flow of the distance measurement device according to the second embodiment will be described with reference to FIG. FIG. 9A is a flow chart for distance measurement in the second embodiment, which is carried out in the measurement unit 13 in FIG. 4 in the same manner as in the first embodiment.

[0081] The flowchart of the second embodiment shown in FIG. 9 is obtained by adding a flowchart of a contrast direction selection process to the first embodiment. In the second embodiment, the pixel of the image sensor includes four photoelectric conversion units, and the pixel configuration allows not only horizontal pupil division but also vertical pupil division. The four-division configuration enables detection of an object in either the horizontal or vertical direction. That is, the direction of the two exit pupil regions used for phase difference detection can be switched between the horizontal direction and the vertical direction.

[0082] Steps S201 and S202 are the same processes as steps S101 and S102 in the first embodiment. In step S203, the contrast direction is selected by referring to the image information. The process of step S203 is shown in the flowchart of FIG.

[0083] As shown in Fig. 9(B), in step S2031, an image is read. In step S2032, it is determined from the image whether the contribution of the horizontal contrast component or the vertical contrast component is greater at the edge of the image. In step S2033, either horizontal ranging (distance measurement in the horizontal direction) or vertical ranging (distance measurement in the vertical direction) is selected for each pixel. In step S2034, a map is generated with the selected ranging direction. That is, for example, the ranging direction can be switched between the horizontal and vertical directions depending on the contrast characteristics of the subject, and the aberration information for correcting the distance information is changed depending on the switching between the horizontal and vertical directions.

[0084] Fig. 10 is a diagram showing an example of edge directions in Example 2. In the case of an image having edges A and B as shown in Fig. 10, since edge A is mainly composed of vertical components, horizontal distance measurement is selected for the corresponding edge portion, and since edge B is mainly composed of horizontal components, vertical distance measurement is selected, and a distance measurement direction map is generated and saved.

[0085] As with embodiment 1, the horizontal ranging pixels use the angle between the horizontal direction and the drive direction of the vibration isolation group as the drive phase θ, multiply the sagittal and meridional tables by sin θ and cos θ, respectively, and add the tables to create a correction table. On the other hand, the vertical ranging pixels use the angle between the vertical direction and the drive direction of the vibration isolation group as the phase φ, multiply the sagittal and meridional tables by sin φ and cos φ, respectively, and add the tables to create a correction table. In this way, in this embodiment, the distance information is corrected based on different aberration information depending on the contrast characteristics of the subject in the image, etc., so that highly accurate distance measurement can be achieved.

[0086] In step S203, the distance measurement direction may be determined not from the image, but from a map in which the distance measurement direction is selected using information on the distance measurement direction stored as image information. For example, whether vertical or horizontal distance measurement has been performed for each pixel is determined during distance measurement, and the result is stored as two-dimensional information. By determining whether vertical or horizontal distance measurement has been performed based on this two-dimensional information, it becomes unnecessary to sequentially calculate the contrast of the image, and therefore the processing speed can be increased.

[0087] Steps S204 to S206 are the same processes as steps S103 to S105 in the embodiment 1. The contents of the process in step S204 are shown in the flowchart of Fig. 9(C). Steps S2041 to S2044 in the flowchart of Fig. 9(C) are the same as steps S1031 to S1034 in Fig. 8(B), so the explanation will be omitted. According to the second embodiment, when the image is captured with an effective vibration isolation mechanism, accurate distance measurement is possible even for a subject in any contrast direction. [Other Examples]

[0088] The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications are possible based on the gist of the present invention, and these modifications are not excluded from the scope of the present invention. A computer program for implementing all or part of the control in this embodiment may be supplied to the distance measurement device or the like via a network or various storage media. Then, a computer (or a CPU, MPU, or the like) in the distance measurement device or the like may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention.

Claims

1. an image information acquiring means for acquiring an image captured by an imaging optical system having an anti-vibration optical system; a distance information acquisition means for acquiring distance information relating to a distance to a subject based on the image; a distance correction unit that corrects the distance information based on aberration information corresponding to a drive phase and a drive amount of the vibration-proof optical system, the distance correction means includes a correction table based on the aberration information in a sagittal direction and a correction table based on the aberration information in a meridional direction, A distance measurement device characterized in that the distance correction means corrects the distance information using data from the correction table in the sagittal direction and data from the correction table in the meridional direction based on the drive phase and the drive amount of the vibration-proof optical system.

2. 2. The distance measurement device according to claim 1, wherein the vibration-proof optical system performs vibration proofing by driving at least one optical element in a direction perpendicular to an optical axis or by tilting the optical element with respect to the optical axis.

3. 2. The distance measurement device according to claim 1, wherein the distance correction means corrects the distance information based on the aberration information corresponding to at least one of a focal length, a focus position, an aperture value, and a shooting magnification of the imaging optical system.

4. 2. The distance measurement device according to claim 1, wherein the distance correction means corrects the distance information based on the aberration information corresponding to the driving amount of the vibration-proof optical system during the period in which the image is captured.

5. 2. The distance measurement device according to claim 1, wherein the distance correction means includes a memory that stores at least one correction table based on the aberration information corresponding to the driving amount of the vibration-proof optical system.

6. 2. The distance measurement device according to claim 1, wherein the distance correction means corrects the distance information based on the aberration information corresponding to the drive amount of the vibration-proof optical system and a drive direction of the vibration-proof optical system.

7. 2. The distance measurement device according to claim 1, wherein the distance correction means corrects the distance information based on the aberration information that differs depending on the subject.

8. 2. The distance measurement device according to claim 1, wherein the distance information acquisition means acquires the distance information by calculating a phase difference between images passing through two exit pupil regions shifted in a predetermined direction.

9. 9. The distance measurement device according to claim 8, wherein the distance information acquisition means is capable of switching between a horizontal direction and a vertical direction as the predetermined direction.

10. 10. The distance measurement device according to claim 9, wherein, when the distance information acquisition means switches the predetermined direction, the distance correction means changes the aberration information for correcting the distance information.

11. 10. The distance measurement device according to claim 9, wherein the distance information acquisition means switches between the horizontal direction and the vertical direction as the predetermined direction depending on the subject.

12. 2. The distance measurement device according to claim 1, wherein the distance information acquisition means acquires the distance information as a two-dimensional distance map.

13. 2. The distance measurement device according to claim 1, wherein the image information acquisition means includes an image pickup means for picking up an image of a subject.

14. A computer program for controlling each means of the distance measurement device according to any one of claims 1 to 13 by a computer.

15. an image information acquiring step of acquiring an image captured by an imaging optical system having an anti-vibration optical system; a distance information acquisition step of acquiring distance information relating to a distance to a subject based on the image; a distance correction step of correcting the distance information based on aberration information corresponding to a drive phase and a drive amount of the vibration-proof optical system, the distance correction step is a step of using a correction table based on the aberration information in a sagittal direction and a correction table based on the aberration information in a meridional direction, A distance measurement method characterized in that the distance correction process corrects the distance information using data from the correction table in the sagittal direction and data from the correction table in the meridional direction based on the drive phase and the drive amount of the vibration-proof optical system.

Images