Image pickup apparatus and control method thereof

Abstract

IMAGE PICKUP APPARATUS CAPABLE OF MORE FLEXIBLY CONTROLLING RANGE OF OPTICAL LOW-PASS FILTER EFFECT THAT CAN BE ACHIEVED WITHOUT USING OPTICAL LOW-PASS FILTER, AND CONTROL METHOD THEREFOR SOLUTION: An imaging apparatus includes a first movable member and a second movable member which are movable in a direction orthogonal to an optical axis of an imaging optical system and exist in an optical path. The imaging apparatus controls a first driving means for moving the first movable member and a second driving means for moving the second movable member so as to realize a function of a pseudo optical low-pass filter by periodically moving the first movable member and the second movable member in a direction orthogonal to the optical axis.SELECTED DRAWING: Figure 1B

Description

[Technical Field]

[0001] The present invention relates to an imaging apparatus and a control method thereof. [Background technology]

[0002] Image capture devices that use an image sensor with a pixel array in which multiple pixels are arranged two-dimensionally generally have an optical low-pass filter located on the subject side of the imaging surface to suppress false colors and moiré.

[0003] However, other techniques have also been proposed to achieve the same effect as an optical low pass filter (hereinafter referred to as the LPF effect).Patent Document 1 achieves the same effect as an optical low pass filter by driving the image sensor during exposure so that it traces a circular locus in a plane perpendicular to the optical axis. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-220993 Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, the magnitude of the LPF effect is controlled by adjusting the radius of the circular locus of drive. However, the adjustable range of the LPF effect is limited by the drive capability of the image sensor.

[0006] In view of the problems with the conventional technology, one aspect of the present invention provides an imaging apparatus that allows for more flexible control of the range of optical low pass filter effects that can be achieved without using an optical low pass filter, and a control method for the imaging apparatus. [Means for solving the problem]

[0007] In one aspect, the present invention provides an imaging device comprising: a first movable member and a second movable member located in an optical path and movable in a direction perpendicular to the optical axis of an imaging optical system; a first driving means for moving the first movable member; a second driving means for moving the second movable member; and a control means for controlling the operation of the first driving means and the second driving means, wherein the control means controls the first driving means and the second driving means so as to realize the function of a pseudo optical low pass filter by periodically moving the first movable member and the second movable member in the direction perpendicular to the optical axis. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide an imaging apparatus and a control method thereof that can more flexibly control the range of optical low-pass filter effects that can be achieved without using an optical low-pass filter. [Brief explanation of the drawings]

[0009] [Figure 1A] 1 is a vertical cross-sectional view of a digital camera according to an embodiment; [Figure 1B] FIG. 1 is a block diagram showing an example of the functional configuration of a digital camera according to an embodiment; [Figure 2A] A diagram showing an example of temporal changes in the incident position of a light beam due to LPF drive. [Figure 2B] FIG. 10 is a diagram showing an example of spatial frequency response by LPF driving. [Figure 3] FIG. 10 is a diagram showing an example of LPF driving in an embodiment. [Figure 4] FIG. 10 is a diagram showing another example of LPF driving in the embodiment. [Figure 5] FIG. 10 is a diagram showing an example of temporal change in the incident position of a light beam that can be achieved by driving an LPF in an embodiment. [Figure 6] 10 is a flowchart illustrating an example of a phase matching process for drive timing according to an embodiment. [Figure 7] FIG. 10 is a diagram showing an example of the frequency response of a movable member used for driving an LPF. DETAILED DESCRIPTION OF THE INVENTION

[0010] The present invention will be described in detail below based on exemplary embodiments with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Furthermore, although multiple features are described in the embodiments, not all of them are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] The following description will be given of an embodiment of the present invention using a digital camera. However, the present invention can be implemented in any electronic device having an imaging function. Such electronic devices include video cameras, computer devices (personal computers, tablet computers, media players, PDAs, etc.), smartphones, game consoles, robots, drones, and drive recorders. However, these are merely examples, and the present invention can also be implemented in other electronic devices.

[0012] 1A is a vertical cross-sectional view of a digital camera 1000 (hereinafter simply referred to as camera 1000). Here, the vertical cross-section including the optical axis 4 of the imaging optical system 3 is shown.

[0013] The camera 1000 is composed of a camera body 1 and a lens unit 2 that is detachable from the camera body 1. The camera body 1 and lens unit 2 can communicate with each other via electrical contacts 13 that come into contact when the lens unit 2 is attached to the camera body 1. The electrical contacts 13 are also used to supply the power required for the operation of circuits within the lens unit 2 from the camera body 1 to the lens unit 2. The lens unit 2 may also be fixed to the camera body 1.

[0014] The camera body 1 has an image sensor 6 that converts an optical image of a subject formed by an imaging optical system 3 into electrical pixel signals through photoelectric conversion processing. The camera body 1 also has a peer-type electronic viewfinder (EVF) 10a. The EVF 10a has a display device such as an LCD and a finder optical system that enlarges the display screen of the display device. Although not shown in FIG. 1A, the camera body 1 may also have a display device provided on the surface of the housing.

[0015] The lens unit 2 is formed as a so-called interchangeable lens. The lens unit 2 has an imaging optical system 3 that forms an optical image of a subject. The imaging optical system 3 can have multiple movable lenses, such as a focus lens, an anti-vibration lens 16, and a zoom lens. 4 indicates the optical axis of the imaging optical system 3. Of the movable lenses, the anti-vibration lens 16 is movable in a direction perpendicular to the optical axis 4, and the focus lens and zoom lens are movable in a direction along the optical axis 4. The movable lenses are driven by motors, actuators, and the like that the lens unit 2 has. In addition, the position of the movable lens in the movement direction can be detected by a sensor or the like that the lens unit 2 has.

[0016] FIG. 1B is a block diagram showing an example of the functional configuration of camera 1000. The functional blocks shown in FIG. 1B can be implemented by software or a combination of software and hardware, except for parts that can clearly be realized only by hardware (e.g., memory 8, display device 10, electrical contacts 13, etc.). For example, the functional blocks may be realized by dedicated hardware such as an ASIC. Alternatively, the functional blocks may be realized by a processor such as a CPU executing a program stored in memory. Note that multiple functional blocks may be realized by a common configuration (e.g., one ASIC). Also, hardware that realizes part of the functions of one functional block may be included in hardware that realizes another functional block.

[0017] The system control unit 5 has, for example, a processor (CPU, MPU, microprocessor, etc.) capable of executing programs, and non-volatile memory. The system control unit 5 loads programs stored in the non-volatile memory into memory 8 and executes them on the processor, thereby controlling the operation of each unit of the camera 1000 and realizing the functions of the camera 1000. The system control unit 5 also controls the operation of the lens unit 2 by communicating with the lens control unit 14 via electrical contacts 13.

[0018] The memory 8 is used as a main memory used by the processor of the system control unit 5, a buffer for temporarily storing image data, a work memory for temporarily storing data being processed by the image processing unit, etc. Furthermore, a part of the memory 8 may be used as a video memory for storing image data to be displayed on the display device 10.

[0019] The image sensor 6 may be, for example, a known CCD or CMOS color image sensor with a primary-color Bayer array of color filters. The image sensor 6 has a pixel array in which multiple pixels are arranged two-dimensionally, and peripheral circuits for reading out signals from each pixel. Each pixel has a photoelectric conversion region and accumulates electric charge according to the amount of incident light. By reading out signals from each pixel having a voltage according to the amount of electric charge accumulated during the exposure period, a group of pixel signals (analog image signals) representing the subject image formed on the imaging surface is obtained.

[0020] In this embodiment, the photoelectric conversion region of each pixel in the image sensor 6 is divided into multiple regions. Furthermore, signals can be read out for each divided photoelectric conversion region (sub-pixel). For example, if one pixel has sub-pixels A and B, the image signals read out from the sub-pixel group A and the image signals read out from the sub-pixel group B form a parallax image pair. Therefore, for multiple pixels in the focus detection region, autofocus detection using the phase difference detection method is possible using the image signals read out from the sub-pixel group A and the image signals read out from the sub-pixel group B.

[0021] The image processing unit 7 applies predetermined image processing to the analog image signal read from the image sensor 6 to generate signals and image data according to the application, and acquires and / or generates various information. The image processing unit 7 may be a dedicated hardware circuit such as an ASIC (Application Specific Integrated Circuit) designed to realize a specific function. Alternatively, the image processing unit 7 may be configured such that a processor such as a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit) executes software to realize a specific function. The image processing unit 7 outputs the acquired or generated information and data to the system control unit 5, memory 8, focus detection unit 11, etc., depending on the application.

[0022] The image processing applied by the image processing unit 7 can include, for example, pre-processing, color interpolation processing, correction processing, detection processing, data processing, evaluation value calculation processing, special effect processing, and the like. The pre-processing may include A / D conversion, signal amplification, reference level adjustment, defective pixel correction, and the like. Color interpolation, also known as demosaicing, is performed when the image sensor is equipped with a color filter, and is a process of interpolating the values ​​of color components that are not included in the individual pixel data that make up the image data. The correction processing may include white balance adjustment, tone correction, correction of image degradation caused by optical aberration of the imaging optical system 3 (image restoration), correction of the effects of peripheral light falloff of the imaging optical system 3, color correction, and the like. The detection process may include detection of characteristic regions (for example, face regions or human body regions) and their movements, person recognition processing, and the like. Data processing can include processes such as area extraction (trimming), compositing, scaling, encoding and decoding, header information generation (data file generation), etc. Data processing also includes the generation of image data for display or image data for recording. The evaluation value calculation process can include processes such as generating signals and evaluation values ​​used in autofocus (AF) detection, and generating evaluation values ​​used in automatic exposure control (AE). Special effect processing can include adding a blur effect, changing color tones, relighting, and the like. It should be noted that these are examples of processes that can be applied by the image processing unit 7, and do not limit the processes that can be applied by the image processing unit 7.

[0023] The focus detection unit 11 calculates the defocus amount of the focus detection area using the AF signal of the phase difference detection method generated by the image processing unit 7. The focus detection unit 11 outputs the defocus amount to the system control unit 5.

[0024] The system control unit 5 transmits focus adjustment information, including the defocus amount supplied from the focus detection unit 11, to the lens control unit .

[0025] The camera motion sensor 18 is a sensor that detects the motion of the camera body 1. The camera motion sensor 18 may be a combination of an acceleration sensor that detects motion along each of three orthogonal axes and a gyro sensor that detects motion around each axis. The camera motion sensor 18 outputs a signal representing the detected motion to the system control unit 5.

[0026] When the image stabilization function for moving the image sensor 6 is enabled, the system control unit 5 generates a movement command for the image sensor 6 to offset the movement of the camera body 1 based on a signal obtained from the camera motion sensor 18. The movement command includes, for example, information indicating the amount of movement for each movement direction. The system control unit 5 outputs the generated movement command to the sensor drive unit 12. The sensor drive unit 12 has a movement mechanism that moves the image sensor 6 in a plane perpendicular to the optical axis 4. The sensor drive unit 12 operates the movement mechanism in accordance with the movement command supplied from the system control unit 5 to move the image sensor 6.

[0027] When the LPF effect is to be achieved by moving the image sensor 6, the system control unit 5 also generates a movement command for the image sensor 6. The movement command may include, for example, information indicating the movement amount for each movement direction, as well as information indicating the movement period or frequency. The system control unit 5 outputs the generated movement command to the sensor driving unit 12 as an instruction to start LPF drive. The sensor driving unit 12 operates the movement mechanism in accordance with the movement command supplied from the system control unit 5 to move the image sensor 6.

[0028] The system control unit 5 also generates a movement command for the vibration-reduction lens 16 when the LPF effect is to be achieved by moving the vibration-reduction lens 16. The movement command may include information indicating the movement amount for each movement direction, as well as information indicating the movement period or frequency. The system control unit 5 outputs the generated movement command to the lens unit 2 as an instruction to start LPF drive. The lens control unit 14 transfers the received instruction to start LPF drive to the vibration-reduction lens driving unit 15. The vibration-reduction lens driving unit 15 operates the movement mechanism in accordance with the movement command supplied from the lens control unit 14, thereby moving the vibration-reduction lens 16.

[0029] If the drive amplitude and drive cycle or frequency of the image sensor 6 and the vibration-proof lens 16 in the LPF drive are fixed, the LPF drive start instruction does not need to include a movement command. The sensor drive unit 12 and the vibration-proof lens drive unit 15 move the image sensor 6 and the vibration-proof lens 16, respectively, at a preset drive amplitude and drive cycle or frequency.

[0030] The camera operation unit 9 is a collective term for input devices (such as buttons, switches, and dials) provided for the user to input various instructions to the camera 1000. The input devices constituting the camera operation unit 9 are named according to their assigned functions. For example, the camera operation unit 9 includes a release switch, a video recording switch, a shooting mode selection dial for selecting a shooting mode, a menu button, directional keys, and a confirm key. The release switch is a switch for recording still images, and the system control unit 5 recognizes a half-pressed state of the release switch as an instruction to prepare for shooting and a full-pressed state as an instruction to start shooting. The system control unit 5 also recognizes a press of the video recording switch in shooting standby mode as an instruction to start video recording, and a press of the video recording switch during video recording as an instruction to stop recording. Note that the functions assigned to the same input device may be variable. The input device may also be software buttons or keys using a touch display. The camera operation unit 9 may also include an input device compatible with non-contact input methods such as voice input and eye-gaze input.

[0031] Display device 10 is a general term for display devices provided in camera body 1, and includes a display device included in EVF 10a and a display device provided on the back or top of the housing of camera body 1. Display device 10 displays live view images, playback images, menu screens, settings and information of camera 1000, etc.

[0032] The lens motion sensor 19 is a sensor that detects the movement of the lens unit 2. The lens motion sensor 19 may be a combination of an acceleration sensor that detects movement along each of three orthogonal axes and a gyro sensor that detects movement around each axis. The lens motion sensor 19 outputs a signal representing the detected movement to the lens control unit 14.

[0033] Lens control unit 14 is, for example, a one-chip microcomputer, and includes a processor (CPU, MPU, microprocessor, etc.) capable of executing programs, ROM, RAM, a communication interface, etc. Lens control unit 14 acquires information about and controls the operation of each part of lens unit 2 by loading programs stored in ROM into RAM and executing them with the processor. Lens control unit 14 also drives the focus lens and transmits information about lens unit 2 to system control unit 5 in response to requests received from system control unit 5 through communication with system control unit 5 via electrical contacts 13.

[0034] For example, lens control unit 14 controls focus lens driving unit 17 based on focus adjustment information received from system control unit 5 to move the focus lens in the direction of optical axis 4. Lens control unit 14 also controls anti-vibration lens driving unit 15 based on drive information received from system control unit 5 to move anti-vibration lens 16 in a direction perpendicular to optical axis 4.

[0035] When the image stabilization function for moving the vibration-reduction lens 16 is enabled, the lens control unit 14 generates a movement command for the vibration-reduction lens 16 to offset the movement of the lens unit 2 based on a signal obtained from the lens movement sensor 19. The movement command includes, for example, information indicating the amount of movement for each movement direction. The lens control unit 14 outputs the generated movement command to the vibration-reduction lens driving unit 15. The vibration-reduction lens driving unit 15 has a movement mechanism that moves the vibration-reduction lens 16 in a plane perpendicular to the optical axis 4. The vibration-reduction lens driving unit 15 operates the movement mechanism in accordance with the movement command supplied from the lens control unit 14 to move the vibration-reduction lens 16.

[0036] Furthermore, when the lens control unit 14 receives an instruction to start driving the LPF from the system control unit 5, it transfers the instruction to the vibration-proof lens driving unit 15. The vibration-proof lens driving unit 15 operates the movement mechanism in accordance with the start instruction transferred from the lens control unit 14 to move the vibration-proof lens 16.

[0037] Although a motion sensor is provided in each of the camera body 1 and the lens unit 2, a motion sensor may be provided in only one of them. When there is only one motion sensor, a signal representing the motion may be supplied to both the system control unit 5 and the lens control unit 14. Alternatively, the system control unit 5 or the lens control unit 14, to which a signal is supplied from the motion sensor, may generate movement commands for both the image sensor 6 and the vibration-proof lens 16.

[0038] (LPF driven) Next, the LPF drive in the camera 1000 will be described. LPF drive is a function that, instead of using an optical low-pass filter, periodically moves a movable member present in the optical path in a direction perpendicular to the optical axis of the imaging optical system, thereby achieving a pseudo-optical low-pass filter effect. In this embodiment, a movable member used for image blur correction is used for LPF drive. This eliminates the need to add a member dedicated to LPF drive. In this embodiment, both the image sensor 6 and the vibration-proof lens 16 of the camera 1000 can be used for LPF drive as movable members used for image blur correction.

[0039] For example, when LPF drive is enabled and an instruction to prepare for still image capture (half-pressing the release button included in the camera operation unit 9) is detected, the system control unit 5 instructs the sensor drive unit 12 and the lens unit 2 to start LPF drive.

[0040] When the sensor driving unit 12 receives the instruction to start LPF driving, it starts LPF driving of the image sensor 6. Parameters of the LPF driving (e.g., movement direction, amplitude, and period) may be included in the start instruction or may be set in advance in the sensor driving unit 12.

[0041] When the lens control unit 14 receives an instruction to start LPF driving from the system control unit 5, it transfers the instruction to the vibration-proof lens driving unit 15. When the vibration-proof lens driving unit 15 receives the instruction to start LPF driving, it starts LPF driving of the vibration-proof lens 16. Parameters for LPF driving (e.g., movement direction, amplitude, and period) may be included in the start instruction, or may be set in advance in the vibration-proof lens driving unit 15.

[0042] The system control unit 5 executes shooting preparation processing, including AF processing and AE processing, based on an image obtained by shooting while the LPF is driven. When the shooting preparation processing is completed, the system control unit 5 may instruct the LPF drive to stop. Thereafter, when a shooting start instruction (full press of the release button) is detected without the release button being released from its half-pressed state, the system control unit 5 instructs the sensor drive unit 12 and the lens unit 2 to start LPF drive. Note that the system control unit 5 does not need to stop LPF drive even after the shooting preparation processing is completed, unless the shooting preparation instruction is released.

[0043] Note that LPF drive can be performed not only when recording still images but also when recording moving images. When LPF drive is enabled, upon detecting that the moving image recording switch has been pressed in shooting standby mode, the system control unit 5 sends an instruction to start LPF drive to the sensor drive unit 12 and the lens unit 2.

[0044] Note that LPF driving is performed during the shooting preparation operation and exposure period. This is mainly to save power consumption. There are no particular restrictions on the period during which LPF driving is performed, and for example, LPF driving may be performed constantly while the camera 1000 is operating in shooting mode.

[0045] FIG. 2A is a schematic diagram showing an example of the relationship between image movement due to LPF driving and pixel size (or pixel pitch) in the horizontal direction of the pixel array. The upper part shows six pixels arranged in the horizontal direction among the pixels of the image sensor 6. The image sensor 6 has color filters in a primary color Bayer array, and in this example, the pixels are shown with green (G) and red (R) color filters arranged alternately. Each pixel 6P is divided into sub-pixels A and B. GA is the sub-pixel A provided with a green color filter, and will be referred to as a GA pixel below. RB is the sub-pixel B provided with a red color filter, and will be referred to as an RB pixel below. The same applies to the GB and RA pixels.

[0046] The waveform shown at the bottom shows the change over time in the incident position due to LPF drive for a light beam incident on the horizontal center of the GA pixel indicated by a ▼ when no LPF drive is performed. AF is the pixel pitch, T AF indicates the exposure period.

[0047] The waveform of the solid line shows that the period of the change in the incident position of the light beam due to LPF drive is T AF and the amplitude is d AF The dotted waveform indicates that the period of the change in the incident position of the light beam due to LPF drive is T AF and the amplitude is d AF Greater than.

[0048] Figure 2B is a diagram showing a schematic representation of the spatial frequency response of the signal detected by the image sensor 6 with respect to the change over time in the incident position of the light beam shown in Figure 2A. The response also reflects the MTF characteristics of the imaging optical system 3. The solid and dotted lines in Figure 2B correspond to the waveform notations in Figure 2A.

[0049] As shown in Figure 2B, as the amplitude of the change in the incident position over time decreases, the spatial frequency at which the response first becomes zero increases. This means that the frequency characteristics of the LPF effect can be controlled by controlling the amplitude of the LPF drive.

[0050] While the effect of LPF driving in the horizontal direction has been explained here, the same applies to LPF driving in the vertical direction. Furthermore, by driving the LPF so that the locus of the time change in the incident position becomes circular, it is possible to obtain the LFP effect in both the horizontal and vertical directions.

[0051] In this embodiment, by using both the image sensor 6 and the vibration-proof lens 16, which are movable members used for image blur correction, the controllable range of the amplitude of the LPF drive is expanded compared to when using only one of them, which is synonymous with expanding the adjustable range of the LPF effect.

[0052] 3 is a schematic diagram illustrating the principle of enhancing the LPF effect according to this embodiment. Reference numeral 301, like FIG. 2A, illustrates an example of the change over time in the incident position of the light beam due to LPF driving of the image sensor 6. Similarly, reference numeral 302 illustrates an example of the change over time in the incident position of the light beam due to LPF driving of the vibration-proof lens 16. Here, the image sensor 6 is LPF-driven so that the amplitude of the change over time in the incident position of the light beam is A, and the vibration-proof lens 16 is LPF-driven so that the amplitude of the change over time in the incident position of the light beam is B. Note that, while a case where A and B are equal is shown as an example here, A and B may be different.

[0053] It should be noted that the amplitude of the change in the incident position of the light beam over time when the vibration-reduction lens 16 is driven using the LPF may differ from the drive amount of the vibration-reduction lens. This is because the amplitude of the change in the incident position of the light beam over time reflects not only the drive amount of the vibration-reduction lens but also the magnification of the imaging optical system 3. Although not specifically described below, the lens control unit 14 or the system control unit 5 calculates the drive amount of the vibration-reduction lens 16 so that the amplitude reflecting the magnification of the imaging optical system 3 becomes the target amplitude. It should be noted that the magnification of the imaging optical system 3 is known as information specific to the lens unit 2 or can be obtained from the lens unit 2.

[0054] An adder 303 schematically shows the combining operation of 301 and 302. 304 shows an example of the change over time in the incident position of the light beam when the LPF drive of the image sensor 6 shown in 301 and the LPF drive of the vibration-proof lens 16 shown in 302 are performed in parallel.

[0055] Here, the time change 301 of the incident position of the light beam due to the LPF drive of the image sensor 6 and the time change 302 of the incident position of the light beam due to the LPF drive of the vibration-proof lens 16 are synchronized (the phase difference θ is 0). Therefore, the amplitude C of the time change of the incident position of the light beam shown as 304 is equal to the sum of A and B.

[0056] 4 is a schematic diagram illustrating the principle of reduction of the LPF effect according to this embodiment. Reference numeral 401, like FIG. 3, shows an example of the change over time in the incident position of the light beam due to LPF driving of the image sensor 6. Similarly, reference numeral 402 shows an example of the change over time in the incident position of the light beam due to LPF driving of the vibration-proof lens 16. Here, it is assumed that the image sensor 6 and the vibration-proof lens 16 are LPF-driven so that the amplitude of the change over time in the incident position of the light beam is A, but the amplitudes do not have to be equal.

[0057] An adder 403 schematically shows the combining operation of 401 and 402. 404 shows an example of the change over time in the incident position of the light beam when the LPF driving of the image sensor 6 shown in 401 and the LPF driving of the vibration-proof lens 16 shown in 402 are performed in parallel.

[0058] Here, the incident position D of the light beam due to LPF driving is expressed by the following equation (1) when the time is t. D = Asin (2π * t / T AF ) (1)

[0059] Furthermore, when the phase difference θ between the time change 401 of the incident position of the light beam due to the LPF drive of the image sensor 6 and the time change 402 of the incident position of the light beam due to the LPF drive of the vibration-proof lens 16 is not 0, the formula (1) is as follows: D = Asin (2π * t / T AF - θ) (1)' This becomes:

[0060] Therefore, when the LPF drive of the image sensor 6 shown in 401 and the LPF drive of the vibration-proof lens 16 shown in 402 are performed in parallel, the incident position D of the light beam is expressed by equation (2) if the phase difference between 401 and 402 is not 0. D = Asin (2π * t / T AF ) + Asin (2π * t / T AF - θ) (2)

[0061] 4, 402 has a phase difference of 8π / 10 with respect to 401. In this case, the amplitude of 404 (the maximum absolute value of D) is smaller than A.

[0062] In this way, by driving both the image sensor 6 and the vibration-proof lens 16 using the LPF so that the temporal change in the incident position of the light beam has a predetermined phase difference, the controllable range of the incident position of the light beam can be expanded compared to when driving only one of the image sensor 6 and the vibration-proof lens 16 using the LPF. According to the LPF driving of this embodiment, the controllable range of the incident position of the light beam can be expanded compared to when driving only one of the image sensor 6 and the vibration-proof lens 16 using the LPF. Therefore, the magnitude of the LPF effect can be controlled more flexibly.

[0063] Figure 5 shows the time change of the incident position D of the light beam when the value of the phase difference θ is changed under the conditions of Figure 4. In this way, by changing the phase difference θ between 0 and π, the amplitude of the time change of the incident position of the light beam can be continuously controlled in the range of 0 to 2A.

[0064] The phase difference in the temporal change in the incident position of the light beam can be controlled by the phase difference in the operation timing of the sensor driver 12 and the vibration-reduction lens driver 15. Therefore, the LPF drive of this embodiment can also be said to control the phase difference between the drive timing of the image sensor 6 by the sensor driver 12 and the drive timing of the vibration-reduction lens 16 by the vibration-reduction lens driver 15. To control the phase difference in the drive timing, it is necessary to use as a reference a state in which the drive timing of the sensor driver 12 and the vibration-reduction lens driver 15 are synchronized (phase difference 0). If the sensor driver 12 and the vibration-reduction lens driver 15 operate on a common clock, their operations can be synchronized based on the clock.

[0065] Alternatively, if the time difference between the instruction to start LPF drive from the system control unit 5 and the time when the sensor drive unit 12 and the vibration-proof lens drive unit 15 start LPF drive is negligible, the operations of the sensor drive unit 12 and the vibration-proof lens drive unit 15 may be considered to be synchronized. Specifically, if this time difference is, for example, equal to the exposure period T AF If the difference is less than the threshold determined based on the above, it may be considered that the operations of the sensor driver 12 and the vibration-proof lens driver 15 are synchronized.

[0066] For convenience, it is assumed that the difference in delay between the movement of the driven members (image sensor 6 and anti-vibration lens 16) relative to the drive control timing by sensor drive unit 12 and anti-vibration lens drive unit 15 is negligible.

[0067] Another method for synchronizing the drive timing of the sensor drive unit 12 and the vibration-proof lens drive unit 15 will be described using the flowchart shown in FIG. 6. The synchronization (phase alignment) process shown in FIG. 6 can be performed when the camera 1000 enters a shooting standby state after power-on, or at any timing before video shooting is possible and LPF drive starts. Note that to achieve accurate phase alignment, it is desirable for the movement of the camera 1000 to be small. Therefore, if the movement of the camera body 1 detected by the camera motion sensor 18 is equal to or greater than a predetermined amount, the system control unit 5 may display a message on the display device 10 warning the user to keep the camera 1000 still, or may cancel the phase alignment process.

[0068] In S2, the system control unit 5 starts LPF driving of the image sensor 6 and the vibration-proof lens 16 by outputting an instruction to start LPF driving to the sensor drive unit 12 and the lens unit 2. In the phase alignment process, the period and amplitude of the change over time in the incident position of the light beam due to the LPF drive of the image sensor 6 are made equal to the period and amplitude of the change over time in the incident position of the light beam due to the LPF drive of the vibration-proof lens 16. The system control unit 5 may include a movement command in the instruction to start LPF driving, as necessary.

[0069] Furthermore, the system control unit 5 changes the frame rate of the moving image as needed so that the frame period of the moving image captured for live view display is less than the period of LPF drive.

[0070] In S3, the system control unit 5 causes the image processing unit 7 to detect motion vectors between frames of the moving image captured for live view display. The motion vectors can be detected using a known method such as template matching.

[0071] In S4, the system control unit 5 calculates the amplitude (target amplitude) that should be obtained when synchronized (phase difference 0) based on the drive conditions of the image sensor 6 and the vibration-proof lens 16. The target amplitude can be calculated as the sum of the amplitudes of the temporal changes in the incident position obtained by driving each LPF.

[0072] Then, if the magnitude of the motion vector detected in S3 is equal to the target amplitude or is within a predetermined error range with respect to the target amplitude, the system control unit 5 determines that the operation timing of the sensor driving unit 12 and the vibration-proof lens driving unit 15 is synchronized.

[0073] If the system control unit 5 determines that the operation timings of the sensor driving unit 12 and the vibration-proof lens driving unit 15 are synchronized, the system control unit 5 ends the phase alignment process. On the other hand, if the system control unit 5 does not determine that the operation timings of the sensor driving unit 12 and the vibration-proof lens driving unit 15 are synchronized, the system control unit 5 executes S5.

[0074] In S5, the system control unit 5 generates a command to instruct one of the sensor driving unit 12 and the vibration-proof lens driving unit 15 to advance (or delay) the operation timing by a predetermined time (for example, a predetermined number of clocks).The system control unit 5 then transmits the generated command to the sensor driving unit 12 or the lens unit 2.

[0075] When the phase alignment process is completed, the system control unit 5 returns the shooting frame rate of the moving image for live view display. Completion of the phase alignment enables the above-described phase difference control to achieve the desired LPF effect.

[0076] As explained with reference to Figure 2B, the cutoff frequency in the LPF effect can be controlled by controlling the magnitude of the amplitude of the change in the incident position of the light beam over time. Therefore, the system control unit 5 can control the phase difference between the drive timing of the image sensor 6 and the vibration-proof lens 16 by the sensor drive unit 12 and the vibration-proof lens drive unit 15, depending on the desired cutoff frequency. The correspondence between the cutoff frequency and the phase difference can be measured in advance and stored in non-volatile memory.

[0077] The desired cutoff frequency can be determined so as to suppress the occurrence of moiré and false colors, for example. For example, spatial frequency components that may cause moiré and false colors can be determined by analyzing an image captured without driving the LPF using the image processing unit 7. The system control unit 5 can then control the phase difference between the drive timings of the sensor driving unit 12 and the vibration-proof lens driving unit 15 to achieve a cutoff frequency that removes the determined spatial frequency components.

[0078] The cutoff frequency may be set by the user via a menu screen, for example, or may be set in advance as a fixed value because the spatial frequency at which moiré or false colors may occur depends on the pixel pitch of the image sensor 6.

[0079] (Variation 1) Another application example of the phase difference control for LPF driving in this embodiment will be described below. Figure 7 shows an example of frequency response when driving a movable member. The horizontal axis is frequency in Hz, and the vertical axis is response in dB.

[0080] Generally, the frequency response of a moving part decreases as the driving frequency increases. In particular, if the driving frequency is included in the range where the response is below zero dB (the driving frequency is equal to or greater than frequency f), the moving part will not move as intended.

[0081] For example, if only one of the image sensor 6 and the vibration-proof lens 16 is driven in a manner such that the LPF driving cycle is equal to the exposure period T AF When driven to be equal to the exposure period T AF The shorter the exposure period T, the higher the driving frequency must be. However, if the driving frequency is greater than f, the desired driving amplitude cannot be obtained. AF If the drive frequency is less than the threshold, the desired LPF effect cannot be achieved by using only one of the image sensor 6 and the vibration-proof lens 16. Alternatively, if the drive frequency is equal to or greater than the threshold, the desired LPF effect cannot be achieved by using only one of the image sensor 6 and the vibration-proof lens 16.

[0082] Therefore, the exposure period T AF≧ the reciprocal of the driving frequency f AF Regarding (2), both the image sensor 6 and the vibration-proof lens 16 are driven, and the phase difference is controlled so that the amplitude increases.

[0083] As with the control of the cutoff frequency, the phase difference for obtaining a desired drive amplitude for the section where the drive frequency is equal to or greater than f (above the threshold) can be measured in advance and stored in nonvolatile memory. When the exposure period satisfies the above-mentioned conditions, the system control unit 5 controls the drive timing of the sensor drive unit 12 and the vibration-proof lens drive unit 15 so that the phase difference obtained by referring to the nonvolatile memory is obtained.

[0084] (Variation 2) Next, an example will be described in which the phase difference between the LPF drive of the image sensor 6 and the vibration-proof lens 16 is changed between focus detection and photography.

[0085] For example, when capturing still images, if a moving image for live view display is used for focus detection, the exposure period is determined so as to maintain the frame rate. On the other hand, the exposure period (shutter speed) during shooting is determined based on AE processing, user settings, etc. Therefore, the exposure period during focus detection and shooting may differ.

[0086] If the exposure period differs between focus detection and shooting, the drive frequencies of the image sensor 6 and the vibration-reduction lens 16 when driving the LPF also differ between focus detection and shooting. Therefore, the frequency responses of the image sensor 6 and the vibration-reduction lens 16 also differ between focus detection and shooting. For this reason, if the phase difference in the drive timing of the image sensor 6 and the vibration-reduction lens 16 by the sensor drive unit 12 and the vibration-reduction lens drive unit 15 during focus detection is also used during shooting, an appropriate LPF effect cannot be obtained for the captured image.

[0087] When the exposure period differs between focus detection and shooting, the system control unit 5 changes the phase difference between the drive timing of the image sensor 6 and the vibration-proof lens 16 between focus detection and shooting so that the cutoff frequency of the LPF effect does not change between focus detection and shooting.

[0088] Furthermore, the pixel pitch used during focus detection may differ from that used during image capture. For example, this may occur when pixels with photoelectric conversion areas divided into multiple discrete regions are provided. In this case, the pixel pitch used during focus detection is wider than that used during image capture. In this case, the drive amplitude of the LPF drive during focus detection must be larger than that during image capture in order to achieve the same LPF effect as during image capture.

[0089] When the pixel pitch used during focus detection differs from that used during shooting, the system control unit 5 changes the phase difference between the drive timing of the image sensor 6 and the vibration-proof lens 16 during focus detection and shooting so that the drive amplitude is such that the same LPF effect is obtained during focus detection and shooting.

[0090] In addition, if one or more of the exposure period and pixel pitch differ between focus detection and shooting, the system control unit 5 may differ not only in the phase difference of the drive timing but also in the drive cycle between focus detection and shooting.

[0091] (Other embodiments) In the above-described embodiment, for ease of explanation and understanding, the combination of LPF driving and image blur correction has not been described. However, LPF driving and image blur correction can be implemented in combination. For example, the system control unit 5 may output a movement command that combines the movement amount for LPF driving and the movement amount for image blur correction to the sensor drive unit 12 and the vibration-proof lens drive unit 15. Alternatively, the system control unit 5 may output movement commands to each of the image sensor 6 and the vibration-proof lens 15, so that one of the image sensor 6 and the vibration-proof lens 15 is responsible for LPF driving, and the other is responsible for image blur correction.

[0092] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0093] The disclosure of this embodiment includes the following imaging device, imaging device control method, and program. (Item 1) a first movable member and a second movable member that are movable in a direction perpendicular to the optical axis of the imaging optical system and that are present in the optical path; a first driving means for moving the first movable member; a second driving means for moving the second movable member; a control means for controlling the operation of the first driving means and the second driving means; the control means controls the first driving means and the second driving means to periodically move the first movable member and the second movable member in a direction perpendicular to the optical axis, thereby realizing a pseudo optical low pass filter function. (Item 2) The imaging device described in item 1, characterized in that the control means controls the first driving means and the second driving means to move the first movable member and the second movable member when movement of one of the first movable member and the second movable member is not enough to achieve the required optical low-pass filter function. (Item 3) Item 2. The imaging device according to item 2, wherein the case where the required optical low-pass filter function cannot be realized by moving either the first movable member or the second movable member is when the exposure period is less than a threshold. (Item 4) The imaging device described in item 2 is characterized in that the case in which the required optical low-pass filter function cannot be achieved by moving either the first movable member or the second movable member is when the drive frequency of the movable member is equal to or higher than a threshold. (Item 5) 5. The imaging device according to any one of items 1 to 4, wherein the control unit controls the phase difference between the timing at which the first driving unit drives the first movable member and the timing at which the second driving unit drives the second movable member in order to achieve the required optical low pass filter function. (Item 6) 6. The imaging device according to item 5, wherein the required function of the optical low-pass filter is a cutoff frequency. (Item 7) 7. The imaging device according to item 6, wherein the cutoff frequency is determined so as to remove spatial frequency components that cause moire or false color. (Item 8) The imaging device described in any one of items 1 to 7, characterized in that the control means executes the control of the first driving means and the second driving means during focus detection and shooting of still images. (Item 9) Item 9. The imaging device according to item 8, characterized in that, when the exposure period differs between the focus detection and the shooting, the control means changes the phase difference between the timing at which the first driving means drives the first movable member and the timing at which the second driving means drives the second movable member between the focus detection and the shooting. (Item 10) Item 10. The imaging device according to item 8 or 9, characterized in that, when the pixel pitch used during focus detection is different from that used during shooting, the control means changes the phase difference between the timing at which the first driving means drives the first movable member and the timing at which the second driving means drives the second movable member between the timing at which the focus detection is performed and the timing at which the shooting is performed. (Item 11) the first movable member is a movable lens included in the imaging optical system, the second movable member is an imaging element; 11. The imaging device according to any one of items 1 to 10, (Item 12) a first movable member and a second movable member that are movable in a direction perpendicular to the optical axis of the imaging optical system and that are present in the optical path; a first driving means for moving the first movable member; a second driving means for moving the second movable member, controlling the first driving means and the second driving means so as to realize a pseudo optical low pass filter function by periodically moving the first movable member and the second movable member in a direction perpendicular to the optical axis. (Item 13) a first movable member and a second movable member that are movable in a direction perpendicular to the optical axis of the imaging optical system and that are present in the optical path; a first driving means for moving the first movable member; A program for causing a computer included in an imaging device having a second driving means for moving the second movable member to function as a control means included in the imaging device described in any one of items 1 to 11.

[0094] The present invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Therefore, the following claims are appended to clarify the scope of the invention. [Explanation of symbols]

[0095] 1... camera body, 2... lens unit, 3... imaging optical system, 6... imaging element, 12... sensor drive unit, 14... lens control unit, 15... lens drive unit, 16... vibration-proof lens, 1000... camera

Claims

1. a first movable member and a second movable member that are movable in a direction perpendicular to the optical axis of the imaging optical system and that are present in the optical path; a first driving means for moving the first movable member; a second driving means for moving the second movable member; a control means for controlling the operation of the first driving means and the second driving means; the control unit controls the first driving unit and the second driving unit to periodically move the first movable member and the second movable member in a direction perpendicular to the optical axis, thereby realizing a pseudo optical low pass filter function.

2. 2. The imaging device according to claim 1, wherein the control means controls the first driving means and the second driving means to move the first movable member and the second movable member when movement of one of the first movable member and the second movable member is insufficient to achieve the required optical low pass filter function.

3. 3. The imaging device according to claim 2, wherein the case where the required optical low pass filter function cannot be achieved by movement of either the first movable member or the second movable member is a case where an exposure period is less than a threshold value.

4. 3. The imaging device according to claim 2, wherein the case in which the required optical low pass filter function cannot be achieved by movement of either the first movable member or the second movable member is a case in which the drive frequency of the movable member is equal to or higher than a threshold value.

5. 2. The image pickup apparatus according to claim 1, wherein the control means controls a phase difference between a timing at which the first driving means drives the first movable member and a timing at which the second driving means drives the second movable member, in order to realize a required optical low pass filter function.

6. 6. The imaging device according to claim 5, wherein the required function of the optical low-pass filter is a cutoff frequency.

7. 7. The imaging device according to claim 6, wherein the cutoff frequency is determined so as to remove spatial frequency components that cause moire or false color.

8. 2. The imaging apparatus according to claim 1, wherein the control means controls the first driving means and the second driving means during focus detection for still image shooting and during shooting.

9. 9. The imaging device according to claim 8, wherein, when an exposure period differs between the focus detection and the shooting, the control means causes a phase difference between a timing at which the first driving means drives the first movable member and a timing at which the second driving means drives the second movable member to differ between the focus detection and the shooting.

10. 9. The imaging device according to claim 8, wherein, when a pixel pitch used during focus detection is different from that used during shooting, the control means changes a phase difference between a timing at which the first driving means drives the first movable member and a timing at which the second driving means drives the second movable member between the focus detection and the shooting.

11. the first movable member is a movable lens included in the imaging optical system, the second movable member is an imaging element; 2. The imaging device according to claim 1, wherein:

12. a first movable member and a second movable member that are movable in a direction perpendicular to the optical axis of the imaging optical system and that are present in the optical path; a first driving means for moving the first movable member; a second driving means for moving the second movable member, controlling the first driving means and the second driving means so as to realize a pseudo optical low pass filter function by periodically moving the first movable member and the second movable member in a direction perpendicular to the optical axis.

13. a first movable member and a second movable member that are movable in a direction perpendicular to the optical axis of the imaging optical system and that are present in the optical path; a first driving means for moving the first movable member; and a second driving means for moving the second movable member.

Images