Imaging device, its control method, and program

Abstract

To provide an imaging apparatus and a control method therefor, and a program that enable accurate camera shake correction by accurately acquiring a motion vector rapidly or according to an imaging environment.SOLUTION: An imaging apparatus composed of a camera body 1 and a lens barrel 2 comprises: an imaging element 6 for imaging a picture; an imaging optical system 3 for focusing light from an object on an imaging surface of the imaging element 6; and photometry means 18 for performing photometry of an imaging environment. The imaging apparatus comprises: motion vector calculation means 5a for calculating a motion vector on the basis of the picture imaged by the imaging element 6; a camera system control unit 5 for calculating reliability of the motion vector; and imaging condition decision means for deciding a picture imaging condition in continuously imaging a plurality of pictures by the imaging element 6 depending on a photometry result by the photometry means 18 and the calculated reliability.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an imaging device, a control method thereof, and a program, and particularly to an imaging device that performs shake correction during shooting by driving an imaging element and a photographing optical system, a control method thereof, and a program.

Background Art

[0002] In recent years, due to the high performance of imaging devices, many imaging elements and photographing optical systems are equipped with a shake correction mechanism. With such a shake correction mechanism, when a user performs hand-held shooting using an imaging device, it becomes possible to reduce the influence of camera shake on the captured image.

[0003] In addition, due to the high performance of imaging devices, the readout speed of imaging elements has also been increased, and a technique of correcting and reducing camera shake by aligning and synthesizing a plurality of consecutive images is also known. On the other hand, this technique has a problem that if the exposure time of each image is shortened in order to reduce the camera shake in each of the plurality of images to be synthesized, the signal-to-noise ratio of each image deteriorates and noise increases.

[0004] In response to such problems, Non-Patent Document 1 discloses a technique of determining the exposure time of each image based on the balance between the magnitude of shake (camera shake, subject shake) and noise.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

[0006] However, the technology disclosed in Non-Patent Document 1 has the problem that, depending on the shooting environment, it is not possible to accurately acquire the positional displacement information (hereinafter referred to as motion vector) of each image used when performing positional synthesis, and as a result, it is not possible to perform image stabilization with high accuracy.

[0007] This invention has been made in view of the above problems, and aims to provide an imaging device, a control method therefor, and a program that can accurately acquire motion vectors according to the shooting environment and perform image stabilization with high precision. [Means for solving the problem]

[0008] To solve the above problems, the imaging device according to claim 1 of the present invention comprises an image sensor for capturing images, an imaging optical system for forming an image of light from an object on the imaging surface of the image sensor, and a photometering means for measuring the light of the shooting environment, wherein the imaging device comprises a motion vector calculation means for calculating a motion vector based on a plurality of images continuously captured by the image sensor, a reliability calculation means for calculating the reliability of the motion vector, and a control means for controlling development parameters for a plurality of images continuously captured by the image sensor according to the photometering result of the photometering means and the reliability calculated by the reliability calculation means, wherein the control means controls development parameters for a plurality of images continuously captured by the image sensor if the reliability of the motion vector calculated based on a first plurality of images continuously captured by the image sensor is lower than a predetermined threshold, For each of the output signals from the image sensor, Used when calculating the motion vector using the aforementioned motion vector calculation means A first development process for generating an image, Used when aligning and combining A second development process is performed to generate an image, and the gain in the first development process is set higher than the gain in the second development process. It is characterized by the following: [Effects of the Invention]

[0011] According to the present invention, motion vectors can be accurately acquired according to the shooting environment, and image stabilization can be performed with high precision. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the central cross-sectional view and electrical configuration of the imaging device according to Embodiment 1 of the present invention. [Figure 2] This figure illustrates the alignment image synthesis in Embodiment 1 of the present invention. [Figure 3] This is a diagram to explain the reliability of motion vectors. [Figure 4] This is a flowchart of the basic control process according to Embodiment 1 of the present invention. [Figure 5] Figure 4 is a flowchart of the motion vector confidence determination process in step S4003. [Figure 6] Figure 4 is a flowchart of the motion vector confidence reassessment process in step S4006. [Figure 7] Figure 4 is a flowchart of the process for determining the shooting conditions in step S4007. [Figure 8] Figure 4 is a flowchart of the alignment and synthesis process in step S4009. [Figure 9] This is a modified example of the flowchart for the alignment and synthesis process in step S4009 of Figure 4. [Figure 10] This is a block diagram showing the electrical configuration for determining the exposure time and shooting conditions of each image captured in the alignment and synthesis process in Embodiment 2 of the present invention. [Figure 11] This is a flowchart of the basic control process according to Embodiment 2 of the present invention. [Figure 12] Figure 11 is a flowchart of the process for determining the shooting conditions in step S1102. [Figure 13] Figure 11 is a flowchart of the alignment and synthesis process in step S1104. [Modes for carrying out the invention]

[0013] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0014] (Example 1) Hereinafter, referring to FIGS. 1 to 9, an imaging device including a camera body 1 and a lens barrel 2 attached thereto in Example 1 of the present invention will be described. FIG. 1(a) is a central cross-sectional view of the imaging device according to this example, and FIG. 1(b) is a block diagram showing an electrical configuration. In FIGS. 1(a) and 1(b), the same reference numerals denote the same configurations.

[0015] As shown in FIG. 1, a lens barrel 2 is attached to the camera body 1. When such attachment is made, the camera body 1 and the lens barrel 2 can communicate via an electrical contact 11.

[0016] The lens barrel 2 includes a photographing optical system 3 composed of a plurality of lenses including a shake correction lens 3a for performing shake correction on the optical axis 4, a lens system control unit 12, a lens-side shake correction means 13, and a lens-side shake detection means 16.

[0017] The camera body 1 includes a camera system control unit 5, an imaging element 6, an image processing unit 7, a memory means 8, a display means 9, an operation detection unit 10, a camera-side shake correction means 14, a camera-side shake detection means 15, a shutter 417, a photometry means 18, and a finder 19.

[0018] [[ID=ID=19]] The camera system control unit 5 includes a motion vector calculation means 5a for calculating a motion vector and an image synthesis means 5b for aligning and synthesizing a plurality of acquired images.

[0019] The display means 9 includes a rear display device 9a provided on the back surface of the camera body 1 and an EVF (electronic viewfinder) 9b provided in the finder 19. [[ID=ID=25]]

[0020] The operation detection unit 10 detects a signal from an operation means including a shutter release button (not shown).

[0021] The lens-side shake correction means 13 drives the shake correction lens 3a for performing hand shake correction in a plane perpendicular to the optical axis 4.

[0022] The camera-side image stabilization means 14 drives the image sensor 6 in a plane perpendicular to the optical axis 4.

[0023] The camera-side shake detection means 15 is provided on the camera body 1 and detects the amount of shake of the image sensor 6.

[0024] The lens-side shake detection means 16 is provided on the lens barrel 2 and detects the amount of shake of the imaging optical system 3.

[0025] The shutter 17 is located in front of the image sensor 6.

[0026] The photometering means 18 is provided on the camera body 1 and performs photometering of the shooting environment.

[0027] The imaging device according to this embodiment includes an imaging means, an image processing means, a recording and playback means, and a control means.

[0028] The imaging means includes an imaging optical system 3 and an image sensor 6.

[0029] The image processing means includes an image processing unit 7.

[0030] The recording and playback means includes a memory means 8 and a display means 9 (rear display device 9a and EVF 9b).

[0031] The control means includes a camera system control unit 5, an operation detection unit 10, a camera-side shake detection means 15, a camera-side shake correction means 14, a lens system control unit 12, a lens-side shake detection means 16, and a lens-side shake correction means 13.

[0032] Furthermore, the lens system control unit 12 can also drive other components, such as a focus lens (not shown) and an aperture, in addition to the image stabilization lens 3a, using other drive means (not shown).

[0033] The camera-side shake detection means 15 and the lens-side shake detection means 16 are capable of detecting angular shake with respect to the optical axis 4 applied to the imaging device, and this is achieved using, for example, a vibration gyroscope. Based on the amount of angular shake detected by the camera-side shake detection means 15 and the lens-side shake detection means 16, the camera-side shake correction means 14 drives the image sensor 6 and the lens-side shake correction means 13 drives the shake correction lens 3a on a plane perpendicular to the optical axis 4.

[0034] Furthermore, the camera-side shake detection means 15 and the lens-side shake detection means 16 may be equipped with acceleration sensors or the like, and configured to detect translational shake applied to the imaging device. In such cases, the camera-side shake correction means 14 and the lens-side shake correction means 13 drive the image sensor 6 and the shake correction lens 3a in a plane perpendicular to the optical axis 4 based on the detected angular shake and translational shake.

[0035] The imaging means described above is an optical processing system that focuses light from an object onto the imaging surface of the image sensor 6 via the imaging optical system 3. Since the focus evaluation amount / appropriate exposure amount is obtained from the image sensor 6, the imaging optical system 3 is appropriately adjusted based on this signal, so that the image sensor 6 is exposed to an appropriate amount of object light, and the subject image is formed near the image sensor 6.

[0036] The image processing unit 7 contains an A / D converter, a white balance adjustment circuit, a gamma correction circuit, an interpolation calculation circuit, etc., and can generate still images and videos for recording. The color interpolation processing means is provided in this image processing unit 7 and generates a color image by performing color interpolation (demosaiking) processing on the Bayer array signal. The image processing unit 7 also compresses still images, videos, and audio for recording using a predetermined method.

[0037] The memory means 8 comprises an actual storage unit consisting of ROM, RAM, HDD, etc. The camera system control unit 5 outputs to the recording unit of the memory means 8 and displays the image to be presented to the user on the display means 9.

[0038] The camera system control unit 5 generates timing signals for imaging in response to external operations and outputs them to the imaging means, image processing means, and recording / playback means as needed, thereby controlling them. For example, when the operation detection unit 10 detects that a shutter release button (not shown) has been pressed, the camera system control unit 5 controls the driving of the image sensor 6 and the operation of the image processing unit 7. Furthermore, it controls the state of each segment of the imaging device that displays information via the display means 9. In addition, the rear display device 9a is a touch panel and may also serve the roles of the display means 9 and the operation detection unit 10.

[0039] The adjustment operation of the imaging means by the control means will be described below.

[0040] The camera system control unit 5 is connected to the image processing unit 7 and acquires signals from the image sensor 6 via the image processing unit 7. Based on these acquired signals, it determines the appropriate focus position and aperture position. The camera system control unit 5 issues commands to the lens system control unit 12 via the electrical contacts 11, and the lens system control unit 12 appropriately controls the focus lens drive means and aperture drive means (not shown) based on these commands. Furthermore, in the mode for performing image stabilization, the camera system control unit 5 appropriately controls the camera-side image stabilization means 14 based on the signals obtained from the camera-side image stabilization means 15. Similarly, the lens system control unit 12 appropriately controls the lens-side image stabilization means 13 based on the signals obtained from the lens-side image stabilization means 16.

[0041] Specifically, the control means performs the following basic blur control operations on the imaging means.

[0042] First, the camera system control unit 5 and the lens system control unit 12 detect the camera shake detection means 15 and the lens shake detection means 16, respectively, which detect camera shake signals (angular shake and translational shake). Based on these detection results, the camera system control unit 5 and the lens system control unit 12 each calculate the amount of drive required for the image sensor 6 and the image stabilization lens 3a to correct the image shake. Subsequently, the calculated drive amount is sent as a command value to the camera-side image stabilization means 14 and the lens-side image stabilization means 13, which then drive the image sensor 6 and the image stabilization lens 3a, respectively.

[0043] In this embodiment, in addition to the basic blur control operation described above, the blur control operation for the imaging means is changed according to the resolution of the imaging optical system 3, the resolution of the image sensor 6, and shooting conditions such as focal length and shutter speed. A detailed control method will be described later.

[0044] Furthermore, as described above, the camera system control unit 5 and the lens system control unit 12 control the operation of each part of the camera body 1 and lens barrel 2 in response to user operations on operating means (not shown) provided on the camera body 1 and lens barrel 2. This enables the capture of both still images and videos.

[0045] Furthermore, in the imaging device according to this embodiment, it is also possible to perform an image synthesis blur correction operation to obtain a blur correction effect by aligning and combining images continuously captured by the image sensor 6.

[0046] In the basic image synthesis blur correction control operation, first, the camera system control unit 5 acquires images continuously captured by the image sensor 6 via the image processing unit 7.

[0047] Subsequently, the camera system control unit 5 uses the motion vector calculation means 5a to calculate positional displacement information (motion vector) between images acquired via the image processing unit 7.

[0048] Furthermore, the camera system control unit 5 uses the image synthesis means 5b to perform alignment between images acquired via the image processing unit 7, based on the motion vector calculated by the motion vector calculation means 5a, and then synthesizes them.

[0049] This basic image synthesis blur correction control operation makes it possible to obtain images with reduced blur by simulating an extended exposure time.

[0050] Next, using Figure 2, we will explain the alignment image synthesis in this embodiment.

[0051] In Figure 2, Image 21 is the first image captured in a series of images by the image sensor 6, while Images 22 and 23 are the second and third images captured after Image 21. The dashed line 24 represents the field of view of the first image 21 relative to the second image 22, and the dashed line 25 represents the field of view of the first image 21 relative to the third image 23.

[0052] Motion vector 26 is calculated by the motion vector calculation means 5a using images 21 and 22, and motion vector 27 is calculated by the motion vector calculation means 5a using images 21 and 23.

[0053] Image 28 is an image obtained by aligning and combining images 21-23 in the image synthesis means 5b based on motion vectors 26 and 27.

[0054] By aligning the consecutively acquired images 21-23 and then combining them, Figure 2 generates an image that appears equivalent to taking three separate photos with the same exposure time. Furthermore, in image 28, blur can be suppressed more effectively than in an image obtained by taking a single photo with the same exposure time required for the three images 21-23. Therefore, it becomes possible to obtain a photograph with a seemingly longer shutter speed while suppressing blur.

[0055] In Figure 2, motion vectors 26 and 27 were calculated from three images 21 to 23, and a single image 28 was created by aligning and combining images 21 to 23 based on motion vectors 26 and 27. However, the number of images used for alignment and combining is not limited to three. That is, the minimum number of images used for alignment and combining is two, and more images can be combined.

[0056] Next, we will explain how to calculate the motion vectors 26 and 27 in Figure 2.

[0057] The motion vector is calculated by extracting feature points from two images and determining the amount of movement of these feature points between the two images. Methods for detecting feature points include detecting edges (corners) or using brightness gradients for each of the two images. Methods for determining the amount of feature point movement include matching the two images and calculating the distance between the feature points. In this way, using two images, it becomes possible to calculate a motion vector that indicates how much the imaging device moved between the acquisition of each of the two images.

[0058] As shown in Figure 2, if multiple feature points are present in an image, multiple motion vectors will generally be detected. Furthermore, there may be cases where the maximum amount of motion vectors detected is predetermined, or where the total amount is defined for each region to prevent the detection of motion vectors from being concentrated in a particular area of ​​the screen. Basically, if feature points are detected and matching can be performed, motion vectors will be calculated for each detected feature point.

[0059] Furthermore, not all motion vectors necessarily show the same direction and magnitude. For example, if the subject itself moves, the amount of movement may differ depending on the feature point, meaning that the motion vectors may differ. On the other hand, when aligning two images, except in cases where each image is affine transformed and combined, the basic method of alignment and compositing is to move the entire screen in the same way and then combine them. Therefore, in this embodiment, a representative motion vector is selected from multiple motion vectors, and alignment and compositing are performed assuming that the direction and magnitude indicated by the representative motion vector correspond to the amount of positional displacement between the two images.

[0060] A representative motion vector is determined based on the variation in the direction and magnitude of multiple detected motion vectors. Furthermore, based on this variation, it becomes possible to determine the reliability of each motion vector, that is, a measure of how well it matches the direction and magnitude shown by the representative vector. In this embodiment, the more motion vectors detected that approximately match the direction and magnitude shown by the representative motion vector, the higher the reliability of the motion vector is judged to be, and the fewer motion vectors that approximately match, the lower the reliability of the motion vector is judged to be.

[0061] (Explanation regarding the reliability of motion vectors) Next, using Figure 3, we will provide examples of shooting environments where the reliability of the acquired vectors is low, and then explain how to change the shooting conditions, development parameters, and image processing to improve the reliability of the motion vectors in these examples.

[0062] Figure 3 is a diagram illustrating the reliability of motion vectors.

[0063] Figure 3(a) shows motion vector 31 with relatively high confidence, while Figures 3(b) and (c) show motion vectors 32 and 33 with relatively low confidence.

[0064] In Figure 3(b), the shooting environment is darker compared to Figure 3(a), so the reliability of the motion vector 32 is lower than that of the motion vector 31. Similarly, in Figure 3(c), while the subject is stationary in Figure 3(a), the subject is moving due to camera shake, wind, etc., so the reliability of the motion vector 33 is lower than that of the motion vector 31.

[0065] As described above using Figure 2, motion vectors are calculated as the amount of movement of feature points in two images. Therefore, if the shooting environment is dark, as in Figure 3(b), and feature points cannot be properly extracted from each captured image, it becomes difficult to calculate motion vectors. Consequently, in Figure 3(b), the number of detected motion vectors is less than in Figure 3(a), and the reliability of motion vector 32 is lower than that of motion vector 31.

[0066] Furthermore, as shown in Figure 3(c), when the subject is moving due to wind, the direction of the motion vector at each feature point differs (i.e., the calculated motion vectors point in various directions). In such shooting environments, the number of motion vectors that closely match the direction and magnitude indicated by the representative motion vector decreases, and the reliability of the motion vectors decreases. In other words, the greater the variance of the calculated motion vectors, the lower the reliability of the motion vectors.

[0067] In this embodiment, if the shooting environment is one in which the reliability of the motion vector is low, as illustrated in Figures 3(b) and (c), the shooting conditions, development parameters, and image processing used when acquiring the image used to calculate the motion vector are changed in order to improve the reliability of the motion vector.

[0068] As shown in Figure 3(b), if the shooting environment is dark and feature points cannot be extracted well, the shooting conditions and development parameters should be changed to acquire an image that is brighter and has higher contrast. Specifically, this can be done by changing the shooting conditions to increase the exposure time (lower the acquisition frame rate) or by changing the development parameters to increase at least one of the gain and contrast during image development.

[0069] As shown in Figure 3(c), if the images used to calculate the motion vector are significantly blurred due to camera shake, wind, etc., the shooting conditions, development parameters, and image processing should be changed to obtain sharper images when calculating the motion vector. Specifically, this can be done by shortening the exposure time (increasing the acquisition frame rate) or increasing the sharpness when developing the acquired images. Details on changing the shooting conditions when acquiring motion vectors in cases of significant camera shake will be described later in Example 2.

[0070] The above describes methods for changing the shooting conditions and image processing when acquiring images used to calculate motion vectors. However, the methods described above are not limited to those described above, as long as they can increase the reliability of the motion vectors. For example, the image processing performed on the images used to calculate motion vectors may differ from the image processing performed on the images used for alignment and synthesis.

[0071] For example, in the case of a dark shooting environment as shown in Figure 3(b), when developing the image acquired by the image sensor 6 to calculate the motion vector, one method is to increase the gain during development or increase the contrast during development.

[0072] Furthermore, additive blending can be used when calculating motion vectors. For example, in the case of a dark shooting environment as shown in Figure 3(b), instead of calculating motion vectors for each acquired image, multiple images can be generated by additively blending two or more images (additive blended images), and the motion vectors between these additive blended images can be calculated. By using additive blended images in this way, the apparent exposure time becomes longer, which increases the reliability of the calculated motion vectors. This apparent exposure time can also be used as a reference when determining the actual exposure time when taking images for alignment blending.

[0073] In this embodiment, it is desirable that the reliability of the motion vector be detected during the preparation phase before actual shooting, in the so-called live view state. This is because calculating the reliability of the motion vector in the live view state makes it possible to determine what shooting conditions will yield a highly reliable motion vector and enable accurate alignment and synthesis when actually shooting the image to be used for alignment and synthesis.

[0074] Next, the control flow according to this embodiment will be explained using Figures 4 to 9.

[0075] Figure 4 is a flowchart of the basic control process according to this embodiment. This process starts when the power button (not shown) of the camera body 1 is operated and the power of the imaging device is turned on. It is executed when the camera system control unit 5 reads a program stored on the HDD (not shown).

[0076] In step S4001, the camera system control unit 5 determines whether the shutter release button (not shown) on the camera body 1 has been half-pressed by the user and whether the shooting preparation operation has started. If the shooting preparation operation has started (YES in step S4001), the system proceeds to step S4002; otherwise (NO in step S4001), the system remains in step S4001 until the shooting preparation operation starts.

[0077] In step S4002, the camera system control unit 5 performs photometry in the photometering means 18, acquires an image at a predetermined frame rate for the live view image, calculates a motion vector in the motion vector calculation means 5a, and proceeds to step S4003. In this embodiment, the calculation of the motion vector is started when the shooting preparation operation is initiated, but the timing of the start of motion vector calculation is not limited to the timing in this embodiment, as long as it is before the image to be used for alignment and synthesis is captured. For example, the calculation of the motion vector may be started when the power is turned on.

[0078] In step S4003, the camera system control unit 5 performs a motion vector reliability determination process to determine the reliability of the motion vector calculated in step S4002, and then proceeds to step S4004. The motion vector reliability determination process will be described later with reference to Figure 5.

[0079] In step S4004, the camera system control unit 5 determines whether the reliability of the motion vector determined in step S4003 is lower than a predetermined threshold. If the result of this determination is lower than the predetermined threshold (YES in step S4004), the process proceeds to step S4005; otherwise (NO in step S4004), the process proceeds to step S4007.

[0080] In step S4005, the camera system control unit 5 changes the shooting conditions to improve the reliability of the motion vector and performs photometry and image acquisition in the same manner as in step S4002. After that, the motion vector calculation means 5a calculates the motion vector again, and then the process proceeds to step S4006. The method of changing the shooting conditions to improve the reliability of the motion vector is as explained with reference to Figure 3. For example, in step S4002, the photometry result from the photometering means 18 may determine that the shooting environment is dark. In this case, in step S4005, for example, the gain or contrast when shooting the image used to calculate the motion vector may be increased, or a composite image of multiple images taken in step S4002 may be generated to lengthen the apparent exposure time. Alternatively, in step S4002, the photometry result from the photometering means 18 may determine that the shooting environment is bright. In this case, in step S4005, for example, the image used to calculate the motion vector is captured at a faster frame rate (shorter exposure time) than the image acquired in step S4002.

[0081] In step S4006, the camera system control unit 5 performs a motion vector reliability reassessment process to determine the reliability of the motion vector calculated in step S4005, updates the current reliability of the motion vector, and proceeds to step S4007. The motion vector reliability reassessment process will be described later with reference to Figure 6.

[0082] In step S4007, the camera system control unit 5 (shooting condition determination means) performs a shooting condition determination process to determine the exposure time and shooting conditions for each image to be taken for alignment and synthesis, according to the reliability of the current motion vector, and proceeds to step S4008. The shooting condition determination process in step S4007 will be described later with reference to Figure 7. The reliability of the current motion vector is calculated in step S4003 if the system proceeds directly from step S4004 to step S4007, and in step S4006 if the system proceeds from step S4006 to step S4007.

[0083] In step S4008, the camera system control unit 5 determines whether the shutter release button (not shown) on the camera body 1 has been fully pressed by the user and whether the shooting operation has started. If the shooting operation has started (YES in step S4008), the process proceeds to step S4009; otherwise (NO in step S4008), the process returns to step S4001.

[0084] In step S4009, the camera system control unit 5 performs a positional merging process that captures multiple images in succession under the shooting conditions determined in step S4007, and then aligns and combines the obtained images, before proceeding to step S4010. The positional merging process in step S4009 will be described later with reference to Figures 8 and 9.

[0085] In step S4010, the camera system control unit 5 determines whether or not the shooting in the alignment and compositing process in step S4009 has been completed. If the result of this determination is that the shooting has been completed (YES in step S4010), the process proceeds to step S4011; otherwise (NO in step S4010), the process returns to step S4009.

[0086] In step S4011, the camera system control unit 5 determines whether the power button (not shown) on the camera body 1 has been operated and whether the power to the imaging device has been turned off. If the power has not been turned off (NO in step S4011), the process returns to step S4001. If the power has been turned off (YES in step S4011), the process ends.

[0087] Next, using the flowchart in Figure 5, we will explain in detail the process of determining the reliability of the motion vector in step S4003 of Figure 4.

[0088] In step S5001, the camera system control unit 5 detects the number of motion vectors calculated by the motion vector calculation means 5a and proceeds to step S5002. As explained in Figure 2, if the image used to calculate the motion vectors is dark, has low contrast, or is blurry, making it difficult to detect feature points, the motion vector calculation means 5a cannot calculate the motion vectors accurately. As a result, the number of motion vectors calculated may be small.

[0089] In step S5002, the camera system control unit 5 calculates the variance of the motion vectors calculated by the motion vector calculation means 5a and proceeds to step S5003. Here, variance refers to the variation in magnitude and direction of the motion vectors calculated by the motion vector calculation means 5a. If many motion vectors of similar magnitude and direction are calculated by the motion vector calculation means 5a, the variance is calculated to be small in step S5002. On the other hand, if various motion vectors with different magnitudes and directions are calculated by the motion vector calculation means 5a, the variance is calculated to be large in step S5002.

[0090] In step S5003, the camera system control unit 5 (reliability calculation means) refers to at least one of the detection results from step S5001 or the calculation results from step S5002, calculates the reliability of the motion vectors under the current shooting conditions, and terminates this process. Basically, the more motion vectors there are and the less dispersion there is among the motion vectors, the higher the reliability of the motion vectors calculated in step S5003.

[0091] Next, we will explain the details of the motion vector confidence reassessment process in step S4006 using the flowchart in Figure 6.

[0092] In step S6001, the camera system control unit 5 detects the number of motion vectors calculated by the motion vector calculation means 5a and proceeds to step S6002. As explained in Figure 2, if the image used to calculate the motion vectors is dark, has low contrast, or is blurry, making it difficult to detect feature points, the motion vector calculation means 5a cannot calculate the motion vectors accurately. As a result, the number of motion vectors calculated may be small.

[0093] In step S6002, the camera system control unit 5 calculates the variance of the motion vectors calculated by the motion vector calculation means 5a and proceeds to step S6003. Here, variance refers to the variation in magnitude and direction of the motion vectors calculated by the motion vector calculation means 5a. If many motion vectors of similar magnitude and direction are calculated by the motion vector calculation means 5a, the variance is calculated to be small in step S6002. On the other hand, if various motion vectors with different magnitudes and directions are calculated by the motion vector calculation means 5a, the variance is calculated to be large in step S6002.

[0094] In step S6003, the reliability of the motion vector under the current shooting conditions is calculated by referring to at least one of the detection results from step S6001 or the calculation results from step S6002, and then the process proceeds to step S6004. Basically, the more motion vectors there are, and the less variance there is in the motion vectors, the higher the reliability of the motion vector calculated in step S6003.

[0095] In step S6004, the camera system control unit 5 compares the reliability of the initial motion vector calculated in step S4003 with the reliability of the motion vector after changing the shooting conditions, calculated in step S6003, and proceeds to step S6005.

[0096] In step S6005, the camera system control unit 5 determines, based on the comparison in step S6004, whether the reliability of the motion vector has improved after changing the shooting conditions compared to the initial state. If the result of this determination is that the reliability of the motion vector has not improved after changing the shooting conditions compared to the initial state (NO in step S6005), the process proceeds to step S6006. On the other hand, if the reliability of the motion vector has improved after changing the shooting conditions compared to the initial state (YES in step S6005), this process is terminated.

[0097] In step S6006, the camera system control unit 5 adjusts a part of the imaging optical system 3 to obtain information on the autofocus point (the position in focus within the image) that was referenced when focusing (hereinafter referred to as "autofocus point information"), and proceeds to step S6007.

[0098] In step S6007, the camera system control unit 5 extracts motion vectors near the distance measurement point based on the distance measurement point information acquired in step S6006, and proceeds to step S6008.

[0099] In step S6008, the camera system control unit 5 calculates the reliability of the motion vector extracted in step S6007 and terminates this process.

[0100] For example, if the image acquired in step S4005 contains many subjects with low contrast and few feature points, such as the sky, changing the shooting conditions in step S4005 may not improve the reliability of the motion vectors, and may even worsen them. Therefore, if the reliability of the motion vectors does not improve even after changing the shooting conditions, steps S6006 to S6008 are performed to determine the reliability of the motion vectors that will be referenced when determining the shooting conditions in step S4007. If subjects that are difficult to acquire motion vectors from, such as the sky, occupy a large portion of the screen, the camera system control unit 5 extracts motion vectors near the focused distance measurement point, rather than the entire screen, and calculates the reliability of the extracted motion vectors. The greater the number of motion vectors near the distance measurement point compared to the number detected in step S6001, the higher the reliability of the motion vectors near the distance measurement point calculated in step S6008 will be compared to the value calculated in step S6003. Furthermore, the less the variance of the motion vectors near the distance measurement point is compared to the variance calculated in step S6002, the higher the reliability of the motion vectors near the distance measurement point calculated in step S6008 will be compared to the value calculated in step S6003.

[0101] Next, we will explain the details of the shooting condition determination process in step S4007 using the flowchart in Figure 7.

[0102] In step S7001, the camera system control unit 5 refers to the reliability of the motion vector calculated by the motion vector calculation means 5a, the image shooting conditions used to calculate the motion vector, and the photometric results obtained in step S4002, and proceeds to step S7002. The shooting conditions here also include the number of images to be added together (number of images added) when generating the additive composite image described above.

[0103] In step S7002, the camera system control unit 5 determines the exposure time and shooting conditions for each image during the alignment and synthesis process in step S4009, based on the information referenced in step S7001, and then terminates this process.

[0104] For example, from the information referenced in step S7001, it may be found that even if the shooting environment is dark, the reliability of the motion vector can be improved if the gain of each image used to calculate the motion vector is increased from the initial setting. In this case, the shooting conditions are determined in step S7002 so that only the image used to calculate the motion vector among the images taken during the alignment and compositing process has its gain increased. In the alignment and compositing process, the images developed with normal gain are aligned and composited according to the motion vector calculated using that image. This makes it possible to obtain an image with less camera shake.

[0105] Furthermore, the information referenced in step S7001 may indicate that using an additive composite image increases the reliability of the motion vector. In this case, the shooting conditions are determined in step S7002 to generate an additive composite image from the images captured during the alignment and compositing process. In the alignment and compositing process, the motion vector is calculated using this generated additive composite image. This makes it possible to obtain images with less camera shake.

[0106] On the other hand, from the information referenced in step S7001, it may be found that the reliability of the motion vector improves if the shooting environment is bright and the exposure time of each image used to calculate the motion vector is shorter than the initial setting. In this case, in step S7002, it is decided to shorten the exposure time when shooting each image during the alignment and synthesis process.

[0107] In this way, the exposure time and shooting conditions for each image during the alignment and compositing process are determined by referring to the information referenced in step S7001.

[0108] Furthermore, in step S7002, the shooting conditions for each image used for alignment and synthesis may be determined according to the performance of either or both of the camera-side image stabilization means 14 located in the camera body 1 and the lens-side image stabilization means 13 located in the lens barrel 2.

[0109] The operation of the image stabilization system may change between the preparation phase and the actual shooting (shooting with alignment and compositing). Therefore, it is preferable to determine the shooting conditions for shooting with alignment and compositing by considering the difference in image stabilization performance between the preparation phase and the actual shooting phase. For example, the image stabilization performance of the image stabilization system is better during actual shooting than during the preparation phase, so it is possible to set a longer exposure time (even if the exposure time is extended during actual shooting compared to the preparation phase, blurring will be less likely).

[0110] The specific operation method of the image stabilization means is determined by the model of the camera body 1 and lens barrel 2, and the combination of both. For this reason, it is preferable to also refer to the image stabilization performance during preparation for shooting and during actual shooting in step S7001, and to determine the shooting conditions in step S7002.

[0111] Next, we will explain the details of the alignment and synthesis process in step S4009 of Figure 4 using Figures 8 and 9.

[0112] Figures 8 and 9 represent essentially the same process. However, in Figure 8, all images for alignment and synthesis are captured using the exposure time and shooting conditions determined in the shooting condition determination process in step S4007. On the other hand, in Figure 9, in parallel with the continuous capture of images for alignment and synthesis, a motion vector is sequentially calculated for each image captured, its reliability is determined, and the shooting conditions are changed according to the result of that determination.

[0113] First, let's explain the alignment and compositing process shown in Figure 8.

[0114] In Figure 8, in step S8001, the camera system control unit 5 captures all images for alignment synthesis in a time-series sequence using the exposure time and shooting conditions determined in the shooting condition determination process in step S4007, and then proceeds to step S8002.

[0115] In step S8002, the camera system control unit 5 calculates motion vectors from each image acquired in step S8001 using the motion vector calculation means 5a, and then proceeds to step S8003.

[0116] In step S8003, the camera system control unit 5, using the image synthesis means 5b, performs alignment synthesis of all images acquired in step S8001 based on the motion vector calculated in step S8002, and then terminates this process.

[0117] In step S8002, when calculating the motion vector, it is generally advantageous to use the image captured in step S8001 as is, in terms of reducing processing time and processing load. However, this does not apply to the method of calculating the motion vector. For example, as explained in Figures 2 and 3, methods such as changing the gain and contrast processing during development, or adding images for motion vector calculation and using the resulting added composite image to calculate the motion vector, can also be employed.

[0118] Next, a modified example of the alignment and synthesis process will be explained using Figure 9.

[0119] In Figure 9, first, in step S9001, the camera system control unit 5 sets the number of frames N at the end of image capture for alignment synthesis to the initial value of 0, and proceeds to step S9002.

[0120] In step S9002, images for alignment and synthesis are captured, similar to the process in step S8001 in Figure 8. However, unlike in Figure 8, only two frames of images, which will be used to calculate the motion vector in step S9004 (described later), are captured here. If additive composite images are used to calculate the motion vector, images with twice the number of frames as the number of images to be added are captured.

[0121] In step S9003, the camera system control unit 5 increments the number of frames to be completed N by 2 and proceeds to step S9004. Note that if additive composite images are used to calculate the motion vector, the number of frames to be completed N is incremented by twice the number of frames added.

[0122] In step S9004, similar to the process in step S8002 in Figure 8, the motion vector is calculated using two consecutive images captured in step S9002, and the process proceeds to step S9005. Note that if additive composite images are used to calculate the motion vector, first, two additive composite images are generated from the images captured in step S9002, and the motion vector is calculated using these two additive composite images.

[0123] In step S9005, the camera system control unit 5 determines, similar to step S4004 in Figure 4, whether the reliability of the motion vector calculated in step S9004 is lower than a predetermined threshold. If the result of this determination is lower than the predetermined threshold (YES in step S9005), the process proceeds to step S9006; otherwise (NO in step S9005), the process proceeds to step S9007.

[0124] In step S9006, the camera system control unit 5 performs a shooting condition determination process to determine the shooting conditions for the image to be used for alignment and synthesis, similar to step S4007 in Figure 4. It updates the currently set exposure time and shooting conditions and proceeds to step S9007. In step S7001 of the shooting condition determination process performed here, the photometric results and shooting conditions from step S9002 and the confidence level of the motion vector determined in step S9005 are referenced.

[0125] In step S9007, it is determined whether the number of frames N at the end of shooting has reached the total number of frames Nmax. If it has reached the total number of frames Nmax (YES in step S9007), the process proceeds to step S9009. On the other hand, if it has not reached Nmax (NO in step S9007), the process returns to step S9002, and the camera system control unit 5 takes two images for alignment synthesis to be used in calculating the next motion vector, using the currently set exposure time and shooting conditions.

[0126] In step S9008, the camera system control unit 5, using the image synthesis means 5b, aligns and synthesizes the images of each frame captured in step S9002 based on the motion vector of each frame calculated in step S9004, and then terminates this process.

[0127] As explained above, by referring to the reliability of the motion vector, the exposure time and shooting conditions of the image for alignment and synthesis, and the photometric results of the photometric means 18, it is possible to accurately acquire the motion vector regardless of differences in the shooting environment, such as in dark environments or when there is a lot of blur. Furthermore, by using the motion vector acquired in this way, it becomes possible to accurately perform image stabilization by alignment and synthesis.

[0128] (Example 2) The imaging apparatus in Embodiment 2 of the present invention will now be described with reference to Figures 10 to 13.

[0129] In the prior art disclosed in Non-Patent Document 1, angular shake and translational shake are evaluated together as a single shake. As a result, in imaging devices that can optically correct angular shake well, there is a problem in that the exposure time is shortened unnecessarily when capturing images for alignment and synthesis, which unnecessarily increases noise.

[0130] This embodiment was made in view of the above-mentioned problems, and performs image stabilization while minimizing noise increase by determining the exposure time for each image based on translational blur among the types of blur.

[0131] In the following, hardware components identical to those in Example 1 will be denoted by the same reference numerals, and redundant explanations will be omitted.

[0132] Figure 10 is a block diagram showing the electrical configuration for determining the exposure time and shooting conditions of each image captured in the alignment and compositing process in this embodiment.

[0133] As shown in Figure 10, the camera system control unit 5 includes motion vector calculation means 5a and image synthesis means 5b (Figure 5), as well as total exposure time determination means 104, on-image plane translational blur amount prediction means 105, and shooting condition determination means 106. The camera-side blur detection means 15 includes angular blur detection means 101 and translational blur detection means, and the electrical contact 11 also functions as a shooting magnification acquisition means 103.

[0134] The angular shake detection means 101 is located within the camera-side shake detection means 15 and detects angular shake applied to the imaging device. The angular shake detection means 101 can be any means capable of detecting angular shake, such as a vibration gyroscope (not shown). The angular shake detection means 101 outputs the detected angular shake as an angular velocity signal to the image plane translational shake amount prediction means 105.

[0135] The translational shake detection means 102 is located within the camera-side shake detection means 15 and detects translational shake applied to the imaging device. The translational shake detection means 102 can be any means capable of detecting translational shake, such as an acceleration sensor (not shown). The translational shake detection means 102 converts the detected acceleration signal into a velocity signal by integrating it and outputs it to the image plane translational shake amount prediction means 105.

[0136] The shooting magnification acquisition means 103 acquires the shooting magnification of the shooting optical system 3 from the lens system control unit 12. The shooting magnification acquisition means 103 outputs the acquired shooting magnification to the image plane translation blur amount prediction means 105.

[0137] The total exposure time determination means 104 is located within the camera system control unit 5 and determines the total exposure time based on the photometric results from the photometric means 18. However, it is not always necessary to determine the total exposure time based on the photometric results; the user may specify it directly. The total exposure time here is the sum of the exposure times for each frame taken during the alignment and compositing process, multiplied by the number of alignment and compositing frames (number of consecutive shots). The total exposure time determination means 104 outputs the determined total exposure time to the image plane translational blur amount prediction means 105 and the shooting condition determination means 106, respectively.

[0138] The image plane translational blur amount prediction means 105 is located within the camera system control unit 5 and predicts the amount of translational blur that appears on the imaging plane during the total exposure time (hereinafter referred to as "image plane translational blur amount"). Specifically, the image plane translational blur amount prediction means 105 predicts the translational blur amount based on the angular velocity signal from the angular blur detection means 101, the velocity signal from the translational blur detection means 102, the shooting magnification from the shooting magnification acquisition means 103, and the total exposure time from the total exposure time determination means 104. The image plane translational blur amount prediction means 105 outputs the predicted image plane translational blur amount to the shooting condition determination means 106.

[0139] An example of a method for predicting the amount of translational blur on the image plane using the image plane translational blur prediction means 105 is described below.

[0140] First, the amount of angular shake θ during the total exposure time is predicted from the angular velocity signal from the angular shake detection means 101 and the total exposure time from the total exposure time determination means 104. For example, the angular velocity signal up to just before the prediction is integrated over the total exposure time to make the prediction.

[0141] Next, the radius of rotation r is calculated using equation (1) from the angular velocity signal (hereinafter referred to as the angular velocity signal ω) from the angular runout detection means 101 and the velocity signal (hereinafter referred to as the velocity signal v) from the translational runout detection means 102. r = v / ω ···(1)

[0142] Finally, using the predicted angular shake amount θ, the calculated rotation radius r, and the shooting magnification from the shooting magnification acquisition means 103 (hereinafter referred to as the shooting magnification β), the predicted amount x of on-image plane translation shake that appears on the imaging surface of the image sensor 6 is calculated using equation (2). x = θ × r × β ... (2)

[0143] The above method allows for the prediction of on-image translational blur, but this is only one example. The on-image translational blur can also be predicted from the total exposure time, velocity signal v, and magnification β. For example, the on-image translational blur can be predicted by integrating the velocity signal v over the total exposure time and multiplying it by the magnification β.

[0144] The shooting condition determination means 106 determines the exposure time for each image to be taken for alignment and synthesis, based on the total exposure time from the total exposure time determination means 104 and the amount of translational blur on the image plane predicted by the image plane translational blur amount prediction means 105. At this time, the smaller the predicted amount of translational blur on the image plane, the longer the exposure time for each image and the smaller the number of continuous shots (total number of frames Nmax). On the other hand, the larger the predicted amount of translational blur on the image plane, the shorter the exposure time for each image and the larger the number of continuous shots (total number of frames Nmax).

[0145] Figure 11 is a flowchart of the basic control process in this embodiment. This process starts when the power button (not shown) of the camera body 1 is operated, and the power to the imaging device is turned on. It is executed when the camera system control unit 5 reads a program stored on the HDD (not shown).

[0146] In step S1101, the camera system control unit 5 determines whether the shutter release button (not shown) on the camera body 1 has been half-pressed by the user and whether the shooting preparation operation has started. If the shooting preparation operation has started (YES in step S1101), the system proceeds to step S1102; otherwise (NO in step S1101), the system remains in step S1101 until the shooting preparation operation starts.

[0147] In step S1102, the shooting condition determination means 106 performs a shooting condition determination process to determine the exposure time and shooting conditions for each image used for alignment and synthesis, and then proceeds to step S1103. The shooting condition determination process in step S1102 will be described later with reference to Figure 12.

[0148] In step S1103, the camera system control unit 5 determines whether the shutter release button (not shown) on the camera body 1 has been fully pressed by the user and whether the shooting operation has started. If the shooting operation has started (YES in step S1103), the process proceeds to step S1104; otherwise (NO in step S1103), the process returns to step S1101.

[0149] In step S1104, the camera system control unit 5 performs an alignment and compositing process that captures multiple images in succession and aligns and combines the obtained images, then proceeds to step S1105. The alignment and compositing process in step S1104 will be described later with reference to Figures 8 and 13.

[0150] In step S1105, the camera system control unit 5 determines whether or not the shooting in the alignment and compositing process in step S1104 has been completed. If the shooting is completed (YES in step S1105), the process proceeds to step S1106; otherwise (NO in step S1105), the process returns to step S1104.

[0151] In step S1106, the camera system control unit 5 determines whether the power button (not shown) on the camera body 1 has been operated and whether the power to the imaging device has been turned off. If the power is not turned off (NO in step S1106), the process returns to step S1101. If the power is turned off (YES in step S1106), the process ends.

[0152] Next, the process of determining the shooting conditions in step S1102 of Figure 11 will be explained using the flowchart in Figure 12.

[0153] In step S1201, the angular shake detection means 101 detects the angular shake, outputs the detected angular shake as an angular velocity signal to the image plane translational shake amount prediction means 105, and then proceeds to step S1202.

[0154] In step S1202, the translational shake detection means 102 detects the translational shake, converts the detected acceleration signal into a velocity signal by integrating it, outputs it to the image plane translational shake amount prediction means 105, and then proceeds to step S1203.

[0155] In step S1203, the shooting magnification acquisition means 103 acquires the shooting magnification of the shooting optical system 3 from the lens system control unit 12, outputs it to the image plane translation blur amount prediction means 105, and then proceeds to step S1204.

[0156] In step S1204, the total exposure time determination means 104 determines the total exposure time and outputs it to the image plane translation blur amount prediction means 105 and the shooting condition determination means 106, respectively, before proceeding to step S1205.

[0157] In step S1205, the image plane translational blur prediction means 105 predicts the amount of image plane translational blur based on the angular velocity signals, shooting magnification, and total exposure time acquired in steps S1201, S1203, and S1204, respectively, and proceeds to step S1206.

[0158] In step S1206, the shooting condition determination means 106 determines the exposure time for each image to be captured for alignment and synthesis, based on the total exposure time and the amount of translational blur on the image plane acquired in steps S1204 and S1205, respectively, and then terminates this process.

[0159] Next, the details of the alignment and synthesis process in step S1104 of Figure 11 will be explained using the flowchart in Figure 13.

[0160] Figures 9 and 13 represent essentially the same process. However, in Figure 13, the exposure time is changed for each frame of the image used for alignment and synthesis based on the total exposure time and the amount of translational blur on the image plane. In other words, Figure 13 differs from Figure 9 in that the shooting condition determination process in step S1102 is executed immediately before step S9002.

[0161] In this embodiment, the case in which the angular shake detection means 101 and the translational shake detection means 102 are provided within the camera-side shake detection means 15 has been described, but the present invention is not limited to this case. For example, the angular shake detection means 101 and the translational shake detection means 102 may be provided within the lens-side shake detection means 16 so that angular velocity signals and velocity signals are transmitted to the camera body 1 side as shake signals via the electrical contacts 11. In addition, in the imaging device according to this embodiment, the lens barrel 2 was attached to the camera body 1, but the lens barrel portion and the camera body portion may be integrally configured. In this case, the camera-side shake detection means 15 and the lens-side shake correction means 13 may also be configured as an integrated shake detection means, and the angular shake detection means 101 and the translational shake detection means 102 may be provided within that shake detection means. The location of the shake detection means within the imaging device is not limited.

[0162] Furthermore, the translational shake detection means 102 may detect translational shake using the following detection method, which utilizes the motion vector vect calculated by the motion vector calculation means 5a, the signal c' obtained by differentiating the shake correction amount, and the angular velocity signal ω of the angular shake.

[0163] If the lens-side image stabilization means 13 and / or the camera-side image stabilization means 14 are correcting the image stabilization when translational blur is detected, the velocity signal v of the translational blur can be calculated using equation (3). v = vect + c' - ω ···(3)

[0164] Furthermore, if translational shake is not corrected when it is detected, the velocity signal v of the translational shake can be calculated using equation (4). v = vector - ω··(4)

[0165] As explained above, in an imaging device that can optically correct angular shake effectively, the shooting conditions (exposure time) for each frame captured by the alignment and synthesis process are determined based on translational shake. This prevents the exposure time of each frame from being shortened unnecessarily due to the effects of angular shake, thereby reducing the number of composite images and suppressing noise.

[0166] In the embodiments described above, the imaging device consisted of a camera body 1 and a lens barrel 2. However, the imaging device is not limited to this case, as long as it performs image stabilization during shooting by driving an image sensor and an imaging optical system. For example, the imaging device according to the present invention can also be applied to the camera section of a smartphone or tablet terminal.

[0167] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist.

[0168] Furthermore, this embodiment can also be implemented by supplying a program that implements one or more functions to a computer of a system or device via a network or storage medium, and the system control unit of that system or device reads and executes the program. The system control unit has one or more processors or circuits and may include a plurality of separate system control units or a network of a plurality of separate processors or circuits in order to read and execute executable instructions.

[0169] A processor or circuit may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA). Alternatively, a processor or circuit may include a digital signal processor (DSP), a dataflow processor (DFP), or a neural processing unit (NPU).

[0170] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist. [Explanation of symbols]

[0171] 3. Imaging optical system 5. Camera System Control Unit 5a Motion vector calculation means 5b Image synthesis means 6 Image sensor 12 Lens System Control Unit 13 Lens-side image stabilization means 14. Camera-side image stabilization means 15 Camera-side shake detection means 16 Lens-side shake detection means 18 Photometric means

Claims

1. An imaging device comprising an image sensor for capturing an image, an imaging optical system for forming an image of light from an object on the imaging surface of the image sensor, and a photometering means for measuring the light of the shooting environment, A motion vector calculation means that calculates a motion vector based on a plurality of images continuously captured by the image sensor, A reliability calculation means for calculating the reliability of the motion vector, The system includes a control means for controlling development parameters for a plurality of images continuously captured by the image sensor, according to the photometric results from the photometric means and the reliability calculated by the reliability calculation means. The imaging apparatus is characterized in that, when the reliability of the motion vector calculated based on a first plurality of images continuously captured by the image sensor is lower than a predetermined threshold, the control means performs a first development process for generating an image used when the motion vector calculation means calculates the motion vector, and a second development process for generating an image used when the images are aligned and combined, on the output signals from the image sensor corresponding to each of the second plurality of images continuously captured by the image sensor, the gain in the first development process is set higher than the gain in the second development process.

2. The imaging apparatus according to claim 1, further comprising image synthesis means for aligning and combining a plurality of images captured in succession.

3. The motion vector calculation means calculates a motion vector based on the image generated by the first development process on the output signal from the image sensor corresponding to each of the second plurality of images, The imaging apparatus according to claim 2, characterized in that the image synthesis means aligns and synthesizes images generated by the second development process on the output signals from the image sensor corresponding to each of the second plurality of images, based on the calculated motion vector.

4. The imaging apparatus according to any one of claims 1 to 3, characterized in that the reliability calculation means calculates the reliability according to the number of motion vectors calculated by the motion vector calculation means from a plurality of images continuously captured by the image sensor during preparation for shooting.

5. The imaging apparatus according to any one of claims 1 to 4, characterized in that the reliability calculation means calculates the reliability according to the variance of the motion vector calculated by the motion vector calculation means from a plurality of images continuously captured by the image sensor during preparation for shooting.

6. The imaging apparatus according to any one of claims 1 to 5, further comprising a shooting condition determination means for determining the shooting conditions when capturing a second plurality of images, in accordance with the reliability of the motion vector calculated based on the first plurality of images, the shooting conditions when capturing the first plurality of images, and the photometric results of the photometric means.

7. The imaging apparatus according to claim 6, characterized in that the shooting condition determination means changes the exposure time, which is one of the shooting conditions when shooting the second plurality of images, according to the brightness of the shooting environment determined by the photometric result of the photometric means, when the reliability of the motion vector calculated based on the first plurality of images is lower than the predetermined threshold.

8. The system further includes a blur correction means for correcting blur in at least one of the image sensor and the imaging optical system, The imaging apparatus according to claim 6 or 7, characterized in that the shooting condition determination means changes the shooting conditions when capturing the second plurality of images according to the image stabilization performance of the image stabilization means.

9. A control method for an imaging device comprising an image sensor for capturing an image, an imaging optical system for forming an image of light from an object onto the imaging surface of the image sensor, and a photometric means for measuring the light of the shooting environment, A motion vector calculation step, which calculates a motion vector based on multiple images continuously captured by the image sensor, A reliability calculation step for calculating the reliability of the motion vector, The system includes a control step that controls development parameters for a plurality of images continuously captured by the image sensor, according to the photometric results from the photometric means and the reliability calculated in the reliability calculation step. The control step is characterized in that, if the reliability of the motion vector calculated based on a first plurality of images continuously captured by the image sensor is lower than a predetermined threshold, a first development process is performed on the output signal from the image sensor corresponding to each of the second plurality of images continuously captured by the image sensor to generate an image used when calculating the motion vector by the motion vector calculation means, and a second development process is performed to generate an image used when aligning and combining the images, and the gain in the first development process is set higher than the gain in the second development process.

10. A program for causing a computer to function as a motion vector calculation means, a reliability calculation means, and a control means for an imaging apparatus described in any one of claims 1 to 8.

Images