Imaging device, control method for imaging device, and program

Abstract

This system provides a mechanism that can generate video frames suitable for video using multiple images. [Solution] The imaging device 100 acquires a series of images and generates a video based on the series of images. The imaging device 100 also sets multiple exposure times to be used for capturing the series of images. The imaging device 100 calculates blur information for generating video frames based on the blur information acquired by the blur information acquisition unit 202 from the image (interpolated frame 300) obtained from the first exposure time 300a, and generates video frames using the blur information for generating video frames.

Description

Technical Field

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program.

Background Art

[0002] The characteristics required for the image quality of each image (video frame) constituting a video and the image in a still image are different. For example, in a video, images are displayed one after another at a predetermined frame rate. Therefore, each image constituting the video is required to have a natural connection with the images before and after it rather than a high degree of perfection as a single image where blurring is not noticeable. Therefore, it is better for the images constituting the video to have a certain degree of blurring. On the other hand, in a still image, the image constituting the still image continues to be displayed. Therefore, the image constituting the still image is required to have a high degree of perfection as a single image where blurring is not noticeable.

[0003] In contrast, as a technique capable of obtaining both a high-quality still image and a video simultaneously, the technique of Patent Document 1 has been proposed. In Patent Document 1, the exposure time of the video is divided into exposure times suitable for still images, a still image is generated based on the image obtained by imaging with one of the divided exposure times, and a video is generated based on a video frame obtained by synthesizing a plurality of images obtained by imaging with other exposure times.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, with the aforementioned technology, since multiple images are combined to generate video frames, there is a concern that the resulting video frames may resemble images with multiple exposures, meaning they are unsuitable for composing a video. Such images cannot maintain a natural connection with the preceding and succeeding images, making it impossible to generate a smooth video.

[0006] The present invention aims to provide a mechanism that can generate video frames suitable for video using multiple images. [Means for solving the problem]

[0007] To achieve the above objective, the imaging device of the present invention comprises: a generation means for acquiring a plurality of consecutive images and generating a video based on the plurality of images; a setting means for setting a plurality of exposure times to be used for capturing the plurality of images; an acquisition means for acquiring blur information of the acquired images; and a calculation means for calculating blur information for generating video frames based on the blur information acquired by the acquisition means from images obtained by shooting with a first exposure time which is set to be longer than a second exposure time, wherein the generation means generates images constituting the video using the blur information for generating video frames. [Effects of the Invention]

[0008] According to the present invention, it is possible to generate video frames suitable for video using multiple images. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram schematically showing the configuration of the imaging device according to this embodiment. [Figure 2] Figure 1 is a processing block diagram of the imaging device. [Figure 3] This timing chart shows the relationship between the exposure time set in the exposure time setting unit and the unexposed time when simultaneously shooting video and continuous still images in this embodiment. [Figure 4]This flowchart shows the procedure for generating still images from video using the imaging device shown in Figure 1. [Figure 5] Figure 4 is a flowchart showing the procedure for acquiring shake information in S401. [Figure 6] This diagram illustrates the extraction process in S517 of Figure 5. [Figure 7] Figure 4 is a flowchart showing the procedure for the blur addition process in S402. [Figure 8] Figure 4 is a flowchart showing another procedure for acquiring shake information in S401. [Modes for carrying out the invention]

[0010] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0011] Figure 1 is a schematic block diagram showing the configuration of the imaging device 100 according to this embodiment. In Figure 1, the imaging device 100 includes a CPU 101, RAM 102, ROM 103, imaging unit 104, input interface 105, and output interface 106. These are connected to each other via a system bus 107. A recording device 108 is connected to the input interface 105, and a display device 109 is connected to the output interface 106.

[0012] The CPU 101 is a processor that comprehensively controls the components described above. The RAM 102 is the main memory and work area of ​​the CPU 101. The ROM 103 stores programs and other data used for processing within the imaging device 100. The CPU 101 uses the RAM 102 as a work area and executes various processes described later by loading the programs stored in the ROM 103 into the RAM 102 and running them.

[0013] The input interface 105 is, for example, a serial bus interface such as USB or IEEE1394. The imaging device 100 can acquire image data to be processed from the recording device 108 (for example, a hard disk, memory card, CF card, SD card, or USB memory) via this input interface 105.

[0014] The output interface 106 is, for example, a video output terminal such as DVI or HDMI (registered trademark). The imaging device 100 can output image data processed by the imaging device 100 to the display device 109 (an image display device such as a liquid crystal display) via this output interface 106. Note that the imaging device 100 may include components other than those shown in Figure 1.

[0015] Figure 2 is a processing block diagram of the imaging device 100 shown in Figure 1. In this embodiment, the CPU 101 loads the program stored in the ROM 103 into the RAM 102 and executes it, thereby realizing the functions of each processing block shown in Figure 2. Note that the CPU 101 does not necessarily have the functions of all the processing blocks shown in Figure 2, and the imaging device 100 may have processing circuits corresponding to each processing block. In the following description, as an example, the functions of each processing block will be explained assuming that they are executed by the CPU 101, but a dedicated processing module may be provided, and that module may perform the processing. As shown in Figure 2, the imaging device 100 has the functions of processing blocks such as an exposure time setting unit 201, a blur information acquisition unit 202, a blur addition unit 203, and a compositing unit 204.

[0016] The exposure time setting unit 201 sets a plurality of exposure times and imaging intervals according to the purpose of imaging, and causes the imaging unit 104 to perform imaging with the exposure time. The imaging unit 104 generates an image for each imaging interval. The exposure time may be, for example, a preset exposure time or an exposure time determined based on an operation by the user. For example, as the preset exposure time, there are an exposure time determined based on the shutter speed in video shooting and an exposure time determined based on the shutter speed in continuous shooting of still images. Also, the imaging interval may be, for example, a preset interval or an interval determined based on an operation by the user. For example, as the preset interval, there are the frame rate in video shooting, the shooting interval in continuous shooting of still images, and the like. That is, the exposure time setting unit 201 sets a plurality of exposure times including at least a first exposure time corresponding to the purpose of imaging and a second exposure time shorter than the first exposure time, and an unexposed time that is the imaging interval. In this embodiment, as an example, the exposure time setting unit 201 will be described with a configuration that sets two different exposure times, but the configuration is not limited to this, and a configuration that sets three or more different exposure times may also be used.

[0017] As an example, FIG. 3 shows, as a timing chart, the relationship between the exposure time and the unexposed time set by the exposure time setting unit 201 when simultaneously performing video shooting and continuous shooting of still images.

[0018] In FIG. 3, 302 represents a video frame which is one frame during video shooting, and the exposure time of this video frame is 302a. 301 represents a still image frame which is one frame during still image shooting, the exposure time of this still image frame is 301a, and the imaging interval of this still image frame is 301b. 303 represents an unexposed time that cannot be exposed due to hardware constraints of the imaging unit 104 or the like. 300 represents a complementary frame for performing complementary exposure to generate a video frame. The exposure time 300a of the complementary frame 300 is the time obtained by subtracting the total time of the exposure time 301a of the still image frame and the unexposed time 303 from the time obtained by dividing the exposure time 302a of the video frame by '4'. That is, the exposure of the video frame 302 is divided into 4 times each by the complementary frame 300 and the still image frame 301 and then exposed.

[0019] In FIG. 3, the imaging frames represent the timings of the still image frames and the complementary frames actually imaged by the imaging unit 104. Based on the imaging frames, an exposure time 300a is set as the first exposure time, an exposure time 301a is set as the second exposure time, and 303 is set as the unexposed time in the imaging unit 104. Note that this timing chart is an example, and the exposure time and the unexposed time set by the exposure time setting unit 201 are not limited to this.

[0020] Returning to FIG. 2, the blur information acquisition unit 202 acquires blur information based on a plurality of images continuously captured by the imaging unit 104, the exposure time set by the exposure time setting unit 201, and the unexposed time. The detailed operation of the blur information acquisition unit 202 will be described later.

[0021] The blur addition unit 203 selects an image to which blur is to be added based on the exposure time and the unexposed time set by the exposure time setting unit 201. The blur addition unit 203 applies a blur filter to the selected image based on the blur information acquired from the blur information acquisition unit 202. The detailed operation of the blur addition unit 203 will be described later.

[0022] The synthesis unit 204 synthesizes a predetermined number of images, including multiple images with added blur generated by the blur addition unit 203 and multiple images without added blur.

[0023] Figure 4 is a flowchart showing the procedure for the video-still image generation process performed by the imaging device 100 in Figure 1. This video-still image generation process is realized by the CPU 101 loading a program stored in ROM 103 into RAM 102 and executing it. This video-still image generation process is started, for example, when the imaging device 100 receives an instruction from the user to generate a video using images obtained from imaging with different exposure times. An instruction to generate a video using images obtained from imaging with different exposure times is, for example, an instruction to simultaneously perform video recording and continuous still image shooting. When such an instruction is received, the exposure time setting unit 201 sets a first exposure time (for example, exposure time 300a in Figure 3), a second exposure time (for example, exposure time 301a in Figure 3), and an unexposed time (for example, exposure time 303 in Figure 3).

[0024] In Figure 4, first, in S401, the CPU 101 controls the blur information acquisition unit 202 to perform the blur information acquisition process shown in Figure 5(a), which will be described later. This provides the blur information used to generate video frames (hereinafter referred to as "blur information for video frame generation").

[0025] Next, in S402, the CPU 101 controls the blur addition unit 203 to perform the blur addition process shown in Figure 7, which will be described later. As a result, blur is added to the video frames that make up the video based on the blur information for generating video frames obtained in S401.

[0026] Next, in S403, the CPU 101 generates video and still images. Specifically, the CPU 101 controls the synthesis unit 204 to synthesize a predetermined number of images with blur added in S402 (for example, four processed images obtained from four still image frames 301 by the blur addition process described later) and images without blur (for example, four interpolated frames 300) to generate video frames. This predetermined number may be, for example, a number set in advance, or a number determined based on user operation. For example, the number set in advance is set based on the exposure time in video shooting, or the exposure time and imaging interval in continuous shooting. A video is generated based on the video frames thus generated. The CPU 101 also generates still images based on the still image frames 301 obtained from imaging with a second exposure time. In other words, in this embodiment, by setting an exposure time suitable for still images to the second exposure time, it is possible to obtain still images with high quality as a single image, such as those with less noticeable blur.

[0027] Next, in S404, the CPU 101 outputs the generated video and still images to the output interface 106. After that, this process ends.

[0028] As described above, in this embodiment, it is possible to simultaneously obtain a smooth video with a natural connection between preceding and succeeding images, and still images with minimal blur, by using multiple images captured with different exposure times.

[0029] Figure 5(a) is a flowchart showing the procedure for acquiring shake information in S401 of Figure 4. The shake information acquisition process is performed by the shake information acquisition unit 202 in accordance with the control of the CPU 101.

[0030] In Figure 5(a), first, in S500, the blur information acquisition unit 202 acquires multiple images obtained by the imaging unit 104. The multiple images are multiple interpolation frames 300, which are images obtained during imaging with a first exposure time 300a, and multiple still image frames 301, which are images obtained during imaging with a second exposure time 301a.

[0031] Next, in S501, the blur information acquisition unit 202 obtains the exposure time at the time of image capture for each acquired image from the exposure time setting unit 201. Also, in S502, the blur information acquisition unit 202 obtains the unexposed time from the exposure time setting unit 201.

[0032] Next, in S503, the blur information acquisition unit 202 determines for each acquired image whether the exposure time for that image is longer than the threshold T. The threshold T may be a preset value, or it may be a value set based on user operation. In this embodiment, as an example, the threshold T is set to a second exposure time. For images in S503 where it is determined that the exposure time for that image is longer than the threshold T, that is, images with an exposure time of the first exposure time, the blur information acquisition unit 202 performs the blur information calculation process shown in Figure 5(b) (S504).

[0033] Figure 5(b) is a flowchart showing the procedure for calculating the shake information in S504 of Figure 5(a).

[0034] In Figure 5(b), in S511, the blur information acquisition unit 202 performs subject area detection processing to detect the area of ​​the subject from the target image. Examples of subject areas detected by the subject area detection processing include the area of ​​an animal, the area of ​​a vehicle, the area of ​​a person, etc. However, the subject area detection processing may also detect the area of ​​subjects of other types.

[0035] Next, in S512, it is determined whether or not the area of ​​the subject has been detected in the target image. If it is determined that the area of ​​the subject has been detected in the target image, the process proceeds to S513. In S513, the blur information acquisition unit 202 extracts the detected area of ​​the subject from the image.

[0036] Next, in S514, the blur information acquisition unit 202 calculates a point image distribution function for each region of the subject extracted in S513. Here, the point image distribution function is a function that represents the trajectory of image degradation when the image is out of focus and blurred, or when image quality deteriorates due to blur.

[0037] Next, in S515, the blur information acquisition unit 202 extracts the background region from the target image. Here, the background region is the region of the target image other than the region of the subject extracted in S513. Next, in S516, the blur information acquisition unit 202 calculates the point image distribution function of the background region extracted in S515. After this, the process proceeds to S520, which will be described later.

[0038] If it is determined in S512 that no subject area is detected in the target image, the process proceeds to S517. In S517, the blur information acquisition unit 202 extracts predetermined areas from the target image. For example, the image 600 in Figure 6 does not contain the subject areas (animal area, vehicle area, person area) mentioned above, but it does contain the house area 601 and the road area 602. In S517, for example, predetermined areas, such as areas 603a to 603i, are extracted from such an image 600. In this embodiment, as an example, a configuration in which nine areas are extracted as areas at predetermined positions is described, but the number of areas to be extracted is not limited to nine, and the positions of the extracted areas are not limited to the positions shown in Figure 6.

[0039] Next, in S518, the blur information acquisition unit 202 calculates a point image distribution function for each region extracted in S517. Then, this process proceeds to S519. Here, if the region of the subject is not detected in the image, there is a very high probability that the image contains a region of a stationary object. In other words, there is a high probability that blur occurs in the same direction throughout the entire image.

[0040] Therefore, in this embodiment, in S519, the blur information acquisition unit 202 acquires the principal components of the point image distribution function of each region extracted in S518. The principal components indicate the variability of the point image distribution function of each region. Representative values ​​are obtained from the point image distribution function of each region using statistical methods. In this way, a representative point image distribution function within the target image is obtained. Thus, in this embodiment, the point image distribution function is calculated for the extracted subject region and background region from an image in which the subject region is detected. On the other hand, a representative point image distribution function is calculated from a predetermined region from an image in which the subject region is not detected.

[0041] Next, in S520, the blur information acquisition unit 202 calculates first blur information based on the point image distribution function calculated in S514 and S516, or the point image distribution function calculated in S518. Note that the point image distribution function also includes blur information such as diffraction in addition to blur information, so in S520, only the blur information for the region extracted in the above-described process is calculated. Thus, in this embodiment, multiple blur information corresponding to each of the multiple regions extracted in S513 and S515, or S517, is calculated as first blur information. After that, the blur information calculation process ends, and the blur information acquisition process proceeds to S505 in Figure 5(a).

[0042] Returning to Figure 5(a), for images where the exposure time for imaging is determined to be less than or equal to the threshold T in S503, the blur information calculation process in S504 is not performed, and the process in S505 is performed instead.

[0043] In S505, the blur information acquisition unit 202 calculates blur information for generating video frames to be used when adding blur in the blur addition process shown in Figure 7, which will be described later, based on the exposure time acquired in S501 and the unexposed time acquired in S502. In this embodiment, blur information for generating video frames is calculated for each region extracted in S513 and S515, or for each region extracted in S517. For example, if the first blur information is calculated in S504, then in S505, the blur information for generating video frames is calculated from this first blur information. The method for calculating the blur information for generating video frames is, for example, to calculate the blur information for the unexposed time as the blur information for generating video frames from the ratio of the first exposure time 300a to the unexposed time 303. Also, if the unexposed time 303 is 0 seconds, the blur information for generating video frames is calculated from the ratio of the first exposure time 300a to the second exposure time 301a. After that, this process ends.

[0044] Figure 7 is a flowchart showing the procedure for the blur addition process in S402 of Figure 4. The blur addition process is performed by the blur addition unit 203 according to the control of the CPU 101.

[0045] In Figure 7, first, at S700, the blur addition unit 203 acquires multiple images obtained by the imaging unit 104, similar to S500 described above.

[0046] Next, in S701, the blur addition unit 203 obtains the exposure time at the time of image capture for each acquired image from the exposure time setting unit 201, similar to S501 described above. Also, in S702, the blur addition unit 203 obtains the unexposed time from the exposure time setting unit 201, similar to S502 described above.

[0047] Next, in S703, the blur addition unit 203, similar to S503 described above, determines for each acquired image whether the exposure time for that image is longer than the threshold T.

[0048] In step S703, the blur addition unit 203 does not perform any processing on the image whose exposure time is determined to be longer than the threshold T, that is, the image with the first exposure time 300a, and the process ends. Thus, in this embodiment, no blur addition processing is performed on the image with the first exposure time 300a.

[0049] In S703, the image whose exposure time is determined to be less than or equal to the threshold T, that is, the image with the second exposure time 301a, is subjected to the processes in S704 to S706, which add blur. Thus, in this embodiment, the image with the second exposure time 301a is subjected to a process to add blur.

[0050] In S704, the blur addition unit 203 acquires blur information for generating video frames for each region, which is calculated in the blur information acquisition process described above. Here, for example, blur information for generating video frames is acquired based on the image frame obtained in the image taken immediately before the target image.

[0051] Next, in S705, the blur addition unit 203 generates a blur filter for each region based on the acquired blur information for each region.

[0052] Next, in S706, the generated region-specific blur filter is applied to each pixel of the target image. After that, this process is terminated.

[0053] According to the embodiment described above, based on the blur information acquired by the blur information acquisition unit 202 from the image (interpolated frame 300) obtained from the first exposure time 300a, blur information for generating video frames is calculated, and video frames are generated using the blur information for generating video frames. This makes it possible to generate video frames suitable for video with a natural connection between preceding and succeeding images, and thus it is possible to generate video frames suitable for video using multiple images.

[0054] Furthermore, in the embodiment described above, the second exposure time 301a is an exposure time suitable for still images, and a still image is generated based on the image (still image frame 301) obtained during shooting at the second exposure time 301a. This makes it possible to simultaneously obtain smooth video with a natural connection between preceding and succeeding images, and still images with minimal blurring.

[0055] Furthermore, in the embodiment described above, a point image distribution function is calculated for a region extracted from the image obtained during imaging with a first exposure time of 300a, and blur information is obtained from the image based on the point image distribution function. This makes it possible to calculate blur information for generating video frames based on the degree of blur in the image obtained during imaging with a first exposure time of 300a.

[0056] Furthermore, in the embodiment described above, blur information for generating video frames (blur information for the unexposed time 303) is calculated based on the ratio of the first exposure time 300a to the unexposed time 303. This allows the image obtained from imaging with the second exposure time 301a (still image frame 301) to be processed as if the unexposed time 303 period had also been exposed, thereby generating smooth video frames.

[0057] In the above-described embodiment, a configuration was described in which blurring is added to the image (still image frame 301) obtained by imaging with a second exposure time 301a, but the configuration is not limited to this. For example, blurring may be added to the image (interpolated frame 300) obtained by imaging with a first exposure time 300a.

[0058] Furthermore, in this embodiment, a motion vector may be calculated from an image in which the exposure time of the image capture is determined to be less than or equal to a threshold T, and a second blur information may be calculated based on this motion vector. In this configuration, the content of the blur information acquisition process in S401 differs in part from the flowchart in Figure 5(a) described above. The following will explain in particular the differences from the flowchart in Figure 5(a) described above.

[0059] Figure 8 is a flowchart showing another procedure for the shake information acquisition process in S401 of Figure 4. Note that the shake information acquisition process in Figure 8 is similar to the shake information acquisition process in Figure 5(a) described above, and the differences from the shake information acquisition process in Figure 5(a) described above will be explained below. The shake information acquisition process in Figure 8 is also implemented by the shake information acquisition unit 202 in accordance with the control of the CPU 101, similar to the shake information acquisition process in Figure 5(a) described above.

[0060] In Figure 8, steps S501 to S503 described above are performed first.

[0061] In S503, the image whose exposure time is determined to be longer than the threshold T, that is, the image obtained with the first exposure time of 300a (interpolated frame 300), is subjected to the process described in S504. In this way, the first blur information is calculated from the image obtained with the first exposure time of 300a (interpolated frame 300). After that, the process proceeds to S810.

[0062] On the other hand, for the image whose exposure time is determined to be less than or equal to the threshold T in S503, that is, the image obtained with the second exposure time 301a (still image frame 301), the processing in S800 to S809 is performed. In this way, a motion vector is calculated from the image obtained with the second exposure time 301a (still image frame 301), and second blur information is calculated based on this motion vector. Here, a motion vector is a vector that indicates how much and in which direction the region of each subject moved between images. When estimating motion vectors from multiple images, feature point matching, block matching, or optical flow estimation using the gradient method should be used. Note that other methods may be used to estimate motion vectors.

[0063] In S800, the blur information acquisition unit 202 performs subject area detection processing to detect the area of ​​the subject from the target image, similar to S511 described above.

[0064] Next, in S801, the blur information acquisition unit 202 determines whether or not the area of ​​the subject has been detected in the target image, similar to S512 described above. If it is determined that the area of ​​the subject has been detected in the target image, the process proceeds to S802.

[0065] In S802, the blur information acquisition unit 202 extracts the detected subject area from the image, similar to S513 described above. Next, in S803, the blur information acquisition unit 202 calculates a motion vector for each area of ​​the subject (motion vector calculation means).

[0066] Next, in S804, the blur information acquisition unit 202 extracts the background region from the target image, similar to S515 described above. Then, in S805, the blur information acquisition unit 202 calculates the motion vector of the background region (motion vector calculation means). After that, this process proceeds to S809, which will be described later.

[0067] If it is determined in S801 that no subject area is detected in the target image, the process proceeds to S806. In S806, the blur information acquisition unit 202 extracts a predetermined area (for example, areas 603a to 603i in Figure 6) from the target image, similar to S517 described above.

[0068] Next, in S807, the blur information acquisition unit 202 calculates a motion vector for each extracted region (motion vector calculation means). Then, in S808, the blur information acquisition unit 202 acquires the principal components of the motion vectors for each region and obtains a representative motion vector for the target image. Thus, in this embodiment, through the processing in S802 to S808, if the target image includes the region of the subject, the motion vectors for the region of the subject and the background region are calculated. On the other hand, if the target image does not include the region of the subject, a representative motion vector is calculated from a predetermined number of regions. As mentioned above, a motion vector is a vector that represents how much and in which direction each subject moved between images. In other words, a motion vector is synonymous with blur information between images. Therefore, in this embodiment, in S809, the blur information acquisition unit 202 converts the motion vector into blur information data and calculates second blur information. The calculated second blur information is, for example, the blur information for the period of the second exposure time 301a and the shooting interval 301b of the still image frame in Figure 3. Furthermore, the first blur information represents the blur information for the period of the first exposure time of 300a.

[0069] Next, in S810, the blur information acquisition unit 202 calculates blur information for generating video frames. For example, the blur information acquisition unit 202 calculates blur information for the unexposed time as blur information for generating video frames by acquiring the difference between the second blur information and the first blur information. Alternatively, if the unexposed time 303 is 0 seconds, the blur information acquisition unit 202 calculates blur information for generating video frames from the ratio of the first exposure time 300a and the second exposure time 301a. After that, this process ends.

[0070] In the embodiment described above, blur information for generating video frames is calculated based on the difference between the first blur information and the second blur information. This allows for more accurate calculation of the unexposed time, thereby enabling the generation of video frames with blur that creates a natural connection between preceding and succeeding images.

[0071] Furthermore, in the embodiment described above, the second blur information is obtained based on a motion vector calculated from the image obtained during the second exposure time 301a. This makes it possible to obtain blur information even from images with short exposure times and relatively little blur, such as the image obtained during the second exposure time 301a, and to use this blur information to generate motion frames with blur that create a natural connection between preceding and succeeding images.

[0072] The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0073] Furthermore, the disclosure of this embodiment includes the following configurations and methods. (Configuration 1) An imaging device comprising: a generation means for acquiring a plurality of consecutive images and generating a video based on the plurality of images; a setting means for setting a plurality of exposure times to be used for capturing the plurality of images; an acquisition means for acquiring blur information of the acquired images; and a calculation means for calculating blur information for generating video frames based on the blur information acquired by the acquisition means from images obtained by shooting with a first exposure time which is set to be longer than a second exposure time, wherein the generation means generates images constituting the video using the blur information for generating video frames. (Configuration 2) The imaging apparatus according to Configuration 1, wherein the second exposure time is an exposure time suitable for still images, and the generation means generates a still image based on an image obtained by shooting with the second exposure time. (Configuration 3) The imaging apparatus according to Configuration 1 or 2, characterized in that the acquisition means calculates a point image distribution function of a region extracted from an image obtained from imaging with the first exposure time, and acquires blur information from the image obtained from imaging with the first exposure time based on the point image distribution function. (Configuration 4) The imaging apparatus according to any one of Configurations 1 to 3, characterized in that the calculation means calculates blur information for generating the video frame based on the ratio of the first exposure time to the unexposed time. (Configuration 5) The imaging apparatus according to any one of Configurations 1 to 3, characterized in that the calculation means calculates the blur information for generating the video frame based on the difference between the blur information obtained from the image obtained from the first exposure time and the blur information obtained from the image obtained from the second exposure time. (Configuration 6) The imaging device according to Configuration 5, further comprising motion vector calculation means for calculating a motion vector of a subject from an image obtained from an image taken with the second exposure time, wherein the acquisition means acquires blur information from an image obtained from an image taken with the second exposure time based on the motion vector. (Configuration 7) The imaging apparatus according to any one of Configurations 1 to 6, further comprising means for applying a process to the image obtained in the second exposure time, which adds blur based on the blur information for generating the video frame, wherein the generation means generates images constituting the video using the image obtained in the first exposure time and the processed image obtained in the second exposure time, which has undergone the above processing. (Configuration 8) The imaging apparatus according to any one of Configurations 1 to 6, further comprising means for applying a process to the image obtained from the first exposure time to add blur based on the blur information for generating the video frame, wherein the generation means generates images constituting the video using the processed image obtained from the first exposure time to which the process has been applied, and the image obtained from the second exposure time. (Configuration 9) A method for controlling an imaging device, comprising: a generation step of acquiring a plurality of consecutive images and generating a video based on the plurality of images; a setting step of setting a plurality of exposure times to be used for capturing the plurality of images; an acquisition step of acquiring blur information of the acquired images; and a calculation step of calculating blur information for generating video frames from images obtained by shooting with a first exposure time which is set and is longer than a second exposure time, based on the blur information acquired in the acquisition step, wherein the generation step generates images constituting the video using the blur information for generating video frames. (Configuration 10) A program that causes a computer to execute a control method for an imaging device, wherein the control method for the imaging device comprises a generation step of acquiring a plurality of consecutive images and generating a video based on the plurality of images; a setting step of setting a plurality of exposure times to be used for capturing the plurality of images; an acquisition step of acquiring blur information of the acquired images; and a calculation step of calculating blur information for generating video frames from images obtained by shooting with a first exposure time which is set and is longer than a second exposure time, based on the blur information acquired in the acquisition step, wherein the generation step generates images constituting the video using the blur information for generating video frames. [Explanation of Symbols]

[0074] 100 Imaging device 101 CPU 104 Imaging Unit 201 Exposure time setting section 202 Blur Information Acquisition Unit 203 Brake-adding section 204 Synthesis section

Claims

1. A generation means that acquires a series of images and generates a video based on the series of images, Setting means for setting multiple exposure times used for capturing the multiple images, An acquisition means for acquiring blur information of the acquired image, The system includes a calculation means for calculating motion blur information for generating motion picture frames based on motion blur information acquired by the acquisition means from an image obtained from a first exposure time which is set longer than the second exposure time, which is set above. The generation means is an imaging device characterized by generating images that constitute the video using the blur information for generating the video frames.

2. The second exposure time is an exposure time suitable for still images. The imaging apparatus according to claim 1, characterized in that the generation means generates a still image based on the image obtained from the second exposure time.

3. The imaging apparatus according to claim 1, characterized in that the acquisition means calculates a point image distribution function of a region extracted from an image obtained by imaging with the first exposure time, and acquires blur information from the image obtained by imaging with the first exposure time based on the point image distribution function.

4. The imaging apparatus according to claim 1, characterized in that the calculation means calculates blur information for generating video frames based on the ratio of the first exposure time to the unexposed time.

5. The imaging apparatus according to claim 1, characterized in that the calculation means calculates the blur information for generating the video frame based on the difference between the blur information obtained from the image obtained from the first exposure time and the blur information obtained from the image obtained from the second exposure time.

6. The system further includes motion vector calculation means for calculating the motion vector of a subject from an image obtained during the second exposure time, The imaging apparatus according to claim 5, characterized in that the acquisition means acquires blur information from the image obtained in the second exposure time based on the motion vector.

7. The system further includes means for applying a process to the image obtained from the second exposure time, which adds blur based on the blur information for generating the video frame, The imaging apparatus according to claim 1, characterized in that the generation means generates images constituting the video using the image obtained from the first exposure time and the processed image obtained from the second exposure time, after applying the processing described above.

8. The system further includes means for applying a process to the image obtained from the first exposure time to add blur based on the blur information for generating the video frame, The imaging apparatus according to claim 1, characterized in that the generation means generates images constituting the video using the processed image obtained by applying the processing to the image obtained by shooting with the first exposure time, and the image obtained by shooting with the second exposure time.

9. A method for controlling an imaging device, A generation process that acquires multiple consecutive images and generates a video based on the multiple images, A setting step of setting multiple exposure times used for capturing the multiple images, The acquisition process involves acquiring blur information from the acquired image, The system includes a calculation step which calculates motion blur information for generating video frames based on motion blur information acquired in the acquisition step from an image obtained by taking a picture for a first exposure time which is set longer than the second exposure time set, The generation step is a control method for an imaging device characterized by generating images that constitute the video using blur information for generating the video frames.

10. A program that causes a computer to execute a control method for an imaging device, The control method for the imaging device is as follows: A generation process that acquires multiple consecutive images and generates a video based on the multiple images, A setting step of setting multiple exposure times used for capturing the multiple images, The acquisition process involves acquiring blur information from the acquired image, The system includes a calculation step which calculates motion blur information for generating video frames based on motion blur information acquired in the acquisition step from an image obtained by taking a picture for a first exposure time which is set longer than the second exposure time set, The generation step is a program characterized by generating images that constitute the video using the blur information for generating the video frames.

Images