Information processing device and method, camera device and control method thereof, and storage medium

Abstract

The present invention relates to an information processing device and method, an imaging device, a control method thereof, and a storage medium. The information processing device includes: an acquisition component for acquiring a plurality of first captured images captured by a first imaging process using first imaging parameters, and acquiring motion information related to a subject in the plurality of first captured images; an estimation component for estimating, based on the second imaging parameters, the motion blur of the subject in a second captured image captured by the second imaging process to be performed using second imaging parameters; and a setting component for setting the first imaging parameters and the second imaging parameters. The acquisition component acquires motion information from the plurality of first captured images captured using the first imaging parameters changed in such a manner that the changed first imaging parameters correspond to the exposure time of the second imaging process associated with the second imaging parameters.

Description

Technical Field

[0001] The present invention relates to a technique for notifying blurring of a subject in a captured image. Background Art

[0002] Some imaging devices, such as digital still cameras, have an imaging mode that prioritizes shutter speed. In imaging mode, the user sets a desired shutter speed, and the imaging device automatically sets exposure settings other than the shutter speed, such as the aperture value and International Organization for Standardization (ISO) sensitivity. Using this mode, for example, the user can set a high shutter speed before capturing an image to capture an image with less motion blur.

[0003] Japanese Patent Application Laid-Open No. 2008-172667 discusses a technique for detecting moving areas from time-series images captured during preliminary imaging and highlighting these moving areas. Preliminary imaging, as used herein, refers to imaging performed prior to main imaging, in which the user sets the composition and imaging conditions while viewing the electronic viewfinder or rear LCD display of the imaging device. Furthermore, main imaging is triggered by the user pressing the shutter button and performed by the imaging device based on the composition and imaging conditions set during preliminary imaging.

[0004] However, Japanese Patent Application Laid-Open No. 2008-172667 fails to mention capturing images at a frame rate and shutter speed appropriate for the subject's speed and amount of movement in order to extract motion regions from time-series images. For example, capturing a portrait image at night results in a longer exposure time due to the reduced amount of light collected by the imaging device compared to daytime. Therefore, even slight shaking or movement of a person can cause subject blur. However, an appropriate frame rate value is not always pre-set to detect slight subject movement, leading to a problem in which the user is not adequately informed of the occurrence of subject blur. Summary of the Invention

[0005] According to one aspect of the present invention, an information processing device includes: an acquisition unit configured to acquire a plurality of first captured images captured by first imaging using first imaging parameters and acquire motion information related to a subject in the plurality of first captured images; an estimation unit configured to estimate, for a second imaging to be performed using second imaging parameters set independently of the first imaging parameters, motion blur of the subject in a second captured image captured by the second imaging based on the motion information and the second imaging parameters; a setting unit configured to set the first imaging parameters and the second imaging parameters; and a notification unit configured to notify information related to the motion blur. The acquisition unit acquires the motion information from the plurality of first captured images captured using the first imaging parameters changed so that the changed first imaging parameters correspond to an exposure time of the second imaging associated with the second imaging parameters.

[0006] According to another aspect of the present invention, an information processing device includes: an acquisition unit configured to acquire a plurality of first captured images captured by first imaging using first imaging parameters and acquire motion information related to a subject in the plurality of first captured images; an estimation unit configured to estimate, for a second imaging to be performed using second imaging parameters set independently of the first imaging parameters, motion blur of the subject in a second captured image captured by the second imaging based on the motion information and the second imaging parameters; a setting unit configured to set the first and second imaging parameters; and a notification unit configured to notify information related to the motion blur. The acquisition unit selects an image for use in acquiring the motion information from the plurality of first captured images based on an exposure time of the second imaging associated with the second imaging parameters.

[0007] According to another aspect of the present invention, an imaging device includes an imaging component, and the imaging device is configured to, in response to an imaging instruction issued by a user during the imaging component sequentially outputting a plurality of first captured images captured by first imaging using first imaging parameters, output a second captured image captured based on the imaging instruction by second imaging using second imaging parameters. The imaging device further includes: an acquisition component configured to acquire motion information based on the plurality of first captured images output from the imaging component; an estimation component configured to estimate, for the second imaging to be performed using the second imaging parameters set independently of the first imaging parameters, motion blur of a subject in the second captured image captured by the second imaging based on the motion information and the second imaging parameters; a setting component configured to set the first imaging parameters and the second imaging parameters; and a notification component configured to notify information related to the motion blur. The acquisition component acquires the motion information from the multiple first captured images, and the multiple first captured images are captured using the first imaging parameter changed in a manner corresponding to the changed first imaging parameter and the exposure time of the second imaging related to the second imaging parameter.

[0008] According to another aspect of the present invention, an imaging device includes an imaging component, and the imaging device is configured to, in response to a user's imaging instruction issued during the imaging component sequentially outputting a plurality of first images captured by first imaging using first imaging parameters, output a second image captured based on the imaging instruction by second imaging using second imaging parameters. The imaging device further includes: an acquisition component configured to acquire motion information based on the plurality of first images outputted from the imaging component; an estimation component configured to estimate, for the second imaging to be performed using second imaging parameters set independently of the first imaging parameters, motion blur of a subject in the second image captured by the second imaging based on the motion information and the second imaging parameters; a setting component configured to set the first and second imaging parameters; and a notification component configured to notify information related to the motion blur. The acquisition component selects an image from the plurality of first images for use in acquiring the motion information based on an exposure time of the second imaging associated with the second imaging parameters.

[0009] According to another aspect of the present invention, an information processing method includes: acquiring a plurality of first captured images captured by first imaging using first imaging parameters, and acquiring motion information related to a subject in the plurality of first captured images; estimating, for second imaging to be performed using second imaging parameters set independently of the first imaging parameters, motion blur of the subject in the second captured images captured by the second imaging based on the motion information and the second imaging parameters; setting the first imaging parameters and the second imaging parameters; and notifying information related to the motion blur. The motion information is acquired from the plurality of first captured images captured using the first imaging parameters changed so that the changed first imaging parameters correspond to an exposure time of the second imaging associated with the second imaging parameters.

[0010] According to another aspect of the present invention, an information processing method includes: acquiring a plurality of first captured images captured by first imaging using first imaging parameters, and acquiring motion information related to a subject in the plurality of first captured images; estimating, for second imaging to be performed using second imaging parameters set independently of the first imaging parameters, motion blur of the subject in a second captured image captured by the second imaging based on the motion information and the second imaging parameters; setting the first imaging parameters and the second imaging parameters; and notifying information related to the motion blur. An image to be used for acquiring the motion information is selected from the plurality of first captured images based on an exposure time of the second imaging associated with the second imaging parameters.

[0011] According to another aspect of the present invention, a control method for an imaging device includes an imaging component and is configured to, in response to a user's instruction issued during the imaging component sequentially outputting a plurality of first captured images captured by first imaging using first imaging parameters, output a second captured image captured based on the instruction by second imaging using second imaging parameters. The control method includes: acquiring motion information based on the plurality of first captured images outputted from the imaging component; estimating motion blur of a subject in the second captured image captured by the second imaging using the second imaging parameters set independently of the first imaging parameters, based on the motion information and the second imaging parameters; setting the first and second imaging parameters; and notifying information related to the motion blur. The motion information is acquired from the plurality of first captured images captured using the first imaging parameters changed so that the changed first imaging parameters correspond to an exposure time of the second imaging associated with the second imaging parameters.

[0012] According to another aspect of the present invention, a method for controlling an imaging device includes an imaging component and is configured to, in response to a user's instruction issued during the imaging component sequentially outputting a plurality of first captured images captured by first imaging using first imaging parameters, output a second captured image captured based on the instruction by second imaging using second imaging parameters. The method includes: acquiring motion information based on the plurality of first captured images output from the imaging component; estimating motion blur of a subject in the second captured image captured by the second imaging using the second imaging parameters set independently of the first imaging parameters based on the motion information and the second imaging parameters; setting the first and second imaging parameters; and notifying information related to the motion blur. An image for use in acquiring the motion information is selected from the plurality of first captured images based on an exposure time of the second imaging associated with the second imaging parameters.

[0013] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a diagram showing an example of a structure for implementation of the present invention.

[0015] Figure 2 1 is a flowchart illustrating a process of image capturing by the image capturing apparatus 100 .

[0016] Figure 3 : is a diagram showing an example of the structure of the motion blur notification image generation unit according to the first embodiment.

[0017] Figure 4 is a diagram showing the first embodiment of the present invention. Figure 2 FIG. 5 is a diagram of the processing of step S205 in .

[0018] Figure 5A and Figure 5B : is a diagram showing the relationship between the amount of motion of a subject and the allowable amount of motion.

[0019] Figure 6A and Figure 6B is a diagram showing preliminary images and motion vectors.

[0020] Figure 7 is a flowchart showing a motion vector calculation process.

[0021] Figure 8 3 is a diagram illustrating a motion vector calculation process.

[0022] Figures 9A to 9C is a diagram illustrating motion blur notification processing.

[0023] Figure 10A and Figure 10B : is a timing chart showing image acquisition timing in preliminary imaging and main imaging.

[0024] Figure 11 : is a diagram showing the exposure time in the main imaging and the frame period in the preliminary imaging.

[0025] Figure 12 : is a graph showing the integration of the amount of motion in a frame.

[0026] Figure 13A and Figure 13B is a diagram showing frame rate control.

[0027] Figure 14 The second embodiment of the present invention is shown in FIG. Figure 2 FIG. 5 is a diagram of the processing of step S205 in . DETAILED DESCRIPTION

[0028] Some embodiments of the present invention will be described below with reference to the accompanying drawings. The various embodiments of the present invention described below can be implemented individually or as a combination of multiple embodiments or features of these embodiments when necessary or when it is beneficial to combine elements or features of the various embodiments into one embodiment.

[0029] According to the first embodiment of the present invention, motion blur in main imaging is estimated using motion information calculated based on images during preliminary imaging, and a motion blur notification image indicating motion blur estimated to have occurred in main imaging is displayed to the user as a notification.

[0030] Figure 1 1 is a block diagram showing an example of a structure according to the first embodiment, and shows an example of an image pickup apparatus. Figure 1 An example of the structure according to the first embodiment of the present invention will be described.

[0031] The control unit 101 is, for example, a central processing unit (CPU). The control unit 101 reads a control program for each block of the imaging device 100 (e.g., a camera) from a read-only memory (ROM) 102 described below, loads the read control program into a random access memory (RAM) 103 described below, and runs the loaded control program, thereby controlling the operation of each block of the imaging device 100. The ROM 102 is a non-volatile electrically erasable and recordable memory. The ROM 102 stores the operating program of each block of the imaging device 100 and parameters used for the operation of each block. The RAM 103 is a volatile rewritable memory. The RAM 103 is used to load the program executed by the control unit 101 and temporarily store data generated during the operation of each block of the imaging device 100.

[0032] The optical system 104 includes a lens group including a zoom lens and a focus lens, and forms a subject image on an imaging surface of the imaging unit 105. The imaging unit 105 is an image sensor such as a charge-coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor. The imaging unit 105 photoelectrically converts the optical image formed on the imaging surface of the imaging unit 105 by the optical system 104 into an analog image signal, and outputs the analog image signal to the analog / digital (A / D) conversion unit 106. The A / D conversion unit 106 converts the input analog image signal into digital image data. The digital image data output from the A / D conversion unit 106 is temporarily stored in the RAM 103.

[0033] The image processing unit 107 performs various types of image processing, such as white balance adjustment, color interpolation, and gamma processing, on the image data stored in the RAM 103. Furthermore, the image processing unit 107 includes a motion blur notification image generation unit 300, described below, and generates a motion blur notification image by superimposing an image plane on the image stored in the RAM 103 to facilitate motion blur inspection. The recording unit 108 is a recording medium such as a removable memory card. The recording unit 108 records the image data processed by the image processing unit 107 as a recorded image via the RAM 103.

[0034] The display unit 109 is a display device such as a liquid crystal display (LCD). The display unit 109 displays images stored in the RAM 103, images recorded in the recording unit 108, and an operation user interface for receiving user instructions. Furthermore, the display unit 109 displays images captured by the imaging unit 105 for use in adjusting the composition during preliminary imaging. The instruction input unit 110 is a touch panel and a mouse, and inputs user instructions.

[0035] Next, the following will refer to Figure 2 The processing described below is executed by the control unit 101 controlling each unit based on the program stored in the ROM 102. Figure 2 The flowchart is implemented in response to the start of preliminary imaging.

[0036] In step S201, the user turns on the imaging apparatus 100. Then, in step S201, when the imaging apparatus 100 is turned on, the control unit 101 controls the optical system 104 and the imaging unit 105 and starts preliminary imaging. The control unit 101 maintains a predetermined frame rate during preliminary imaging, and the imaging apparatus 100 sequentially captures images. The acquired captured images are displayed on the display unit 109. The user can adjust the composition while checking the sequentially displayed preliminary images.

[0037] In step S202, the imaging device 100 sets the imaging conditions (imaging parameters) for the preliminary image to be captured in order to detect the amount of motion of the subject in the composition. The imaging conditions here mainly refer to the frame rate and shutter speed. The highest frame rate and high shutter speed are set within a range that does not affect the processing for calculating the evaluation value used in the general automatic function control performed by the imaging device (such as automatic exposure (AE) and automatic focus (AF) control, etc.). In addition, the optical system 104 is controlled to appropriately adjust the shutter speed by changing the lens aperture or the ISO sensitivity setting of the imaging unit 105, so that the image will be captured under appropriate exposure conditions even at the set high shutter speed. The imaging device 100 performs imaging under the imaging conditions set in step S202 during the preliminary imaging. At a higher shutter speed, the blur of the moving subject in the captured image accumulates less, and the amount of movement of the subject between consecutive captured images is reduced. This enables detailed subject motion detection.

[0038] In step S203, the user sets the imaging parameters of the main camera using the instruction input unit 110. The control unit 101 can set the imaging parameters of the main camera independently of the imaging parameters of the preliminary camera based on the input from the instruction input unit 110. For example, the exposure time can be set as the imaging parameter of the main camera. The imaging parameters of the main camera can be automatically set by the imaging device 100. According to this embodiment, the set imaging parameters of the main camera are used as the imaging parameters of the main camera after the pressing of the shutter button (for the instruction to perform the main camera) described below is detected.

[0039] In step S204, the control unit 101 determines whether the motion blur notification setting is set to "ON" or "OFF." The motion blur notification setting is set to "ON" or "OFF" by the user using the instruction input unit 110, for example. Once the motion blur notification setting is set, the setting value is maintained. If the control unit 101 determines in step S204 that the motion blur notification setting is "ON" ("YES" in step S204), then in step S205, the motion blur notification image generation unit 300, described below, generates a motion blur notification image by superimposing a motion blur notification plane on the preliminary image. Then, in step S206, the control unit 101 displays the generated motion blur notification image on the display unit 109.

[0040] On the other hand, if the control unit 101 determines in step S204 that the motion blur notification setting is "off" ("No" in step S204), no motion blur notification image is generated. Therefore, in step S206, a preliminary image without motion blur notification is displayed on the display unit 109.

[0041] In step S207, the control unit 101 determines whether the shutter button of the instruction input unit 110 has been pressed by a user operation. For a two-stage input to be received (for example, half-pressing the shutter button to input an imaging preparation operation instruction and fully pressing the shutter button to issue a main imaging instruction), the control unit 101 determines whether the shutter button has been fully pressed. For a single input to be received, the control unit 101 determines whether a single input has been made.

[0042] If the control unit 101 determines that the shutter button has not been pressed ("No" in step S207), the process returns to step S202, and steps S202 to S206 are performed again. Therefore, during the preparatory imaging, the user can easily check the motion blur of the subject that is likely to occur in the main imaging performed using the currently set imaging parameters. If the checked motion blur is not the motion blur that the user wants, the user does not press the shutter button to return to step S202, and in step S203, the user can set the shutter speed (exposure time) of the main imaging again. As described above, the user repeats the process for setting the exposure time of the main imaging while checking the motion blur notification image displayed on the display unit 109 during the preparatory imaging until the desired motion blur is obtained, and when the opportunity to capture an image comes, the user presses the shutter button.

[0043] In step S207, if the control unit 101 determines that the shutter button has been pressed ("YES" in step S207), the control unit 101 determines that a main image capture instruction has been received, and the process proceeds to step S208. In step S208, the control unit 101 controls the optical system 104 and the image capture unit 105, and performs main image capture using the image capture parameters set by the preliminary image capture. The image captured in the main image capture is output by the control unit 101 to the display unit 109 and the recording unit 108. The output image is displayed on the display device of the display unit 109, recorded on the recording medium of the recording unit 108, or output to an external device.

[0044] Next, the following will refer to Figure 3 An example of the structure of the motion blur notification image generating unit 300 of the image processing unit 107 which is a feature of the present invention will be described.

[0045] Figure 3is a diagram showing an example of the structure of the motion blur notification image generation unit 300. The motion blur notification image generation unit 300 includes a motion vector calculation unit 301 and an estimated motion blur calculation unit 302. The motion vector calculation unit 301 calculates the motion vector of the subject by comparing images. The estimated motion blur calculation unit 302 estimates the motion blur of the subject in the main camera image based on the calculated motion vector. The motion blur notification image generation unit 300 also includes a notification plane generation unit 303 and an image superposition unit 304. The notification plane generation unit 303 generates data for motion blur notification based on the estimated motion blur of the subject. According to this embodiment, the motion blur notification plane is superimposed on the captured image, and the image superposition unit 304 performs superposition processing. The details of the operation of these units will be described below.

[0046] Figure 3 One or more of the functional blocks shown may be implemented by hardware such as an application specific integrated circuit (ASIC) or a programmable logic array (PLA), by a programmable processor such as a CPU or a microprocessor unit (MPU) running software, or by a combination of software and hardware. Therefore, different functional blocks described below as operation executors may be implemented by the same hardware as the executor.

[0047] Next, the following will refer to Figure 4 The flowchart shown in FIG. 1 is used to explain the operation performed by the motion blur notification image generation unit 300. Figure 2 The steps shown in this flowchart are performed by the control unit 101 or by each unit of the imaging apparatus 100 including the motion blur notification image generation unit 300 based on an instruction from the control unit 101.

[0048] In step S401, the control unit 101 sets the imaging conditions set in step S202. Then, the control unit 101 controls the optical system 104 and the imaging unit 105, and the imaging apparatus 100 captures consecutive preliminary images for detecting subject motion blur in main imaging.

[0049] In step S402 , the control unit 101 acquires shutter speed information for main imaging set by the user or automatically set by the camera.

[0050] In step S403, the control unit 101 selects a preliminary image frame for use in subject motion detection (step S404 described below) based on the frame rate information of the preliminary imaging and the shutter speed of the main imaging. Figure 10A 、 Figure 10B and Figure 11 The method of selecting the preliminary image frame will be described in detail.

[0051] Figure 10A and Figure 10B 1 is a timing chart showing the timing at which a frame ( 1001 ) is acquired in the preliminary imaging and the timing at which an image ( 1011 ) is acquired in the main imaging, wherein the horizontal axis represents time T.

[0052] The horizontal axis represents the time of recording. Figure 10A Control 1021 is shown in FIG, and Figure 10B Frame selection 1022 is shown in the figure. The amount of motion of the subject at a given moment is calculated using two frames connected by arrows, representing selected preliminary image frames for use in calculating the amount of motion of the subject. Control 1021 represents control for selecting two preceding and succeeding frames when performing motion detection, and frame selection 1022 represents an example of preliminary image frame selection based on the frame rate information used in preliminary imaging according to this embodiment and the shutter speed of the main imaging. Captured image frame selection 1022 represents a method typically used when calculating the amount of motion of the subject, and according to this embodiment, a captured image frame is selected as indicated by frame selection 1022.

[0053] The following will refer to Figure 11 The method of selecting a preliminary image frame according to this embodiment is described below.

[0054] Figure 111 is a diagram showing the relationship between the frame time period of a plurality of preliminary images (1111 to 1114) and the exposure time of an image (1101) captured in the main camera at the shutter speed of the main camera. Specific numerical values ​​will be used in the following description. If the shutter speed of the main camera is set to 1 / 30, an image is captured with an exposure time of 1 / 30 second in the main camera. Specifically, in the case of slight subject movement during 1 / 30 second, an image with less cumulative blur is acquired. On the other hand, since the preliminary image is used for motion analysis, the preliminary image is captured at a shutter speed that makes cumulative blur less likely to occur between frames. For example, a plurality of frames are captured with an exposure time shorter than 1 / 120 second. The frame rate of the preliminary camera for performing continuous camera shooting is set to 120fps (continuous camera shooting 120 images in one second). In order to detect subject movement during the exposure time of the main camera, the control unit 101 selects the frame of the preliminary image so that the frame time period of the preliminary image is close to the exposure time of the main camera. Since adjacent frames of the preliminary images 1111 to 1114 have equal time periods, the four-frame time period (from 1111 to 1114) is 1 / 30 second, which is the same time as the exposure time of the main camera. This means that a frame including the subject motion during the time corresponding to the exposure time of the main camera (i.e., 1 / 30 second) is selected. The smaller the difference between the frame time period of the selected preliminary image frame and the shutter speed of the main camera, the smaller the error during step S405 described below, thereby providing an accurate estimate. Therefore, the control unit 101 selects the frame of the preliminary image so that the frame time period of the preliminary image is as close as possible to the shutter speed set for the main camera. Therefore, the control unit 101 selects a preliminary image with a frame time period that satisfies the following formula (2).

[0055] F_Period (frame time period) ≥ (shutter speed of main camera × frame rate of standby camera) × α Formula (2)

[0056] In formula (2), F_Period is the minimum integer that satisfies the formula, and α is a constant less than or equal to 1.

[0057] The control unit 101 selects preliminary image frames for calculating the amount of motion of the subject that satisfy Expression (2) so that the selected frames are close to a time period corresponding to the shutter speed set for the main imaging.

[0058] The above reference Figure 11 The relationship between the frame time period of the preliminary image and the shutter speed of the main camera is described. According to this embodiment, the frame time period of the preliminary image is controlled by the control unit 101 to select the above-mentioned Figure 10B1022 frames are selected from the frames in the equation (2). In addition, at higher ISO sensitivities, the constant α in equation (2) is ideally set to a smaller number. For strict imaging conditions, the use of the constant α prevents an excessively long frame time period. For example, in a case where the shutter speed of the main camera is long and the frame time period of the preliminary image is too long, complex subject motion may be included between frames, and this may result in a decrease in the accuracy of subject motion detection. Using different constants α for different shutter speeds of the main camera prevents a decrease in the accuracy of subject motion detection.

[0059] exist Figure 4 In step S404, the preliminary image frame selected in step S403 is input to the motion blur notification image generation unit 300, and the motion vector calculation unit 301 calculates the motion amount of the subject using the input image. Figure 6A The calculation of the motion amount of the subject by the motion vector calculation unit 301 is described below. Figure 6A A captured image of a scene including a dog 601 running to the left and a dog 602 standing still is shown.

[0060] The motion vector calculation unit 301 calculates the motion vector between the consecutive preliminary images in the time series as the motion amount of the subject. The motion vector is a vector indicating the horizontal movement amount of the subject and the vertical movement amount of the subject between the preliminary images. Figure 7 and Figure 8 The motion vector calculation method is described in detail.

[0061] Figure 7 is a flowchart illustrating motion vector calculation processing. While the block matching method will be described below as an example of a motion vector calculation method according to aspects of the present invention, the motion vector calculation method is not limited to this example and may be, for example, a gradient method. The steps shown in this flowchart are performed by the control unit 101 or by various units of the imaging apparatus 100, including the motion blur notification image generation unit 300, based on instructions from the control unit 101.

[0062] In step S701 , the control unit 101 receives two temporally adjacent preliminary images acquired during preliminary imaging, and sets the preliminary image of the Mth frame as a standard frame and the preliminary image of the (M+1)th frame as a reference frame.

[0063] In step S702, if Figure 8 As shown, the control unit 101 places a standard block 802 of N×N pixels in a standard frame 801.

[0064] In step S703, Figure 8As shown, the control unit 101 sets (N+n)×(N+n) pixels around coordinates 804 corresponding to the coordinates of the center of the standard block 802 in the standard frame 801 as a search range 805 in the reference frame 803 .

[0065] In step S704, the control unit 101 performs a correlation operation on the standard block 802 in the standard frame 801 and the reference block 806 of N×N pixels at different coordinates within the search range 805 in the reference frame 803, and calculates a correlation value. The correlation value is calculated based on the sum of absolute differences (SAD) between the pixels in the standard block 802 in frames 801 and 803 and the corresponding pixels in the reference block 806. Specifically, the coordinates with the smallest value of the sum of absolute differences between frames 801 and 803 are the coordinates with the highest correlation value. The correlation value calculation method is not limited to the method by calculating the SAD between frames, and may be, for example, a method of calculating the correlation value based on the sum of square differences or normalized mutual correlation values ​​between frames. Figure 8 In the example of , reference block 806 is indicated as having the highest correlation.

[0066] In step S705, the motion vector calculation unit 301 calculates a motion vector based on the coordinates having the highest correlation value calculated in step S704 in the reference block 806. Figure 8 In the example shown in FIG, a motion vector is calculated based on coordinates 804 corresponding to the coordinates of the center of standard block 802 in standard frame 801 within search range 805 in reference frame 803, and the coordinates of the center of reference block 806. Specifically, the distance and direction from coordinates 804 to the coordinates of the center of reference block 806 are calculated as the motion vector.

[0067] In step S706, the motion vector calculation unit 301 determines whether motion vectors have been calculated for all pixels of the standard frame 801. In step S706, if the motion vector calculation unit 301 determines that motion vectors have not been calculated for all pixels ("No" in step S706), the process returns to step S702. Then, in step S702, the standard block 802 of N×N pixels is placed in the standard frame 801 centered at a pixel for which a motion vector has not been calculated, and steps S703 to S705 are performed as described above. Specifically, the motion vector calculation unit 301 calculates the motion vectors for all pixels of the standard frame 801 by moving the standard block 802 to the pixel for which the motion vector has not been calculated. Figure 8 Repeat steps S702 to S705 while calculating the motion vectors of all pixels in the standard frame 801. Figure 6B An example of a motion vector is shown in . Figure 6B It shows Figure 6A FIG. 1 is a diagram showing an example of a motion vector in a preliminary image of FIG.

[0068] Figure 6AThe preliminary image is an example of a dog 601 running to the left. Figure 6B A representative example of a motion vector of a moving object is shown. Figure 6B In the example of FIG. 5 , the running dog 601 is detected as a motion vector to the left, while the remaining parts (such as the stationary dog ​​602 and the background fence) are detected as a motion vector 0 (not shown).

[0069] In addition, the motion vector calculation unit 301 may calculate the motion vector of each predetermined pixel instead of calculating the motion vector of all pixels. The motion vector between the frames of the preliminary image that are temporally adjacent is calculated through the above-mentioned process.

[0070] exist Figure 4 In step S405, the estimated motion blur calculation unit 302 calculates the relationship between the calculated amount of subject motion and the target amount of motion blur. If the time between the preliminary image frames selected in step S403 does not correspond to the shutter speed (exposure time) of the main camera, the subject blur in the main camera cannot be accurately calculated. Therefore, the amount of subject motion detected in step S404 is multiplied by the correction gain obtained using equation (4) to convert the amount of motion and estimate the motion blur in the main camera.

[0071] Correction gain = shutter speed of main camera / time between selected preliminary image frames (4)

[0072] Then, the control unit 101 determines whether the motion blur in the main camera image estimated by multiplying the correction gain of formula (4) (i.e., the estimated motion blur) is greater than or less than the target motion blur amount. The target motion blur amount here refers to the allowable motion blur amount in which the motion blur is less visible in the camera image taken at a predetermined shutter speed. The allowable motion blur amount is determined based on the size of the image sensor such as a CCD sensor or a CMOS sensor, the number of pixels, and the resolution of the display device used for display. For example, under the conditions of an Advanced Photo System C-type (APS-C) image sensor as the image sensor, 200,000 pixels as the number of pixels, a full high-definition (Full HD) (1920×1080 pixels) personal computer (PC) display as the display, and five pixels or less as the allowable motion blur amount, motion of less than or equal to five pixels between frames will be detected. Specifically, sufficient resolution is provided for detecting motion less than or equal to the allowable motion blur amount. The following will refer to Figure 5A and 5B Let's illustrate a specific example.

[0073] Figure 5A and Figure 5B A motion vector indicating the relationship between the amount of motion of the subject calculated from the preliminary image and the permissible amount of motion blur is shown. Figure 5AThis shows a case where a motion amount of a subject larger than an allowable motion blur amount is detected. Figure 5B The case where the motion amount of the subject is less than or equal to the permissible motion amount is shown. The relationship between the calculated motion amount of the subject and the target motion blur amount is expressed by equation (1).

[0074] n = subject's movement amount / allowable movement amount formula (1)

[0075] If n is greater than 1, the subject's motion exceeds the permissible motion amount, causing the control unit 101 to determine that subject blur will occur. On the other hand, if n is less than or equal to 1, the control unit 101 determines that the subject's motion does not exceed the permissible motion amount. Specifically, the control unit 101 determines that subject blur will be within the permissible range. The target permissible motion amount can be determined by a setting set by the user.

[0076] exist Figure 4 In step S406 , the motion blur notification image generation unit 300 generates a motion blur notification image according to the relationship between the motion amount of the subject calculated in step S405 and the target motion blur amount based on an instruction from the control unit 101 . Figures 9A to 9C The figure shows motion blur notification in the process of notifying the user of the motion blur level when performing main imaging. Therefore, the notification plane generation unit 303 converts the motion amount calculated from the preliminary image into a motion amount (converted motion blur amount) converted using the exposure conditions for main imaging, and generates a blur notification image that allows the converted motion amount to be visually checked.

[0077] First, the notification plane generation unit 303 uses the result of the equation (1) calculated in step S405 to generate an image plane for motion blur notification for each amount of motion of the subject. Thereafter, the image superimposition unit 304 superimposes the motion blur notification plane on the preliminary image and generates a motion blur notification image. Figures 9A to 9C To illustrate an example of blurring the notification image.

[0078] Figures 9A to 9C Three examples of the motion blur notification image are shown. The display unit 109 displays the motion blur notification image during preliminary image capture so that the user can easily check motion blur.

[0079] first, Figure 9A An example of motion blur notification using an icon display is shown. In step S405, the control unit 101 calculates the ratio of the motion blur amount of the subject whose n is greater than 1 as a result of equation (1) relative to the overall motion blur amount of the subject. If the ratio is greater than or equal to a predetermined value, a motion blur notification is generated. Figure 9AThe motion blur icon 901 shown serves as a motion blur notification plane, and is generated by drawing the motion blur icon 901 on the preliminary image. Figure 9A Motion blur notification image shown.

[0080] Figure 9B An example of motion blur notification using a motion blur frame display is shown. A method of generating a motion blur notification image using a motion blur frame display will be described. In step S405, the control unit 101 divides the image into regions and calculates the motion blur amount of the subject in each divided region where n is greater than 1 as a result of equation (1). Specifically, the control unit 101 generates a motion blur notification image for each divided region where the ratio is greater than or equal to a predetermined value. Figure 9B The motion blur frame 902 shown serves as a motion blur notification plane, and is generated by drawing the motion blur notification plane on the preliminary image. Figure 9B Motion blur notification image shown.

[0081] Figure 9C An example of a motion blur notification in which the edge where motion blur occurs is highlighted is shown. A method of generating a motion blur notification image by highlighting the motion blur edge will be described. In step S405, the notification plane generation unit 303 detects the edge strength in the preliminary image. When calculating the edge strength, an existing method such as a Sobel filter is used, and its detailed description will be omitted. Then, the notification plane generation unit 303 calculates the edge strength for each region where the edge strength value is greater than or equal to a predetermined value and n in equation (1) exceeds 1, as shown in FIG. Figure 9C As shown, a motion blur notification plane highlighting the motion blur edge 903 is generated, and the notification plane generation unit 303 generates a motion blur notification plane by superimposing the motion blur notification plane on the preliminary image. Figure 9C The motion blur notification image shown. Figure 9C In the example shown, motion-blurred edge 903 is indicated by bolding. Motion-blurred edges are highlighted to enable visual recognition of motion blur on a small portion of the subject in the primary image. Another example of highlighting is extracting pixels having an edge strength greater than or equal to a predetermined value and an estimated motion blur greater than or equal to a predetermined value, and then changing the hue, saturation, and / or brightness of the extracted pixels, for example, displaying the extracted pixels in red.

[0082] The above description explains that the motion blur notification image generation unit 300 performs Figure 4 The process of generating a motion blur notification image is performed by performing steps S401 to S406 in FIG. Figure 2The processing in the flowchart is performed by the control unit 101, or by each unit of the camera device 100 including the motion blur notification image generation unit 300 based on instructions from the control unit 101, which makes it possible to appropriately determine the camera conditions for preparatory camera shooting for checking motion blur that may occur in the main camera shooting.

[0083] According to one aspect of the present invention, an appropriate preliminary image for checking for motion blur that may occur in main imaging is selected, and motion is detected. This makes it possible to check for motion blur that may occur in main imaging.

[0084] According to one aspect of the present invention, the above control is performed to select a frame corresponding to the frame time period of the preliminary image and detect the amount of motion, thereby estimating the amount of subject blur that may occur during the exposure time of the main camera. For further detailed motion detection, all preliminary images included in the frame time period of the preliminary image corresponding to the exposure time of the main camera can be used to detect the amount of subject motion. For example, Figure 11 and Figure 12 In the case of the main camera exposure time Figure 11 (1 / 30 sec) and the frame rate of the prepared image is as follows Figure 11 If the frame rate is 120 fps, the corresponding time period of the preparatory image is a time period of four frames (1 / 30 seconds). Therefore, according to the first embodiment, the motion vector calculation unit 301 uses Figure 11 The motion amount of the subject is calculated by using the preliminary images 1111 and 1114 in FIG. Figure 12 The control unit 101 calculates the calculated motion amounts ( 1201 ) of the subject between the preliminary images 1111 and 1112, between the preliminary images 1112 and 1113, and between the preliminary images 1113 and 1114. Figure 12 This makes it possible to accurately calculate the motion of the subject during the 1 / 30 second period from preliminary image 1111 to preliminary image 1114, even if the subject moves in a complex manner (rotational motion, accelerated motion) during the 1 / 30 second period between preliminary images.

[0085] Furthermore, if the user capturing the image intentionally does not determine the permissible amount of motion, the image capture device 100 may set the size of the motion vector within the limits of motion detection to the permissible amount of motion. In this case, the control unit 101 is configured to determine whether the subject's motion exceeds the motion detection range. This determination may be based on whether the change in the sum of absolute differences (i.e., SAD) value during motion detection is less than a predetermined value, or using a method for measuring the motion of the image capture device 100, such as gyroscope information. If the control unit 101 determines that the permissible amount of motion has been exceeded, the shutter speed and / or frame rate is controlled to increase.

[0086] Although the motion blur notification method in which the display unit 109 according to an aspect of the present invention displays a motion blur notification has been described above as an example, the motion blur notification method is not limited to the above. For example, sound, light, or vibration may be used as the motion blur notification. Specifically, if the ratio of the number of pixels in which the motion blur is estimated to be greater than or equal to a predetermined value relative to the entire screen is higher than or equal to a predetermined ratio, a motion blur notification sound, notification light, or notification vibration is provided. According to the present embodiment, the structure of the notification plane generation unit 303 and the image superposition unit 304 and the processing of steps S405 and S406 are not used. Instead, the camera device 100 includes a speaker, and in step S205, together with the display of the preliminary image on the display unit 109, the control unit 101 causes the speaker to provide a notification sound, light up a notification light, or provide a notification vibration.

[0087] According to one aspect of the present invention, although the control unit 101 selects different frames of the preliminary image for motion detection for different shutter speeds of the main camera, the selected frame of the preliminary image for motion detection can be changed if the shutter speed of the main camera is stable. Controlling the frame of the preliminary image for motion detection when the shutter speed of the main camera is stable prevents an undesirable motion blur notification image from being displayed on the display unit 109 while the user is still confused about shutter speed control or when the evaluation value calculation by the control unit 101 is unstable.

[0088] An example of a method for determining whether subject motion is stable is as follows. Specifically, after the preparatory imaging conditions have been changed, the control unit 101 calculates whether the subject motion is stable as a change in the motion vector. If the change in the motion vector per unit time is significant, the control unit 101 determines that the subject motion is unstable and may change the preparatory imaging conditions again.

[0089] According to one aspect of the present invention, while the control unit 101 selects different frames of the preliminary image for motion detection in response to different shutter speeds for the main camera, in addition to selecting frames from the preliminary image, the shutter speed for the preliminary image can also be changed. For example, if the shutter speed for the main camera is longer than a predetermined value and the ISO sensitivity of the main camera is higher than a predetermined value, the shutter speed for the preliminary image is changed to a longer shutter speed. On the other hand, if the ISO sensitivity of the main camera is low, the shutter speed for the preliminary image is changed to a shorter shutter speed. Thus, the imaging conditions for the preliminary image for motion detection are balanced against the effects of noise and cumulative blur, enabling the capture of an appropriate preliminary image for motion detection.

[0090] A second embodiment will be described. This embodiment describes an example in which the frame rate of preliminary imaging is changed relative to the shutter speed of the main imaging to estimate the amount of subject blur in the main imaging and thereby generate a motion blur notification image. This embodiment provides efficient control of the frame rate of the preliminary image to notify subject blur in the main imaging.

[0091] According to this embodiment, the Figure 1 The structure of the imaging apparatus 100 in the embodiment will be described, and redundant description of the block diagram of the imaging apparatus 100 will be omitted. In addition, the entire processing performed by the control unit 101 according to the second embodiment is the same as that performed by the control unit 101 in the embodiment. Figure 2 The processing shown in the flowchart of is similar, and its redundant description will be omitted.

[0092] Next, the features of the present invention will be described in detail. Figure 2 The processing of steps S202 and S205 in .

[0093] In the process of step S202, the control unit 101 changes the frame rate of the preliminary imaging with respect to the shutter speed of the main imaging. Figure 13A and Figure 13B Provide explanation.

[0094] Figure 13A : is a timing chart showing typical preliminary imaging and main imaging. Figure 13B An example of changing the frame rate of the preliminary image capture with respect to the shutter speed of the main image capture according to the second embodiment is shown. As an example, the shutter speed of the main image capture is 1 / 30 second, and the frame rate before the change is Figure 13A The frame rate of the preliminary camera in the main camera 1311 is 120fps. When the shutter speed is 1 / 30 second, the image is captured with an exposure time of 1 / 30 second in the main camera 1311. Figure 13B As shown, the control unit 101 changes the frame rate of the preliminary shooting so that the changed frame rate is close to the exposure time of the main shooting. For example, if the frame rate of the preliminary shooting is 120 fps and the exposure time of the main shooting is adjusted to 1 / 30 second, the control unit 101 changes the frame rate of the preliminary shooting to 30 fps. Specifically, the frame rate of the preliminary shooting is expressed by the following equation (3).

[0095] The changed frame rate of the preliminary image = β / the shutter speed of the main image Formula (3)

[0096] In formula (3), β is a constant less than or equal to 1.

[0097] Similar to the constant α in equation (2) according to the first embodiment, the constant β is a predetermined constant that changes based on the ISO sensitivity and shutter speed of the main image capture. The control unit 101 changes the frame rate of the preliminary image capture so as to satisfy equation (3) so as to set a frame period that can appropriately detect subject blur at the shutter speed of the main image capture. Figure 13B shows the changed frame rate and the difference between the two Figure 13A Compared with the subject motion detection of adjacent frames used in the preliminary imaging performed at the frame rate shown, Figure 13B The frame time period (1321) in the preliminary imaging in the 1321 is increased and is a time period close to the exposure time of the main imaging 1311. Therefore, by controlling Figure 13B At the frame rate of the preliminary imaging shown, subject motion corresponding to the exposure time of the main imaging 1311 becomes detectable.

[0098] Will refer to Figure 14 The steps shown in the flowchart are performed by the control unit 101 or by each unit of the imaging apparatus 100 including the motion blur notification image generation unit 300 based on an instruction from the control unit 101.

[0099] In step S1401, the control unit 101 performs processing similar to step S401 and sets imaging conditions.Then, the control unit 101 controls the optical system 104 and the imaging unit 105, and the imaging apparatus 100 captures consecutive preliminary images.

[0100] In step S1402 , the control unit 101 performs processing similar to that of step S404 .

[0101] In step S1403 , the control unit 101 performs processing similar to that of step S405 , and the motion vector calculation unit 301 calculates the amount of motion of the subject from the preliminary image captured in step S1401 .

[0102] In step S1404 , the control unit 101 performs processing similar to that of step S406 , and the estimated motion blur calculation unit 302 calculates the relationship between the calculated motion amount of the subject and the target motion blur amount.

[0103] In step S205, the control unit 101 performs Figure 4 The notification plane generation unit 303 generates an image plane for motion blur notification based on the estimated motion blur amount in the main imaging in step S1404, and the image superimposition unit 304 generates a motion blur notification image by superimposing the motion blur notification plane on the preliminary image.

[0104] The above is a detailed description of the control unit 101 according to the second embodiment. Figure 2 The feature processing steps S202 and S205 in the entire process.

[0105] The structure according to the second embodiment enables the control unit 101 to control the frame rate of preliminary imaging for detecting subject motion that affects the exposure time of the main imaging. This allows accurate detection of motion blur in the main imaging. Furthermore, the absence of a structure that captures continuous preliminary images at a constant high frame rate for subject motion detection allows for accurate motion detection with reduced power consumption.

[0106] According to one aspect of the present invention, the control unit 101 selects different preliminary image frames for motion detection based on the shutter speed of the main camera. If the shutter speed of the main camera is higher than the maximum frame rate that can be set by the camera 100, the frame rate of the preliminary camera is fixed at the maximum value, and the preliminary image closest in time to the frame is selected for motion detection. This condition effectively eliminates motion blur in the main camera, allowing motion blur estimation to be omitted. If the shutter speed of the main camera is higher than a given value (i.e., the exposure time of the main camera is shorter than a given time), the frame rate can be fixed to a specific value.

[0107] Furthermore, a threshold value may be set for the shutter speed of the main camera, and the control unit 101 may control the frame rate of the preliminary camera based on the set value. For example, a first threshold value and a second threshold value greater than the first threshold value may be set for the shutter speed of the main camera. If the shutter speed of the main camera is less than the first threshold value, the control unit 101 controls the frame rate using the first frame rate. Furthermore, if the shutter speed of the main camera is greater than or equal to the first threshold value and less than the second threshold value, the control unit 101 controls the frame rate using the second frame rate. If the shutter speed of the main camera is greater than or equal to the second threshold value, the control unit 101 controls the frame rate using the third frame rate.

[0108] Although the above describes an example of providing a motion blur notification when the estimated motion blur is greater than or equal to a predetermined value according to one aspect of the present invention, the present invention can also be applied to a case where multiple motion blur displays are provided based on the amount of motion, for example. An example is provided in which three colors (red, blue, and yellow) are used for representation based on the size of the estimated motion blur. In this case, the allowable amount of motion that can detect the narrowest range of the amount of motion is set so that the range of the size of the amount of motion can be represented by these colors. For example, if the representation colors of the motion blur based on the size of the vector are yellow representing 0 to 10 pixels, blue representing 11 to 15 pixels, and red representing 16 pixels or more, the blue range is the narrowest range of the amount of motion. Specifically, five pixels (11 to 15) of the blue range are set as the allowable amount of motion.

[0109] Although the allowable motion blur amount is set and the frame rate as the shooting condition of the preliminary image is controlled so that motion less than or equal to the allowable motion blur amount can be detected according to one aspect of the present invention, the shooting condition can also be changed (motion priority mode) if the detected motion blur is less than or equal to a predetermined value.

[0110] If the subject's motion is detected to be stable, the shooting conditions for the preliminary image may be changed. Furthermore, whether the motion vector is stable is calculated as the change in the motion vector after the shooting conditions for the preliminary image are changed, and if the change in the motion vector per unit time increases, the shooting conditions for the preliminary image may be changed again.

[0111] If the permissible amount of motion is not pre-settable, the size of the motion vector within the limit range of motion detection can be set as the permissible amount of motion. In this case, it is determined whether the subject's motion exceeds the limit range of motion detection. Specifically, it is determined whether the subject's motion exceeds the motion detection range. This determination can be made based on whether the amount of change in the value of the sum of absolute differences (SAD) in motion detection is less than a predetermined value, or using a method of measuring the motion of the camera 100 such as gyroscope information. If it is determined that the permissible amount of motion is exceeded, the shutter speed and / or frame rate is controlled to increase.

[0112] While the above description uses an example of a motion blur notification method in which the display unit 109 displays a motion blur notification according to an aspect of the present invention, the motion blur notification method is not limited to the above. For example, a sound, light, or vibration may be used as a motion blur notification. Specifically, if the ratio of the number of pixels with estimated motion blur greater than or equal to a predetermined value relative to the entire screen is greater than or equal to a predetermined ratio, a motion blur notification sound, a notification light, or a notification vibration may be provided.

[0113] (Other embodiments)

[0114] The purpose of the present invention can also be achieved as described below. Specifically, a storage medium having a program code describing the software for performing the processing of the functions of the embodiment is fed to a system or device. Then, a computer (or CPU or microprocessor unit (MPU)) of the system or device reads the program code stored in the storage medium and executes the read program code.

[0115] In this case, the program code read from the storage medium realizes the novel functions of the present invention, and the storage medium storing the program code and the program are included in the present invention.

[0116] In addition, examples of storage media for feeding program code include floppy disks, hard disks, optical disks, and magneto-optical disks. In addition, compact disk ROM (CD-ROM), recordable compact disk (CD-R), rewritable compact disk (CD-RW), digital versatile disk ROM (DVD-ROM), digital versatile disk RAM (DVD-RAM), rewritable digital versatile disk (DVD-RW), recordable digital versatile disk (DVD-R), magnetic tape, nonvolatile memory card, and ROM can also be used.

[0117] In addition, the computer is configured to execute the read program code to perform the functions of the embodiment. In addition, the following situation is included: the operating system (OS) running on the computer partially or completely performs actual processing based on the instructions of the program code, and its processing performs the functions of the embodiment.

[0118] The following scenario is also included. First, the program code read from the storage medium is written to the memory of a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU on the function expansion board or function expansion unit partially or fully performs actual processing based on the instructions of the program code.

[0119] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications, equivalent structures and functions.

Claims

1. An information processing device, comprising: an acquiring component configured to acquire a plurality of first captured images captured by first imaging using first imaging parameters, and acquire motion information related to a subject in the plurality of first captured images; an estimating unit configured to estimate, for second imaging to be performed using second imaging parameters set independently of the first imaging parameters, a motion blur of the subject in a second captured image captured by the second imaging based on the motion information and the second imaging parameters; A setting component configured to set the first camera parameter and the second camera parameter; as well as a notification component configured to notify information related to the motion blur, wherein the acquiring unit acquires the motion information from the plurality of first captured images, the plurality of first captured images being captured using the first imaging parameter changed in such a manner that the changed first imaging parameter corresponds to the exposure time of the second imaging associated with the second imaging parameter; and The setting component changes a frame rate included in the first imaging parameter based on an exposure time of the second imaging related to the second imaging parameter, so as to acquire the plurality of first photographed images.

2. The information processing device according to claim 1, wherein The first camera parameter includes a shutter speed of the first camera.

3. The information processing device according to claim 1, wherein The estimating section estimates the motion blur using the motion information and the frame rate. The information processing device according to claim 1 , wherein: The setting component changes the frame rate based on the exposure time of the second imaging to extend the exposure time of the first imaging. The information processing device according to claim 1 , wherein: The acquiring component acquires the motion information from two consecutive time-series images among the plurality of first captured images acquired using the changed frame rate. The information processing device according to claim 1 , wherein: The notification section notifies information related to the motion blur by displaying the information corresponding to the motion blur on a display unit.

7. The information processing device according to claim 1, wherein The notification section notifies the information related to the motion blur by displaying at least one first captured image on which the information related to the motion blur is superimposed among the plurality of first captured images on a display unit.

8. The information processing device according to claim 1, wherein In response to the estimation section estimating a motion blur greater than a predetermined value, the notification section notifies information related to the motion blur.

9. The information processing device according to claim 1, wherein In a case where the setting of notification by the notification section is set to off, the notification section does not notify the information related to the motion blur.

10. The information processing device according to claim 1, wherein The first imaging is a preliminary imaging for capturing an image at a different time from the second imaging. The information processing device according to claim 10 , wherein: The second imaging is main imaging, and the first imaging is preliminary imaging performed before the main imaging.

12. A camera device comprising: a camera unit configured to perform a first camera operation; as well as The information processing device according to claim 1.

13. An imaging device comprising an imaging component, wherein the imaging device is configured to, in response to an imaging instruction issued by a user during the period in which the imaging component sequentially outputs a plurality of first images captured by first imaging using first imaging parameters, output a second image captured based on the imaging instruction by second imaging using second imaging parameters, the imaging device comprising: an acquisition component configured to acquire motion information based on the plurality of first captured images output from the imaging component; an estimating unit configured to estimate, for the second imaging to be performed using the second imaging parameters set independently of the first imaging parameters, a motion blur of the subject in the second captured image captured by the second imaging based on the motion information and the second imaging parameters; A setting component configured to set the first camera parameter and the second camera parameter; as well as a notification component configured to notify information related to the motion blur, wherein the acquiring unit acquires the motion information from the plurality of first captured images, the plurality of first captured images being captured using the first imaging parameter changed in such a manner that the changed first imaging parameter corresponds to the exposure time of the second imaging associated with the second imaging parameter; and The setting component changes a frame rate included in the first imaging parameter based on an exposure time of the second imaging related to the second imaging parameter, so as to acquire the plurality of first photographed images.

14. An information processing method, comprising: acquiring a plurality of first captured images captured by first imaging using first imaging parameters, and acquiring motion information related to a subject in the plurality of first captured images; For second imaging to be performed using second imaging parameters set independently of the first imaging parameters, estimating, based on the motion information and the second imaging parameters, a motion blur of the subject in a second captured image captured by the second imaging; Setting the first camera parameter and the second camera parameter; as well as Notify information related to said motion blur, wherein the motion information is obtained from the plurality of first captured images, the plurality of first captured images being captured using the first imaging parameter changed in such a manner that the changed first imaging parameter corresponds to the exposure time of the second imaging associated with the second imaging parameter; and Wherein, the frame rate included in the first imaging parameter is changed based on the exposure time of the second imaging related to the second imaging parameter, so as to obtain the plurality of first captured images.

15. A method for controlling an imaging device, the imaging device comprising an imaging component, and the imaging device being configured to, in response to an imaging instruction issued by a user during a period in which the imaging component sequentially outputs a plurality of first images captured by first imaging using first imaging parameters, output a second image captured based on the imaging instruction by second imaging using second imaging parameters, the method comprising: acquiring motion information based on the plurality of first captured images output from the imaging component; for the second imaging to be performed using the second imaging parameter set independently of the first imaging parameter, estimating, based on the motion information and the second imaging parameter, a motion blur of the subject in the second captured image captured by the second imaging; Setting the first camera parameter and the second camera parameter; as well as Notify information related to said motion blur, wherein the motion information is obtained from the plurality of first captured images, the plurality of first captured images being captured using the first imaging parameter changed in such a manner that the changed first imaging parameter corresponds to the exposure time of the second imaging associated with the second imaging parameter; and Wherein, the frame rate included in the first imaging parameter is changed based on the exposure time of the second imaging related to the second imaging parameter, so as to obtain the plurality of first captured images.

16. A non-transitory computer-readable storage medium storing a program configured to cause a computer to execute steps of a control method for an information processing device, the control method comprising: acquiring a plurality of first captured images captured by first imaging using first imaging parameters, and acquiring motion information related to a subject in the plurality of first captured images; For second imaging to be performed using second imaging parameters set independently of the first imaging parameters, estimating, based on the motion information and the second imaging parameters, a motion blur of the subject in a second captured image captured by the second imaging; Setting the first camera parameter and the second camera parameter; as well as Notify information related to said motion blur, wherein the motion information is obtained from the plurality of first captured images, the plurality of first captured images being captured using the first imaging parameter changed in such a manner that the changed first imaging parameter corresponds to the exposure time of the second imaging associated with the second imaging parameter; and Wherein, the frame rate included in the first imaging parameter is changed based on the exposure time of the second imaging related to the second imaging parameter, so as to obtain the plurality of first captured images.

17. A computer program product comprising a program configured to cause a computer to execute the steps of a method for controlling an information processing device, the method comprising: acquiring a plurality of first captured images captured by first imaging using first imaging parameters, and acquiring motion information related to a subject in the plurality of first captured images; For second imaging to be performed using second imaging parameters set independently of the first imaging parameters, estimating, based on the motion information and the second imaging parameters, a motion blur of the subject in a second captured image captured by the second imaging; Setting the first camera parameter and the second camera parameter; as well as Notify information related to said motion blur, wherein the motion information is obtained from the plurality of first captured images, the plurality of first captured images being captured using the first imaging parameter changed in such a manner that the changed first imaging parameter corresponds to the exposure time of the second imaging associated with the second imaging parameter; and Wherein, the frame rate included in the first imaging parameter is changed based on the exposure time of the second imaging related to the second imaging parameter, so as to obtain the plurality of first captured images.

Images