Information processing apparatus and method, image pickup apparatus, control method thereof, and storage

Abstract

The invention relates to an information processing apparatus and method, an imaging apparatus, a control method thereof, and a storage medium. The information processing apparatus includes: an acquisition section configured to acquire a plurality of first captured images captured by first imaging using a first imaging parameter, and acquire motion information related to an object in the plurality of first captured images; an estimation means for estimating, for a second imaging to be performed using a second imaging parameter, motion blur of a subject in a second captured image captured by the second imaging on the basis of the second imaging parameter; and the setting component is used for setting the first shooting parameter and the second shooting parameter. The acquisition section acquires motion information from a plurality of first captured images captured using a first imaging parameter changed in such a manner that the changed first imaging parameter corresponds to an exposure time of a second imaging related to a second imaging parameter.

Description

[0001] (This application is a divisional application of the application filed on June 1, 2022, with application number 2022106194768, entitled "Information Processing Apparatus and Method, Camera Apparatus and Control Method Thereof and Storage Medium".) Technical Field

[0002] This invention relates to a technique for notifying that a subject is blurred in a captured image. Background Technology

[0003] Video recording devices, such as digital still cameras, exist that feature a shutter speed priority mode. In this mode, the user sets the desired shutter speed, and the recording device automatically sets exposure settings other than the shutter speed, such as aperture and ISO sensitivity. Using this mode, for example, a user can set a high shutter speed before recording to capture an image with less motion blur.

[0004] Japanese Patent Application Publication No. 2008-172667 describes a technique for detecting and highlighting motion regions from time-series images captured during pre-capture photography. Pre-capture photography here refers to photography performed before the main capture, to set composition and shooting conditions while viewing the electronic viewfinder or rear LCD display of the camera equipment. Furthermore, the main capture refers to photography triggered by the user pressing the shutter button and performed by the camera equipment based on the composition and shooting conditions set during pre-capture photography.

[0005] However, Japanese Patent Application Publication No. 2008-172667 does not mention the following: recording video with an appropriate frame rate and shutter speed for the speed and amount of movement of the subject in order to extract motion regions from time-series images. For example, shooting portrait images at night results in a longer exposure time due to less light collected by the camera compared to shooting portrait images during the day. Therefore, even slight shaking or movement of the subject can cause blurring. However, an appropriate frame rate value is not always preset to detect slight movement of the subject, which raises the problem of not providing the user with appropriate notification of the occurrence of subject blur. Summary of the Invention

[0006] According to one aspect of the present invention, an information processing apparatus includes: an acquisition unit configured to acquire a plurality of first captured images taken by a first camera using first camera parameters, and to acquire motion information relating to a subject in the plurality of first captured images; an estimation unit configured to estimate motion blur of the subject in a second captured image taken by the second camera using second camera parameters set independently of the first camera parameters, based on the motion information and the second camera parameters; a setting unit configured to set the first camera parameters and the second camera parameters; and a notification unit configured to notify information relating to the motion blur. The acquisition unit acquires the motion information from the plurality of first captured images, the plurality of first captured images being captured using first camera parameters changed in such a manner that the changed first camera parameters correspond to the exposure time of the second camera associated with the second camera parameters.

[0007] According to another aspect of the present invention, an information processing apparatus includes: an acquisition unit configured to acquire a plurality of first captured images taken by a first camera using first camera parameters, and to acquire motion information relating to a subject in the plurality of first captured images; an estimation unit configured to estimate motion blur of the subject in a second captured image taken by the second camera using second camera parameters set independently of the first camera parameters, based on the motion information and the second camera parameters; a setting unit configured to set the first camera parameters and the second camera parameters; and a notification unit configured to notify information relating to the motion blur. The acquisition unit selects an image from the plurality of first captured images for use in acquiring the motion information based on the exposure time of the second camera image associated with the second camera parameters.

[0008] According to another aspect of the present invention, a camera device includes a camera component, and the camera device is configured to output a second captured image based on the camera instruction issued by a user during the sequential output of a plurality of first captured images captured by a first camera using a first camera parameter, in response to the camera component sequentially outputting a camera instruction, a second captured image captured by a second camera using a second camera parameter. The camera device further includes: an acquisition component configured to acquire motion information based on the plurality of first captured images output from the camera component; an estimation component configured to estimate motion blur of a subject in the second captured image captured by the second camera using the motion information and the second camera parameter for a second camera to be performed using second camera parameters set independently of the first camera parameter; a setting component configured to set the first camera parameter and the second camera parameter; and a notification component configured to notify information related to the motion blur. The acquisition component acquires the motion information from the plurality of first captured images, which are captured using first camera parameters that are changed in such a way that the changed first camera parameters correspond to the exposure time of the second camera associated with the second camera parameters.

[0009] According to another aspect of the invention, a camera device includes a camera component, and the camera device is configured to output a second captured image based on the camera instruction issued by a user during the sequential output of a plurality of first captured images captured by a first camera using a first camera parameter, in response to the camera component sequentially outputting a plurality of first captured images captured by a second camera using a second camera parameter. The camera device further includes: an acquisition component configured to acquire motion information based on the plurality of first captured images output from the camera component; an estimation component configured to estimate motion blur of a subject in the second captured image captured by the second camera using second camera parameters set independently of the first camera parameters, based on the motion information and the second camera parameters; a setting component configured to set the first camera parameters and the second camera parameters; and a notification component configured to notify information related to the motion blur. The acquisition component selects an image from the plurality of first captured images for use when acquiring the motion information based on the exposure time of the second camera captured image associated with the second camera parameters.

[0010] According to another aspect of the present invention, an information processing method includes: acquiring a plurality of first captured images taken by a first camera using first camera parameters, and acquiring motion information related to a subject in the plurality of first captured images; estimating motion blur of the subject in a second captured image taken by the second camera based on the motion information and the second camera parameters for a second captured image to be taken using second camera parameters set independently of the first camera parameters; setting the first camera parameters and the second camera parameters; and notifying information related to the motion blur. The motion information is acquired from the plurality of first captured images, the plurality of first captured images being captured using first camera parameters that are changed in a manner corresponding to the exposure time of the second captured image associated with the second camera parameters.

[0011] According to another aspect of the present invention, an information processing method includes: acquiring a plurality of first captured images taken by a first camera using first camera parameters, and acquiring motion information relating to a subject in the plurality of first captured images; estimating motion blur of the subject in a second captured image taken by the second camera using the motion information and the second camera parameters for a second camera shot to be taken using second camera parameters set independently of the first camera parameters; setting the first camera parameters and the second camera parameters; and notifying information relating to the motion blur. An image is selected from the plurality of first captured images for use when acquiring the motion information based on the exposure time of the second camera shot associated with the second camera parameters.

[0012] According to another aspect of the present invention, a control method for a camera device includes a camera component, and the camera device is configured to output a second captured image based on the camera instruction during the sequential output of a plurality of first captured images captured by a first camera using a first camera parameter, in response to a camera instruction issued by a user during the sequential output of a plurality of first captured images by a first camera using a first camera parameter. The control method includes: acquiring motion information based on the plurality of first captured images output from the camera component; estimating motion blur of a subject in the second captured image captured by the second camera using the motion information and the second camera parameter for a second camera shot to be performed using second camera parameters set independently of the first camera parameter; setting the first camera parameter and the second camera parameter; and notifying information related to the motion blur. The motion information is acquired from the plurality of first captured images, which are captured using first camera parameters that are changed in a manner corresponding to the exposure time of the second camera shot associated with the second camera parameter.

[0013] According to another aspect of the present invention, a control method for a camera device, the camera device including a camera component, and the camera device configured to output a second captured image based on the camera instruction issued by a user during the sequential output of a plurality of first captured images captured by a first camera using a first camera parameter, in response to the camera component sequentially outputting a plurality of first captured images captured by a second camera using a second camera parameter, the control method comprising: acquiring motion information based on the plurality of first captured images output from the camera component; estimating motion blur of a subject in the second captured image captured by the second camera using the motion information and the second camera parameter for a second camera to be performed using second camera parameters set independently of the first camera parameter; setting the first camera parameter and the second camera parameter; and notifying information related to the motion blur. An image is selected from the plurality of first captured images for use when acquiring the motion information based on the exposure time of the second camera associated with the second camera parameter.

[0014] Further features of the invention will become apparent from the following description of typical embodiments with reference to the accompanying drawings. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating an example of the structure of an implementation of the present invention.

[0016] Figure 2 This is a flowchart illustrating the image processing of the camera device 100.

[0017] Figure 3 This is a diagram illustrating an example of the structure of the motion blur notification image generation unit according to the first embodiment.

[0018] Figure 4 This illustrates the first embodiment. Figure 2 The diagram shows the processing of step S205 in the process.

[0019] Figure 5A and Figure 5B It is a graph showing the relationship between the amount of motion of the subject and the permissible amount of motion.

[0020] Figure 6A and Figure 6B It is a diagram showing the preliminary image and motion vectors.

[0021] Figure 7 This is a flowchart illustrating the motion vector calculation process.

[0022] Figure 8 This is a diagram illustrating the motion vector calculation process.

[0023] Figures 9A to 9C This is a diagram illustrating motion blur notification processing.

[0024] Figure 10A and Figure 10B This is a timing diagram showing the timing of image acquisition in the pre-camera and main camera modes.

[0025] Figure 11 It is a diagram showing the exposure time in the main camera and the frame time interval in the pre-camera.

[0026] Figure 12 It is a graph showing the integral of motion in a frame.

[0027] Figure 13A and Figure 13B This is a diagram illustrating frame rate control.

[0028] Figure 14 This illustrates the second embodiment. Figure 2 The diagram shows the processing of step S205 in the process. Detailed Implementation

[0029] Some embodiments of the present invention will now be described with reference to the accompanying drawings. The various embodiments of the present invention described below can be implemented individually, or, where necessary or where it is advantageous to combine elements or features from the various embodiments into one embodiment, as a combination of multiple embodiments or features of these embodiments.

[0030] According to a first embodiment of the present invention, motion blur in the main camera is estimated using motion information calculated based on images during the pre-shooting period, and a motion blur notification image representing the motion blur estimated to occur in the main camera is displayed to the user as a notification.

[0031] Figure 1 This is a block diagram illustrating an example of a structure according to the first embodiment, and also illustrating an example of a camera device. Reference will be made below. Figure 1 An example illustrating the structure according to the first embodiment of the present invention will be provided.

[0032] The control unit 101 is, for example, a central processing unit (CPU). The control unit 101 reads control programs for each block of the imaging device 100 (e.g., a camera) from the read-only memory (ROM) 102 (described below), loads the read control programs into the random access memory (RAM) 103 (described below), and runs the loaded control programs, thereby controlling the operation of each block of the imaging device 100. The ROM 102 is a non-volatile electrically erasable and recordable memory. The ROM 102 stores the operating programs for each block of the imaging device 100 and the parameters used for the operation of each block. The RAM 103 is a volatile rewritable memory. The RAM 103 is used to load the programs run by the control unit 101 and temporarily store data generated during the operation of each block of the imaging device 100.

[0033] The optical system 104 includes a lens group with a zoom lens and a focusing lens, and forms an image of the subject on the imaging surface of the imaging unit 105. The imaging unit 105 is an image sensor such as a charge-coupled device (CCD) sensor or a complementary metal-oxide-semiconductor (CMOS) sensor. The imaging unit 105 photoelectrically converts the optical image formed by the optical system 104 on the imaging surface of the imaging unit 105 into an analog image signal, and outputs the analog image signal to the analog-to-digital (A / D) conversion unit 106. The A / D conversion unit 106 converts the input analog image signal into digital image data. The digital image data output from the A / D conversion unit 106 is temporarily stored in RAM 103.

[0034] Image processing unit 107 performs various types of image processing on the image data stored in RAM 103, such as white balance adjustment, color interpolation, and gamma processing. Furthermore, image processing unit 107 includes a motion blur notification image generation unit 300, as described below, and generates a motion blur notification image by superimposing an image plane onto the image stored in RAM 103 to facilitate motion blur detection. Recording unit 108 is a recording medium such as a removable memory card. Recording unit 108 records the image data processed by image processing unit 107 as a recorded image via RAM 103.

[0035] Display unit 109 is a display device such as a liquid crystal display (LCD). Display unit 109 displays images stored in RAM 103, images recorded in recording unit 108, and an operation user interface for receiving user instructions. Furthermore, display unit 109 displays images captured by camera unit 105 for adjusting composition during pre-capture recording. Instruction input unit 110 is a touch panel and a mouse, and is used to input user instructions.

[0036] Next, the following will refer to Figure 2 The flowchart below will be used to explain the first embodiment in detail. The processes described below are executed by the control unit 101 controlling each unit based on the program stored in the ROM 102. Figure 2 The flowchart is implemented in response to the start of pre-capture.

[0037] In step S201, the user turns on the camera device 100. Then, in step S201, when the camera device 100 is turned on, the control unit 101 controls the optical system 104 and the camera unit 105 and begins pre-capture preparation. During pre-capture preparation, the control unit 101 maintains a predetermined frame rate, and the camera device 100 sequentially captures images. The acquired images are displayed on the display unit 109. The user can adjust the composition while reviewing the sequentially displayed pre-capture images.

[0038] In step S202, the imaging device 100 is set to imaging conditions (imaging parameters) for capturing a preliminary image to detect the amount of motion of the subject in the composition. The imaging conditions here primarily refer to the frame rate and shutter speed. The highest frame rate and high shutter speed are set within a range that does not affect the processing used to calculate evaluation values ​​for use in general automatic function control (such as automatic exposure (AE) and automatic focus (AF) control) performed by the imaging device. Furthermore, the optical system 104 is controlled to appropriately adjust the shutter speed by changing the lens aperture or the ISO sensitivity setting of the imaging unit 105, so that images are captured under appropriate exposure conditions even at the set high shutter speed. The imaging device 100 performs imaging under the imaging conditions set in step S202 during the preliminary imaging period. At higher shutter speeds, less blur accumulates in the captured image of a moving subject, and the amount of subject movement between consecutive captured images is reduced. This enables detailed subject motion detection.

[0039] In step S203, the user sets the main camera's shooting parameters using the instruction input unit 110. The control unit 101 can set the main camera's shooting parameters independently of the pre-shooting shooting parameters based on the input from the instruction input unit 110. For example, the exposure time can be set as the main camera's shooting parameter. The main camera's shooting parameters can be automatically set by the camera device 100. According to this embodiment, the set main camera's shooting parameters are used as the main camera's shooting parameters after detecting the pressing of the shutter button (for indicating main shooting) as described below.

[0040] In step S204, the control unit 101 determines whether the motion blur notification setting is set to "ON" or "OFF". The motion blur notification setting is set to "ON" or "OFF" by the user using the instruction input unit 110, and the setting is maintained once it is set. If the control unit 101 determines in step S204 that the motion blur notification is set to "ON" (yes in step S204), then in step S205, the motion blur notification image generation unit 300, as described below, generates a motion blur notification image by superimposing a motion blur notification plane onto a prepared image. Then, in step S206, the control unit 101 displays the generated motion blur notification image on the display unit 109.

[0041] On the other hand, if the control unit 101 determines in step S204 that the motion blur notification is set to "off" ("no" in step S204), then no motion blur notification image is generated. Therefore, in step S206, a preliminary image without motion blur notification is displayed on the display unit 109.

[0042] In step S207, the control unit 101 determines whether the shutter button of the instruction input unit 110 has been pressed by the user. For two-level inputs to be received (e.g., half-pressing the shutter button to input a camera preparation operation instruction and fully pressing the shutter button to issue a main camera instruction), the control unit 101 determines whether the shutter button has been fully pressed. For a single input to be received, the control unit 101 determines whether an input has been made.

[0043] If the control unit 101 determines that the shutter button has not been pressed ("No" in step S207), the process returns to step S202, and steps S202 to S206 are repeated. Therefore, during the pre-capture period, the user can easily check for potential motion blur of the subject in the main camera shot using the currently set capture parameters. If the checked motion blur is not the desired motion blur, the user does not press the shutter button to return to step S202, and in step S203, the user can reset the main camera shutter speed (exposure time). As described above, during the pre-capture period, the user repeatedly performs the process of setting the main camera exposure time while checking the motion blur notification image displayed on the display unit 109 until the desired motion blur is obtained, and when the opportunity to capture the image arises, the user presses the shutter button.

[0044] In step S207, if the control unit 101 determines that the shutter button has been pressed ("Yes" in step S207), the control unit 101 determines that it has received a main camera instruction and proceeds to step S208. In step S208, the control unit 101 controls the optical system 104 and the camera unit 105, and performs main camera recording using the camera parameters set by the pre-camera setup. The image captured by the main camera is output by the control unit 101 to the display unit 109 and the recording unit 108, and the output image is displayed on the display device of the display unit 109 and recorded in the recording medium of the recording unit 108 or output to an external device.

[0045] Next, the following will refer to Figure 3 Here is an example illustrating the structure of the motion blur notification image generation unit 300 of the image processing unit 107, which is a feature of the present invention.

[0046] Figure 3This is a diagram illustrating an example of the structure of the motion blur notification image generation unit 300. The motion blur notification image generation unit 300 includes a motion vector calculation unit 301 and a motion blur estimation calculation unit 302. The motion vector calculation unit 301 calculates the motion vector of the subject by comparing images. The motion blur estimation calculation unit 302 estimates the motion blur of the subject in the main camera based on the calculated motion vector. The motion blur notification image generation unit 300 also includes a notification plane generation unit 303 and an image overlay unit 304. The notification plane generation unit 303 generates data for motion blur notification based on the estimated motion blur of the subject. According to this embodiment, the motion blur notification plane is overlaid on the captured image, and the image overlay unit 304 performs the overlay processing. Details of the operation of these units will be described below.

[0047] Figure 3 One or more of the functional blocks shown can be implemented by hardware such as an application-specific integrated circuit (ASIC) or a programmable logic array (PLA), by a programmable processor such as a CPU or a microprocessor unit (MPU) running software, or by a combination of software and hardware. Therefore, the different functional blocks described below as the executor can be implemented by the same hardware as the executor.

[0048] Next, the following will refer to Figure 4 The flowchart shown illustrates the process performed by the motion blur notification image generation unit 300. Figure 2 The processing of step S205 in the flowchart. The steps shown in this flowchart are performed by the control unit 101 or by each unit of the camera device 100, including the motion blur notification image generation unit 300, based on instructions from the control unit 101.

[0049] In step S401, the control unit 101 sets the imaging conditions set in step S202. Then, the control unit 101 controls the optical system 104 and the imaging unit 105, and the imaging device 100 captures a series of preliminary images for detecting motion blur of the subject in the main camera.

[0050] In step S402, the control unit 101 acquires shutter speed information for main video recording, which is set by the user or automatically by the camera.

[0051] In step S403, the control unit 101 selects a preliminary image frame for use in subject motion detection (step S404 described below) based on the frame rate information of the preliminary camera and the shutter speed of the main camera. Reference will be made below. Figure 10A , Figure 10B and Figure 11 This section details the method for selecting the preparatory image frame.

[0052] Figure 10A and Figure 10B This is a timing diagram showing the timing of acquiring a frame (1001) in the pre-camera and the timing of acquiring an image (1011) in the main camera, where the horizontal axis represents time T.

[0053] The horizontal axis represents the recording time. Figure 10A The control 1021 is shown in the diagram, and... Figure 10B Frame selection 1022 is shown. The amount of motion of the subject at a given instant is calculated using two frames connected by arrows, representing selected preliminary image frames used in calculating the amount of motion of the subject. Control 1021 represents control for selecting two consecutive frames during motion detection, and frame selection 1022 represents an example of selecting preliminary image frames based on frame rate information used in the preliminary shooting according to this embodiment and the shutter speed of the main camera. Image capture frame selection 1022 represents a method typically performed when calculating the amount of motion of the subject, and according to this embodiment, an image capture frame is selected as shown by frame selection 1022.

[0054] The following will be referenced Figure 11 This embodiment will explain the method for selecting a preparatory image frame.

[0055] Figure 11This is a graph showing the relationship between the frame time intervals of multiple preparatory images (1111 to 1114) and the exposure time of the image (1101) captured by the main camera at the main camera's shutter speed. Specific numerical values ​​will be used in the following description. If the main camera's shutter speed is set to 1 / 30, an image is captured in the main camera with an exposure time of 1 / 30 of a second. Specifically, in the case of slight subject movement during 1 / 30 of a second, an image with less accumulated blur is obtained. On the other hand, since the preparatory images are used for motion analysis, the preparatory images are captured at a shutter speed that makes it unlikely for accumulated blur to occur between frames. For example, multiple frames are captured with an exposure time shorter than 1 / 120 of a second. The frame rate of the preparatory images, which are captured continuously, is set to 120fps (continuous capture of 120 images per second). In order to detect subject movement during the main camera's exposure time, the control unit 101 selects frames of the preparatory images such that the frame time interval of the preparatory images is close to the main camera's exposure time. Since adjacent frames of the preparation images 1111 to 1114 have equal time intervals, the four-frame time interval (from 1111 to 1114) is 1 / 30 of a second, which is the same as the exposure time of the main camera. This means that frames that include the subject movement during the time corresponding to the exposure time of the main camera (i.e., 1 / 30 of a second) have been selected. The smaller the difference between the frame time interval of the selected preparation image frame and the shutter speed of the main camera, the smaller the error during step S405 described below, thus providing an accurate estimate. Therefore, the control unit 101 selects frames of the preparation images such that the frame time interval of the preparation images is as close as possible to the shutter speed set for the main camera. Therefore, the control unit 101 selects a preparation image with a frame time interval that satisfies the following equation (2).

[0056] F_Period (frame duration) ≥ (main camera shutter speed × pre-camera frame rate) × α Equation (2)

[0057] In equation (2), F_Period is the smallest integer that satisfies the equation, and α is a constant less than or equal to 1.

[0058] Control unit 101 selects a preliminary image frame that satisfies equation (2) for calculating the amount of motion of the subject, such that the selected frame is close to the time period corresponding to the shutter speed set for the main camera.

[0059] The above is for reference only. Figure 11 This explains the relationship between the frame duration of the preparatory image and the shutter speed of the main camera. According to this embodiment, the frame duration of the preparatory image is controlled by the control unit 101 to select the frame speed as described above. Figure 10BThe frame selected is 1022. Furthermore, for higher ISO sensitivity, the constant α in equation (2) is ideally set to a smaller number. Using a constant α prevents excessively long frame intervals for demanding shooting conditions. For example, in cases where the main camera shutter speed is long and the frame interval for preparing the image is too long, complex subject motion may be included between frames, potentially leading to a decrease in the accuracy of subject motion detection. Using different constants α for different main camera shutter speeds prevents a decrease in the accuracy of subject motion detection.

[0060] exist Figure 4 In step S404, the preparatory image frame selected in step S403 is input to the motion blur notification image generation unit 300, and the motion vector calculation unit 301 uses the input image to calculate the motion of the subject. The following will refer to... Figure 6A To illustrate the calculation of the motion of the subject using the motion vector calculation unit 301, wherein... Figure 6A The image shows a scene including a dog 601 running to the left and a dog 602 standing still.

[0061] The motion vector calculation unit 301 calculates the motion vectors between consecutive pre-recorded images of the time series as the motion of the subject. The motion vector represents the horizontal and vertical movement of the subject between pre-recorded images. The following will refer to... Figure 7 and Figure 8 This section will explain in detail the method for calculating motion vectors.

[0062] Figure 7 This is a flowchart illustrating the motion vector calculation process. Although the block matching method will be described below as an example of a motion vector calculation method according to aspects of the present invention, the motion vector calculation method is not limited to this example and may, for example, be a gradient method. The steps shown in this flowchart are performed by the control unit 101, or by the various units of the camera device 100, including the motion blur notification image generation unit 300, based on instructions from the control unit 101.

[0063] In step S701, the control unit 101 receives two temporally adjacent preparatory images acquired during the preparatory imaging period, sets the preparatory image of the Mth frame as the standard frame, and sets the preparatory image of the (M+1)th frame as the reference frame.

[0064] In step S702, as Figure 8 As shown, the control unit 101 places a standard block 802 of N×N pixels in the standard frame 801.

[0065] In step S703, as Figure 8As shown, the control unit 101 sets the (N+n)×(N+n) pixels around the coordinate 804 corresponding to the coordinate of the center of the standard block 802 in the standard frame 801 as the search range 805 in the reference frame 803.

[0066] In step S704, the control unit 101 performs correlation operations on the standard block 802 in the standard frame 801 and the reference block 806 at different coordinates within the search range 805 in the reference frame 803, and calculates the correlation value. The correlation value is calculated based on the sum of absolute differences (SAD) between the pixels in the standard block 802 and the corresponding pixels in the reference block 806 within frames 801 and 803. Specifically, the coordinate with the smallest sum of absolute differences between frames 801 and 803 is the coordinate with the highest correlation value. The correlation value calculation method is not limited to calculating the SAD between frames, and can, for example, be based on the sum of squared differences or normalized cross-correlation values ​​between frames. Figure 8 In the example, reference block 806 is represented as having the highest relevance.

[0067] In step S705, the motion vector calculation unit 301 calculates the motion vector based on the coordinates with the highest correlation value calculated in step S704 in the reference block 806. Figure 8 In the example, the motion vector is calculated based on coordinate 804, which corresponds to the coordinates of the center of standard block 802 in standard frame 801 within the search range 805 in reference frame 803, and the coordinates of the center of reference block 806. Specifically, the distance and direction from coordinate 804 to the coordinates of the center of reference block 806 are calculated as the motion vector.

[0068] In step S706, the motion vector calculation unit 301 determines whether motion vectors have been calculated for all pixels of the standard frame 801. If, in step S706, the motion vector calculation unit 301 determines that motion vectors have not been calculated for all pixels ("No" in step S706), the process returns to step S702. Then, in step S702, an N×N pixel standard block 802 is placed in the standard frame 801 centered at the pixel where no motion vector has been calculated, and steps S703 to S705 are performed as described above. Specifically, the motion vector calculation unit 301 calculates motion vectors by moving... Figure 8 While processing standard block 802, steps S702 to S705 are repeated to calculate the motion vectors of all pixels in standard frame 801. Figure 6B An example of a motion vector is shown in the figure. Figure 6B It is shown Figure 6A An example diagram of motion vectors in the preparatory image.

[0069] Figure 6AThe preparatory image is an example of dog 601 running to the left. Figure 6B This shows a representative example of the motion vector of a moving subject. Figure 6B In the example, the running dog 601 is detected as a leftward motion vector, while the rest (such as the stationary dog ​​602 and the background fence) are detected as motion vector 0 (not shown).

[0070] Furthermore, the motion vector calculation unit 301 can calculate the motion vector of each predetermined pixel, rather than calculating the motion vector of all pixels. Through the above processing, the motion vectors between frames of temporally adjacent pre-images are calculated.

[0071] exist Figure 4 In step S405, the motion blur estimation calculation unit 302 calculates the relationship between the calculated motion amount of the subject and the target motion blur amount. If the time between the selected pre-image frames in step S403 does not correspond to the shutter speed (exposure time) of the main camera, the subject blur in the main camera cannot be accurately calculated. Therefore, the motion amount is converted by multiplying the motion amount of the subject detected in step S404 by the correction gain obtained using equation (4), and the motion blur in the main camera is estimated.

[0072] Correction gain = shutter speed of the main camera / time between selected preparatory image frames (4)

[0073] Then, the control unit 101 determines whether the motion blur in the main camera (i.e., the estimated motion blur) estimated by multiplying the correction gain by equation (4) is greater than or less than the target motion blur amount. The target motion blur amount here refers to the permissible motion blur amount where motion blur is less visible in a shot taken at a predetermined shutter speed. The permissible motion blur amount is determined based on the size of the image sensor, such as a CCD sensor or a CMOS sensor, the number of pixels, and the resolution of the display device used for display. For example, under the conditions of using an Advanced Photo System Type C (APS-C) image sensor as the image sensor, 200,000 pixels as the number of pixels, a Full HD (1920×1080 pixels) personal computer (PC) display as the display, and five pixels or less as the permissible motion blur amount, motion of five pixels or less between frames will be detected. Specifically, sufficient resolution is provided for detecting motion less than or equal to the permissible motion blur amount. Reference will be made below. Figure 5A and 5B Let's illustrate with a specific example.

[0074] Figure 5A and Figure 5B The diagram shows a motion vector representing the relationship between the amount of motion of the subject calculated from the pre-existing image and the allowable amount of motion blur. Figure 5AThis shows a case where the amount of motion of the subject is detected to be greater than the allowable amount of motion blur. Figure 5B This shows the case where the motion of the subject is less than or equal to the permissible motion. The relationship between the calculated motion of the subject and the target motion blur is expressed by equation (1).

[0075] n = (Amount of motion of the subject) / Permissible amount of motion (Equation 1)

[0076] If n is greater than 1, the motion of the subject exceeds the permissible motion limit, causing the control unit 101 to determine that subject blurring will occur. On the other hand, if n is less than or equal to 1, the control unit 101 determines that the motion of the subject does not exceed the permissible motion limit. Specifically, the control unit 101 determines that subject blurring will be within the permissible range. The target permissible motion limit can be determined by settings configured by the user.

[0077] exist Figure 4 In step S406, the motion blur notification image generation unit 300 generates a motion blur notification image based on the instruction from the control unit 101 and the relationship between the amount of motion of the subject and the amount of target motion blur calculated in step S405. Figures 9A to 9C This illustrates motion blur notification during the process of notifying the user of the motion blur level when the main camera is in operation. Therefore, the notification plane generation unit 303 converts the motion calculated from the preliminary image into a motion calculated using the exposure conditions of the main camera (converted motion blur), and generates a blurred notification image that allows for visual inspection of the converted motion.

[0078] First, the notification plane generation unit 303 uses the result of equation (1) calculated in step S405 to generate an image plane for motion blur notification for each motion amount of the subject. Then, the image overlay unit 304 overlays the motion blur notification plane onto the prepared image and generates a motion blur notification image. The following will refer to... Figures 9A to 9C Here's an example to illustrate a blurred notification image.

[0079] Figures 9A to 9C Three examples of motion blur notification images are shown. The display unit 109 displays motion blur notification images during pre-capture, allowing the user to easily check for motion blur.

[0080] first, Figure 9A An example of motion blur notification displayed using an icon is shown. In step S405, the control unit 101 calculates the ratio of the motion blur amount of the subject, which is greater than 1 (n) as a result of equation (1), to the overall motion blur amount of the subject. If this ratio is greater than or equal to a predetermined value, a motion blur notification is generated. Figure 9AThe motion blur icon 901 shown serves as a motion blur notification plane, and is generated by drawing the motion blur icon 901 on the prepared image. Figure 9A The motion-blurred notification image shown.

[0081] Figure 9B An example of a motion-blurred notification displayed using a motion-blurred bounding box is shown. A method for generating a motion-blurred notification image using a motion-blurred bounding box will be explained. In step S405, the control unit 101 divides the image into regions and calculates the motion blur amount of the subject in each segmented region, which is greater than 1 n, as a result of equation (1). Specifically, the control unit 101 generates a motion-blurred notification image for each segmented region with a proportion greater than or equal to a predetermined value. Figure 9B The motion blur box 902 shown serves as the motion blur notification plane, and is generated by drawing this motion blur notification plane on the prepared image. Figure 9B The motion-blurred notification image shown.

[0082] Figure 9C An example of a motion blur notification is shown where the edges of the motion blur are highlighted. A method for generating a motion blur notification image by highlighting the motion blur edges will be explained. In step S405, the notification plane generation unit 303 detects the edge intensity in the prepared image. Existing methods such as the Sobel filter are used when calculating the edge intensity, and their detailed description will be omitted. Then, for each region where the edge intensity value is greater than or equal to a predetermined value and n in equation (1) exceeds 1, such as... Figure 9C The motion blur notification plane that highlights the motion blur edge 903 is shown, and the notification plane generation unit 303 generates the notification plane by superimposing the motion blur notification plane onto the preparation image. Figure 9C The motion blur notification image shown. Figure 9C In the example shown, the motion blur edge 903 is indicated by bolding. The motion blur edge is highlighted to enable visual recognition of motion blur in small parts of the subject in the main camera. Another example highlighted is: extracting pixels whose edge intensity is greater than or equal to a predetermined value and whose estimated motion blur is greater than or equal to a predetermined value, and modifying the extracted pixels in terms of hue, saturation, and / or brightness, for example, displaying the extracted pixels in red.

[0083] The above explains how the motion blur notification image generation unit 300 generates the image by performing... Figure 4 The process of generating motion-blurred notification images is carried out in steps S401 to S406. Figure 2The processing in the flowchart is performed by the control unit 101 or by each unit of the camera device 100, including the motion blur notification image generation unit 300, based on instructions from the control unit 101, which makes it appropriate to determine the camera conditions for checking for motion blur that may occur in the main camera.

[0084] According to one aspect of the invention, a suitable preparatory image is selected for checking for motion blur that may occur in the main camera, and motion is detected. This makes it possible to check for motion blur that may occur in the main camera.

[0085] According to one aspect of the invention, the above-described control is performed to select frames corresponding to the frame time interval of the preparation images and detect motion, thereby estimating the amount of subject blur that may occur during the exposure time of the main camera. For more detailed motion detection, all preparation images included in the frame time interval of the preparation images corresponding to the exposure time of the main camera can be used to detect the amount of subject motion. For example, in Figure 11 and Figure 12 In the middle, if the exposure time of the main camera is as follows: Figure 11 As shown (1 / 30 second) and the frame rate for preparing the camera is as follows: Figure 11 The image is shown at 120fps, so the corresponding time period for the preparatory image is four frames (1 / 30 second). Therefore, according to the first embodiment, the motion vector calculation unit 301 uses... Figure 11 The preliminary images 1111 and 1114 are used to calculate the motion of the subject. Figure 12 The motion amount 1201 in the image). The control unit 101 calculates the motion amount of the subject between preparatory images 1111 and 1112, between preparatory images 1112 and 1113, and between preparatory images 1113 and 1114. Figure 12 The motion (1202) in the image is integrated. This allows for accurate calculation of the motion of the subject during the 1 / 30th of a second time interval from the preparatory image 1111 to the preparatory image 1114, even if the subject moves in a complex manner (rotational motion, acceleration) between preparatory images.

[0086] Furthermore, if the user capturing the image intentionally fails to determine the permissible amount of motion, the camera device 100 can set the magnitude of the motion vector within the motion detection limit range as the permissible amount of motion. In this case, the control unit 101 determines whether the motion of the subject exceeds the motion detection range. This determination can be made based on whether the change in the sum of absolute differences (i.e., SAD) in motion detection is less than a predetermined value or using a method that measures the motion of the camera device 100, such as gyroscope information. If the control unit 101 determines that the permissible amount of motion has been exceeded, it controls the shutter speed and / or frame rate to increase.

[0087] Although the above description of a motion blur notification method for displaying motion blur notifications using a display unit 109 according to one aspect of the present invention is given as an example, the motion blur notification method is not limited to those described above. For example, sound, light, or vibration can be used as motion blur notification. Specifically, if the estimated number of pixels with motion blur greater than or equal to a predetermined value is higher than or equal to a predetermined ratio relative to the entire image, a motion blur notification sound, notification light, or notification vibration is provided. According to this embodiment, the structure of the notification plane generation unit 303 and the image overlay unit 304, as well as the processing of steps S405 and S406, are not used. Instead, the camera device 100 includes a speaker, and in step S205, together with the display of a preliminary image on the display unit 109, the control unit 101 causes the speaker to provide a notification sound, illuminate a notification light, or provide a notification vibration.

[0088] According to one aspect of the invention, although the control unit 101 selects different frames of the preparation image for motion detection for different shutter speeds of the main camera, the selected frame of the preparation image for motion detection can be changed if the shutter speed of the main camera is stable. Controlling the frame of the preparation image for motion detection when the shutter speed of the main camera is stable prevents unwanted motion blur notification images from being displayed on the display unit 109 when the user is still confused about shutter speed control or when the evaluation value calculation of the control unit 101 is unstable.

[0089] An example of a method for determining whether the subject's motion is stable is as follows. Specifically, after the pre-capture shooting conditions are changed, the stability of the subject's motion is calculated as the change in the motion vector. If the change in the motion vector per unit time is large, the control unit 101 determines that the subject's motion is unstable and can change the pre-capture shooting conditions again.

[0090] According to one aspect of the invention, although the control unit 101 selects different frames of the preparatory image for motion detection for different shutter speeds of the main camera, in addition to selecting frames from the preparatory image, the shutter speed of the preparatory image can also be changed. For example, if the shutter speed of the main camera is longer than a predetermined value and the ISO sensitivity of the main camera is higher than a predetermined value, the shutter speed of the preparatory image is changed to a longer shutter speed. On the other hand, if the ISO sensitivity of the main camera is low, the shutter speed of the preparatory image is changed to a shorter shutter speed. Therefore, the shooting conditions of the preparatory image for motion detection are balanced with the effects of noise and cumulative blur, thereby enabling the capture of an appropriate preparatory image for motion detection.

[0091] The second embodiment will be described. An example will be described whereby, according to this embodiment, the frame rate of a pre-shot image is changed based on the shutter speed of the main camera to estimate the amount of subject blur in the main camera image, thereby generating a motion blur notification image. According to this embodiment, the structure provides efficient control over the frame rate of the pre-shot image for notification of subject blur in the main camera image.

[0092] According to this embodiment, the method according to the first embodiment is used. Figure 1 The structure of the camera device 100 will be described, and redundant descriptions of the block diagram of the camera device 100 will be omitted. Furthermore, the entire process performed by the control unit 101 according to the second embodiment is... Figure 2 The process shown in the flowchart is similar, and its redundant descriptions will be omitted.

[0093] Next, the features of the present invention will be described in detail. Figure 2 The processing of steps S202 and S205 in the process.

[0094] In step S202, the control unit 101 changes the frame rate of the pre-capture image in relation to the shutter speed of the main camera. This will refer to... Figure 13A and Figure 13B Please provide an explanation.

[0095] Figure 13A This is a timing diagram showing typical pre-camera and main camera shots. Figure 13B An example is shown where the frame rate of the pre-capture video is changed according to the second embodiment of the shutter speed for the main camera. As an example, the shutter speed of the main camera is 1 / 30 second, and the frame rate is changed before... Figure 13A The frame rate of the pre-recorded video is 120fps. At a shutter speed of 1 / 30 second, an image is captured in the main camera 1311 with an exposure time of 1 / 30 second. According to this embodiment, as... Figure 13B As shown, the control unit 101 changes the frame rate of the pre-camera to be close to the exposure time of the main camera. For example, if the frame rate of the pre-camera is 120fps and the exposure time of the main camera is adjusted to 1 / 30 second, the control unit 101 changes the frame rate of the pre-camera to 30fps. Specifically, the frame rate of the pre-camera is represented by the following equation (3).

[0096] The changed frame rate of the pre-camera is equal to β / the shutter speed of the main camera (3).

[0097] In equation (3), β is a constant less than or equal to 1.

[0098] Similar to the constant α in equation (2) according to the first embodiment, the constant β is a predetermined constant that changes based on the ISO sensitivity and shutter speed of the main camera. The control unit 101 changes the frame rate of the pre-captured image to satisfy equation (3), thereby setting a frame time period that can appropriately detect subject blur at the shutter speed of the main camera. Figure 13B The changed frame rate is shown, and compared with the one used in... Figure 13A Compared to the subject motion detection in adjacent frames used in the pre-capture process at the frame rate shown, Figure 13B The frame time interval (1321) in the pre-capture camera is increased, and it is a time interval close to the exposure time of the main camera 1311. Therefore, by controlling such... Figure 13B The frame rate of the pre-capture camera, corresponding to the exposure time of the main camera 1311, makes the subject motion detectable.

[0099] Reference Figure 14 The flowchart illustrates the processing of step S205. The steps shown in the flowchart are performed by the control unit 101 or by the various units of the camera device 100, including the motion blur notification image generation unit 300, based on instructions from the control unit 101.

[0100] In step S1401, the control unit 101 performs a process similar to that in step S401 and sets the imaging conditions. Then, the control unit 101 controls the optical system 104 and the imaging unit 105, and the imaging device 100 captures a series of pre-images.

[0101] In step S1402, the control unit 101 performs a process similar to that in step S404.

[0102] In step S1403, the control unit 101 performs a similar process to step S405, and the motion vector calculation unit 301 calculates the motion of the subject based on the preparatory image captured in step S1401.

[0103] In step S1404, the control unit 101 performs a process similar to that in step S406, and the motion blur calculation unit 302 calculates the relationship between the calculated motion amount of the subject and the target motion blur amount.

[0104] In step S205, the control unit 101 performs a communication with... Figure 4 The process is similar to step S406 in the previous step. The notification plane generation unit 303 generates the image plane used for motion blur notification based on the estimated motion blur amount in the main camera in step S1404, and the image overlay unit 304 generates the motion blur notification image by overlaying the motion blur notification plane onto the preparation image.

[0105] The above describes in detail the control unit 101 according to the second embodiment. Figure 2 The feature processing steps S202 and S205 in the entire process.

[0106] The structure according to the second embodiment enables the control unit 101 to control the frame rate of the pre-capture images used to detect subject motion affecting the exposure time of the main camera. Therefore, motion blur in the main camera is accurately detected. Furthermore, the absence of a structure that requires capturing continuous pre-capture images at a constant high frame rate for subject motion detection allows for accurate motion detection with reduced power consumption.

[0107] According to one aspect of the invention, the control unit 101 selects different preliminary image frames for motion detection for different shutter speeds of the main camera. If the shutter speed of the main camera is higher than the maximum value of the frame rate that can be set by the imaging device 100, the frame rate of the preliminary image is fixed at the maximum value, and the preliminary image closest in frame time interval is selected for motion detection. Using this condition, no motion blur actually occurs in the main camera, making motion blur estimation unnecessary. If the shutter speed of the main camera is greater than a given value, i.e., the exposure time of the main camera is shorter than a given time, the frame rate can be fixed to a specific value.

[0108] Furthermore, a threshold can be set for the shutter speed of the main camera, and the control unit 101 can control the frame rate of the pre-capture video based on the set value. For example, a first threshold and a second threshold larger than the first threshold can be set for the shutter speed of the main camera. If the shutter speed of the main camera is less than the first threshold, the control unit 101 uses the first frame rate for control. Furthermore, if the shutter speed of the main camera is greater than or equal to the first threshold and less than the second threshold, the control unit 101 uses the second frame rate for control. If the shutter speed of the main camera is greater than or equal to the second threshold, the control unit 101 uses the third frame rate for control.

[0109] While the foregoing has described an example of providing motion blur notification when the estimated motion blur is greater than or equal to a predetermined value according to one aspect of the invention, the invention can also be applied, for example, to providing multiple motion blur displays based on the amount of motion. An example is provided where the estimated size of the motion blur is represented using three colors (red, blue, and yellow). In this case, a permissible range of motion magnitude is set that enables the detection of the narrowest range of motion magnitude, such that these colors can be used to represent the range of motion magnitude. For example, if the colors representing motion blur based on the vector size are yellow for 0 to 10 pixels, blue for 11 to 15 pixels, and red for 16 or more pixels, then the blue range is the narrowest range of motion magnitude. Specifically, five pixels (11 to 15) of the blue range are set as the permissible motion magnitude.

[0110] Although the permissible motion blur amount is set and the frame rate of the shooting conditions as a preparatory image is controlled according to one aspect of the invention so that motion less than or equal to the permissible motion blur amount can be detected, the shooting conditions (motion priority mode) can also be changed if the detected motion blur is less than or equal to a predetermined value.

[0111] If a stable subject motion is detected, the pre-recorded image conditions can be changed. Furthermore, the stability of the motion vector is calculated as the change in the motion vector after the pre-recorded image conditions are changed, and if the change in the motion vector per unit time increases, the pre-recorded image conditions can be changed again.

[0112] If the permissible motion amount is not preset, the magnitude of the motion vector within the motion detection limit range can be set as the permissible motion amount. In this case, it is necessary to determine whether the subject motion exceeds the motion detection limit range. Specifically, it is necessary to determine whether the subject motion exceeds the motion detection range. This determination can be made based on whether the change in the sum of absolute differences (SAD) in motion detection is less than a predetermined value, or by using a method that measures the motion of the camera device 100, such as gyroscope information. If it is determined that the permissible motion amount has been exceeded, the shutter speed and / or frame rate are increased.

[0113] While the above description of a motion blur notification method for displaying motion blur notifications using a display unit 109 according to one aspect of the present invention is provided as an example, the motion blur notification method is not limited to those described above. For example, sound, light, or vibration can be used as motion blur notification. Specifically, if the estimated number of pixels with motion blur greater than or equal to a predetermined value is greater than or equal to a predetermined ratio relative to the entire screen, a motion blur notification sound, notification light, or notification vibration can be provided.

[0114] (Other embodiments)

[0115] The object of the present invention can also be achieved as follows. Specifically, a storage medium containing program code of software describing the processing for performing the functions of the embodiments is fed to a system or device. Then, the computer (or CPU or microprocessor unit (MPU) of the system or device reads the program code stored in the storage medium and executes the read program code.

[0116] In this case, the program code read from the storage medium implements the novel functionality of the invention, and the storage medium storing the program code and the program are included in the invention.

[0117] Furthermore, examples of storage media used for feeding program code include floppy disks, hard disks, optical disks, and magneto-optical disks. Additionally, compact disc ROM (CD-ROM), recordable compact disc (CD-R), rewritable compact disc (CD-RW), digital universal disc ROM (DVD-ROM), digital universal disc RAM (DVD-RAM), rewritable digital universal disc (DVD-RW), recordable digital universal disc (DVD-R), magnetic tape, non-volatile memory cards, and ROM can also be used.

[0118] Furthermore, the computer is configured to execute the read program code to perform the functions of the embodiments. Additionally, this includes situations where the operating system (OS) running on the computer performs the actual processing, partially or completely, based on the instructions of the program code, which enables the execution of the functions of the embodiments.

[0119] This also includes the following scenario. First, program code read from a storage medium is written to the memory of a function expansion board or function expansion unit inserted into the computer. Then, the CPU on the function expansion board or function expansion unit performs actual processing, partially or entirely, based on the instructions in the program code.

[0120] Although the invention has been described with reference to exemplary embodiments, it should be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims is to be interpreted in the broadest sense to include all such modifications, equivalent structures, and functions.

Claims

1. An information processing device, comprising: The acquisition component is configured to acquire a plurality of first captured images taken by a first camera using first camera parameters, and to acquire motion information related to the subject in the plurality of first captured images; An estimation component is configured to estimate the motion blur of the subject in a second captured image obtained by the second camera based on the motion information and the second camera parameters; as well as The setting component is configured to set the first camera parameters and the second camera parameters. The acquisition component selects an image from the plurality of first captured images for use when acquiring the motion information based on the value difference between the frame rate, which is one of the first camera parameters, and the exposure time of the second camera.

2. The information processing device according to claim 1, wherein, The acquisition component selects two images from the plurality of first captured images, such that the time period of the two selected images corresponds to the exposure time of the second camera.

3. The information processing device according to claim 1 further includes a notification component, the notification component being configured to notify information related to the motion blur, wherein the notification component notifies the motion blur by displaying information corresponding to the motion blur on a display unit.

4. The information processing device according to claim 3, wherein, The notification component notifies the motion blur by displaying at least one of the plurality of first captured images, which contains information corresponding to the motion blur, on a display unit.

5. The information processing device according to claim 3, wherein, In response to the estimation component estimating a motion blur greater than a predetermined value, the notification component notifies the motion blur.

6. The information processing device according to claim 3, wherein, When the notification setting of the notification component is set to off, the notification component does not notify the motion blur.

7. The information processing device according to claim 1, wherein, The first camera is a preparatory camera that captures images at a time different from that of the second camera.

8. The information processing device according to claim 7, wherein, The second camera is the main camera, and the first camera is a pre-recorded shot taken before the main camera.

9. A camera device, comprising: A camera component, configured to perform a first camera shot; as well as The information processing device according to claim 1.

10. A camera device including a camera component, and the camera device being configured to output a second captured image based on the camera instruction during the sequential output of a plurality of first captured images captured by a first camera using a first camera parameter, in response to a camera instruction issued by a user during the sequential output of such images by the camera component. The camera device further includes: An acquisition component is configured to acquire motion information based on the plurality of first captured images output from the camera component; An estimation component is configured to estimate the motion blur of a subject in a second captured image obtained by the second camera based on the motion information and the second camera parameters; as well as The setting component is configured to set the first camera parameters and the second camera parameters. The acquisition component selects an image from the plurality of first captured images for use when acquiring the motion information based on the value difference between the frame rate, which is one of the first camera parameters, and the exposure time of the second camera.

11. An information processing method, comprising: Acquire multiple first images captured by a first camera using first camera parameters, and acquire motion information related to the subject in the multiple first images; Based on the motion information and the second camera parameters, estimate the motion blur of the subject in the second captured image obtained by the second camera; and Set the first camera parameters and the second camera parameters. The image to be used when acquiring the motion information is selected from the plurality of first captured images based on the difference between the frame rate, which is one of the first camera parameters, and the exposure time of the second camera.

12. A method for controlling a camera device, the camera device including a camera component, and the camera device being configured to output a second image captured based on the camera instruction by a second camera using second camera parameters in response to a camera instruction issued by a user during the sequential output of a plurality of first images captured by a first camera using first camera parameters, the control method comprising: Motion information is obtained based on the plurality of first captured images output from the camera component; Based on the motion information and the second camera parameters, estimate the motion blur of the subject in the second captured image obtained by the second camera; and Set the first camera parameters and the second camera parameters. The image to be used when acquiring the motion information is selected from the plurality of first captured images based on the difference between the frame rate, which is one of the first camera parameters, and the exposure time of the second camera.

13. A non-transitory computer-readable storage medium for storing a program configured to cause a computer to perform the steps of a control method for an information processing device, the control method comprising: Acquire multiple first images captured by a first camera using first camera parameters, and acquire motion information related to the subject in the multiple first images; Based on the motion information and the second camera parameters, estimate the motion blur of the subject in the second captured image obtained by the second camera; and Set the first camera parameters and the second camera parameters. The image to be used when acquiring the motion information is selected from the plurality of first captured images based on the difference between the frame rate, which is one of the first camera parameters, and the exposure time of the second camera.

Images