Imaging device, control method for imaging device, and program

Abstract

This suppresses the display of images with significant subject positional shift after exposure is complete during shooting. [Solution] The imaging device comprises an image sensor, a field of view control means for controlling the field of view of an image obtained using the image sensor by moving a driven object with a driving means, a display means for displaying an image obtained using the image sensor, and a display control means for changing the image to be displayed on the display means after the exposure for obtaining the first image is completed, according to the amount of movement of the driven object from a reference position by the driving means during exposure for obtaining the first image using the image sensor.

Description

Technical Field

[0001] The present invention relates to an imaging device, a control method for the imaging device, and a program.

Background Art

[0002] There is a technique for correcting blurring in shooting by shooting means. Examples of blurring in shooting include camera shake of the photographer and subject blurring caused by movement of the subject. Patent Document 1 discloses detecting vertical and horizontal vibrations generated in the camera body by a gyro sensor and correcting the detected vibrations by translating a driving unit such as a lens or an imaging element so as to correct blurring caused by camera shake. Further, Patent Document 2 discloses correcting blurring caused by subject blurring by translating a driving unit based on the angular velocity of the subject calculated based on the relative difference between the moving speed of the subject and the panning speed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] When an imaging device continuously shoots a subject a plurality of times, in order to make the operating range in the driving of the driving unit for blurring correction common for each shooting, for each shooting, a return operation may be performed in which the driving unit is driven to return to the position before the start of correction. In this case, not only during the driving of the driving unit for correction, but also during the return operation of the driving unit, a change in the angle of view due to the driving occurs. Furthermore, the imaging device may perform a so-called live view display, which shows images generated by capturing the subject image but not used for image recording as live images (also called through images) on the display unit. The video displayed as a live image on the display unit is updated with each frame of the live image.

[0005] In this case, the imaging device may take multiple consecutive shots of the subject while a live image is displayed on the display unit. In this case, if the drive unit is driven during exposure during shooting, and then the image generated by the image taken during the drive unit's recovery operation after exposure is complete is displayed as the live image, the display unit will show an image affected by the change in field of view due to the drive unit's recovery operation. Furthermore, even if the change in field of view due to the recovery operation is large, if the image generated by the image taken during the recovery operation is displayed as the live image, the positional shift of the subject from the previous live image will be large, which may cause discomfort to the user viewing the live image. The present invention aims to suppress the display of images with significant subject positional shifts after exposure is complete during photography. [Means for solving the problem]

[0006] To solve the above problems, the imaging apparatus according to the present invention is characterized by comprising: an image sensor; a field of view control means for controlling the field of view of an image obtained using the image sensor by moving a driven object with a driving means; a display means for displaying an image obtained using the image sensor; and a display control means for changing the image to be displayed on the display means after the exposure for obtaining the first image is completed, according to the amount of movement of the driven object from a reference position by the driving means during exposure for obtaining the first image using the image sensor. [Effects of the Invention]

[0007] According to the present invention, it is possible to suppress the display of images with significant subject positional shifts after shooting is completed. [Brief explanation of the drawing]

[0008] [Figure 1] This is an overall diagram of the imaging device. [Figure 2] This diagram shows the configuration of the correction control unit and the panning control unit in the lens microcontroller of the lens device. [Figure 3] This is a flowchart illustrating the process of panning assist. [Figure 4] Figures (A) and (B) show the relationship between the amount of movement of the shift lens to correct subject blur and the time for which a replacement image is displayed on the display unit. [Figure 5] This diagram shows the relationship between the return operation of the shift lens and the changes in the content of the through image displayed on the display unit over time. [Figure 6] This diagram shows the relationship between the return operation of the shift lens and the changes in the content of the image displayed on the display unit over time. [Figure 7] This diagram shows the relationship between the return operation of the shift lens and the changes in the content of the image displayed on the display unit over time. [Figure 8] This is a flowchart illustrating the pre-processing steps before image capture. [Figure 9] This is a flowchart illustrating the pre-processing steps for image capture as a variation of the original procedure. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is an overall diagram of the imaging device 1. The imaging device 1 of this embodiment has a function to assist the user in panning. Panning is a shooting technique that involves continuously shooting while tracking a moving subject with the imaging device 1, thereby obtaining multiple photographs in which the subject appears still while the background is blurred. Examples of moving subjects include vehicles, ships, and aircraft squadrons, but the moving subject may also be a different subject, such as a moving person. The function to assist the user in panning is a function that corrects blur in panning. The function of the imaging device 1 to assist the user in panning may be referred to as panning assist below. The shooting mode of the imaging device 1 in which panning assist is performed may be referred to as panning assist mode below. The imaging device 1 comprises a lens device 100 and a camera body 130. Since the lens device 100 is detachable from the camera body 130, the lens device 100 is provided in the imaging device 1 in an interchangeable manner. However, the lens device 100 and the camera 130 may be provided as an integrated unit in the imaging device 1.

[0010] The lens device 100 includes a lens unit 101, a position detection unit 105, an encoder 106, an angular velocity detection unit 111, a lens microcontroller 112, a driver 113, an amplifier circuit (AMP) 114, and a mount contact unit 115. The lens unit 101 guides subject light to the image sensor 132, which will be described later, located in the camera body 130. The lens unit 101 includes a main imaging optical system 102, a zoom lens 103, and a shift lens 104. The zoom lens 103 is a lens group whose focal length can be changed. The shift lens 104, as an example of an image stabilization lens, is a lens group that corrects blur in the captured image caused by the shaking of the imaging device 1. More specifically, the shift lens 104 corrects blur in the captured image by changing the angle of view by moving in a direction perpendicular to the optical axis. Therefore, the shift lens 104 can also be considered as a driveable object that can change the angle of view of the image by driving it. The encoder 106 detects the position of the zoom lens 103. The position detection unit 105 is, for example, a Hall element, and detects the position of the shift lens 104. The angular velocity detection unit 111 is, for example, a gyro sensor, and detects the shaking of the imaging device 1. The driver 113 is a voice coil motor used as a driving means to drive the shift lens 104. The AMP 114 amplifies the output of the position detection unit 105. The mount contact unit 115 relays communication with the camera body 130.

[0011] The lens microcontroller 112 is an abbreviation for the control microcomputer for the lens device 100, and controls the entire lens device 100. The lens microcontroller 112 includes, for example, a CPU (Central Processing Unit) and memory. The lens microcontroller 112 also includes a correction control unit 121 and a panning control unit 122. The image stabilization control unit 121 corrects image blur caused by shaking of the image device 1 by driving the shift lens 104 based on the detection results of the angular velocity detection unit 111 when the image device 1 is in a normal mode other than panning assist mode. The driving direction of the shift lens 104 includes the direction perpendicular to the optical axis of the optical element. Furthermore, image blur correction by the image stabilization control unit 121 is performed with respect to two orthogonal axes, such as the vertical and horizontal directions. In addition, the lens microcontroller 112 performs control such as focus lens control and aperture control.

[0012] The panning control unit 122 controls the panning assist. Specifically, when the imaging device 1 is in panning assist mode, the panning control unit 122 corrects subject blur caused by subject movement by driving the shift lens 104 based on information such as angular velocity obtained from the camera body 130. The panning control unit 122 drives the shift lens 104 by controlling the driver 113. Furthermore, the panning control unit 122 may drive the shift lens 104 to correct subject blur in individual shots during continuous subject photography in the panning assist mode by the imaging device 1. In this case, the panning control unit 122 causes the shift lens 104 to perform a return operation to return to its reference position before the start of correction each time a shot is taken while the shift lens 104 is driven. In this case, the operating range for driving the shift lens 104 for blur correction becomes common for each shot. The return operation may include centering the position of the shift lens 104. As described above, the lens microcontroller 112 functions as a field-of-view control unit that controls the field of view of the image obtained using the image sensor 132. Alternatively, it may communicate with the camera microcontroller 141, which will be described later, to control the field of view, or at least one of the lens microcontroller 112 and the camera microcontroller 141 may function as a field-of-view control unit.

[0013] The camera body 130 includes a shutter 131, an image sensor 132, an analog signal processing circuit (AFE) 133, a signal processing circuit 134, a timing generator (TG) 135, an operating unit 136, a driver 137, and a shutter drive motor 138. The camera body 130 also includes a memory card 139, a display unit 140, a camera microcontroller 141, a mount contact unit 144, an angular velocity detection unit 171, and a distance detection unit 181. The shutter 131 controls the exposure time of the imaging device 132. The imaging device 132 is an image sensor using a CMOS (Complementary Metal Oxide Semiconductor) device or the like. The imaging device 132 photoelectrically converts subject light imaged through the lens unit 101 and outputs an electrical signal. The imaging device 132 can be regarded as imaging means for generating an image by imaging a subject image. Also, the imaging device 132 can be regarded as imaging means for taking an image as a captured image according to an imaging instruction from the user. Examples of the imaging instruction from the user include an imaging instruction performed by the user operating the operation unit 136. The AFE 133 processes the electrical signal output from the imaging device 132 and outputs it to the signal processing circuit 134. The signal processing circuit 134, which is an example of the changing means, is a signal processing circuit of the camera body 130 and generates an image signal by processing the electrical signal output from the AFE 133.

[0014] The signal processing circuit 134 has a motion vector detection unit 145. The motion vector detection unit 145 detects a motion vector of a subject reflected in an image based on a plurality of images with different imaging times. That is, the motion vector detection unit 145 detects the motion of a subject between different frames (between images) as a motion vector. Note that information indicating the motion vector detected by the motion vector detection unit 145 may be referred to as vector information hereinafter. Also, each time a new image is generated by imaging, the motion vector detection unit 145 detects a motion vector of a subject reflected in the image based on the motion of the subject between frames including the latest image, and generates vector information. Then, each time the motion vector detection unit 145 generates vector information, it transmits the generated vector information to the camera microcomputer 141. Also, the signal processing circuit 134 functions as a display control unit that controls the image displayed on the display unit 140. TG135 sets the operation timings of the imaging device 132 and the AFE 133. The operation unit 136 is used for operation inputs by the photographer. Examples of the operation unit 136 include a power switch, a release switch, and the like. The driver 137 controls the shutter drive motor. The shutter drive motor 138 drives the shutter 131. The memory card 139 records the image obtained by imaging.

[0015] The display unit 140 displays the image obtained by imaging. Examples of the display unit 140 include a liquid crystal panel (LCD) and the like. The display unit 140 of the present embodiment displays the image signal generated by the signal processing circuit 134 as a through image. This through image is a video obtained by the output from the imaging device 132 being displayed on the display unit 140 in real time. Therefore, the through image can also be regarded as a live image. Further, in the present embodiment, when the imaging device 1 assists in continuous shooting, the through image is displayed on the display unit 140. The mount contact portion 144 relays communication with the lens device 100. The angular velocity detection unit 171 detects the shake of the imaging device 1 as an angular velocity and outputs the detection result to the camera microcomputer 141. The distance detection unit 181 detects the distance to the subject (subject distance) and outputs the detection result to the camera microcomputer 141. As an example of an estimation means, the camera microcontroller 141 is an abbreviation for the control microcomputer for the camera body 130 and controls the entire camera body 130. The camera microcontroller 141 has, for example, a CPU and memory. The camera microcontroller 141 also has a shutter control unit 151, an angular velocity calculation unit 152, and a shutter speed calculation unit 153. The shutter control unit 151 controls the driver 137. The angular velocity calculation unit 152 calculates the angular velocity of the subject used for correcting subject blur based on vector information transmitted from the motion vector detection unit 145 and the focal length, etc. The vector information used in the calculation by the angular velocity calculation unit 152 is the latest vector information. The angular velocity calculation unit 152 transmits information indicating the calculated result to the lens microcontroller 112. The information indicating the angular velocity of the subject calculated by the angular velocity calculation unit 152 may be referred to as angular velocity information below. The shutter speed calculation unit 153 calculates a shutter speed suitable for panning.

[0016] In this embodiment, when the user operates the control unit 136 and power is supplied to the camera body 130, the power input is detected by the camera microcontroller 141, and the camera microcontroller 141 controls the supply of power to each circuit of the camera body 130 and performs initial settings. Also, when power is supplied to the camera body 130, power is supplied from the camera body 130 to the lens device 100, and the lens microcontroller 112 controls the initial settings within the lens device 100. Furthermore, communication between the lens device 100 and the camera body 130 is started at a predetermined timing by the control of the lens microcontroller 112 and the camera microcontroller 141. In the communication, the camera body 130 transmits camera information, such as the status of the camera body 130 and shooting settings, as well as angular velocity information, to the lens device 100. Shooting settings include, for example, the setting of the panning assist mode. In the communication, the lens device 100 transmits information, such as the focal length of the lens, to the camera body 130.

[0017] Figure 2 shows the configuration of the correction control unit 121 and the panning control unit 122 in the lens microcontroller 112 of the lens device 100. The correction control unit 121 includes an offset removal unit 201, a phase calculation unit 202, an integrator 203, a vibration isolation control determination unit 204, a subtractor 205, an A / D converter 206, a controller 207, and a pulse width modulation unit 208. The offset removal unit 201 removes the DC component contained in the output of the angular velocity detection unit 111. The offset removal unit 201 is a filter calculation unit composed of, for example, a high-pass filter (hereinafter referred to as HPF). The phase calculation unit 202 has an amplifier that amplifies the information indicating the angular velocity from which the offset component has been removed by the offset removal unit 201 at a predetermined gain, and a phase compensation filter. The integrator 203 has a function that allows its characteristics to be changed in any frequency band. The integrator 203 integrates the output of the phase calculation unit 202, calculates the drive amount of the shift lens 104 used for image stabilization control, and outputs it to the vibration stabilization control determination unit 204. If the angular velocity output by the angular velocity detection unit 111 remains above a certain value for a predetermined time, the integrator 203 determines that the camera is panning and gradually changes the cutoff frequency of the high-pass filter (HPF) in the offset removal unit 201 to the high-frequency side. Furthermore, the integrator 203 gradually changes the cutoff frequency towards the higher frequency side, thereby gradually decreasing the target signal for image stabilization control and returning the shift lens 104 to its optical center position. In this case, if the correction is performed without changing the cutoff frequency towards the higher frequency side based on a large angular velocity such that the imaging device 1 is determined to be panning, the shift lens 104 will reach the correction limit point, which would suppress the occurrence of unnatural changes in the field of view for the photographer.

[0018] The vibration isolation control determination unit 204 switches the control signal for driving the shift lens 104 according to the information transmitted from the panning control unit 122. The information transmitted from the panning control unit 122 includes information indicating whether or not the imaging device 1 is in panning assist mode. Depending on whether or not the imaging device 1 is in panning assist mode, the vibration isolation control determination unit 204 decides whether to use the output from the integrator 225 described later in the panning control unit 122 or the output from the integrator 203. The A / D converter 206 digitizes the output of the AMP 114 and outputs it. The subtractor 205 subtracts the output of the A / D converter 206 from the output of the vibration isolation control determination unit 204 and outputs the deviation data to the controller 207. The controller 207 has an amplifier that amplifies the deviation data output by the subtractor 205 with a predetermined gain, and a phase compensation filter. The deviation data is processed by an amplifier and a phase compensation filter in the controller 207 and then output to the pulse width modulation unit 208. The pulse width modulation unit 208 modulates the output of the controller 207 into a waveform that changes the duty cycle of the pulse wave (PWM) and outputs it to the driver 113 for driving the shift lens. In this case, the driver 113 drives the shift lens 104, thereby correcting image blur.

[0019] The panning control unit 122 includes a communication control unit 211, an angular velocity output unit 222, an angular velocity acquisition unit 223, an adder 224, an integrator 225, and a camera information acquisition unit 226. The communication control unit 221 communicates with the camera microcontroller 141. The communication control unit 211 receives angular velocity information and camera information from the camera microcontroller 141. The angular velocity acquisition unit 223 calculates the angular velocity of the subject during the exposure period based on the angular velocity of the subject identified from the angular velocity information received by the communication control unit 211, and outputs the calculation result to the adder 224. The camera information acquisition unit 226 acquires the camera information received by the communication control unit 221 and transmits the acquired camera information to the vibration isolation control determination unit 204. The angular velocity output unit 222 acquires information indicating the generated angular velocity from the offset removal unit 201 and outputs the acquired information to the communication control unit 211. In this case, the communication control unit 211 transmits the information acquired from the angular velocity output unit 222 to the camera microcontroller 141. The adder 224 takes the output of the offset removal unit 201 as a positive input and the output of the angular velocity acquisition unit 223 as a negative input and performs subtraction. The subtraction result is output to the integrator 225. The integrator 225 integrates the output of the adder 224 to calculate the drive amount of the shift lens 104 used for controlling the panning assist, and outputs the calculated result to the vibration damping control determination unit 204. The information indicating the drive amount of the shift lens 104 calculated by the integrator 225 may be referred to as drive information below. As described above, if the panning assist is set, the vibration damping control determination unit 204 adopts the drive information output to the integrator 225 and outputs it to the subtractor 205. As a result, the shift lens 104 is driven based on the drive amount calculated by the panning control unit 122, and subject blur is corrected.

[0020] Figure 3 is a flowchart showing the flow of the panning assist process. The panning assist process is the process in which the imaging device 1 assists with panning. In this embodiment, the panning assist process is started when the imaging device 1 is instructed to take multiple consecutive shots in panning assist mode. The instruction to take multiple shots in panning assist mode is given by the user operating the control unit 136. First, the image sensor 132 starts capturing images. The signal processing circuit 134 starts displaying the image, which is a through image generated by the capture, on the display unit 140 (step 301, sometimes referred to as "S" below). The through image, which is displayed starting in step 301, is updated at a predetermined period (frame rate).

[0021] The angular velocity calculation unit 152 of the camera microcontroller 141 calculates the angular velocity of the subject and generates angular velocity information (S302). The angular velocity calculation unit 152 transmits the generated angular velocity information to the lens microcontroller 112. The camera microcontroller 141 determines whether or not there is a shooting instruction (S303). The camera microcontroller 141 makes the determination in step 303 based on whether or not the user has made an operation on the operation unit 136 to instruct shooting. If there is no shooting instruction (No in S303), the process from step 302 is repeated. That is, the angular velocity calculation unit 152 calculates the angular velocity of the subject until a shooting instruction is given. In addition, each time the frame of the through image is updated, the motion vector detection unit 145 generates vector information indicating the latest detection result, and the angular velocity calculation unit 152 performs the calculation using this latest vector information. Therefore, until a shooting instruction is given, the latest angular velocity information is generated each time the frame of the through image is updated.

[0022] Furthermore, if there is an instruction to take a picture (Yes in S303), the lens microcontroller 112 calculates the amount of drive required for the shift lens 104 to correct subject blur based on the latest angular velocity information generated in step 302. Then, as the image sensor 132 starts exposure, the lens microcontroller 112 drives the shift lens 104 according to the calculated amount of drive (S304). Subject blur is corrected by driving the shift lens 104. In this embodiment, the drive of the shift lens 104 continues until the exposure time for the image sensor 132 has elapsed. The camera microcontroller 141 determines whether a predetermined exposure time for the image sensor 132 has elapsed since the image sensor 132 began exposure (S305). As long as the result remains negative (No in S305), the process in step 305 is repeated.

[0023] Furthermore, if the exposure time has elapsed (Yes in S305), the camera microcontroller 141 determines the amount of drive of the shift lens 104 (S306). This amount of drive is the amount of drive of the shift lens 104 from the time the image sensor 132 starts exposure until the exposure time has elapsed, that is, the amount of drive during exposure in the image capture. In other words, the amount of drive that becomes the target of the determination in step 306 is the amount of drive due to the drive of the shift lens 104 that was started in step 304. In this embodiment, the lens microcontroller 112 transmits the drive information generated by the integrator 225 in the panning control unit 122 to the camera microcontroller 141. The camera microcontroller 141 determines the amount of drive of the shift lens 104 from the received drive information. Therefore, the camera microcontroller 141 can also be considered as an acquisition means for acquiring drive information transmitted from the lens device 100. Alternatively, the camera microcontroller 141 may determine the amount of drive of the shift lens 104 based on the angular velocity information generated in step 302, the exposure time of the image sensor 132, and information indicating the angular velocity previously acquired from the angular velocity output unit 222. In this case, even if a communication failure occurs between the camera microcontroller 141 and the lens microcontroller 112, the amount of drive of the shift lens 104 can still be determined.

[0024] The camera microcontroller 141 determines whether the drive amount of the shift lens 104 identified in step 306 is less than the change threshold (S307). The change threshold is a predetermined value used by the camera microcontroller 141 to determine whether or not to change the display content of the display unit 140 from the through image. In this embodiment, as described above, when the shift lens 104 is driven to correct subject blur during individual shots among multiple shots of the subject in panning assist mode, the shift lens 104 returns to its original position after the exposure for shooting is completed. In addition, during the return operation, the content of the through image displayed on the display unit 140 may be updated due to a new image captured by the image sensor 132. In this case, if the amount of drive of the shift lens 104 during the return operation is large and the change in the angle of view is also large, the positional shift of the subject in the newly displayed through image will be large compared to the through image that was displayed immediately before. Therefore, in this embodiment, a change threshold is set so that the display of the through image on the display unit 140 is suppressed when the return operation of the shift lens 104 is performed with a drive amount that results in a large change in the angle of view and a large positional shift of the subject in the through image. The change threshold is held in the camera microcontroller 141. The change threshold may also be set for each type of lens device 100. More specifically, the change threshold may be set to a drive amount of the shift lens 104 that results in a large change in the angle of view and a large positional shift of the subject in the through image, depending on the optical system configuration of the lens device 100. Furthermore, the camera microcontroller 141 identifies the amount of drive of the shift lens 104 identified in step 306 as the amount of drive of the shift lens 104 after the exposure of the image sensor 132 is completed. Therefore, the camera microcontroller 141 can also be considered as a means for identifying the amount of drive of the shift lens 104 after the exposure is completed.

[0025] If the drive amount of the shift lens 104 identified in step 306 is less than the change threshold (Yes in S307), the camera microcontroller 141 instructs the lens microcontroller 112 to return the shift lens 104 to its original position. Upon receiving the instruction from the camera microcontroller 141, the lens microcontroller 112 starts the return operation of the shift lens 104 (S308). In this case, the through image continues to be displayed on the display unit 140. In other words, if the image sensor 132 takes a new image while the shift lens 104 is returning to its original position, the through image displayed on the display unit 140 is updated with the newly captured image.

[0026] Furthermore, if the drive amount of the shift lens 104 identified in step 306 is greater than or equal to the change threshold (No in S307), the camera microcontroller 141 determines the time for displaying the alternative image on the display unit 140 (S309). The alternative image is a predetermined image that is displayed on the display unit 140 in place of the through image. In other words, if the drive amount of the shift lens 104 is greater than or equal to the change threshold, the through image is not displayed on the display unit 140. Examples of alternative images include an image whose entire surface is a predetermined color (a black image). The camera microcontroller 141 determines the display time of the alternative image on the display unit 140 according to the amount of drive in the return operation of the shift lens 104. As described above, the amount of drive in the return operation of the shift lens 104 refers to the amount of drive of the shift lens 104 identified in step 306. Therefore, the camera microcontroller 141 determines the display time of the alternative image on the display unit 140 according to the amount of drive of the shift lens 104 identified in step 306. More specifically, the camera microcontroller 141 displays the alternative image on the display unit 140 until the return operation of the shift lens 104 is completed. Also, the larger the amount of drive in the return operation of the shift lens 104, the longer the time required for the return operation of the shift lens 104. In other words, the camera microcontroller 141 increases the display time of the alternative image on the display unit 140 as the amount of drive of the shift lens 104 increases. In this embodiment, the camera microcontroller 141 maintains table information in which the time for which the alternative image is displayed on the display unit 140 is pre-associated for each amount of drive of the shift lens 104. Furthermore, the time for which the alternative image is displayed on the display unit 140 may be pre-associated with each range of the drive amount of the shift lens 104. Also, the number of stages in the display time of the alternative image associated with the drive amount or range of the drive amount of the shift lens 104 may be any number. In addition, the drive amount or range of the drive amount of the shift lens 104 and the associated display time of the alternative image may be determined for each type of lens device 100. More specifically, the drive amount or range of the drive amount of the shift lens 104 and the associated display time of the alternative image may be determined according to the configuration of the optical system in the lens device 100. Furthermore, the display time of the alternative image may be determined as the number of frames of the through image. When the display time of the alternative image is determined as the number of frames of the through image, the alternative image becomes easier to replace the through image.

[0027] The signal processing circuit 134 starts displaying the alternative image on the display unit 140 (S310). The camera microcontroller 141 instructs the lens microcontroller 112 to return the shift lens 104 to its original position. Upon receiving the instruction from the camera microcontroller 141, the lens microcontroller 112 initiates the return operation of the shift lens 104 (S311). This return operation changes the field of view, but a substitute image is displayed on the display unit 140, and the change in field of view does not affect the substitute image.

[0028] The camera microcontroller 141 determines whether the display time for the substitute image by the display unit 140 has elapsed (S312). As long as the negative result persists (No in S312), the process in step 312 is repeated. Furthermore, if the display time for the alternative image has elapsed (Yes in S312), the signal processing circuit 134 starts displaying the through image to the display unit 140 again (S313). The through image displayed on the display unit 140 is updated at a predetermined period (frame rate).

[0029] The camera microcontroller 141 determines whether or not the image sensor 132 will continue shooting (S314). If the user operates the control unit 136 to signal the end of shooting in panning assist mode, the camera microcontroller 141 determines that the image sensor 132 will not take a picture (No in S314). In this case, the panning assist process ends. Furthermore, if no instruction has been given to end shooting in panning assist mode, the camera microcontroller 141 determines that the image sensor 132 will continue shooting (Yes in S314). In this case, pre-shooting processing is performed (S315). As will be explained in detail later, pre-shooting processing is a process that is performed before the next shot is taken in panning assist mode.

[0030] In this embodiment, the return operation of the shift lens 104 is started after the alternative image is displayed on the display unit 140 (see steps 310 and 311 in Figure 3), but this is not the only way to go. The alternative image may be displayed on the display unit 140 after the shift lens 104 has started to return, or it may be displayed on the display unit 140 at the start of the shift lens 104's return operation. More specifically, the alternative image may be displayed on the display unit 140 at the timing of updating the through image frame reached during the return operation of the shift lens 104, at either timing.

[0031] Figure 4 shows the relationship between the amount of drive applied to the shift lens 104 to correct subject blur and the time it takes for a replacement image to be displayed on the display unit 140. The amount of drive applied to the shift lens 104 to correct subject blur refers to the amount of drive applied to the shift lens 104 from the time the image sensor 132 starts exposure during shooting until the exposure time has elapsed. In Figures 4(A) and 4(B), the horizontal axis represents time, and the vertical axis represents the position of the shift lens 104. In the examples shown in Figures 4(A) and 4(B), it is assumed that the imaging device 1 is instructed to take multiple shots in panning assist mode, and that panning assist processing is being performed.

[0032] First, we will explain the case where the amount of drive of the shift lens 104 is large, as shown in Figure 4(A). At time T1, the image sensor 132 begins exposure, and the shift lens 104 begins to move from its initial position P0. The process that begins at time T1 is the process that begins in step 304 of the panning assist process (see Figure 3).

[0033] Next, at time T2, which is later than time T1, the image sensor 132 finishes exposure. Therefore, the exposure time t1 of the image sensor 132 is the period from time T1 to time T2. At this time, the shift lens 104 finishes its drive for correcting motion blur and starts its return operation from the position P1 at which the drive for correction ended. Therefore, the drive amount D1 of the shift lens 104 for correcting motion blur is the distance from the initial position P0 to position P1. This drive amount D1 is assumed to be greater than or equal to the change threshold (No in S307 in Figure 3). Also, since the field of view may change during the execution of this return operation, an alternative image is displayed on the display unit 140.

[0034] When the return operation begins, the shift lens 104 is driven toward the initial position P0. During the return operation, the shift lens 104 moves back and forth around the initial position P0, converging toward the initial position P0. Therefore, when the shift lens 104 first reaches the initial position P0 during the return operation, it moves beyond the initial position P0, and at time T3, which is later than time T2, it is located at position P2, which is closer to the initial position P0 than position P1. Next, at time T4, which is later than time T3, the shift lens 104 returns to its initial position P0, and the recovery operation ends. At this time, the image displayed on the display unit 140 switches from the substitute image to the through image. Therefore, the display time of the substitute image is the period t2 from time T2 to time T4.

[0035] The display time of the alternative image is not limited to the example described above. At time T3, which is during the return operation by the shift lens 104, the shift lens 104 is located at position P2, which is close to the initial position P0. In this case, during the period from time T3 to time T4, when the return operation is completed, the degree of change in the field of view due to the return operation of the shift lens 104 is small. Therefore, the display time of the alternative image may be the period t3 from time T2 to time T3, when the shift lens 104 is located at a position determined to minimize the degree of change in the field of view due to the subsequent return operation of the shift lens 104. In the illustrated example, the position determined to minimize the degree of change in the field of view due to the return operation is position P2.

[0036] Next, we will explain the case where the amount of drive of the shift lens 104 is small, as shown in Figure 4(B). At time T11, the image sensor 132 begins exposure, and the shift lens 104 begins to move from its initial position P0. The process that begins at time T11 is the process that begins in step 304 of the panning assist process (see Figure 3).

[0037] Next, at time T12, which is later than time T11, the image sensor 132 finishes exposure. Therefore, the exposure time t4 of the image sensor 132 is the period from time T11 to time T12. This exposure time t4 is the same period as the exposure time t1 described above. At this time, the shift lens 104 finishes its drive for correcting motion blur and starts its return operation from position P3, where the drive for correction ended. Therefore, the drive amount D2 of the shift lens 104 for correcting motion blur is the distance from the initial position P0 to position P3. This drive amount D2 is assumed to be greater than or equal to the change threshold (No in S307 in Figure 3). Also, this drive amount D2 is smaller than the drive amount D1 described above. In other words, position P3 is closer to the initial position P0 than to position P1. Furthermore, since the field of view may change during the execution of this return operation, an alternative image is displayed on the display unit 140.

[0038] Next, at time T13, which is later than time T12, the shift lens 104 returns to its initial position P0, and the recovery operation ends. At this time, the image displayed on the display unit 140 switches from the alternate image to the through image. Therefore, the display time of the alternate image is the period t5 from time T12 to time T13. This period t5 is shorter than both period t2 and period t3. If the drive amount D2 of the shift lens 104 is less than the change threshold (Yes in S307 of Figure 3), the alternative image is not displayed, and the through image to the display unit 140 continues to be displayed from time T11 to time T13. Thus, the display time of the alternative image is determined according to the amount of drive in the return operation of the shift lens 104, in other words, the amount of drive for correcting subject blur of the shift lens 104. In this case, the through image is displayed on the display unit 140 during the time period when the positional shift of the subject in the through image becomes large due to the change in the field of view accompanying the return operation.

[0039] Figure 5 shows the relationship between the return operation of the shift lens 104 and the changes in the content of the through image displayed on the display unit 140 over time. In the example shown in Figure 5, it is assumed that the through image is continuously displayed on the display unit 140 regardless of the amount of drive during the return operation of the shift lens 104. Furthermore, the operation of the shift lens 104 shown in Figure 5 is assumed to be the same as the operation of the shift lens 104 shown in Figure 4(B). First, at time T21, the signal processing circuit 134 causes the display unit 140 to display a through image. As described above, the through image displayed on the display unit 140 is updated at a predetermined period (frame rate), and at time T21, the through image 401 is displayed.

[0040] Next, as a new image is captured, at time T22, which is later than time T21, the signal processing circuit 134 updates the image displayed on the display unit 140 to the latest through image 402. Next, when the user gives a shooting command, at time T23, which is later than time T22, the image sensor 132 starts exposure and the shift lens 104 starts driving to correct subject blur. Next, at time T24, which is later than time T23, the image sensor 132 finishes exposure and the readout of pixels from the image sensor 132 begins. At this time, the shift lens 104 also begins its return operation.

[0041] Then, as the shift lens 104 is returning to its original position, a new image is taken, and at time T25, which is later than time T24, the signal processing circuit 134 updates the image displayed on the display unit 140 to the latest through image 403. In this through image 403, the subject's position is shifted, or in other words, the subject is blurred, compared to the through image 402 that was displayed on the display unit 140 just before. This subject blurring occurs because the image is taken while the angle of view is changing due to the return operation of the shift lens 104. Next, at time T26, which is later than time T25, the reading of pixels from the image sensor 132 is completed.

[0042] Next, at time T27, which is later than time T26, the return operation of the shift lens 104 is completed. Next, as a new image is captured, at time T28, which is later than time T27, the signal processing circuit 134 updates the image displayed on the display unit 140 to the latest through image 404. Next, as a new image is captured, at time T29, which is later than time T28, the signal processing circuit 134 updates the image displayed on the display unit 140 to the latest through image 405. Note that through images 404 and 405 are images captured after the return operation of the shift lens 104 has finished. Therefore, through image 405 is not affected by the return operation of the shift lens 104, unlike through image 404 which was displayed on the display unit 140 immediately before.

[0043] Thus, if the through image is continuously displayed on the display unit 140 regardless of the amount of drive during the return operation of the shift lens 104, a through image with a large amount of subject blur will be displayed on the display unit 140 when the amount of drive during the return operation is large.

[0044] Figure 6 shows the relationship between the return operation of the shift lens 104 and the changes in the content of the image displayed on the display unit 140 over time. In the example shown in Figure 6, it is assumed that the imaging device 1 is instructed to take multiple shots in panning assist mode and that panning assist processing is being performed. Furthermore, the operation of the shift lens 104 shown in Figure 6 is assumed to be the same as the operation of the shift lens 104 shown in Figure 4(B). Also, the timings of time points T31 to T39 shown in Figure 6 are assumed to have the same relationship as the timings of time points T21 to T29 shown in Figure 5. Therefore, the through images 401, 402, 404, and 405 shown in Figure 6 are the same images as the through images 401, 402, 404, and 405 shown in Figure 5, respectively.

[0045] Furthermore, the processing at each of the time points T31 to T34 shown in Figure 6 is the same as the processing at each of the time points T21 to T24 shown in Figure 5. That is, in the example shown in Figure 6, at time point T31 the through image 401 is displayed on the display unit 140, at time point T32 the display content of the display unit 140 is updated to the through image 402, and at time point T33 exposure and the driving of the shift lens 104 are started. Also, at time point T34 exposure is completed, pixel reading from the image sensor 132 is started, and the return operation of the shift lens 104 is started.

[0046] Next, at time T35, which is later than time T34, the timing for updating the through image frame is reached. Meanwhile, the return operation of the shift lens 104 is still ongoing. In this case, the signal processing circuit 134 updates the image to be displayed on the display unit 140 to the alternative image 411. That is, the image generated by imaging during the return operation of the shift lens 104 is not displayed on the display unit 140. Furthermore, if an alternative image is displayed on the display unit 140 during the panning assist process, the image sensor 132 may restrict imaging during the return operation of the shift lens 104. Also, if an alternative image is displayed on the display unit 140 during the panning assist process, the image sensor 132 may continue to capture images even during the return operation, while the signal processing circuit 134 may restrict the display of the through image generated by the imaging during the return operation on the display unit 140.

[0047] Next, at time T36, which is later than time T35, the reading of pixels from the image sensor 132 is completed. Next, at time T37, which is later than time T36, the return operation of the shift lens 104 is completed. Next, at time T38, which is later than time T37, the timing for updating the through image frame is reached. Also, at this time, as described above, the return operation of the shift lens 104 has finished. In this case, the signal processing circuit 134 updates the image displayed on the display unit 140 from the alternative image 411 to the latest through image 404 obtained from a new image capture. This through image 404 is an image captured after the return operation of the shift lens 104 has finished. In other words, the through image 404 is an image captured when the shift lens 104 is in its initial position P0 (see Figure 4(B)), just like the through image 402. Next, as a new image is captured, at time T39, which is later than time T38, the signal processing circuit 134 updates the image displayed on the display unit 140 to the latest through image 405.

[0048] In this way, when an alternative image is displayed on the display unit 140 instead of the through image captured during the return operation of the shift lens 104, the display of a through image with significant subject blur on the display unit 140 is suppressed.

[0049] Figure 7 shows the relationship between the return operation of the shift lens 104 and the changes in the content of the image displayed on the display unit 140 over time. In the example shown in Figure 7, it is assumed that the imaging device 1 is instructed to take multiple shots in panning assist mode and that panning assist processing is being performed. Furthermore, the operation of the shift lens 104 shown in Figure 7 is assumed to be the same as the operation of the shift lens 104 shown in Figure 4(A).

[0050] Furthermore, the processing at each of the time points T41 to T46 shown in Figure 7 is the same as the processing at each of the time points T31 to T36 shown in Figure 6. That is, in the example shown in Figure 7, at time point T41 the through image 401 is displayed on the display unit 140, at time point T42 the display content of the display unit 140 is updated to the through image 402, and at time point T43 exposure and the driving of the shift lens 104 are started. At time point T44 exposure is finished, pixel reading from the image sensor 132 is started, and the return operation of the shift lens 104 is started. At time point T45 the display of the alternative image 411 on the display unit 140 is started, and at time point T46 the pixel reading from the image sensor 132 is finished. Furthermore, the timings T41 to T46 shown in Figure 7 are assumed to have the same relationship as the timings T31 to T36 shown in Figure 6. Therefore, the through image 401, through image 402, and alternative image 411 shown in Figure 7 are identical to the through image 401, through image 402, and alternative image 411 shown in Figure 6, respectively.

[0051] Next, at time T47, which is later than time T46, the timing for updating the through image frame is reached. Meanwhile, the return operation of the shift lens 104 is still ongoing. In this case, the signal processing circuit 134 causes the display unit 140 to display the alternative image 413. Thus, if the shift lens 104 is in the return operation for a period spanning two frames of the through image, the alternative image is displayed on the display unit 140 for these two frames. Note that the alternative image 413 may be the same image as the alternative image 411, or it may be a different image from the alternative image 411, as long as it is a different image from the image generated by imaging during the return operation of the shift lens 104. Next, at time T48, which is later than time T47, the return operation of the shift lens 104 is completed.

[0052] Next, at time T49, which is later than time T48, the timing for updating the through-image frame is reached. Also, at this time, as described above, the return operation of the shift lens 104 is completed. In this case, the signal processing circuit 134 updates the image displayed on the display unit 140 from the substitute image 413 to the latest through-image 405 obtained from the new image capture. Note that the period from time T41 to time T49 shown in Figure 7 is the same period from time T31 to time T39 shown in Figure 6. Therefore, the through-image 405 shown in Figure 7 is the same image as the through-image 405 shown in Figure 6.

[0053] Thus, in this embodiment, the time for which an alternative image is displayed on the display unit 140 is determined according to the amount of drive during the return operation of the shift lens 104. In this case, even if the return operation of the shift lens 104 takes a long time, the display of a through image with a large amount of subject blur on the display unit 140 is suppressed. Furthermore, the amount of movement of the shift lens 104 until the exposure of the image sensor 132 is completed can also be considered as the amount of motion blur correction. Therefore, if the amount of motion blur correction by the user during panning is small, such as for a user who is accustomed to panning, the display unit 140 will continue to display the through image without displaying an alternative image. Thus, it is possible to switch whether or not an alternative image is displayed depending on the user's level of skill in panning.

[0054] Figure 8 is a flowchart showing the pre-processing steps (see step 315 in Figure 3). The camera microcontroller 141 determines whether the shooting instruction given in the panning assist process (see step 303 in Figure 3) is still in effect (S801). The camera microcontroller 141 makes the determination in step 801 in response to the user's operation on the control unit 136. If no shooting instruction is given (No in S801), the panning assist process repeats from step 302. In other words, if no shooting instruction is given, continuous shooting is not performed, and the latest angular velocity information is generated each time the through image frame is updated until the user gives another shooting instruction (see steps 302 and 303 in Figure 3).

[0055] Furthermore, if the shooting instruction continues (Yes in S801), the camera microcontroller 141 determines whether the drive amount of the shift lens 104 identified in step 306 of the panning assist processing is less than the standby threshold (S802). The standby threshold is a predetermined value used by the camera microcontroller 141 to determine whether or not to set a waiting time for shooting before the next shooting starts. The standby threshold may be the same value as the change threshold, or it may be a different value from the change threshold. When the shift lens 104 returns to its initial position P0 (see Figure 4) after the completion of shooting, if the next shooting session starts before the shift lens 104 returns to its initial position P0 (see Figure 4) due to the return operation, the drive range of the shift lens 104 in the next shooting session may be limited. Therefore, in this embodiment, a waiting threshold is set so that the next shooting session starts after the return operation of the shift lens 104 is completed. In other words, if the amount of drive required for the return operation of the shift lens 104 is large, and the time required for the return operation of the shift lens 104 is long, the waiting threshold is set so that a waiting time is provided if the next shooting session starts during the return operation.

[0056] If the drive amount of the shift lens 104 identified in step 306 of the panning assist processing is less than the standby threshold (Yes in S802), the process proceeds to the next step. The angular velocity calculation unit 152 of the camera microcontroller 141 calculates the angular velocity of the subject and generates angular velocity information (S803). The angular velocity calculation unit 152 transmits the generated angular velocity information to the lens microcontroller 112.

[0057] Furthermore, if the drive amount of the shift lens 104 identified in step 306 of the panning assist processing is greater than or equal to the standby threshold (No in S802), the camera microcontroller 141 determines the standby time for the next shot (S804). The camera microcontroller 141 determines the waiting time for the next shot based on the drive amount identified in step 306 of the panning assist processing, in other words, the drive amount in the return operation of the shift lens 104. More specifically, the camera microcontroller 141 determines the waiting time for the next shot as the time until the return operation of the shift lens 104 is completed. Therefore, the larger the drive amount identified in step 306 of the panning assist processing, the longer the waiting time determined. The waiting time is stored in the camera microcontroller 141 in advance as a value corresponding to the drive amount in the return operation of the shift lens 104. In addition, the camera microcontroller 141 may store table information in which the waiting time is pre-associated for each drive amount of the shift lens 104. In addition, the waiting time may be pre-associated for each range of the drive amount of the shift lens 104. Furthermore, the number of stages of the waiting time associated with the drive amount or range of the drive amount of the shift lens 104 may be any number. In addition, the drive amount or range of the drive amount of the shift lens 104 and the associated waiting time may be defined for each type of lens device 100. More specifically, the amount of drive of the shift lens 104 or the range of the amount of drive, and the associated waiting time may be determined according to the configuration of the optical system in the lens device 100. The waiting time may also be determined as the number of frames of the through image.

[0058] The angular velocity calculation unit 152 of the camera microcontroller 141 calculates the angular velocity of the subject and generates angular velocity information (S805). The angular velocity calculation unit 152 transmits the generated angular velocity information to the lens microcontroller 112. The camera microcontroller 141 determines whether the waiting time for the next shot has elapsed (S806). If the waiting time has not elapsed (No in S806), the process from step 805 is repeated. In other words, the latest angular velocity information is generated each time a frame of the through image is updated until the waiting time has elapsed. Furthermore, if the waiting time elapses (Yes in S806), the process from step 304 of the panning assist process (see Figure 3) is repeated, and the next shot is taken. In addition, the next shot is taken without limiting the drive range of the shift lens 104 for correcting subject blur.

[0059] In this way, once the pre-processing for shooting is performed, the waiting time for the next shot is set according to the amount of movement of the shift lens 104 after the exposure of the image sensor 132 is completed, and after the waiting time has elapsed, the panning assist process is performed. In other words, the calculation of the amount of movement of the shift lens 104 necessary for correcting subject blur in the next shot is started at a timing corresponding to the amount of movement of the shift lens 104 after the exposure of the image sensor 132 is completed (see step 304 in Figure 3).

[0060] Figure 9 is a flowchart illustrating the pre-processing steps for a modified image. Pre-processing for imaging is not limited to what is shown in Figure 8. The camera microcontroller 141 determines whether the shooting instruction given in the panning assist process (see step 303 in Figure 3) is still in effect (S901). If no shooting instructions are given (No in S901), the panning assist process will repeat from step 302.

[0061] Furthermore, if the shooting instruction continues (Yes in S901), the camera microcontroller 141 determines whether the drive amount of the shift lens 104 identified in step 306 of the panning assist processing is less than the limit threshold (S902). The limit threshold is a predetermined value used by the camera microcontroller 141 to determine whether or not to limit the calculation of the angular velocity of the subject. The limit threshold may be the same value as the change threshold, or it may be a different value from the change threshold. Alternatively, the change threshold and the standby threshold may be the same value, while the limit threshold is different from both the change threshold and the standby threshold. When motion vectors are detected from images captured during the return operation of the shift lens 104, and the angular velocity of the subject is calculated from the vector information indicating the detected motion vectors, the calculation of the angular velocity may be affected by the change in the field of view due to the return operation. Therefore, in this embodiment, a limiting threshold is set so as to restrict the calculation of the angular velocity of the subject that is greatly affected by the change in the field of view due to the return operation. In other words, if motion vectors are detected from images captured when the change in the field of view is greatly affected during the return operation, a limiting threshold is set so as to provide a waiting time for the calculation of the angular velocity of the subject.

[0062] If the amount of drive of the shift lens 104 identified in step 306 of the panning assist processing is greater than or equal to a limit threshold (No in S902), the camera microcontroller 141 determines the waiting time for calculating the angular velocity of the subject, etc. (S903). The camera microcontroller 141 determines the calculation waiting time according to the amount of drive in the return operation of the shift lens 104, in other words, the amount of drive of the shift lens 104 identified in step 306 of the panning assist processing. More specifically, the camera microcontroller 141 determines the calculation waiting time as the time until the effect of the change in angle of view accompanying the return operation on the calculation of the subject's angular velocity, etc., becomes small. Therefore, the larger the amount of drive identified in step 306 of the panning assist processing, the longer the waiting time determined. The calculation waiting time is stored in the camera microcontroller 141 in advance as a value corresponding to the amount of drive in the return operation of the shift lens 104. In addition, the camera microcontroller 141 may also store table information in which the calculation waiting time is pre-associated for each amount of drive of the shift lens 104. In addition, the calculation waiting time may be pre-associated for each range of drive of the shift lens 104. Furthermore, the number of stages of the calculation waiting time associated with the drive amount or range of drive of the shift lens 104 may be any number. Furthermore, the drive amount or range of the drive amount of the shift lens 104 and the associated calculation waiting time may be determined for each type of lens device 100. More specifically, the drive amount or range of the drive amount of the shift lens 104 and the associated calculation waiting time may be determined according to the configuration of the optical system in the lens device 100. The calculation waiting time may also be determined as the number of frames of the through image.

[0063] The camera microcontroller 141 determines whether the waiting time for calculating the angular velocity of the subject has elapsed (S904). As long as the negative result continues (No in S904), the process in step 904 is repeated. Furthermore, if the waiting time has elapsed (Yes in S904), or if the drive amount of the shift lens 104 identified in step 306 of the panning assist processing is less than the limit threshold (Yes in S902), the process proceeds to the next step. The angular velocity calculation unit 152 of the camera microcontroller 141 calculates the angular velocity of the subject based on the latest vector information and generates angular velocity information (S905). The angular velocity calculation unit 152 transmits the generated angular velocity information to the lens microcontroller 112.

[0064] Furthermore, after the angular velocity information is generated, the process from step 304 of the panning assist process (see Figure 3) is repeated, and the next shot is taken. In this modified form, when pre-processing is performed, the calculation of the subject's angular velocity and other parameters based on the motion vector detected from the image captured when the angle of view changes significantly due to the return movement of the shift lens 104 is suppressed. Furthermore, in the case of pre-processing as a modified example, the calculation of the angular velocity of the subject is started at a timing corresponding to the amount of drive of the shift lens 104 after the exposure of the image sensor 132 is completed.

[0065] In this embodiment, the camera microcontroller 141 determines whether or not to display an alternative image and, if so, the display time of the alternative image, based on the amount of drive of the shift lens 104 during exposure of the image sensor 132, but is not limited to this. The camera microcontroller 141 may, for example, determine whether or not to display an alternative image and, if so, the display time of the alternative image, based on the amount of motion blur correction. In this case, the amount of motion blur correction may be determined from the amount of movement of the shift lens 104 during exposure of the image sensor 132. Alternatively, the amount of motion blur correction may be determined from the latest angular velocity information. Furthermore, the amount of motion blur correction may be determined by the camera microcontroller 141 or by the lens microcontroller 112.

[0066] Furthermore, although this embodiment describes the shift lens 104 as the drive target used for correcting subject blur, it is not limited to this. The drive target may be, for example, at least one of the shift lens 104 and the image sensor 132. Furthermore, the driving means may be at least one of a gimbal mechanism and a pan-tilt mechanism that changes the optical axis direction of the imaging device 1 itself.

[0067] Furthermore, while the above example explained that the alternative image is an image that is entirely black, it is not limited to this. For example, the signal processing circuit 134 may continue to display the frame of the through image (see through image 402 in Figures 6 and 7) that was displayed on the display unit 140 until just before the content displayed on the display unit 140 switched to the alternative image, as the alternative image. In this case, while the through image 402 is displayed as the alternative image, not only the moving subject but also the background is still in the image displayed on the display unit 140. In other words, because a still image is displayed on the display unit 140, the user recognizes that the image displayed on the display unit 140 is different from the live image. The alternative image may also be an image that is entirely black, or an image that is different from both the through image and the live image (for example, an image that is entirely a color other than black).

[0068] Furthermore, although this embodiment explains that angular velocity information indicates the angular velocity of the subject, it is not limited to this. The angular velocity information may also be information indicating the angular acceleration of the subject. In other words, the angular velocity calculation unit 152 of the camera microcontroller 141 may calculate the angular acceleration of the subject based on the latest vector information and generate angular velocity information indicating the result of the calculation.

[0069] Furthermore, in this embodiment, the camera microcontroller 141 identifies the amount of drive of the drive unit after exposure is completed during shooting by the image sensor 132, and causes the drive unit to return to its original position based on the identified result, but the embodiment is not limited to this. For example, instead of determining the amount of drive of the drive unit after exposure is complete in shooting with the image sensor 132, the camera microcontroller 141 may determine the position of the drive unit at the end of exposure in shooting with the image sensor 132. In this case, the camera microcontroller 141 may determine the position of the drive unit at the end of exposure in shooting with the image sensor 132 from the drive information received from the lens microcontroller 112. Alternatively, the camera microcontroller 141 may determine (estimate) the position of the drive unit at the end of exposure in shooting with the image sensor 132 based on angular velocity information, the exposure time of the image sensor 132, and information indicating the angular velocity previously obtained from the angular velocity output unit 222. Even in such cases, the camera microcontroller 141 can cause the drive unit to return to its original position based on the specific result. Furthermore, the camera microcontroller 141 may determine whether or not an alternative image is displayed instead of the through image, and determine the display time of the alternative image, based on the position of the drive unit at the end of exposure in shooting with the image sensor 132.

[0070] Furthermore, for example, if a specific lens device 100 is used in the imaging device 1, the display of the through image on the display unit 140 may continue even if the amount of drive of the shift lens 104 during exposure of the image sensor 132 is greater than or equal to the change threshold. Even in this case, the image generated by imaging during the return operation of the shift lens 104 may not be used for the detection of the motion vector of the subject by the motion vector detection unit 145.

[0071] Furthermore, when the imaging device 1 takes multiple shots in panning assist mode, the results of the panning assist processing for the first shot may be reflected in the panning assist processing for the second and subsequent shots (see Figure 3). More specifically, the presence or absence of display of an alternative image and the display time of the alternative image, etc., determined in the panning assist processing for the first shot, may also be applied to the panning assist processing for the second and subsequent shots. In this case, there will be no variation in the presence or absence of display of an alternative image and the display time of the alternative image for each shot. Furthermore, for example, the longer the display time of the alternative image in the panning assist process for the first shot, the longer the time the through image is displayed on the display unit 140 before the next shot. In other words, the more frames in which the alternative image is displayed in the panning assist process for the first shot, the more frames of the through image are displayed on the display unit 140 before the next shot. In this case, while the number of shots per unit of time decreases, the user can check the through image for a longer period of time before taking a shot, making it less likely for subject blur to occur even for users who are not accustomed to panning.

[0072] Furthermore, the change threshold, standby threshold, and limit threshold mentioned above may be set according to the user's operation on the operation unit 136. In other words, the change threshold, standby threshold, and limit threshold may be set to values ​​that are appropriate for the user.

[0073] As described above, in this embodiment, the signal processing circuit 134 changes the image to be displayed on the display unit 140 after the exposure for obtaining the first image is completed, according to the amount of movement of the object driven by the driving means from the reference position during exposure for obtaining the first image using the image sensor 132. In this case, the display of images with significant subject positional shift after the exposure is complete during shooting is suppressed.

[0074] Furthermore, in this embodiment, if the amount of movement of the driven object from the reference position is less than a threshold, the signal processing circuit 134 displays the through image obtained using the image sensor 132 on the display unit 140 after the exposure for obtaining the first image is completed. The threshold can be a change threshold. In this case, even if the subject's positional shift is small or nonexistent after exposure, the disappearance of the through-image display is suppressed.

[0075] Furthermore, in this embodiment, if the amount of movement of the driven object from the reference position is greater than or equal to a threshold, the signal processing circuit 134 displays a predetermined image on the display unit 140 that is different from the through image obtained using the image sensor 132 after the exposure for obtaining the first image is completed. In this case, even if the amount of movement of the driven object from the reference position by the driving means during exposure is large, the display of an image with a large displacement of the subject after exposure is suppressed.

[0076] Furthermore, in this embodiment, if the amount of movement of the driven object from the reference position is greater than or equal to a threshold, the signal processing circuit 134 changes the number of frames or time for displaying a predetermined image according to the amount of movement. In this case, regardless of the amount of movement of the driven object, the display of images with a large displacement of the subject's position after exposure is suppressed.

[0077] Furthermore, in this embodiment, the predetermined image is an image of a predetermined color. In this case, it becomes easier for the user to recognize that the displayed image has been changed from a background image to a predetermined image.

[0078] Furthermore, in this embodiment, the predetermined image is a through image acquired before obtaining the first image. In this case, the displayed image is less likely to cause discomfort to the user, while making it easier for the user to recognize that the displayed image has been changed from a background image to a predetermined image.

[0079] Furthermore, in this embodiment, the driving means moves the object to be driven in a direction approaching the reference position after the exposure for obtaining the first image is completed. In this case, the restriction on the movement of the driven object during the exposure following the exposure for obtaining the first image is suppressed.

[0080] Furthermore, in this embodiment, if the amount of movement is greater than or equal to a threshold, the signal processing circuit 134 displays a predetermined image on the display unit 140 instead of the through image obtained using the image sensor 132 while moving the target object after the exposure for obtaining the first image is completed. In this case, the display of images with significant subject positional shifts due to the movement of the driven object after exposure is suppressed.

[0081] Furthermore, in this embodiment, the angle of view control means, such as the lens microcontroller 112 and the camera microcontroller 141, move the object to be driven by the driving means in a direction that suppresses changes in the position of the subject within the angle of view. In this case, the displacement of the subject in the first image is suppressed.

[0082] Furthermore, in this embodiment, the drive target is at least one of the lens included in the imaging optical system that guides subject light to the image sensor 132 and the image sensor 132. Examples of the lens included in the imaging optical system that guides subject light to the image sensor 132 include vibration-damping lenses such as the shift lens 104. In this case, the lens or image sensor 132 helps to suppress the display of images with significant subject positional shift after exposure is complete during shooting.

[0083] Furthermore, in this embodiment, the driving means is at least one of a gimbal mechanism and a pan-tilt mechanism. In this case, the gimbal mechanism lens or pan-tilt mechanism helps to suppress the display of images with significant subject positional shift after exposure is complete during shooting.

[0084] Furthermore, in this embodiment, the driving means corrects the blur of the subject image by changing the angle of view in accordance with the driving. The camera microcontroller 141 then estimates the position of the driving means at the end of exposure by the image sensor 132 or the amount of driving after exposure is completed, based on information regarding the amount of driving of the driving means during exposure or information regarding the amount of correction of the blur of the subject image by the driving means. In this case, even if the position of the drive means at the end of exposure and the amount of drive after exposure are not detected, the display content of the display unit 140 after exposure is changed from the live image according to the amount of drive of the drive means after exposure.

[0085] In this embodiment, the lens device 100 has a drive mechanism and transmits drive information regarding the drive mechanism until exposure by the image sensor 132 is completed to the camera microcontroller 141. The camera microcontroller 141 then identifies the position of the drive mechanism at the end of exposure by the image sensor 132 or the amount of drive of the drive mechanism after exposure is completed from the drive information. In this case, the processing load on the camera microcontroller 141 to determine the position of the drive means at the end of exposure by the image sensor 132 or the amount of drive of the drive means after the end of exposure is reduced.

[0086] Furthermore, in this embodiment, it was explained that the time for which the through image is displayed on the display unit 140 is determined according to the amount of drive of the drive means at the end of exposure. That is, there is a first threshold and a second threshold that is greater than the first threshold. The signal processing circuit 134 changes the display content of the display unit 140 from the through image for a first time if the amount of drive of the drive means at the end of exposure is greater than or equal to the first threshold and less than the second threshold. The signal processing circuit 134 also changes the display content of the display unit 140 from the through image for a second time, which is longer than the first time, if the amount of drive of the drive means at the end of exposure is greater than or equal to the second threshold. In this case, even if the drive time of the drive means is long after the exposure is completed, the display unit 140 will not display a through image with a large displacement of the subject's position after the exposure is completed during shooting.

[0087] In this embodiment, the calculation means of the imaging device 1 calculates an index related to the blur of the subject image. Examples of the index related to the blur of the subject image include the angular velocity and angular acceleration of the subject, the amount of drive of the drive means during exposure, and the amount of correction for the blur of the subject image. Examples of the calculation means include the lens microcontroller 112 and the camera microcontroller 141. The camera microcontroller 141 identifies the position of the drive means at the end of exposure by the image sensor 132 or the amount of drive after the end of exposure based on the calculated index. The calculation means then starts calculating the index at a timing corresponding to at least one of the type of lens device 100 and the amount of drive of the drive means after the end of exposure. In this case, the calculation of an index affected by the operation of the drive mechanism after exposure is suppressed.

[0088] Furthermore, the signal processing circuit 134 may determine whether to change the display content of the display unit 140 from the through image to an alternative image, and the display time of the alternative image, depending on the frame rate of the through image. For example, if the frame rate of the through image is 60fps, the signal processing circuit 134 may display an alternative image for one frame in place of the through image during the return operation of the drive means after exposure is complete, depending on the amount of drive of the drive means during exposure in the shooting. On the other hand, if the frame rate of the through image is 120fps, the signal processing circuit 134 may display an alternative image for two frames in place of the through image during the return operation of the drive means after exposure is complete. Also, if the frame rate of the through image is 30fps and no return operation is performed at the timing of frame updates, the signal processing circuit 134 may continue to display the through image without displaying an alternative image. In this way, the signal processing circuit 134 may change the display content of the display unit 140 after exposure is complete from the through image, depending on the frame rate of the through image. In this case, regardless of the frame rate of the through-image, the display unit 140 will not display through-images with a large displacement of the subject's position after the exposure is completed during shooting.

[0089] Furthermore, the image sensor 132 may change the frame rate according to the amount of drive of the drive means after the exposure is completed during shooting. For example, if an alternative image is displayed for a first time according to the amount of drive of the drive means after the exposure is completed, the image sensor 132 sets the number of shots per unit time to the first number. Then, if the alternative image is displayed for a second time, which is shorter than the first time, according to the amount of drive of the drive means after the exposure is completed, the image sensor 132 sets the number of shots per unit time to the second number, which is greater than the first number. Furthermore, if the alternative image is not displayed and the through image continues to be displayed according to the amount of drive of the drive means after the exposure is completed, the image sensor 132 sets the number of shots per unit time to the third number, which is greater than the second number. In this way, the image sensor 132 may take a number of shots per unit time according to the amount of drive of the drive means after the exposure is completed during shooting. In this case, shooting is performed at a frame rate that matches the user's skill level in panning.

[0090] Furthermore, the present invention can also be realized by supplying a program that implements one or more of the functions of this embodiment to the imaging device 1 via a network or storage medium, and by having one or more processors in the computer of the imaging device 1 read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0091] This embodiment includes the following configuration. (Composition 1) Image sensor and An angle of view control means that controls the angle of view of an image obtained using the image sensor by moving the object to be driven by the driving means, A display means for displaying an image obtained using the aforementioned image sensor, A display control means that changes the image to be displayed on the display means after the exposure for obtaining the first image is completed, according to the amount of movement of the driven object from a reference position by the driving means during exposure for obtaining the first image using the image sensor, An imaging device characterized by being equipped with the following features. (Configuration 2) The imaging apparatus according to configuration 1, characterized in that, when the amount of movement is less than a threshold, the display control means displays the through image obtained using the image sensor after the exposure for obtaining the first image on the display means. (Composition 3) The imaging apparatus according to configuration 2, characterized in that, when the amount of movement is greater than or equal to the threshold, the display control means causes a predetermined image different from the through image obtained using the image sensor to be displayed on the display means after the exposure for obtaining the first image is completed. (Composition 4) The imaging apparatus according to configuration 3, characterized in that the display control means changes the number of frames or time for displaying the predetermined image according to the amount of movement when the amount of movement is greater than or equal to the threshold. (Composition 5) The imaging apparatus according to configuration 3 or 4, characterized in that the predetermined image is an image of a predetermined color. (Composition 6) The imaging apparatus according to configuration 3 or 4, characterized in that the predetermined image is a through image acquired before obtaining the first image. (Composition 7) The imaging apparatus according to any one of configurations 1 to 6, characterized in that the driving means moves the object to be driven in a direction approaching the reference position after the exposure for obtaining the first image is completed. (Composition 8) The imaging apparatus according to configuration 7, characterized in that, when the amount of movement is greater than or equal to the threshold, the display control means moves the driven object after the exposure for obtaining the first image and displays the predetermined image on the display means instead of the through image obtained using the image sensor. (Composition 9) The imaging apparatus according to any one of configurations 1 to 8, characterized in that the angle of view control means moves the driven object by the driving means in a direction that suppresses changes in the position of the subject within the angle of view. (Composition 10) The imaging apparatus according to any one of configurations 1 to 9, characterized in that the target of the drive is at least one of a lens included in the imaging optical system that guides subject light to the image sensor and the image sensor. (Composition 11) The imaging apparatus according to any one of configurations 1 to 9, characterized in that the driving means is at least one of a gimbal mechanism and a pan-tilt mechanism. (Composition 12) A control method for an imaging device comprising an image sensor and a display means for displaying an image obtained using the image sensor, A step of moving the object to be driven by the driving means and controlling the field of view of the image obtained using the image sensor, A step of changing the image to be displayed on the display means after the exposure for obtaining the first image is completed, according to the amount of movement of the driven object from the reference position by the driving means during exposure for obtaining the first image using the image sensor, A control method characterized by having the following features. (Composition 13) A program for causing a computer to function as an imaging device according to any one of claims 1 to 11.

[0092] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to the above embodiments, and various modifications are possible in accordance with the spirit of the present invention, and these modifications are not excluded from the scope of the present invention. [Explanation of Symbols]

[0093] 1...Imaging device, 100...Lens device, 112...Lens microcontroller, 130...Camera body, 134...Signal processing circuit, 141...Camera microcontroller

Claims

1. Image sensor and An angle of view control means that controls the angle of view of an image obtained using the image sensor by moving the object to be driven by the driving means, A display means for displaying an image obtained using the aforementioned image sensor, A display control means that changes the image to be displayed on the display means after the exposure for obtaining the first image is completed, according to the amount of movement of the driven object from a reference position by the driving means during exposure for obtaining the first image using the image sensor, An imaging device characterized by being equipped with the following features.

2. The imaging apparatus according to claim 1, characterized in that, when the amount of movement is less than a threshold, the display control means causes the through image obtained using the image sensor to be displayed on the display means after the exposure for obtaining the first image is completed.

3. The imaging apparatus according to claim 2, characterized in that, when the amount of movement is greater than or equal to the threshold, the display control means causes a predetermined image different from the through image obtained using the image sensor to be displayed on the display means after the exposure for obtaining the first image is completed.

4. The imaging apparatus according to claim 3, characterized in that the display control means changes the number of frames or time for displaying the predetermined image according to the amount of movement when the amount of movement is greater than or equal to the threshold.

5. The imaging device according to claim 3, characterized in that the predetermined image is an image of a predetermined color.

6. The imaging apparatus according to claim 3, characterized in that the predetermined image is a through image acquired before obtaining the first image.

7. The imaging apparatus according to claim 3, characterized in that the driving means moves the object to be driven in a direction approaching the reference position after the exposure for obtaining the first image is completed.

8. The imaging apparatus according to claim 7, characterized in that, when the amount of movement is greater than or equal to the threshold, the display control means moves the driven object after the exposure for obtaining the first image and displays the predetermined image on the display means instead of the through image obtained using the image sensor.

9. The imaging apparatus according to claim 1, characterized in that the angle of view control means moves the driven object by the driving means in a direction that suppresses changes in the position of the subject within the angle of view.

10. The imaging apparatus according to claim 1, characterized in that the target of the drive is at least one of a lens included in the imaging optical system that guides subject light to the image sensor and the image sensor.

11. The imaging apparatus according to claim 1, characterized in that the driving means is at least one of a gimbal mechanism and a pan-tilt mechanism.

12. A control method for an imaging device comprising an image sensor and a display means for displaying an image obtained using the image sensor, A step of moving the object to be driven by the driving means and controlling the field of view of the image obtained using the image sensor, A step of changing the image to be displayed on the display means after the exposure for obtaining the first image is completed, according to the amount of movement of the driven object from the reference position by the driving means during exposure for obtaining the first image using the image sensor, A control method characterized by having the following features.

13. A program for causing a computer to function as an imaging device according to any one of claims 1 to 11.

Images