Imaging device and control method thereof

Abstract

To provide an imaging device which can correct composition deviation.SOLUTION: An imaging device comprises: an instruction unit which instructs first and second compositions; an image deviation detection unit 18; and an image deviation correction unit 14. The first composition is, for example, an ideal composition decided by a user. The second composition is a composition adjusted to the approximate position or attitude with respect to the first composition when a composition is deviated from the first composition once decided by the user. The image deviation detection unit 18 compares data of a first image stored at the time of instruction of the first composition with data of a second image stored at the time of instruction of the second composition and detects a motion vector of the second image with respect to the first image. The image deviation correction unit 14 performs image deviation correction by moving an image pick-up device 6 on the basis of the detected motion vector.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for correcting image displacement with respect to compositional displacement.

Background Art

[0002] When a photographer takes a picture of a subject as intended, if a deviation occurs from the composition that has been determined once, the photographer may be forced to adjust the composition again. For example, when trying to take a picture with an ideal composition determined by the photographer in a situation where a tripod cannot be used, there is a possibility of compositional deviation. The ideal composition is the composition in the background determined in advance by the photographer, and the compositional deviation corresponds to a composition that is not the ideal composition. For example, assume that after the photographer adjusts to the ideal composition, the subject is visually confirmed later, or the photographer waits for a subject outside the composition to enter the composition before taking the picture. In this case, especially in hand-held shooting or shooting using a monopod, displacement of the imaging device occurs, increasing the possibility of compositional deviation.

[0003] Patent Document 1 discloses a technique for calculating the correction amount of compositional deviation from the values of a gyro sensor or an acceleration sensor and correcting the composition by an image displacement correction unit.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the prior art disclosed in Patent Document 1, the amount of composition shift correction is calculated by integrating the outputs of a gyro sensor and an accelerometer. However, if there is a long time between the time the photographer sets the ideal composition and the time the actual shooting takes place, the error in the amount of composition shift correction becomes large. Also, if the photographer moves the imaging device significantly, that movement may exceed the measurement range of the gyro sensor and accelerometer. If the error in the amount of composition shift correction becomes large, a composition shift correction residual will occur, requiring the photographer to readjust the composition. In the meantime, there is a possibility of losing the timing for shooting. The objective of the present invention is to provide an imaging device that enables correction of compositional misalignment. [Means for solving the problem]

[0006] The apparatus of the embodiment of the present invention is an imaging apparatus that performs image shift correction for compositional shift, A first instruction means for giving instructions for a first composition, a second instruction means for giving instructions for a second composition, and the The data of the first image is stored according to the instruction for the first composition, and this occurs after the instruction for the first composition. The aforementioned A storage means for storing data of a second image according to instructions for a second composition; a first detection means for detecting the motion vector of the second image relative to the first image; and a background vector acquisition means for acquiring a background vector representing the movement of the background based on the motion vector. When the second composition is instructed by the second instruction means, the first image and the second image are compared, and if a shift occurs in the background of the second image relative to the background of the first image, the system includes a correction means that controls the movement of the optical elements of the imaging optical system or the image sensor, or corrects the image shift by image processing. The correction means calculates the amount of movement of the optical elements of the imaging optical system or the image sensor or the amount of correction by image processing using the background vector, and corrects the image shift of the second image relative to the first image. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an imaging device that enables correction of compositional misalignment. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows the configuration of an imaging device according to an embodiment of the present invention. [Figure 2] This is a flowchart illustrating the process in the first embodiment. [Figure 3] This is a flowchart illustrating the process in the second embodiment. [Figure 4] This is a flowchart illustrating the process in the third embodiment. [Figure 5] This is a flowchart explaining the process that follows Figure 4. [Figure 6] This is a flowchart illustrating the process in the fourth embodiment. [Modes for carrying out the invention]

[0009] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this embodiment, a digital camera is shown as an example of an imaging device, but the invention can also be applied to information processing devices such as camera-equipped mobile phones and their variants, smartphones, tablet terminals, and personal computers (PCs) with cameras.

[0010] Figure 1(A) is a central cross-sectional view of the imaging device according to this embodiment, and Figure 1(B) is a block diagram showing the electrical configuration of the imaging device. The imaging system is composed of a camera body 1, which is the main body of the imaging device, and a photographic lens 2 attached to the camera body 1. For the sake of explanation, as shown in Figure 1(A), the X, Y, and Z axes, which are mutually orthogonal to each other with respect to the imaging device, are defined. The Z axis is an axis parallel to the optical axis 4 of the photographic lens 2 and is an axis approximately orthogonal to the light-receiving surface (imaging surface) of the image sensor 6. The X axis is an axis perpendicular to the Z axis in the horizontal plane when the Z axis is parallel to the horizontal direction. The Y axis is an axis parallel to the vertical direction when the Z axis is parallel to the horizontal direction.

[0011] The photographic lens 2 is a lens device equipped with a lens barrel and is detachable from the camera body (hereinafter referred to as the main body) 1. When the main body 1 and the photographic lens 2 are mechanically coupled, the main body 1 and the photographic lens 2 are electrically connected at the electrical contact 11. It should be noted that the present invention is not limited to interchangeable lens systems, but is also applicable to embodiments in which the photographic lens 2 and the main body 1 are integrally configured (non-detachable).

[0012] The photographic lens 2 has an imaging optical system 3 consisting of multiple lenses. The lens drive unit 13 drives the various lenses that make up the imaging optical system 3, such as the focus lens, zoom lens, and image shift correction lens, as well as the aperture. The lens system control unit (hereinafter referred to as the lens control unit) 12 controls the photographic lens 2, for example, it controls the operation of the lens drive unit 13.

[0013] The main unit 1 includes a camera system control unit (hereinafter referred to as the camera control unit) 5. The camera control unit 5 can communicate with the lens control unit 12 and comprehensively controls the operation of the imaging system. For example, the CPU (Central Processing Unit) in the camera control unit 5 controls the operation of each part that constitutes the imaging system by executing a program.

[0014] Light from the subject that has passed through the imaging optical system 3 is imaged onto the image sensor 6. The image sensor 6 converts the optical image into an electrical signal via photoelectric conversion and outputs it to the image processing unit 7. At this time, the image sensor 6 can obtain a focus evaluation amount corresponding to the focus detection state and an appropriate exposure amount, and the imaging optical system 3 is appropriately adjusted based on these amounts. As a result, the image sensor 6 is exposed to an appropriate amount of object light. In addition, the exposure amount to the image sensor 6 is controlled by the shutter unit 16 moving the shutter curtain. The drive control of the shutter unit 16 is performed by the shutter drive unit 17 according to commands from the camera control unit 5.

[0015] The image processing unit 7 contains an A / D converter, a white balance adjustment circuit, a gamma correction circuit, an interpolation calculation circuit, etc., and generates digital data for recording images (video). For example, the image processing unit 7 has a color interpolation processing circuit and generates a color image by applying color interpolation (demosaicing) processing to the Bayer array signal. The image processing unit 7 also compresses image data, video data, and audio data using a predetermined method.

[0016] The memory unit 8 includes a storage device that stores programs and the like executed by the camera control unit 5. A semiconductor storage device such as an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a memory card that stores various data generated by the image processing unit 7 is used.

[0017] The display unit 9 displays captured images, setting screens, and the like. The display unit 9 includes a rear display device 9a and an EVF (electronic viewfinder) 9b. The rear display device 9a displays various information such as a subject image, a captured image, and setting information of the imaging device. The rear display device 9a has a touch panel and has a function of detecting a touch operation as one of the operation detection units 10. Also, the user can confirm the subject image by looking through the EVF 9b.

[0018] The operation detection unit 10 detects operations on various operation members provided on the main body unit 1 and notifies the detection results to the camera control unit 5. The camera control unit 5 executes processing corresponding to the notification from the operation detection unit 10. For example, when the operation detection unit 10 detects that the release button is pressed, the camera control unit 5 controls a series of imaging operations such as imaging preparation processing, imaging processing, image processing, and recording processing according to the operation instruction.

[0019] The image shift correction unit 14 performs image shift correction by driving the imaging element 6 according to an instruction from the camera control unit 5. The shake detection unit 15 detects the shake of the main body unit 1 and outputs a detection signal to the camera control unit 5. For example, the shake detection unit 15 includes a shake detection sensor such as a vibration gyro and detects the rotational shake of the main body unit 1 including rotations around the X-axis, Y-axis, and Z-axis, and transmits a signal of the shake detection result to the camera control unit 5.

[0020] Based on the blur detection results from the blur detection unit 15, the camera control unit 5 controls the movement of the image sensor 6 using the image shift correction unit 14 so that the detected blur is canceled out. For example, the image shift correction unit 14 includes a drive mechanism that moves the image sensor 6 in a direction perpendicular to the optical axis 4 (Z axis) (X direction, Y direction). The image shift correction unit 14 can translate the image sensor 6 in a plane perpendicular to the optical axis 4 (XY plane) and rotate it around the optical axis 4 (around the Z axis).

[0021] The image shift detection unit 18 detects motion vectors by comparing two images acquired through imaging and transmits them to the camera control unit 5. Motion vectors can be calculated using known methods. The subject detection unit 19 and the motion vector separation unit 20 will be described later.

[0022] The camera control unit 5 transmits commands to the lens control unit 12 via the electrical contacts 11. The lens control unit 12 drives the lens drive unit 13 according to the commands from the camera control unit 5 and transmits the results of the drive (such as operation information of the photographic lens 2) to the camera control unit 5.

[0023] [First Embodiment] Figure 2 is a flowchart illustrating image misalignment correction in this embodiment. The following processes are implemented by the CPU of the camera control unit 5 executing a program. First, the camera control unit 5 performs an image update process (S101). An image for adjusting the composition is displayed on the display unit 9 (rear display device 9a or EVF 9b). The user can adjust the composition while viewing the updated display image. In the following, the first composition will be described as the ideal composition intended by the user.

[0024] Once the user has determined a first composition as the ideal composition, they perform an operation to instruct the imaging device to use the first instruction unit. The first instruction unit is, for example, a release button provided on the main unit 1. A switch linked to the release button is electrically connected to the operation detection unit 10. The release button is an operating member with two operating strokes. A first stroke operation by the user turns on the first switch (denoted as SW1), and instructions for preparing for shooting, such as distance measurement and light metering, are given. A deeper second stroke operation turns on the second switch (denoted as SW2), and instructions for shooting are given.

[0025] Another example of the first instruction unit is a configuration that uses an eyepiece sensor located on the main unit 1 to detect when the user approaches the EVF 9b. When the user looks through the EVF 9b and the first composition is determined, and the user takes their eye away from the EVF 9b, the eyepiece sensor outputs an off signal. The operation detection unit 10 detects the off signal from the eyepiece sensor as an instruction signal for the first instruction unit. Alternatively, since the main unit 1 and the imaging lens 2 are provided with various operating members, one method is to assign the function of the first instruction unit to any existing operating member. Or, a dedicated operating member for the first instruction unit may be provided.

[0026] The camera control unit 5 determines whether the operation detection unit 10 has detected the instruction signal from the first instruction unit (S102). For example, if the operation detection unit 10 detects the ON signal of SW1 as the instruction signal from the first instruction unit, the camera control unit 5 accepts the shooting preparation instruction and proceeds to S103. If the operation detection unit 10 does not detect the instruction signal from the first instruction unit, the image is updated in S101, and the user continues to adjust the composition.

[0027] When an instruction signal is detected from the first instruction unit, the camera control unit 5 executes the process of saving the data of the first image related to the first composition to the storage unit 8 (S103). The first image is used for motion vector detection, which will be described later. Next, the camera control unit 5 performs the process of updating the image (S104). While the image is being updated, the user can check the status of the subject visually or by viewing it on the rear display device 9a or EVF 9b. At this time, the position and orientation of the imaging device may deviate from the position and orientation corresponding to the ideal composition. If a compositional deviation occurs, the user needs to make adjustments to return to the ideal composition before it is time to take a picture.

[0028] As the shooting time approaches, the user adjusts the position and orientation of the imaging device to achieve the ideal composition. However, when shooting handheld or with a monopod, it may be possible to return the imaging device to an approximate position and orientation relative to the ideal composition. In such cases, the user performs an operation to instruct the imaging device to use the composition achieved when the imaging device is adjusted to an approximate position and orientation as a second composition. In other words, the user instructs the imaging device to use a second instruction unit to specify the second composition. The second instruction unit may be the same operating member as the first instruction unit, or it may be a different operating member from the first instruction unit. The operation detection unit 10 is capable of detecting the instruction signal of the predetermined second instruction unit.

[0029] The camera control unit 5 determines whether the operation detection unit 10 has detected the instruction signal from the second instruction unit (S105). If the operation detection unit 10 detects the instruction signal from the second instruction unit, the process proceeds to S106. If the operation detection unit 10 does not detect the instruction signal from the second instruction unit, the image is updated in S104, and the user continues to make approximate adjustments to the composition.

[0030] When an instruction signal from the second instruction unit is detected, the camera control unit 5 processes data of the second image in the second composition, that is, a composition that does not strictly match the first composition but is in an approximate position and orientation, and stores it in the storage unit 8 (S106). The image shift detection unit 18 then compares the first image and the second image (S107) and detects a motion vector from the comparison result between the first image and the second image (S108). The motion vector detected by the image shift detection unit 18 corresponds to the amount of compositional shift of the second image in the second composition relative to the first image in the ideal composition. The motion vector detection result is sent to the camera control unit 5.

[0031] The camera control unit 5 calculates the amount of movement of the image sensor 6 by the image shift correction unit 14 based on the motion vector detected in S108, and controls the movement of the image sensor 6 (S109). The amount of movement of the image sensor 6 based on the motion vector is calculated by the camera control unit 5 so as to cancel out the motion vector detected in S108. In other words, the image shift correction unit 14 moves the image sensor 6 by the calculated amount of movement, thereby correcting the image shift between the first composition and the second composition. The user can then immediately take a picture with the ideal composition.

[0032] For example, if the first and second instruction units are configured by SW1, the ON signal of SW1 is used as the instruction signal for the first instruction unit. In this case, image data is saved in the composition at the time of the shooting preparation instruction, such as distance measurement or photometering, in S103. Furthermore, when SW1 is turned ON again while the image data is saved in the composition from S103, the signal is used as the instruction signal for the second instruction unit. By turning SW1 ON again, the user can quickly return to the composition at the time SW1 was turned ON the first time. Also, if the first instruction unit is configured by SW1 and the second instruction unit by SW2, suppose the user turns SW2 ON while the image data is saved in the composition from S103. In this case, the shooting process can start immediately after quickly returning to the composition at the time SW1 was turned ON.

[0033] Furthermore, assuming that the first and second indicator units are configured by eyepiece sensors, the OFF signal from the eyepiece sensor is used as the indicator signal for the first indicator unit. In S103, image data is saved with the composition at the moment the user takes their eye away from the EVF9b. When the user looks into the EVF9b again, an ON signal is input from the eyepiece sensor. With the image data saved with the composition from S103, the ON signal acquired from the eyepiece sensor is used as the indicator signal for the second indicator unit. By looking into the EVF9b again, the user can quickly return to the composition at the moment they took their eye away from the EVF9b.

[0034] In this embodiment, image misalignment correction is performed by translating the image sensor 6 based on the detection results of motion vectors related to the first and second images. According to this embodiment, an imaging device can be provided that allows the user to take pictures with an ideal composition without missing any shooting opportunities.

[0035] [Second Example] A second embodiment of the present invention will be described with reference to Figure 3. Figure 3 is a flowchart illustrating image misalignment correction in this embodiment. Matters similar to those in the first embodiment will be omitted from the explanation, and the differences will be explained mainly. This method of omitting explanations will be the same in the embodiments described later.

[0036] In this embodiment, after the data for the first and second images are saved, a motion vector is calculated (S101 to S108), and the process proceeds to S201. The camera control unit 5 compares the amount of movement that the image shift correction unit 14 can move the image sensor 6 (hereinafter referred to as the first amount of movement) with the amount of movement of the image sensor 6 by the image shift correction unit 14 based on the motion vector (hereinafter referred to as the second amount of movement). The camera control unit 5 determines whether the first amount of movement is greater than the second amount of movement (S202). If the first amount of movement is greater than the second amount of movement, the process proceeds to S109. If the first amount of movement is less than or equal to the second amount of movement, the process proceeds to S203.

[0037] If the first amount of movement is greater than the second amount of movement, the image shift correction unit 14 can move the image sensor 6 based on the motion vector. In this case, the camera control unit 5 corrects the composition shift using the image shift correction unit 14 (S109).

[0038] On the other hand, if the first amount of movement is less than or equal to the second amount of movement, no correction is performed by the image shift correction unit 14. The camera control unit 5 controls the display of a notification message indicating that the composition shift has not been corrected on the rear display device 9a or EVF 9b (S203). Alternatively, a notification sound may be generated by the speaker provided in the main unit 1.

[0039] According to this embodiment, the wasted time caused by shooting without correcting the composition to an ideal state is eliminated, thus improving convenience.

[0040] [Third Embodiment] A third embodiment of the present invention will be described with reference to Figures 4 and 5. Figures 4 and 5 are flowcharts illustrating image shift correction in this embodiment. First, the rear display device 9a and EVF 9b are updated with images for adjusting the composition (S101). The blur detection unit 15 performs blur detection, and the blur detection result is updated (S301).

[0041] The user determines an ideal composition (first composition) and issues an instruction to the imaging device using the first instruction unit. When the operation detection unit 10 detects the instruction signal from the first instruction unit (S102), the data of the first image is stored in the storage unit 8 (S103). The detection result by the blur detection unit 15 is stored in the storage unit 8 as the first blur detection result (S302).

[0042] The image is updated after the data for the first image is saved (S104). During the image update, the user checks the subject's condition visually or using the rear display device 9a or EVF9b. At this time, a deviation from the ideal composition may occur, so it is necessary to adjust the composition to the ideal before taking the picture.

[0043] The data for the first image is saved, and as the image is updated, the blur detection result by the blur detection unit 15 is updated (S303). Next, the processes in S105 and S106 are executed, and the blur detection result by the blur detection unit 15 is saved in the storage unit 8 as the second blur detection result (S304).

[0044] The image shift detection unit 18 compares the first image and the second image (Figure 5: S107) and detects a motion vector from the comparison result (S108). The motion vector detection result is sent to the camera control unit 5. The camera control unit 5 calculates the amount of movement of the image sensor 6 by the image shift correction unit 14 based on the detected motion vector.

[0045] The camera control unit 5 also calculates an image blur correction amount based on the first blur detection result in S302 or the second blur detection result in S304 in Figure 4. The image blur correction amount based on the blur detection result is a correction amount calculated so that the detected blur is canceled out by the movement of the image sensor 6. For the blur detection result, the result obtained by averaging the first blur detection result and the second blur detection result may be used. Alternatively, the image blur correction amount may be calculated based on a blur detection result that satisfies predetermined conditions, for example, the larger blur detection result.

[0046] Next, the camera control unit 5 compares the amount of movement of the image sensor 6 by the image shift correction unit 14 based on the calculated motion vector and blur detection result (hereinafter referred to as the third amount of movement) with the first amount of movement (the amount of movement of the image sensor 6 that can be moved by the image shift correction unit 14) (S305). Then, the camera control unit 5 determines whether the first amount of movement is greater than the third amount of movement (S306). If the first amount of movement is greater than the third amount of movement, image shift correction is possible based on the motion vector, and the process proceeds to S109. The image shift correction unit 14 corrects the composition shift. The image shift correction unit 14 can correct both the blur detected by the blur detection unit 15 and the composition shift detected by the image shift detection unit 18.

[0047] Furthermore, if the first movement amount in S306 is less than or equal to the third movement amount, the camera control unit 5 calculates a movement amount obtained by subtracting the image shift correction amount (movement amount of the image sensor 6) based on the blur detection result from the first movement amount (S307). Then, the image shift correction unit 14 moves the image sensor 6 according to the calculated movement amount. In this way, it is possible to correct image blur during shooting while correcting the composition to be as close to the ideal composition as possible before shooting.

[0048] In S308, following S307, the camera control unit 5 controls the display of a message indicating insufficient correction of composition shift on the rear display device 9a or EVF 9b to show that the image shift correction unit 14 has not been able to correct the image shift. Alternatively, the camera control unit 5 controls the speaker provided in the main unit 1 to emit an alarm.

[0049] According to this embodiment, wasted time caused by shooting with a composition that has not been corrected to an ideal composition is eliminated, and it is possible to adjust the composition by only the remaining correction difference before shooting.

[0050] [Fourth embodiment] A fourth embodiment of the present invention will be described with reference to Figures 1 and 6. The main unit 1 according to this embodiment further comprises a subject detection unit 19 and a motion vector separation unit 20. The subject detection unit 19 detects feature quantities such as the face portion and object portion of the subject, as well as the amount of motion, from the captured image. Based on the feature quantities and the amount of motion, the subject detection unit 19 detects, for example, the main subject and outputs the subject detection result data to the camera control unit 5.

[0051] The motion vector separation unit 20 separates the motion vectors detected by the image displacement detection unit 18 into multiple regions. The motion vectors separated for each region include, for example, the following vectors. • The first motion vector representing the movement of the background. • A second motion vector representing the movement of the subject detected by the subject detection unit 19. In this embodiment, the motion vectors are separated into two types, but it is sufficient that at least the first motion vector can be separated or extracted, and known motion vector detection methods (such as methods for extracting background vectors using histograms) can be used.

[0052] Referring to the flowchart in Figure 6, the image shift correction in this embodiment will be explained. In this embodiment, as in the first embodiment, the data for the first image in the first composition and the data for the second image in the second composition are saved, and motion vectors are calculated (S101 to S108). The differences, such as the processing in S601 and S602, will be explained.

[0053] The motion vector separation unit 20 separates the motion vector detected in S108 into a first motion vector representing the movement of the background and a second motion vector representing the movement of the subject (S601). Based on the separated first motion vector representing the movement of the background, the image shift correction unit 14 moves the image sensor 6 (S602).

[0054] In the first to third embodiments, if a subject other than the background is included in the first image when determining the first composition or in the second image when determining the second composition, the motion vector may be affected by the subject. For example, if the image sensor 6 moves by an amount of movement calculated based on the motion vector affected by the subject by the image shift correction unit 14, a correction residual may occur with respect to the first composition.

[0055] According to this embodiment, the image sensor 6 moves according to the amount of movement calculated based on a first motion vector representing the movement of the separated background, thereby suppressing compositional shifts and increasing the number of scenes in which it is possible to correct for an ideal composition. In this embodiment, the amount of movement of the image sensor 6 in image shift correction is calculated based on the first motion vector, but the first to third embodiments can also be used to perform shooting operations using the first motion vector by adopting the configuration of this embodiment. According to the above embodiment, by enabling correction of compositional deviations, it is possible to provide an imaging device that can take pictures with the composition determined by the first instruction without losing a shooting opportunity.

[0056] [Differentiation] Modifications of the first to fourth embodiments are described below. In the first modified example, image shift correction is performed by the lens control unit 12 controlling the movement of the corrective lens at the photographic lens 2. In other words, the lens drive unit 13 has a function equivalent to the image shift correction unit 14, and the lens control unit 12 controls the movement of the corrective lens (shift lens, etc.) that constitutes the imaging optical system 3.

[0057] A second modification involves a configuration that combines composition shift correction performed by an image shift correction unit 14 within the main body 1 with composition shift correction performed by a lens drive unit 13 within the shooting lens 2. A third modification involves the image processing unit 7 performing image shift correction on the captured image so that the detected motion vectors cancel each other out. In other words, the image processing unit 7 has an image correction unit that corrects image shift by cropping and rotating the image.

[0058] The above aspects can be summarized as follows: (A) An embodiment in which image misalignment is corrected by driving an optical element. (B) An embodiment in which image shift is corrected by driving the image sensor. (C) An embodiment in which image misalignment is corrected by image processing of captured images. (D) An embodiment that uses two or more of (A) through (C) in combination. In the case of (A), the amount of movement of the image sensor shall be appropriately replaced with the amount of movement of the optical component in this specification. Also, in the case of (C), the amount of movement of the image sensor shall be appropriately replaced with the correction amount in image processing (image extraction, processing, etc.) in this specification. Although preferred embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications and changes are possible within the scope of its gist. [Explanation of symbols]

[0059] 5. Camera System Control Unit 7 Image Processing Unit 14 Image misalignment correction unit 18 Image shift detection unit

Claims

1. An imaging device that performs image shift correction for compositional misalignment, A first instruction means for giving instructions for the first composition, A second instruction means for giving instructions for a second composition, A storage means that stores data of a first image according to the instruction for the first composition, and stores data of a second image according to the instruction for the second composition, which is after the instruction for the first composition, A first detection means for detecting the motion vector of the second image relative to the first image, A background vector acquisition means for acquiring a background vector representing the movement of the background based on the aforementioned motion vector, When the second composition is instructed by the second instruction means, the first image and the second image are compared, and if a shift occurs in the background of the second image relative to the background of the first image, the system includes a correction means that controls the movement of the optical elements of the imaging optical system or the image sensor, or corrects the image shift by image processing. The correction means calculates the amount of movement of the optical element or image sensor of the imaging optical system or the correction amount by the image processing using the background vector, and corrects the image misalignment of the second image with respect to the first image. An imaging device characterized by the following features.

2. The first detection means detects the motion vector by comparing the data of the first image stored in the storage means according to the instructions of the first instruction means with the data of the second image stored in the storage means according to the instructions of the second instruction means. The imaging apparatus according to feature 1.

3. If the amount of movement or correction is greater than the amount of movement that allows the optical element or image sensor to be moved or the amount of correction that allows the image processing to be performed, the correction means will not move the optical element or image sensor based on the background vector or perform the correction by image processing. The imaging apparatus according to claim 1 or 2.

4. The correction means performs a process to notify that image misalignment correction will not be performed if the optical element or image sensor is not moved based on the background vector or correction is not performed by image processing. The imaging device according to feature 3.

5. It is equipped with a second detection means for detecting vibration of the imaging device, The correction means calculates the amount of movement of the optical element or image sensor based on the background vector and the detection result of the second detection means, and if the amount of movement that the optical element or image sensor can move is greater than the calculated amount of movement, it performs a correction by moving the optical element or image sensor based on the background vector. The imaging device according to feature 3.

6. If the amount of movement of the optical element or image sensor based on the background vector and the detection result of the second detection means is greater than the amount of movement the optical element or image sensor can move, the correction means calculates an amount obtained by subtracting the amount of movement of the optical element or image sensor based on the detection result of the second detection means from the amount of movement the optical element or image sensor can move, and performs correction due to the movement of the optical element or image sensor. The imaging device according to claim 5, characterized in that it is a feature of the present invention.

7. The correction means performs a process to notify of insufficient image shift correction if the amount of movement of the optical element or image sensor based on the background vector and the detection result of the second detection means is greater than the amount of movement that the optical element or image sensor can move. The imaging device according to feature 6.

8. The background vector acquisition means acquires the background vector by separating the motion vector into motion vectors in multiple regions. The imaging apparatus according to any one of claims 1 to 7.

9. It is equipped with a subject detection means for detecting a subject from an captured image, The background vector acquisition means separates a first motion vector representing the movement of the background and a second motion vector representing the movement of the subject detected by the subject detection means. The correction means uses the first motion vector as the background vector. The imaging apparatus according to feature 8.

10. A control method performed in an imaging device that includes a first instruction means for giving instructions for a first composition and a second instruction means for giving instructions for a second composition, and which performs image shift correction for composition shift, The process involves the storage means storing data of a first image in accordance with the instruction for the first composition, and the storage means storing data of a second image in accordance with the instruction for a second composition, which occurs after the instruction for the first composition. A step of detecting the motion vector of the second image relative to the first image, A step of obtaining a background vector representing the background movement based on the aforementioned motion vector, When the second composition is instructed by the second instruction means, the first image and the second image are compared, and if a shift occurs in the background of the second image relative to the background of the first image, the system includes a correction step of controlling the movement of the optical components of the imaging optical system or the image sensor, or correcting the image shift by image processing. In the correction step, the amount of movement of the optical elements of the imaging optical system or the image sensor, or the amount of correction by image processing, is calculated using the background vector, and a process is performed to correct the image misalignment of the second image with respect to the first image. A control method for an imaging device, characterized by the following:

Images