Image capturing apparatus, method of controlling same, and image processing apparatus and method

By prioritizing rotational actuation in the imaging device to correct jitter, the problems of image quality and time series stability in still and moving images are solved, and sensor displacement stability and image synthesis quality improvement suitable for both are achieved.

CN118974646APending Publication Date: 2024-11-15CANON KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202280094295.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2022-03-29
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The prior art is difficult to achieve image quality and time series stability in both still and moving images, resulting in the failure to obtain the desired image quality in image stabilization of still and moving images.

Method used

Using an imaging device, including an imaging component, a jitter detection component and an actuation component, the rotational actuation is preferred to correct jitter by translating and rotating actuation in a plane perpendicular to the optical axis of the imaging optical system, and the rotational actuation is preferred to correct jitter, which is suitable for static and motion The sensor displacement of the image is stable and the quality of the composite image is improved during image synthesis.

Benefits of technology

Sensor displacement stability control suitable for moving images and still images is realized, and image quality is improved during image synthesis, meeting the image quality of still images and the time series stability requirements of moving images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118974646A_ABST
    Figure CN118974646A_ABST
Patent Text Reader

Abstract

An image pickup apparatus includes: an image pickup section for photoelectrically converting light incident through an image pickup optical system and outputting an image; a shake detection means for detecting shake of the image pickup apparatus; and an actuating member for translating and rotationally actuating the imaging member in a plane perpendicular to the optical axis of the imaging optical system to correct the shake detected by the shake detecting member. When the plurality of images output from the imaging unit are combined, the actuating unit performs rotational actuation prior to translational actuation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an image pickup device and a control method thereof as well as an image processing device and method, and more particularly to an image stabilization technology in an image pickup device. Background Art

[0002] In recent years, with the advancement of imaging equipment, many imaging equipment are equipped with a technology for correcting camera shake by moving the image sensor (sensor shift stabilization). In this sensor shift stabilization technology, many sensor shift stabilization mechanisms have been proposed, which can perform rotational movement around the optical axis in addition to translational movement in a plane perpendicular to the optical axis.

[0003] Another type of image stabilization is a technique for correcting camera shake by electronically aligning the captured image so that it overlaps with a reference image (electronic image stabilization). Another type of image stabilization is a technique for aligning and synthesizing multiple images to obtain an image with an exposure equivalent to a long exposure (hereinafter referred to as "image synthesis stabilization"). Electronic image stabilization does not require image synthesis and is generally used for moving images that require real-time performance. On the other hand, image synthesis stabilization requires synthesis processing and is designed for long exposures, so it is generally used for still images.

[0004] Furthermore, Patent Document 1 discloses a technique for correcting camera shake by coordinating sensor shift stabilization for correcting rotational shake by rotating an image sensor with electronic image stabilization.

[0005] Citation List

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2012-242563 Summary of the invention

[0008] Problem that the invention aims to solve

[0009] Generally, in image stabilization of still images, the image quality of each image is considered important, whereas in image stabilization of moving images, the time series stability of images is considered important. Therefore, the requirements in image composition stabilization of still images and the requirements in electronic image stabilization of moving images are different, so if sensor shift stabilization or electronic image stabilization is performed on still images and moving images using the same method, the desired image quality of still images or moving images may not be obtained.

[0010] The present invention has been made in view of the above circumstances, and an object thereof is to perform sensor shift stabilization control suitable for both moving images and still images.

[0011] A second object of the present invention is to improve the quality of a synthesized image when a plurality of images are synthesized.

[0012] Solutions for solving problems

[0013] In order to achieve the above-mentioned purpose, there is provided an image pickup device, comprising: an image pickup component for photoelectrically converting light incident via an image pickup optical system and outputting an image; a shake detection component for detecting shake of the image pickup device; and an actuation component for performing translational actuation and rotational actuation on the image pickup component in a plane perpendicular to an optical axis of the image pickup optical system to correct the shake detected by the shake detection component, wherein, in a case where a plurality of images output from the image pickup component are synthesized, the actuation component causes the rotational actuation to be performed in priority to the translational actuation.

[0014] Effects of the Invention

[0015] According to the present invention, it is possible to perform sensor shift stabilization control suitable for moving images and still images, and also improve the image quality of a synthesized image when a plurality of images are synthesized.

[0016] Other features and advantages of the present invention will be apparent from the following description in conjunction with the accompanying drawings. Note that throughout the accompanying drawings, the same reference numerals represent the same or similar components. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present invention and, together with the description, serve to explain the principles of the present invention.

[0018] Figure 1A is a cross-sectional view at the center of the imaging system according to the first embodiment of the present invention.

[0019] Figure 1B is a block diagram illustrating a brief configuration of an image pickup system according to a first embodiment of the present invention.

[0020] Figure 2 is an exploded perspective view of the image stabilization mechanism according to the first embodiment.

[0021] Figure 3 is a flowchart illustrating the processing in the first embodiment.

[0022] Figure 4 1 is a diagram illustrating the relationship between the rotation and translation of the image stabilization mechanism in the first embodiment.

[0023] Figure 5A 2 is a diagram illustrating tilt jitter according to the second embodiment.

[0024] Figure 5B2 is a diagram illustrating tilt jitter according to the second embodiment.

[0025] Figure 6 is a flowchart illustrating the processing in the second embodiment.

[0026] Figure 7 : is a diagram illustrating the relationship between the amount of rotational shake and tilting shake and the focal length according to the second embodiment.

[0027] Figure 8 is a block diagram illustrating a brief configuration of an image pickup system according to a third embodiment of the present invention.

[0028] Fig. 9A is a diagram for explaining the structure of an image sensor according to a third embodiment.

[0029] Fig. 9B is a diagram for explaining the structure of an image sensor according to a third embodiment.

[0030] Fig. 10A : is a diagram for explaining processing performed by an image displacement calculation unit and a reliability detection unit according to the third embodiment.

[0031] Fig. 10B : is a diagram for explaining processing performed by an image displacement calculation unit and a reliability detection unit according to the third embodiment.

[0032] Fig. 10C : is a diagram for explaining processing performed by an image displacement calculation unit and a reliability detection unit according to the third embodiment.

[0033] Fig. 10D : is a diagram for explaining processing performed by an image displacement calculation unit and a reliability detection unit according to the third embodiment.

[0034] Fig.11A is a schematic external view of an image pickup apparatus according to a fourth embodiment.

[0035] Fig. 11B is a block diagram illustrating the configuration of an image processing unit of an image pickup apparatus according to a fourth embodiment. DETAILED DESCRIPTION

[0036] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. A plurality of features are described in the embodiments, but are not limited to the invention requiring all of these features, and a plurality of these features may be appropriately combined. In addition, in the accompanying drawings, the same reference numerals are given to the same or similar configurations, and their repeated descriptions are omitted.

[0037] <First Embodiment>

[0038] Figure 1A and Figure 1B An imaging apparatus according to an embodiment of the present invention is exemplified. Figure 1A is a cross-sectional view at the center of an image pickup system as an image pickup apparatus according to an embodiment of the present invention, and Figure 1B is a block diagram illustrating a schematic configuration of an imaging system.

[0039] like Figure 1A As shown, the imaging system of the present embodiment mainly includes a camera body 1 and a lens unit 2 detachable from the camera body 1. The camera body 1 and the lens unit 2 are electrically connected via an electrical contact 11. Note that the imaging apparatus of the present invention is not limited to this configuration, and may be an imaging apparatus in which the camera body and the lens unit are integrally configured.

[0040] The lens unit 2 includes: an imaging optical system 3, which is arranged on an optical axis 4 and is composed of a plurality of lenses including a focus lens, a zoom lens, and an image stabilization lens, and an aperture; a lens system control circuit 12; and a lens memory 17. The camera body 1 includes an image sensor 6, a rear display device 9a, an EVF 9b, an image stabilization mechanism 14, a shake detection unit 15, and a shutter mechanism 16.

[0041] Figure 1B is a diagram showing an electrical configuration of an imaging system, wherein the lens unit 2 further includes a lens actuation mechanism 13 that moves a focus lens, a zoom lens, an image stabilization lens, an aperture, and the like included in the imaging optical system 3. The camera body 1 further includes a camera system control circuit 5, an image processing unit 7, a memory 8, a display unit 9, and an operation detection unit 10. The display unit includes a rear display device 9a and an EVF 9b.

[0042] In the imaging system having the above-described configuration, light from the subject incident via the imaging optical system 3 converges on the imaging surface of the image sensor 6. The image sensor 6 performs photoelectric conversion on the incident light and outputs an electrical signal (image signal) corresponding to the amount of light. The image sensor 6 can output image signals of images in various formats (such as so-called still images and moving images, etc.), and can output moving images in multiple formats by changing the aspect ratio, the resolution of the image to be recorded, etc. In addition, the imaging system of the present embodiment has a mode for shooting and synthesizing a group of still images that are continuous in time (dynamic range extension, noise reduction, etc.), and there is a case where a group of images that are continuous in time are shot regardless of whether the images are still images or moving images.

[0043] The image processing unit 7 includes an A / D converter, a white balance adjustment circuit, a gamma correction circuit, an interpolation calculation circuit, and the like, and processes an image signal output from the image sensor 6 to generate image data for recording.

[0044] The image processing unit 7 is also connected to the camera system control circuit 5. The camera system control circuit 5 can obtain a focus evaluation value and an exposure amount based on the image signal from the image sensor 6 processed by the image processing unit 7, and obtain a focus position and exposure conditions (F value, shutter speed, etc.) based on these values. Then, the camera system control circuit 5 issues a command to the lens system control circuit 12 via the electrical contact 11 based on the obtained focus position and exposure conditions, and the lens system control circuit 12 controls the lens actuation mechanism 13 based on the command. The shutter mechanism 16 controls the incidence / blocking of the subject image on the image sensor 6 by moving the shutter curtain. Through the above control, the image sensor 6 is exposed to an appropriate amount of light, and a subject image is formed near the image sensor 6.

[0045] The image processing unit 7 in this embodiment also includes: an alignment unit 71 that performs electronic image stabilization by shifting and aligning the positions of multiple images; and an image synthesis unit 72 that synthesizes multiple images. The specific operations of the alignment unit 71 and the image synthesis unit 72 will be described later. The image processing unit 7 also compresses data such as images, moving images, and audio using a predetermined method.

[0046] The memory 8 has a storage unit of an image, and the camera system control circuit 5 outputs the image to the recording unit of the memory 8 .

[0047] Under the control of the camera system control circuit 5, the display unit 9 turns off the back display device 9a and displays images and information on the EVF 9b when the user is looking at the EVF 9b, and displays images and information on the back display device 9a when the user is not looking at the EVF 9b. The back display device 9a serves as a touch panel and is connected to the operation detection unit 10.

[0048] The shake detection unit 15 can detect the translation shake in the translation direction on the plane perpendicular to the optical axis 4 and the rotation shake of the device in the rotation direction around the optical axis 4, and a vibration gyro or an acceleration sensor or the like can be used. It is also possible to detect the shake by comparing the image between frames based on the image signal output from the image sensor 6. The image stabilization mechanism 14 is a mechanism that causes the image sensor 6 to translate in the plane perpendicular to the optical axis 4 and rotate around the optical axis 4, and its specific structure will be referred to later. Figure 2 Provide explanation.

[0049] The camera system control circuit 5 generates and outputs a timing signal and the like when capturing an image. In addition, in response to an external operation detected by the operation detection unit 10, the camera system control circuit 5 controls a capture process, an image process, and a recording / playback process. For example, when the operation detection unit 10 detects that a shutter release button (not shown) is pressed, the camera system control circuit 5 controls the actuation of the image sensor 6, the operation of the image processing unit 7, the operation of the compression process, and the like. The camera system control circuit 5 also includes an alignment on / off unit 51 and an image synthesis on / off unit 52 for respectively turning on / off the operations of the alignment unit 71 and the image synthesis unit 72 included in the image processing unit 7, and also includes an image stabilization control unit 53. The image stabilization control unit 53 generates a target value of the actuation amount of the image sensor 6 using the image stabilization mechanism 14 based on a signal from the shake detection unit 15 according to the on / off conditions of alignment and image synthesis as described below, and performs image stabilization (sensor shift stabilization) control.

[0050] In addition, in the mode for performing normal image stabilization, the camera system control circuit 5 can also perform known optical image stabilization by controlling the image stabilization lens included in the imaging optical system 3 via the lens actuation mechanism 13 based on a signal obtained from the image sensor 6 .

[0051] The flow of the sensor shift stabilization control in the camera system having the above configuration is briefly described here.

[0052] In the present embodiment, sensor shift stabilization control is performed using a shake detection unit 15 that detects shake, an image stabilization mechanism 14 that performs sensor shift stabilization, and an image stabilization control system provided in the camera system control circuit 5. Then, during a so-called framing operation in which the operation detection unit 10 detects an operation (SW1) of half-pressing a shutter release button (not shown) to start a shooting preparation operation, sensor shift stabilization is performed using the image stabilization mechanism 14 to facilitate framing. That is, sensor shift stabilization is performed by controlling the image stabilization mechanism 14 based on a signal from the shake detection unit 15. Thereafter, when the operation detection unit 10 detects an operation (SW2) of fully pressing the shutter release button to start a shooting operation, sensor shift stabilization is performed using the image stabilization mechanism 14 to suppress blurring of a subject image acquired by exposure. When a certain time has passed after the exposure, the sensor shift stabilization operation stops.

[0053] Next, we will refer to Figure 2 The image stabilization mechanism 14 of the present embodiment will be described. Figure 2 is an exploded perspective view of a portion of the image stabilization mechanism 14 for image stabilization. Note that there is a separate electrical mechanism for control, but this is not included in the Figure 2As shown in the figure. Figure 2 In the case of Figure 2 In the figure, the non-moving member (fixed part) is numbered as 100s. The moving member (movable part) is numbered as 200s. In addition, the ball held between the fixed part and the movable part is numbered as 300s.

[0054] exist Figure 2 , 101 denotes an upper yoke, 102a, 102b and 102c denote screws, 103a, 103b, 103c, 103d, 103e and 103f denote upper magnets, 104a and 104b denote auxiliary spacers, 105a, 105b and 105c denote main spacers, 106a, 106b and 106c denote fixed portion rolling plates, 107a, 107b, 107c, 107d, 107e and 107f denote lower magnets, 108 denotes a lower yoke, 109a, 109b and 109c denote screws, and 110 denotes a substrate.

[0055] Reference numeral 201 denotes an FPC, 202a, 202b and 202c denote position detection element mounting positions, 203 denotes a movable PCB, 204a, 204b and 204c denote movable portion rolling plates, 205a, 205b and 205c denote coils, 206 denotes a movable frame, and 301a, 301b and 301c denote ball bearings.

[0056] The upper yoke 101, the upper magnets 103a, 103b, 103c, 103d, 103e and 103f, the lower magnets 107a, 107b, 107c, 107d, 107e and 107f, and the lower yoke 108 form a magnetic circuit, thereby forming a so-called closed magnetic circuit. The upper magnets 103a, 103b, 103c, 103d, 103e and 103f are attached and fixed in a state of being attracted to the upper yoke 101. Similarly, the lower magnets 107a, 107b, 107c, 107d, 107e and 107f are attached and fixed in a state of being attracted to the lower yoke 108. The upper magnets 103a, 103b, 103c, 103d, 103e and 103f and the lower magnets 107a, 107b, 107c, 107d, 107e and 107f are arranged along the optical axis direction ( Figure 2 In addition, adjacent magnets (such as magnets in a positional relationship between magnets 103a and 103b) are magnetized in different directions, and in addition, opposing magnets (such as magnets in a positional relationship between magnets 103a and 107a) are magnetized in the same direction. In this way, a strong magnetic flux density is generated between the upper yoke 101 and the lower yoke 108 in the optical axis direction.

[0057] Since a strong attraction is generated between the upper yoke 101 and the lower yoke 108, the main spacers 105a, 105b and 105c and the auxiliary spacers 104a and 104b are configured to maintain an appropriate distance therebetween. The appropriate distance is a distance that allows the coils 205a, 205b and 205c and the FPC 201 to be placed between the upper magnets 103a, 103b, 103c, 103d, 103e and 103f and the lower magnets 107a, 107b, 107c, 107d, 107e and 107f while ensuring an appropriate gap. The main spacers 105a, 105b and 105c have screw holes, and the upper yoke 101 is fixed to the main spacers 105a, 105b and 105c by screws 102a, 102b and 102c.

[0058] The side portions of the main spacers 105a, 105b and 105c are provided with rubber, thereby forming mechanical ends (so-called stoppers) of the movable parts.

[0059] The substrate 110 has a hole while avoiding the lower magnets 107a, 107b, 107c, 107d, 107e and 107f, and the surface of the magnet protrudes from the hole. That is, the substrate 110 and the lower yoke 108 are fixed by screws 109a, 109b and 109c, and the lower magnets 107a, 107b, 107c, 107d, 107e and 107f having a larger size than the substrate 110 in the thickness direction are fixed in a manner protruding from the substrate 110.

[0060] The movable PCB 203 is made of magnesium die-casting or aluminum die-casting, and is lightweight and highly rigid. Each element of the movable part is fixed to the movable PCB 203 to form the movable part. The position detection element is Figure 2 The position detection element mounting positions 202a, 202b and 202c are attached to the FPC 201 on the surface not visible in FIG. For example, a Hall element or the like can be used to detect the position using the magnetic circuit described above. Since the Hall element is small, the Hall element is arranged to be nested in the windings of the coils 205a, 205b and 205c.

[0061] The image sensor 6 (not shown), the coils 205a, 205b and 205c, and the Hall element are connected to the movable PCB 203. Electrical communication with the outside is performed via a connector on the movable PCB 203.

[0062] The fixed part rolling plates 106a, 106b and 106c are attached to the substrate 110, and the movable part rolling plates 204a, 204b and 204c are attached to the movable PCB 203, thereby forming the rolling surfaces of the balls 301a, 301b and 301c. By separately setting the rolling plates, the surface roughness, hardness and other properties can be easily designed to a desired state.

[0063] In the above configuration, by passing current through the coils 205a, 205b, and 205c, a force following Fleming's left-hand rule is generated, and the movable frame 206 can be moved. In addition, feedback control can be performed by using a signal from a Hall element (which is the above-mentioned position detection element). By appropriately controlling the value of the signal to the Hall element, the movable frame 206 can be moved in a translational motion in a plane perpendicular to the optical axis, and the movable frame 206 can be rotated around the optical axis.

[0064] By actuating so that the signals from the Hall elements at the position detecting element mounting positions 202b and 202c have opposite phases while keeping the signal from the Hall element at the position detecting element mounting position 202a constant, rotational movement of the movable frame 206 approximately around the optical axis 4 can be generated.

[0065] The magnetic flux density in the direction of the optical axis is detected at the position detection element installation positions 202a, 202b, and 202c. The characteristics of the magnetic circuit composed of the upper magnets 103a, 103b, 103c, 103d, 103e, and 103f and the lower magnets 107a, 107b, 107c, 107d, 107e, and 107f are generally nonlinear. Therefore, the magnetic flux density detected at the position detection element installation positions 202a, 202b, and 202c does not necessarily have a constant resolution throughout the actuation range, so the detection resolution changes. This is because there are positions where the magnetic flux density changes steeply and positions where the change is gentle, and at positions where the change is steeper, the detection resolution is higher, that is, the greater the change in magnetic flux density relative to the amount of movement. Figure 2 In the magnetic circuit shown, the change in magnetic flux density is maximum at the boundary position of the magnets (eg, the boundary position between the upper magnets 103a and 103b), and the detection resolution is high at the boundary position.

[0066] Since many proposals have been made regarding the control method of the image stabilization mechanism 14, it will not be described in detail here.

[0067] Next, we will refer to Figure 3 The shown flowchart describes processing performed in the camera body 1 having the above-described configuration.

[0068] In step S301, the image synthesis on / off unit 52 in the camera system control circuit 5 controls whether image synthesis is to be performed. Note that the user may be able to directly instruct whether image synthesis is to be performed, or it may be automatically switched according to the camera settings (such as the shooting mode, etc.) and the shooting conditions (such as the shutter speed, etc.). For example, if the shutter speed is slower than the threshold in the still image shooting mode, image synthesis is performed, and in the moving image shooting mode or if the shutter speed is faster than the threshold in the still image shooting mode, image synthesis is not performed. Note that image synthesis is performed after alignment by the alignment unit 71, so if image synthesis is performed, the alignment on / off unit 51 is also turned on.

[0069] If image synthesis is performed, the process proceeds to step S302 , and if image synthesis is not performed, the process proceeds to step S303 .

[0070] In step S302, the image stabilization control unit 53 of the camera system control circuit 5 controls the sensor shift stabilization to prioritize the correction of rotational shake between translational shake and rotational shake (rolling shake). Hereinafter, this control is referred to as "rotational shake correction priority control". The reason for prioritizing the correction of rotational shake is to suppress the image quality degradation caused by the image alignment in step S304 and the image synthesis in step S305 described later. This control is suitable for performing image synthesis stabilization accompanied by image alignment in still image shooting where image quality is important.

[0071] Here, we will refer to Figure 4 The relationship between the rotation and translation of the image stabilizing mechanism 14 in the present embodiment will be described. Figure 4 yes Figure 3 The image stabilization mechanism 14 shown is a view viewed from the direction of the optical axis 4 (the Z-axis direction of the coordinate system at the lower right). Figure 4 The coordinate system of the lower right corner Figure 4 40A, 40B, and 40C are common. 40A shows a state where the movable frame 206 does not move, 40B shows a state where the movable frame 206 moves in the positive X direction without rotating, and 40C shows a state where the movable frame 206 moves in the positive X direction with rotating.

[0072] Figure 4The dashed line 400 drawn in common in 40A, 40B, and 40C in FIG. 40 is a reference line indicating the reference of the fixed part in the X direction, and the dashed line 401 indicates the reference line of the fixed part in the Y direction. In 40B, the dashed line 402 indicates the reference line after the movable frame 206 moves in the positive X direction. In 40C, the dashed line 403 indicates the reference line after the movable frame 206 is rotated, and the dashed line 404 indicates the reference line after the movable frame 206 moves in the positive X direction. In 40A, the reference line of the movable part 206 overlaps with the reference line of the fixed part. In other words, the movable part 206 is in a non-moving state.

[0073] 40B shows a state where the movable frame 206 moves in the positive X direction until it contacts the main spacer 105a without rotating the movable frame 206. In other words, the movable frame 206 can move in the positive X direction by an amount equivalent to the interval between the reference line 400 and the reference line 402.

[0074] 40C shows a state where the movable frame 206 rotates and moves in the positive X direction until it contacts the main spacer 105a. The amount of rotation is shown as the angle formed by the reference line 401 and the reference line 403. At this time, the amount of movement of the movable frame 206 in the X direction corresponds to the distance between the reference line 400 and the reference line 404.

[0075] As is apparent from comparing 40B and 40C, the possible translation in the X direction changes when rotation occurs. To suppress this change, a mechanical end can be provided for the movable portion 206 on the optical axis 4, but this is not easy due to the presence of the image sensor 6 and the processing board behind it. For this reason, the movable portion ( Figure 2 and Figure 4 The mechanical ends of the main spacers 105a, 105b and 105c) in the example of FIG. 1 are typically arranged at positions different from the optical axis 4.

[0076] In addition, as is apparent from 40C, the gap in the Y direction between the main spacer 105c and the movable frame 206 changes compared to the case where no rotation occurs. In other words, it can be seen that the possible movement amount in the positive Y direction is also reduced due to the rotation.

[0077] Although 40B and 40C do not show the specific relationship between the amount of rotation and the amount of translation, it can be clearly seen from 40C that, in general, as the amount of rotation increases, the amount of translation may decrease. In addition, in order to obtain the specific relationship, the position of the mechanical end of the movable part can be determined, and numerical calculation can be performed. In order to have a margin for the amount of rotation and the amount of translation, the main spacers 105a, 105b, and 105c can be designed to be away from the movable frame 206, but this will lead to an increase in the size of the device. Since the camera body 1 is carried by the user for use, it is very meaningful not to increase the size of the device.

[0078] As described above, since the rotation amount limits the maximum translation amount and vice versa, in step S302, the upper limit value of the rotational shake correction amount (rotational actuation amount) is set higher than the upper limit value of the translational shake correction amount (translational actuation amount) to prioritize compensation of rotational shake.

[0079] In step S304 , the alignment unit 71 of the image processing unit 7 detects misalignment between the plurality of images input from the image sensor 6 , and geometrically deforms the images to correct the misalignment.

[0080] Misalignment between images can be detected by calculating the similarity between images using known template matching methods, etc. In geometric transformation, images are subjected to affine transformation or projective transformation to correct misalignment of images in translation and rotation. This alignment achieves electronic image stabilization.

[0081] In addition, geometric transformation requires that the pixels of the image before and after the transformation correspond to each other. When the image is translated or rotated in sub-pixel units, a one-to-one correspondence cannot be achieved between pixels. Therefore, pixels whose values ​​are interpolated from multiple surrounding pixels are usually used to establish the correspondence.

[0082] However, such pixel interpolation reduces image resolution and causes image quality degradation. Therefore, in order to prevent image quality degradation, it is desirable to avoid translation and rotation in sub-pixel units. Regarding translation, by limiting the movement to a movement of an integer multiple of pixels, image quality degradation due to pixel interpolation can be avoided, although alignment accuracy will be reduced. In addition, if the rotational shake correction priority control in step S302 is used, the image stabilization mechanism 14 will sufficiently correct the rotational shake, thereby eliminating the need for rotational movement and avoiding image quality degradation.

[0083] In step S305, the image synthesis unit 72 of the image processing unit 7 synthesizes the multiple images by adding the multiple images aligned in step S304. This achieves image synthesis stabilization. Note that in the case where each of the multiple images is shot with appropriate exposure, it is necessary to average the images after adding them.

[0084] On the other hand, in the case where image synthesis is not performed, in step S303, the image stabilization control unit 53 controls the sensor shift stabilization to correct the translational shake and the rotational shake in a well-balanced manner. Hereinafter, this control is referred to as "standard shake correction control". This control is suitable for image stabilization in moving image shooting where time series stability is important. This control is also suitable for still image shooting without image alignment, that is, without image synthesis stabilization.

[0085] In the standard shake correction control, the upper limit of the rotational shake correction amount is set lower than that of the rotational shake correction priority control in step S302 and is substantially the same as that of the translational shake correction amount, thereby achieving balanced correction of translational shake and rotational shake.

[0086] Next, in step S306, the alignment on / off unit 51 of the camera system control circuit 5 controls whether image alignment is to be performed. The user can directly set whether image alignment is to be performed, or image alignment on / off can be automatically switched according to the camera settings and shooting conditions. For example, image alignment is performed in the moving image shooting mode, and in the still image shooting mode, image alignment is not performed when the shutter speed is faster than a threshold value.

[0087] If image alignment (ie, electronic image stabilization) is to be performed, the process proceeds to step S307, and if electronic image stabilization is not to be performed, the process ends.

[0088] In step S307 , the same image alignment processing as in step S304 is performed, and the processing ends.

[0089] As described above, according to the first embodiment, in the case where image synthesis stabilization is performed, sensor shift stabilization is performed under the rotational shake correction priority control, thereby enabling image stabilization suitable for still images that prioritize image quality. In addition, in the case where image synthesis stabilization is not performed, sensor shift stabilization is performed under the standard shake correction control, thereby enabling image stabilization suitable for moving images that prioritize stability.

[0090] In the above-described embodiment, an example is described in which rotational shake is corrected preferentially by setting the upper limit of the rotational shake correction amount (rotational actuation amount) to be higher than the upper limit of the translational shake correction amount (translational actuation amount) in the rotational shake correction priority control. However, the method of correcting rotational shake preferentially relative to translational shake is not limited to this. For example, the rotational shake correction amount may be calculated first, and the translational shake may be corrected within the remaining movable range. In addition, if the sum of the rotational shake correction amount and the translational shake correction amount exceeds the movable range, the gain used when obtaining the rotational shake correction amount based on the detected rotational shake may be made greater than the gain used when obtaining the translational shake correction amount based on the detected translational shake, so that the rotational shake is corrected preferentially. The gain may be set as the correction amount so that when the gain is 1, the detected shake is corrected 100%.

[0091] <Second Embodiment>

[0092] Next, a second embodiment of the present invention will be described.

[0093] In the first embodiment, the image stabilization mechanism 14 translates the image sensor 6 in a plane perpendicular to the optical axis 4 and rotates it around the optical axis 4. In the second embodiment, in addition to Figure 4 In addition to the configuration shown, the image stabilization mechanism 14 also includes a mechanism for rotating the image sensor 6 about two axes perpendicular to the optical axis 4. This mechanism allows the sensor shift stabilization to correct not only translation and rotation shakes, but also tilt components caused by camera shake in the yaw and pitch directions (hereinafter referred to as "tilt shake"). Note that the overall configuration of the camera system is similar to Figure 1A and Figure 1B The configuration shown is the same, so its description will be omitted.

[0094] Figure 5A and Figure 5B It is a diagram for explaining tilt shake. Figure 5A 501 indicates an original image, and 502 indicates an image affected by the camera shake in the yaw direction. When the camera shake in the yaw direction occurs in the camera in this way, in addition to the translation shake Tx, a tilt shake α also occurs in the image, which causes the shape of the original image to be distorted into a trapezoidal shape.

[0095] Similarly, Figure 5B The state when the camera shake in the pitch direction occurs is shown. 501 indicates an original image, and 503 indicates an image affected by the camera shake in the pitch direction. When the camera shake in the pitch direction occurs in the camera in this way, in addition to the translation shake Ty, a tilt shake β also occurs in the image, which causes the shape of the original image to be distorted into a trapezoidal shape.

[0096] Next, we will refer to Figure 6 The flowchart shown describes the sensor displacement stabilization operation according to the second embodiment. Figure 6 In, with Figure 3 The same processing shown is denoted by the same step numbers, and the description thereof will be appropriately omitted.

[0097] In the second embodiment, when performing image synthesis, in step S601, the image stabilization control unit 53 of the camera system control circuit 5 controls the sensor shift stabilization to prioritize correction of a specific shake among translational shake, rotational shake, and tilting shake. Hereinafter, this control will be referred to as "specific shake correction priority control".

[0098] When attempting to correct tilt shake during image alignment, as in the case of rotational shake, pixel interpolation occurs, resulting in degraded image quality. Therefore, when performing image synthesis stabilization, it is preferable to use sensor shift stabilization to correct tilt shake in addition to rotational shake.

[0099] The correction priority of rotational shake and tilt shake differs depending on the focal length. Figure 7 The relationship between the amount of rotational shake and tilt shake and the focal length is shown. In the area where the focal length is shorter than ft, the tilt shake is dominant, and in the area where the focal length is longer than ft, the rotation shake is dominant. Therefore, when the focal length is shorter than a predetermined focal length, correction is prioritized in the order of tilt shake>rotational shake>translation shake, and when the focal length is equal to or greater than the predetermined focal length, correction is prioritized in the order of rotational shake>tilt shake>translation shake.

[0100] As described above, according to the second embodiment, by using sensor shift stabilization to correct tilt shake in addition to rotation shake, image synthesis stabilization can be performed with less image quality degradation.

[0101] <Third Embodiment>

[0102] Next, a third embodiment of the present invention will be described.

[0103] As described in the first and second embodiments, when correcting camera shake by synthesizing a plurality of images, if the images are not properly aligned and are misaligned before synthesis, a ghost image will be generated. Therefore, in this embodiment, a case where the reliability of alignment is determined will be described.

[0104] Figure 8 is a diagram showing a schematic configuration of an imaging system according to a third embodiment. Figure 1B Compared with the configuration of the camera system shown in FIG. 1 , the difference is that the image processing unit 7′ further includes an image displacement calculation unit 74 and a reliability detection unit 75. The rest of the configuration is the same as that of the reference Figure 1A , Figure 1Band Figure 2 The configurations described are the same, so their descriptions will be omitted.

[0105] Fig. 9A and Fig. 9B is a diagram showing the configuration of the image sensor 6 in the third embodiment. Fig. 9A is an enlarged view of a portion of pixels 20 (4×4 pixels) of the image sensor 6 as seen from the direction of the optical axis 4, and Fig. 9B 2 is a diagram showing a cross-sectional view of one pixel 20 (lower part) and its correspondence with the exit pupil of the imaging lens. Note that the projection direction is Fig. 9B , but this illustration method is used to make it easier to understand the corresponding relationship state on the pupil plane.

[0106] exist Fig. 9A In the example shown, the image sensor 6 is covered by a color filter of a so-called Bayer array, and R, G, and B indicate the colors of the color filter, namely red, green, and blue, respectively. The circle in each pixel 20 indicates a microlens 25, and the vertical line in each pixel 20 indicates that the light receiving portion of each pixel 20 is divided into two light receiving portions 22a and 22b (photoelectric conversion portions) in the X direction. Arrows 21 indicate the distance between pixels covered by color filters of the same color.

[0107] exist Fig. 9B , reference numerals 23 a and 23 b denote regions on a pupil plane of the imaging optical system 3 corresponding to the light receiving portions 22 a and 22 b , respectively, and reference numeral 24 denotes an exit pupil of the imaging optical system 3 .

[0108] like Fig. 9B As shown, light receiving sections 22 a and 22 b of the pixel 20 are conjugated with partial areas 23 a and 23 b of the pupil plane of the imaging optical system 3 via microlenses 25 , respectively, and are configured to receive light beams passing through different pupil areas of the exit pupil 24 .

[0109] From the image sensor 6 having the above configuration, signals with parallax (or signals from different viewpoints) obtained by the light receiving parts 22a and 22b are acquired, and phase difference calculation can be performed to realize known phase difference AF. The calculation of phase difference AF is performed by the camera system control circuit 5. A normal pixel signal can be obtained by adding and outputting the signals obtained by the light receiving parts 22a and 22b for each pixel 20. Fig. 9B, regions 23a and 23b are shown with a gap therebetween to clearly indicate that they are different regions, but they may actually be in contact with each other or may have overlapping regions. Hereinafter, an image obtained by collecting pixel signals obtained from the light receiving portions 22a of each pixel 20 will be referred to as "image A", and an image obtained by collecting pixel signals obtained from the light receiving portions 22b of each pixel 20 will be referred to as "image B".

[0110] Next, we will refer to FIG. 10A to FIG. 10D The processing performed by the image displacement calculation unit 74 and the reliability detection unit 75 in the third embodiment is described.

[0111] Fig. 10A The horizontal axis in indicates time. Let t 1 is the time to start acquisition, with a predetermined period at t 2 and t 3 Images A and B are acquired continuously. Images A and B acquired at time t are designated as A 1 and B 1 Similarly, at time t n The acquired images A and B are designated as A n and B n .

[0112] Fig. 10B and Fig. 10C Shown at time t 2 The images A and B acquired at subsequent times are compared with the images A and B acquired at time t 2 An example of a loop path formed between previously acquired images A and B. Fig. 10B and Fig. 10C compared to, Fig. 10D Shown at time t 2 The images A and B acquired at subsequent times are compared with the images A and B acquired at time t 2 An example where no loop path is formed between previously acquired images A and B.

[0113] refer to FIG. 10B to FIG. 10D , consider at time t n-1 and t n Images A and B are obtained at FIG. 10B to FIG. 10D In time t n-1 The images A and B acquired at time t are used as templates for calculating motion vectors. n-1 The acquired image was rejected as a template at the previous time, then at time t n-1 Previously acquired images can be used as templates. Template rejection will be described later.

[0114] exist Fig. 10B In time t n At the timing of A, calculaten To B n The motion vector 30 from B n To B n-1 The motion vector 31, and the n-1 To A n As can be seen from the figure, by repeating the above motion vector calculation, since at time t n-1 The motion vector in the opposite direction has already been calculated, so it can be used as the motion vector from B after reversing its sign. n-1 To A n-1 The important thing here is that the four motion vectors 30, 31, 32 and 33 form the motion vector 32 of the moving object A. n and return to A without following the same path n In this embodiment, this is called "forming a loop path". Fig. 10B In the example of FIG. 1 , it is obvious that the image group (A n ,B n ) and at time t 2 The image group (A n-1 ,B n-1 ) form a circular path between them.

[0115] Similarly, in Fig. 10C In the example, at time t n At the timing of A, calculate n To B n The motion vector 30 from B n To A n-1 The motion vector 34, and the n-1 To A n The motion vector 56. Note that at time t n-1 The motion vector 35 is calculated at 30. As can be seen from the figure, these four motion vectors 30, 34, 35 and 36 form a cyclic path. The image displacement detection unit 54 of the present embodiment operates in this way to form a cyclic path.

[0116] like Fig. 10B and Fig. 10C As shown in the example of , various methods can be used to form the loop path, and any method can be set as desired. However, comparison between images A and between images B and between images acquired simultaneously is convenient because they are stable with respect to vignetting conditions and the state of the subject. In other words, Fig. 10B The example shown in is a better example.

[0117] Fig. 10D FIG. 2 shows the operation of image displacement detection which is usually performed. Fig. 10D In the example, byn The timing of A n and B n Add to generate the image (A n +B n Then, calculate the image (A n +B n ) and template image A n-1 +B n-1 The motion vector between Fig. 10D It is obvious that no path is formed that allows loops.

[0118] Next, the method using the loop path will be described. In the present embodiment, the reliability detection unit 75 accumulates the motion vectors that loop back to the original image. If there is no detection error, the accumulated result will be zero because it returns to the original position. On the other hand, if the detection fails due to some reason (such as movement of the subject, errors in the repeated pattern, or changes in ambient light, etc.), the result will not be zero.

[0119] Therefore, if the result of the reliability detection unit 75 is greater than the threshold value, it is determined that a detection error has occurred. This makes it possible to n The timing is determined at time t n The obtained motion vector is good or bad. When aligning and synthesizing images, it can be determined whether to add them based on the good or bad judgment at this time, which can save memory.

[0120] In addition, based on the comparison with the threshold, it can be determined whether to update the template. If it is equal to or less than the threshold, the time t n The image is used as a template for the image to be acquired next time, and if it is greater than the threshold, the template is not updated. This allows stable synthesis processing by excluding the image when an image of a single frame with different characteristics (for example, positioning failure due to flicker) is acquired.

[0121] In the above example, the light receiving portion of each pixel 20 is divided into two light receiving portions 22a and 22b in the X direction, but the present invention is not limited thereto. For example, it may be divided in the Y direction, or the division directions may be mixed. In addition, the number of divisions is not limited to two, and the number of divisions may be three or more. In either case, as described above, the reliability of the alignment may be determined by using a method utilizing motion vectors between images.

[0122] In the present embodiment, the imaging device is provided with the image displacement calculation unit 74 and the reliability detection unit 75, but the present invention is not limited thereto. As another configuration, for example, the image group A and the image group B may be stored using the optical system shown in the present embodiment and processed later. In this case, the image displacement calculation unit 74 and the reliability detection unit 75 are implemented in the form of a computer program or the like, and process the above-mentioned stored image group A and image group B. The processing content is as illustrated in the present embodiment, and the image displacement amount is calculated to obtain the image group A and the image group B at time t 2 And the image groups and time t acquired at subsequent times 2 A cyclic path is formed between the previous image groups, and the quantities detected along the above cyclic path are accumulated. By using this result, it is possible to n The timing is determined at time t n The quality of the calculated motion vector. In this case, as in the example of the camera device, memory can be saved and memory access can be reduced, allowing high-speed calculation.

[0123] As described above, according to the third embodiment, the reliability of alignment can be determined with a simple configuration.

[0124] <Fourth Embodiment>

[0125] Next, a fourth embodiment of the present invention will be described.

[0126] Fig.11A is a schematic external view of a smartphone 41 as an imaging device in the fourth embodiment, and Fig. 11B 2 is a block diagram mainly showing the configuration of the image processing unit 7″. The smartphone 41 in the fourth embodiment will be described as including a so-called multi-lens camera having two imaging optical systems 3a, 3b and two image sensors 6a, 6b. The imaging optical systems 3a, 3b have different focal lengths and can capture images using different viewing angles. Here, it is assumed that the imaging optical system 3b has a shorter focal length. In addition, since the imaging optical systems 3a and 3b are spaced apart, image groups can be captured from multiple viewpoints at the same time. Reference numerals 4a and 4b indicate optical axes of the imaging optical systems 3a and 3b, respectively.

[0127] Note that circuit configurations other than those described above are Figure 1B The circuit configuration shown is similar. Since the processing in the image processing unit 7" is different, this will be explained below. In this embodiment, the image processing unit 7" includes Figure 8 In addition to the configuration of the image processing unit 7′, it also includes a magnification adjustment unit 73.

[0128] In the present embodiment, the image obtained from the image sensor 6a is referred to as image A. On the other hand, the image obtained from the image sensor 6b is sent to the magnification adjustment unit 73 and converted into an image having a state equivalent to that of the image obtained from the image sensor 6a. Specifically, a part of the image obtained at a wider viewing angle is cropped, and processing is performed to match the pixel pitch. The output image of the magnification adjustment unit 73 is referred to as image B.

[0129] Then, by using the acquired images A and B, as in the third embodiment, reference FIG. 10A to FIG. 10D The images A and B are processed. That is, the image displacement calculation unit 74 calculates the image displacement amount so that a loop path is formed between the second image group and the subsequent image group and the previously acquired image group. Then, the reliability detection unit 75 accumulates the detection amount along the loop path. As a result, the same effect as in the third embodiment can be obtained.

[0130] In the present embodiment, the smartphone 41 has been described as having two imaging optical systems 3a, 3b and two image sensors 6a, 6b. However, the imaging systems may be three or more, in which case a required number of magnification adjustment units 73 are provided.

[0131] Furthermore, the present embodiment is not limited to application to smartphones, but may also be applied to electronic devices capable of processing images.

[0132] Furthermore, the present invention can be applied to a system composed of a plurality of devices, or an apparatus composed of a single device.

[0133] <Other embodiments>

[0134] The present invention can be implemented by supplying a program for implementing one or more functions of the above-mentioned embodiments to a system or device via a network or storage medium, and causing one or more processors in a computer of the system or device to read out and execute the program. The present invention can also be implemented by a circuit (e.g., ASIC) for implementing one or more functions.

[0135] The present invention is not limited to the above-described embodiments, and various changes and modifications can be made within the spirit and scope of the present invention. Therefore, in order to make the public aware of the scope of the present invention, the following claims are attached.

Claims

1. A camera device, comprising: An imaging component, used for performing photoelectric conversion on light incident through the imaging optical system and outputting an image; A shake detection component, used for detecting the shake of the camera device; as well as an actuating member for performing translational actuation and rotational actuation on the imaging member in a plane perpendicular to the optical axis of the imaging optical system to correct the vibration detected by the vibration detecting member, Here, when a plurality of images output from the imaging component are synthesized, the actuation component performs the rotational actuation in priority to the translational actuation.

2. The imaging device according to claim 1, wherein: In a case where the plurality of images are synthesized, the actuation section sets an upper limit value of the rotational actuation to be higher than an upper limit value of the translational actuation.

3. The imaging device according to claim 1 or 2, wherein: The actuation section sets an upper limit value of the rotational actuation to be higher when the plurality of images are synthesized than when the synthesis is not performed.

4. A camera device, comprising: An imaging component, used for performing photoelectric conversion on light incident through the imaging optical system and outputting an image; A shake detection component, used for detecting the shake of the camera device; as well as an actuating member for performing translational actuation and rotational actuation on the imaging member in a plane perpendicular to the optical axis of the imaging optical system to correct the vibration detected by the vibration detecting member, Here, when a plurality of images are synthesized, the actuation member sets an upper limit value of the rotational actuation to be higher than an upper limit value of the rotational actuation when the synthesis is not performed.

5. The imaging device according to any one of claims 1 to 4, further comprising: A setting component is used to set whether to synthesize the multiple images. The setting component sets to combine the plurality of images when a user instructs to combine the images or when a shutter speed is slower than a threshold in a still image shooting mode.

6. The imaging device according to any one of claims 1 to 5, further comprising: A setting component is used to set whether to synthesize the multiple images. The setting component is configured not to synthesize images when the user does not instruct to synthesize images, when the shutter speed is faster than a threshold in the still image shooting mode, or when the image shooting mode is a moving image shooting mode.

7. The imaging device according to any one of claims 1 to 6, wherein: The actuating member is also tilt-actuated in a rotational direction around an axis perpendicular to the optical axis, and In the case of synthesizing the multiple images, the actuating component causes the rotation actuation and the tilt actuation to take precedence over the translation actuation, and in the case where the focal length of the imaging optical system is shorter than a predetermined focal length, causes the tilt actuation to take precedence over the rotation actuation, and in the case where the focal length is equal to or longer than the predetermined focal length, causes the rotation actuation to take precedence over the tilt actuation.

8. The imaging device according to any one of claims 1 to 7, further comprising: an alignment component for aligning the plurality of images when the plurality of images output from the camera component are synthesized; as well as A synthesis component is used to synthesize the multiple images aligned by the alignment component.

9. The imaging device according to claim 8, wherein: The imaging means is capable of outputting a first image and a second image having different viewpoints based on light incident via the imaging optical system, and the imaging apparatus further includes: a calculation unit configured to calculate a motion vector indicating a mutual deviation between a plurality of the first images obtained at a first timing and a second timing prior to the first timing and a plurality of the second images obtained at the first timing and the second timing aligned by the alignment unit; and a reliability detection unit for detecting the reliability of the alignment using the alignment unit by accumulating the motion vectors obtained by the calculation unit, Here, when the reliability determined by the reliability detection unit is lower than a predetermined threshold, the first image and the second image obtained at the first timing are not used for synthesis by the synthesis unit.

10. The imaging device according to claim 9, wherein: The calculation unit forms a cyclic path of the motion vector between the plurality of the first images and the plurality of the second images aligned by the alignment unit.

11. The imaging device according to claim 10, wherein: the calculation means calculating a first motion vector from the first image obtained at the first timing and aligned by the alignment means to the second image obtained at the first timing and aligned by the alignment means, a second motion vector from the second image obtained at the first timing to the second image obtained at the second timing, a third motion vector from the second image obtained at the second timing to the first image obtained at the second timing, and a fourth motion vector from the first image obtained at the second timing to the first image obtained at the first timing, and The reliability detection section detects the reliability by accumulating the first motion vector, the second motion vector, the third motion vector, and the fourth motion vector.

12. The imaging device according to claim 9, wherein: the calculation means calculating a first motion vector from the first image obtained at the first timing and aligned by the alignment means to the second image obtained at the first timing and aligned by the alignment means, a second motion vector from the second image obtained at the first timing to the first image obtained at the second timing, a third motion vector from the first image obtained at the second timing to the second image obtained at the second timing, and a fourth motion vector from the second image obtained at the second timing to the first image obtained at the first timing, and The reliability detection section detects the reliability by accumulating the first motion vector, the second motion vector, the third motion vector, and the fourth motion vector.

13. The imaging device according to any one of claims 9 to 12, wherein: The imaging section includes an image sensor having a plurality of pixels, each pixel including a first photoelectric conversion unit and a second photoelectric conversion unit that photoelectrically convert light having passed through different pupil areas of the imaging optical system, and The first image corresponds to an image signal photoelectrically converted by the first photoelectric conversion unit, and the second image corresponds to an image signal photoelectrically converted by the second photoelectric conversion unit.

14. The imaging device according to any one of claims 9 to 12, wherein: The imaging section includes a first image sensor and a second image sensor that photoelectrically convert light incident via a plurality of imaging optical systems, and The first image corresponds to an image signal photoelectrically converted by the first image sensor, and the second image corresponds to an image signal photoelectrically converted by the second image sensor.

15. An image processing device, further comprising: an acquisition unit for acquiring, from the imaging unit, a first image and a second image having different viewpoints obtained based on light incident via the imaging optical system; an aligning component, configured to align a plurality of the first images acquired by the acquiring component at a first timing and a second timing prior to the first timing, and to align a plurality of the second images acquired by the acquiring component at the first timing and the second timing; A synthesis component, used for synthesizing the plurality of first images and the plurality of second images aligned by the alignment component; a calculation unit configured to calculate a motion vector indicating a displacement amount between the plurality of first images and the plurality of second images aligned by the alignment unit; as well as a reliability detection unit for detecting the reliability of the alignment using the alignment unit by accumulating the motion vectors obtained by the calculation unit, Here, when the reliability detected by the reliability detection unit is lower than a predetermined threshold, the first image and the second image obtained at the first timing are not used for synthesis by the synthesis unit.

16. A method for controlling a camera device, comprising: An imaging step for performing photoelectric conversion on light incident through an imaging optical system by an imaging component and outputting an image; A shaking detection step, used to detect the shaking of the camera device; as well as an actuation step for performing translation actuation and rotation actuation on the imaging component in a plane perpendicular to the optical axis of the imaging optical system by an actuation component to correct the vibration detected by the vibration detection step, In the case where a plurality of images outputted in the photographing step are synthesized, in the actuating step, the rotation actuation is performed with priority over the translation actuation.

17. A method for controlling a camera device, comprising: An imaging step for performing photoelectric conversion on light incident through an imaging optical system by an imaging component and outputting an image; A shaking detection step, used to detect the shaking of the camera device; as well as an actuation step for performing translation actuation and rotation actuation on the imaging component in a plane perpendicular to the optical axis of the imaging optical system to correct the vibration detected by the vibration detection step, Here, in a case where a plurality of images are synthesized, an upper limit value of the rotational actuation in the actuation step is set higher than an upper limit value of the rotational actuation in a case where the synthesis is not performed.

18. An image processing method, comprising: An acquisition step for acquiring, from an imaging component by an acquisition component, a first image and a second image having different viewpoints obtained based on light incident via an imaging optical system; an alignment step for, in a case where the plurality of first images and the plurality of second images obtained from the imaging means are synthesized, aligning, by an alignment means, the plurality of first images obtained by the acquisition means at a first timing and a second timing prior to the first timing, and aligning, by an alignment means, the plurality of second images obtained by the acquisition means at the first timing and the second timing; a synthesis step for synthesizing the plurality of first images and the plurality of second images aligned by the alignment component by means of a synthesis component when synthesizing the plurality of first images and the plurality of second images; a calculation step for calculating, by a calculation means, a motion vector indicating an amount of displacement between the plurality of first images and the plurality of second images aligned in the alignment step; as well as a reliability detection step for detecting the reliability of the alignment using the alignment component by accumulating the motion vectors obtained in the calculation step, through a reliability detection component, Wherein, when the reliability detected in the reliability detection step is lower than a predetermined threshold, the first image and the second image obtained at the first timing are not used for synthesis in the synthesis step.

19. A program for causing a computer to function as each component of the image processing apparatus according to claim 15.

20. A program for causing a computer to execute each step of the control method of the imaging apparatus according to claim 16 or 17.

Citation Information

Patent Citations

  • Camera shake correction device and digital camera

    JP2012242563A