Imaging device, electronic device, imaging device control method, electronic device control method, and program

Abstract

To display an appropriate image at an output destination in an imaging device capable of outputting an image to a display unit of the imaging device and also to a part other than the display unit.SOLUTION: An imaging device 100 having an imaging element 111 that captures an image, and a display unit 121 that displays the image includes an image processing unit 116 that performs roll correction on the image, a control unit 140 that controls the output of the image, and a display control unit 122 that displays the image on the display unit 121, and the image processing unit 116 performs roll correction such that the degree of roll correction when the image is output only to the display unit 121 is smaller than the degree of the roll correction when the image is output to other than the display unit 121.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] There is a technology for displaying an image captured by a camera on a display separate from the camera's display device so that it can be viewed by a primary observer other than the photographer. For example, the photographer wears a camera on his or her head, and the image captured by the camera is displayed on a display viewed by a secondary observer. In this case, if the photographer tilts his or her head, the subject displayed on the display viewed by the observer will also tilt. Therefore, to ensure that the subject is displayed appropriately on the display, it is necessary to minimize the change in the orientation of the subject image displayed on the display even when the photographer tilts his or her head. Patent Document 1 discloses an image processing device that performs image tilt correction on an image displayed on a monitor device viewed by a secondary observer when an image captured by a camera worn on the observer's head is displayed on the monitor device. In Patent Document 1, the observer wearing the camera visually observes objects in the real world as see-through images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-160898 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, an observer wearing a camera observes a subject visually, and the case where a photographer observes the captured image on a display or electronic viewfinder, as in the case of electronic monoculars and electronic binoculars, is not taken into consideration. Therefore, in an imaging device that can output the captured image to both the camera's display unit where the main observer views the image and a display other than the camera's display unit where another observer views the image, it is necessary to display an appropriate image that does not feel strange to the photographer.

[0005] An object of the present invention is to enable an appropriate image to be displayed at an output destination in an imaging device that has a display unit and that can output an image to a unit other than the display unit. [Means for solving the problem]

[0006] In order to solve the above problem, an imaging device of the present invention is an imaging device having an imaging section for capturing an image and a display section for displaying the image, Image blur The image processing device includes a correction unit that performs correction, an output unit that controls output of the image, and a display control unit that displays the output image on the display unit. Among the pitch correction, yaw correction, and roll correction in the image stabilization, The roll correction is performed so that the correction degree of the roll correction when the output means outputs an image only to the display unit is smaller than the correction degree of the roll correction when the output means outputs an image to other units besides the display unit. and performing the pitch correction and the yaw correction when the output means outputs an image only to the display unit and when the output means outputs an image to other units as well. cormorant. [Effects of the Invention]

[0007] According to the present invention, in an imaging device that has a display unit and is capable of outputting images to a unit other than the display unit, it is possible to display an appropriate image at the output destination. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is a perspective view of the appearance of the imaging device. [Figure 2] FIG. 1 is a diagram illustrating a configuration of an imaging device. [Figure 3] FIG. 10 is a diagram illustrating roll correction. [Figure 4] 10 is a flowchart showing a process of switching roll correction. [Figure 5] FIG. 10 is a diagram showing an example of a display indicating horizontality. [Figure 6] 10 is a flowchart showing a process for switching between roll corrections and a process for displaying a display indicating the horizontal; [Figure 7] 10 is a flowchart showing a process of switching between roll corrections and a process of switching between sound collection methods. DETAILED DESCRIPTION OF THE INVENTION

[0009] (First embodiment) The configuration of an imaging device 100 as an example of an electronic device to which the present invention can be applied will be described with reference to Figs. 1 and 2. Fig. 1 is an external perspective view of the imaging device 100. Fig. 1(A) is an external perspective view of the imaging device 100 as seen from the front side (subject side), and Fig. 1(B) is an external perspective view as seen from the rear side opposite the subject side. Fig. 1(C) is a diagram showing the imaging device 100 with a terminal cover 105 opened. Fig. 2 is a diagram illustrating the configuration of the imaging device 100.

[0010] The imaging device 100 is an example of an electronic device that has an imaging element and a display unit integrated together, such as an electronic monocular, electronic binoculars, or digital camera. The imaging device 100 has a side cover 101, a rear panel 102, and a front panel 103, which form a body housing. The front panel 103 has an opening 103a, through which a subject image is incident on an imaging optical system 110. The subject image is formed on an imaging element 111, which serves as an imaging unit, through the imaging optical system 110. An optical axis A of a photographing lens of the imaging optical system 110 faces in the extension direction of the imaging device 100. Note that, in this embodiment, an example will be described in which the main body of the imaging device and the lens device are integrated, but this is not limiting, and the lens device may be detachable from the main body.

[0011] Two microphone holes 103b are provided on the front panel 103, facing the subject side. A corresponding microphone 150 is provided in each of the two microphone holes 103b. The microphone holes 103b guide external sound to the microphones 150. The imaging device 100 is capable of collecting sound in monaural, and can also collect sound in stereo using the two microphones 150.

[0012] A terminal cover 105 is attached to the side of the side cover 101 and is openably and closably joined to the imaging device 100. FIG. 1(B) shows the terminal cover 105 in a closed state, and FIG. 1(C) shows the terminal cover 105 in an open state. An I / O unit 143 and a connection terminal 131 are provided inside the terminal cover 105 of the side cover 101. The I / O unit 143 is an input / output connector unit for inserting a removable data storage medium 130 such as an SD card and exchanging information. The connection terminal 131 is a connection terminal 131 such as a USB connection terminal for exchanging information and power.

[0013] There are five operation buttons 104 on the top surface of the side cover 101. The five operation buttons 104 are a power button 104a, a zoom button 104b, a release button 104c, a video recording button 104d, and a setting change button 104e. The power button 104a is an operation button for turning the power of the imaging device 100 on and off. The zoom button 104b is an operation button for performing a zoom operation to change the imaging magnification of the imaging device 100. The release button 104c is an operation button for moving a focus adjustment group such as a zoom lens in the imaging optical system 110 to focus on a subject and for performing a release to capture a still image. The video recording button 104d is an operation button for starting and stopping video recording. The setting change button 104e is an operation button for changing various settings of the imaging device 100, and pressing the setting change button 104e displays a menu related to the settings. Various settings can be selected and switched using the zoom button 104b, release button 104c, video recording button 104d, and setting change button 104e. Note that the shapes, functions, and layout of the operation buttons shown here are merely examples, and they may have combined functions with other operation buttons or may be arranged in a different manner.

[0014] The lower part of the side cover is provided with a diopter adjustment dial 124 and a sound hole 106. The diopter adjustment dial 124 is a dial for adjusting the focus of the diopter adjustment means. The sound hole 106 transmits sound from a speaker 151.

[0015] The rear panel 102 is provided with an eyepiece window 120 and an eyepiece detection sensor 123. A display is located inside the eyepiece window 120. The display is an electronic viewfinder equipped with a display unit 121, an eyepiece group (not shown), and a diopter adjustment unit. The display unit 121 is a display such as an organic electroluminescence (EL) or liquid crystal display (LCD). The eyepiece group enlarges the display on the display unit 121 to make it easier for the photographer to view. The photographer can view the display at the most comfortable position by aligning their line of sight with the eyepiece optical axis B of the eyepiece group and then aligning their eyes with eyepoint C of the eyepiece group on the eyepiece optical axis B. The photographing lens optical axis A and the eyepiece optical axis B of the imaging device 100 are parallel. The diopter adjustment unit can adjust the focal position to suit the photographer's eyesight. The eyepiece detection sensor 123 is located above the eyepiece window 120 and detects when the photographer is looking through the eyepiece window 120. In this embodiment, an example will be described in which the display means is an electronic finder, but the display means is not limited to this and may be a display or the like.

[0016] Next, we will explain the electrical configuration of the imaging device 100. The imaging device 100 includes a control unit 140, a ROM 141, a RAM 142, and an I / O unit 143, and is equipped with a removable data storage medium 130. The control unit 140 is a CPU (Central Processing Unit) that controls the entire imaging device 100. A ROM (Read Only Memory) 141 is connected to the control unit 140, and the control unit 140 performs various controls of the imaging device 100 based on a control program stored in the ROM 141.

[0017] The ROM (Read Only Memory) 141 is a non-volatile memory that stores a control program. The RAM (Random Access Memory) 142 is a volatile memory. The RAM 142 is a temporary storage area such as a main memory and a work area for the operation of the control unit 140. For example, the RAM 142 has an image development area, a work area, a VRAM (Video RAM), a temporary save area, etc. The image development area is used as a temporary buffer for temporarily storing captured image data sent from the image processing unit 116 and compressed and converted image data read from the data storage medium 130. The image development area is also used as an image-only work area for image compression processing and decompression processing. The work area is an area for executing various programs. The VRAM is an area for storing display data to be displayed on the display unit 121. The temporary save area is an area for temporarily saving various data.

[0018] The data storage medium 130 is a removable recording medium such as an SD card or HDD. Converted and compressed captured image data and video data are stored in file format in the data storage medium 130. The I / O unit 143 controls the exchange of data with the data storage medium 130 based on instructions from the control unit 140.

[0019] The imaging device 100 also includes an imaging optical system 110, an imaging element 111, a shutter control unit 112, a zoom control unit 113, a focus control unit 114, an imaging element control unit 115, and an image processing unit 116. The imaging optical system 110 includes multiple lenses, such as a zoom lens and a focus lens, an aperture, and a shutter. The aperture is a light amount adjusting unit that controls the amount of light passing through the imaging optical system 110. The imaging optical system 110 forms an optical image of a subject on the imaging element 111. The imaging element 111 is a photoelectric conversion element, such as a CMOS or CCD, and is an imaging unit that outputs an output signal (analog signal) corresponding to the optical image. If the imaging element 111 is a CMOS, it can output thinned pixel data in the horizontal and vertical directions in accordance with a resolution conversion instruction from the imaging element control unit 115.

[0020] The image sensor control unit 115 has a timing generator, a signal processing circuit, and an A / D conversion circuit, and controls the image sensor 111 and the output signal from the image sensor 111. The timing generator supplies a transfer clock signal and a shutter signal to the image sensor 111. The signal processing circuit performs noise removal and gain processing on the output signal output from the image sensor 111. The A / D conversion circuit converts an analog signal into a digital signal. The specifications for digital signal conversion by the A / D conversion circuit can be changed in various ways depending on the specifications of the image sensor, etc.

[0021] The image processing unit 116 performs various image processing operations, such as gamma conversion, color space conversion, white balance adjustment, and exposure adjustment, on the digital signal output from the image sensor control unit 115. Furthermore, the image processing unit 116 performs image processing operations, such as pitch correction, yaw correction, and roll correction, to correct image shake and rotation based on the attitude information of the image capture device 100 obtained by the attitude sensor 125. Image shake corrections, such as pitch correction, yaw correction, and roll correction, are electronically performed by the image processing unit 116 by, for example, cropping an image. The pitch correction, yaw correction, and roll correction performed by the image processing unit 116 may be performed based on subject movement information obtained from captured image data. Furthermore, the pitch correction, yaw correction, and roll correction may be performed by controlling the imaging plane of the image sensor 111 or by shifting a shift lens in the imaging optical system 110. The pitch correction, yaw correction, and roll correction performed by the image processing unit 116, including whether or not to perform the correction processing, are controlled by the control unit 140.

[0022] The shutter control unit 112 drives the shutter of the imaging optical system 110 based on instructions from the control unit 140. The zoom control unit 113 drives the zoom lens of the imaging optical system 110 based on instructions from the control unit 140. The focus control unit 114 drives the focus lens of the imaging optical system 110 based on instructions from the control unit 140. The control unit 140 calculates how far the focus lens needs to be driven from its current position to achieve focus (or whether focus was achieved at the current position of the imaging optical system 110), and instructs the focus control unit 114 on the amount of focus lens drive. The focus control unit 114 performs a focusing operation by moving the focus lens according to the drive amount instructed by the control unit 140. Note that although a method of driving a lens has been described for the focusing operation, the same effect can be achieved by changing the relative position of the image sensor 111 and the imaging optical system 110, and therefore a method of driving the image sensor 111 back and forth along the imaging lens optical axis A may also be used.

[0023] The imaging device 100 includes a DC / DC converter 145 and a battery 146. The DC / DC converter 145 generates voltage from power supplied from the battery 146 and supplies it to each element including the control unit 140. The DC / DC converter 145 controls the start and stop of each voltage supply based on a control signal from the control unit 140. The battery 146 is a primary battery or a rechargeable secondary battery, and supplies power to the DC / DC converter 145. Note that it is sufficient that power is supplied to the DC / DC converter 145; it does not have to be supplied from a battery. The DC / DC converter 145 may be supplied with power from an external power source such as a mobile battery connected to the connection terminal 131, for example.

[0024] The imaging device 100 has a power button 104a, a zoom button 104b, a release button 104c, a video capture button 104d, and a setting change button 104e. Each button is connected to a control unit 140. The control unit 140 starts when the power button 104a is turned on, and stops when the power button 104a is turned off. The control unit 140 controls a DC / DC converter 145 in response to the operation of the power button 104a.

[0025] Operating the zoom button 104b changes the zoom position of the imaging optical system 110. The control unit 140 controls the zoom control unit 113 in response to the operation of the zoom button 104b, thereby driving the zoom lens of the imaging optical system 110 to change the zoom position of the imaging optical system 110. The release button 104c has two switch positions depending on the position at which it is pressed. When the first position (SW1 ON) is detected, camera settings such as white balance, photometry, and focus are locked. When the second position (SW2 ON) is detected, imaging is performed and a subject image signal is captured. The control unit 140 controls the focus control unit 114, shutter control unit 112, and other components in response to control of the release button 104c. Operating the video capture button 104d starts / ends capturing of a subject moving image signal and sound recording. Pressing the video capture button 104d a first time starts capturing of a subject moving image signal and sound recording, and pressing the video capture button 104d a second time ends capturing of a subject moving image signal and sound recording. In response to operation of the video capture button 104d, the control unit 140 instructs the image sensor control unit 115 to capture a video and instructs the microphone 150 to record sound.

[0026] The imaging device 100 includes a display unit 121 and a display control unit 122. The display unit 121 has a display device such as an organic EL or liquid crystal display, and displays images (including still images and moving images) captured by the imaging element 111 and information related to the image capture. The display unit 121 is controlled by the display control unit 122. The display control unit 122 adjusts the display size, brightness, etc. of the image in accordance with instructions from the control unit 140, and controls the display on the display unit 121. The display control unit 122 controls the information to be superimposed and displayed on the image.

[0027] The display control unit 122 also has an eye proximity detection sensor 123. The eye proximity detection sensor 123 is provided near the display unit 121, and when the photographer looks into the display unit 121, the eye proximity detection sensor 123 detects that the photographer is approaching at a predetermined distance or less, and turns on the display unit 121. Conversely, when the eye proximity detection sensor 123 detects that the photographer has moved away at a predetermined distance or more, the eye proximity detection sensor 123 turns off the display unit 121. This operation has the effect of reducing power consumption of the imaging device and suppressing heat generation.

[0028] The imaging device 100 is equipped with a microphone 150 for collecting external sound. The imaging device 100 of this embodiment is equipped with two microphones 150, and is capable of collecting sound in stereo as well as monaural. The control unit 140 synchronizes the audio data obtained by the microphones 150 with the video data, and records them on the data storage medium 130 as a single video data.

[0029] The imaging device 100 includes a speaker 151 and an audio output terminal 152. The speaker 151 is an electro-acoustic transducer that can output sounds such as electronic sounds and audio included in video data. The audio output terminal 152 is an audio output terminal such as an earphone jack to which headphones or the like can be connected. The audio output terminal 152 can output not only monaural audio but also stereo audio to the connected headphones or the like. The audio output terminal 152 may be a connection terminal 131 such as a USB connection terminal, for example.

[0030] When video data is played back in imaging device 100, video data recorded on data storage medium 130 is selected. When video data is played back, the video data included in the video data is displayed on display unit 121 by display control unit 122. At the same time, audio data included in the video data is played back and output from speaker 151 or audio output terminal 152 by control unit 140 in synchronization with the video data.

[0031] The imaging device 100 has a communication unit 144 that connects to external devices such as other imaging devices, smartphones, and display devices to exchange data and power. The communication unit 144 may be a wired connection terminal 131 such as a USB, or a wireless connection means such as Wi-Fi or Bluetooth (registered trademark). Using the wired connection terminal 131 allows not only data exchange but also power exchange. On the other hand, in the case of a wireless connection means, data exchange is possible by connecting to an external device. The control unit 140 communicates with the connected external device via the communication unit 144 and controls the output of video data to the external device. Note that the control to output video data to the external device may be performed only when the external device is a display device or an electronic device with a display unit, such as an imaging device or smartphone.

[0032] The control unit 140 controls the entire imaging device 100. The control unit 140 controls various image processes, including roll correction, in the image processing unit 116. The control unit 140 also reads captured image data output from the image processing unit 116, transfers it to the RAM 142, and outputs it to each output destination. Destinations for captured image data include the display unit 121, the data storage medium 130, and an external device. The control unit 140 transfers the captured image data from the RAM 142 to the display unit 121 via the display control unit 122. The control unit 140 also stores the captured image data in the data storage medium 130 via the I / O unit 143. The control unit 140 also outputs data to an external device via the communication unit 144. Note that if the captured image data is video data, audio data is also output along with the video data. The control unit 140 also controls the method of collecting sound by the microphone 150 when recording video data.

[0033] Next, roll correction will be described. Fig. 3 is a diagram for explaining roll correction. In the following, a person who is shooting a subject 301 with the image capture device 100 will be called a primary observer, and a person who is watching the video captured by the image capture device 100 on a display device other than the image capture device 100 will be called a viewer.

[0034] FIG. 3A is a diagram showing an example of a screen display when the primary observer is observing the subject 301 with the imaging device 100. That is, FIG. 3A is an example of a screen displayed on the display unit 121 of the imaging device 100 without roll correction. The subject 301 is standing vertically on a horizontal ground. Screen 302 is an example of a screen display when the primary observer holds the imaging device 100 horizontally. Screen 303 is an example of a screen display when the primary observer holds the imaging device 100 at an angle. When the imaging device 100 is tilted, the screen tilts, but the tilt of the subject 301 remains the same as the actual tilt of the subject 301. Therefore, the tilt of the subject 301 observed by the primary observer through the imaging device 100 is the same as the actual tilt of the subject 301, regardless of whether the imaging device 100 is held flat or tilted. In this case, the primary observer does not feel uncomfortable even if roll correction processing is not performed on the image displayed on the display unit 121 in accordance with the tilt of the imaging device 100. That is, when the main observer is observing the subject with the image capture device 100, roll correction is not necessary.

[0035] 3(B) is a diagram showing an example of a screen display when video captured by imaging device 100 is displayed on a display device other than imaging device 100 without roll correction. Screen frame 305 is the screen frame of a horizontally placed display device. The video displayed on screen frame 305 in FIG. 3(B) is the video when imaging device 100 is held tilted (screen 303 in FIG. 3(A)). On the screen of the horizontal display device, subject 301 tilts in the opposite direction by the amount of tilt of screen frame 303 of imaging device 100. Therefore, the tilt of subject 301 displayed on screen frame 305 with respect to the horizontal direction is different from the actual tilt of subject 301, which gives the viewer a sense of discomfort.

[0036] Fig. 3(C) is a diagram showing an example of a screen display when video that has undergone roll correction processing is displayed on a display device other than imaging device 100. The video displayed in screen frame 305 in Fig. 3(B) is video obtained by performing roll correction according to the tilt of imaging device 100 on the video captured when imaging device 100 is held tilted (screen 303 in Fig. 3(A)). By performing roll correction according to the tilt of imaging device 100, the tilt of subject 301 displayed on the display device becomes the same as the actual tilt of subject 301, allowing the viewer to view the video without feeling uncomfortable.

[0037] Fig. 3(D) is a diagram showing an example of a screen display when an image that has undergone roll correction processing is displayed on display unit 121 of imaging device 100. That is, Fig. 3(D) is a diagram showing a case where the roll-corrected image of Fig. 3(C) is displayed on display unit 121 of imaging device 100 that is held in an inclined state. The inclination of subject 301 displayed on screen 303 of imaging device 100 that is held in an inclined state will be different from the inclination of subject 301, which will give the primary observer an uncomfortable feeling.

[0038] When the imaging device 100 is held horizontally, neither the primary observer viewing the display on the imaging device 100 nor the viewer viewing the display on an external display device feels uncomfortable. However, it is difficult to always maintain the imaging device 100 horizontal when the imaging device 100 is handheld. Furthermore, it becomes difficult to shoot at a free angle, such as by leaning to the side. When the primary observer is shooting at a free angle with the imaging device 100 and simultaneously outputting the captured video to another display device, roll correction can be performed to ensure an appropriate display. However, if roll correction is performed, the video viewed by the primary observer and displayed on the imaging device 100 will also be roll corrected, causing the primary observer to feel uncomfortable. For this reason, it is preferable not to perform roll correction all the time, and not to perform roll correction when only the primary observer is observing. Therefore, in this embodiment, roll correction is not performed when only viewing video captured in real time by the imaging device 100, but is performed when using a display device other than the imaging device 100 or when recording the recorded video for later viewing.

[0039] Next, the roll correction switching process in this embodiment will be described. As described above, in this embodiment, roll correction is not performed when the photographer simply views the video displayed on the display unit 121 of the imaging device 100 in real time, i.e., when the output destination of the captured video is only the display unit 121. On the other hand, roll correction is performed when the viewer views the video displayed on an external device or checks the recorded video later, i.e., when the output destination of the captured video includes devices other than the display unit 121, such as a display device other than the imaging device 100 or a recording medium of the imaging device 100. Therefore, in this embodiment, a process is performed to switch roll correction on and off depending on whether or not the video is output to devices other than the display unit 121 of the imaging device 100.

[0040] Fig. 4 is a flowchart showing the roll correction switching process. Each process shown in Fig. 4 is realized by the control unit 140 executing a program stored in a readable storage medium such as the ROM 141. This process is started when the power button 104a is pressed to turn on the power of the imaging device 100. First, in step S401, the control unit 140 sets the yaw and pitch correction by the image processing unit 116 to on. Then, the process proceeds to step S402.

[0041] In step S402, the control unit 140 determines whether or not video is being output to devices other than the display unit 121 of the imaging device 100. The control unit 140 determines whether or not video is being output to devices other than the display unit 121 of the imaging device 100 based on whether or not video is being output to an external device or a recording medium. Specifically, the control unit 140 communicates with the connected external device via the communication unit 144 and determines whether or not the external device is an electronic device having a display unit. If the external device is an electronic device having a display unit, the control unit 140 determines that video is being output. The control unit 140 also determines whether or not video data is being recorded to the data storage medium 130. Even if video data is being recorded to the data storage medium 130, the control unit 140 determines that video is being output to devices other than the display unit 121. Furthermore, even if a predetermined mode, such as a video shooting mode for shooting video, is selected by switching modes or pressing the video shooting button 104d, the control unit 140 determines that video is being output to devices other than the display unit 121, and determines that video is being output. If video is being output to devices other than the display unit 121 of the imaging device 100, the process proceeds to step S403. On the other hand, if there is no video output to anything other than the display unit 121 of the imaging device 100, the process proceeds to step S405.

[0042] If there is video output to devices other than the display unit 121 of the imaging device 100, then in step S403 the control unit 140 sets roll correction to ON. When roll correction is set to ON, the image processing unit 116 performs roll correction processing on the captured video. As a result, roll correction is performed on video that is output to devices other than the display unit 121. Then, the process proceeds to step S404. On the other hand, if there is no video output to devices other than the display unit 121 of the imaging device 100, then in step S405 the control unit 140 sets roll correction to OFF. As a result, roll correction is not performed on video that is output only to the display unit 121. Then, the process proceeds to step S404.

[0043] In step S404, the control unit 140 detects the on / off state of the power button 104a. If it is detected that the switch is off, this process ends. On the other hand, if it is detected that the switch is on, the process returns to step S402, and in order to determine whether to switch the roll correction, it is determined whether there is video output to anything other than the display unit 121.

[0044] As described above, according to this embodiment, roll correction is performed when video is output to a device other than the display unit of the imaging device, and roll correction is not performed when video is output only to the display unit of the imaging device. This allows viewers viewing video captured by the imaging device on an external device, etc., to view video with minimal discomfort, regardless of the tilt of the imaging device. Furthermore, when no one other than the photographer is viewing the video, roll correction is not performed, allowing the display unit 121 to display video that is minimally uncomfortable for the photographer, in which the actual tilt of the subject matches the tilt of the subject in the video displayed on the display unit 121. This allows observation at any position or angle when the imaging device 100 is used as a monocular or binocular without video output. Note that this embodiment describes an example in which roll correction is not performed when video is output only to the display unit of the imaging device. However, roll correction does not need to be completely turned off, as this can be effective if it reduces the discomfort felt by the photographer. In other words, control may be performed so that the degree of roll correction is smaller when proceeding to step S405 than when proceeding to step S403. The degree of roll correction refers to the amount of roll shake correction for the roll shake that has occurred. The smaller the amount of roll shake correction when the same roll shake has occurred, in other words, the larger the remaining roll shake, the smaller the degree of roll shake correction. For example, when a photographer is looking through the eyepiece window 120, while the photographer's head is fixed, the hand holding the imaging device may shake slightly, and the small roll shake may cause the photographer discomfort. Therefore, when proceeding to step S405, the degree of correction may be reduced by correcting only the high-frequency components of the roll shake that is the target of correction in step S403, without correcting the low-frequency components. In addition to reducing the degree of correction by applying a smaller gain than in step S403, the shake components in a narrower frequency band than in step S403 may be corrected in this way.

[0045] (Second embodiment) In the first embodiment, an example was described in which roll correction is also performed on the image output to the display unit 121 when the image is output to a device other than the display unit 121. In this case, the inclination of the subject on the image on the display unit 121 that has undergone roll correction differs from the inclination of the real subject, and the photographer (primary observer) viewing both images feels uncomfortable. Therefore, in the second embodiment, an example will be described in which roll correction is performed only on the image that is output.

[0046] When roll correction is performed only on the output video, both the photographer using the imaging device 100 and the viewer viewing the output video can observe the subject at a tilt that feels natural. On the other hand, if the tilt of the imaging device 100 is particularly large, and roll correction of the output video is performed by image cropping, the output video may be excessively cropped. In order to prevent excessive cropping of the output video due to roll correction, it is necessary to suppress the tilt of the imaging device 100. Therefore, in this embodiment, the tilt of the imaging device 100 is suppressed by displaying an indication of the tilt of the imaging device on the display unit 121, thereby informing the photographer who is shooting while looking at the display unit 121 of the tilt of the imaging device 100.

[0047] Fig. 5 is a diagram showing an example of a display indicating the horizontal that is displayed on the display unit 121. Fig. 5 shows an example of an image displayed on the display unit 121 of the imaging device 100 in this embodiment, in which a subject 501 is captured while the imaging device 100 is held at an angle. In this embodiment, roll correction is not performed on the image displayed on the display unit 121, and therefore roll correction is not performed on the example of the image displayed in Fig. 5 either. Therefore, the screen frame 502 of the display unit 121 is also tilted in accordance with the tilt of the imaging device 100.

[0048] A display 503 indicating the horizontal is displayed within the screen frame 502. The display 503 indicating the horizontal is a straight line that is horizontal with respect to the screen frame 502. The photographer can recognize the tilt of the image capturing device 100 by comparing the display 503 indicating the horizontal with the tilt of the subject 501. Note that the display 503 indicating the horizontal is an example of a display indicating the tilt of the image capturing device. The display indicating the tilt of the image capturing device is not limited to a straight line that is horizontal with respect to the screen frame 502, as long as it allows the photographer to recognize the tilt of the image capturing device 100. For example, a spirit level may be displayed as an display indicating the horizontal direction. Furthermore, the display indicating the tilt of the image capturing device may display the screen frame of the image after roll correction, or may display the direction and amount of roll correction. Furthermore, the display indicating the tilt of the image capturing device may attenuate roll correction.

[0049] Furthermore, if the tilt of the imaging device 100 exceeds a predetermined tilt, a warning may be issued. The warning may be, for example, a display of a mark or the like, the generation of a warning sound, or a change in color of the display 503 indicating the horizontal. By displaying a display indicating the tilt of the imaging device on the image displayed on the display unit 121, the photographer using the imaging device 100 can know how much the imaging device 100 is tilted. By notifying the photographer of how much the imaging device 100 is tilted, the tilt of the imaging device 100 can be suppressed.

[0050] Fig. 6 is a flowchart showing the process of switching roll correction and the process of displaying a display indicating horizontality. The processes shown in Fig. 6 are realized by the control unit 140 and the display control unit 122 executing a program stored in a readable storage medium such as the ROM 141. This process is started when the power button 104a is pressed to turn on the power of the imaging device 100, for example. In Fig. 6, the same processes as in the first embodiment are denoted by the same reference numerals and their description will be omitted, and different processes will be described.

[0051] If it is determined in step S402 that there is video output to devices other than the display unit 121, the process proceeds to step S601. In step S601, the control unit 140 sets roll correction to on for video to be output to devices other than the display unit 121. When roll correction is set to on, the image processing unit 116 performs roll correction processing on the captured video. The process then proceeds to step S602. In step S602, the control unit 140 sets roll correction to off for video to be output to the display unit 121. The process then proceeds to step S603. In step S603, the display control unit 122 superimposes an indication indicating the tilt of the imaging device on the video to be output to the display unit 121. For example, the display control unit 122 superimposes an indication 503 indicating horizontal as an indication indicating the tilt of the imaging device. The process then proceeds to step S404. With the above processing, when there is video output to a device other than the display unit 121, roll correction is not performed on the video displayed on the display unit 121, but a display to notify the photographer of the tilt of the imaging device 100 is superimposed, and roll correction can be performed only on the output video.

[0052] As described above, according to this embodiment, roll correction can be performed only on images output to devices other than the display unit of the imaging device. The primary observer will see images at a different angle from the viewer, but the primary observer can know the degree to which the imaging device 100 is tilted by the display that shows the tilt of the imaging device along with the image on the display unit of the imaging device. This prevents the imaging device 100 from being used at an extreme angle, and prevents the output image from being excessively cropped.

[0053] (Third embodiment) In the first and second embodiments, switching of roll correction was described, but in the third embodiment, switching of the sound collection method accompanying switching of roll correction will be described. The image capture device 100 can collect sound in monaural or stereo when capturing video. As shown in FIGS. 1(A) and 2, the image capture device 100 of this embodiment is equipped with a pair of left and right microphone elements (a microphone 150 and a microphone hole 103b).

[0054] When collecting sound in stereo using a pair of left and right microphone elements, as the imaging device 100 tilts in the roll direction from the horizontal, the pair of microphone elements also tilt in the roll direction from the horizontal. Without roll correction, the tilt of the image and the stereoscopic feel of the sound collected in stereo match. On the other hand, when roll correction is performed, the tilt of the image after roll correction and the stereoscopic feel of the sound collected in stereo do not match, causing the viewer to mistake sounds from above and below for sounds from the left and right, resulting in an inappropriate stereoscopic feel and a sense of discomfort. Therefore, in this embodiment, when roll correction is performed, sound is controlled to be collected in mono rather than stereo.

[0055] Fig. 7 is a flowchart showing the roll correction switching process and the sound collection method switching process. The processes shown in Fig. 7 are realized by the control unit 140 executing a program stored in a readable storage medium such as the ROM 141. This process is started when the power button 104a is pressed to turn on the power of the image capture device 100, for example. In Fig. 7, the same processes as in the first embodiment are denoted by the same reference numerals and their description will be omitted, and different processes will be described.

[0056] If there is video output to a device other than the display unit 121 and roll correction is to be performed (YES in step S402, step S403), then in step S701 the control unit 140 determines that audio should be collected in mono. On the other hand, if there is video output only to the display unit 121 and roll correction is not to be performed (NO in step S402, step S405), then in step S702 the control unit 140 determines whether or not there is audio output. If there is audio output from the speaker 151 or the audio output terminal 152, then it is determined that there is audio output, and the process proceeds to step S703. On the other hand, if it is determined that there is no audio output, the process proceeds to step S404. In step S703, the control unit 140 determines that audio should be collected in stereo. Then, the process proceeds to step S404.

[0057] In step S701, an example in which sound collection is uniformly switched to monaural when roll correction is performed has been described, but this is not limiting. It is sufficient to switch to monaural sound collection only when a large discrepancy occurs between the image and stereo sound after roll correction that causes a viewer to feel uncomfortable. For example, whether to use the monaural or stereo sound collection method may be determined based on the tilt of the image capture device 100. In this case, stereo sound collection is performed when the tilt of the image capture device 100 is less than a predetermined tilt, and switching to monaural sound collection is performed when the tilt of the image capture device 100 is equal to or greater than the predetermined tilt. In addition to the tilt of the image capture device 100, whether the tilt has continued for a predetermined period of time or more may also be taken into consideration when making this determination. Furthermore, although an example in which sound collection methods are switched has been described in this embodiment, sound collection may always be performed in stereo, and the output sound may be switched between monaural and stereo.

[0058] As described above, according to this embodiment, when there is video output to a device other than the display unit 121, the audio collection method is switched to mono, thereby making it possible to suppress discrepancies between video and audio even when the imaging device 100 is used at an angle.

[0059] In each embodiment, an example has been described in which roll correction is performed by image processing, but roll correction may also be performed by driving the image sensor 111 relative to the image capturing optical system 110.

[0060] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0061] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these examples and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0062] 100 Imaging device 111 Image sensor 115 Image sensor control unit 116 Image Processing Unit 121 Display section 130 Data storage media 131 Connection terminal 140 Control Means 144 Means of communication

Claims

1. An imaging device having an imaging unit that captures an image and a display unit that displays the image, a correction unit for correcting an image blur of the image; an output means for controlling the output of the image; a display control means for displaying the output image on the display unit, the correction means performs, among pitch correction, yaw correction, and roll correction in the image shake correction, the roll correction so that a correction degree of the roll correction when the output means outputs an image only to the display unit is smaller than a correction degree of the roll correction when the output means outputs an image to a unit other than the display unit, and performs the pitch correction and the yaw correction when the output means outputs an image only to the display unit and when the output means outputs an image to a unit other than the display unit.

2. 2. The imaging device according to claim 1, wherein the correction means stops the roll correction when the output means outputs the image only to the display unit.

3. 3. The imaging device according to claim 1, wherein the image is output to a device other than the display device when a moving image shooting mode is set.

4. the correction means electronically performs roll correction on the image captured by the imaging unit, 3. The imaging device according to claim 1, wherein when the output means outputs an image to a device other than the display device, roll correction is performed on each of the image to be output to a device other than the display device and the image to be output to the display device, so that the image to be output to a device other than the display device has a higher degree of roll correction than the image to be output to the display device.

5. 5. The imaging device according to claim 1, wherein the display control means displays a display indicating the tilt of the imaging device.

6. 6. The imaging device according to claim 5, wherein the display control means superimposes a display indicating horizontality as the display indicating the tilt of the imaging device.

7. equipped with a sound collection means for collecting sound in either monaural or stereo; The imaging device according to any one of claims 1 to 6, characterized in that the sound collection means collects sound in stereo when the output means outputs an image only to the display unit, and collects sound in mono when the output means outputs an image to other units as well.

8. equipped with a sound collection means for collecting sound in either monaural or stereo; The imaging device described in any one of claims 1 to 6, characterized in that the sound collection means collects sound in stereo when the output means outputs an image only to the display unit, and switches between collecting sound in mono and stereo depending on the tilt of the imaging device when the output means outputs an image to other units as well as the display unit.

9. 9. The imaging device according to claim 1, wherein an output destination of the image other than the display unit is a recording medium or an external device.

10. 10. The imaging apparatus according to claim 9, wherein the external device is an electronic device having a display means for displaying an image.

11. An electronic device having a display unit that displays an acquired image, a correction unit for correcting an image blur of the image; an output means for controlling output of the image, the correction means, among pitch correction, yaw correction, and roll correction in the image shake correction, varies a correction degree of the roll correction depending on whether the output means outputs an image to a unit other than the display unit, and performs the pitch correction and the yaw correction regardless of whether the output means outputs an image to a unit other than the display unit.

12. 12. The electronic device according to claim 11, wherein the correction unit stops the roll correction when the output unit outputs an image only to the display unit.

13. 13. The electronic device according to claim 11, wherein an image is output to a device other than the display device when a moving image shooting mode is set.

14. the correction means performs electronic roll correction on the image; The electronic device according to claim 11 or 12, characterized in that when the output means outputs an image to a device other than the display device, roll correction is performed on each of the image to be output to a device other than the display device and the image to be output to the display device, so that the image to be output to a device other than the display device has a higher degree of roll correction than the image to be output to the display device.

15. 15. The electronic device according to claim 11, wherein the display unit displays a display indicating the tilt of the electronic device.

16. The electronic device according to claim 15, wherein the display unit displays a display indicating horizontality in a superimposed manner as the display indicating the inclination of the electronic device.

17. 17. The electronic device according to claim 11, wherein an output destination of the image other than the display unit is a recording medium or an external device.

18. 18. The electronic device according to claim 17, wherein the external device comprises a display means for displaying an image.

19. A control method for an imaging device having an imaging unit that captures an image and a display unit that displays the image, comprising: determining whether an image captured by the imaging unit is output only to the display unit; and controlling a correction unit that performs image blur correction on the image in accordance with the determination, a step of controlling the correction means, in which, of pitch correction, yaw correction, and roll correction in the image shake correction, roll correction is not performed when it is determined that an image will be output only to the display unit, roll correction is performed when it is determined that an image will be output to a unit other than the display unit, and the pitch correction and the yaw correction are performed when it is determined that an image will be output only to the display unit and when it is determined that an image will be output to a unit other than the display unit.

20. A method for controlling an electronic device having a display unit that displays an acquired image, comprising: a control method for an electronic device, characterized in that, among pitch correction, yaw correction, and roll correction for image shake correction for the acquired image, a correction degree of the roll correction is varied depending on whether the acquired image is to be output to a unit other than the display unit, and the pitch correction and the yaw correction are performed regardless of whether the acquired image is to be output to a unit other than the display unit.

21. A program for causing a computer to function as the correction means, the output means, and the display control means of the imaging device according to any one of claims 1 to 10.

22. A program for causing a computer to function as each means of the electronic device according to any one of claims 11 to 18.

Images