Imaging device, control method for imaging device, and storage medium

Abstract

The present invention provides an imaging device, a control method for the imaging device, and a storage medium. An imaging device, wherein an imaging sensor that captures an image and a display unit that displays the image are provided, and the imaging device includes: an image processing unit configured to perform roll correction on the image; a control unit configured to control the output of the image; and a display control unit configured to display the output image on the display unit. The image processing unit performs roll correction such that the degree of roll correction in the case where the image is to be output only to the display unit is less than the degree of roll correction in the case where the image is to be output to the display unit and an output destination other than the display unit.

Description

Technical Field

[0001] The present invention relates to a photographing apparatus, a control method for the photographing apparatus, and a recording medium. Background Art

[0002] In order to allow a main observer other than the photographer to view an image captured by a camera, there is a technique of displaying the image on a display other than the display device included in the camera. For example, the photographer can wear a camera on his / her head, and the image captured by the camera can be displayed on a display seen by other observers. At this time, when the photographer tilts the head wearing the camera, the subject displayed on the display seen by the observer also tilts. Therefore, there is a need to minimize the change in the direction of the image of the subject displayed on the display even when the photographer tilts the head, so that the subject is appropriately displayed on the display. Japanese Patent Laid-Open No. 2012-160898 discloses an image processing apparatus that corrects image tilt of an image displayed on a monitor unit seen by a person other than the observer when the image captured by a camera worn on the observer's head is displayed on the monitor unit. In the technique disclosed in Japanese Patent Laid-Open No. 2012-160898, an observer wearing a camera visually observes an object in the real outside world as a perspective image.

[0003] However, in Japanese Patent Laid-Open No. 2012-160898, the observer wearing the camera visually observes the subject, and does not consider the case where the photographer uses a display or an electronic viewfinder to view the captured image as in an electronic monocular telescope or an electronic binocular telescope. Therefore, in a photographing apparatus capable of outputting a captured image to both a display unit of the camera seen by the main observer and a display other than the display unit of the camera seen by other observers, it is necessary to display an appropriate image that causes less discomfort to the photographer. Summary of the Invention

[0004] The present invention provides a photographing apparatus that can output an image to a display unit of the photographing apparatus and an output destination other than the display unit, so that an appropriate image is displayed on the output destination.

[0005] According to the present invention, there is provided an imaging device, including an imaging unit that captures an image and a display unit that displays the image, the imaging device comprising: a correction component configured to perform roll correction on the image; an output component configured to control the output of the image; and a display control component configured to display an output image on the display unit, wherein the correction component is configured to perform the roll correction such that a correction degree of the roll correction when the output component is to output the image to only the display unit is less than a correction degree of the roll correction when the output component is to output the image to the display unit and an output destination other than the display unit.

[0006] A control method for an imaging device, in which an imaging unit that captures an image and a display unit that displays the image are provided, the control method including the steps of: determining whether an output destination of an image captured by the imaging unit is only the display unit; and controlling a roll correction component based on a result of the determination, wherein the step of controlling the roll correction component includes performing control such that: when it is determined that the image is to be output to only the display unit, no roll correction is performed, and when it is determined that the image is to be output to the display unit and an output destination other than the display unit, roll correction is performed.

[0007] A non-transitory storage medium storing a computer program for causing a computer of an imaging device to execute a control method, in which an imaging sensor that captures an image and a display unit that displays the image are provided, the control method including the steps of: determining whether an output destination of an image captured by the imaging unit is only the display unit; and controlling a roll correction component based on a result of the determination, wherein the step of controlling the roll correction component includes performing control such that: when it is determined that the image is to be output to only the display unit, no roll correction is performed, and when it is determined that the image is to be output to the display unit and an output destination other than the display unit, roll correction is performed.

[0008] Other features of the present invention will become apparent from the following description of embodiments with reference to the accompanying drawings. Description of the Drawings

[0009] Figures 1A to 1C is a perspective view showing an external appearance of the imaging device.

[0010] Figure 2 is a diagram showing a structure of the imaging device.

[0011] Figures 3A to 3D is a diagram showing roll correction.

[0012] Figure 4 is a flowchart showing the roll correction switching process.

[0013] Figure 5 is a diagram showing an example of a mark indicating the horizontal state.

[0014] Figure 6 is a flowchart showing the roll correction switching process and the process of displaying a mark indicating the horizontal state.

[0015] Figure 7 is a flowchart showing the roll correction switching process and the radio reception method switching process. Detailed Description of the Invention

[0016] (First Embodiment)

[0017] will refer to Figures 1A to 2 to describe the structure of a camera device 100 as an example of an electronic device to which the present invention can be applied. Figures 1A to 1C is a perspective view showing the appearance of the camera device 100. Figure 1A is a perspective view showing the appearance of the camera device 100 when viewed from the front (subject side). Figure 1B is a perspective view showing the appearance of the camera device 100 when viewed from the back opposite to the subject side. Figure 1C is a diagram showing a state in which the terminal cover 105 of the camera device 100 is open. Figure 2 is a diagram showing the structure of the camera device 100.

[0018] The camera device 100 is an example of an electronic device integrally including an imaging element such as a digital single-lens reflex camera, a digital binocular telescope, or a digital camera, and a display unit. The camera device 100 includes a side cover 101, a rear panel 102, and a front panel 103 that serve as a main body housing. An opening 103a is formed in the front panel 103, and a subject image is input to the imaging optical system 110 through the opening 103a. The subject image is formed on the imaging element 111 as an imaging component through the imaging optical system 110. The imaging lens optical axis A of the imaging optical system 110 extends in the extending direction of the camera device 100. In this embodiment, an example in which the main body and the lens unit of the camera device are integrated will be described, but the present invention is not limited thereto, and the lens unit can be detached from the main body.

[0019] Two microphone holes 103b facing the subject side are provided in the front panel 103. A corresponding microphone 150 is provided in each of the two microphone holes 103b. The microphone holes 103b guide sound from the outside to the microphone 150. The camera device 100 can collect sound in mono using the two microphones 150, and can also collect sound in stereo.

[0020] A terminal cover 105 that is coupled to the imaging device 100 so as to be openable and closable is provided on a side surface of the side cover 101. Figure 1B The state in which the terminal cover 105 is closed is shown, and Figure 1C The state in which the terminal cover 105 is open is shown. An I / O unit 143 and connection terminals 131 are provided in the terminal cover 105 of the side cover 101. The I / O unit 143 is an input / output connector unit that is configured to send and receive information by allowing a detachable data storage medium 130 such as an SD card to be mounted thereon. The connection terminals 131 are connection terminals 131 such as USB connection terminals for sending and receiving information or power.

[0021] Five operation buttons 104 are provided on an upper surface of the side cover 101. The five operation buttons 104 include a power button 104a, a zoom button 104b, a release button 104c, a video shooting button 104d, and a setting change button 104e. The power button 104a is an operation button for turning on / off the power of the imaging device 100. The zoom button 104b is an operation button for performing a zoom operation for changing a shooting magnification of the imaging device 100. The release button 104c is an operation button for focusing on a subject by moving a focusing group such as a zoom lens in the imaging optical system 110 or for performing a release for shooting a still image. The video shooting button 104d is an operation button for starting and ending shooting a moving image. The setting change button 104e is an operation button for changing various settings of the imaging device 100, and when the setting change button 104e is pressed, a menu associated with the settings is displayed. Selective changes of various settings are made using the zoom button 104b, the release button 104c, the video shooting button 104d, and the setting change button 104e. The form, functions, and arrangement of the operation buttons mentioned here are merely examples, and the operation buttons may be functionally combined with other operation buttons or may have other arrangements.

[0022] A visibility adjustment dial 124 and a sound hole 106 are provided in a bottom portion of the side cover. The visibility adjustment dial 124 is a dial for adjusting the focus of a visibility adjustment member. The sound hole 106 emits sound from a speaker 151.

[0023] An eyepiece window 120 and an eyepiece sensor 123 are provided in the rear panel 102. A display component is arranged in the eyepiece window 120. The display component is an electronic viewfinder (EVF) including a display unit 121, an eyepiece lens group (not shown), and a visibility adjustment component. The display unit 121 is a display formed of an organic EL display, a liquid crystal display, or the like. The eyepiece lens group magnifies the display on the display unit 121 so that the photographer can easily view the display unit 121. The photographer can view the display at the optimal visible position by setting the line of sight on the eyepiece lens optical axis B of the eyepiece lens group and setting the eye position to the eye point C of the eyepiece lens group on the eyepiece lens optical axis B. The imaging lens optical axis A and the eyepiece lens optical axis B in the imaging device 100 are parallel to each other. The visibility adjustment component can adjust the focus position according to the eyesight of the photographer. The eyepiece sensor 123 is provided above the eyepiece window 120 and detects that the photographer is peeping into the eyepiece window 120. In this embodiment, an example in which the display component is an electronic viewfinder is described, but the present invention is not limited thereto, and the display component may be a display or the like.

[0024] The electrical structure of the imaging device 100 will be described below. The imaging device 100 includes a control unit 140, a read-only memory (ROM) 141, a random access memory (RAM) 142, and an I / O unit 143, and a removable data storage medium 130 is attached thereto. The control unit 140 is a central processing unit (CPU) and controls the imaging device 100 as a whole. The ROM 141 is connected to the control unit 140, and the control unit 140 performs various types of control of the imaging device 100 based on the control program stored in the ROM 141.

[0025] The ROM 141 is a non-volatile memory storing a control program. The RAM 142 is a volatile memory. The RAM 142 is a temporary storage area such as a main memory or a working area for the operation of the control unit 140. For example, the RAM 142 includes an image loading area, a working area, a video RAM (VRAM), and a temporary save area. The image loading area serves as a temporary buffer that temporarily stores captured image data sent from the image processing unit 116, compressed and converted image data read from the data storage medium 130, etc. The image loading area also serves as an image dedicated area for image compression processing and image decompression processing. The working area is an area for executing various programs. The VRAM is an area for storing display data displayed on the display unit 121. The temporary save area is an area for temporarily saving various types of data.

[0026] The data storage medium 130 is a detachable storage medium such as an SD card or an HDD. In the data storage medium 130, the captured image data or moving image data that has been compressed and converted is stored in the form of files. The I / O unit 143 controls the transmission of data to and the reception of data from the data storage medium 130 based on an instruction from the control unit 140.

[0027] The imaging device 100 includes an imaging optical system 110, an imaging element 111, a shutter control unit 112, a zoom control unit 113, a focus control unit 114, an imaging element control unit 115, and an image processing unit 116. The imaging optical system 110 includes a plurality of lenses (such as a zoom lens and a focus lens), an aperture, and a shutter. The aperture is a light intensity adjustment component configured to control the amount of light flux passing through the imaging optical system 110. The imaging optical system 110 forms an optical image of the subject on the imaging element 111. The imaging element 111 is a photoelectric conversion element such as a CMOS or a CCD, and is an imaging unit that outputs an output signal (analog signal) corresponding to the optical image. When the imaging element 111 is a CMOS element, pixel data that is thinned out at intervals in the horizontal and vertical directions can be output according to a resolution change instruction from the imaging element control unit 115.

[0028] The imaging element control unit 115 includes a timing generator, a signal processing circuit, and an A / D conversion circuit, and controls the imaging element 111 and the output signal from the imaging element 111. The timing generator supplies a transfer clock signal or a shutter signal to the imaging element 111. The signal processing circuit performs noise removal processing or gain setting processing on the output signal output from the imaging element 111. The A / D conversion circuit converts the analog signal into a digital signal. The digital signal conversion specification in the A / D conversion circuit can be changed in various forms based on the specifications of the imaging element and the like.

[0029] The image processing unit 116 performs various types of image processing such as gamma conversion, color space conversion, white balance adjustment, and exposure adjustment on the digital signal output from the imaging element control unit 115. The image processing unit 116 performs image processing for correcting image shake or rotation such as pitch correction, yaw correction, and roll correction based on the pose information of the imaging device 100 acquired by the pose sensor 125. Image shake correction such as pitch correction, yaw correction, and roll correction is electronically performed by image cropping or the like in the image processing unit 116. The pitch correction, yaw correction, and roll correction performed by the image processing unit 116 can be based on the motion information of the subject obtained from the captured image data. The pitch correction, yaw correction, and roll correction can be performed by controlling the imaging surface of the imaging element 111 or shifting the shift lens of the imaging optical system 110. The pitch correction, yaw correction, and roll correction performed by the image processing unit 116 are controlled by the control unit 140, and the control unit 140 also determines whether to perform the correction process.

[0030] The shutter control unit 112 drives the shutter of the imaging optical system 110 based on an instruction from the control unit 140. The zoom control unit 113 drives the zoom lens of the imaging optical system 110 based on an instruction from the control unit 140. The focus control unit 114 drives the focus lens of the imaging optical system 110 based on an instruction from the control unit 140. The control unit 140 calculates whether the focus is achieved by driving the focus lens by how much from the current position (or whether the current position of the imaging optical system 110 is in focus), and instructs the drive amount of the focus lens to the focus control unit 114. The focus control unit 114 performs a focusing operation by moving the focus lens based on the drive amount indicated by the control unit 140. Although the lens drive method for the focusing operation has been described above, the same effect can also be achieved by changing the relative position between the imaging element 111 and the imaging optical system 110. Therefore, a method of driving the imaging element 111 along the optical axis A of the imaging lens can be adopted.

[0031] The imaging device 100 includes a DC / DC converter 145 and a battery 146. The DC / DC converter 145 generates a voltage from the power supplied by the battery 146 and supplies the voltage to components such as the control unit 140. The DC / DC converter 145 controls the start and stop of the voltage supply based on a control signal from the control unit 140. The battery 146 is a primary battery or a rechargeable secondary battery and supplies power to the DC / DC converter 145. As long as power is supplied to the DC / DC converter 145, it is not necessary to supply power from the battery. For example, power can be supplied to the DC / DC converter 145 from an external power source such as a mobile battery connected to the connection terminal 131.

[0032] The imaging device 100 includes a power button 104a, a zoom button 104b, a release button 104c, a video shooting button 104d, and a setting change button 104e. These buttons are connected to the control unit 140. The control unit 140 is turned on by turning on the power button 104a, and the control unit 140 is stopped by turning off the power button 104a. The control unit 140 controls the DC / DC converter 145 in response to the operation of the power button 104a.

[0033] When the zoom button 104b is operated, the zoom position of the imaging optical system 110 can be changed. The control unit 140 drives the zoom lens of the imaging optical system 110 to change the zoom position of the imaging optical system 110 by controlling the zoom control unit 113 in response to the operation of the zoom button 104b. The release button 104c has two-stage switch positions according to the pressed position. When the first-stage position (SW1 ON) is detected, operations for locking camera settings such as white balance, photometry, and focusing are performed. When the second-stage position (SW2 ON) is detected, an image is captured and a subject image signal is acquired. The control unit 140 controls the focus control unit 114, the shutter control unit 112, etc. according to the control of the release button 104c. When the video shooting button 104d is operated, acquisition of a subject moving image signal and recording of sound can be started / ended. Shooting of the subject moving image signal and recording of sound are started by pressing the video shooting button 104d for the first time, and shooting of the subject moving image signal and recording of sound are ended by pressing the video shooting button 104d for the second time. The control unit 140 sends a video shooting instruction to the imaging element control unit 115 and sends a sound recording instruction to the microphone 150 through the operation of the video shooting button 104d.

[0034] The imaging device 100 includes a display unit 121 and a display control unit 122. The display unit 121 includes an organic EL display device or a liquid crystal display device, and displays an image (including a still image or a moving image) captured by the imaging element 111 or information related to imaging. The display unit 121 is controlled by the display control unit 122. The display control unit 122 adjusts the display size, brightness, etc. of the image according to an instruction from the control unit 140, and controls the display of the image on the display unit 121. The display control unit 122 controls the information superimposed on the image.

[0035] The display control unit 122 includes an eyepiece sensor 123. The eyepiece sensor 123 is provided near the display unit 121. When the imaging user peeks at the display unit 121, the eyepiece sensor 123 detects that the imaging user has approached to a predetermined distance or less, and lights up the display unit 121. On the other hand, when the eyepiece sensor 123 detects that the photographer is separated from the eyepiece sensor 123 by a predetermined distance or more, the eyepiece sensor 123 turns off the display unit 121. By this operation, the power consumption of the imaging device can be reduced and heat generation can be suppressed.

[0036] The imaging device 100 includes a microphone 150 to collect external sounds. The imaging device 100 according to the present embodiment includes two microphones 150 and can perform monaural sound recording and stereophonic sound recording. The control unit 140 synchronizes the moving image data with the sound data acquired by the microphone 150 and stores the two pieces of data as a single video data in the data storage medium 130.

[0037] The imaging device 100 includes a speaker 151 and a sound output terminal 152. The speaker 151 is an electroacoustic transducer that can output sounds such as electronic sounds or sounds included in video data. The sound output terminal 152 is a sound output terminal such as a headphone jack that can connect headphones or the like. In addition to monaural sound, the sound output terminal 152 can also output stereophonic sound to the headphones connected thereto. For example, the sound output terminal 152 can be a connection terminal 131 such as a USB connection terminal.

[0038] When reproducing video data in the imaging device 100, the video data stored in the data storage medium 130 is selected. When reproducing video image data, the moving image data included in the video data is displayed on the display unit 121 by the display control unit 122. At the same time, the sound data included in the video data is reproduced in synchronization with the moving image data by the control unit 140 and output from the speaker 151 or the sound output terminal 152.

[0039] The imaging device 100 includes a communication unit 144 that transmits and receives data or power by connecting to an external device such as another imaging device, a smartphone, or a display device. The communication unit 144 can be a connection terminal 131 such as a USB connection terminal, or can be a radio connection component such as Wi-Fi or Bluetooth. When using the wired connection terminal 131, in addition to being able to transmit and receive data, it can also transmit and receive power. On the other hand, when using the radio connection component, it can transmit and receive data by connecting to an external device. The control unit 140 communicates with the external device connected to the communication unit 144 via the communication unit 144, and performs control for outputting video data to the external device. The control unit 140 can perform control for outputting video data to the external device only when the external device is a display device or an electronic device including a display unit such as an imaging device or a smartphone.

[0040] The control unit 140 controls the imaging device 100 as a whole. The control unit 140 performs various types of image processing, including roll correction in the image processing unit 116. The control unit 140 reads the captured image data output from the image processing unit 116, sends the captured image data to the RAM 142, and outputs the captured image data to the output destination. The output destinations of the captured image data include the display unit 121, the data storage medium 130, and the external device. The control unit 140 sends the captured image data from the RAM 142 to the display unit 121 via the display control unit 122. The control unit 140 stores the captured image data in the data storage medium 130 via the I / O unit 143. The control unit 140 outputs the captured image data to the external device via the communication unit 144. When the captured image data is moving image data, in addition to outputting the moving image data, sound data is also output. When recording the moving image data, the control unit 140 also controls the sound collection method in the microphone 150.

[0041] The roll correction will be described below. Figures 3A to 3D is a diagram showing the roll correction. In the following description, a person who uses the imaging device 100 to photograph the subject 301 is referred to as the main observer, and a person who uses a display device other than the imaging device 100 to view the video captured by the imaging device 100 is referred to as the viewer.

[0042] Figure 3A is a diagram showing a display example of the screen when the main observer uses the imaging device 100 to observe the subject 301. That is, Figure 3AAn example of a screen displayed on the display unit 121 of the imaging device 100 without roll correction is shown. The subject 301 stands vertically on a horizontal ground. The screen 302 is a display example of the screen when the main observer holds the imaging device 100 horizontally. The screen 303 is a display example of the screen when the main observer holds the imaging device 100 at an angle. When the imaging device 100 is tilted, although the screen is tilted, the tilt angle of the subject 301 is the same as the actual tilt angle of the subject 301. Therefore, regardless of whether the imaging device 100 is tilted or not, the tilt angle of the subject 301 observed by the main observer using the imaging device 100 is the same as the actual tilt angle of the subject 301. In this case, without performing roll correction processing on the image displayed on the display unit 121 according to the tilt angle of the imaging device 100, the main observer will not feel discomfort. That is, when the main observer observes the subject using the imaging device 100, roll correction is not required.

[0043] Figure 3B FIG. is a display example of a screen showing the image captured by the imaging device 100 without roll correction and displayed on a display device other than the imaging device 100. The screen frame 305 is the screen frame of the display device held horizontally. Figure 3B The image displayed in the screen frame 305 in is the image when the imaging device 100 is tilted ( Figure 3A the screen 303 in). On the screen of the horizontal display device, the subject 301 is tilted in the opposite direction at the tilt angle of the screen 303 of the imaging device 100. Therefore, the tilt angle of the subject 301 displayed on the screen frame 305 with respect to the horizontal direction is different from the actual tilt angle of the subject 301, and the viewer feels discomfort.

[0044] Figure 3C FIG. is a display example of a screen showing the image after roll correction processing and displayed on a display device other than the imaging device 100. Figure 3C The image displayed in the screen frame 305 in is the image obtained by performing roll correction on the image ( Figure 3A the screen 303 in) according to the tilt angle of the imaging device 100 when the imaging device 100 is tilted. By performing roll correction according to the tilt angle of the imaging device 100, the tilt angle of the subject 301 displayed on the display device is the same as the actual tilt angle of the subject 301, and the viewer can view the image without feeling discomfort.

[0045] Figure 3D FIG. is a display example of a screen showing the image after roll correction processing and displayed on the display unit 121 of the imaging device 100. That is, Figure 3D FIG. shows the display on the display unit 121 of the imaging device 100 held at an angleFigure 3C A diagram showing an example of an image with roll correction. Since the inclination of the subject 301 displayed on the screen 303 of the tilted imaging device 100 is different from the actual inclination of the subject 301, the viewer feels discomfort.

[0046] When the imaging device 100 is held horizontally, the main observer who views the screen displayed on the imaging device 100 and the viewer who views the screen displayed on the external display device do not feel discomfort. However, when the user carries the imaging device 100, it is difficult to always keep the imaging device 100 horizontal. The user cannot capture images at a free angle such as the user tilting to one side. When the main observer uses the imaging device 100 to capture an image at a free angle and outputs the captured image to another display device, the image can be appropriately displayed by performing roll correction on the captured image. However, when performing roll correction, the image displayed on the imaging device 100 and observed by the main observer is roll-corrected, so the main observer feels discomfort. Therefore, it is preferable not to perform roll correction when only the main observer observes the image, rather than always performing roll correction. Therefore, in the present embodiment, when only the main observer observes the captured image in real time through the imaging device 100, roll correction is not performed, and when the video is output to a display device other than the imaging device 100, or the video is recorded for later viewing of the video, etc., roll correction is performed.

[0047] The roll correction switching process according to the present embodiment will be described below. As described above, in the present embodiment, when only the cameraman observes the image displayed on the display unit 121 of the imaging device 100 in real time, that is, when the output destination of the captured image is only the display unit 121, roll correction is not performed. On the other hand, when the viewer views the image displayed on the external device or confirms the recorded image later, that is, when the output destination of the captured image includes an output destination other than the display unit 121 (such as a display device other than the imaging device 100 or a recording medium of the imaging device 100, etc.), roll correction is performed. Therefore, in the present embodiment, a process of turning on / off roll correction depending on whether the image is to be output to an output destination other than the display unit 121 of the imaging device 100 is performed.

[0048] Figure 4 It is a flowchart showing the roll correction switching process. It is implemented by causing the control unit 140 to execute a program stored in a computer-readable storage medium such as the ROM 141 Figure 4 The method shown in. For example, when the power button 104a is pressed to turn on the imaging device 100, this method starts. First, in step S401, the control unit 140 turns on the yaw correction and pitch correction in the image processing unit 116. Then, the method proceeds to step S402.

[0049] In step S402, the control unit 140 determines whether to output the image to an output destination other than the display unit 121 of the imaging device 100. The control unit 140 determines whether to output the image to an output destination other than the display unit 121 of the imaging device 100 based on whether to output the image to an external device or a storage medium. Specifically, the control unit 140 communicates with an external device connected via the communication unit 144 and determines whether the external device is an electronic device including a display unit. When the external device is an electronic device including a display unit, it is determined that the image is to be output to an output destination other than the display unit 121 of the imaging device 100. The control unit 140 also determines whether to store the video data in the data storage medium 130. When the video data is to be stored in the data storage medium 130, it is also determined that the image is to be output to an output destination other than the display unit 121 of the imaging device 100. When a predetermined mode such as a video shooting mode for shooting a video is selected by switching the mode or by pressing the video shooting button 104d, the control unit 140 also determines that the video is to be arranged to be output to an output destination other than the display unit 121 of the imaging device 100 and determines that the video is to be output to an output destination other than the display unit 121 of the imaging device 100. When the video is to be output to an output destination other than the display unit 121 of the imaging device 100, the method proceeds to step S403. On the other hand, when the video is not to be output to an output destination other than the display unit 121 of the imaging device 100, the method proceeds to step S405.

[0050] When the video is to be output to an output destination other than the display unit 121 of the imaging device 100, the control unit 140 turns on the roll correction in step S403. When the roll correction is turned on, the image processing unit 116 performs roll correction processing on the captured video. Therefore, the roll correction is performed on the video to be output to an output destination other than the display unit 121 of the imaging device 100. Then, the method proceeds to step S404. On the other hand, when the video is not to be output to an output destination other than the display unit 121 of the imaging device 100, the control unit 140 turns off the roll correction in step S405. Therefore, the roll correction is not performed on the video to be output to an output destination other than the display unit 121 of the imaging device 100. Then, the method proceeds to step S404.

[0051] In step S404, the control unit 140 detects the ON / OFF state of the power button 104a. When it is determined that the power button 104a is in the OFF state, the method ends. On the other hand, when it is detected that the power button 104a is in the ON state, the method returns to step S402, and it is determined whether to output the video to an output destination other than the display unit 121 to determine whether to switch the roll correction.

[0052] As described above, according to the present embodiment, when an image is to be output to an output destination other than the display unit of the imaging device, roll correction is performed, and when the image is to be output only to the display unit of the imaging device, roll correction is not performed. Therefore, regardless of the inclination of the imaging device, a viewer who views the image captured by the imaging device using an external device or the like can view the image with less discomfort. When only the photographer views the image, by not performing roll correction, an image in which the actual inclination of the subject is the same as the inclination of the subject in the image displayed on the display unit 121 and the photographer feels less discomfort can be displayed on the display unit 121. Therefore, when the image is not to be output to an output destination other than the display unit of the imaging device and the imaging device 100 is used as a monocular telescope or a binocular telescope, the image can be observed in a free posture and angle. In the present embodiment, an example in which roll correction is not performed when the image is to be output only to the display unit of the imaging device has been described above. The same advantages can be achieved as long as the discomfort of the photographer can be reduced. Therefore, the roll correction must be turned off. That is, control can be performed such that the roll shake correction degree in step S405 is smaller than that in step S403. The roll shake correction degree indicates the correction value of the generated roll shake, and is defined as follows: when roll shake has occurred, in the case of a smaller roll shake correction value, the correction degree of roll correction is smaller, that is, there is a larger residual roll shake. For example, when the photographer peeks through the eyepiece window 120, the head of the photographer is fixed, but the hand supporting the imaging device shakes slightly. Therefore, there is a possibility that a slight roll shake will make the photographer feel discomfort. Therefore, in step S405, by not correcting the low-frequency component of the roll shake to be corrected in step S403 and only correcting the high-frequency component, the correction amount can be reduced. By applying a smaller gain in step S403, the correction degree can be reduced, and the shake component in a frequency band narrower than that in step S403 can be corrected.

[0053] (Second Embodiment)

[0054] In the first embodiment, an example of performing roll correction on a video to be output to an output destination other than the display unit 121 when the video to be output to the display unit 121 has been described. In this case, the inclination of the subject in the image that has undergone roll correction and is displayed on the display unit 121 is different from the actual inclination of the subject, so a cameraman (main observer) who views these two images feels discomfort. Therefore, in the second embodiment, an example of performing roll correction only on an image to be output to an output destination other than the display unit of the imaging device will be described.

[0055] When performing roll correction only on an image to be output to an output destination other than the display unit of the imaging device, both the user of the imaging device 100 who uses it and the viewer who observes the output image can observe the subject with an inclination that does not cause discomfort. On the other hand, when the inclination of the imaging device 100 is particularly large and roll correction of the output image is performed by image cropping, the output image is extremely cropped. In order not to extremely crop the output image by roll correction, it is necessary to suppress the inclination of the imaging device 100. Therefore, in the present embodiment, by displaying a mark indicating the inclination of the imaging device on the display unit 121, the inclination of the imaging device 100 is notified to the user of the imaging device who takes an image while viewing the display unit 121, and the inclination of the imaging device 100 is suppressed.

[0056] Figure 5 FIG. is a diagram showing an example of a mark indicating a horizontal state displayed on the display unit 121. Figure 5 FIG. shows an example of an image obtained by photographing a subject 501 while the imaging device 100 according to the present embodiment is held at an inclination and displayed on the display unit 121. In the present embodiment, since roll correction is not performed on the image to be displayed on the display unit 121, roll correction has not been performed on Figure 5 the image shown in. Therefore, the frame 502 of the display unit 121 is also inclined according to the inclination of the imaging device 100.

[0057] In the frame 502, a mark 503 indicating the horizontal state is displayed. The mark 503 indicating the horizontal state is a horizontal straight line in the frame 502. The photographer can identify the inclination of the imaging device 100 by comparing the mark 503 indicating the horizontal state with the inclination of the subject 501. The mark 503 indicating the horizontal state is an example of a mark indicating the inclination of the imaging device. As long as the mark indicating the inclination of the imaging device allows the photographer to identify the inclination of the imaging device 100, the mark is not limited to a horizontal straight line in the frame 502. For example, a spirit level can be displayed as a mark indicating the inclination of the imaging device, or it can be a mark indicating the horizontal direction. The mark indicating the inclination of the imaging device can be the inclination of the frame of the image after roll correction, or it can be a mark indicating the roll correction direction and the correction value. The mark indicating the inclination of the imaging device can be displayed by attenuating the roll correction.

[0058] When the inclination of the imaging device 100 is greater than a predetermined inclination, a warning can be issued. For example, the warning can be to display a mark or output a warning sound, or it can be a change in the color of the mark 503 indicating the horizontal state. By displaying a mark indicating the inclination of the imaging device on the image displayed on the display unit 121, the photographer using the imaging device 100 can know to what extent the imaging device 100 is inclined. By notifying the photographer to what extent the imaging device 100 is inclined, the inclination of the imaging device 100 can be suppressed.

[0059] Figure 6 is a flowchart showing the roll correction switching process and the process of displaying a mark indicating the horizontal state. It is implemented by causing the control unit 140 and the display control unit 122 to execute a program stored in a computer-readable storage medium such as the ROM 141 Figure 6 the method shown in. For example, when the imaging device 100 is turned on by pressing the power button 104a, this method starts. In Figure 6 the same steps as in the first embodiment will be referred to with the same reference numerals and their description will be omitted. Different processing steps will be described below.

[0060] When it is determined in step S402 that the image is to be output to an output destination other than the display unit 121, the method proceeds to step S601. In step S601, the control unit 140 enables roll correction for the image to be output to an output destination other than the display unit 121. When roll correction is enabled, the image processing unit 116 performs roll correction processing on the captured image. Then, the method proceeds to step S602. In step S602, the control unit 140 disables roll correction for the image to be output to the display unit 121. Then, the method proceeds to step S603. In step S603, the display control unit 122 displays a mark indicating the tilt of the imaging device in a superimposed manner on the image to be output to the display unit 121. For example, the display control unit 122 displays the mark 503 indicating the horizontal state as a mark indicating the tilt of the imaging device in a superimposed manner. Then, the method proceeds to step S404. By the foregoing method, when the image is to be output to an output destination other than the display unit 121, roll correction is not performed on the image to be output to the display unit 121, a mark for notifying the imager of the tilt of the imaging device 100 is displayed in a superimposed manner, and roll correction is performed only on the image to be output to an output destination other than the display unit of the imaging device.

[0061] As described above, according to the present embodiment, roll correction can be performed only on the image to be output to an output destination other than the display unit of the imaging device. Although the main observer observes the image at a tilt angle different from that at which the viewer views it, the main observer can know to what extent the imaging device 100 is tilted by referring to the mark indicating the tilt of the imaging device displayed together with the image on the display unit of the imaging device. Therefore, extreme tilting of the imaging device 100 can be suppressed and extreme cropping of the output image can be suppressed.

[0062] (Third Embodiment)

[0063] The switching of roll correction has been described in the first and second embodiments, and the switching of the sound collection method accompanying the switching of roll correction will be described in the third embodiment. The imaging device 100 can collect sound in mono or stereo when shooting a moving image. As Figure 1A and Figure 2 shown, the imaging device 100 according to the present embodiment includes a pair of microphone elements (microphone 150 and microphone hole 103b) on each of the right and left sides.

[0064] When performing stereo sound collection using a pair of microphone elements on each of the right and left sides, as the imaging device 100 tilts from the horizontal state in the roll direction, the pair of microphone elements tilt from the horizontal state in the roll direction. When roll correction is not performed, the degree of tilt of the image is consistent with the perception of the stereo sound collected in stereo. On the other hand, when roll correction is performed, the degree of tilt of the image after roll correction and the perception of the stereo sound collected in stereo are inconsistent, and since the sound from the vertical direction may be misrecognized as the sound from the horizontal direction, an appropriate stereo feeling cannot be obtained, and the viewer feels discomfort. Therefore, in the present embodiment, control is performed such that when roll correction is performed, mono sound collection is performed instead of stereo sound collection.

[0065] Figure 7 is a flowchart showing roll correction switching processing and sound collection method switching processing. It is implemented by causing the control unit 140 to execute a program stored in a computer-readable storage medium such as the ROM 141 Figure 7 shown in the method. For example, when the imaging device 100 is turned on by pressing the power button 104a, this method starts. In Figure 7 this, the same processing steps as in the first embodiment will be referred to by the same reference numerals, and their descriptions will be omitted. Different processing steps will be described below.

[0066] When the image is to be output to an output destination other than the display unit 121 and roll correction is performed on the image (Yes in step S402, step S403), in step S701, the control unit 140 determines to perform mono sound collection. On the other hand, when the image is to be output to only the display unit 121 and thus roll correction is not performed on the image (No in step S402, step S405), in step S702, the control unit 140 determines whether to output sound. When it is determined to output sound from the speaker 151 or the sound output terminal 152, it is determined to output sound, and the method proceeds to step S703. On the other hand, when it is determined not to output sound, the method proceeds to step S404. In step S703, the control unit 140 determines to perform stereo sound collection. Then, the method proceeds to step S404.

[0067] An example has been described above in which monaural sound recording is uniformly performed during roll correction in step S701, but the present invention is not limited thereto. Monaural sound recording may be performed only when a large inconsistency between the image and the stereo sound that occurs after roll correction reaches a level that makes the viewer feel uncomfortable. For example, it is possible to determine whether to perform monaural sound recording or stereo sound recording based on the tilt angle of the imaging device 100. In this case, when the tilt angle of the imaging device 100 is less than a predetermined tilt angle, stereo sound recording is performed, and when the tilt angle of the imaging device 100 is equal to or greater than the predetermined tilt angle, monaural sound recording is performed. In addition to the tilt angle of the imaging device 100, this determination may be made considering whether the tilt angle is maintained for a predetermined time or longer. In the present embodiment, an example of switching the sound recording method has been described above, but stereo sound recording may be performed normally, and the sound to be output may be switched between monaural and stereo.

[0068] As described above, according to the present embodiment, by switching the sound recording method to monaural when the image is to be output to an output destination other than the display unit 121, even when the imaging device 100 is used in a tilted state, it is possible to suppress the inconsistency between the image and the sound.

[0069] In the foregoing embodiment, an example of performing roll correction by image processing has been described, but roll correction may be performed by driving the imaging element 111 with respect to the imaging optical system 110.

[0070] (Other embodiments)

[0071] Embodiments of the present invention can also be implemented by a method in which software (program) that executes the functions of the above-described embodiments is provided to a system or device via a network or various storage media, and the computer or central processing unit (CPU) or microprocessing unit (MPU) of the system or device reads and executes the program.

[0072] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims will be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

[0073] This application claims the benefit of Japanese Patent Application No. 2021-023159, filed on Feb. 17, 2021, the entire contents of which are incorporated herein by reference.

Claims

1. An imaging device, wherein, A camera unit that captures an image and a display unit that displays the image, the imaging device comprising: a correction component configured to perform image processing on the image including pitch correction, yaw correction, and roll correction; an output component configured to control the output of the image; and a display control component configured to display the output image on the display unit, wherein the correction component is configured to perform the roll correction such that the degree of roll correction in a case where the output component is to output the image to only the display unit is less than the degree of roll correction in a case where the output component is to output the image to the display unit and an output destination other than the display unit, and perform the pitch correction and the yaw correction in a case where the output component is to output the image to only the display unit and in a case where the output component is to output the image to the display unit and an output destination other than the display unit.

2. The imaging device according to claim 1, wherein, The correction component is configured to stop the roll correction in a case where the output component is to output the image to only the display unit.

3. The imaging device according to claim 1, wherein, The case where the output component is to output the image to the display unit and an output destination other than the display unit includes a case where a predetermined mode is set.

4. The imaging device according to claim 1, wherein, The correction component is configured to electronically perform roll correction on the image captured by the camera unit, and wherein the correction component is adapted to: in a case where the output component is to output the image to the display unit and the output destination other than the display unit, perform the roll correction on the image to be output to the output destination other than the display unit and the image to be output to the display unit such that the degree of roll correction for the image to be output to the output destination other than the display unit is higher than the degree of roll correction for the image to be output to the display unit.

5. The imaging device according to claim 1, wherein, The display control component is configured to display a mark indicating the inclination of the imaging device.

6. The imaging device according to claim 5, wherein, The display control component is configured to display a mark indicating a horizontal state in a superimposed manner as a mark indicating the inclination of the imaging device.

7. The imaging device according to claim 1, further comprising a sound collection component configured to perform sound collection using one of mono and stereo, and Among them, the sound collection component is configured to: in a case where the output component is to output the image to only the display unit, perform the sound collection in stereo, and in a case where the output component is to output the image to the display unit and an output destination other than the display unit, perform the sound collection in mono.

8. The imaging device according to claim 1, further comprising a sound collection component configured to perform sound collection using one of mono and stereo, and Among them, The sound receiving component is configured to: perform sound reception in stereo when the output component is to output the image to only the display unit, and switch between mono sound reception and stereo sound reception according to the tilt of the imaging device when the output component is to output the image to an output destination other than the display unit.

9. The imaging device according to any one of claims 1 to 8, wherein, The output destination of the image other than the display unit is a recording medium or an external device.

10. The imaging device according to claim 9, wherein, The external device is an electronic device including a display component configured to display an image.

11. A control method for an imaging device, the imaging device including an imaging unit that captures an image and a display unit that displays the image, the control method including the following steps: A step of determining whether the output destination of the image captured by the imaging unit is only the display unit; And A step of performing image processing including pitch correction, yaw correction, and roll correction based on the result of the determination, wherein the step of performing image processing includes controlling so that: when it is determined that the image is to be output to only the display unit, the roll correction is not performed, and when it is determined that the image is to be output to the display unit and an output destination other than the display unit, the roll correction is performed, and the pitch correction and the yaw correction are performed when it is determined that the image is to be output to only the display unit and when it is determined that the image is to be output to the display unit and an output destination other than the display unit.

12. A non-transitory storage medium storing a computer program for causing a computer of an imaging device to execute a control method, the imaging device including an imaging unit that captures an image and a display unit that displays the image, the control method including the following steps: A step of determining whether the output destination of the image captured by the imaging unit is only the display unit; And A step of performing image processing including pitch correction, yaw correction, and roll correction based on the result of the determination, wherein the step of performing image processing includes controlling so that: when it is determined that the image is to be output to only the display unit, the roll correction is not performed, and when it is determined that the image is to be output to the display unit and an output destination other than the display unit, the roll correction is performed, and the pitch correction and the yaw correction are performed when it is determined that the image is to be output to only the display unit and when it is determined that the image is to be output to the display unit and an output destination other than the display unit.

13. A computer program product including a computer program for causing a computer of an imaging device to execute a control method, the imaging device including an imaging unit that captures an image and a display unit that displays the image, the control method including the following steps: A step of determining whether the output destination of the image captured by the imaging unit is only the display unit; And Steps of performing image processing including pitch correction, yaw correction, and roll correction based on the result of the determination Among them, the steps of performing image processing include performing control such that: when it is determined that the image is to be output only to the display unit, the roll correction is not performed, and when it is determined that the image is to be output to the display unit and an output destination other than the display unit, the roll correction is performed, and the pitch correction and the yaw correction are performed when it is determined that the image is to be output only to the display unit and when it is determined that the image is to be output to the display unit and an output destination other than the display unit.

Images