Electronic apparatus, method for controlling electronic apparatus, program, and recording medium

Abstract

To provide an imaging device which can easily take a three-dimensionally viewable image when taking two image regions with a parallax.SOLUTION: An electronic apparatus includes: acquisition means for acquiring information of the distance from an imaging device to an object; and control means for controlling to notify a user of the relation between a recommendation distance as an imaging distance in which a taken image with two image regions with a parallax can be viewed three-dimensionally and the distance from the imaging device to the object.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an electronic device, a control method for an electronic device, a program, and a storage medium.

Background Art

[0002] There is known a technique of acquiring an image having two image regions with a parallax using two optical systems facing the same direction, and displaying the two image regions in a stereoscopically viewable manner. If a circumferential fisheye lens is used as each optical system, an image region representing a wide range of 180 degrees or more (hemisphere, 90 degrees in all directions from the image center) can be obtained as each image region.

[0003] Patent Document 1 discloses a stereoscopic camera configured to photograph the same subject from different viewpoints with parallel lines of sight. Patent Document 2 discloses a stereoscopic imaging device that detects corresponding points between a plurality of captured images obtained by imaging a subject field from a plurality of viewpoints.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, an image having two image regions with a parallax may be an image without a three-dimensional effect depending on the distance to the subject. For this reason, there is a problem that the user does not know whether the captured image can be captured so as to be a three-dimensional image when viewed on a display device when capturing two image regions.

[0006] An object of the present invention is to provide an imaging device capable of easily capturing a stereoscopic image when capturing two image regions with parallax.

Means for Solving the Problems

[0007] A first aspect of the present invention is an electronic device comprising acquisition means for acquiring information on the distance from the imaging device to the subject, a recommended distance that is a shooting distance at which stereoscopic viewing of an imaging image having two image regions with parallax is possible, and control means for controlling to notify the relationship between the distance from the imaging device to the subject.

[0008] A second aspect of the present invention is a method for controlling an electronic device, comprising the steps of acquiring information on the distance from the imaging device to the subject, a recommended distance that is a shooting distance at which stereoscopic viewing of an imaging image having two image regions with parallax is possible, and controlling to notify the relationship between the distance from the imaging device to the subject.

[0009] A third aspect of the present invention is a program for causing a computer to function as each means of the above-described electronic device. A fourth aspect of the present invention is a computer-readable storage medium storing a program for causing a computer to function as each means of the above-described electronic device.

Advantages of the Invention

[0010] According to the present invention, it is possible to provide an imaging device capable of easily capturing a stereoscopic image when capturing two image regions with parallax.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Mode for Carrying Out the Invention

[0012] <Embodiment 1> Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIGS. 1(A) to 1(C) are schematic diagrams of the eyes of a user looking at a subject and an imaging device that images the subject. With reference to FIGS. 1(A) to 1(C), the reason why the stereoscopic effect when an image having two image regions with parallax is viewed on a display device is different from the stereoscopic effect when the user actually views the subject will be explained.

[0013] Fig. 1(A) is a schematic diagram when the user views a subject with the left and right eyes. The convergence angle S1 is the angle formed by the straight lines (lines of sight) connecting the subject to each of the left and right eyes. Fig. 1(B) is a schematic diagram when imaging a subject with an imaging device having two optical systems. The distance from the imaging device to the subject in Fig. 1(B) is assumed to be the same as the distance from the user to the subject in Fig. 1(A). The convergence angle S2 is the angle formed by the straight lines connecting the subject to each of the left and right optical systems.

[0014] Since the distance between the left and right eyeballs is different from the distance between the two optical systems, the convergence angle S1 is different from the convergence angle S2. That is, the parallax of the left and right image regions included in the image captured by the imaging device is different from the parallax of the region viewed by the user with the left and right eyes from the same position as the imaging device. When the convergence angle S2 is smaller than the convergence angle S1, the parallax is smaller when imaging the subject with the imaging device than when the user actually views the subject. When the convergence angle S2 at the time of imaging with the imaging device is smaller than the convergence angle S1, the stereoscopic effect when viewing the image captured by the imaging device on the display device decreases. Thus, the stereoscopic effect is different between the case where the user views the subject from the imaging position in Fig. 1(A) and the case where the captured image is viewed on a display device such as an HMD (head-mounted display).

[0015] Fig. 1(C) is a schematic diagram when imaging a subject with the imaging device from a position closer to the subject than Fig. 1(B). The convergence angle S3 is the angle formed by the straight lines connecting the subject to the left and right optical systems. By bringing the imaging device closer to the subject, the convergence angle S3 becomes approximately the same angle as the convergence angle S1 in Fig. 1(A). The stereoscopic effect when viewing the image captured by bringing the imaging device closer to the subject on the display device becomes closer to the stereoscopic effect when the user views the subject from the imaging position in Fig. 1(A).

[0016] Therefore, the imaging device (camera) according to Embodiment 1 notifies the user of an appropriate distance to the subject in order to enable the user to capture a stereoscopic image. Based on the notification from the imaging device, the user can easily capture a stereoscopic image by adjusting the distance to the subject. to capture.

[0017] Figures 2(A) and 2(B) are diagrams showing an example of the appearance of digital camera (camera) 100. Camera 100 is an example of an electronic device according to the present embodiment. Figure 2(A) is a perspective view of camera 100 seen from the front. Figure 2(B) is a perspective view of camera 100 seen from the back.

[0018] On the upper surface of camera 100, there are a shutter button 101, a power switch 102, a mode switch 103, a main electronic dial 104, a sub - electronic dial 105, a video button 106, and an external - finder display section 107. The shutter button 101 is an operation member for giving an instruction for shooting preparation or a shooting instruction. The power switch 102 is an operation member for switching the power of camera 100 on and off. The mode switch 103 is an operation member for switching various modes. The main electronic dial 104 is a rotary operation member for changing setting values such as shutter speed and aperture. The sub - electronic dial 105 is a rotary operation member for moving a selection frame (cursor) or advancing an image. The video button 106 is an operation member for giving an instruction to start or stop video shooting (recording). The external - finder display section 107 displays various setting values such as shutter speed and aperture.

[0019] On the back surface of camera 100, there are a display section 108, a touch panel 109, direction keys 110, a SET button 111, an AE - lock button 112, a zoom - in button 113, a playback button 114, a menu button 115, an eyepiece section 116, an eyepiece detection section 118, and a touch bar 119.

[0020] The display unit 108 displays images and various types of information. The touch panel 109 is an operation member that detects touch operations on the display surface (touch operation surface) of the display unit 108. The direction keys 110 are operation members composed of keys (4-direction keys) that can be pressed respectively in the up, down, left, and right directions. Processing can be performed according to the pressed position of the direction keys 110. The SET button 111 is an operation member that is mainly pressed when determining a selected item. The AE lock button 112 is an operation member that is pressed when fixing the exposure state in the shooting standby state. The zoom button 113 is an operation member for switching between on and off of the zoom mode in the live view display (LV display) in the shooting mode. When the zoom mode is on, by operating the main electronic dial 104, the live view image (LV image) is enlarged or reduced. Also, the zoom button 113 is used when enlarging a playback image or increasing the magnification rate in the playback mode. The playback button 114 is an operation member for switching between the shooting mode and the playback mode. When the playback button 114 is pressed in the shooting mode, the camera 100 shifts to the playback mode, and the display unit 108 displays the latest image among the images recorded on the recording medium 227.

[0021] The menu button 115 is an operation member that is pressed to display a menu screen on the display unit 108 where various settings can be made. The user can intuitively perform various settings using the direction keys 110 and the SET button 111 on the menu screen displayed on the display unit 108. The eyepiece part 116 is a part where the user looks into the eyepiece finder (peeping type finder) 117. The user can visually recognize the video displayed on the EVF 217 (Electronic View Finder) inside the camera 100 through the eyepiece part 116. The eyepiece detection unit 118 is a sensor that detects whether or not the user is looking into the eyepiece part 116 (eyepiece finder 117).

[0022] The touch bar 119 is a linear touch operation member (line touch sensor) capable of receiving touch operations. The touch bar 119 is arranged at a position where it can be touched (touched) with the right thumb in a state where the grip portion 120 is held with the right hand (a state where it is held with the little finger, ring finger, and middle finger of the right hand) so that the shutter button 101 can be pressed with the index finger of the right hand. That is, the touch bar 119 is operable in a state (shooting posture) where the user looks into the eyepiece finder 117 and peers into the eyepiece portion 116, and can press the shutter button 101 at any time. The touch bar 119 can receive a tap operation (an operation of touching and releasing without moving the touch position within a predetermined period) on the touch bar 119, a slide operation to the left and right (an operation of moving the touch position while keeping the touch after touching), and the like. The touch bar 119 is an operation member different from the touch panel 109 and does not have a display function. The touch bar 119 functions as, for example, a multifunction bar (M-Fn bar) to which various functions can be assigned.

[0023] In addition, the camera 100 includes a grip portion 120, a thumb rest portion 121, a terminal cover 122, a lid 123, a communication terminal 124, and the like. The grip portion 120 is a holding portion formed in a shape that is easy to hold with the right hand when the user holds the camera 100. With the grip portion 120 held by the little finger, ring finger, and middle finger of the right hand, the shutter button 101 and the main electronic dial 104 are arranged at positions operable by the index finger of the right hand. Also, in the same state, the sub-electronic dial 105 and the touch bar 119 are arranged at positions operable by the thumb of the right hand. The thumb rest portion 121 (thumb standby position) is a grip portion provided on the back side of the camera 100 at a location where it is easy to place the thumb of the right hand that is holding the grip portion 120 without operating any operation members. The thumb rest portion 121 is composed of a rubber member or the like for enhancing the holding force (grip feeling). The terminal cover 122 protects connectors such as connection cables for connecting the camera 100 to an external device (external apparatus). The lid 123 protects the recording medium 227 and the slot by closing the slot for storing the recording medium 227 described later. The communication terminal 124 is a terminal for communicating with the lens unit (such as the lens unit 200 and the lens unit 300) side that is detachable from the camera 100.

[0024] FIG. 3 is a block diagram showing an example of the configuration of the camera 100. In FIG. 3, the same components as those in FIGS. 2(A) and 2(B) are denoted by the same reference numerals as in FIGS. 2(A) and 2(B), and the description of those components is omitted as appropriate. In FIG. 3, the lens unit 200 is attached to the camera 100.

[0025] First, the lens unit 200 will be described. The lens unit 200 is a type of interchangeable lens unit (interchangeable lens) that is detachable from the camera 100. The lens unit 200 is a single-lens unit (single-lens) and is an example of a normal lens unit. The lens unit 200 includes a diaphragm 201, a lens 202, a diaphragm drive circuit 203, an AF (auto focus) drive circuit 204, a lens system control circuit 205, a communication terminal 206, and the like.

[0026] The aperture 201 is configured such that its aperture diameter is adjustable. The lens 202 is composed of a plurality of lenses. The aperture drive circuit 203 adjusts the light amount by controlling the aperture diameter of the aperture 201. The AF drive circuit 204 drives the lens 202 to focus. The lens system control circuit 205 controls the aperture drive circuit 203, the AF drive circuit 204, etc. based on the instructions of the system control unit 50. The lens system control circuit 205 controls the aperture 201 via the aperture drive circuit 203 and focuses by changing the position of the lens 202 via the AF drive circuit 204. The lens system control circuit 205 is capable of communicating with the camera 100. Specifically, the camera 100 and the lens unit 200 communicate via the communication terminal 206 of the lens unit 200 and the communication terminal 124 of the camera 100. The communication terminal 206 is a terminal for the lens unit 200 to communicate with the camera 100 side.

[0027] Next, the camera 100 will be described. The camera 100 includes a shutter 210, an imaging unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, an EVF 217, a display unit 108, and a system control unit 50.

[0028] The shutter 210 is a focal plane shutter that can freely control the exposure time of the imaging unit 211 based on the instructions of the system control unit 50. The imaging unit 211 is an image sensor composed of a CCD or CMOS element that converts an optical image into an electrical signal. The imaging unit 211 may have an imaging surface phase difference sensor that outputs defocus amount information to the system control unit 50. The A / D converter 212 converts the analog signal output from the imaging unit 211 into a digital signal. The image processing unit 214 performs predetermined processing (such as pixel interpolation, resizing processing such as reduction, color conversion processing, etc.) on the data from the A / D converter 212 or the data from the memory control unit 213. Also, the image processing unit 214 performs predetermined arithmetic processing using the captured image data. Based on the obtained arithmetic result, the system control unit 50 performs exposure control or distance measurement control. By this processing, AF processing, AE (automatic exposure) processing, EF (flash pre-emission) processing, etc. in the TTL (through-the-lens) method are performed. Further, the image processing unit 214 performs predetermined arithmetic processing using the captured image data. Based on the obtained arithmetic result, the system control unit 50 performs AWB (auto white balance) processing in the TTL method.

[0029] The image data from the A / D converter 212 is written into the memory 215 via the image processing unit 214 and the memory control unit 213. Alternatively, the image data from the A / D converter 212 is written into the memory 215 via the memory control unit 213 without passing through the image processing unit 214. The memory 215 stores the image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212, and the image data for display on the display unit 108 and the EVF 217. The memory 215 has a storage capacity sufficient to store a predetermined number of still images, a moving image for a predetermined time, and audio. Also, the memory 215 doubles as a memory for image display (video memory).

[0030] The D / A converter 216 converts the image data for display stored in the memory 215 into an analog signal and supplies it to the display unit 108 and the EVF 217. Therefore, the image data for display written in the memory 215 is displayed on the display unit 108 and the EVF 217 via the D / A converter 216. The display unit 108 and the EVF 217 perform display according to the analog signal from the D / A converter 216. The display unit 108 and the EVF 217 are, for example, displays such as an LCD or an organic EL. The digital data A / D-converted by the A / D converter 212 is stored in the memory 215 as a digital signal. The digital signal is converted into an analog signal by the D / A converter 216. The converted analog signal is sequentially transferred to the display unit 108 and the EVF 217. The display unit 108 and the EVF 217 perform live view display by displaying the transferred analog signal.

[0031] The system control unit 50 is a control unit composed of at least one processor and / or at least one circuit. That is, the system control unit 50 may be a processor, may be a circuit, or may be a combination of a processor and a circuit. The system control unit 50 controls the entire camera 100. The system control unit 50 realizes each process of the flowchart described later by executing a program recorded in the non-volatile memory 219. In addition, the system control unit 50 also performs display control by controlling the memory 215, the D / A converter 216, the display unit 108, the EVF 217, etc.

[0032] In addition, the camera 100 includes a system memory 218, a non-volatile memory 219, a system timer 220, a communication unit 221, an attitude detection unit 222, and an eyepiece detection unit 118.

[0033] As the system memory 218, for example, a RAM (Random Access Memory) is used. Constants, variables for the operation of the system control unit 50, programs read from the non-volatile memory 219, etc. are expanded in the system memory 218. The non-volatile memory The non-volatile memory 219 is an electrically erasable and recordable memory. For example, EEPROM (Electrically Erasable Programmable Read-Only Memory) (registered trademark) is used as the non-volatile memory 219. Constants for the operation of the system control unit 50, programs, etc. are recorded in the non-volatile memory 219. The program here is a program for executing the flowchart described later. The system timer 220 is a timing unit that measures the time used for various controls and the time of the built-in clock.

[0034] The communication unit 221 transmits and receives video signals and audio signals to and from external devices connected by wireless or wired cables. The communication unit 221 can also be connected to a wireless LAN (Local Area Network) or the Internet. Further, the communication unit 221 can communicate with external devices using Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 221 can transmit the images captured by the imaging unit 211 (including live images) and the images recorded on the recording medium 227. The communication unit 221 can receive images and various other information from external devices.

[0035] The attitude detection unit 222 is an attitude detection sensor that detects the attitude of the camera 100 with respect to the direction of gravity. Based on the attitude detected by the attitude detection unit 222, it is possible to determine whether the image captured by the imaging unit 211 is an image captured with the camera 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the attitude detected by the attitude detection unit 222 to the image file of the image captured by the imaging unit 211. Also, the system control unit 50 can rotate and record the image according to the attitude detected by the attitude detection unit 222. As the attitude detection unit 222, for example, an acceleration sensor, a gyro sensor, etc. can be used. It is also possible to detect the movement of the camera 100 (such as panning, tilting, lifting, and whether it is stationary or not) using the attitude detection unit 222.

[0036] The eyepiece detection unit 118 can detect the approach of an object to the eyepiece unit 116 (eyepiece finder 117). As the eyepiece detection unit 118, for example, an infrared proximity sensor can be used. When an object approaches, the infrared rays projected from the light projecting unit of the eyepiece detection unit 118 are reflected by the object and received by the light receiving unit of the infrared proximity sensor. The eyepiece detection unit 118 can determine the distance from the eyepiece unit 116 to the object based on the amount of the received infrared rays. In this way, the eyepiece detection unit 118 performs eyepiece detection for detecting the proximity distance of the object to the eyepiece unit 116. The eyepiece detection unit 118 is an eyepiece detection sensor that detects the approach (eyepiece) and separation (eyeleaving) of the eye (object) to the eyepiece unit 116. The eyepiece detection unit 118 detects that the eyepiece is made when an object approaching within a predetermined distance to the eyepiece unit 116 is detected from a non-eyepiece state (non-approaching state). On the other hand, the eyepiece detection unit 118 detects that the eye has left when an object that has been detected approaching moves away by a predetermined distance or more from the eyepiece state (approaching state). The threshold value for detecting the eyepiece and the threshold value for detecting the eye leaving may be different, for example, by providing hysteresis.

[0037] Also, after detecting the eyepiece, it is assumed to be in the eyepiece state until the eye leaving is detected. After detecting the eye leaving, it is assumed to be in the non-eyepiece state until the eyepiece is detected. The system control unit 50 switches the display (display state) / non-display (non-display state) of the display unit 108 and the EVF 217 according to the state detected by the eyepiece detection unit 118. Specifically, the system control unit 50 is at least in the shooting standby state, and when the switching setting of the display destination is automatic switching, the display unit 108 is set as the display destination and is in the display state, and the EVF 217 is in the non-display state when not in the eyepiece. Also, the system control unit 50 sets the EVF 217 as the display destination and is in the display state, and the display unit 108 is in the non-display state when in the eyepiece. Note that the eyepiece detection unit 118 is not limited to the case of being an infrared proximity sensor. Other sensors may be used for the eyepiece detection unit 118 as long as a state that can be regarded as the eyepiece can be detected.

[0038] In addition, the camera 100 includes an external viewfinder display unit 107, an external viewfinder display drive circuit 223, a power control unit 224, a power supply unit 225, a recording medium I / F 226, an operation unit 228, and the like.

[0039] The external viewfinder display unit 107 is driven by the external viewfinder display drive circuit 223 and displays various setting values of the camera 100 such as the shutter speed and aperture.

[0040] The power control unit 224 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching the energized block, and the like. The power control unit 224 detects the presence or absence of a battery, the type of battery, and the remaining battery level. Further, the power control unit 224 controls the DC-DC converter based on the detection result and the instruction of the system control unit 50, and supplies voltage to each unit including the recording medium 227 for a required period.

[0041] The power supply unit 225 includes primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, and Li batteries, and an AC adapter.

[0042] The recording medium I / F 226 is an interface with a recording medium 227 such as a memory card and a hard disk. The recording medium 227 is, for example, a memory card for recording the captured image. The recording medium 227 includes a semiconductor memory, a magnetic disk, and the like. The recording medium 227 may be detachable from the camera 100. The recording medium 227 may be built into the camera 100.

[0043] The operation unit 228 is an input unit (reception unit) capable of receiving operations (user operations) from the user. The operation unit 228 is used to input various instructions to the system control unit 50. The operation unit 228 includes a shutter button 101, a power switch 102, a mode changeover switch 103, a touch panel 109, and other operation units 229. The other operation units 229 include a main electronic dial 104, a sub electronic dial 105, a video button 106, a direction key 110, a SET button 111, an AE lock button 112, a zoom button 113, a play button 114, a menu button 115, a touch bar 119, and the like.

[0044] The shutter button 101 has a first shutter switch 230 and a second shutter switch 231. The first shutter switch 230 turns on during the operation of the shutter button 101, that is, a so-called half-press (shooting preparation instruction), and outputs a first shutter switch signal SW1. The system control unit 50 starts shooting preparation processes such as AF processing, AE processing, AWB processing, and EF processing in response to the first shutter switch signal SW1. The second shutter switch 231 turns on when the operation of the shutter button 101 is completed, that is, a so-called full-press (shooting instruction), and outputs a second shutter switch signal SW2. The system control unit 50 starts a series of shooting processes from reading the signal from the imaging unit 211 to generating an image file including the captured image and writing it to the recording medium 227 in response to the second shutter switch signal SW2.

[0045] The mode switching switch 103 switches the operation mode of the system control unit 50 to any one of a still image shooting mode, a moving image shooting mode, a playback mode, etc. The still image shooting mode includes an auto shooting mode, an auto scene discrimination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), a program AE mode (P mode). Further, the still image shooting mode includes various scene modes and custom modes which are shooting settings according to the shooting scene. The user can directly switch to any one of the above-described shooting modes by the mode switching switch 103. Alternatively, the user can once switch to the list screen of the shooting modes by the mode switching switch 103, and then selectively switch to any one of the plurality of displayed modes by using the operation unit 228. Similarly, the moving image shooting mode may also include a plurality of modes. Similarly, the moving image shooting mode may also include a plurality of modes.

[0046] The touch panel 109 is a touch sensor that detects various touch operations on the display surface of the display unit 108 (the operation surface of the touch panel 109). The touch panel 109 and the display unit 108 can be integrally configured. For example, the touch panel 109 is attached to the upper layer of the display surface of the display unit 108 so that the light transmittance does not interfere with the display of the display unit 108. By associating the input coordinates on the touch panel 109 with the display coordinates on the display surface of the display unit 108, a GUI (Graphical User Interface) can be configured as if the user can directly operate the screen displayed on the display unit 108. Any one of various methods such as a resistive film method, a capacitance method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, an image recognition method, and an optical sensor method can be used for the touch panel 109. Depending on the method, there are a method of detecting a touch when there is contact with the touch panel 109, and a method of detecting a touch when there is an approach of a finger or a pen to the touch panel 109, and any method can be used.

[0047] The system control unit 50 can detect the following operations or states with respect to the touch panel 109. · A finger or pen that was not touching the touch panel 109 newly touches the touch panel 109, that is, the start of a touch (hereinafter referred to as Touch-Down). · A state where the touch panel 109 is being touched by a finger or pen (hereinafter referred to as Touch-On). · The finger or pen that is touching the touch panel 109 is moving while touching it (hereinafter referred to as Touch-Move). · The finger or pen that was touching the touch panel 109 has left (been released from) the touch panel 109, that is, the end of a touch (hereinafter referred to as Touch-Up). · A state where nothing is touching the touch panel 109 (hereinafter referred to as Touch-Off).

[0048] When Touch-Down is detected, Touch-On is also detected at the same time. After Touch-Down, Touch-On is usually continuously detected unless Touch-Up is detected. Even when Touch-Move is detected, Touch-On is continuously detected. If the touch position is not moving even though Touch-On is detected, Touch-Move is not detected. After it is detected that all the fingers or pens that were touching have performed Touch-Up, it becomes Touch-Off.

[0049] These operations, states, and the position coordinates of a finger or pen touching the touch panel 109 are notified to the system control unit 50 through the internal bus. Based on the notified information, the system control unit 50 determines what kind of operation (touch operation) has been performed on the touch panel 109. Regarding a touch move, the moving direction of the finger or pen moving on the touch panel 109 can also be determined for each vertical and horizontal component on the touch panel 109 based on the change in the position coordinates. When it is detected that a touch move has been made over a predetermined distance, it is determined that a slide operation has been performed. An operation of quickly moving a finger a certain distance while touching the touch panel 109 and then releasing it is called a flick. In other words, a flick is an operation of quickly tracing on the touch panel 109 as if flicking with a finger. When it is detected that a touch move has been made over a predetermined distance at a predetermined speed or more and a touch up is detected immediately afterwards, it is determined that a flick has been performed (it can be determined that there is a flick following a slide operation). Furthermore, a touch operation of touching multiple locations (for example, two points) simultaneously (multi-touching) and bringing the touch positions closer to each other is called a pinch-in, and a touch operation of moving the touch positions away from each other is called a pinch-out. The pinch-out and pinch-in are collectively called a pinch operation (or simply a pinch).

[0050] FIG. 4 is a schematic diagram showing an example of the configuration of the lens unit 300. FIG. 4 shows a state in which the lens unit 300 is attached to the camera 100. Among the components of the camera 100 shown in FIG. 4, the same components as those described in FIG. 3 are denoted by the same reference numerals as in FIG. 3, and the description of those components is omitted as appropriate. For the components related to the right eye, an R is added to the end of the reference numeral, for the components related to the left eye, an L is added to the end of the reference numeral, and for the components related to both the right and left eyes, neither R nor L is added to the end.

[0051] The lens unit 300 is a type of interchangeable lens unit that is detachable from the camera 100. The lens unit 300 is a binocular lens unit that enables imaging of a right image and a left image with parallax. The lens unit 300 has two optical systems, and each of the two optical systems can image a range with a wide viewing angle of approximately 180 degrees. Specifically, the lens unit 300 can image a subject within a viewing field (angle of view) of 180 degrees in the left-right direction (horizontal angle, azimuth angle, yaw angle) and 180 degrees in the up-down direction (vertical angle, pitch angle, elevation angle) with each of the two optical systems. That is, the lens unit 300 can image the range of the front hemisphere with each of the two optical systems.

[0052] The lens unit 300 includes an optical system 301R having a plurality of lenses and a reflection mirror, etc., an optical system 301L having a plurality of lenses and a reflection mirror, etc., and a lens system control circuit 303. The optical system 301R has a lens 302R disposed on the subject side, and the optical system 301L has a lens 302L disposed on the subject side. The lens 302R and the lens 302L face the same direction, and their optical axes are substantially parallel.

[0053] The lens unit 300 is a binocular lens unit (VR180 lens unit) for obtaining a VR180 image, which is one of the formats of VR (Virtual Reality) images enabling binocular stereoscopic vision. The lens unit 300 has a fisheye lens capable of capturing a range of approximately 180 degrees in each of the optical system 301R and the optical system 301L. Note that the range that can be captured by the lenses of each of the optical system 301R and the optical system 301L may be about 120 degrees or about 160 degrees, which is narrower than the 180-degree range.

[0054] The lens unit 300 can form a right image formed through the optical system 301R and a left image formed through the optical system 301L on one or two imaging elements of the camera to which the lens unit 300 is attached. In the camera 100, the right image and the left image are formed on one imaging element (imaging sensor), and one image (stereoscopic image) in which the right image area (area of the right image) and the left image area (area of the left image) are arranged side by side is generated.

[0055] The lens unit 300 is attached to the camera 100 via the lens mount portion 304 and the camera mount portion 305 of the camera 100. The system control unit 50 of the camera 100 and the lens system control circuit 303 of the lens unit 300 are electrically connected via the communication terminal 124 of the camera 100 and the communication terminal 306 of the lens unit 300.

[0056] In the configuration of FIG. 4, the right image formed through the optical system 301R and the left image formed through the optical system 301L are formed side by side on the imaging unit 211 of the camera 100. That is, two optical images (subject images) are respectively formed on two regions of one imaging element (imaging sensor) by the optical system 301R and the optical system 301L. The imaging unit 211 converts the formed optical image (optical signal) into an analog electrical signal. In this way, by using the lens unit 300, one image including two image regions with parallax can be acquired from two locations (optical systems) of the optical system 301R and the optical system 301L. The acquired image is used as an image for the left eye and an image for the right eye and divided for VR display, so that the user can view a stereoscopic VR image in a range of approximately 180 degrees. That is, the user can view the VR180 image stereoscopically.

[0057] The lens unit 300 includes a focus ring (not shown) for performing focus adjustment. The lens unit 300 includes, for example, two focus rings: a focus ring for performing focus adjustment on the right image formed via the optical system 301R and a focus ring for performing focus adjustment on the left image formed via the optical system 301L. The lens unit 300 may include a focus ring for simultaneously performing focus adjustment on the right image formed via the optical system 301R and the left image formed via the optical system 301L, and a focus ring for performing focus adjustment on one of the right image or the left image.

[0058] Here, the VR image is an image that can be VR-displayed as described later. The VR image includes an omnidirectional image (omnispherical image) captured by an omnidirectional camera (full-sphere camera), a panoramic image having a video range (effective video range) wider than the display range that can be displayed on the display unit at one time, and the like. Further, the VR image is not limited to a still image, and also includes a moving image and a live image (an image acquired from a camera almost in real time). The VR image has a video range (effective video range) of up to 360 degrees in the left-right direction and 360 degrees in the up-down direction. The VR image also includes an image having an angle of view wider than the angle of view that can be captured by a normal camera or a video range wider than the display range that can be displayed on the display unit at one time, even if it is less than 360 degrees in the left-right direction and less than 360 degrees in the up-down direction. The image captured by the camera 100 using the above-described lens unit 300 is a type of VR image. The VR image can be VR-displayed, for example, by setting the display mode of a display device (a display device capable of displaying a VR image) to "VR view". By displaying a partial range of a VR image having a 360-degree angle of view and changing the posture of the display device in the left-right direction (horizontal rotation direction), the displayed range can be moved, and an omnidirectional video without seams in the left-right direction can be viewed.

[0059] VR display (VR view) is a display method (display mode) with a variable display range that displays the video within the field of view corresponding to the posture of the display device among VR images. In VR display, there is "monocular VR display (monocular VR view)" that performs a transformation (distortion correction) of mapping the VR image onto a virtual sphere to display a single image. Also, in VR display, there is "binocular VR display (binocular VR view)" that performs a transformation of mapping the VR image for the left eye and the VR image for the right eye onto a virtual sphere respectively and arranges and displays them in the left and right regions. By performing "binocular VR display" using the VR images for the left eye and the right eye with a parallax from each other, it is possible to perform stereoscopic viewing of those VR images. In any VR display, for example, when the user wears a display device such as an HMD (head-mounted display), the video within the field of view corresponding to the orientation of the user's face is displayed. For example, assume that among the VR images, the video within the field of view centered at 0 degrees in the left-right direction (a specific azimuth, for example, north) and 90 degrees in the up-down direction (90 degrees from the zenith, that is, horizontal) is being displayed at a certain point in time. When the posture of the display device is reversed front and back from this state (for example, changing the display surface from south-facing to north-facing), the display range is changed to the video within the field of view centered at 180 degrees in the left-right direction (the opposite azimuth, for example, south) and 90 degrees in the up-down direction among the same VR images. That is, when the user wears an HMD and turns the face from north to south (that is, turns to the back), the video displayed on the HMD is also changed from the video of the north to the video of the south. Note that the VR image captured using the lens unit 300 is an image that captures the range of approximately 180 degrees in the front (180° image), and there is no video in the range of approximately 180 degrees in the back. When such an image is VR-displayed and the posture of the display device is changed to the side where there is no video, a blank area is displayed.

[0060] By thus performing VR display of the VR image, the user can visually obtain a feeling (immersive feeling) as if being inside the VR image (inside the VR space). Note that the front The display method is not limited to the method of changing the posture of the display device. For example, the display range may be moved (scrolled) according to a user operation via a touch panel, a direction button, or the like. Also, during VR display (when the display mode is "VR view"), in addition to changing the display range due to a change in posture, the display range may be changed according to a touch move on the touch panel, a drag operation with a mouse, etc., or pressing a direction button. Note that a smartphone mounted on a VR goggle (head-mounted adapter) is a type of HMD.

[0061] Referring to FIG. 5, the guide display process according to Embodiment 1 will be described. The camera 100 displays a guide for notifying the user whether the distance to the subject is a shooting distance at which stereoscopic viewing is possible with a display device such as an HMD. The guide shows the relationship between the recommended distance and the distance from the camera 100 to the subject (hereinafter also referred to as the subject distance). The guide may be represented by a graphic or character information, or may be represented by a display mode such as the color tone of an image. The recommended distance is a shooting distance at which stereoscopic viewing of an imaging image having two image regions with parallax is possible. By displaying a guide showing the relationship between the recommended distance and the subject distance, the camera 100 can assist the user in shooting the subject in a stereoscopically viewable manner.

[0062] In step S501, the system control unit 50 determines whether the display setting of the guide for notifying the user whether the imaging image is stereoscopically viewable with a display device such as an HMD is on. The user can display a screen for setting the display setting of the guide from the menu screen and set the guide display to on or off. When the display setting of the guide is on, the system control unit 50 displays a guide indicating whether the imaging is a stereoscopically viewable imaging when the user takes a picture.

[0063] FIG. 6 shows an example of a setting screen for setting the display of the guide. The user can turn on the display setting of the guide by checking the check box 601 of "guide display". Also, the user can turn off the display setting of the guide by unchecking the check box 601. In step S501, the system control unit 50 can determine whether the display setting of the guide is on or not based on the value set on the setting screen illustrated in FIG. 6. If the display setting of the guide is on, the process proceeds to step S502. If the display setting of the guide is off, the process shown in FIG. 5 ends.

[0064] In step S502, the system control unit 50 acquires information on the recommended distance of the lens unit 300 (hereinafter referred to as recommended distance information). The recommended distance is the shooting distance at which stereoscopic viewing of a captured image having two image regions with parallax is possible. That is, the user can capture an image so that stereoscopic viewing of the captured image including the subject is possible by setting the distance to the subject as the recommended distance.

[0065] The recommended distance varies depending on the baseline length, which is the distance between the left and right lenses (optical systems) of the lens unit 300 that image the two image regions respectively. The distance to the subject for stereoscopically capturing the subject is in a proportional relationship with the baseline length. Therefore, the recommended distance information can be acquired based on the baseline length. Also, the proportional relationship can be obtained based on a model of the interpupillary distance indicating the width between the assumed left and right eyes of a human and a model of the distance to the subject (fixation object) that can be seen stereoscopically with the naked eye. The model of the interpupillary distance is a reference value of the interpupillary distance and is hereinafter referred to as the reference interpupillary distance. The model of the distance to the subject that can be seen stereoscopically with the naked eye is a reference value of the distance to the subject that can be seen stereoscopically and is hereinafter referred to as the reference stereoscopic viewing distance.

[0066] Specifically, the recommended distance can be obtained by the following formulas 1 and 2. The proportionality constant K indicates the proportional relationship between the interpupillary distance and the stereoscopic viewing distance. K = reference stereoscopic viewing distance / reference interpupillary distance ··· (formula 1) Recommended distance = baseline length × K ··· (Equation 2)

[0067] For example, assume that the reference interpupillary distance is 6 cm, which is a common interpupillary distance, and the reference stereoscopic viewing distance is 5 m (500 cm), which is the longest distance assumed to be stereoscopically visible with the naked eye. Also, assume that the baseline length of the lens unit 300 is, for example, 3 cm. In this case, the proportionality constant K = 500 / 6. The longest recommended stereoscopic viewing distance is 3 cm × (500 / 6) = 250 cm (2.5 m).

[0068] If the proportionality constant K is determined, the recommended distance information can be obtained in advance based on the baseline length of the lens unit 300. The camera 100 can hold the obtained recommended distance information in advance.

[0069] Also, the obtained recommended distance information may be held in a recording area within the lens unit 300. The lens system control circuit 303 transmits the recommended distance information held in the recording area to the camera 100 via the communication terminal 306 and the communication terminal 124 of the camera 100. In this way, the system control unit 50 may obtain the recommended distance information from the lens unit 300 attached to the camera 100, which has two optical systems for respectively imaging two image areas.

[0070] Although the recommended distance information has been described as the longest shooting distance at which the captured image can be stereoscopically viewed, it may also include the shortest shooting distance at which the captured image can be stereoscopically viewed. That is, the recommended distance information may be the range of shooting distances at which stereoscopic viewing is possible when a captured image having two image areas is displayed on a display device.

[0071] Also, the lens unit 300 may transmit the information of the baseline length to the camera 100 instead of the recommended distance information. The camera 100 can calculate and obtain the recommended distance using the information of the baseline length obtained from the lens unit 300 according to the above Equations 1 and 2.

[0072] In step S503, the system control unit 50 acquires information on the distance from the camera 100 to the subject (subject distance). The system control unit 50 can acquire the subject distance based on the focal length of the camera 100. For example, the lens system control circuit 303 of the lens unit 300 can acquire the subject distance based on the position of the focus ring that performs focus adjustment. The lens system control circuit 303 transmits the acquired subject distance via the communication terminal 306 and the communication terminal 124 of the camera 100. In this way, the system control unit 50 can acquire the subject distance from the lens unit 300.

[0073] Note that the method for acquiring the subject distance is not limited to the method of acquiring based on the position of the focus ring. For example, the system control unit 50 of the camera 100 can acquire subject distance information based on the parallax between two image regions captured using the lens unit 300. Also, the system control unit 50 can acquire subject distance information based on the phase difference information output by the image plane phase difference sensor included in the camera 100. Further, the system control unit 50 can acquire subject distance information based on the output of the TOF (Time of Flight) sensor included in the camera 100.

[0074] In step S504, the system control unit 50 determines whether the adjustment setting of the recommended distance is on. The user can display a screen for adjusting the recommended distance from the menu screen and set the adjustment of the recommended distance to on or off. When the adjustment setting of the recommended distance is on, the system control unit 50 adjusts the recommended distance based on the user operation.

[0075] FIG. 6 shows an example of a setting screen for adjusting the recommended distance. The user can turn on the adjustment setting of the recommended distance by checking the check box 602 of "Recommended Distance Adjustment". Also, the user can turn off the adjustment setting of the recommended distance by unchecking the check box 602.

[0076] In step S504, the system control unit 50 can determine whether the adjustment setting of the recommended distance is on or off based on the value set on the setting screen illustrated in FIG. 6. If the adjustment setting of the recommended distance is on, the process proceeds to step S505. If the adjustment setting of the recommended distance is off, the process proceeds to step S507.

[0077] In step S505, the system control unit 50 acquires an adjustment value for adjusting the recommended distance. The adjustment of the recommended distance is a process of adjusting the recommended distance acquired in step S502. Since the recommended distance is a value predetermined based on an assumed model, the three-dimensional feeling when viewing on a display device such as an HMD may be different from the three-dimensional feeling assumed during shooting for some users. Therefore, the camera 100 may have means for the user to adjust the recommended distance.

[0078] FIG. 6 shows an example of a setting screen for setting an adjustment value for adjusting the recommended distance. The user can instruct the recommended distance to be increased by moving the handle 604 to the + side with the slider 603 for changing the adjustment value. In this case, the system control unit 50 can adjust the recommended distance by multiplying the recommended distance by a coefficient α1 (α1>1) corresponding to the scale of the slider 603 as the adjustment value. Also, the user can instruct the recommended distance to be decreased by moving the handle 604 to the - side. In this case, the system control unit 50 can adjust the recommended distance by multiplying the recommended distance by a coefficient α2 (0<α2<1) corresponding to the scale of the slider 603 as the adjustment value. Note that the adjustment value of the recommended distance may be set while the user views the captured image displayed on a display device such as an HMD.

[0079] Alternatively, the adjustment value may be automatically set based on the distance between the pupils obtained from the user's face image. FIG. 7 is a flowchart illustrating the setting process of the adjustment value for the recommended distance. In step S701, the system control unit 50 acquires the user's face image. For example, the user's face image is an captured image captured using a single-eye lens provided in the camera 100, rather than the lens unit 300 having two optical systems. Further, the user's face image may be an image captured in advance with the single-eye lens unit attached to the camera 100.

[0080] In step S702, the system control unit 50 detects the pupils from the user's face image acquired in step S701, and acquires the center coordinates of each of the detected pupils. The system control unit 50 can detect the pupils by a known detection method. In step S702, it is only necessary to acquire the center coordinates of each of the pupils detected from the image in which the user's face is captured, and the system control unit 50 may use any detection method.

[0081] In step S703, the system control unit 50 determines whether two pupils are detected in step S702. If two pupils are detected, the process proceeds to step S704. If the number of detected pupils is less than two or three or more, since the face image is not appropriate, the system control unit 50 returns to step S701 to acquire the face image again.

[0082] In step S704, the system control unit 50 acquires the inter-pupillary distance based on the coordinate information of the two pupils detected in step S702. The system control unit 50 can calculate the inter-pupillary distance using the following formulas 3 and 4. Inter-pupillary distance on the image plane = Distance between the coordinates of the two pupils × Pixel size ··· (Formula 3) Inter-pupillary distance = Inter-pupillary distance on the image plane × Subject distance / Focal length ··· (Formula 4)

[0083] The pixel size of Equation 3 is a value corresponding to the size per pixel of the imaging device. The subject distance of Equation 4 can be obtained, for example, based on the position of the focus ring of the lens at the time of face image shooting. The focal length is the focal length of the lens at the time of face image shooting. When the distortion due to lens distortion aberration or the like is large, the system control unit 50 can improve the accuracy of the calculation result by calculating the interpupillary distance in consideration of the distortion.

[0084] In step S705, the system control unit 50 acquires an adjustment value. The system control unit 50 can acquire an adjustment value α3 by the following Equation 5 based on the interpupillary distance acquired in step S704 and the reference interpupillary distance. α3 = reference interpupillary distance / interpupillary distance acquired in step S704 ··· (Equation 5) The system control unit 50 stores the acquired adjustment value α in the memory 215 or the like.

[0085] As described above, in step S505 of FIG. 5, the system control unit 50 can acquire the adjustment value (for example, coefficient α1 and coefficient α2) set by the user or the automatically set adjustment value α3.

[0086] In step S506, the system control unit 50 adjusts the recommended distance using the adjustment value acquired in step S505. The system control unit 50 adjusts the recommended distance, for example, using the adjustment values (coefficient α1 and coefficient α2) acquired based on the user's instruction on the setting screen shown in FIG. 6. Specifically, the system control unit 50 can adjust the recommended distance by multiplying the coefficient α1 and the coefficient α2 by the recommended distance.

[0087] In addition, the system control unit 50 may adjust the recommended distance using the adjustment value obtained based on the user's interpupillary distance. Specifically, the system control unit 50 can adjust the recommended distance by multiplying the adjustment value α3 obtained by Equation 5 by the recommended distance. That is, when the interpupillary distance obtained in step S704 of FIG. 7 is shorter than the assumed reference interpupillary distance, the system control unit 50 can adjust the recommended distance to be longer based on the adjustment value α3. Also, when the interpupillary distance obtained in step S704 of FIG. 7 is longer than the assumed reference interpupillary distance, the system control unit 50 can adjust the recommended distance to be shorter based on the adjustment value α3.

[0088] With reference to FIGS. 1(A) and 1(C), the recommended distances when the user's interpupillary distance is longer and shorter than the reference interpupillary distance will be described. The distance between the left and right eyes shown in FIG. 1(A) is defined as the reference interpupillary distance. The convergence angle S1 is set as the minimum angle at which the subject appears three-dimensional. The convergence angle S3 in FIG. 1(C) is approximately the same angle as the convergence angle S1, and the recommended distance is the distance from the camera in FIG. 1(C) to the subject.

[0089] When the user's interpupillary distance obtained in step S704 of FIG. 7 is shorter than the reference interpupillary distance in FIG. 1(A), the minimum angle at which the subject appears three-dimensional becomes smaller than the convergence angle S1. Since the convergence angle S3 in FIG. 1(C) becomes smaller, the recommended distance, which is the distance from the camera 100 to the subject, becomes longer. On the other hand, when the user's interpupillary distance obtained in step S704 of FIG. 7 is longer than the reference interpupillary distance in FIG. 1(A), the minimum angle at which the subject appears three-dimensional becomes larger than the convergence angle S1. Since the convergence angle S3 in FIG. 1(C) becomes larger, the recommended distance, which is the distance from the camera 100 to the subject, becomes shorter.

[0090] In step S507, the system control unit 50 compares the subject distance with the recommended distance obtained in step S502. When it is determined in step S504 that the adjustment setting of the recommended distance is on and the recommended distance is adjusted in step S506, the system control unit 50 Compare the recommended distance with the subject distance.

[0091] In step S508, the system control unit 50 displays a guide on the display unit 108 based on the relationship between the subject distance and the recommended distance compared in step S507. Display examples of the guide will be described with reference to FIGS. 8(A) to 8(C), FIGS. 9(A) to 9(D), and FIGS. 10(A) to 10(D).

[0092] FIGS. 8(A) to 8(C) are diagrams for explaining display examples of a guide showing the relationship between the recommended distance and the subject distance according to the color tone of the image. The display unit 108 performs a live view display for displaying, in real time, a captured image including two image regions captured by the two optical systems 301R and 301L of the lens unit 300. The display unit 108 controls to color-code the two image regions based on the relationship between the recommended distance and the subject distance.

[0093] FIG. 8(A) shows a state where the subject distance and the recommended distance are substantially the same. When the subject distance and the recommended distance are substantially the same, the display unit 108 does not perform any special display. Also, when the recommended distance information is within the range from the shortest value to the longest value and the subject distance is within this range, the display unit 108 does not perform any special display as shown in FIG. 8(A).

[0094] FIG. 8(B) shows a state where the subject distance is longer than the recommended distance. When the subject distance is longer than the recommended distance, the display unit 108 displays, for example, the color tone of the live view image, which is an image obtained by the live view display, in warm colors. The slashes in the live view image of FIG. 8(B) indicate that the image is color-coded in warm colors. Also, when the recommended distance information is within the range from the shortest value to the longest value and the subject distance is longer than the longest value of this range, the display unit 108 displays the color tone of the live view image in warm colors as shown in FIG. 8(B).

[0095] FIG. 8(C) shows a state where the subject distance is shorter than the recommended distance. When the subject distance is shorter than the recommended distance, the display unit 108 displays, for example, the color tone of the live view image in a cold color. The shading in the live view image of FIG. 8(C) indicates that the image is colored in a cold color. Also, when the recommended distance information is information in the range from the shortest value to the longest value, and the subject distance is shorter than the shortest value of the range, the display unit 108 also displays the color tone of the live view image in a cold color as shown in FIG. 8(C).

[0096] The display unit 108 can notify the user who is the photographer whether the subject distance is the recommended distance or not by changing the color tone of the display image based on whether the subject distance is longer or shorter than the recommended distance. The color indicating whether the subject distance is the recommended distance or not is not limited to examples of warm colors and cold colors, and other colors may be used as long as the user can recognize the relationship between the subject distance and the recommended distance.

[0097] In the examples of FIGS. 8(A) to 8(C), the user can recognize the relationship between the subject distance and the recommended distance from the difference in the color tone of the live view image. The user can adjust the shooting position so that the subject can be photographed at the recommended distance for three-dimensional display of the subject by approaching or moving away from the subject according to the relationship between the subject distance and the recommended distance.

[0098] FIGS. 9(A) to 9(D) are diagrams for explaining display examples of a guide showing the positional relationship between the recommended distance, the subject, and the camera 100. The display unit 108 controls to display two image areas in live view and display a diagram (item) showing the positional relationship between the recommended distance, the subject, and the camera 100 as a guide indicating the relationship between the subject distance and the recommended distance.

[0099] In the example of FIG. 9(A), the guide indicating the relationship between the subject distance and the recommended distance is, for example, the live view It is displayed in the display area 900 at the lower part of the new image. With reference to FIGS. 9(B) to 9(D), a specific example of a guide showing the relationship between the subject distance and the recommended distance will be described. In FIGS. 9(B) to 9(D), the recommended distance is shown in the range from the shortest value to the longest value, but it may also be shown depending on the position of the longest value of the recommended distance.

[0100] FIG. 9(B) shows an example of a guide when the subject distance is longer than the recommended distance. The position 901 of the camera 100 is displayed at a position farther from the subject 903 than the range 902 of the recommended distance. The subject distance is the distance from the position 901 of the camera 100 to the subject 903. The recommended distance is the distance from the position 901 of the camera 100 to any position within the range 902 of the recommended distance. From the guide shown in FIG. 9(B), the user can recognize that the subject distance is longer than the recommended distance.

[0101] FIG. 9(C) shows an example of a guide when the subject distance substantially coincides with the recommended distance, or when the subject distance is included in the range 902 of the recommended distance. The position 901 of the camera 100 is displayed within the range 902 of the recommended distance. From the guide shown in FIG. 9(C), the user can recognize that the subject distance is included in the range 902 of the recommended distance.

[0102] FIG. 9(D) shows an example of a guide when the subject distance is shorter than the recommended distance. The position 901 of the camera 100 is displayed at a position closer to the subject 903 than the range 902 of the recommended distance. From the guide shown in FIG. 9(D), the user can recognize that the subject distance is shorter than the recommended distance.

[0103] Figures 10(A) to 10(D) are diagrams for explaining display examples of a guide showing the relationship between the recommended distance and the subject distance by text (character information). The display unit 108 controls to display two image regions in live view and display text showing the relationship between the subject distance and the recommended distance. In the example of Fig. 10(A), the text showing the relationship between the subject distance and the recommended distance is displayed, for example, in the display area 1000 at the bottom of the live view image. With reference to Figs. 10(B) to 10(D), specific examples of the text showing the relationship between the subject distance and the recommended distance will be described.

[0104] Fig. 10(B) shows an example of the text displayed when the subject distance is longer than the recommended distance. The text indicates that the position of the camera 100 is farther from the subject than the position of the recommended distance or the range of the recommended distance. Fig. 10(C) shows an example of the text displayed when the subject distance substantially coincides with the recommended distance or the subject distance is included in the range of the recommended distance. The text indicates that the position of the camera 100 substantially coincides with the position of the recommended distance or is included in the range of the recommended distance. Fig. 10(D) shows an example of the text displayed when the subject distance is shorter than the recommended distance. The text indicates that the position of the camera 100 is closer to the subject than the position of the recommended distance or the range of the recommended distance. By displaying the text described in Figs. 10(B) to 10(D) in the live view image, the user can recognize the relationship between the recommended distance and the subject distance.

[0105] The display examples of the guide are not limited to the examples described in Figs. 8(A) to 8(C), Figs. 9(A) to 9(D), and Figs. 10(A) to 10(D). Other methods may be used as long as the user, who is the photographer, can recognize the relationship between the subject distance and the recommended distance.

[0106] In the above-described Embodiment 1, the camera 100 performs a guide display for notifying the user of the relationship between the recommended distance and the subject distance. Thereby, the user can easily capture an image that can be viewed stereoscopically on the display device.

[0107] <Embodiment 2> In the first embodiment, the relationship between the distance from the camera 100 to a specific subject and the recommended distance is displayed to the user. In the second embodiment, a distance map is generated for each of two image regions, and the relationship between the recommended distance and the subject distance is notified to the user for each region (block) in each image region. The image region can be divided into blocks based on the relationship between the recommended distance and the subject distance.

[0108] The guide display process according to the second embodiment will be described with reference to Fig. 11. Detailed description of the process similar to the guide display process according to the first embodiment in Fig. 5 will be omitted.

[0109] In step S1101, the system control unit 50 determines whether a guide display setting that notifies the user whether a captured image can be displayed stereoscopically on a display device such as an HMD is on. The process of step S1101 is similar to the process of step S501 in Fig. 5, so a detailed description is omitted. If the guide display setting is on, the process proceeds to step S1102. If the guide display setting is off, the process shown in Fig. 11 ends.

[0110] In step S1102, the system control unit 50 determines whether or not recommended distance information for the lens unit 300 attached to the camera 100 is stored in the non-volatile memory 219 of the camera 100. The recommended distance information is stored, for example, in a recommended distance information table illustrated in FIG.

[0111] FIG. 12 is a diagram illustrating a recommended distance information table. The recommended distance differs for each type of lens unit 300. The recommended distance information table holds information on the minimum and maximum recommended distances for each lens ID indicating the type of lens unit used for shooting. For example, the minimum recommended distance for a lens unit 300 with a lens ID of "001" is "x1" and the maximum recommended distance is "x2". The system control unit 50 can refer to the recommended distance information table to obtain recommended distance information corresponding to the lens ID of the lens unit 300 attached to the camera 100.

[0112] In step S1102, when the lens ID of the lens unit 300 mounted on the camera 100 at the time of shooting exists in the recommended distance information table, the system control unit 50 can determine that the recommended distance information of the lens unit 300 is held. When the recommended distance information of the lens unit 300 is held, the process proceeds to step S1103. When the recommended distance information of the lens unit 300 is not held, the process shown in FIG. 11 ends.

[0113] In step S1103, the system control unit 50 acquires the recommended distance information corresponding to the lens unit 300 mounted on the camera 100. The system control unit 50 can acquire the recommended distance information from the recommended distance information table. By holding the recommended distance information in the camera 100, the system control unit 50 can acquire the recommended distance information without receiving it from the lens unit 300.

[0114] In step S1104, the system control unit 50 generates a distance map (subject distance map) indicating the subject distance information for the entire captured image. The system control unit 50 can generate a distance map for the entire captured image by calculating the distance using a stereo camera by imaging using the two-lens unit 300. Since the method of calculating the distance using a stereo camera is a known technique, a detailed description thereof is omitted.

[0115] The system control unit 50 can calculate the subject distance, for example, by the following equation 6. Let W be the baseline length of the left and right lenses, F be the focal length, and D be the parallax. Subject distance L = W × F / D ··· (Equation 6)

[0116] The system control unit 50 performs the calculation according to Equation 6 over the entire captured image. Disparity information can be calculated in areas where subject information such as edges exists, but cannot be obtained in flat portions of the captured image. Therefore, the system control unit 50 can generate a distance map for the entire captured image by, for example, interpolating the distance information of the flat portion using the surrounding distance information.

[0117] The method for generating the distance map is not limited to the method based on distance calculation using a stereo camera. For example, when the imaging unit 211 has an imaging plane phase difference sensor, the system control unit 50 may generate a distance map of the captured image based on the phase difference information output by the imaging plane phase difference sensor. Also, when the camera 100 has a distance measurement sensor such as a TOF (Time of Flight) sensor, a distance map of the captured image may be generated based on the output of the sensor.

[0118] FIG. 15(A) is a diagram showing an example of a distance map. Two image areas each hold distance information, and the distance map is an image diagram showing each image area in shading based on the value of the subject distance in order to visualize the distance information.

[0119] In step S1105, the system control unit 50 determines whether the adjustment setting of the recommended distance is on. Since the process of step S1105 is the same as the process of step S504 in FIG. 5, a detailed description thereof is omitted.

[0120] In step S1106, the system control unit 50 acquires an adjustment value for adjusting the recommended distance. The system control unit 50 can acquire the adjustment value of the recommended distance using the inter-pupillary distance of the user acquired from an external device. FIG. 13 is a flowchart illustrating the setting process of the adjustment value of the recommended distance.

[0121] In step S1301, the system control unit 50 connects to an external device. The external device is, for example, an HMD. FIG. 14 is a diagram for explaining the connection between the camera 100 and the HMD 400 according to Embodiment 2. The camera 100 is connected to the HMD 400, for example, by wireless connection. Note that the camera 100 may be connected to the HMD 400 by wire. The system control unit 50 wirelessly connects to the HMD 400 via the communication unit 221, enabling the transmission and reception of various data between the camera 100 and the HMD 400.

[0122] In step S1302, the system control unit 50 acquires the interpupillary distance from the HMD 400, which is an external device. The HMD 400 has a built-in camera for imaging the user's eyes to detect the user's line of sight, and can measure the user's interpupillary distance using the captured image of the eyes. The HMD 400 transmits the acquired interpupillary distance to the camera 100. In this way, the camera 100 can acquire the interpupillary distance from the HMD 400.

[0123] In step S1303, the system control unit 50 acquires an adjustment value α3 according to Equation 5 in the same manner as in step S705 of FIG. 7 based on the interpupillary distance acquired in step S1302. Note that the external device is not limited to the HMD 400, and any other device that can image the user's face image to measure the interpupillary distance may be used. Further, the system control unit 50 may acquire the adjustment value calculated by the external device using the interpupillary distance.

[0124] In step S1106 of Embodiment 2, an example of acquiring an adjustment value based on information from an external device was described. However, the adjustment value may be acquired by the camera 100 in the same manner as in Embodiment 1. In step S505 of Embodiment 1, an example of the camera 100 acquiring an adjustment value was described. However, the adjustment value in Embodiment 1 may also be acquired based on information from an external device in the same manner as in Embodiment 2.

[0125] In step S1107 of FIG. 11, the system control unit 50 adjusts the recommended distance using the adjustment value acquired in step S1106. Since the process of step S1107 is the same as the process of step S506 in FIG. 5, a detailed description thereof is omitted.

[0126] In step S1108, the system control unit 50 compares the distance map generated in step S1104 with the recommended distance acquired in step S1103. Based on the comparison result, the system control unit 50 generates a comparison map indicating the magnitude relationship between the subject distance and the recommended distance for each area (block). When it is determined in step S1105 that the adjustment setting of the recommended distance is on and the recommended distance is adjusted in step S1107, the system control unit 50 generates a comparison map using the adjusted recommended distance.

[0127] FIG. 15(B) is a diagram showing an example of the comparison map. The comparison map is an image diagram that distinguishes blocks farther than the recommended distance, blocks at the same distance as the recommended distance, and blocks closer than the recommended distance by shading. When the recommended distance information is within the range of the recommended distance, the system control unit 50 may generate a comparison map that distinguishes blocks farther than the maximum value of the recommended distance, blocks within the range of the recommended distance, and blocks closer than the minimum value of the recommended distance by shading.

[0128] In step S1109, the system control unit 50 displays the captured image on the display unit 108 based on the comparison map generated in step S1108 so that the relationship between the recommended distance and the subject distance can be understood.

[0129] FIG. 15(C) is a diagram showing an example of a guide display of a captured image including two image regions. In the example of FIG. 15(C), blocks farther than the recommended distance and blocks closer than the recommended distance are indicated by hatching. Each block may be colored in a different color based on the relationship between the recommended distance and the subject distance. A user, who is a photographer, can recognize whether the subject distance for each block is the recommended distance or not based on differences in display modes such as hatching or coloring of each block. The user can adjust the shooting position so that a desired subject is included in a block at the recommended distance by approaching or moving away from the subject according to the guide display as shown in FIG. 15(C).

[0130] In the above-described Second Embodiment, the camera 100 can notify the user of the relationship between the recommended distance and the subject distance for each block (region) within the image region by generating a distance map of a captured image including two image regions. The user can easily adjust the shooting distance so that a desired subject is stereoscopically displayed on the display device.

[0131] Note that the above-described embodiments are merely examples, and configurations obtained by appropriately modifying or changing the configurations of the above-described embodiments within the scope of the gist of the present invention are also included in the present invention. Configurations obtained by appropriately combining the configurations of the above-described embodiments are also included in the present invention.

[0132] <Other Embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium and causing one or more processors in a computer of the system or device to read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0133] The disclosure of the present embodiment includes the following configurations, methods, programs, and media. (Configuration 1) An acquisition unit that acquires information on the distance from the imaging device to the subject, a shooting distance at which stereoscopic viewing of a captured image having two image regions with different parallaxes is possible control means for controlling to notify the relationship between the recommended distance and the distance from the imaging device to the subject An electronic device characterized by comprising: (Configuration 2) The acquisition means acquires the distance from the imaging device to the subject based on the focal length of the imaging device. The electronic device according to Configuration 1, characterized in that. (Configuration 3) The acquisition means acquires information on the distance from the imaging device to the subject based on the parallax between the two image regions. The electronic device according to Configuration 1, characterized in that. (Configuration 4) The acquisition means acquires information on the distance from the imaging device to the subject based on the phase difference information output by the image plane phase difference sensor of the imaging device. The electronic device according to Configuration 1, characterized in that. (Configuration 5) The acquisition means acquires information on the distance from the imaging device to the subject based on the output of the TOF (Time of Flight) sensor of the imaging device. The electronic device according to Configuration 1, characterized in that. (Configuration 6) further comprising setting means for setting whether to display information indicating the relationship between the recommended distance and the distance from the imaging device to the subject The electronic device according to any one of Configurations 1 to 5, characterized in that. (Configuration 7) further comprising second acquisition means for acquiring information on the recommended distance The electronic device according to any one of Configurations 1 to 6, characterized in that. (Configuration 8) The second acquisition means acquires information on the recommended distance based on the baseline length, which is the distance between two optical systems that respectively image the two image regions. The electronic device according to Configuration 7, characterized in that. (Configuration 9) The second acquisition means acquires the information on the baseline length from a lens unit attached to the imaging device, the lens unit having two optical systems for imaging the two image areas. The electronic device according to Configuration 8, characterized in that. (Configuration 10) The second acquisition means acquires the information on the recommended distance from a lens unit attached to the imaging device, the lens unit having two optical systems for imaging the two image areas respectively. The electronic device according to Configuration 7, characterized in that. (Configuration 11) The electronic device according to Configuration 7, characterized in that the second acquisition means acquires the information on the recommended distance that the imaging device holds in advance. (Configuration 12) Further comprising adjustment means for adjusting the recommended distance The electronic device according to any one of Configurations 1 to 11, characterized in that. (Configuration 13) The adjustment means adjusts the information on the recommended distance based on the interpupillary distance of the user. The electronic device according to Configuration 12, characterized in that. (Configuration 14) When the interpupillary distance is shorter than a reference value of the interpupillary distance, the adjustment means adjusts so that the recommended distance becomes longer, and when the interpupillary distance is longer than the reference value, the recommended distance is adjusted to be shorter. The electronic device according to Configuration 13, characterized in that. (Configuration 15) The adjustment means acquires the interpupillary distance from the captured image of the user. The electronic device according to Configuration 13 or 14, characterized in that. (Configuration 16) The adjustment means acquires the interpupillary distance measured by an external device. The electronic device according to Configuration 13 or 14, characterized in that. (Configuration 17) The adjustment means adjusts the recommended distance based on an instruction from the user. The electronic device according to Configuration 12, characterized by the above. (Configuration 18) The information on the recommended distance includes the range of shooting distances at which stereoscopic viewing of the captured image having the two image regions is possible. The electronic device according to any one of Configurations 1 to 17, characterized by the above. (Configuration 19) The control means controls to color-code the two image regions based on the relationship between the recommended distance and the distance from the imaging device to the subject. The electronic device according to any one of Configurations 1 to 18, characterized by the above. (Configuration 20) The control means controls to display an item indicating the positional relationship among the recommended distance, the subject, and the imaging device. The electronic device according to any one of Configurations 1 to 18, characterized by the above. (Configuration 21) The control means controls to display character information indicating the relationship between the recommended distance and the distance from the imaging device to the subject. The electronic device according to any one of Configurations 1 to 18, characterized by the above. (Configuration 22) The control means controls to notify the relationship between the recommended distance and the distance from the imaging device to the subject for each block within each of the two image regions. The electronic device according to any one of Configurations 1 to 19, characterized by the above. (Method) A step of acquiring information on the distance from the imaging device to the subject, A step of controlling to notify the relationship between the recommended distance, which is a shooting distance at which stereoscopic viewing of the captured image having two image regions with parallax from each other is possible, and the distance from the imaging device to the subject, A control method for an electronic device, characterized by including the above steps. (Program) A program for causing a computer to function as each means of the electronic device according to any one of Configurations 1 to 22. (Medium) A computer-readable storage medium storing a program for causing a computer to function as each means of the electronic device described in any one of Configurations 1 to 22.

Explanation of Signs

[0134] 100: Camera (electronic device), 50: System control unit

Claims

1. An acquisition means for acquiring information on the distance from the imaging device to the subject, a recommended distance that is a shooting distance at which stereoscopic vision of an imaging image having two image regions with parallax with each other is possible, and a control means for controlling to notify the relationship between the distance from the imaging device to the subject An electronic device characterized by comprising.

2. The acquisition means acquires the distance from the imaging device to the subject based on the focal length of the imaging device The electronic device according to claim 1, characterized in that.

3. The acquisition means acquires the information on the distance from the imaging device to the subject based on the parallax of the two image regions The electronic device according to claim 1, characterized in that.

4. The acquisition means acquires the information on the distance from the imaging device to the subject based on the phase difference information output by the image plane phase difference sensor of the imaging device The electronic device according to claim 1, characterized in that.

5. The acquisition means acquires the information on the distance from the imaging device to the subject based on the output of the TOF (Time of Flight) sensor of the imaging device The electronic device according to claim 1, characterized in that.

6. Further comprising a setting means for setting whether to display information indicating the relationship between the recommended distance and the distance from the imaging device to the subject The electronic device according to claim 1, characterized in that.

7. Further comprising a second acquisition means for acquiring information on the recommended distance The electronic device according to claim 1, characterized in that.

8. The second acquisition means acquires the information on the recommended distance based on the baseline length which is the distance between two optical systems that respectively image the two image regions The electronic device according to claim 7, characterized in that.

9. The second acquisition means acquires the information on the baseline length from a lens unit attached to the imaging device, which has two optical systems for imaging the two image regions The electronic device according to claim 8, characterized in that.

10. The second acquisition means acquires the information on the recommended distance from a lens unit attached to the imaging device, which has two optical systems that respectively image the two image regions The electronic device according to claim 7, characterized in that.

11. The second acquisition means acquires the information on the recommended distance held in advance by the imaging device, The electronic device according to claim 7, characterized in that.

12. further comprising adjustment means for adjusting the recommended distance The electronic device according to claim 1, characterized in that

13. The adjustment means adjusts the information on the recommended distance based on the inter-pupillary distance of the user The electronic device according to claim 12, characterized in that

14. When the inter-pupillary distance is shorter than a reference value of the inter-pupillary distance, the adjustment means adjusts the recommended distance to be longer, and when the inter-pupillary distance is longer than the reference value, the adjustment means adjusts the recommended distance to be shorter The electronic device according to claim 13, characterized in that

15. The adjustment means obtains the inter-pupillary distance from the captured image of the user The electronic device according to claim 13, characterized in that

16. The adjustment means obtains the inter-pupillary distance measured by an external device The electronic device according to claim 13, characterized in that

17. The adjustment means adjusts the recommended distance based on an instruction from the user The electronic device according to claim 12, characterized in that

18. The information on the recommended distance includes a range of shooting distances at which stereoscopic viewing of a captured image having the two image areas is possible The electronic device according to claim 1, characterized in that

19. The control means controls to color-code the two image areas based on the relationship between the recommended distance and the distance from the imaging device to the subject The electronic device according to claim 1, characterized in that

20. The control means controls to display an item indicating the positional relationship between the recommended distance, the subject, and the imaging device The electronic device according to claim 1, characterized in that

21. The control means controls to display character information indicating the relationship between the recommended distance and the distance from the imaging device to the subject The electronic device according to claim 1, characterized in that

22. The control means controls to notify the relationship between the recommended distance and the distance from the imaging device to the subject for each block within each of the two image areas The electronic device according to claim 1, characterized in that

23. A step of obtaining information on the distance from the imaging device to the subject, and A step of controlling to notify the relationship between a recommended distance, which is a shooting distance at which stereoscopic viewing of a captured image having two image areas with a parallax therebetween is possible, and the distance from the imaging device to the subject A control method for an electronic device, characterized by comprising

24. A program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 22.

25. A computer-readable storage medium storing a program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 22.

Images