Electronic device, control method of electronic device, computer program product, and computer readable storage medium

Abstract

The invention discloses an electronic device, a control method of the electronic device, a computer program product and a computer readable storage medium. The electronic apparatus includes: an acquisition unit configured to acquire information related to a distance from an imaging apparatus to a subject; a control unit configured to control notification of a relationship between a recommended distance and a distance from the imaging apparatus to the subject, the recommended distance representing a photographing distance that enables three-dimensional viewing of a photographed image including two image regions having a parallax with respect to each other; and a setting unit configured to set whether to display information showing a relationship between the recommended distance and a distance from the imaging apparatus to the subject.

Description

Technical Field

[0001] Aspects of embodiments relate to an electronic device, a control method for an electronic device, a computer program product, and a computer-readable storage medium. Background Art

[0002] There is known a technique of obtaining an image including two image regions having parallax with respect to each other by using two optical systems facing the same direction and displaying the two image regions for three-dimensional viewing. If a circumferential fish-eye lens is used for each optical system, as each image region, an image region having a wide range showing at least 180 degrees (hemispherical, 90 degrees in all directions from the image center) in both the vertical and horizontal directions can be obtained.

[0003] Japanese Unexamined Patent Application Publication No. 2001-281754 discloses a camera capable of stereoscopic photography, which is designed to photograph the same subject from different viewpoints with their lines of sight parallel to each other.

[0004] Japanese Unexamined Patent Application Publication No. 2010-177921 discloses a three-dimensional imaging device that detects corresponding points between a plurality of captured images obtained by photographing a subject field from a plurality of viewpoints. Summary of the Invention

[0005] A first aspect of an embodiment is an electronic device including: an acquisition unit configured to acquire information related to the distance from an imaging device to a subject; a control unit configured to control notification of the relationship between a recommended distance and the distance from the imaging device to the subject, the recommended distance indicating a photographic distance that enables three-dimensional viewing of a captured image including two image regions having parallax with respect to each other; and a setting unit configured to set whether to display information showing the relationship between the recommended distance and the distance from the imaging device to the subject.

[0006] A second aspect of an embodiment is a control method for an electronic device, the control method including the steps of: acquiring information related to the distance from an imaging device to a subject; controlling notification of the relationship between a recommended distance and the distance from the imaging device to the subject, the recommended distance indicating a photographic distance that enables three-dimensional viewing of a captured image including two image regions having parallax with respect to each other; and setting whether to display information showing the relationship between the recommended distance and the distance from the imaging device to the subject.

[0007] A third aspect of an embodiment is a computer program product including a program that causes a computer to execute each of the steps of the control method for the electronic device described above. A fourth aspect of an embodiment is a computer-readable storage medium storing a program that causes a computer to execute each of the steps of the control method for the electronic device described above.

[0008] Other features of the present disclosure will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1A is a schematic diagram of the eyes of a user viewing a subject;

[0010] Figure 1B and Figure 1C is a schematic diagram of an imaging device that captures a subject;

[0011] Figure 2A and Figure 2B is an external view of a camera;

[0012] Figure 3 is a block diagram of the camera;

[0013] Figure 4 is a schematic diagram showing the configuration of a lens unit;

[0014] Figure 5 is a flowchart showing the guidance display process according to the first embodiment;

[0015] Figure 6 is a diagram showing an example of a setting screen;

[0016] Figure 7 is a flowchart showing the setting process of adjustment values according to the first embodiment;

[0017] Figures 8A to 8C is a diagram describing a display example of guidance using image tones;

[0018] Figures 9A to 9D is a diagram describing a display example of guidance showing a positional relationship;

[0019] Figures 10A to 10D is a diagram describing a display example of guidance using text;

[0020] Figure 11 is a flowchart showing the guidance display process according to the second embodiment;

[0021] Figure 12 is a diagram showing a recommended distance information table;

[0022] Figure 13 is a flowchart showing the setting process of adjustment values according to the second embodiment;

[0023] Figure 14 is a diagram describing the connection between the camera and the HMD according to the second embodiment;

[0024] Figure 15A is a diagram showing an example of a distance map;

[0025] Figure 15B is a diagram showing an example of a comparison diagram; and

[0026] Figure 15C is a diagram showing an example of a guided display. Detailed Description

[0027] First Embodiment

[0028] Embodiments of the present disclosure will be described below with reference to the drawings. Figures 1A to 1C is a schematic diagram of the eyes of a user viewing a subject and an imaging device that images the subject. Referring to Figures 1A to 1C , the reason why the three-dimensional effect perceived when a user views an image including two image regions with parallax on a display device is different from the three-dimensional effect perceived when the user actually views the subject will be described.

[0029] Figure 1A is a schematic diagram of a user viewing a subject with the left and right eyes. The convergence angle S1 represents the angle formed by the lines (lines of sight) connecting the subject to the left and right eyes. Figure 1B is a schematic diagram showing a subject being imaged by an imaging device having two optical systems. Figure 1B The distance from the imaging device to the subject in Figure 1A is the same as the distance from the user to the subject in

[0030] The distance between the left and right eyes is different from the distance between the two optical systems. Therefore, the convergence angle S1 is different from the convergence angle S2. In other words, the parallax between the left image region and the right image region included in the image captured by the imaging device is different from the parallax between the regions viewed by the user with the left and right eyes from the same position as the imaging device. When the convergence angle S2 is smaller than the convergence angle S1, the parallax becomes smaller in the case where the subject is imaged by the imaging device compared to the case where the user actually views the subject. When the convergence angle S2 of the imaging by the imaging device is smaller than the convergence angle S1, the three-dimensional effect perceived when viewing the image captured by the imaging device on the display device is reduced. As described above, the three-dimensional effect perceived when the user views the subject from the imaging position in Figure 1A is different from the three-dimensional effect perceived when the user views the captured image on a display device such as a head-mounted display (HMD).

[0031] Figure 1C is a schematic diagram of a subject being imaged by an imaging device from a position closer to the subject than the position in Figure 1B . The convergence angle S3 represents the angle formed by the lines connecting the subject to the left and right optical systems. By bringing the imaging device closer to the subject, the convergence angle S3 becomes the same asFigure 1A The converging angle S1 therein is substantially the same. The three-dimensional effect perceived when viewing an image captured by bringing the imaging device closer to the subject on the display device becomes similar to the three-dimensional effect perceived when the user views the subject from the Figure 1A imaging position therein.

[0032] In view of the above, the imaging device (camera) according to the first embodiment notifies the user of an appropriate distance to the subject, enabling the user to capture an image that can be viewed in three dimensions. Based on the notification from the imaging device, the user can easily capture an image that can be viewed in three dimensions by adjusting the distance to the subject.

[0033] Figure 2A and Figure 2B are diagrams showing an external appearance example of a digital camera (camera) 100. The camera 100 shows an example of an electronic device according to the present embodiment. Figure 2A is a perspective view of the camera 100 when viewed from the front. Figure 2B is a perspective view of the camera 100 when viewed from the back.

[0034] The camera 100 has a shutter button 101, a power switch 102, a mode selection switch 103, a main electronic dial 104, a sub-electronic dial 105, a moving image button 106, and an external viewfinder display unit 107, all of which are provided on the upper surface. The shutter button 101 is an operation member for providing a photography preparation instruction or a photography instruction. The power switch 102 is an operation member for switching between the on and off states of the power supply of the camera 100. The mode selection switch 103 is an operation member for selecting various modes. The main electronic dial 104 is a rotary operation member for changing setting values such as shutter speed and aperture. The sub-electronic dial 105 is a rotary operation member for moving a selection box (cursor), image feed, etc. The moving image button 106 is an operation member for providing an instruction to start or stop photographing (recording) a moving image. The external viewfinder display unit 107 displays various setting values including shutter speed or aperture.

[0035] The camera 100 has a display unit 108, a touch panel 109, direction keys 110, a SET button 111, an AE lock button 112, a zoom button 113, a playback button 114, a menu button 115, an eyepiece unit 116, an eyepiece detection unit 118, and a touch bar 119, all of which are provided on the back surface.

[0036] The display unit 108 displays images and various information. The touch panel 109 is an operation member for detecting a touch operation on the display surface (touch operation surface) of the display unit 108. The direction keys 110 are operation members composed of keys (four-way keys) that can be pressed in both the vertical and horizontal directions. Processing can be performed based on the pressed position of the direction keys 110. The setting button 111 is an operation member that is mainly pressed to determine selected items. The AE lock button 112 is an operation member that is pressed in the shooting standby state to maintain the exposure state. The zoom button 113 is an operation member for switching between the on and off states of the zoom mode in the live view display (LV display) in the shooting mode. When the zoom mode is on, the live view image (LV image) is enlarged or reduced as the main electronic dial 104 is operated. In addition, the zoom button 113 is used to enlarge the reproduced image or increase the magnification in the playback mode. The playback button 114 is an operation member for switching between the shooting mode and the playback mode. When the playback button 114 is pressed during the shooting mode, the camera 100 transitions to the playback mode, and the display unit 108 displays the latest image among the images recorded on the recording medium 227.

[0037] The menu button 115 is an operation member that is pressed to display a menu screen on the display unit 108 where various settings can be made. The user can intuitively perform various settings on the menu screen displayed on the display unit 108 using the direction keys 110 and the setting button 111. The eyepiece unit 116 is an observation part through which the user observes the eyepiece viewfinder (optical viewfinder) 117. Through the eyepiece unit 116, the user can visually recognize the video displayed on the electronic viewfinder (EVF) 217 inside the camera 100. The eyepiece detection unit 118 is a sensor for detecting whether the user's eye is in contact with the eyepiece unit 116 (or the eyepiece viewfinder 117).

[0038] The touch bar 119 is a linear touch operation member (linear touch sensor) capable of receiving touch operations. The touch bar 119 is disposed at a touchable position where the user can perform a touch operation with the thumb of the right hand while holding the grip unit 120 with the right hand (i.e., holding the grip unit 120 with the little finger, ring finger, and middle finger of the right hand), so that the shutter button 101 can be pressed with the index finger of the right hand. In other words, when the user observes the eyepiece unit 116 through the eyepiece viewfinder 117 and holds up the camera 100 (i.e., the shooting posture) so as to be able to press the shutter button 101 at any time, the touch bar 119 is operable. The touch bar 119 can receive operations such as tapping on the touch bar 119 (after touching, releasing the thumb from the touch position within a specified period without moving) and sliding in the left-right direction on the touch bar 119 (moving the touch position while maintaining contact after touching). The touch bar 119 is an operation member different from the touch panel 109 and does not include a display function. The touch bar 119 serves as, for example, a multi-function bar (M-Fn bar) to which various functions can be assigned.

[0039] In addition, the camera 100 also has a grip unit 120, a thumb rest unit 121, a terminal cover 122, a cover 123, a communication terminal 124, etc. The grip unit 120 is a holding unit formed in a shape that is easily held by the right hand when the user holds up the camera 100. The shutter button 101 and the main electronic dial 104 are disposed at positions where the user can operate them with the index finger of the right hand while holding the camera 100 with the little finger, ring finger, and middle finger of the right hand. In addition, in the same state, the sub-electronic dial 105 and the touch bar 119 are disposed at positions where the user can operate them with the thumb of the right hand. The thumb rest unit 121 (thumb standby position) is a holding unit provided on the back side of the camera 100 where the user can easily place the thumb of the right hand that holds the grip unit 120 when not operating any operation member. The thumb rest unit 121 is composed of a rubber member or the like for increasing the holding force (grip feeling). The terminal cover 122 protects connectors such as connection cables that connect the camera 100 to an external device (external equipment). The cover 123 closes the slot for storing the recording medium 227 described later to protect the recording medium 227 and the slot. The communication terminal 124 is a terminal for communicating with a lens unit (such as the lens unit 200 and the lens unit 300, etc.) that can be attached to and detached from the camera 100.

[0040] Figure 3 is a block diagram showing a configuration example of the camera 100. Note that in Figure 3 in, the components identical to those of Figure 2A and Figure 2B are denoted by the same reference numerals, and descriptions thereof will be omitted as necessary. In Figure 3In this case, the lens unit 200 is attached to the camera 100.

[0041] First, the lens unit 200 will be described. The lens unit 200 is an interchangeable lens unit (replaceable lens) that can be attached to and detached from the camera 100. The lens unit 200 is a single-lens unit (monocular lens), and an example of a general lens unit is shown. The lens unit 200 includes an aperture 201, lenses 202, an aperture drive circuit 203, an autofocus (AF) drive circuit 204, a lens system control circuit 205, a communication terminal 206, and the like.

[0042] The aperture 201 is configured to be able to adjust the aperture diameter. The lenses 202 are composed of a plurality of lenses. The aperture drive circuit 203 adjusts the light amount by controlling the aperture diameter of the aperture 201. The AF drive circuit 204 drives the lenses 202 to obtain focus. The lens system control circuit 205 controls the aperture drive circuit 203, the AF drive circuit 204, etc. based on instructions from the system control unit 50. The lens system control circuit 205 controls the aperture 201 via the aperture drive circuit 203 and obtains focus by changing the position of the lenses 202 via the AF drive circuit 204. The lens system control circuit 205 can communicate with the camera 100. Specifically, the camera 100 and the lens unit 200 communicate via the communication terminal 206 of the lens unit 200 and the communication terminal 124 of the camera 100. The communication terminal 206 is a terminal used when the lens unit 200 communicates with the camera 100.

[0043] Next, the camera 100 will be described. The camera 100 includes a shutter 210, an imaging unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, an EVF 217, a display unit 108, and a system control unit 50.

[0044] The shutter 210 is a focal plane shutter that can freely control the exposure time of the imaging unit 211 based on an instruction from the system control unit 50. The imaging unit 211 is an imaging element (image sensor) composed of a CCD element, a CMOS element, etc. that convert an optical image into an electrical signal. The imaging unit 211 may have an imaging surface phase difference sensor that outputs defocus amount information to the system control unit 50. The A / D converter 212 converts the analog signal output from the imaging unit 211 into a digital signal. The image processing unit 214 performs prescribed processing (such as resizing processing such as pixel interpolation and reduction, color conversion processing, etc.) on the data from the A / D converter 212 or the data from the memory control unit 213. In addition, the image processing unit 214 performs prescribed calculation processing using the data from the captured image. The system control unit 50 performs exposure control or distance measurement control based on the obtained calculation results. Through this processing, AF processing, automatic exposure (AE) processing, electronic flash pre-flash (EF) processing, etc. are performed through the lens (TTL) system. In addition, the image processing unit 214 performs prescribed calculation processing using the data from the captured image. The system control unit 50 performs automatic white balance (AWB) processing of the TTL system based on the obtained calculation results.

[0045] The image data from the A / D converter 212 is written into the memory 215 via the image processing unit 214 and the memory control unit 213. Alternatively, the image data from the A / D converter 212 is written into the memory 215 via the memory control unit 213 without passing through the image processing unit 214. The memory 215 stores the image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212, or the image data to be displayed on the display unit 108 or the EVF 217. The memory 215 includes a storage capacity sufficient to store a prescribed number of still images or moving images and sounds for a prescribed period of time. In addition, the memory 215 also serves as a memory for image display (video memory).

[0046] The D / A converter 216 converts the image data for display stored in the memory 215 into an analog signal, and supplies the converted signal to the display unit 108 and the EVF 217. Therefore, the image data for display written into the memory 215 is displayed on the display unit 108 or the EVF 217 via the D / A converter 216. The display unit 108 and the EVF 217 perform display based on the analog signal from the D / A converter 216. The display unit 108 and the EVF 217 are displays such as an LCD and an organic EL display, for example. The digital data A / D-converted by the A / D converter 212 is accumulated in the memory 215 as a digital signal. The digital signal is converted into an analog signal by the D / A converter 216. After the conversion, the analog signal is sequentially transmitted to the display unit 108 and the EVF 217. The display unit 108 and the EVF 217 perform live view display by displaying the transmitted analog signal.

[0047] The system control unit 50 is a control unit including at least one processor and / or at least one circuit. In other words, the system control unit 50 can be a processor, a circuit, or a combination thereof. The system control unit 50 controls the entire camera 100. The system control unit 50 implements each process of the flowchart described later by executing a program recorded on the non-volatile memory 219. In addition, the system control unit 50 also performs display control by controlling the memory 215, the D / A converter 216, the display unit 108, the EVF 217, and the like.

[0048] In addition, the camera 100 has a system memory 218, a non-volatile memory 219, a system timer 220, a communication unit 221, a posture detection unit 222, and an eyepiece detection unit 118.

[0049] For example, a random access memory (RAM) is used as the system memory 218. Constants, variables for the operation system control unit 50, programs read from the non-volatile memory 219, and the like are expanded into the system memory 218. The non-volatile memory 219 is an electrically erasable / recordable memory. For example, an electrically erasable programmable read-only memory (EEPROM) (registered trademark) is used as the non-volatile memory 219. On the non-volatile memory 219, constants, programs, etc. for the operation system control unit 50 are recorded. Here, the program refers to a program for executing the flowchart described later. The system timer 220 is a timing unit that measures the time used in various control processes or the time of an embedded clock.

[0050] The communication unit 221 transmits and receives video signals or audio signals to and from an external device connected via a wireless or wired cable. The communication unit 221 is also capable of connecting to a wireless local area network (LAN) or the Internet. In addition, the communication unit 221 can communicate with an external device via Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 221 can transmit an image (including a live view image) captured by the imaging unit 211 or an image recorded on the recording medium 227. The communication unit 221 can receive an image or various other information from an external device.

[0051] The posture detection unit 222 is a posture detection sensor that detects the posture of the camera 100 relative to the direction of gravity. Based on the posture detected by the posture detection unit 222, it can be determined whether the image captured by the imaging unit 211 is captured with the camera 100 held in a horizontal orientation or a vertical orientation. The system control unit 50 can add orientation information corresponding to the posture detected by the posture detection unit 222 to the image file of the image captured by the imaging unit 211. In addition, the system control unit 50 can also rotate and record the image based on the posture detected by the posture detection unit 222. As the posture detection unit 222, for example, an acceleration sensor, a gyro sensor, etc. can be used. The movement (panning, tilting, lifting, stationary, etc.) of the camera 100 can also be detected using the posture detection unit 222.

[0052] The eyepiece detection unit 118 can detect the approach of an object to the eyepiece unit 116 (eyepiece viewfinder 117). As the eyepiece detection unit 118, for example, an infrared proximity sensor can be used. When an object approaches, the infrared light projected from the projection unit of the eyepiece detection unit 118 is reflected by the object and received by the light receiving unit of the infrared proximity sensor. Based on the amount of the received infrared light, the eyepiece detection unit 118 can determine the distance from the eyepiece unit 116 to the object. As described above, the eyepiece detection unit 118 performs eyepiece detection to detect the proximity distance of the object to the eyepiece unit 116. The eyepiece detection unit 118 is an eyepiece detection sensor that detects the approach (contact) and retreat (separation) of the eye (object) relative to the eyepiece unit 116. When an eye approaching within a specified distance from a non-contact state (non-approaching state) to the eyepiece unit 116 is detected, the eyepiece detection unit 118 detects the contact of the eye. On the other hand, when an eye that has approached the eyepiece unit 116 separates by at least a specified distance from the contact state (approaching state), the eyepiece detection unit 118 detects the separation of the eye. The threshold for detecting contact and the threshold for detecting separation can be different from each other, for example, by setting hysteresis, etc.

[0053] In addition, after contact is detected, it is assumed that the contact state is maintained until separation is detected. After separation is detected, it is assumed that the non-contact state is maintained until contact is detected. The system control unit 50 switches between the display (display state) and non-display (non-display state) of the display unit 108 and the EVF 217 based on the state detected by the eyepiece detection unit 118. Specifically, when the camera 100 is at least in the shooting standby state and its display destination setting is set to automatic switching, when not in contact with the eye, the system control unit 50 causes the display unit 108 to enter the display state as the display destination, and causes the EVF 217 to enter the hidden state. In addition, when in contact with the eye, the system control unit 50 causes the EVF 217 to enter the display state as the display destination, and causes the display unit 108 to enter the hidden state. Note that the eyepiece detection unit 118 is not limited to an infrared proximity sensor. Any sensor that can detect a state considered to be contact can be used as the eyepiece detection unit 118.

[0054] In addition, the camera 100 includes an external viewfinder display unit 107, an external viewfinder display unit driving circuit 223 for the display unit located outside the viewfinder, a power control unit 224, a power supply unit 225, a recording medium I / F 226, an operation unit 228, and the like.

[0055] The external viewfinder display unit 107 is driven by the external viewfinder display unit driving circuit 223 for the display unit located outside the viewfinder, and displays various setting values of the camera 100 such as the shutter speed and aperture.

[0056] The power control unit 224 is composed of a battery detection circuit, a DC-DC converter, a switching circuit for switching the blocks to be powered on, and the like. The power control unit 224 detects the presence or absence of a battery, the type of battery, the remaining battery power, and the like. In addition, the power control unit 224 controls the DC-DC converter based on the detection result and an instruction from the system control unit 50, and supplies voltage to each unit including the recording medium 227 during a required period.

[0057] The power supply unit 225 includes primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, and Li batteries, an AC adapter, and the like.

[0058] The recording medium I / F 226 is an interface with a recording medium 227 such as a memory card and a hard disk. The recording medium 227 is, for example, a memory card for recording captured images. The recording medium 227 includes a semiconductor memory, a magnetic disk, and the like. The recording medium 227 can be attached to and detached from the camera 100. The recording medium 227 can be embedded in the camera 100.

[0059] The operation unit 228 is an input unit (reception unit) capable of receiving operations (user operations) from a user. The operation unit 228 is used to input various instructions to the system control unit 50. The operation unit 228 includes a shutter button 101, a power switch 102, a mode selection switch 103, a touch panel 109, other operation units 229, etc. The operation unit 229 includes a main electronic dial 104, a sub-electronic dial 105, a moving image button 106, arrow keys 110, a setting button 111, an AE lock button 112, a zoom button 113, a playback button 114, a menu button 115, a touch bar 119, etc.

[0060] The shutter button 101 has a first shutter switch 230 and a second shutter switch 231. The first shutter switch 230 is turned on during the operation of the shutter button 101, i.e., when it is half-pressed (photographing preparation instruction), and outputs a first shutter switch signal SW1. The system control unit 50 starts photographing preparation processes such as AF processing, AE processing, AWB processing, and EF processing based on the first shutter switch signal SW1. When the operation of the shutter button 101 is completed, i.e., when it is fully pressed (photographing instruction), the second shutter switch 231 is turned on and outputs a second shutter switch signal SW2. The system control unit 50 starts a series of photographing processes from reading the signal from the imaging unit 211 to writing the image file including the captured image to the recording medium 227 after generating the image file based on the second shutter switch signal SW2.

[0061] The mode selection switch 103 switches the operation mode of the system control unit 50 to any one of a still image photographing mode, a moving image photographing mode, a playback mode, etc. The still image photographing mode includes an automatic photographing mode, an automatic scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode (P mode). In addition, the still image photographing mode also includes various scene modes, custom modes, etc. for setting photographing parameters for each photographing scene. Through the mode selection switch 103, the user can directly switch the operation mode to any of the above photographing modes. Alternatively, after temporarily switching to a list screen of photographing modes through the mode selection switch 103, the user can use the operation unit 228 to select any one of multiple display modes. Similarly, the moving image photographing mode can include multiple modes.

[0062] The touch panel 109 is a touch sensor that detects various touch operations on the display surface of the display unit 108 (the operation surface of the touch panel 109). The touch panel 109 and the display unit 108 can be integrally configured. For example, the touch panel 109 is attached to the upper layer of the display surface of the display unit 108 such that the light transmittance does not interfere with the display of the display unit 108. In addition, by associating the input coordinates in the touch panel 109 with the display coordinates on the display surface of the display unit 108, a graphical user interface (GUI) can be configured that makes the user feel as if he / she can directly operate the screen displayed on the display unit 108. The touch panel 109 can be any one of various types such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type. There are types that detect a touch operation when the touch panel 109 is touched and types that detect a touch operation when a finger or a pen approaches the touch panel 109. However, any of these types can be used.

[0063] The system control unit 50 is capable of detecting the following operations or states of the touch panel 109.

[0064] · The state in which a finger or a pen that has not touched the touch panel 109 newly touches the touch panel 109, that is, the start of a touch (hereinafter referred to as a touch-on).

[0065] · The state in which the touch panel 109 is touched by a finger or a pen (hereinafter referred to as a touch-continue).

[0066] · The state in which a finger or a pen moves while touching the touch panel 109 (hereinafter referred to as a touch-move).

[0067] · The state in which a finger or a pen that has touched the touch panel 109 separates (releases) from the touch panel 109, that is, the end of a touch (hereinafter referred to as a touch-stop).

[0068] · The state in which the touch panel 109 is not touched (hereinafter referred to as a non-touch).

[0069] When a touch-on is detected, a touch-continue is detected simultaneously. Generally, unless a touch-stop is detected after a touch-on, a touch-continue is continuously detected. When a touch-move is detected, a touch-continue is continuously detected. Even if a touch-continue has been detected, a touch-move is not detected unless the touch position has moved. After the touch-stop of all the fingers or pens that have touched is detected, a non-touch is detected.

[0070] Via the internal bus, the system control unit 50 is notified of these operations and states or the position coordinates of a finger or a pen that has touched the touch panel 109. Based on the notified information, the system control unit 50 determines what operation (touch operation) has been performed on the touch panel 109. For a touch movement, the system control unit 50 can also determine, based on the change in the position coordinates, the moving direction of the finger or pen moving on the touch panel 109 for each of the vertical component and the horizontal component on the touch panel 109. When it is detected that the touch movement has been made by at least a prescribed distance, the system control unit 50 determines that a swipe operation has been performed. An operation of quickly moving a finger a certain distance while touching the touch panel 109 and then releasing the finger will be referred to as a "flick". In other words, a "flick" is an operation of quickly swiping across the touch panel 109 in the manner of flicking a finger. The system control unit 50 can determine that a flick has been performed (a flick has been performed after a swipe operation) when it is detected that the touch movement has been made by at least a prescribed distance and at least at a prescribed speed and then a touch stop has been made after the touch movement. In addition, a touch operation of simultaneously touching multiple places (e.g., two points) (multi-touch) and bringing the touch positions closer to each other will be referred to as a "pinch", and a touch operation of moving the touch positions away from each other will be referred to as a "spread". "Spread" and "pinch" will be collectively referred to as a pinch-separate operation (or simply "pinch-separate").

[0071] Figure 4 is a schematic diagram showing a configuration example of the lens unit 300. Figure 4 shows a state in which the lens unit 300 is attached to the camera 100. Note that, in Figure 4 the camera 100 shown, components identical to those described in Figure 3 are denoted by the same reference numerals, and descriptions thereof will be omitted where necessary. For components related to the right eye, the letter R is added to the end of each reference numeral. For components related to the left eye, the letter L is added to the end of each reference numeral. For components related to both the right eye and the left eye, neither R nor L is added to the end of each reference numeral.

[0072] The lens unit 300 is a replaceable lens unit that can be attached to and detached from the camera 100. The lens unit 300 is a binocular lens unit that enables the capture of a right image and a left image having parallax with respect to each other. The lens unit 300 has two optical systems and can capture images with a wide viewing angle of approximately 180 degrees using each of the two optical systems. Specifically, the lens unit 300 can capture an image of a subject with a field of view (viewing angle) of 180 degrees in the left-right direction (horizontal angle, azimuth angle, and yaw angle) and 180 degrees in the up-down direction (vertical angle, elevation angle, and pitch angle) using each of the two optical systems. In other words, the lens unit 300 can capture images within the range of the front hemisphere using each of the two optical systems.

[0073] The lens unit 300 includes an optical system 301R including a plurality of lenses, mirrors, etc.; an optical system 301L including a plurality of lenses, mirrors, etc.; and a lens system control circuit 303. The optical system 301R has a lens 302R disposed on the subject side, and the optical system 301L has a lens 302L disposed on the subject side. The lenses 302R and 302L face the same direction, and their optical axes are substantially parallel to each other.

[0074] The lens unit 300 is a binocular lens unit (VR180 lens unit) for capturing images in the VR180 format, which is one of the virtual reality (VR) image formats capable of binocular three-dimensional viewing. The lens unit 300 has a fisheye lens in each of the optical systems 301R and 301L that can each capture a field of view of approximately 180 degrees. Note that the field of view that the lenses in each of the optical systems 301R and 301L can capture may be narrower than 180 degrees, such as approximately 120 degrees or 160 degrees, etc.

[0075] The lens unit 300 can form a right image formed via the optical system 301R and a left image formed via the optical system 301L on one or two imaging elements of the camera to which the lens unit 300 is attached. In the camera 100, the right image and the left image are formed on one imaging element (imaging sensor), generating an image (binocular image) in which a right image region (region of the right image) and a left image region (region of the left image) are arranged side by side.

[0076] The lens unit 300 is attached to the camera 100 via a lens mounting unit 304 and a camera mounting unit 305 of the camera 100. The system control unit 50 of the camera 100 and the lens system control circuit 303 of the lens unit 300 are electrically connected to each other via a communication terminal 124 of the camera 100 and a communication terminal 306 of the lens unit 300.

[0077] InFigure 4 In the configuration, the right image formed via the optical system 301R and the left image formed via the optical system 301L are formed side by side on the imaging unit 211 of the camera 100. In other words, two optical images (subject images) are formed on two regions of one imaging element (imaging sensor) by the optical systems 301R and 301L. The imaging unit 211 converts the formed optical images (optical signals) into analog electrical signals. By using the lens unit 300 as described above, it is possible to capture one image including two image regions having parallax with respect to each other from two locations of the optical systems 301R and 301L. When the captured image is divided into a left-eye image and a right-eye image for VR display, the user can view a three-dimensional VR image with a field of view of approximately 180 degrees. In other words, the user can view an image in the VR180 format three-dimensionally.

[0078] The lens unit 300 includes a focus ring (not shown) for adjusting focus. The lens unit 300 includes, for example, two focus rings, that is, a focus ring for adjusting the focus of the right image formed via the optical system 301R and a focus ring for adjusting the focus of the left image formed via the optical system 301L. The lens unit 300 may also include a focus ring for simultaneously adjusting the focus of both the right image formed via the optical system 301R and the left image formed via the optical system 301L, and a focus ring for adjusting the focus of one of the right image and the left image.

[0079] Here, the VR image refers to an image that can be displayed in VR as described below. The VR image includes an omnidirectional image (celestial sphere image) captured by an omnidirectional camera (celestial sphere camera), a panoramic image whose video range (effective video range) is wider than the display range that can be displayed on the display unit at one time, and the like. In addition, the VR image is not limited to a still image, and also includes a moving image and a live image (an image captured from the camera almost in real time). The VR image has a video range (effective video range) with a field of view of 360 degrees in the left-right direction and 360 degrees in the up-down direction at maximum. In addition, the VR image also includes an image captured at a wider angle of view than that of a normal camera even if the horizontal and vertical coverage of the viewing angle is less than 360 degrees. Alternatively, the VR image also includes an image whose video range exceeds the display range that can be displayed on the display unit at one time. The image captured by the camera 100 using the above lens unit 300 is a type of VR image. This VR image can be displayed in VR, for example, by setting the display mode of a display device (a display device capable of displaying VR images) to "VR view". By displaying a part of the VR image with a 360-degree viewing angle and changing the posture of the display device in the left-right direction (horizontal rotation direction), the user can move the display range and view a seamless omnidirectional video in the left-right direction.

[0080] VR display (VR view) refers to a display method (display mode) that displays the video within the field of view corresponding to the posture of the display device as a VR image and enables the display range to be changed. VR display includes "monocular VR display (monocular VR view)", in which the VR image is transformed (distortion corrected) to be mapped onto a virtual sphere to display a single image. In addition, VR display includes "binocular VR display (binocular VR view)", in which the VR image for the left eye and the VR image for the right eye are transformed to be mapped onto a virtual sphere and displayed side by side in the left and right regions. By using the VR image for the left eye and the VR image for the right eye that have parallax with respect to each other for "binocular VR display", the VR image can be viewed in three dimensions. For example, when a user wears a display device such as a head-mounted display (HMD), the video within the field of view corresponding to the orientation of the user's face is displayed in both monocular VR display and binocular VR display. For example, assume that at a certain point in time, the video within the field of view centered at 0 degrees in the left-right direction (in a specific direction, such as north) and 90 degrees in the up-down direction (90 degrees relative to the zenith, that is, the horizontal line) is displayed as a VR image. When the posture of the display device is flipped inside out (for example, when the orientation of the display surface changes from south to north), the display range is adjusted so that the video within the field of view centered at 180 degrees in the left-right direction (in the opposite direction, such as south) and 90 degrees in the up-down direction is displayed as the same VR image. In other words, when the user turns the face from north to south while wearing the HMD (that is, when the user looks backward), the video displayed on the HMD also changes from the video of the north to the video of the south. Note that the VR image captured by the lens unit 300 is an image obtained by capturing a range of approximately 180 degrees in the forward direction (180° image) and does not include the video within the range of approximately 180 degrees in the backward direction. When such an image is displayed in VR and the posture of the display device changes to the side where there is no video, a blank area is displayed.

[0081] As described above, when a VR image is displayed in VR, the user can obtain a feeling (immersive feeling) as if he / she is visually present in the VR image (VR space). Note that the method for displaying the VR image is not limited to changing the posture of the display device. For example, the display range can be moved (scrolled) according to the operation of the user via a touch panel or a direction button, etc. In addition, during VR display (display mode "VR view"), in addition to changing the display range based on the change in posture, the display range can also be moved according to the touch movement on the touch panel, the drag operation using a mouse, etc., or the pressing of the direction button. Note that a smartphone attached to a VR goggle (head-mounted adapter) is a type of HMD.

[0082] Reference Figure 5, the guidance display process according to the first embodiment will be described. The camera 100 displays guidance to notify the user whether the distance to the subject is consistent with the shooting distance at which three-dimensional viewing is possible on a display device such as an HMD. This guidance shows the relationship between the recommended distance and the distance from the camera 100 to the subject (hereinafter referred to as the subject distance). This guidance can be represented by graphics or text information, or it can be expressed by a display mode such as image tone. The recommended distance represents the shooting distance at which a captured image including two image regions having parallax with respect to each other can be three-dimensionally viewed. By displaying guidance showing the relationship between the recommended distance and the subject distance, the camera 100 assists the user in photographing the subject so that three-dimensional viewing is possible.

[0083] In step S501, the system control unit 50 determines whether the display setting for guiding the user to notify whether the captured image can be three-dimensionally viewed on a display device such as an HMD is turned on. The user can display a screen for the display setting of the guidance from the menu screen and turn on or off the guidance display. When the display setting of the guidance is turned on, the system control unit 50 displays guidance showing whether the captured image can be three-dimensionally viewed when the user takes a photo.

[0084] Figure 6 An example of a setting screen showing the display setting of the guidance is shown. By checking the checkbox 601 of "Guidance Display", the user can turn on the display setting of the guidance. On the contrary, by unchecking the checkbox 601, the user can turn off the display setting of the guidance. In step S501, the system control unit 50 can determine whether the display setting of the guidance is turned on based on the value set on the Figure 6 setting screen shown. When the display setting of the guidance is turned on, the process proceeds to step S502. When the display setting of the guidance is turned off, Figure 5 the process shown ends.

[0085] In step S502, the system control unit 50 acquires information related to the recommended distance of the lens unit 300 (hereinafter referred to as recommended distance information). The recommended distance represents the shooting distance at which a captured image including two image regions having parallax with respect to each other can be three-dimensionally viewed. In other words, by adjusting the distance to the subject to be consistent with the recommended distance, the user can perform shooting so that the captured image including the subject can be three-dimensionally viewed.

[0086] The recommended distance varies according to the baseline length representing the distance between the left and right lenses (optical systems) of the lens unit 300 that respectively capture two image regions. The distance to the subject for capturing an image such that the subject can be viewed in three dimensions is proportional to the baseline length. Therefore, recommended distance information can be obtained based on the baseline length. In addition, the proportional relationship can be calculated based on a model of the interpupillary distance and a model of the distance to the subject (the observed subject) at which three-dimensional viewing can be performed with the naked eye, where the model of the interpupillary distance represents the assumed distance between the left and right eyes of a person. The model of the interpupillary distance is a reference value of the interpupillary distance and will be referred to as the reference interpupillary distance hereinafter. The model of the distance to the subject at which three-dimensional viewing can be performed with the naked eye is a reference value of the distance to the subject at which three-dimensional viewing can be performed and will be referred to as the reference three-dimensional viewing distance hereinafter.

[0087] Specifically, the recommended distance can be calculated according to the following equations 1 and 2. The proportionality constant K represents the proportional relationship between the interpupillary distance and the three-dimensional viewing distance.

[0088] K = reference three-dimensional viewing distance / reference interpupillary distance... Equation 1

[0089] Recommended distance = baseline length × K... Equation 2

[0090] For example, the reference interpupillary distance is assumed to be 6 cm, which is a typical interpupillary distance. The reference three-dimensional viewing distance is assumed to be 5 m (500 cm), which is the maximum distance at which three-dimensional viewing is assumed to be possible with the naked eye. In addition, the baseline length of the lens unit 300 is assumed to be 3 cm, for example. In this case, the proportionality constant K = 500 / 6. The maximum recommended distance at which three-dimensional viewing is possible is 3 cm × (500 / 6) = 2,500 cm (2.5 m).

[0091] If the proportionality constant K is set, the recommended distance information can be obtained in advance based on the baseline length of the lens unit 300. The camera 100 can pre-retain the obtained recommended distance information.

[0092] In addition, the obtained recommended distance information can be retained in the recording area inside the lens unit 300. The lens system control circuit 303 transmits the recommended distance information retained in the recording area to the camera 100 via the communication terminal 306 and the communication terminal 124 of the camera 100. As described above, the system control unit 50 can obtain the recommended distance information from the lens unit 300, which is attached to the camera 100 and has two optical systems that respectively capture two image regions.

[0093] Note that the recommended distance information is the maximum shooting distance at which the captured image can be viewed in 3D as described above, but it may also include the minimum shooting distance at which the captured image can be viewed in 3D. In other words, the recommended distance information may also include the range of shooting distances at which the captured image with two image regions can be viewed in 3D when displayed on the display device.

[0094] In addition, the lens unit 300 may transmit information related to the baseline length to the camera 100 instead of information related to the recommended distance. Using the information related to the baseline length obtained from the lens unit 300, the camera 100 can calculate and obtain the recommended distance according to the above formulas 1 and 2.

[0095] In step S503, the system control unit 50 obtains information related to the distance from the camera 100 to the subject (subject distance). The system control unit 50 can obtain the subject distance based on the focal length of the camera 100. For example, the lens system control circuit 303 of the lens unit 300 can obtain the subject distance based on the position of the focus adjustment ring for adjusting focus. The lens system control circuit 303 transmits the obtained subject distance via the communication terminal 306 and the communication terminal 124 of the camera 100. As a result, the system control unit 50 can obtain the subject distance from the lens unit 300.

[0096] Note that the method of obtaining the subject distance is not limited to obtaining the subject distance based on the position of the focus adjustment ring. For example, the system control unit 50 of the camera 100 can obtain information related to the subject distance based on the parallax between two image regions captured using the lens unit 300. In addition, the system control unit 50 can also obtain information related to the subject distance based on the phase difference information output from the image plane phase difference sensor provided in the camera 100. In addition, the system control unit 50 can also obtain information related to the subject distance based on the output from the time-of-flight (TOF) sensor provided in the camera 100.

[0097] In step S504, the system control unit 50 determines whether the adjustment setting for the recommended distance is enabled. The user can display a screen for adjusting the recommended distance from the menu screen and enable or disable the adjustment of the recommended distance. When the adjustment setting for the recommended distance is enabled, the system control unit 50 adjusts the recommended distance based on the user operation.

[0098] Figure 6 An example of a setting screen for the adjustment setting of the recommended distance is shown. By checking the checkbox 602 for "Recommended Distance Adjustment", the user can enable the adjustment setting for the recommended distance. Conversely, by unchecking the checkbox 602, the user can disable the adjustment setting for the recommended distance.

[0099] In step S504, the system control unit 50 may determine whether the adjustment setting of the recommended distance is enabled based on the value set on the setting screen shown in Figure 6 When the adjustment setting for the recommended distance is enabled, the process proceeds to step S505. When the adjustment setting for the recommended distance is disabled, the process proceeds to step S507.

[0100] In step S505, the system control unit 50 obtains an adjustment value for adjusting the recommended distance. The adjustment of the recommended distance refers to the process of adjusting the recommended distance obtained in step S502. The recommended distance is a value preset based on a hypothetical model. Therefore, depending on the user, the three-dimensional effect perceived when viewing on a display device such as an HMD may be different from the three-dimensional effect assumed during photography. In view of this, the camera 100 may have a unit that enables the user to adjust the recommended distance.

[0101] Figure 6 An example of a setting screen for setting the adjustment value used to adjust the recommended distance is shown. By moving the handle 604 to the + side using the slider 603 to change the adjustment value, the user can provide an instruction to increase the recommended distance. In this case, the system control unit 50 adjusts the recommended distance by multiplying the recommended distance by a coefficient α1 (α1>1) corresponding to the scale of the slider 603 used as the adjustment value. Conversely, by moving the handle 604 to the – side, the user can provide an instruction to decrease the recommended distance. In this case, the system control unit 50 adjusts the recommended distance by multiplying the recommended distance by a coefficient α2 (0<α2<1) corresponding to the scale of the slider 603 used as the adjustment value. Note that the adjustment value of the recommended distance can be set while the user is viewing the captured image displayed on a display device such as an HMD.

[0102] In addition, the adjustment value can be automatically set based on the distance between the pupils obtained from the user's face image. Figure 7 is a flowchart showing the process of setting the adjustment value of the recommended distance. In step S701, the system control unit 50 obtains the user's face image. For example, the user's face image is an image captured using a monocular lens provided in the camera 100 instead of a lens unit 300 having two optical systems. In addition, the user's face image can also be an image pre-captured using a monocular lens unit attached to the camera 100.

[0103] In step S702, the system control unit 50 detects the pupils from the user's face image obtained in step S701 and obtains the center coordinates of each detected pupil. The system control unit 50 can use a known detection method to detect the pupils. In step S702, it is sufficient to obtain the center coordinates of each pupil detected from the user's face image. The system control unit 50 can use any detection method.

[0104] In step S703, the system control unit 50 determines whether the number of pupils detected in step S702 is two. When two pupils are detected, the process proceeds to step S704. When fewer than two pupils or at least three pupils are detected, the facial image is regarded as invalid. Therefore, the system control unit 50 returns to step S701 to obtain another facial image.

[0105] In step S704, the system control unit 50 obtains the inter-pupil distance based on the information related to the coordinates of the two pupils detected in step S702. The system control unit 50 can use the following equations 3 and 4 to calculate the inter-pupil distance.

[0106] The inter-pupil distance on the image plane = the distance between the coordinates of the two pupils × the pixel size... Equation 3

[0107] The inter-pupil distance = the inter-pupil distance on the image plane × the subject distance / the focal length... Equation 4

[0108] The pixel size in Equation 3 represents the value corresponding to the size of each pixel of the imaging element. For example, the subject distance in Equation 4 can be calculated based on the position of the focus ring of the lens during the capture of the facial image. The focal length represents the focal length of the lens during the capture of the facial image. When the distortion is significant due to lens distortion aberration, etc., the system control unit 50 can improve the accuracy of the calculation result by calculating the inter-pupil distance while considering the distortion.

[0109] In step S705, the system control unit 50 obtains an adjustment value. The system control unit 50 can obtain the adjustment value α3 using the following Equation 5 based on the inter-pupil distance obtained in step S704 and the reference inter-pupil distance.

[0110] α3 = the reference inter-pupil distance / the inter-pupil distance obtained in step S704... Equation 5

[0111] The system control unit 50 stores the obtained adjustment value α3 in the memory 215 or the like.

[0112] As described above, in Figure 5 step S505, the system control unit 50 can obtain the adjustment value (for example, the coefficients α1 and α2) set by the user or the automatically set adjustment value α3.

[0113] In step S506, the system control unit 50 uses the adjustment value obtained in step S505 to adjust the recommended distance. For example, the system control unit 50 uses the value based on the Figure 6The recommended distance is adjusted using the adjustment value (coefficient α1 or α2) obtained from the user instruction on the setting screen shown. Specifically, the system control unit 50 can adjust the recommended distance by multiplying the recommended distance by the coefficient α1 or α2.

[0114] In addition, the system control unit 50 can use the adjustment value obtained based on the inter-pupillary distance of the user to adjust the recommended distance. Specifically, the system control unit 50 can adjust the recommended distance by multiplying the recommended distance by the adjustment value α3 obtained using Equation 5. In other words, when the inter-pupillary distance obtained in Figure 7 step S704 is shorter than the assumed reference inter-pupillary distance, the system control unit 50 can adjust the recommended distance to be longer based on the adjustment value α3. Conversely, when the inter-pupillary distance obtained in Figure 7 step S704 is longer than the assumed reference inter-pupillary distance, the system control unit 50 can adjust the recommended distance to be shorter based on the adjustment value α3.

[0115] Reference Figure 1A and Figure 1C will describe the recommended distance for both the case where the user's inter-pupillary distance is longer than the reference inter-pupillary distance and the case where the user's inter-pupillary distance is shorter than the reference inter-pupillary distance. Figure 1A The distance between the left and right eyes shown is assumed to be the reference inter-pupillary distance. The convergence angle S1 is assumed to be the minimum angle at which the subject can be viewed in three dimensions. Figure 1C The convergence angle S3 in Figure 1C is approximately the same as the convergence angle S1, and the recommended distance is the

[0116] distance from the camera to the subject in Figure 7 . Figure 1A When the inter-pupillary distance of the user obtained in Figure 1C step S704 is shorter than the reference inter-pupillary distance in Figure 7 , the minimum angle at which the subject can be viewed in three dimensions becomes smaller than the convergence angle S1. Since Figure 1A the convergence angle S3 in Figure 1C becomes smaller, the recommended distance from the camera 100 to the subject increases. On the other hand, when the inter-pupillary distance of the user obtained in

[0117] In step S507, the system control unit 50 compares the subject distance with the recommended distance obtained in step S502. When it is determined in step S504 that the adjustment setting for the recommended distance is turned on and the recommended distance is adjusted in step S506, the system control unit 50 compares the adjusted recommended distance with the subject distance.

[0118] In step S508, based on the relationship between the subject distance and the recommended distance compared in step S507, the system control unit 50 displays a guidance on the display unit 108. Refer to Figures 8A to 8C , Figures 9A to 9D and Figures 10A to 10D , and the display examples of the guidance will be described.

[0119] Figures 8A to 8C is a diagram showing a display example of a guidance that uses an image tone to show the relationship between the recommended distance and the subject distance. The display unit 108 performs live view display in real time, showing a captured image including two image areas captured by the two optical systems 301R and 301L of the lens unit 300. The display unit 108 controls the coloring of the two image areas based on the relationship between the recommended distance and the subject distance.

[0120] Figure 8A shows a state where the subject distance is substantially the same as the recommended distance. When the subject distance is substantially the same as the recommended distance, the display unit 108 does not perform any special display. Additionally, when the recommended distance information shows a range from the shortest value to the longest value of the recommended distance and the subject distance falls within this range, the display unit 108 also Figure 8A shows no special display.

[0121] Figure 8B shows a state where the subject distance is longer than the recommended distance. When the subject distance is longer than the recommended distance, the display unit 108 displays, for example, the tone of the live view image representing the image in the live view display in warm colors. Figure 8B The slashes in the live view image in Figure 8B show that the image is colored in warm colors. Additionally, when the recommended distance information shows a range from the shortest value to the longest value of the recommended distance and the subject distance is longer than the longest value of this range, the display unit 108 also

[0122] Figure 8C shows a state where the subject distance is shorter than the recommended distance. When the subject distance is shorter than the recommended distance, the display unit 108 displays, for example, the tone of the live view image in cool colors. Figure 8CThe shadow of the live view image in shows that the image is colored in cool colors. Additionally, when the recommended distance information shows a range from the shortest value to the longest value and the subject distance is shorter than the shortest value of this range, the display unit 108 displays the hue of the live view image in cool colors as shown in Figure 8C

[0123] The display unit 108 changes the hue of the displayed image based on whether the subject distance is longer or shorter than the recommended distance, thereby enabling the user who acts as the photographer to be notified whether the subject distance is consistent with the recommended distance. The colors indicating whether the subject distance is consistent with the recommended distance are not limited to examples of warm colors and cool colors, but may include other colors as long as the user can recognize the relationship between the subject distance and the recommended distance.

[0124] In the example of , the user can recognize the relationship between the subject distance and the recommended distance through the hue difference of the live view image. By moving closer to or farther from the subject according to the relationship between the subject distance and the recommended distance, the user can adjust the photography position so that photography can be performed at the recommended distance where the subject is displayed as three-dimensionally viewable. Figures 8A to 8C

[0125] Figures 9A to 9D FIG. is a diagram illustrating a display example of a guide showing the positional relationship between the recommended distance, the subject, and the camera 100. The display unit 108 performs live view display of two image areas and controls the display of a graphic (item) showing the positional relationship between the recommended distance, the subject, and the camera 100 as a guide showing the relationship between the subject distance and the recommended distance.

[0126] In the example of , the guide showing the relationship between the subject distance and the recommended distance is displayed in the display area 900 at the lower part of the live view image, for example. Referring to Figures 9B to 9D Figure 9A , specific examples of the guide showing the relationship between the subject distance and the recommended distance will be described. In , the recommended distance is shown by the range from the shortest value to the longest value of the recommended distance, but it may also be shown by the position of the longest value of the recommended distance. Figures 9B to 9D

[0127] Figure 9B FIG. shows a guide example when the subject distance is longer than the recommended distance. The position 901 of the camera 100 is displayed at a position farther from the subject 903 than the range 902 of the recommended distance. The subject distance refers to the distance from the position 901 of the camera 100 to the subject 903. The recommended distance refers to the distance from the position 901 of the camera 100 to any position within the range 902 of the recommended distance. According to the guide shown in Figure 9B the user can recognize that the subject distance is longer than the recommended distance.​​​​​​​

[0128] Figure 9C Shows a guidance example when the subject distance is substantially the same as the recommended distance or when the subject distance falls within the range 902 of the recommended distance. The position 901 of the camera 100 is shown within the range 902 of the recommended distance. According to Figure 9C the guidance shown, the user can recognize that the subject distance falls within the range 902 of the recommended distance.

[0129] Figure 9D Shows a guidance example when the subject distance is shorter than the recommended distance. The position 901 of the camera 100 is shown at a position closer to the subject 903 than the range 902 of the recommended distance. According to Figure 9D the guidance shown, the user can recognize that the subject distance is shorter than the recommended distance.

[0130] Figures 10A to 10D is a diagram showing a display example of guidance that shows the relationship between the recommended distance and the subject distance through text (text information). The display unit 108 performs live view display of two image areas and controls the display of text showing the relationship between the subject distance and the recommended distance. In Figure 10A the example, the text showing the relationship between the subject distance and the recommended distance is displayed in the display area 1000, for example, at the lower part of the live view image. Referring to Figures 10B to 10D , specific examples of the text showing the relationship between the subject distance and the recommended distance will be described.

[0131] Figure 10B Shows an example of the text displayed when the subject distance is longer than the recommended distance. The text shows that the position of the camera 100 is farther from the subject than the position of the recommended distance or the range of the recommended distance. Figure 10C Shows an example of the text displayed when the subject distance is substantially the same as the recommended distance or when the subject distance falls within the range of the recommended distance. The text shows that the position of the camera 100 is substantially the same as the position of the recommended distance or falls within the range of the recommended distance. Figure 10D Shows an example of the text displayed when the subject distance is shorter than the recommended distance. The text shows that the position of the camera 100 is closer to the subject than the position of the recommended distance or the range of the recommended distance. By displaying the text described in Figures 10B to 10D , the user can recognize the relationship between the recommended distance and the subject distance.

[0132] The display examples of the guidance are not limited to those described in Figures 8A to 8C , Figures 9A to 9D and Figures 10A to 10D . However, other methods can be adopted as long as the user acting as the photographer can recognize the relationship between the subject distance and the recommended distance.

[0133] According to the first embodiment described above, the camera 100 performs guidance display to notify the user of the relationship between the recommended distance and the subject distance. As a result, the user can easily capture an image that can be three-dimensionally viewed on the display device.

[0134] Second Embodiment

[0135] The first embodiment describes an example of notifying the user of the relationship between the distance from the camera 100 to a specific subject and the recommended distance. In the second embodiment, a distance map is generated for each of two image regions, and the relationship between the recommended distance and the subject distance is notified to the user for each region (block) within the image region. The image region can be divided into blocks based on the relationship between the recommended distance and the subject distance.

[0136] Reference Figure 11 , the guidance display process according to the second embodiment will be described. For the same processes as the guidance display process according to the first embodiment in Figure 5 , the detailed description will be omitted.

[0137] In step S1101, the system control unit 50 determines whether the display setting for guidance is turned on, which notifies the user whether the captured image can be displayed on a display device such as an HMD for three-dimensional viewing. The process in step S1101 is the same as the process in step S501 in Figure 5 , so the detailed description thereof will be omitted. When the display setting for guidance is turned on, the process proceeds to step S1102. When the display setting for guidance is turned off, Figure 11 the process shown ends.

[0138] In step S1102, the system control unit 50 determines whether recommended distance information related to the lens unit 300 attached to the camera 100 is stored in the non-volatile memory 219 of the camera 100. The recommended distance information is stored in, for example, Figure 12 the recommended distance information table shown.

[0139] Figure 12 is a diagram showing the recommended distance information table. For each type of lens unit 300, the recommended distance is different. For each lens ID showing the type of lens unit for photography, the recommended distance information table stores information related to the minimum value and the maximum value of the recommended distance. For example, in the lens unit 300 with the lens ID of "001", the minimum value of the recommended distance of the lens unit 300 is "x1", and its maximum value is "x2". By referring to the recommended distance information table, the system control unit 50 can acquire the recommended distance information corresponding to the lens ID of the lens unit 300 attached to the camera 100.

[0140] In step S1102, when the lens ID of the lens unit 300 attached to the camera 100 during photography exists in the recommended distance information table, the system control unit 50 can determine that the recommended distance information related to the lens unit 300 is retained. When the recommended distance information related to the lens unit 300 is retained, the process proceeds to step S1103. When the recommended distance information related to the lens unit 300 is not retained, Figure 11 the processing shown ends.

[0141] In step S1103, the system control unit 50 acquires the recommended distance information corresponding to the lens unit 300 attached to the camera 100. The system control unit 50 can acquire the recommended distance information from the recommended distance information table. When the recommended distance information is retained inside the camera 100, the system control unit 50 can acquire the recommended distance information without receiving the recommended distance information from the lens unit 300.

[0142] In step S1104, the system control unit 50 generates a distance map (subject distance map) showing the subject distance information of the entire captured image. By using the binocular lens unit 300 to capture an image, the system control unit 50 can generate a distance map of the entire captured image by utilizing the distance calculation of the stereo camera. The method of calculating the distance using a stereo camera is a known technique, and thus its detailed description will be omitted.

[0143] The system control unit 50 can calculate the subject distance, for example, using the following formula 6. W represents the baseline length between the left lens and the right lens, F represents the focal length, and D represents the parallax.

[0144] Subject distance L = W × F / D... Formula 6

[0145] The system control unit 50 performs the calculation on the entire captured image using Formula 6. In the area where there is subject information such as an edge, the parallax information is calculable, but in the flat area of the captured image, the parallax information is not obtainable. Therefore, by interpolating the distance information in the flat area using the surrounding distance information, the system control unit 50 can generate a distance map of the entire captured image.

[0146] The method for generating the distance map is not limited to using the distance calculation of a stereo camera. For example, when the imaging unit 211 has an imaging surface phase difference sensor, the system control unit 50 can generate a distance map of the captured image based on the phase difference information output from the imaging surface phase difference sensor. In addition, when the camera 100 has a distance measurement sensor such as a time-of-flight (TOF) sensor, the system control unit 50 can generate a distance map of the captured image based on the output from the sensor.

[0147] Figure 15AThis is a diagram showing an example of a distance map. Each of the two image regions stores distance information, and the distance map refers to an image map in which distance information is visualized by shading based on values of the subject distance for each image region.

[0148] In step S1105, the system control unit 50 determines whether the adjustment setting for the recommended distance is enabled. The process in step S1105 is the same as the process in Figure 5 step S504, and thus its detailed description will be omitted.

[0149] In step S1106, the system control unit 50 obtains an adjustment value for adjusting the recommended distance. The system control unit 50 can use the inter-pupillary distance of the user obtained from an external device to obtain the adjustment value for the recommended distance. Figure 13 This is a flowchart showing the process of setting the adjustment value for the recommended distance.

[0150] In step S1301, the system control unit 50 connects to an external device. The external device refers to, for example, an HMD. Figure 14 This is a diagram describing the connection between the camera 100 and the HMD 400 according to the second embodiment. The camera 100 is connected to the HMD 400 via a wireless connection, for example. Note that the camera 100 can also be connected to the HMD 400 via a wired connection. The system control unit 50 is wirelessly connected to the HMD 400 through the communication unit 221 and enables the transmission and reception of various data between the camera 100 and the HMD 400.

[0151] In step S1302, the system control unit 50 obtains the inter-pupillary distance from the HMD 400 serving as an external device. The HMD 400 includes a camera that captures an image of the eyes to detect the user's line of sight and is capable of measuring the inter-pupillary distance of the user using the captured eye image. The HMD 400 transmits the obtained inter-pupillary distance to the camera 100. As a result, the camera 100 can obtain the inter-pupillary distance from the HMD 400.

[0152] In step S1303, the system control unit 50, similar to Figure 7 step S705, obtains the adjustment value α3 using Equation 5 based on the inter-pupillary distance obtained in step S1302. Note that the external device is not limited to the HMD 400, but can include other devices as long as the device is capable of capturing a facial image of the user and measuring the inter-pupillary distance. In addition, the system control unit 50 can also obtain the adjustment value calculated by the external device using the inter-pupillary distance.

[0153] Note that in step S1106 of the second embodiment, an example of obtaining an adjustment value based on information from an external device is provided. However, the adjustment value can also be obtained by the camera 100 in the same manner as in the first embodiment. Further, in step S505 of the first embodiment, an example of the camera 100 obtaining an adjustment value is provided. However, the adjustment value in the first embodiment can also be obtained based on information from an external device as in the second embodiment.

[0154] In Figure 11 step S1107, the system control unit 50 uses the adjustment value obtained in step S1106 to adjust the recommended distance. The processing in step S1107 is the same as the processing in Figure 5 step S506, and thus its detailed description will be omitted.

[0155] In step S1108, the system control unit 50 compares the distance map generated in step S1104 with the recommended distance obtained in step S1103. Based on the comparison result, the system control unit 50 generates a comparison map showing the magnitude relationship between the subject distance and the recommended distance for each region (block). When it is determined in step S1105 that the adjustment setting for the recommended distance is turned on and the recommended distance is adjusted in step S1107, the system control unit 50 uses the adjusted recommended distance to generate the comparison map.

[0156] Figure 15B is a diagram showing an example of the comparison map. The comparison map refers to an image map that distinguishes blocks farther from the recommended distance, blocks at the recommended distance, and blocks closer than the recommended distance by color shading. Note that when the recommended distance information falls within the range of the recommended distance, in one embodiment, the system control unit 50 only needs to generate a comparison map that distinguishes, by color shading, the block with the maximum value farther from the recommended distance, the blocks falling within the range of the recommended distance, and the blocks closer than the shortest value of the recommended distance.

[0157] In step S1109, based on the comparison map generated in step S1108, the system control unit 50 displays the captured image on the display unit 108 to show the relationship between the recommended distance and the subject distance.

[0158] Figure 15C is a diagram showing an example of the guidance display of the captured image including two image regions. In Figure 15C the example, the blocks farther from the recommended distance and the blocks closer than the recommended distance are indicated by diagonal lines. Based on the relationship between the recommended distance and the subject distance, each block can be colored with a different color. The user acting as the photographer can identify whether the subject distance coincides with the recommended distance for each block based on differences in the display mode (such as diagonal lines and coloring of each block). By according to as Figure 15CThe guided display shown moves closer to or farther from the subject, allowing the user to adjust the shooting position to include the desired subject within the block at the recommended distance.

[0159] According to the second embodiment described above, the camera 100 generates a distance map for a captured image including two image regions, enabling the relationship between the recommended distance and the subject distance to be notified to the user for each block (region) within the image region. As a result, the user can easily adjust the shooting distance so that the desired subject is displayed as three-dimensionally viewable on the display device.

[0160] Note that the above embodiments are provided only as examples, and configurations obtained by appropriately modifying or changing the configurations of the above embodiments within the scope of the present disclosure are also included in the present disclosure. Configurations obtained by appropriately combining the configurations of the above embodiments are also included in the present disclosure.

[0161] According to the present disclosure, an imaging device capable of easily capturing an image that can be three-dimensionally viewed when capturing two image regions with parallax can be provided.

[0162] Note that the above various types of control can be processing executed by one piece of hardware (e.g., a processor or a circuit), or not processing executed by one piece of hardware (e.g., a processor or a circuit). The processing can be shared among multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) to perform the control of the entire device.

[0163] Furthermore, the above processor is a processor in a broad sense and includes a general-purpose processor and a dedicated processor. Examples of general-purpose processors include a central processing unit (CPU), a microprocessing unit (MPU), a digital signal processor (DSP), and the like. Examples of dedicated processors include a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), and the like. Examples of PLDs include a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), and the like.

[0164] Other embodiments

[0165] Embodiments of the present invention can also be implemented by the following method, that is, by providing software (program) that performs the functions of the above embodiments to a system or device via a network or various storage media, and the computer or central processing unit (CPU), microprocessing unit (MPU) of the system or device reads and executes the program.

[0166] Although the present disclosure has been described with reference to exemplary embodiments, it should be understood that the present disclosure is not limited to the disclosed exemplary embodiments. The scope of the appended claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An electronic device, comprising: an acquisition unit configured to acquire information related to a distance from the camera device to the subject; a control unit configured to control notification of a relationship between a recommended distance indicating a photographing distance that enables three-dimensional viewing of a captured image including two image areas having parallax with respect to each other, and a distance from the imaging device to the subject; as well as A setting unit configured to set whether to display information showing a relationship between the recommended distance and the distance from the imaging device to the subject.

2. The electronic device according to claim 1, wherein: The acquisition unit acquires information about the distance from the imaging device to the subject based on the focal length of the imaging device.

3. The electronic device according to claim 1, wherein: The acquisition unit acquires information about the distance from the imaging device to the subject based on the parallax between the two image areas.

4. The electronic device according to claim 1, wherein: The acquisition unit acquires information on the distance from the imaging device to the subject based on phase difference information output from an image plane phase difference sensor of the imaging device.

5. The electronic device according to claim 1, wherein: The acquisition unit acquires information on the distance from the imaging device to the subject based on an output from a TOF sensor, which is a time-of-flight sensor of the imaging device.

6. The electronic device according to any one of claims 1 to 5, further comprising: A second acquiring unit is configured to acquire information related to the recommended distance.

7. The electronic device according to claim 6, wherein: The second acquisition unit acquires information related to the recommended distance based on a base line length indicating a distance between two optical systems that respectively capture the two image areas.

8. The electronic device according to claim 6, wherein: The second acquisition unit acquires information related to the recommended distance from a lens unit that has two optical systems that respectively capture the two image areas and is attached to the imaging device.

9. The electronic device according to claim 6, wherein: The second acquisition unit acquires information related to the recommended distance that is previously retained by the imaging device.

10. The electronic device according to any one of claims 1 to 5, further comprising: An adjusting unit is configured to adjust the recommended distance.

11. The electronic device according to claim 10, wherein: The adjustment unit adjusts the information related to the recommended distance based on an interpupillary distance of a user.

12. The electronic device according to claim 11, wherein: The adjustment unit adjusts the recommended distance to be longer if the interpupillary distance is shorter than a reference value of the interpupillary distance, and adjusts the recommended distance to be shorter if the interpupillary distance is longer than the reference value.

13. The electronic device according to claim 10, wherein: The adjustment unit adjusts the recommended distance based on an instruction of a user.

14. The electronic device according to any one of claims 1 to 5, wherein: The information about the recommended distance includes a range of photographing distances that enables three-dimensional viewing of a captured image including the two image areas.

15. The electronic device according to any one of claims 1 to 5, wherein: The control unit controls coloring of the two image areas based on a relationship between the recommended distance and a distance from the imaging device to the object.

16. The electronic device according to any one of claims 1 to 5, wherein: The control unit controls display of items showing a positional relationship among the recommended distance, the object, and the imaging device.

17. The electronic device according to any one of claims 1 to 5, wherein: The control unit controls display of text information showing a relationship between the recommended distance and a distance from the imaging device to the subject.

18. The electronic device according to any one of claims 1 to 5, wherein: The control unit controls notification of a relationship between the recommended distance and the distance from the imaging device to the subject for each block within the two image areas.

19. A control method for an electronic device, the control method comprising the following steps: Acquiring information related to the distance from the camera device to the subject; controlling notification of a relationship between a recommended distance indicating a photographing distance that enables three-dimensional viewing of a captured image including two image areas having parallax with respect to each other, and a distance from the imaging device to the subject; as well as It is set whether to display information showing the relationship between the recommended distance and the distance from the imaging device to the subject. 20 . A computer program product comprising a program for causing a computer to execute each step of the control method of an electronic device according to claim 19 . 21 . A computer-readable storage medium storing a program for causing a computer to execute each step of the control method of an electronic device according to claim 19 .

Images