Electronic equipment, its control method, program, and recording medium

Abstract

To allow a user to check all over an image and also to enlarge and display the image so as to prevent a user from becoming confused.SOLUTION: Electronic equipment includes: acquisition means of acquiring a third image in which a first image captured via a first optical system and a second image captured via a second optical system and having parallax with respect to the first image are arranged side by side; and setting means of setting a target range which is included in the third image and to which predetermined processing is applied, according to a user operation, the setting means setting the target region such that the target region includes at least part of the first image or the second image.SELECTED DRAWING: Figure 13

Description

Technical Field

[0001] The present invention relates to an electronic device, a control method thereof, a program, and a recording medium.

Background Art

[0002] In recent years, digital cameras having two lens optical systems are known. If the two optical systems are arranged to image in the same direction, an image (hemispherical image) of an imaging range of 180 degrees or a stereoscopic image can be created from two images with parallax obtained respectively. If the two optical systems are arranged to image in opposite directions, an image (omnidirectional image) of a range of 360 degrees can be created from the two images obtained respectively. When a fisheye lens is used to photograph such a wide-angle image, it is obtained as a circular image.

[0003] When photographing two images with parallax using such a digital camera having two optical systems, the user will perform the photographing while checking two live view images. In a normal digital camera having one optical system, the user can enlarge and finely check one live view image.

[0004] In Patent Document 1, it is disclosed that two live view images obtained by two optical systems can be displayed on one screen. It is disclosed that when the user performs a pinch-out operation (enlargement instruction) in the display area of any one of the two live view images, the live view image is enlarged and displayed.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, the magnification instruction disclosed in Patent Document 1 does not allow the user to know in advance which area will be magnified according to the instruction. Furthermore, if an image is displayed in a circular shape on a rectangular screen, and only the area outside the image is magnified, the user may not be able to recognize which area of ​​the original image has been magnified even when looking at the magnified image.

[0007] Therefore, the present invention aims to enable users to examine images thoroughly and to enlarge them in a way that does not confuse them. [Means for solving the problem]

[0008] To solve the above problems, the present invention provides An acquisition means for acquiring a third image in which a first image captured through a first optical system and a second image captured through a second optical system and having parallax with respect to the first image are arranged side by side, It includes a setting means for setting the target range of the third image to which a predetermined process is applied, in response to user operation, The setting means sets the target range such that the target range includes at least a part of the first image or the second image. Display the third image such that the first image is on the left and the second image is on the right. In the first image, there is a first region where the item indicating the target range can be moved, and in the second image, there is a second region where the item can be moved. If the position of the item is specified when the rightmost edge of the item reaches the rightmost edge of the first region, the position of the item is changed to the second region. If the position of the item is specified while the left edge of the item has reached the left edge of the second area, the system is characterized by controlling the position of the item to be changed to the first area. [Effects of the Invention]

[0009] According to the present invention, images can be viewed in detail, and enlarged without causing confusion to the user. [Brief explanation of the drawing]

[0010] [Figure 1] This is an external view of the Digital Camera 100. [Figure 2] This is a schematic block diagram showing an example of the hardware configuration of digital camera 100. [Figure 3] This is a schematic diagram showing an example of the configuration of a lens unit. [Figure 4] This is a flowchart showing the process for switching the camera's display mode when a dual-lens system (for VR180) is attached, and the control process during touchdown operations. [Figure 5] This figure shows an example of the live view display of the camera when a dual-lens system (VR 180-degree lens) is attached according to this embodiment. [Figure 6] This is a control flowchart for moving the magnification frame of the camera when a dual-lens system is attached. [Figure 7] This diagram illustrates the left and right boundaries of the camera's live view when a dual-lens system is attached. [Figure 8] This is a flowchart of the camera's magnification process and shooting operation when a dual-lens system is attached. [Figure 9] This is a diagram illustrating the magnification operation of a camera with a dual-lens system. [Figure 10] This is a control flowchart showing the method for calculating the movement and display position of the enlarged frame during touchdown. [Figure 11] This is a control flowchart showing the method for calculating the movement and display position of the enlarged frame when a direction indication is given. [Figure 12] This figure shows an example of the physical coordinate system of an optical image acquired by the imaging unit of a camera equipped with a twin-lens system. [Figure 13] This diagram shows the transformation of the physical coordinate system of the optical image acquired by the imaging unit of a camera equipped with a twin-lens system to a logical coordinate system. [Modes for carrying out the invention]

[0011] Hereinafter, a preferred embodiment of the present invention will be described with reference to the drawings. In this embodiment, the case where the electronic device is a digital camera (imaging device) will be described as an example.

[0012] The digital camera 100 according to this embodiment can acquire a stereoscopic image in which a left image and a right image having a predetermined parallax in the left - right direction are arranged side by side in one image and display it on the display unit. Also, the digital camera 100 can apply a predetermined image processing to the image displayed on the display unit with respect to the target range. The predetermined image processing is, for example, an enlargement process. In the following description, the case of the enlargement process will be described in detail. The digital camera 100 displays an enlarged image obtained by enlarging the target range on the display unit in response to an enlargement instruction. Note that the predetermined image processing is not limited to the enlargement process. For example, it may be a process of detecting the luminance distribution or chromaticity distribution within the target range and generating a histogram or a waveform monitor, or a process of applying a filter process such as contrast enhancement processing to the target range. In this embodiment, it is assumed that the enlargement instruction is given by pressing an enlargement button 78 which is a physical member that can be pressed, but it may also be an enlargement instruction by pinching in on the touch panel 70a and the cancellation of the enlarged display by pinching out.

[0013] The digital camera 100 according to this embodiment displays an item indicating the target range in the stereoscopic image in accordance with the stereoscopic image. The item is, for example, a frame - shaped indicator indicating the target range, a semi - transparent color image synthesized with the target range, or the like. The digital camera 100 can change the display position of the item (that is, the target range indicated by the item) according to a user operation. When the digital camera 100 is displaying a stereoscopic image, it displays the item at a position that does not straddle the left image and the right image. That is, the target range is determined so as not to include both the left image and the right image. In other words, it can be said that the target range is set to include only one of the left image and the right image.

[0014] Even if there is a user operation to change the display position of an item (the position of the target range), the digital camera 100 controls the display of the item so that the target range indicated by the item does not span the left and right images.

[0015] FIGS. 1(a) and (b) show an external view of a digital camera 100 as an example of an apparatus to which the present invention is applicable. FIG. 1(a) is a front perspective view of the digital camera 100, and FIG. 1(b) is a rear perspective view of the digital camera 100. In FIG. 1, a display unit 28 is a display unit provided on the back of the camera that displays images and various types of information. A touch panel 70a is a touch detection means capable of detecting a touch operation on the display surface (operation surface) of the display unit 28. An external viewfinder display unit 43 is a display unit provided on the upper surface of the camera, and various setting values of the camera, such as shutter speed and aperture, are displayed.

[0016] A shutter button 61 is an operation unit for giving a shooting instruction. A mode changeover switch 60 is an operation unit for switching various modes. A terminal cover 40 is a cover for protecting a connector (not shown) such as a connection cable for connecting a connection cable with an external device and the digital camera 100. A main electronic dial 71 is a rotary operation member included in an operation unit 70, and by turning this main electronic dial 71, setting values such as shutter speed and aperture can be changed. A power switch 72 is an operation member for switching ON and OFF of the power supply of the digital camera 100.

[0017] A sub - electronic dial 73 is included in the operation unit 70 and is a rotary operation member included in the operation unit 70, and can perform operations such as moving a selection frame and image scrolling. A cross key 74 is included in the operation unit 70 and is a cross key (4 - direction key) in which the upper, lower, left, and right parts can be respectively pushed in. Operations corresponding to the pushed part of the cross key 74 are possible. A SET button 75 is included in the operation unit 70 and is a push button, mainly used for determining a selected item, etc. A movie button 76 is used for instructing the start and stop of movie shooting (recording).

[0018] The magnification button 78 is included in the control unit 70 and is used to turn the magnification mode ON and OFF in the live view display in shooting mode. By turning the magnification mode ON and then operating the main electronic dial 71, the LV image can be enlarged or reduced. In playback mode, it functions as a magnification button to enlarge the playback image and increase the magnification ratio. The playback button 79 is also included in the control unit 70 and is used to switch between shooting mode and playback mode. By pressing the playback button 79 while in shooting mode, the camera switches to playback mode, and the latest image recorded on the recording medium 200 can be displayed on the display unit 28.

[0019] The menu button 81 is included in the operation unit 70, and when the menu button 81 is pressed, various configurable menu screens are displayed on the display unit 28. The user can intuitively make various settings using the menu screen displayed on the display unit 28, the directional keys 74, and the SET button 75.

[0020] The multi-controller 82 can be used to control the eight directional keys (up, down, left, right, etc.) by tilting it 360 degrees. It can also be used to activate assigned functions by pressing it down. The display mode switching button 83 is an operating element for switching between multiple different display modes for information such as the live view image and shooting information displayed on the display unit 28 or EVF 29. Each time the display mode switching button 83 is pressed, the display mode is switched, allowing the user to view the information of the image being shot or played back in the display mode of their choice.

[0021] The communication terminal 10 is a communication terminal used by the digital camera 100 to communicate with the lens side (which is detachable).

[0022] The eyepiece section 16 is the eyepiece of an eyepiece viewfinder (a type of viewfinder that you look through), and the user can view the image displayed on the internal EVF 29 through the eyepiece section 16. The eyepiece detection section 57 is an eyepiece detection sensor that detects whether or not the photographer is looking through the eyepiece section 16. The cover 202 is the cover of the slot that houses the recording medium 200. The grip section 90 is a holding section shaped to be easy for the user to grip with their right hand when holding the digital camera 100. With the digital camera held by gripping the grip section 90 with the little finger, ring finger, and middle finger of the right hand, the shutter button 61 and the main electronic dial 71 are positioned to be operated by the index finger of the right hand. Also, in the same position, the sub electronic dial 73 is positioned to be operated by the thumb of the right hand.

[0023] Figure 2 is a block diagram showing an example configuration of the digital camera 100 according to this embodiment. In Figure 2, the lens unit 150 is a lens unit equipped with an interchangeable photographic lens. The lens 103 is usually composed of multiple lenses, but here it is shown simply as a single lens. Communication terminal 6 is a communication terminal for the lens unit 150 to communicate with the digital camera 100, and communication terminal 10 is a communication terminal for the digital camera 100 to communicate with the lens unit 150. The lens unit 150 communicates with the system control unit 50 via these communication terminals 6 and 10, controls the aperture 1 via the aperture drive circuit 2 using the internal lens system control circuit 4, and focuses by displacing the position of the lens 103 via the AF drive circuit 3. In addition, the type of lens unit 150 attached to the digital camera 100 is identified via the communication terminals 6 and 10.

[0024] The shutter 101 is a focal-plane shutter that allows the exposure time of the imaging unit 22 to be freely controlled by the system control unit 50.

[0025] The imaging unit 22 is an image sensor composed of a CCD or CMOS element, etc., which converts an optical image into an electrical signal. The imaging unit 22 may have an image plane phase difference sensor that outputs defocus amount information to the system control unit 50. The A / D converter 23 converts an analog signal into a digital signal. The A / D converter 23 is used to convert the analog signal output from the imaging unit 22 into a digital signal.

[0026] The image processing unit 24 performs resizing and color conversion processing, such as predetermined pixel interpolation and reduction, on the data from the A / D converter 23 or the data from the memory control unit 15. The image processing unit 24 also performs predetermined calculations using the captured image data. Based on the calculation results obtained by the image processing unit 24, the system control unit 50 performs exposure control and distance measurement control. This enables TTL (through-the-lens) AF (autofocus), AE (automatic exposure), and EF (flash pre-flash) processing. The image processing unit 24 further performs predetermined calculations using the captured image data and performs TTL AWB (auto white balance) processing based on the calculation results obtained.

[0027] The output data from the A / D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15, or directly via the memory control unit 15. The memory 32 stores image data obtained by the imaging unit 22 and converted into digital data by the A / D converter 23, as well as image data for display on the display unit 28 and EVF 29. The memory 32 has sufficient storage capacity to store a predetermined number of still images, a predetermined amount of video footage, and audio.

[0028] Furthermore, memory 32 also serves as memory for image display (video memory). The D / A converter 19 converts the image display data stored in memory 32 into an analog signal and supplies it to the display unit 28 and EVF 29. In this way, the display image data written to memory 32 is displayed by the display unit 28 and EVF 29 via the D / A converter 19. The display unit 28 and EVF 29 display the image on a display device such as an LCD or organic EL according to the analog signal from the D / A converter 19. The digital signal, which has been A / D converted once by the A / D converter 23 and stored in memory 32, is converted to analog by the D / A converter 19 and sequentially transferred to the display unit 28 or EVF 29 for display, thereby enabling live view display (LV display). Hereinafter, the image displayed in live view will be referred to as a live view image (LV image).

[0029] The external LCD display unit 43 displays various camera settings, including shutter speed and aperture, via the external display unit drive circuit 44.

[0030] The non-volatile memory 56 is an electrically erasable and recordable memory, such as an EEPROM. The non-volatile memory 56 stores constants for the operation of the system control unit 50, programs, etc. The program referred to here is a program for executing various flowcharts described later in this embodiment.

[0031] The system control unit 50 is a control unit consisting of at least one processor or circuit, and controls the entire digital camera 100. It realizes each of the processes of this embodiment, which will be described later, by executing the program recorded in the non-volatile memory 56 mentioned above. For example, RAM is used in the system memory 52, and constants, variables for the operation of the system control unit 50, the program read from the non-volatile memory 56, etc. are stored there. The system control unit 50 also performs display control by controlling the memory 32, the D / A converter 19, the display unit 28, etc.

[0032] The system timer 53 is a timekeeping unit that measures the time used for various controls and the time of the built-in clock.

[0033] The operation unit 70 is an operating means for inputting various operation instructions to the system control unit 50. The mode switching switch 60 is an operating component included in the operation unit 70 and switches the operating mode of the system control unit 50 to one of the following: still image shooting mode, video shooting mode, playback mode, etc. Modes included in the still image shooting mode include auto shooting mode, auto scene detection mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). There are also various scene modes and custom modes that provide shooting settings for different shooting scenes. The user can switch directly to any of these modes using the mode switching switch 60. Alternatively, the user can switch to a list screen of shooting modes using the mode switching switch 60, select one of the displayed modes, and then switch using another operating component. Similarly, the video shooting mode may also include multiple modes.

[0034] The first shutter switch 62 turns ON during the operation of the shutter button 61 on the digital camera 100, specifically when it is half-pressed (indicating preparation for shooting), and generates the first shutter switch signal SW1. The first shutter switch signal SW1 initiates shooting preparation operations such as AF (autofocus), AE (automatic exposure), AWB (auto white balance), and EF (flash pre-flash).

[0035] The second shutter switch 64 turns ON when the shutter button 61 is fully pressed (shooting instruction), generating the second shutter switch signal SW2. The system control unit 50 starts a series of shooting processes, from reading the signal from the imaging unit 22 to writing the captured image to the recording medium 200 as an image file, in response to the second shutter switch signal SW2.

[0036] The operation unit 70 consists of various operating components (reception means) that act as an input unit for receiving operations from the user. The operation unit 70 includes at least the following operating components: shutter button 61, main electronic dial 71, power switch 72, sub electronic dial 73, cross key 74, SET button 75, video button 76, AF lock button 77, zoom button 78, playback button 79, menu button 81, and multi-controller 82. Other operating components 70b represent a collection of operating components that are not individually described in the block diagram.

[0037] The power control unit 80 consists of a battery detection circuit, a DC-DC converter, a switch circuit for switching which blocks are energized, and detects whether a battery is installed, the type of battery, and the remaining battery level. The power control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, supplying the necessary voltage to each part, including the recording medium 200, for the required period. The power supply unit 30 consists of primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, and Li batteries, and an AC adapter.

[0038] The recording medium I / F18 is an interface to the recording medium 200, such as a memory card or hard disk. The recording medium 200 is a recording medium such as a memory card for recording captured images, and is composed of semiconductor memory, magnetic disks, etc.

[0039] The communication unit 54 is connected wirelessly or via a wired cable and transmits and receives video and audio signals. The communication unit 54 can also connect to a wireless LAN (Local Area Network) or the internet. Furthermore, the communication unit 54 can communicate with external devices using Bluetooth® or Bluetooth Low Energy. The communication unit 54 can transmit images (including LV images) captured by the imaging unit 22 and images recorded on the recording medium 200, and can also receive images and other various information from external devices.

[0040] The attitude detection unit 55 detects the orientation of the digital camera 100 relative to the direction of gravity. Based on the orientation detected by the attitude detection unit 55, it is possible to determine whether the image captured by the imaging unit 22 was taken with the digital camera 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the orientation detected by the attitude detection unit 55 to the image file of the image captured by the imaging unit 22, or rotate the image before recording. An acceleration sensor or gyro sensor can be used as the attitude detection unit 55. It is also possible to detect the movement of the digital camera 100 (pan, tilt, lift, whether it is stationary or not, etc.) using the acceleration sensor or gyro sensor in the attitude detection unit 55.

[0041] The eyepiece detection unit 57 is an eyepiece detection sensor that detects the approach (eye-to-eye contact) and retraction (eye-away) of an eye (object) to the eyepiece 16 of the viewfinder (proximity detection). The system control unit 50 switches the display (display state) / hidden (hidden state) of the display unit 28 and the EVF 29 according to the state detected by the eyepiece detection unit 57. More specifically, at least in the shooting standby state and when the display destination switching is automatic, when the eye is not focused, the display destination is set to the display unit 28 and the display is turned on, and the EVF 29 is hidden. When the eye is focused, the display destination is set to the EVF 29 and the display is turned on, and the display unit 29 is hidden.

[0042] The eyepiece detection unit 57 can, for example, use an infrared proximity sensor to detect the approach of any object to the eyepiece 16 of the viewfinder, which houses the EVF 29. When an object approaches, infrared light emitted from the light emitter (not shown) of the eyepiece detection unit 57 is reflected and received by the light receiver (not shown) of the infrared proximity sensor. The amount of infrared light received can also be used to determine how close the object is to the eyepiece 16 (eyepiece distance). In this way, the eyepiece detection unit 57 performs eyepiece detection to detect the proximity distance of an object to the eyepiece 16.

[0043] The system detects that the eyepiece has been used when an object approaches the eyepiece 16 within a predetermined distance from the non-eyepiece state (non-proximity state). The system detects that the eye has been removed when the object that was detected approaching moves beyond a predetermined distance from the eyepiece state (proximity state). The threshold for detecting eyepiece use and the threshold for detecting eye removal may be different, for example, by providing hysteresis. After detecting eyepiece use, the system remains in the eyepiece state until eye removal is detected. After detecting eye removal, the system remains in the non-eyepiece state until eyepiece use is detected again. Note that the infrared proximity sensor is just one example, and the eyepiece detection unit 57 may use any other sensor that can detect the approach of an eye or object that can be considered as eyepiece use.

[0044] The touch panel 70a and the display unit 28 can be configured as an integrated unit. For example, the touch panel 70a is configured such that its light transmittance does not interfere with the display of the display unit 28, and is mounted on the upper layer of the display surface of the display unit 28. Then, the input coordinates on the touch panel 70a are associated with the display coordinates on the display screen of the display unit 28. This makes it possible to provide a GUI (Graphical User Interface) that makes it seem as if the user can directly operate the screen displayed on the display unit 28. The system control unit 50 can detect the following operations or states on the touch panel 70a. - A finger or pen that was not previously touching the touch panel 70a now touches the touch panel 70a. In other words, the start of a touch (hereinafter referred to as Touch-Down). The touch panel 70a is being touched with a finger or pen (hereinafter referred to as Touch-On). - The user is moving while touching the touch panel 70a with their finger or pen (hereinafter referred to as Touch-Move). • The finger or pen that was touching the touch panel 70a is lifted. In other words, the touch action ends (hereinafter referred to as Touch-Up). • The touch panel 70a is not being touched (hereinafter referred to as Touch-Off).

[0045] When a touchdown is detected, a touch-on state is also detected simultaneously. After a touchdown, a touch-on state is usually detected unless a touch-up is detected. Touch move is also detected when a touch-on state is detected. Even if a touch-on state is detected, a touch move will not be detected if the touch position has not moved. After all fingers or pens that were touching have been detected as having touched up, the touch-off state is activated.

[0046] These operations and states, as well as the position coordinates of the finger or pen touching the touch panel 70a, are notified to the system control unit 50 via the internal bus. Based on the notified information, the system control unit 50 determines what kind of operation (touch operation) was performed on the touch panel 70a. For touch moves, the direction of movement of the finger or pen moving on the touch panel 70a can also be determined for each vertical and horizontal component on the touch panel 70a based on the change in position coordinates.

[0047] If a touch movement exceeding a predetermined distance is detected, it will be determined that a slide operation has been performed. A flick is an operation in which a finger is quickly moved a certain distance while touching the touch panel and then released. In other words, a flick is an operation in which the finger is quickly swiped across the touch panel 70a. If a touch movement exceeding a predetermined distance and at a predetermined speed is detected, and a touch-up is then detected, it can be determined that a flick has been performed (it can be determined that a flick followed a slide operation).

[0048] Furthermore, touching multiple points (for example, two points) simultaneously (multitouch) to bring them closer together is called a pinch-in, and touching them further apart is called a pinch-out. Pinch-out and pinch-in are collectively referred to as a pinch operation (or simply a pinch). The touch panel 70a may use any of the various types of touch panels, such as resistive, capacitive, surface acoustic wave, infrared, electromagnetic induction, image recognition, and optical sensor types. Depending on the type, a touch may be detected when there is contact with the touch panel, or when a finger or pen approaches the touch panel; either type is acceptable.

[0049] Figure 3 is a schematic diagram showing an example of the configuration of the lens unit 300. Figure 3 shows the lens unit 300 attached to the digital camera 100. Note that components of the digital camera 100 shown in Figure 3 that are the same as those described in Figure 2 are denoted by the same reference numerals, and their descriptions are omitted as appropriate.

[0050] The lens unit 300 is a type of interchangeable lens that can be attached to and removed from the digital camera 100. The lens unit 300 is a twin-lens system capable of acquiring optical images with parallax between the left and right images. The lens unit 300 has two optical systems (photographic lenses), each with a wide field of view of 180 degrees, and can capture an area of ​​the front hemisphere. Specifically, the two optical systems of the lens unit 300 can each input optical images of subjects contained within a field of view (angle of view) of 180 degrees in the left-right direction (horizontal angle, azimuth angle, yaw angle) and 180 degrees in the up-down direction (vertical angle, elevation / depression angle, pitch angle).

[0051] The lens unit 300 includes a right-eye optical system 301R having multiple lenses and a reflective mirror, a left-eye optical system 301L having multiple lenses and a reflective mirror, and a lens system control circuit 303. The right-eye optical system 301R corresponds to an example of a first optical system, and the left-eye optical system 301L corresponds to an example of a second optical system. In the right-eye optical system 301R and the left-eye optical system 301L, the lenses 302R and 302L located on the subject side are oriented in the same direction, and their optical axes are parallel. In addition, each optical system is a so-called fisheye lens, and a circular optical image is formed on the imaging unit 22 (sensor). The optical image input via the left-eye optical system 301L (left image) and the optical image input via the right-eye optical system 301R (right image) are imaged onto the imaging plane of one imaging unit 22, and the imaging unit 22 acquires an image including each optical image.

[0052] The lens unit 300 of this embodiment is a VR180 lens for capturing images for so-called VR180, a VR image format that enables binocular stereoscopic viewing. The VR180 lens has a fisheye lens in which the right eye optical system 301R and the left eye optical system 301L are each capable of capturing a range of 180 degrees. The VR180 lens only needs to be capable of capturing images that enable binocular VR display as VR180 in which the right eye optical system 301R and the left eye optical system 301L are each capable of capturing a wide field of view of about 160 degrees, which is narrower than the 180-degree range. The VR180 lens can form a right image (first image) formed via the right eye optical system 301R and a left image (second image) formed via the left eye optical system 301L, which has parallax with the right image, on one or two image sensors of the attached camera. In the digital camera 100 according to this embodiment, the right image and the left image are formed on a single image sensor, and a single image (binacular image) is generated in which the image corresponding to the right image and the image corresponding to the left image are arranged side by side. The binacular image at this time includes the image corresponding to the right image, the image corresponding to the left image, and a region where no optical image is formed (shaded area).

[0053] Furthermore, the lens unit 300 is attached to the digital camera 100 via the lens mount portion 304 and the camera mount portion 305 of the digital camera 100. As a result, the system control unit 50 of the digital camera 100 and the lens system control circuit 303 of the lens unit 300 are electrically connected via the communication terminal 10 of the digital camera 100 and the communication terminal 306 of the lens unit 300.

[0054] In this embodiment, the right image formed via the right-eye optical system 301R and the left image formed via the left-eye optical system 301L, which has parallax with respect to the right image, are imaged side by side on the imaging unit 22 of the digital camera 100. That is, two optical images formed by the right-eye optical system 301R and the left-eye optical system 301L are formed on a single image sensor. The imaging unit 22 converts the imaged subject image (optical signal) into an analog electrical signal to acquire image data of the binocular image. In this way, by using the lens unit 300, two images with parallax can be acquired simultaneously (as a set) from two locations (optical systems) - the right-eye optical system 301R and the left-eye optical system 301L. Furthermore, by separating the acquired images into a left-eye image and a right-eye image and displaying them in VR, the user can view a three-dimensional VR image with a 180-degree range, so-called VR180.

[0055] Conventionally, with a standard single-lens reflex camera, the image incident on the lens is inverted point-symmetrically around the optical axis and input to the sensor. Imaging devices like the digital camera 100 generate images that do not appear inverted (uninverted) by adjusting the order in which the sensor reads the images and performing inversion processing on the read images. On the other hand, with a twin-lens reflex camera, the image is inverted point-symmetrically vertically and input to the sensor, but the image is not inverted horizontally. Images acquired through the left-eye optical system are input to the sensor on the left side, and images acquired through the right-eye optical system are input to the sensor on the right side. Therefore, if inversion processing is performed as before, the left and right sides of the digital camera 100 and the image after inversion processing will be reversed; that is, images acquired through the left-eye optical system will be displayed on the right side, and images acquired through the right-eye optical system will be displayed on the left side.

[0056] Here, a VR image is an image that can be displayed in VR, as described later. VR images include omnidirectional images (spherical images) taken with an omnidirectional camera (spherical camera), and panoramic images with a wider field of view (effective field of view) than the display area that can be displayed on the display unit at once. Furthermore, VR images are not limited to still images, but also include videos and live images (images acquired from the camera in near real-time). A VR image has a field of view (effective field of view) of up to 360 degrees horizontally and vertically. In addition, VR images also include images with a wider field of view than that that can be captured by a normal camera, or images with a wider field of view than the display area that can be displayed on the display unit at once, even if the field of view is less than 360 degrees horizontally or vertically. Images taken by the digital camera 100 using the aforementioned lens unit 300 are a type of VR image. VR images can be displayed in VR, for example, by setting the display mode of a display device (a display device that can display VR images) to "VR View". By displaying VR images with a 360-degree field of view in VR, users can change the orientation of the display device left or right (horizontal rotation direction) to view seamless, omnidirectional images in the left-right direction.

[0057] Here, VR display (VR view) refers to a display method (display mode) in which the display range of a VR image is changed, displaying an image within the field of view range corresponding to the orientation of the display device. VR display includes "single-eye VR display (single-eye VR view)," which displays a single image by performing a transformation (a transformation that applies distortion correction) that maps the VR image to a virtual sphere. VR display also includes "two-eye VR display (two-eye VR view)," which displays a VR image for the left eye and a VR image for the right eye side-by-side in the left and right regions by performing transformations that map each to a virtual sphere. Stereoscopic viewing is possible by using "two-eye VR display" with a VR image for the left eye and a VR image for the right eye that have parallax between them.

[0058] Regardless of the type of VR display, when a user wears a display device such as an HMD (Head-Mounted Display), the displayed image will have a field of view corresponding to the orientation of the user's face. For example, suppose a VR image is currently displaying an image with a field of view centered at 0 degrees horizontally (a specific direction, e.g., north) and 90 degrees vertically (90 degrees from the zenith, i.e., horizontal). If the orientation of the display device is then reversed (for example, changing the display surface from facing south to facing north), the display range of the same VR image will change to an image with a field of view centered at 180 degrees horizontally (the opposite direction, e.g., south) and 90 degrees vertically. In other words, if a user wearing an HMD turns their face from north to south (i.e., turns their back), the image displayed on the HMD will also change from a north image to a south image.

[0059] Note that the VR image captured using the lens unit 300 of this embodiment is a VR180 image capturing a 180-degree range in front, and there is no image in the 180-degree range behind. When such a VR180 image is displayed in VR, if the orientation of the display device is changed to the side where there is no image, a blank area will be displayed.

[0060] By displaying VR images in VR in this way, users can visually experience the sensation of being inside the VR image (in VR space). Note that the method of displaying VR images is not limited to changing the orientation of the display device. For example, the display range may be moved (scrolled) in response to user operations via a touch panel or directional buttons. Furthermore, during VR display (in "VR View" display mode), in addition to changing the display range due to changes in orientation, the display range may also be changed in response to touch movements on the touch panel, drag operations with a mouse, or pressing directional buttons. Note that a smartphone mounted on VR goggles (head-mounted adapter) is a type of HMD (Head-Mounted Display).

[0061] In the digital camera 100 configured as described above, the binocular image captured via the lens unit 300 is an image that includes the right image and the left image input to the imaging unit 22 via the right-eye optical system and the left-eye optical system. Furthermore, users of the digital camera 100 may enlarge a portion of the image to check the details of the live view image or recorded image. When enlarging, the center position of the enlarged area may be uniquely set to the center position of the entire image.

[0062] When magnifying a binocular image to examine a portion of it, it is desirable to display only a portion of either the right or left image. If the center of the magnified area is uniquely set to the center of the entire image, and the magnified image includes portions of both the right and left images, it becomes difficult for the user to intuitively understand which part of the original image the magnified image represents. Specifically, the left edge of the right image will be on the right side of the magnified image, and the right edge of the left image will be on the left side, resulting in a magnified image where the left-right positional relationship with the field of view of the subject is different.

[0063] Therefore, in this embodiment, the processing of the digital camera 100, which performs live view magnification processing suitable for shooting with a twin-lens system such as the lens unit 300, will be explained with reference to the flowchart in Figure 4.

[0064] Figure 4 is a flowchart illustrating an example of processing in the shooting mode of the digital camera 100. The flowchart in Figure 4 is realized when the system control unit 50 loads the program recorded in the non-volatile memory 219 into the system memory 52 and executes it. The flowchart in Figure 4 is started when the digital camera 100 is powered on and in shooting standby mode. When starting the control flowchart in Figure 4, the system control unit 50 initializes control variables and other parameters and starts processing.

[0065] Furthermore, an example of the display on the display unit 28 when the control flowchart in Figure 4 is executed will be explained using Figure 5. Details of the display example shown in Figure 5 will be explained after the explanation of the control flowchart in Figure 4.

[0066] In S401, the system control unit 50 obtains the previously used display mode from a flag (N) stored in the non-volatile memory 56, and displays the live view image and shooting-related information on the display unit 28 based on the previously used display mode. The previously used display mode is, for example, the display mode that was in use when the power was turned off if the flow was started by turning on the power. Alternatively, if the flow was started by switching from a mode other than shooting mode, such as playback mode, to shooting mode, it is the display mode that was in use when processing was performed in shooting mode last time.

[0067] In S402, the system control unit 50 determines whether the display mode switching button 83 has been pressed. If it has been pressed, the process proceeds to S403; otherwise, the process proceeds to S412.

[0068] In S403, the system control unit 50 refers to the system memory 52 and determines whether the flag N, which indicates the display mode, is 5 (N=5). If N=5, proceed to S404; otherwise, proceed to S405.

[0069] In S404, the system control unit 50 determines via communication terminals 6 and 10 whether the mounted lens is a twin-lens lens (i.e., a lens for VR180). If it is a twin-lens lens, the process proceeds to S406; otherwise (i.e., it is a normal single-lens lens or no lens is mounted), the process proceeds to S408. A twin-lens lens is a wide-angle fisheye lens equipped with lenses on the left and right sides, each lens capable of capturing at least 180 degrees of the range on the side where the lens is located, i.e., the subject side. It is possible to capture data of the left and right images captured through the left-eye optical system 301L and the right-eye optical system 301R using one or two image sensors.

[0070] In S405, since the result in S403 was determined to be No, the system control unit 50 increments the flag N, which indicates the display mode, by 1 (N = N + 1) and stores it in the system memory 52.

[0071] In S406, the system control unit 50 sets the flag N to 6 (N=6) and stores it in the system memory 52.

[0072] In S407, the system control unit 50 displays a display mode specifically for the twin-lens system on the display unit 28 when N=6. An example of this display is shown in Figure 5(f). Figure 5(f) will be explained in detail later.

[0073] In S408, since the result in S404 was determined to be No, the system control unit 50 sets flag N to 1 (N=1) and stores it in the system memory 52.

[0074] In S409, the system control unit 50 displays the live view image and information in the display mode corresponding to the display unit 28, according to the value of flag N stored in the system memory 52. ​​Examples of displays corresponding to the display modes displayed on the display unit 28 are shown in Figures 5(a) to (e). Since this step is reached even if No is determined in S404, the display is also shown when the dual-lens system is not installed, i.e., when a single-lens system is installed or no lens is installed.

[0075] Figures 5(a) to 5(f) will be used to explain the LV images and information displayed on the display unit 28. Figures 5(a) to 5(f) show two LV images when the digital camera 100 is equipped with a twin-lens system (VR180 lens). However, when a single-lens system is used, only one LV image is displayed, and the rest of the display is the same. When a single-lens system is used, the display mode shown in Figure 5(f) does not change, as described above using Figure 4. Each time the user switches the display mode (by pressing the display mode switching button 83 in this embodiment), the information displayed on the display unit 28 is changed. Specifically, the display in Figures 5(a) to 5(f) is performed according to the number of the flag N which indicates the display mode as described above in Figure 4. In this embodiment, the LV image displayed on the display unit 28 is a circular fisheye display, but it is not limited to this; an equirectangular transformation process may be performed on the circular fisheye LV image to make it an equirectangular display.

[0076] Figure 5(a) shows the display mode when the flag N=1, i.e., when the digital camera 100 is powered on and in shooting standby mode, and the LV image is displayed on the display unit 28. Two LV images (LV500R, LV500L) are displayed side by side, and information displays 501a to 501c are also displayed on the display unit 28. The two LV images displayed side by side at this time are called side-by-side images. Information displays 501a to 501c show shooting information and are the minimum information that the user is expected to need when taking a picture. When the display mode switching button 83 is pressed in the state shown in Figure 5(a), the display transitions to Figure 5(b).

[0077] Figure 5(b) shows the display modes when flag N=2. In Figure 5(b), in addition to LV500R, LV500L, and information displays 501a~501c, information displays 502a and 502b are displayed. Information displays 502a and 502b, like information displays 501a~501c, show shooting information and various information related to shooting (for example, the currently set settings and the type of recording medium 200 inserted). Since the amount of information is greater than that of information displays 501a~501c, the user can see more shooting information. On the other hand, the visibility of the LV image may decrease. When the display mode switching button 83 is pressed in the state shown in Figure 5(b), the display transitions to Figure 5(c).

[0078] Figure 5(c) shows the display mode when flag N=3. In Figure 5(c), in addition to LV500R, LV500L, information displays 501a~501c, 502a, 502b, information display 503, which is a histogram of the currently captured LV image, is displayed. Information display 503 is a graph with brightness on the horizontal axis and the number of pixels on the vertical axis, and serves as a guide to check how bright the currently captured LV image is, the trend of the exposure level, and the overall gradation of the LV image. Some users may want to check information display 503 during shooting, so it is displayed. On the other hand, since it is superimposed on a relatively large area of ​​the LV image, visibility is reduced. Pressing the display mode switching button 83 in the state shown in Figure 5(c) will transition to Figure 5(d).

[0079] Figure 5(d) shows the display mode when flag N=4. In the display mode of Figure 5(d), all information displays 501a~501c, 502a, 502b, and 503 are hidden, and only LV500R and LV500L are displayed. This display allows the user to take pictures while viewing only the LV image without being bothered by various shooting information. Pressing the display mode switching button 83 in the state shown in Figure 5(d) will transition to Figure 5(e).

[0080] Figure 5(e) shows the display mode when flag N=5. In the display mode shown in Figure 5(e), the LV image is not displayed, and only information related to shooting is displayed in the form shown in the table. When the display mode switching button 83 is pressed in the state shown in Figure 5(e), the display transitions to Figure 5(f) if a twin-lens system is attached, and to Figure 5(a) if a single-lens system is attached or no lens is attached.

[0081] Figure 5(f) shows the display mode when flag N=6 (S407 in Figure 4). In other words, this is the display mode that is transitioned to only when a twin-lens reflex camera is attached to the digital camera 100. At this time, the display unit 28 displays LV500R, LV500L, information displays 505, 506, an enlarged frame 511 superimposed on the LV image, a focus guide 512, and a magic window 513. Information display 505 indicates that LV500R is an LV image captured by the right eye optical system 301R, and displays "R" to mean right. Information display 506 indicates that LV500L is an LV image captured by the left eye optical system 301L, and displays "L" to mean left.

[0082] In the case of an optical system like the one shown in digital camera 100, the image captured by the image sensor (image unit 22) is captured upside down. The top and bottom of this upside-down image are then flipped 180 degrees before being displayed on the display unit 28 or EVF 29. Given this structure, let's consider taking pictures using a lens with two optical systems (twin-lens system) as shown in Figure 3. As mentioned above, if the two LV images captured by the left and right optical systems and input to the image unit 22 are displayed on the display unit 28 without inversion processing, the LV image will be displayed upside down, which is inconvenient for the user. Therefore, just as with a single-lens system, inversion processing is performed on the two LV images input to the image unit 22.

[0083] However, even after the inversion process, the two LV images displayed on the display unit 28 will still show the LV image acquired via the right eye optical system 301R (right image) in the left region and the LV image acquired via the left eye optical system 301L (left image) in the right region. In particular, in this embodiment, since imaging is performed with a single image sensor (imaging unit 22), the process of identifying the boundary between the two LV images on a single image sensor and swapping the left and right LV images would place a heavy processing load on the system control unit 50. Therefore, the LV image received by the imaging unit 22 is inverted 180 degrees vertically and displayed on the display unit 28 without swapping the left and right images.

[0084] When two LV images are displayed on the display unit 28, it is common for users to assume that the LV image displayed in the left region was captured by the left-eye optical system 301L, and the LV image displayed in the right region was captured by the right-eye optical system 301R. In other words, without the information displays 505 and 506, users are unlikely to understand that the left and right LV images are horizontally inverted, and if they are not aware of the horizontal inversion during shooting, they may become confused. Therefore, by displaying the information displays 505 and 506, users can clearly see which optical system (left or right) captured each LV image.

[0085] The magnification frame 511 superimposed on the LV image is a frame display that indicates the area of ​​the LV image to be magnified when the user gives a magnification instruction (similar to S801 in Figure 8, described later). The focus guide 512 is an indicator that displays a frame indicating the focus detection area and the degree of focus on the subject at the position where the focus detection area is superimposed, using information such as indicators and colors. In this embodiment, the degree of focus is expressed by three indicators displayed above or below the focus detection area. From the focus guide 512 in Figure 5(f), it can be seen that the subject at the position where the focus detection area is displayed is out of focus and is back-focused (focus is behind the subject and not on the subject). Also, when the image is not in focus, the focus guide is displayed in white. When the number of indicators decreases to two, it indicates that the image is in focus, and the focus guide is displayed in green.

[0086] The Magic Window 513 is an indicator displayed on the display unit 28 that shows the range that is initially displayed before the user moves their viewpoint. When a single 180-degree image (hemispherical image) is generated from the LV500R and LV500L and played back on a browser or head-mounted display (HMD), it shows the range that is initially displayed before the user moves their viewpoint. By displaying the Magic Window 513 on the LV image, the range that is initially displayed during playback, i.e., the range that the viewer sees first, can be visually confirmed during shooting, allowing the user to more effectively capture their desired composition and subject.

[0087] The Magic Window 513 is an indicator that is only necessary when creating a 180-degree image, and is a unique indicator that is expected to be particularly useful for users when shooting with a twin-lens system. Therefore, the display mode shown in Figure 5(f) is not displayed when a single-lens system is attached. Also, since the display position of the Magic Window 513 is fixed, it is not necessarily required that it be displayed at all times during subsequent shooting, as long as the user can confirm the range of the Magic Window 513.

[0088] In this embodiment, the magnification frame 511 and focus guide 512 are superimposed on only one of the two LV images, while the magic window 513 is superimposed on both LV images. Since the magnification frame 511 indicates the magnification position, it is optimal to display only one; however, the focus guide 512 may be displayed on both LV images instead of just one.

[0089] In S410, similar to S404, the system control unit 50 determines whether the lens to be attached is a twin-lens lens via the communication terminals 6 and 10. If it is a twin-lens lens, the process proceeds to S411; otherwise, it proceeds to S412.

[0090] In S411, the system control unit 50 calculates the center and size of the left and right images from the lens information acquired via the communication terminals 6 and 10. The calculation method at this time will be explained using Figure 12.

[0091] Figure 12 shows the physical coordinate system 1200 of the optical image formed on the imaging unit 22 (image sensor) of the digital camera 100 according to this embodiment. The coordinate system 1200 corresponds to the view of the imaging unit 22 from the lens unit 300 side.

[0092] In this embodiment, a circular optical image 1201R formed by the left eye optical system 301L and a circular optical image 1201L formed by the right eye optical system 301R are formed on a single image sensor. The imaging unit 22 converts the formed subject image into an analog electrical signal, forming the logical coordinate system 500 shown in Figure 13.

[0093] The system control unit 50 can acquire the following information from the attached lens via the communication terminals 6 and 10. For example, it can acquire the distance 1203R between the center line 1210 of the image sensor 1200 and the center position of the circular optical image 1201R, and the distance 1203L between the center line 1210 of the image sensor 1200 and the center position of the circular optical image 1201L. In addition, it can acquire the radius 1204R of the optical image 1201R and the radius 1204L of the optical image 1201L.

[0094] From this acquired lens information, the center and size of the two LV images (left image and right image) acquired through the two optical systems shown in Figure 13 can be calculated.

[0095] Next, the processing when a touchdown operation is performed on the touch panel 70a will be described. The system control unit 50 of the digital camera 100 displays items related to focus control and items indicating the magnification range in the image in response to a touchdown operation on the touch panel 70a during image display. If the lens unit connected to the digital camera 100 is not a twin-lens reflex lens (it is a conventional single-lens reflex lens), the system control unit 50 displays a focus guide indicating the degree of focus or an AF frame indicating the target area of ​​autofocus at the position corresponding to the touchdown. If the lens unit connected to the digital camera 100 is a twin-lens reflex lens, the system control unit 50 places a focus guide indicating the degree of focus in the center of either the area displaying the left image or the area displaying the right image in response to the touchdown. Furthermore, the system control unit 50 displays a magnification frame indicating the magnification range at the position corresponding to the touchdown. If the lens unit connected to the digital camera 100 is a twin-lens reflex lens, the system control unit 50 displays the magnification frame so that it includes the area where the optical images of the left and right images are formed (but does not include only the shaded area that does not include the left and right images).

[0096] In S412, the system control unit 50 determines whether or not a touchdown operation has occurred on the touch panel 70a. If a touchdown operation has occurred, the coordinates (xt, xy) indicating the touchdown position (touch position) are stored, and the process proceeds to S413; otherwise, the process proceeds to S601 in Figure 6.

[0097] In S413, similar to S404, the system control unit 50 determines whether the lens to be attached via the communication terminals 6 and 10 is a twin-lens lens. If it is a twin-lens lens, proceed to S414; otherwise, proceed to S423.

[0098] In S414, the system control unit 50 determines whether the touch position (touchdown position) of the touchdown operation performed on the touch panel 70a in S412 is within the left region (region 701L) of the binocular image. Figure 7 is a schematic diagram showing an example of a display with the binocular image shown. If the touch position is within the left region (region 701L), the process proceeds to S415; if the touch position is within the right region (region 701R), the process proceeds to S417. Line 705 is the line that marks the boundary between the live view images (LV700R and LV700L) captured by the left and right optical systems, respectively. In this embodiment, line 705 is the center line when the imaging unit 22 is divided into left and right halves.

[0099] In S415, the system control unit 50 displays the focus guide in the center of the left region. If the focus guide is already superimposed on the LV image in either the left or right region, the display is moved to the center of the left region (the center of the LV image captured by the right eye optical system 301R shown in the left region). The focus guide consists of a frame indicating the focus detection region and multiple indicators that show the degree of focus in the focus detection region based on the relationship between their display positions. The relationship between the display positions of the indicators allows the user to visually confirm whether the focus is in focus at the display position of the focus detection region, to what extent it is out of focus, and, if it is out of focus, whether it is in front of or behind the subject in the focus detection region. Note that the focus guide is not limited to this display form; for example, the degree of focus may be indicated by color.

[0100] In S416, the system control unit 50 performs position calculation processing (left region) for the enlarged frame based on touch operation. This process will be described later using Figure 10(a).

[0101] In S417, since the result in S414 was determined to be No, the system control unit 50 displays the focus guide in the center of the right region (the center of the LV image captured by the right eye optical system 301L shown in the right region). If the focus guide is already displayed superimposed on the LV image at this time, the display is moved to the center of the right region. An example of the display at this time is shown in the guide 512 of Figure 5(f).

[0102] In S418, the system control unit 50 performs position calculation processing for the enlarged frame (right region) based on touch operation. This process will be described later using Figure 10(b).

[0103] Figures 10(a) and 10(b) show the processing steps shown in S416 and S418 of Figure 4. This is a control flowchart for calculating the position of the magnified frame when a user performs a touchdown operation on the touch panel 70a. It determines whether the touchdown position performed by the user is within a specific range in the left and right regions. If it is within the specific range, the display of the magnified frame is moved to the touchdown position; if it is outside the specific range, the display of the magnified frame is moved to the nearest specific range from the touchdown position. Here, the specific range is the range that includes the display range of the left image or the right image. XminL, XmaxL, YminL, and YmaxL are coordinates that indicate the threshold range in the left region where the center of the magnified frame can move. XminR, XmaxR, YminR, and YmaxR are coordinates that indicate the threshold range in the right region where the center of the magnified frame can move. Details of this coordinate system will be explained using Figure 13.

[0104] Using Figure 10(a), the calculation process when the touch position (the location of the touchdown performed in S412) is in the left region will be explained. Here, the left region refers to the region 13010L in Figure 13. Note that region 1301L is the same region as region 701L in Figure 7.

[0105] In S1001, the system control unit 50 determines whether the touch position is to the left of XminL (shown in Figure 13). If it is to the left of XminL, proceed to S1002; otherwise, proceed to S1003.

[0106] In S1002, the system control unit 50 sets the X coordinate of the center of the enlarged frame to X = X min L and stores it in the system memory 52.

[0107] In S1003, the system control unit 50 determines whether the touch position (the position of the touchdown performed in S412) is to the right of XmaxL. If it is to the right of XmaxL, the process proceeds to S1004; otherwise, it proceeds to S1005.

[0108] In S1004, the system control unit 50 sets the X coordinate of the center of the enlarged frame to X = XmaxL and stores it in the system memory 52.

[0109] In S1005, the system control unit 50 sets the X coordinate of the center of the enlarged frame as the touch position X=xt and stores it in the system memory 52.

[0110] In S1006, the system control unit 50 determines whether the touch position is above YminL. If it is above YminL, proceed to S1007; otherwise, proceed to S1008.

[0111] In S1007, the system control unit 50 sets the Y coordinate of the center of the enlarged frame as Y = YminL and stores it in the system memory 52.

[0112] In S1008, the system control unit 50 determines whether the touch position is below YmaxL. If it is below YmaxL, the process proceeds to S1009; otherwise, it proceeds to S1010.

[0113] In S1009, the system control unit 50 sets the Y coordinate of the center of the enlarged frame to Y=YmaxL and assigns it to the system memory 52.

[0114] In S1010, the system control unit 50 sets the Y coordinate of the center of the enlarged frame as the touch position Y=yt and stores it in the system memory 62.

[0115] In S1011, the system control unit 50 moves the enlarged frame according to the X and Y values ​​stored in the system memory 52. ​​At this time, the display position is moved so that the center of the enlarged frame in terms of its vertical and horizontal orientation matches the stored X and Y values.

[0116] Similarly, using Figure 10(b), we will explain the calculation process when the touch position (the location of the touchdown performed in S412) is in the right region. Here, the right region refers to the region 1310R in Figure 13. Note that region 1301R is the same region as region 701R in Figure 7.

[0117] In S1021, the system control unit 50 determines whether the touch position is to the left of XminR (shown in Figure 13). If it is to the left of XminR, proceed to S1022; otherwise, proceed to S1023.

[0118] In S1022, the system control unit 50 sets the X coordinate of the center of the enlarged frame as X = XminR and stores it in the system memory 52.

[0119] In S1023, the system control unit 50 determines whether the touch position is to the right of XmaxR (shown in Figure 13). If it is to the right of XmaxR, proceed to S1024; otherwise, proceed to S1025.

[0120] In S1024, the system control unit 50 sets the X coordinate of the center of the enlarged frame to X = XmaxR and stores it in the system memory 52.

[0121] In step S1025, the system control unit 50 sets the X coordinate of the center of the enlarged frame as the touch position X=xt and stores it in the system memory 52.

[0122] In S1026, the system control unit 50 determines whether the touch position is above YminR (shown in Figure 13). If it is above YminR, proceed to S1027; otherwise, proceed to S1028.

[0123] In S1027, the system control unit 50 sets the Y coordinate of the center of the enlarged frame as Y=YminR and stores it in the system memory 52.

[0124] In S1028, the system control unit 50 determines whether the touch position is below YmaxR (shown in Figure 13). If it is below YmaxR, the process proceeds to S1029; otherwise, it proceeds to S1030.

[0125] In S1029, the system control unit 50 sets the Y coordinate of the center of the enlarged frame to Y=YmaxR and assigns it to the system memory 52.

[0126] In S1030, the system control unit 50 sets the Y coordinate of the center of the enlarged frame as the touch position Y=yt and stores it in the system memory 52.

[0127] In S1031, the system control unit 50 moves the enlarged frame according to the X and Y values ​​stored in the system memory 52. ​​At this time, the display position is moved so that the center of the enlarged frame in terms of its vertical and horizontal orientation corresponds to the stored X and Y values.

[0128] By performing the control described above, the movement of the magnification frame in response to a user's touch operation will not occur in areas where the left or right image is not displayed (where no optical image is formed). In other words, even if the user's touch operation (touchdown) is performed in an area where no optical image is formed (area 1311 in Figure 13), the entire target range included in the magnification frame will not become an area where no optical image is formed. At this time, the magnification frame moves to include at least a part of the area where the left or right image is displayed (circular area 1301L or 1301R in Figure 13). As a result, when the user gives a magnification command, the display unit 28 displays a magnified image that includes at least a part of the left or right image. Since the display unit 28 does not display only the magnified portion that does not include the left or right image at all (area 1311 where no optical image is formed), it is possible to reduce user confusion and the feeling that the operation is poor.

[0129] Next, we will describe the control performed in response to touchdown when the lens attached to the digital camera 100 is not a twin-lens system (it is a single-lens system).

[0130] In S419, the system control unit 50 refers to the non-volatile memory 56 to determine whether the focus mode is set to autofocus mode (AF mode) or manual focus mode (MF mode). If it is MF mode, the process proceeds to S420; if it is AF mode, the process proceeds to S421. If the focus guide display setting is ON in MF mode, the focus guide is displayed to assist with focusing during MF operation.

[0131] In S420, the system control unit 50 moves the display of the focus guide to the touch position specified by the user in S412.

[0132] In S421, the system control unit 50 displays the AF frame (autofocus frame) indicating the focus position at the touch position specified by the user in S412. In S422, the system control unit 50 displays an enlarged frame at the touch position made in S412.

[0133] In other words, if the lens attached to the Digital Camera 100 is not a twin-lens reflex lens (but a single-lens reflex lens), the magnified frame and focus guide will be displayed at the user's touch position regardless of the focus mode setting. Furthermore, the magnified frame will be displayed in conjunction with the focus detection area of ​​the AF frame and focus guide. On the other hand, if a twin-lens reflex lens is attached, the display position of the magnified frame will not be linked to the focus detection area of ​​the focus guide.

[0134] In this embodiment, when a dual-lens system is installed, the display position of the focus guide is fixed to the center of the display area of ​​the left and right LV images. Therefore, if the display position of the focus guide and the display position of the magnification frame were linked, only the center of the LV image could be magnified, which would be inconvenient for the user. Also, with a dual-lens system, two LV images are displayed on the display unit 28, so each LV image becomes less than half the size of the LV image when a single-lens system is installed. Therefore, it is highly likely that the user will magnify the LV image more times than when a single-lens system is installed to examine it in more detail. For this reason, the magnification frame (i.e., the magnification range of the LV image) is not linked to the focus guide, so that the user can freely check the desired position.

[0135] Furthermore, even if the focus guide is not fixed in the center, when a twin-lens system is installed, the focus guide and the magnification frame should not be linked. Users can use the focus guide to make more precise focus adjustments, but there may be times when they want to magnify a different area than the point they want to focus on for more detailed inspection. For example, when users want to magnify and check the area within the range of the magic window (described later) or an area close to the circumference of the LV image in circular fisheye display, it is more user-friendly if the focus guide and the magnification frame are not linked.

[0136] Figure 6 is a control flowchart for the display control of the digital camera 100 when an operation is performed on the directional indicator operating member according to this embodiment.

[0137] The system control unit 50 of the digital camera 100 moves the position of items in the image in response to a direction indication received while displaying the twin-lens image and items such as the magnified frame. If the lens unit connected to the digital camera 100 is not a twin-lens lens (it is a conventional single-lens lens), the system control unit 50 moves the focus guide or AF frame and the magnified frame in response to the direction indication.

[0138] Furthermore, if the lens unit connected to the digital camera 100 is a twin-lens system, the system control unit 50 moves the magnification frame in accordance with the direction indication. The focus guide is not moved. In addition, when the magnification frame is displayed in the right area of ​​the display unit and is near the boundary, so that the magnification frame does not include both the left and right images, the system control unit 50 moves the display position of the magnification frame to the left area in response to an instruction to move to the left. When the magnification frame is displayed in the left area of ​​the display unit and is near the boundary, so that the magnification frame does not include both the left and right images, the system control unit 50 moves the display position of the magnification frame to the right area in response to an instruction to move to the right.

[0139] In S601, the system control unit 50 determines whether a direction instruction has been given using either the multi (MC) controller 82 or the directional pad 74 of the operation unit 70. If an instruction has been given, the process proceeds to S602; otherwise, it proceeds to S610.

[0140] In S602, similar to S411, the system control unit 50 obtains the type of lens installed via communication terminals 6 and 10 and determines whether or not it is a twin-lens system. If it is a twin-lens system, proceed to S607; otherwise, proceed to S603.

[0141] In S603, similar to S421, the system control unit 50 determines whether the current camera focus mode is AF mode or MF mode. If it is AF mode, the process proceeds to S604; if it is MF mode, the process proceeds to S605. In MF mode, if the display setting for the focus guide for MF focusing assistance is ON, the focus guide is displayed on one of the LV images displayed on the display unit 28. If the touch-down operation on the touch panel 70a described above has not been performed in S410, the focus guide is superimposed on the LV image in the right area.

[0142] In S604, the system control unit 50 moves the AF frame displayed on the display unit 28 in the direction indicated in S601.

[0143] In S605, the system control unit 50 moves the focus guide displayed on the display unit 28 in the direction indicated in S601. If the focus guide display setting is turned OFF by user settings, this step is skipped. In this case, the entire focus guide is moved, but only the focus detection area may be moved instead of the entire focus guide.

[0144] In S606, the system control unit 50 moves the enlarged display frame on the display unit 28 in conjunction with the position of the frame indicating the focus detection area of ​​the AF frame or focus guide that was moved in S604 or 605.

[0145] In S607, the system control unit 50 determines whether the currently displayed enlarged frame is displayed in the left region. If it is in the left region, the process proceeds to S608; otherwise, if it is displayed in the right region, the process proceeds to S609. At this time, it is determined whether the entire enlarged frame is displayed in the left region. In other words, it is determined whether the coordinates of the center position of the enlarged frame are at least in region 1310L.

[0146] In S608, the system control unit 50 performs position calculation processing for the enlarged frame (left region) based on the direction indication. This process will be described later using Figure 11(a).

[0147] In S609, the system control unit 50 performs position calculation processing for the enlarged frame (right region) based on the direction indication. This process will be described later using Figure 11(b).

[0148] Figures 11(a) and 11(b) show the processing steps shown in S608 and S609 of Figure 6. This is a control flowchart for calculating the position of the magnified frame to be displayed when the user gives a direction indication to the MC82 or the cross key 74. It is determined whether the display position of the already displayed magnified frame (the center coordinates of the magnified frame) is inside a specific range within the left and right regions. If it is inside the specific range, the magnified frame is moved one step in the direction indicated by the user's instruction. If the center coordinates of the magnified frame are on the boundary of a specific range within the left and right regions, the system is controlled to either not move the magnified frame or to jump the magnified frame from one optical image to the other (for example, from the left image to the right image), depending on the direction of the instruction. The specific range here is the same as the specific range described above in Figure 10, and is defined as the range of the rectangle that includes the display range of the left or right image. The coordinate system used for this determination will be explained using Figure 13.

[0149] Using Figure 11(a), we will explain the calculation process when the display area of ​​the enlarged frame that was displayed before the movement instruction (direction instruction) of the enlarged frame was the left region. Here, the left region refers to the region 1310L in Figure 13.

[0150] In S1101, the system control unit 50 determines whether the instruction is to turn right or not. The instruction in this step refers to the direction instruction given by the user to the MC82 or the directional pad 74 in S601 of Figure 6. If the instruction is to turn right, proceed to S1102; otherwise, proceed to S1104.

[0151] In S1102, the system control unit 50 determines whether the X coordinate of the center of the enlarged frame before the instruction is X = XmaxL. If so, proceed to S1103; otherwise, proceed to S1106.

[0152] In S1103, the system control unit 50 moves the enlarged frame so that the X coordinate of the center of the enlarged frame becomes X = X min R.

[0153] In S1104, the system control unit 50 determines whether the indicator is to turn left or not. If it is to turn left, proceed to S1105; otherwise, proceed to S1108.

[0154] In S1105, the system control unit 50 determines whether the X coordinate of the center of the enlarged frame before the instruction is X = X min L. If so, proceed to S1107; otherwise, proceed to S1106.

[0155] In S1106, the system control unit 50 moves the display position of the enlarged frame in the indicated direction. At this time, the amount of movement of the display position in response to one direction indication is assumed to be equivalent to one pixel of the display unit 28.

[0156] In S1107, the system control unit 50 does not move the magnification frame. Since it was determined to be Yes in S1105, it can be seen that the left side of the magnification frame displayed before the user gives a direction instruction is inscribed in the rectangle 1302L shown in Figure 13. If the magnification frame is moved one pixel to the left in response to a leftward instruction in this state, the magnification range (the range to be magnified) indicated by the magnification frame will include an area (region 1311) where an optical image has not been formed. If such control is performed, a large portion of the magnified display shown on the display unit 28 will be displayed, including parts other than the left or right image, which may make the user feel that it is not user-friendly. Therefore, the magnification frame is not moved outside the rectangle 1302L that is in contact with the circular region 1301L where the left image is displayed (the magnification range indicated by the magnification frame does not include anything outside the range of the rectangle 1302L).

[0157] In S1108, the system control unit 50 determines whether or not it is an upward indicator. If it is an upward indicator, it proceeds to S1109; otherwise, it proceeds to S1111.

[0158] In S1109, the system control unit 50 determines whether the Y coordinate of the center of the enlarged frame before the instruction is Y = Y min L. If so, proceed to S1110; otherwise, proceed to S1112.

[0159] In S1110, the system control unit 50 does not move the magnification frame. It can be seen that the top edge of the magnification frame displayed before the user gives a direction instruction is inscribed within the rectangle 1302L shown in Figure 13. If the magnification frame is moved upward by one pixel in response to an upward instruction in this state, the magnification range (the range of the object to be magnified) indicated by the magnification frame will include an area (region 1311) where an optical image has not been formed. As mentioned in S1107, this state may be inconvenient for the user, so the magnification range indicated by the magnification frame is made so as not to include anything outside the range of rectangle 1302L.

[0160] In S1111, the system control unit 50 determines whether the Y coordinate of the center of the enlarged frame before the instruction is Y = YmaxL. If so, proceed to S1113; otherwise, proceed to S1112.

[0161] In S1112, similar to S11106, the system control unit 50 moves the display position of the enlarged frame in the indicated direction. At this time, the amount of movement of the display position in response to one direction indication is assumed to be equivalent to one pixel of the display unit 28.

[0162] In S1113, the system control unit 50 does not move the magnification frame. Since it was determined to be Yes in S1111, it can be seen that the lower edge of the magnification frame displayed before the user gives a direction instruction is inscribed in the rectangle 1302L shown in Figure 13. If the magnification frame is moved downward by one pixel in response to a downward instruction in this state, the magnification range (the range to be magnified) indicated by the magnification frame will include an area (region 1311) where an optical image has not been formed. As mentioned in S1107 and S1110, the user may find this state inconvenient, so the magnification range indicated by the magnification frame does not include anything outside the range of rectangle 1302L.

[0163] Similarly, using Figure 11(b), we will explain the calculation process when the display area of ​​the enlarged frame that was shown before the movement instruction (direction instruction) of the enlarged frame was the right region. Here, the right region refers to the region 1310R in Figure 13.

[0164] In S1121, the system control unit 50 determines whether the instruction is to turn right or not. The instruction in this step refers to the direction instruction given by the user to the MC82 or the directional pad 74 in S601 of Figure 6. If the instruction is to turn right, proceed to S1122; otherwise, proceed to S1124.

[0165] In S1122, the system control unit 50 determines whether the X coordinate of the center of the enlarged frame before the instruction is X = XmaxR. If so, proceed to S1123; otherwise, proceed to S1126.

[0166] In S1123, the system control unit 50 does not move the magnification frame. As described in S1107, if the magnification frame is moved one pixel to the left according to the user's instruction, the magnification range indicated by the magnification frame (the range to be magnified) will include an area (region 1311) where no optical image is formed. In this state, the user may find it inconvenient to use. Therefore, the magnification frame is not moved outside the range of the rectangle 1302R that is tangent to the circular region 1301R where the right image is displayed (the magnification range indicated by the magnification frame does not include the area outside the range of the rectangle 1302R).

[0167] In S1124, the system control unit 50 determines whether the indicator is for a left turn. If it is for a left turn, proceed to S1125; otherwise, proceed to S1128.

[0168] In S1125, the system control unit 50 determines whether the X coordinate of the center of the enlarged frame before the instruction is X = X min R. If so, proceed to S1127; otherwise, proceed to S1126.

[0169] In S1126, the system control unit 50 moves the display position of the enlarged frame in the indicated direction. At this time, the amount of movement of the display position in response to one direction indication is assumed to be equivalent to one pixel of the display unit 28.

[0170] In S1127, the system control unit 50 moves the enlarged frame to X = X min L. At this time, it moves so that the X coordinate of the center of the enlarged frame becomes X = X min L.

[0171] In S1128, the system control unit 50 determines whether or not it is an upward indicator. If it is an upward indicator, it proceeds to S1129; otherwise, it proceeds to S1131.

[0172] In S1129, the system control unit 50 determines whether the Y coordinate of the center of the enlarged frame before the instruction is Y = Y min R. If so, proceed to S1130; otherwise, proceed to S1132.

[0173] In S1130, the system control unit 50 does not move the magnification frame. As described in S1110, if the magnification frame were moved upward by one pixel in response to user instructions, the magnification range indicated by the magnification frame (the range to be magnified) would include an area (region 1311) where no optical image is formed. Since this situation may be inconvenient for the user, the system control unit 50 ensures that the magnification range indicated by the magnification frame does not include areas outside the rectangle 1302R.

[0174] In S1131, the system control unit 50 determines whether the Y coordinate of the center of the enlarged frame before the instruction is Y=YmaxR. If so, proceed to S1133; otherwise, proceed to S1132.

[0175] In S1132, similar to S1126, the system control unit 50 moves the display position of the enlarged frame in the indicated direction. At this time, the amount of movement of the display position in response to one direction indication is assumed to be equivalent to one pixel of the display unit 28.

[0176] In S1133, the system control unit 50 does not move the magnification frame. As described in S1113, if the magnification frame were moved downward by one step in response to user instruction, the magnification range indicated by the magnification frame (the range of the object to be magnified) would include an area (region 1311) where no optical image is formed. In such a state, the user may find it inconvenient to use, so the magnification range indicated by the magnification frame is made so as not to include anything outside the range of the rectangle 1302R.

[0177] In this embodiment, the proportion of the left or right image included within the magnified range indicated by the magnification frame is controlled to be at least half (50%) of the size of the magnification frame, and the proportion of the shaded area (region 1311) included is 50% or less of the size of the magnification frame. However, it is sufficient that at least one pixel of the left or right image is included within the magnified range indicated by the magnification frame, and that the entire magnified range does not become a region where an optical image is not formed, i.e., the shaded area (region 1311). In other words, in this embodiment, the movable rectangular regions 1302L and 1302R of the magnification frame do not necessarily have to be circumscribing to the circular regions 1301L and 1301R, nor do they necessarily have to be rectangular, nor is it necessary to provide (calculate) such regions. For example, when a touch operation or movement operation is performed, the target range included within the current magnification frame is determined. If a touch operation is performed, in steps S418 and S416 of Figure 4 (Figure 10(a)(b)), it is determined whether the portion of the optical image of the left / right image is included within the target range contained within the magnified frame when the position of the touch operation is moved to the center of the magnified frame. If a movement operation (direction indication) is performed, in steps S608 and S609 of Figure 6 (Figure 11(a)(b)), it is determined whether the portion of the optical image of the left / right image is included within the target range contained within the magnified frame after movement. Alternatively, it may be determined whether the optical image of the left / right image is included by only one pixel within the target range contained within the magnified frame before movement. If even one pixel is included within the target range of the magnified frame after display and movement, the magnified frame can be moved; otherwise, the magnified frame is not moved.

[0178] The above describes the flow of item movement control, such as the enlarged frame and focus guide, when directional input is received via MC82 or the directional pad 74.

[0179] Next, we will describe the control when an operation is performed to return the display position of items such as the magnification frame and focus guide to their predetermined positions. The system control unit 50 displays items such as the magnification frame and focus guide in their predetermined positions in response to the reception of a predetermined operation (such as pressing the MC82 or pressing the SET button 75). If a binocular image is being displayed, the system control unit 50 displays the item in the center of either the right or left area where the item was displayed before the predetermined operation was received, in response to the reception of the predetermined operation. On the other hand, if a binocular image is not being displayed, the system control unit 50 displays the item in the center of the screen in response to the reception of the predetermined operation. This makes it possible to suitably control the display position of items depending on whether the displayed image is a binocular image or not.

[0180] In S610, the system control unit 50 determines whether the MC82 has been pressed down to the center (not moved up, down, left, or right) or whether the SET button 75 has been pressed. If a press or button is pressed, the process proceeds to S611; otherwise, the process proceeds to S801 in Figure 8. Pressing the MC82 or pressing the SET button 75 in this step can be described as a center movement instruction that moves (returns) the currently displayed magnified frame to the center of the LV image in the displayed area.

[0181] In S611, similar to S411, the system control unit 50 obtains the type of lens installed via communication terminals 6 and 10 and determines whether or not it is a twin-lens system. If it is a twin-lens system, proceed to S616; otherwise, proceed to S612.

[0182] In S612, similar to S421, the system control unit 50 determines whether the current camera focus mode is AF mode or MF mode. If it is AF mode, proceed to S613; if it is MF mode, proceed to S614.

[0183] In S613, the system control unit 50 moves the AF frame to the center of the LV image displayed on the display unit 28.

[0184] In S614, the system control unit 50 moves the focus guide frame to the center of the LV image displayed on the display unit 28.

[0185] In S615, the system control unit 50 moves the magnified frame to the center of the LV image displayed on the display unit 28 in conjunction with the position of the focus detection area of ​​the AF frame or focus guide.

[0186] In S616, the system control unit 50 determines whether the enlarged frame is displayed in the LV image of the right region. That is, it determines whether the enlarged frame is displayed within region 701R in Figure 7. If so, proceed to S618; otherwise, proceed to S617.

[0187] In S617, since the result in S616 was determined to be No, the system control unit 50 moves the display position of the enlarged frame to the center of the LV image displayed in the left region. That is, it moves to the center of the LV image displayed in region 701L in Figure 7.

[0188] In S618, since the result in S616 was determined to be Yes, the system control unit 50 moves the display position of the enlarged frame to the center of the LV image displayed in the right region. That is, it moves to the center of the LV image displayed in region 701R in Figure 7.

[0189] The above is a flow diagram illustrating the control process when an operation is performed to return the display position of items such as the magnified frame and focus guide to their predetermined positions.

[0190] Figure 13 is initiated when "Yes" is determined in S410 of Figure 4, that is, when it is determined that a dual-lens camera is installed. By calculating the centers and sizes of the two LV images (left image and right image) using lens information as shown in Figure 12, it is possible to determine the positions in which the two LV images are displayed in the single optical image captured by the imaging unit 22, which consists of the two LV images and a margin (shaded) area.

[0191] As explained in Figures 4 and 6, when moving the zoom frame, if it is possible to move the zoom frame into the blank areas when moving it across two LV images, the LV image may not be included at all in the enlarged display when zooming is performed. Alternatively, most of the enlarged display may be blank areas with only a small portion of the LV image included, which is likely to confuse the user. Therefore, a rectangular area (area 1302L, 1302R) is calculated where the circular display areas of the two LV images meet, and when the user instructs to move the zoom frame, it is made possible to move it only within the calculated rectangular area.

[0192] First, the system control unit 50 calculates the center positions of the rectangular regions 1302L and 1302R from the acquired lens information shown in Figure 12. The rectangular regions 1302L and 1302R are the circumscribing rectangles of the circular regions 1301L and 1301R. The circular regions 1301L and 1301R are the regions obtained by converting the circular optical image 1201L formed by the right eye optical system 301R and the circular optical image 1201R formed by the left eye optical system 301L onto the logical coordinate system 1300. The system control unit 50 acquires the distances 1203L and 1203R between the center position of the physical coordinate system 1200 of the optical image and the circular optical images 1201L and 1201R, respectively, from the mounted lenses via the communication terminals 6 and 10.

[0193] The center positions 1304L and 1304R of the rectangular regions 1302L and 1302R can be expressed by the following equations, using the center position 1303 of the logical coordinate system 1300 of the optical image, the transformation coefficient k determined from the ratio of the sizes of the physical coordinate system and the logical coordinate system, and the distances 1203L and 1203R.

[0194] 1304L = 1303 - (1203L ÷ k) 1304R = 1303 - (1203R ÷ k) Next, the system control unit 50 calculates the size of the rectangular regions 1302L and 1302R from the lens information. As shown in Figure 12, the radii 1204L and 1204R of the circular optical images 1201L and 1201R are obtained from the lens information. The size of the rectangular regions 1302L and 1302R can be expressed by the following formula using the transformation coefficient k, which is determined from the ratio of the sizes of the physical coordinate system and the logical coordinate system, and the radii 1204L and 1204R. Size of 1302L = (1204L ÷ k) × 2 The size of 1302R = (1204R ÷ k) × 2

[0195] The distances 1203L and 1203R, and the radii 1204L and 1204R are determined by the type of lens, and the transformation coefficient k for converting the physical coordinate system of the optical image to the logical coordinate system is determined by the type of image sensor. By performing calculations in this way, the center position and size of the rectangular regions 1302L and 1302R can be determined even if the type of lens or image sensor changes.

[0196] From the center position and size of the rectangular regions 1302L and 1302R, and the size of the enlarged frame, the coordinates for restricting the movement of the enlarged frame, as described above in Figures 10(a) and (b), can be determined.

[0197] From the rectangular region 1302L and the size of the enlarged frame, XminL, XmaxL, YminL, and YmaxL can be calculated.

[0198] XminL and XmaxL are coordinates in the horizontal (X-axis) direction. When the center coordinates of the magnified frame reach XminL and XmaxL, it indicates that it is touching the inside of the left and right sides of rectangle 1302L. When the center coordinates of the magnified frame reach XminL and XmaxL, and a movement instruction is given in the opposite direction from the center position 1304L relative to the magnified frame, the movement of the magnified frame is restricted or it is controlled to jump to the other LV image (rectangular region).

[0199] XminL is the center position of the enlargement frame when its left edge touches the left edge of rectangle 1302L. In other words, XminL is the value obtained by subtracting half the horizontal size (X-axis direction) of the enlargement frame from the X-coordinate of the left edge of rectangle 1302L, on the side of the center position 1304L inside rectangle 1302L.

[0200] XmaxL is the center position of the enlargement frame when its right edge touches the right edge of rectangle 1302L. In other words, XmaxL is the value obtained by subtracting half the size of the enlargement frame in the horizontal direction (X-axis direction) from the X coordinate of the right edge of rectangle 1302L, on the side of the center position 1304L inside rectangle 1302L.

[0201] YminL and YmaxL are coordinates in the vertical (Y-axis) direction. When the center coordinates of the enlarged frame reach YminL and YmaxL, it indicates that it is touching the inside of the top and bottom edges of rectangle 1302L. When the center coordinates of the enlarged frame reach YminL and YmaxL, if a movement instruction is given in the opposite direction from the center position 1304L relative to the enlarged frame, the movement of the enlarged frame is restricted.

[0202] YminL is the center position of the enlargement frame when its top edge touches the top edge of rectangle 1302L. In other words, YminL is the value obtained by subtracting half the size of the enlargement frame in the vertical direction (Y-axis direction) from the Y coordinate of the top edge of rectangle 1302L, on the side of the center position 1304L inside rectangle 1302L.

[0203] YmaxL is the center position of the enlargement frame when the bottom edge of the enlargement frame touches the bottom edge of rectangle 1302L. In other words, XmaxL is the value obtained by subtracting half the size of the vertical direction (Y-axis direction) of the enlargement frame from the Y coordinate of the bottom edge of rectangle 1302L, on the side of the center position 1304L inside rectangle 1302L.

[0204] Similarly, XminR, XmaxR, YminR, and YmaxR can be determined from the size of the rectangular region 1302R and the enlarged frame. Unlike the case of the rectangular region 1302L, it is mirror symmetric with respect to line 1305.

[0205] XminR and XmaxR, like XminL and XmaxL, are coordinates in the horizontal (X-axis) direction. When the center coordinates of the magnified frame reach XminR and XmaxR, it indicates that it is touching the inside of the left and right sides of rectangle 1302L. When the center coordinates of the magnified frame reach XminR and XmaxR, if a movement instruction is given in the opposite direction from the center position 1304L relative to the magnified frame, the movement of the magnified frame is restricted or it is controlled to jump to the other LV image (rectangular region).

[0206] XminR is the center position of the enlargement frame when the left edge of the enlargement frame touches the left edge of rectangle 1302R, and XmaxR is the center position of the enlargement frame when the right edge of the enlargement frame touches the right edge of rectangle 1302R. XminR is the value obtained by subtracting half the horizontal size of the enlargement frame from the X coordinate of the left edge of rectangle 1302R, which is inside rectangle 1302R and on the side of the center position 1304R. XmaxR is the value obtained by subtracting half the horizontal size of the enlargement frame from the X coordinate of the right edge of rectangle 1302R, which is inside rectangle 1302R and on the side of the center position 1304R.

[0207] YminR and YmaxR, like YminL and YmaxL, are coordinates in the vertical (Y-axis) direction. When the center coordinates of the enlarged frame reach YminR and YmaxR, it indicates that it is touching the inside of the top and bottom edges of rectangle 1302L. When the center coordinates of the enlarged frame reach YminR and YmaxR, if a movement instruction is given in the opposite direction from the center position 1304R relative to the enlarged frame, the movement of the enlarged frame is restricted.

[0208] YminR is the center position of the enlargement frame when the top edge of the enlargement frame touches the top edge of rectangle 1302R, and YmaxR is the center position of the enlargement frame when the bottom edge of the enlargement frame touches the bottom edge of rectangle 1302R. YminR is the value obtained by subtracting half the vertical size of the enlargement frame from the Y coordinate of the top edge of rectangle 1302R, which is inside rectangle 1302R and on the side of the center position 1304R. YmaxR is the value obtained by subtracting half the vertical size of the enlargement frame from the Y coordinate of the bottom edge of rectangle 1302R, which is inside rectangle 1302R and on the side of the center position 1304R.

[0209] XminL and XminR, XmaxL and XmaxR, YminL and YminR, and YmaxL and YmaxR change depending on the lens information acquired, namely the values ​​of the center positions 1304L and 1304R and the circular regions 1301L and 1301R. Even with a binocular lens, the lens information may differ depending on the manufacturing parameters. Therefore, XminL and XminR, XmaxL and XmaxR, YminL and YminR, and YmaxL and YmaxR do not necessarily coincide. In other words, the center position and size of the respective rectangular regions 1302L and 1302R are determined from the lens information of each left and right optical system, and the X and Y coordinates that control different positions in the magnified frame are calculated.

[0210] effect As mentioned above, when a position is specified by touch operation or direction (movement instruction) is given using MC82 or the directional keys 74, the center coordinates of the magnification frame indicating the magnified area are controlled so as not to move outside the rectangular area inscribed within the circularly displayed LV image. In other words, even if the user gives an instruction, the center coordinates of the magnification frame will not move to any area other than the rectangular areas 1302L and 1302R that are tangent to the circular areas 1301L and 1301R. This control prevents areas where the LV image is not displayed from being magnified when the user gives a magnification instruction, thus reducing the possibility of user confusion. Furthermore, the magnified display allows for a more detailed examination of every corner of the LV image.

[0211] Next, we will explain the zooming process and the process of changing the zoom level that are executed when the zoom button 78 is pressed.

[0212] Figure 8 is a control flowchart relating to the enlargement and capture operations of the LV image displayed on the display unit 28 according to this embodiment. An example of the display on the display unit 28 when the control flowchart in Figure 8 is executed will be explained using Figure 9. Details of the display example shown in Figure 9 will be explained after the explanation of the control flowchart in Figure 8.

[0213] In S801, the system control unit 50 determines whether or not the zoom button 78 has been pressed. If it has been pressed, the process proceeds to S802; otherwise, it proceeds to S819. In this embodiment, the user's zoom instruction is given by pressing the zoom button 78, but this is not the only option. For example, it is also possible to zoom in by pinching in on the touch panel 70a and cancel the zoom by pinching out.

[0214] In S802, similar to S411, the system control unit 50 obtains the type of lens installed via communication terminals 6 and 10 and determines whether or not it is a twin-lens system. If it is a twin-lens system, proceed to S803; otherwise, proceed to S812.

[0215] In S803, the system control unit 50 enlarges the LV image at the position where the enlargement frame is displayed by a factor of 6 and displays it on the display unit 28. The size of the enlargement frame displayed before the enlargement process is pre-set so that the entire enlarged image is displayed on the display unit 28 when the enlargement process is performed at a magnification of 6. The magnification of 6 in this case is the magnification based on the state in which the LV image displayed on the display unit 28 is not enlarged (1x). An example of the display in this case is shown in Figures 9(c) and 9(d).

[0216] In S804, the system control unit 50 determines whether or not an operation to move the magnified position (the area being magnified) has been performed. If an operation to move the magnified position has been performed, the process proceeds to S805; otherwise, it proceeds to S806. The operation to move the magnified position is assumed to be a directional instruction using the MC82 or the directional keys of the cross key 74, as explained using the control flowchart in Figure 6. This operation can also be performed by operating the touch panel 70a.

[0217] In S805, the system control unit 50 moves the magnification position only within the currently magnified area, which is either the left or right display area of ​​the two LV images displayed on the display unit 28, based on the magnification position movement operation in S804. In other words, even if a movement operation is performed to move the center of the magnification position closer to line 705 while the area to be magnified is touching line 705 in either the left or right area, the movement operation is invalidated and the magnification position is not moved.

[0218] As shown in S607~S614, when the LV image is at 1:1 magnification, the user can see the magnified frame displayed on the display unit 28, so even if the magnified frame can be moved from one area to another, the user will not lose sight of the magnified frame. On the other hand, if the magnified position moves across areas while the magnified image is being displayed, it becomes difficult to intuitively understand which position the magnified image is showing.

[0219] Let's consider a scenario where the user wants to examine the left edge of the LV image in the right region (where the left edge of the magnified position touches line 705) in more detail. If the user is viewing the magnified left edge of the LV image in the right region and unintentionally moves the magnification frame, the magnified area may become the right edge of the LV image in the left region, potentially causing user confusion. Therefore, when a magnified image is displayed, if the user instructs to move the magnification position, and the magnified position is at the edge of each LV image, the magnification position will not move beyond the left or right region.

[0220] In S806, the system control unit 50 determines whether a left / right switching operation has been performed. If a left / right switching operation has been performed, the process proceeds to S807; otherwise, it proceeds to S808. A left / right switching operation refers to the operation of switching from one image to the other of two images arranged side by side. Specifically, a left / right switching operation is the pressing of a button that has a left / right switching function (for example, the INFO button (not shown)).

[0221] In S807, the system control unit 50 moves the magnified position from the area where the magnified position was set before the left / right switching button was pressed to the other area, and performs magnified display. At this time, the magnified position is moved in such a way that the relative position of the magnified position in the area where the magnified position was set before the left / right switching button was pressed is maintained even after moving to the other area. For example, when the LV image in the right area is being magnified, the distance from the center of the LV image in the right area to the magnified position is calculated. Depending on the left / right switching operation performed by the user, the calculated distance from the center of the LV image to the magnified position is applied to the distance from the center of the LV image in the opposite area (in this case, the left area), and the magnified position is displayed on the display unit 28. With this control, if the user wants to check the same position on both the left and right LV images, they can switch between the left and right LV images with fewer steps and more easily, and it does not take time during shooting.

[0222] In S808, the system control unit 50 determines whether or not the zoom button 78 has been pressed. If it has been pressed, the process proceeds to S809; otherwise, it returns to S804.

[0223] In S809, the system control unit 50 refers to the system memory 52 and determines whether the magnification of the LV image displayed on the display unit 28 is 15 times. If it is 15 times, proceed to S810; otherwise, proceed to S811.

[0224] In S810, the system control unit 50 cancels the magnified state of the LV image and displays the unmagnified (1:1 magnification) LV image on the display unit 28. An example of this display is shown in Figures 9(a) and 9(b).

[0225] In S811, the system control unit 50 displays the LV image on the display unit 28 at a magnification of 15 times, at the display position of the magnified frame superimposed on the LV image. This magnification of 15 times is the magnification of the LV image in its unmagnified (1:1) state. An example of the display is shown in Figures 9(e) and 9(f).

[0226] In S812, since the result was determined to be No in S802, the system control unit 50 displays the LV image on the display unit 28 at a magnification of 6 times at the display position of the magnified frame superimposed on the LV image. This magnification of 6 times is the magnification based on the LV image in its unmagnified (1x) state.

[0227] In S813, similar to S804, the system control unit 50 determines whether or not an operation to move the magnified position has been performed. If an operation to move the magnified position has been performed, the process proceeds to S814; otherwise, it proceeds to S815.

[0228] In S814, the system control unit 50 moves the magnified position within the display area of ​​the LV image based on the operation to move the magnified position. In this step, since No was determined in S802, it is understood that the lens attached to the digital camera 010 is either a single-lens reflex lens or no lens is attached. In other words, since only one LV image is displayed on the display unit 28, the magnified position can be moved without considering the left and right areas as in S805.

[0229] In S815, similar to S808, the system control unit 50 determines whether or not the zoom button 78 has been pressed. If it has been pressed, the process proceeds to S816; otherwise, it returns to S813.

[0230] In S816, the system control unit 50 refers to the system memory 52 and determines whether the current magnification of the LV image is 15 times or not. If it is 15 times, proceed to S817; otherwise, proceed to S818.

[0231] In S817, similar to S810, the system control unit 50 cancels the magnified state of the LV image and displays the unmagnified (1:1) LV image on the display unit 28.

[0232] In S818, the system control unit 50 displays the LV image on the display unit 28 at a magnification of 15 times, at the display position of the magnified frame superimposed on the LV image. This magnification of 15 times is the magnification of the LV image in its unmagnified (1:1) state as the reference.

[0233] In S819, the system control unit 50 determines whether the first shutter switch 62 is turned on or not. If the first shutter switch 62 is turned on, the process proceeds to S820; otherwise, it proceeds to S830. The first shutter switch 62 being turned on means that the shutter button 61 is half-pressed, as mentioned above. In other words, it can be assumed that the user is about to take a picture.

[0234] In S820, similar to S411, the system control unit 50 obtains the type of lens installed via communication terminals 6 and 10 and determines whether or not it is a twin-lens system. If it is a twin-lens system, proceed to S823; otherwise, proceed to S821.

[0235] In step S821, the system control unit 50 determines whether the focus mode is set to AF mode. If it is set to AF mode, the process proceeds to S822; otherwise (if it is set to MF mode), the process proceeds to S823. Switching between AF mode and MF mode can be done via the settings menu screen or a switch located on the outside of the lens unit 150.

[0236] In S822, the system control unit 50 performs AF processing based on the AF frame position.

[0237] In S823, the system control unit 50 performs other shooting preparation processing such as AE and AWB.

[0238] In S824, the system control unit 50 determines whether the second shutter switch 64 is turned on or not. If the second shutter switch 64 is turned on, that is, if the shutter button 61 is fully pressed, the process proceeds to S825; otherwise, the process proceeds to S829.

[0239] In S825, similar to S411, the system control unit 50 obtains the type of lens installed via the communication terminals 6 and 10 and determines whether or not it is a twin-lens system. If it is a twin-lens system, proceed to S826; otherwise, proceed to S827.

[0240] In S826, the system control unit 50 performs a series of shooting processes, including recording the image captured by the twin-lens system and twin-lens information as an image file onto the recording medium 200.

[0241] In S827, the system control unit 50 performs a series of shooting processes, including recording the captured image and normal (single-lens) lens information as an image file onto the recording medium 200.

[0242] In S828, the system control unit 50 determines whether the first shutter switch 62 remains ON. If the first shutter switch 62 remains ON, the process returns to S824; otherwise, it proceeds to S830.

[0243] In S829, the system control unit 50 determines whether it has detected any operations other than those mentioned above. If it has detected any other operations, it proceeds to S830; otherwise, it proceeds to S831. Specifically, it determines whether a button such as the menu button 81 or the play button 79 has been pressed.

[0244] In S830, the system control unit 50 starts executing other processes corresponding to the detected other operations. If the menu button 81 is pressed, the settings menu screen is displayed, and if the play button 79 is pressed, the image stored on the recording medium 200 is displayed on the display unit 28.

[0245] In S831, the system control unit 50 determines whether the shooting standby state has ended. For example, if the shooting standby state has ended due to power off or other reasons, the control flowchart in Figure 8 is terminated; otherwise, the process returns to S401 in Figure 4.

[0246] Using Figures 9(a) to 9(f), we will explain the LV image displayed on the display unit 28 when the user requests an enlargement command.

[0247] Figure 9(a) shows an example of a display where the magnification is 1:1 (i.e., no magnification) and the magnification frame is superimposed on the left region, i.e., the LV image (LV900R) captured via the right-eye optical system 301R. The display unit 28 displays the LV900R captured via the right-eye optical system 301R and the LV900L captured via the left-eye optical system 301L. As mentioned above, the LV900R displayed in the left region was captured by the right-eye optical system 301R, and the LV900L displayed in the right region was captured by the left-eye optical system 301L. Display items 901 and 902 are displayed to inform the user that the image is reversed left and right. Display item 901 displays "R" to indicate right, and display item 902 displays "L" to indicate left. When the magnification button 78 is pressed once in the state shown in Figure 9(a), the state changes to the state shown in Figure 9(c). When the left / right switching button (not shown) is pressed once, the magnified frame superimposed on LV900R moves to the same relative position on LV900L, and the state changes to the state shown in Figure 9(b).

[0248] Figure 9(b) shows an example of the display when the magnification is 1:1 (1:1, i.e., no magnification) and the magnification frame is superimposed on the right region, i.e., the LV image (LV900L) captured by the left eye optical system 301L. In Figure 9(b), the magnification frame 904 is superimposed on LV900L. In the state shown in Figure 9(b), pressing the magnification button 78 once transitions to the state shown in Figure 9(d), and pressing the left / right switching button once moves the magnification frame superimposed on LV900L to the same relative position on LV900R, transitioning to the state shown in Figure 9(a).

[0249] Figure 9(c) shows an example of the display when the LV900R is magnified at a magnification of 6x. The magnified image displayed on the display unit 28 has a display item 901 superimposed on the magnified LV image, LV910R, indicating which optical system captured the image.

[0250] Figure 9(d) shows an example of the display when LV900L is magnified at a magnification of 6x. The magnified image displayed on the display unit 28 has a display item 902 superimposed on the magnified LV image, LV910L, indicating which optical system captured the image.

[0251] Figure 9(e) shows an example of the display when the LV900R is magnified at a magnification of 1:5. The magnified image displayed on the display unit 28 is superimposed on the LV920R which is magnified 15 times, and a display item 901 indicating the LV image captured by the right eye optical system 301R is displayed.

[0252] Figure 9(f) shows an example of the display when the LV900L is magnified at a magnification of 1:5. The magnified image displayed on the display unit 29 is superimposed on the LV920L which is magnified 15 times, and a display item 902 indicating the LV image captured by the left eye optical system 301L is displayed.

[0253] Specifically, Figures 9(a) and 9(b) show the LV images displayed on the display unit 28 when the magnification is 1x (i.e., no magnification), Figures 9(c) and 9(d) show the LV images displayed when the magnification is 6x, and Figures 9(e) and 9(f) show the LV images displayed when the magnification is 15x.

[0254] Each time the zoom button 78 is pressed, the magnification of the LV image displayed on the display unit 28 changes in the following order: 1x → 6x → 1x5 → 1x. Also, if the left / right switching button is pressed, and the magnification is 1x, the display area is switched to overlay the magnified frame onto the other LV image. That is, if the magnified frame is overlaid on LV900R displayed in the left area, the display area is switched to overlay it onto LV900L displayed in the right area. If the magnification is anything other than 1x, the magnified position is switched to the area of ​​the other LV image that is in the same relative position. In this case, the magnification does not change.

[0255] Although it was explained that operations such as switching display modes, moving the magnified frame and position, and magnification are performed using dedicated operation buttons, it is also possible to assign these functions to buttons that can be assigned to various arbitrary functions.

[0256] As described above, when using a lens with two optical systems (two-lens system) and taking images, the magnification frame indicating the magnified position of the LV image should be set to include the area where the optical image is formed (but not only the area where the optical image is not formed). Two LV images captured through the two optical systems are displayed side by side on the display unit, and the magnification frame is displayed and moved so as to include at least a portion of the area where the optical image is formed (either the left or right image), rather than only the area where the optical image is not formed. This prevents the display from showing only the area where the optical image is not formed when the user gives a magnification command, thereby reducing user confusion.

[0257] Furthermore, when the two LV images are displayed at their original size (not enlarged), the system controls the center of the enlargement frame so that it does not move outside the rectangular area inscribed within the circularly displayed LV image when position is specified by touch operation or direction (movement instruction) is given using MC82 or the cross key 74. In other words, even if the user gives an instruction, the center coordinates of the enlargement frame do not move to the area 1311, which represents the blank (shaded) area, excluding the rectangular areas 1302L and 1302R that are tangent to the circular areas 1301L and 1301R from the area 1310 of the display unit 28. This control prevents areas where the LV image is not displayed from being enlarged when the user gives an enlargement instruction, thus reducing the possibility of user confusion. Moreover, the enlarged display allows for a more detailed examination of every corner of the LV image.

[0258] On the other hand, if either of the two LV images is being magnified, even if a command to move the magnification position is given when the user has reached the boundary edge of the currently magnified image (the right edge if the left region is being magnified, or the left edge if the right region is being magnified), the image will not move in the direction of the command. In other words, the system will not move the magnification position across the region (the boundary of the LV image) when the user has difficulty visually confirming the current magnification position. This reduces unintended crossing of LV image boundaries and minimizes user confusion.

[0259] Furthermore, when a lens with two optical systems (a twin-lens system) is attached, the focus guide and the magnification frame are not linked (unlinked). This is because, as with a twin-lens system, when two LV images are displayed and it is difficult to examine the entire LV image in detail without magnification, it is highly likely that the user will want to magnify the LV image regardless of the focus position to examine it in more detail. Therefore, instead of only allowing magnification of the focus guide, which has a frame indicating the focus detection area, the focus guide and magnification frame are unlinked so that the user can magnify the position of their choice. On the other hand, when a normal lens (a single-lens system) is attached, the focus guide / AF frame and the magnification frame are mainly linked, and when the focus guide / AF frame is moved, the position of the magnification frame also moves. When a single-lens system is attached, only one LV image is displayed, and since the LV image is displayed larger on the display than when two LV images are displayed, it is easier to examine the LV image in more detail at 1:1 magnification than when a twin-lens system is attached.

[0260] Furthermore, the various controls described above, which are performed by the system control unit 50, may be performed by a single piece of hardware, or multiple pieces of hardware (for example, multiple processors or circuits) may share the processing to control the entire device.

[0261] Furthermore, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is possible to combine each embodiment as appropriate.

[0262] Furthermore, although the above-described embodiment explained the case where the present invention is applied to a digital camera 100, it is not limited to this example and can be applied to any imaging control device that can display images captured by two optical systems. In other words, the present invention can be applied to personal computers and PDAs, mobile phone terminals and portable image viewers, printer devices equipped with displays, digital photo frames, music players, game consoles, e-book readers, home appliances, in-vehicle devices, medical devices, and the like.

[0263] Furthermore, the present invention is applicable not only to the imaging device itself, but also to a control device that communicates with the imaging device (including network cameras) via wired or wireless communication and remotely controls the imaging device. Examples of devices that remotely control the imaging device include smartphones, tablet PCs, and desktop PCs. The imaging device can be remotely controlled by notifying the imaging device of commands to perform various operations and settings based on operations and processes performed on the control device side. In addition, live view images captured by the imaging device may be received via wired or wireless communication and displayed on the control device side.

[0264] (Other embodiments) The present invention can also be realized by performing the following process: supplying software (programs) that realize the functions of the embodiments described above to a system or device via a network or various storage media, and having the computer (or CPU, MPU, etc.) of that system or device read and execute the program code. In this case, the program and the storage medium storing the program constitute the present invention.

Claims

1. An acquisition means for acquiring a third image in which a first image captured through a first optical system and a second image captured through a second optical system and having parallax with respect to the first image are arranged side by side, It includes a setting means for setting the target range of the third image to which a predetermined process is applied, in response to user operation, The setting means sets the target range such that the target range includes at least a part of the first image or the second image. The third image is displayed such that the first image is on the left and the second image is on the right. In the first image, there is a first region where the item indicating the target range can be moved, and in the second image, there is a second region where the item can be moved. If the position of the item is specified when the rightmost edge of the item reaches the rightmost edge of the first region, the position of the item is changed to the second region. An electronic device characterized in that, if the position of the item is specified when the left edge of the item reaches the left edge of the second area, the device controls the position of the item to be changed to the first area.

2. The electronic device according to claim 1, characterized in that the setting means controls the position of the item not to change when the position of the item is specified while the third image is enlarged and the end of the item has reached either the right end of the first region or the left end of the second region.

3. The electronic device according to claim 1 or 2, characterized in that the first region and the second region are rectangular regions.

4. The display unit further includes a touch detection means capable of detecting touch operations on the display surface, The setting means is, If the touch position on the touch detection means is outside the first region or within a predetermined region from the edge of the first region, the target range is set to include both the first region and the edge of the first region. The electronic device according to any one of claims 1 to 3, characterized in that, if the touch position on the touch detection means is outside the second region or within a predetermined region from the edge of the second region, the target range is controlled to be set to include both the second region and the edge of the second region.

5. The electronic device according to claim 4, characterized in that the predetermined area is half the horizontal size of the item.

6. A program for causing a computer to function as one of the means of an electronic device described in any one of claims 1 to 5.

7. A computer-readable recording medium storing a program for causing a computer to function as one of the means of an electronic device described in any one of claims 1 to 5.

Images