Display device, method for controlling display device, and program

Abstract

To provide a display device for presenting a photographing result of a VR image by a method matched with a display system of the VR image in photographing.SOLUTION: A determination unit configured to determine a first display method for displaying a live view image on a display unit from among a plurality of display methods including a display method in which distortion is suppressed by causing a user to select the first display method; And a control unit configured to, in a case where an image capturing instruction is issued while the live view image is displayed on the display unit in the first display method, perform control to display, on the display unit in the first display method, a captured image recorded in a recording unit in response to the image capturing instruction.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a display device, a control method thereof, a program, and a recording medium, and more particularly to display control of VR content. [Background technology]

[0002] A technology that displays virtual reality (hereafter referred to as VR) images with a three-dimensional feel is becoming widespread, using two optical systems to capture wide-angle images with parallax, then mapping and displaying them on a virtual sphere. A twin-lens VR camera for capturing VR images has two optical systems facing the same direction, allowing two images with parallax to be captured with a single shot. Some twin-lens VR cameras can capture a wide range of more than 180 degrees up, down, left, and right with each optical system (a hemisphere, 90 degrees in all directions from the center of the image).

[0003] There are various display methods for displaying VR images, including "non-VR display," which displays the left and right eye images acquired by each optical system of a two-lens VR camera side by side, and "VR display," which maps and converts the left and right eye images onto a virtual sphere to display a three-dimensional image. In addition, each display method may allow you to specify which area of ​​the entire image to display. Technology has been disclosed that allows you to change between these multiple display methods depending on various conditions, such as user operation and the built-in functions of the device.

[0004] For example, Patent Document 1 discloses a technology in which a first image, which is an image of a predetermined range of a VR image, and a second image, which is narrower in range than the first image, are normally displayed. Then, when a user issues an instruction to prepare for shooting, the technology controls the display to switch to the other of the first image and the second image, or to both images.

[0005] For example, Patent Document 2 discloses a technology for outputting image data that can be displayed on a display device based on display device information including information on whether the display device has the function to display VR images and VR identification information including information on whether the image data contains VR images. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2019-012881 [Patent Document 2] WO17 / 145721 Summary of the Invention [Problem to be solved by the invention]

[0007] However, with the conventional technology disclosed in the aforementioned patent documents, the display method of the live view image when capturing a VR image may not necessarily match the display method when the captured image recorded in the recording unit is subsequently played back to confirm the capture results. For example, if VR display is selected as the display method for the live view image at the time of capture and non-VR display is selected as the display method for playing back the recorded image after capture, the photographer will view the subject in VR image until just before capture, but the recorded image after capture will not be displayed as VR image. In other words, even though the photographer viewed the subject in VR display until just before capture, the subject cannot be immediately confirmed as VR display in the recorded image of the capture result. In addition, additional operations such as instructing a change in display method are required.

[0008] Therefore, an object of the present invention is to provide a display device that presents a recorded image of the shooting result in a manner that matches the live view image display method when shooting a VR image. [Means for solving the problem]

[0009] One aspect of the present invention is a display device characterized by having: a determination means for determining, by having a user select, a first display method for displaying a live view image on a display unit from among a plurality of display methods including a display method in which distortion is suppressed; and a control means for controlling the live view image to be displayed on the display unit in the first display method, and, when an image capture instruction is given while the live view image is being displayed on the display unit in the first display method, controlling the display unit to display an image recorded in a recording unit in response to the image capture instruction in the first display method. [Effects of the Invention]

[0010] According to the present invention, the user can check the results of capturing a VR image in a display format that matches the display method of the VR image when it was captured. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of the external configuration of a camera. [Figure 2] FIG. 2 is a diagram illustrating an example of the internal configuration of a camera. [Figure 3] FIG. 2 is a schematic diagram illustrating an example of the configuration of a lens unit. [Figure 4] FIG. 10 is a diagram showing the transition of the display method of a VR image during shooting and playback. [Figure 5] FIG. 10 is a flowchart showing a photographing process of the camera. [Figure 6] FIG. 10 is a flowchart showing a playback process of the camera. [Figure 7] FIG. 10 is a flowchart showing a process of changing the display mode of the camera. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0013] (First embodiment) DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the present embodiment, the display device will be described as a digital camera.

[0014] 1 is a diagram showing an example of the external configuration of a digital camera 100 (hereinafter referred to as camera 100). Fig. 1(a) is a perspective view of the camera 100 seen from the front, and Fig. 1(b) is a perspective view of the camera 100 seen from the back.

[0015] The camera 100 has, on its top surface, a shutter button 101, a power switch 102, a mode switch 103, a main electronic dial 104, a sub electronic dial 105, a video button 106, and an extra-viewfinder display 107. The shutter button 101 is an operation unit for preparing for shooting or issuing shooting instructions (image capture instructions). The power switch 102 is an operation unit for switching the power of the camera 100 on and off. The mode switch 103 is an operation unit for switching between various modes. The main electronic dial 104 is a rotary operation unit for changing setting values ​​such as shutter speed and aperture. The sub electronic dial 105 is a rotary operation unit for moving the selection frame (cursor), scrolling through images, etc. The video button 106 is an operation unit for issuing instructions to start and stop video shooting (recording). The extra-viewfinder display 107 displays various setting values ​​such as shutter speed and aperture.

[0016] The camera 100 also has a display unit 108, a touch panel 109, directional keys 110, a SET button 111, an AE lock button 112, a magnify button 113, a playback button 114, a menu button 115, an eyepiece unit 116, an eyepiece detection unit 118, and a touch bar 119. The display unit 108 displays images and various information. The touch panel 109 is an operation unit that detects touch operations on the display surface (touch operation surface) of the display unit 108. The directional keys 110 are an operation unit consisting of keys (four-way keys) that can be pressed up, down, left, and right. Operations can be performed depending on the position of the directional keys 110 that is pressed. The SET button 111 is an operation unit that is pressed mainly to confirm a selection item. The AE lock button 112 is an operation unit that is pressed to fix the exposure state in a shooting standby state. The magnify button 113 is an operation unit that switches the magnify mode on and off in the live view display (LV display) of the shooting mode. When the enlargement mode is on, the live view image (LV image) is enlarged or reduced by operating the main electronic dial 104. Here, live view refers to a function that displays the image captured by the imaging sensor on an EVF (Electronic View Finder) or monitor, and the live view image refers to the image displayed when this function is being used. The enlargement button 113 is used to enlarge the playback image or increase the magnification in playback mode. The playback button 114 is an operation unit for switching between shooting mode and playback mode. In shooting mode, pressing the playback button 114 switches to playback mode, and the most recent image recorded on a recording medium 227 (described later) can be displayed on the display unit 108.

[0017] The menu button 115 is an operation unit that is pressed when a menu screen that allows various settings to be displayed on the display unit 108. The user can intuitively make various settings using the menu screen displayed on the display unit 108, the direction keys 110, and the SET button 111. The eyepiece unit 116 is a part for placing an eye on the eyepiece finder (peek-in type finder) 117. The user can view an image displayed on an internal EVF 217 (described later) through the eyepiece unit 116. The eyepiece detection unit 118 is a sensor that detects whether the user has placed an eye on the eyepiece unit 116.

[0018] The touch bar 119 is a line-shaped touch operation unit (line touch sensor) capable of receiving touch operations. The touch bar 119 is positioned so that it can be touched by the thumb of the right hand when the grip unit 120 is held in the right hand (held with the little finger, ring finger, and middle finger of the right hand) so that the shutter button 101 can be pressed with the index finger of the right hand. That is, the touch bar 119 can be operated while the eyepiece unit 116 is placed near the viewfinder 117 and the user is in a position (shooting posture) ready to press the shutter button 101 at any time. The touch bar 119 can receive tap operations (operations in which the user touches and then releases the touch bar without moving within a predetermined period of time), slide operations to the left or right (operations in which the user touches and then moves the touched position while keeping the touch), and the like. The touch bar 119 is an operation unit different from the touch panel 109 and does not have a display function. The touch bar 119 of this embodiment is a multi-function bar and functions, for example, as an M-Fn bar.

[0019] The camera 100 also includes a grip section 120, a thumb rest section 121, a terminal cover 122, a lid 123, a communication terminal 124, and the like. The grip section 120 is a holding section shaped to be easily gripped with the user's right hand when holding the camera 100. The shutter button 101 and main electronic dial 104 are positioned so that they can be operated with the index finger of the right hand when the user holds the camera 100 by gripping the grip section 120 with the little finger, ring finger, and middle finger of the right hand. Similarly, the sub electronic dial 105 and touch bar 119 are positioned so that they can be operated with the thumb of the right hand. The thumb rest section 121 (thumb standby position) is a grip section provided on the rear side of the camera 100, in a position where it is easy to place the thumb of the right hand holding the grip section 120 when none of the operation sections are being operated. The thumb rest section 121 is made of a rubber member or the like to enhance holding strength (grip feeling). The terminal cover 122 protects connectors such as connection cables that connect the camera 100 to external devices. The lid 123 protects the recording medium 227 and the slot by closing the slot for storing the recording medium 227. The communication terminal 124 is a terminal for communicating with the detachable lens unit 200 side of the camera 100, which will be described later.

[0020] Fig. 2 is a diagram showing an example of the internal configuration of the camera 100. Note that the same components as those in Fig. 1 are given the same reference numerals and descriptions thereof will be omitted where appropriate. A lens unit 200 is attached to the camera 100.

[0021] First, a description will be given of the lens unit 200. The lens unit 200 is a type of interchangeable lens that can be attached to and detached from the camera 100. The lens unit 200 is a single lens, and is an example of a normal lens.

[0022] The lens unit 200 includes an aperture 201, a lens 202, an aperture drive circuit 203, an AF (autofocus) drive circuit 204, a lens system control circuit 205, a communication terminal 206, and the like.

[0023] The aperture 201 is configured so that its aperture diameter is adjustable. The lens 202 is composed of multiple lenses. The aperture drive circuit 203 adjusts the amount of light by controlling the aperture diameter of the aperture 201. The AF drive circuit 204 drives the lens 202 to adjust the focus. The lens system control circuit 205 controls the aperture drive circuit 203, the AF drive circuit 204, etc. based on instructions from a system control unit 50 (described later). The lens system control circuit 205 controls the aperture 201 via the aperture drive circuit 203, and adjusts the focus by displacing the position of the lens 202 via the AF drive circuit 204. The lens system control circuit 205 can communicate with the camera 100. Specifically, communication is performed via a communication terminal 206 of the lens unit 200 and a communication terminal 124 of the camera 100. The communication terminal 206 is a terminal through which the lens unit 200 communicates with the camera 100.

[0024] Next, a description will be given of the camera 100. The camera 100 has a shutter 210, an imaging unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, an EVF 217, a display unit 108, and a system control unit 50.

[0025] The shutter 210 is a focal plane shutter that can freely control the exposure time of the imaging unit 211 based on instructions from the system control unit 50. The imaging unit 211 is an imaging element (image sensor) composed of a CCD, CMOS, or other element that converts an optical image into an electrical signal. The imaging unit 211 may also have an imaging surface phase difference sensor that outputs defocus amount information to the system control unit 50. The A / D converter 212 converts the analog signal output from the imaging unit 211 into a digital signal. The image processing unit 214 performs predetermined processing (pixel interpolation, resizing such as reduction, color conversion, etc.) on data from the A / D converter 212 or data from the memory control unit 213. The image processing unit 214 also performs predetermined arithmetic processing using captured image data, and the system control unit 50 controls exposure and distance measurement based on the obtained arithmetic results. This processing allows for TTL (through-the-lens) AF processing, AE (auto-exposure) processing, EF (pre-flash) processing, etc. Furthermore, the image processing unit 214 performs predetermined calculation processing using the captured image data, and performs TTL type AWB (auto white balance) processing based on the obtained calculation results.

[0026] The image data from the A / D converter 212 is written to the memory 215 via the image processing unit 214 and the memory control unit 213. Alternatively, the image data from the A / D converter 212 is written to the memory 215 via the memory control unit 213 without going through the image processing unit 214. The memory 215 stores image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212, as well as image data to be displayed on the display unit 108 and the EVF 217. The memory 215 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio. The memory 215 also serves as a memory for displaying images (video memory).

[0027] The D / A converter 216 converts image display data stored in the memory 215 into an analog signal and supplies it to the display unit 108 or the EVF 217. Therefore, the display image data written to the memory 215 is displayed as a live view image on the display unit 108 or the EVF 217 via the D / A converter 216. The display unit 108 or the EVF 217 performs display in accordance with the analog signal from the D / A converter 216. The display unit 108 or the EVF 217 is, for example, an LCD or organic EL display. The digital signal that has been A / D converted by the A / D converter 212 and stored in the memory 215 is converted into an analog signal by the D / A converter 216 and then sequentially transferred to and displayed on the display unit 108 or the EVF 217, thereby performing live view display.

[0028] The system control unit 50 is a control unit including at least one processor and / or at least one circuit. That is, the system control unit 50 may be a processor, a circuit, or a combination of a processor and a circuit. The system control unit 50 controls the entire camera 100. The system control unit 50 executes programs recorded in the nonvolatile memory 219 to realize each process in the flowcharts described below. The system control unit 50 also performs display control by controlling the memory 215, D / A converter 216, display unit 108, EVF 217, etc.

[0029] The camera 100 also includes a system memory 218 , a nonvolatile memory 219 , a system timer 220 , a communication unit 221 , an attitude detection unit 222 , and an eye proximity detection unit 118 .

[0030] The system memory 218 may be, for example, a RAM. Constants and variables for the operation of the system control unit 50, programs read from the nonvolatile memory 219, and the like are loaded into the system memory 218. The nonvolatile memory 219 is an electrically erasable and recordable memory, for example, an EEPROM. The nonvolatile memory 219 stores constants and programs for the operation of the system control unit 50. The programs referred to here are programs for executing flowcharts, which will be described later. The system timer 220 is a timing unit that measures the time used for various controls and the time of a built-in clock. The communication unit 221 transmits and receives video signals and audio signals to and from external devices connected wirelessly or via a wired cable. The communication unit 221 can also connect to a wireless LAN (Local Area Network) or the Internet. The communication unit 221 can also communicate with external devices via Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 221 can transmit images (including live images) captured by the imaging unit 211 and images recorded on a recording medium 227, and can receive image data and various other information from external devices. The orientation detection unit 222 detects the orientation of the camera 100 with respect to the direction of gravity. Based on the orientation detected by the orientation detection unit 222, it is possible to determine whether an image captured by the imaging unit 211 was captured with the camera 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the orientation detected by the orientation detection unit 222 to the image file of the image captured by the imaging unit 211, or rotate and record the image. The orientation detection unit 222 can use, for example, an acceleration sensor or a gyro sensor. The orientation detection unit 222 can also be used to detect movement of the camera 100 (panning, tilting, lifting, whether or not the camera is stationary, etc.).

[0031] The eyepiece detection unit 118 can detect the approach of an object to the eyepiece 116 of the eyepiece finder 117, which incorporates the EVF 217. The eyepiece detection unit 118 can be, for example, an infrared proximity sensor. When an object approaches, infrared light is emitted from a light-emitting unit of the eyepiece detection unit 118, reflected by the object, and received by a light-receiving unit of the infrared proximity sensor. The amount of received infrared light can be used to determine the distance from the eyepiece 116 to the object. In this way, the eyepiece detection unit 118 performs eyepiece detection, which detects the proximity of the object to the eyepiece 116. The eyepiece detection unit 118 is an eyepiece detection sensor that detects the approach (approach) and departure (away from) of an eye (object) from the eyepiece 116 of the eyepiece finder 117. When an object is detected approaching within a predetermined distance from the eyepiece 116 from a non-eyepiece state (not approaching state), it is detected that the eye has been placed in proximity. On the other hand, when an object whose proximity has been detected moves away from the eye-closed state (approaching state) by a distance greater than a predetermined distance, it is detected that the eye has moved away. The threshold for detecting eye-closedness and the threshold for detecting eye-away may be different, for example, by providing hysteresis. Furthermore, after detecting eye-closedness, the eye-closed state is maintained until eye-away is detected. After detecting eye-away, the non-eye-closed state is maintained until eye-closedness is detected. The system control unit 50 switches the display unit 108 and the EVF 217 between display (display state) and non-display (non-display state) depending on the state detected by the eye-closedness detection unit 118. Specifically, when at least in a shooting standby state and the display destination switching setting is automatic switching, the display unit 108 is set as the display destination and the display is turned on, and the EVF 217 is hidden, when the eye is not placed near the object. Furthermore, when the eye is placed near the object, the EVF 217 is set as the display destination and the display is turned on, and the display unit 108 is hidden. The eye proximity detector 118 is not limited to an infrared proximity sensor, and any other sensor may be used as long as it can detect a state that can be considered as eye proximity.

[0032] The camera 100 also has an outside-viewfinder display unit 107, an outside-viewfinder display drive circuit 223, a power supply control unit 224, a power supply unit 225, a recording medium I / F 226, an operation unit 228, and the like.

[0033] The viewfinder display unit 107 displays various settings of the camera 100, such as shutter speed and aperture, via an external viewfinder display drive circuit 223. The power supply control unit 224 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between powered blocks, and other components, and detects whether a battery is installed, the battery type, and the remaining battery charge. The power supply control unit 224 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50 to supply the required voltage for the required period to each component, including the recording medium 227. The power supply unit 225 may be a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, or an AC adapter. The recording medium I / F 226 is an interface with a recording medium 227, such as a memory card or a hard disk. The recording medium 227 is a memory card or the like for recording captured images, and may be composed of a semiconductor memory, a magnetic disk, or the like. The recording medium 227 may be removable or built-in.

[0034] The operation unit 228 is an input unit that accepts operations from the user (user operations) and is used to input various instructions to the system control unit 50. The operation unit 228 includes the shutter button 101, the power switch 102, the mode selector switch 103, the touch panel 109, and other operation units 229. The other operation units 229 include the main electronic dial 104, the sub electronic dial 105, and the video button 106. The other operation units 229 also include the direction keys 110, the SET button 111, the AE lock button 112, the enlargement button 113, the playback button 114, the menu button 115, and the touch bar 119.

[0035] The shutter button 101 has a first shutter switch 230 and a second shutter switch 231. The first shutter switch 230 is turned on when the shutter button 101 is pressed halfway (a shooting preparation command) and generates a first shutter switch signal SW1. The system control unit 50 starts shooting preparation processes such as AF processing, AE processing, AWB processing, and EF processing in response to the first shutter switch signal SW1. The second shutter switch 231 is turned on when the shutter button 101 is pressed fully (a shooting command) and generates a second shutter switch signal SW2. The system control unit 50 starts a series of shooting processes in response to the second shutter switch signal SW2, from reading out a signal from the imaging unit 211 to generating an image file containing the captured image and writing it to the recording medium 227.

[0036] The mode selector switch 103 switches the operation mode of the system control unit 50 to one of still image capture mode, video capture mode, playback mode, etc. Modes included in the still image capture mode include auto capture mode, auto scene determination mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). There are also various scene modes and custom modes that provide capture settings for different capture scenes. The user can directly switch to one of the above-mentioned capture modes using the mode selector switch 103. Alternatively, the user can first switch to a list screen of capture modes using the mode selector switch 103, and then selectively switch to one of the displayed modes using the operation unit 228. Similarly, the video capture mode may also include multiple modes.

[0037] The touch panel 109 is a touch sensor that detects various touch operations on the display surface of the display unit 108 (the operation surface of the touch panel 109). The touch panel 109 and the display unit 108 can be configured as an integrated unit. For example, the touch panel 109 is attached to the upper layer of the display surface of the display unit 108 so that its light transmittance does not interfere with the display of the display unit 108. By associating input coordinates on the touch panel 109 with display coordinates on the display surface of the display unit 108, a GUI (graphical user interface) can be configured that makes it appear as if the user is directly operating the screen displayed on the display unit 108. The touch panel 109 can be any of a variety of types, such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, or an optical sensor type. Depending on the type, there are types that detect a touch by contact with the touch panel 109, and types that detect a touch by the approach of a finger or a pen to the touch panel 109, but either type may be used.

[0038] The system control unit 50 can detect the following operations or states on the touch panel 109.

[0039] A finger or pen that has not been touching the touch panel 109 touches the touch panel 109 again, that is, the start of touching (hereinafter referred to as Touch-Down).

[0040] A state in which the touch panel 109 is touched with a finger or a pen (hereinafter referred to as Touch-On).

[0041] The touch panel 109 is moved while being touched by a finger or a pen (hereinafter referred to as Touch-Move).

[0042] The finger or pen that has been touching the touch panel 109 is released from the touch panel 109, that is, the end of the touch (hereinafter referred to as "touch-up").

[0043] A state in which nothing is touching the touch panel 109 (hereinafter referred to as Touch-Off).

[0044] When a touch down is detected, a touch on is also detected at the same time. After a touch down, a touch on is usually continued to be detected unless a touch up is detected. If a touch move is detected, a touch on is also detected at the same time. Even if a touch on is detected, a touch move is not detected unless the touch position moves. Once it is detected that all fingers or pens that were touching have touched up, a touch off occurs.

[0045] These operation states and the position coordinates of the finger or pen touching the touch panel 109 are notified to the system control unit 50 via the internal bus. The system control unit 50 determines what kind of operation (touch operation) has been performed on the touch panel 109 based on the notified information. Regarding touch-move, the movement direction of the finger or pen moving on the touch panel 109 can also be determined for each vertical and horizontal component on the touch panel 109 based on changes in the position coordinates. If a touch-move of a predetermined distance or more is detected, it is determined that a slide operation has been performed. An operation in which a finger is touched on the touch panel 109, quickly moved a certain distance, and then released is called a flick. In other words, a flick is an operation in which a finger is quickly traced across the touch panel 109 as if flicking. If a touch-move of a predetermined distance or more at a predetermined speed or more is detected and a touch-up is then detected, it is determined that a flick has been performed (it can be determined that a flick occurred following a slide operation). Furthermore, when multiple points (for example, two points) are touched together (multi-touch), the touch operation of bringing the touched positions closer together is called pinch in, and when the touch operation of moving the touched positions farther apart is called pinch out. Pinch out and pinch in are collectively called pinch operations (or simply pinch).

[0046] The line-of-sight detection block 260 is a block for detecting the line of sight of a user who has placed his / her eye near the eyepiece 116 and is looking at the EVF 217, and if so, at what position the user is looking. The line-of-sight detection block 260 includes a dichroic mirror 262, an imaging lens 263, a line-of-sight detection sensor 264, a line-of-sight detection circuit 265, and an infrared light-emitting diode 266.

[0047] The infrared light-emitting diode 266 is a light-emitting element that emits infrared light onto the user's eyeball placed in the eyepiece 116. The infrared light emitted from the infrared light-emitting diode 266 is reflected by the eyeball, and the reflected infrared light reaches the dichroic mirror 262. The dichroic mirror 262 reflects only the infrared light and transmits visible light. The reflected infrared light, whose optical path has been changed, forms an image on the imaging surface of the line-of-sight detection sensor 264 via the imaging lens 263. The imaging lens 263 is an optical component that constitutes the line-of-sight detection optical system. The line-of-sight detection sensor 264 is composed of an imaging device such as a CCD image sensor. The line-of-sight detection sensor 264 photoelectrically converts the incident reflected infrared light into an electrical signal and outputs it to the line-of-sight detection circuit 265. The line-of-sight detection circuit 265 includes at least one processor and detects the user's line-of-sight position from the image or movement of the user's eyeball based on the output signal from the line-of-sight detection sensor 264, and outputs the detection information to the system control unit 50.

[0048] In this embodiment, the gaze detection block 260 is used to detect the gaze using a method called the corneal reflex method. The corneal reflex method detects the gaze direction and position based on the positional relationship between the pupil and the infrared light emitted from the infrared light-emitting diode 266 and reflected by the eyeball, particularly the cornea. There are various other methods for detecting the gaze direction and position, such as the scleral reflex method, which utilizes the difference in light reflectance between the black and white of the eye. Note that other gaze detection methods may be used as long as they can detect the gaze direction and position. In this embodiment, the light-emitting unit and light-receiving unit of the eyepiece detection unit 118 are described as being separate devices from the infrared light-emitting diode 266 and the gaze detection sensor 264. However, this is not a limitation. The infrared light-emitting diode 266 may also serve as the light-emitting unit of the eyepiece detection unit 118, and the gaze detection sensor 264 may also serve as the light-receiving unit.

[0049] The system control unit 50 can detect the following operations or states based on the output from the line-of-sight detection block 260. The gaze of the user who has placed his / her eye close to the eyepiece unit 116 is newly input (detected), that is, the start of gaze input. The eyepiece 116 is in a state where a user has placed their eye on it and is inputting their line of sight. The user is gazing at the eyepiece 116. The user who has placed his / her eye close to the eyepiece 116 moves his / her eye gaze away from the eyepiece 116. In other words, the eye gaze input ends. A state in which the user places his / her eye close to the eyepiece 116 and does not input any gaze.

[0050] The term "gazing" as used herein means that the user continues to look at approximately the same position for a certain period of time. For example, gazing is determined to be present when the user's gaze position does not exceed a predetermined amount of movement for a predetermined period of time (e.g., approximately 0.5 seconds). The predetermined period of time may be user-settable, a fixed period of time, or may vary depending on the distance between the previous and current gaze positions. For example, the system control unit 50 determines that the user is gazing when the duration of the state in which the user's gaze is detected at approximately the same position (no gaze movement state) exceeds a predetermined period of time (threshold period) based on the detection information received from the gaze detection circuit 265. Furthermore, the system control unit 50 determines that the user is gazing when, for example, the average position of the gaze detection position over a short period of time (≦the aforementioned threshold period) including the most recent detection timing falls within a predetermined range and the variation (variance) is less than a predetermined value.

[0051] <Example of the configuration of the lens unit 300> Fig. 3 is a schematic diagram showing an example of the configuration of lens unit 300. Fig. 3 shows a state in which lens unit 300 is attached to camera 100. Note that, among the camera 100 shown in Fig. 3, the same components as those described in Fig. 2 are given the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0052] Lens unit 300 is a type of interchangeable lens that can be attached to and detached from camera 100. Lens unit 300 is a twin lens that can capture images with parallax between left and right images. Lens unit 300 has two optical systems, each with a wide viewing angle of approximately 180 degrees, and can capture images of the range of the forward hemisphere. Specifically, the two optical systems of lens unit 300 can each capture an object within a field of view (angle of view) of 180 degrees in the left-right direction (horizontal angle, azimuth angle, yaw angle) and 180 degrees in the up-down direction (vertical angle, elevation angle, pitch angle).

[0053] The lens unit 300 includes a right-eye optical system 301R having a plurality of lenses and a reflecting mirror, a left-eye optical system 301L having a plurality of lenses and a reflecting mirror, and a lens system control circuit 303. The right-eye optical system 301R corresponds to an example of a first optical system, and the left-eye optical system 301L corresponds to an example of a second optical system. The right-eye optical system 301R and the left-eye optical system 301L have lenses 302R and 302L located on the subject side, respectively, facing the same direction, and their optical axes are approximately parallel. The lens unit 300 of this embodiment is a VR180 lens for capturing images for VR180, a VR image format capable of two-eye stereoscopic viewing. In the VR180 lens, the right-eye optical system 301R and the left-eye optical system 301L each have a fisheye lens capable of capturing a range of approximately 180 degrees. The VR180 lens may be a lens that can capture a wide viewing angle range of about 160 degrees, which is narrower than the 180-degree range, as long as the right-eye optical system 301R and the left-eye optical system 301L can acquire images that allow two-eye VR display as VR180. The VR180 lens can form a right image (first image) formed via the right-eye optical system 301R and a left image (second image) formed via the left-eye optical system 301L, which has parallax from the right image, on one or two image pickup elements of the attached camera.

[0054] Furthermore, the lens unit 300 is attached to the camera 100 via the lens mount section 304 and the camera mount section 305 of the camera 100. When the lens unit 300 is attached to the camera 100, the system control section 50 of the camera 100 and the lens system control circuit 303 of the lens unit 300 are electrically connected via the communication terminal 124 of the camera 100 and the communication terminal 306 of the lens unit 300.

[0055] In this embodiment, the right image formed through the right optical system 301R and the left image formed through the left optical system 301L having a parallax with respect to the right image are imaged side by side on the imaging unit 211 of the camera 100. That is, two optical images formed by the right optical system 301R and the left optical system 301L are formed on one imaging element. The imaging unit 211 converts the imaged subject image (optical signal) into an analog electrical signal. Thus, by using the lens unit 300, two image sets with parallax can be acquired from two locations (optical systems), namely the right optical system 301R and the left optical system 301L. Further, by separately VR-displaying the acquired images as left-eye images and right-eye images, the user can view a three-dimensional VR image in a range of approximately 180 degrees, so-called VR180.

[0056] <Explanation of VR Image> Here, a VR image is an image that can be VR-displayed, which will be described later. VR images include omnidirectional images (whole-spherical images) taken by an omnidirectional camera (whole-spherical camera), panoramic images having a video range (effective video range) wider than the display range that can be displayed on the display unit at one time, and the like. Also, VR images are not limited to still images, but also include moving images and live view images (images acquired almost in real time from a camera). A VR image has a video range (effective video range) for a visual field of up to 360 degrees in the left-right direction and 360 degrees in the up-down direction. Also, VR images include images having an angle of view wider than the angle of view that can be captured by an ordinary camera, or a video range wider than the display range that can be displayed on the display unit at one time, even if they are less than 360 degrees in the left-right direction and less than 360 degrees in the up-down direction. The image captured by the camera 100 using the above-described lens unit 300 is a type of VR image. A VR image can be VR-displayed, for example, by setting the display mode of a display device (a display device capable of displaying a VR image) to "VR view". By VR-displaying a VR image having a 360-degree angle of view and changing the posture of the display device by the user in the left-right direction (horizontal rotation direction), a seamless omnidirectional video can be viewed in the left-right direction.

[0057] Here, VR display (VR view) refers to a display method (display mode) that displays a VR image with a field of view that corresponds to the orientation of the display device, allowing for a change in display range. VR display includes "single-eye VR display (single-eye VR view)," which displays a single image by mapping a VR image onto a virtual sphere (deformation that corrects distortion). VR display also includes "two-eye VR display (two-eye VR view)," which displays a VR image for the left eye and a VR image for the right eye side by side by mapping them onto a virtual sphere. Stereoscopic viewing is possible by performing "two-eye VR display" using a VR image for the left eye and a VR image for the right eye that have parallax. Regardless of the VR display, for example, when a user wears a display device such as an HMD (head-mounted display), the image displayed corresponds to the orientation of the user's face. For example, suppose a VR image with a field of view centered at 0 degrees left and right (a specific direction, e.g., north) and 90 degrees up and down (90 degrees from the zenith, i.e., horizontal) is displayed at a certain point in time. If the orientation of the display device is flipped from this state (for example, by changing the display surface from facing south to facing north), the display range of the same VR image is changed to an image with a field of view centered at 180 degrees left and right (the opposite direction, for example, south) and 90 degrees up and down. In other words, when the user wears the HMD and turns their face from north to south (i.e., turns backward), the image displayed on the HMD also changes from a north image to a south image. Note that the VR image captured using the lens unit 300 of this embodiment is a VR180 image captured in a range of approximately 180 degrees forward, and no image exists in a range of approximately 180 degrees behind. If such a VR180 image is VR displayed and the orientation of the display device is changed to the side where no image exists, a blank area is displayed.

[0058] By displaying VR images in this way, the user visually feels as if they are inside the VR image (in the VR space). Note that the method of displaying VR images is not limited to changing the posture of the display device. For example, the display range may be moved (scrolled) in response to user operation via a touch panel or directional buttons. Furthermore, during VR display (display mode "VR view"), in addition to changing the display range due to posture changes, the display range may also be changed in response to touch-move on the touch panel, dragging with a mouse, pressing directional buttons, etc. Note that a smartphone attached to VR goggles (head-mounted adapter) is a type of HMD.

[0059] In this embodiment, the camera 100 is capable of single-eye VR display of either the left-eye or right-eye VR image in both live view display in shooting mode and playback display of captured images in playback mode. Furthermore, pressing the up direction key 110 allows switching between VR display and non-VR display. Herein, non-VR display refers to displaying the left and right VR images acquired by the right-eye optical system 301R and left-eye optical system 301L as is, without performing the transformation of mapping onto a virtual sphere as in VR display. Hereinafter, VR display in live view display in shooting mode will be referred to as VRLV, and non-VR display in live view display in shooting mode will be referred to as normal LV. Furthermore, VR display in playback display in playback mode will be referred to as VR playback, and non-VR display in playback display in playback mode will be referred to as normal playback.

[0060] <Transition of VR image display methods during shooting and playback> FIG. 4 shows the transition of the display method of a VR image during shooting and playback. The normal LV display 400 displays a subject using a non-VR display method in a live view display during shooting. Here, in the normal LV display 400, a left-eye image and a right-eye image are displayed using a circular fisheye method. That is, a left-eye image in a circular image area and a right-eye image in a circular image area are displayed. The VRLV display 401 displays a subject using a VR display method in a live view display during shooting. The VRLV display 401 is a perspective projection image obtained by mapping a VR image for either the left or right eye onto a virtual sphere and then performing perspective projection conversion onto a virtual plane, and displays a portion of the imaging range of the VR image in a three-dimensional manner. In the normal LV display 400, the image has distortion, but in the VRLV display 401, the distortion is suppressed compared to the normal LV display 400. The display method for VR images during shooting and playback may include a plurality of display methods including at least one of the circular fisheye method, the perspective projection method, and the equirectangular method.

[0061] Here, the user of the camera 100 can switch between the left and right VR images for which perspective projection transformation is to be performed by pressing the downward direction of the directional key 110. Furthermore, the center position of the display range can be moved by performing a slide operation on the touch panel 109, and the magnification rate of the display range can be changed by performing a pinch-in / pinch-out operation. Furthermore, by pressing the upward direction of the directional key 110, the display method can be switched between the normal LV display 400 and the VRLV display 401.

[0062] When the user presses the shutter button 101, the camera 100 performs various image processing for capturing an image, and when preparations for recording to the recording medium 227 are complete, it performs playback processing to present the user with a recorded image (recorded captured image) as the captured image. The normal playback display 402 is a display format during playback that is displayed when capturing an image while the normal LV display 400 is being displayed. The normal playback display 402 is a display format for non-VR display, and its appearance matches that of the normal LV display 400. Here, the normal LV display 400 displays a circular image area for the left eye and a circular image area for the right eye. The VR playback display 403 is a display format for playback that is displayed when capturing an image while the VRLV display 401 is being displayed. The VR playback display 403 is a display format for VR display, and its appearance matches that of the VRLV display 401. That is, the image in the normal playback display 402 has distortion, whereas the distortion in the VR playback display 403 is reduced compared to the normal playback display 402. At this time, the initial display range of the VR playback display 403 and whether the VR image for the left or right eye is displayed are assumed to be inherited from the state of the VRLV display 401 when the user of the camera 100 instructed to shoot. As with shooting, during playback, the display method can be switched between the normal playback display 402 and the VR playback display 403 by pressing the up direction of the directional key 110. Similarly to shooting, the left or right eye can be switched by pressing the down direction of the directional key 110. Similarly to shooting, the center position and magnification of the display range can be changed by performing a slide operation or a pinch-in / pinch-out operation on the touch panel 109. Furthermore, at this time, by pressing the SET button 111, it is possible to return to the initial display range of the VR playback display 403, i.e., the display range of the VRLV display 401 when the user of the camera 100 instructed to shoot. The VR playback display range 404 is a frame-shaped display object (display item) that is displayed when switching from the VR playback display 403 to the normal playback display 402, and indicates where the range that was displayed in the previous VR playback display 403 is located in the normal playback display 402. However, although a frame-shaped display object is used in this embodiment, the shape and color thereof are not important, such as a circle, a dot, or a rectangle.Any display content is acceptable as long as it can indicate the display range during VR playback.

[0063] <Photography processing flowchart> 5 is a flowchart of the shooting process of the camera 100 in this embodiment. First, in step S500, the system control unit 50 starts live view display. The display format of the live view displayed here is the normal LV display 400, but the previous display format may be stored in the non-volatile memory 219 so that the same display format is displayed the next time live view is displayed.

[0064] Next, in step S501, the system control unit 50 receives an instruction from the user to switch between normal LV and VRLV display. That is, the system control unit 50 receives an instruction to change the display method from the user. The system control unit 50 switches between the normal LV display 400 and the VRLV display 401 each time the user presses the up direction key 110.

[0065] Next, in step S502, the system control unit 50 receives a shooting instruction from the user. Upon detecting that the user has pressed the shutter button 101, the system control unit 50 proceeds to shooting processing.

[0066] Next, in step S503, the system control unit 50 determines whether the VRLV was displayed at the time of shooting. If the VRLV was not displayed (No in step S503), in step S505, the system control unit 50 identifies the eye / viewpoint position that the user was focusing on during shooting based on the line-of-sight information and stores it in the memory 215. Here, the line-of-sight information is detected by the line-of-sight detection block 260. Depending on whether the user was looking at the left or right half of the normal LV display 400 during shooting, it is determined whether the user was focusing on the left-eye VR image or the right-eye VR image. After identifying the eyes, it stores in the memory 215 which part of that half was being focused on in the form of XY coordinates with the upper left corner as the origin. Note that a predetermined initial value for the magnification ratio is stored in the memory 215. In this embodiment, the gaze detection block 260 is provided in the EVF 217, and therefore, when the normal LV display 400 is displayed on something other than the EVF 217, the system control unit 50 also stores predetermined initial values ​​for the focused eye and viewpoint position in the memory 215. On the other hand, a mechanism for detecting the gaze may be provided independently of the EVF 217, and in that case, the system control unit 50 performs processing similar to this step in a situation where the gaze can be detected. On the other hand, when the VRLV is displayed (Yes in step S503), the system control unit 50 stores information on the eye and viewpoint positions and magnification ratio displayed in the VRLV in the memory 215 in step S504.

[0067] Next, in step S506, the system control unit 50 performs imaging and development processing, which involves converting the analog signal of the subject image formed on the imaging unit 211 into a digital signal and performing predetermined image processing.

[0068] Next, in step S507, the system control unit 50 writes the captured image as a result of the imaging and development processing in the previous step to the memory 215 and the recording medium 227. That is, the system control unit 50 records the captured image in the recording unit. Note that the system control unit 50 may also record information related to the display method of the live view image in association with the captured image.

[0069] Next, in step S508, the system control unit 50 performs a playback process of the shooting results. This is to allow the shooting results to be checked immediately after shooting, and is automatically displayed without any operation by the user. Note that this playback process is not necessarily limited to cases where the shooting results are automatically displayed after recording, but also applies when the user issues a separate playback instruction after shooting. Details of this playback process will be explained later.

[0070] Finally, in step S509, the system control unit 50 determines whether or not to continue shooting. For example, if the user turns off the power by operating the power switch 102, or presses the playback button 114, menu button 115, or the like to operate another function, shooting ends. On the other hand, if the user presses the SET button 111, for example, shooting continues. If shooting does not continue (No in step S509), the system control unit 50 simply ends the shooting process. On the other hand, if shooting continues (Yes in step S509), the system control unit 50 returns to step S500 and displays the live view again.

[0071] <Regeneration process flowchart> 6 is a flowchart of the playback process of the camera 100 in this embodiment. First, in step S600, the system control unit 50 decodes the recorded image. This is performed by a circuit in the image processing unit 214, which decodes the content encoded in a format such as JPEG and stores it in the memory 215 in a predetermined format such as YUV. The decoded result is the recorded image before conversion processing for VR display is performed, and is a VR image for the left eye and a VR image for the right eye lined up side by side.

[0072] Next, in step S601, the system control unit 50 determines whether VRLV was displayed when shooting. If information related to the display method of live view images is recorded in the recording unit in association with the captured image, the system control unit 50 determines whether VRLV was displayed when shooting based on the information recorded in the recording unit. Here, if VRLV was not displayed (No in step S601), the system control unit 50 proceeds to step S602. On the other hand, if VRLV was displayed when shooting (Yes in step S601), the system control unit 50 proceeds to step S603.

[0073] In step S602, the system control unit 50 resizes the decoded result to match the output resolution, that is, to change the vertical and horizontal sizes of the decoded result to match the resolution of a display device such as the display unit 108 or EVF 217.

[0074] In step S603, the system control unit 50 sets the eye and viewpoint positions and magnification ratio at the time of shooting as parameters for perspective projection transformation.

[0075] Next, in step S604, the system control unit 50 performs perspective projection transformation on the decoded image based on the parameters set in the previous step, thereby rendering a three-dimensional image for a VR image equivalent to one eye with the same viewpoint position and magnification as when the image was captured.

[0076] Next, in step S605, the system control unit 50 displays the result of drawing in step S602 or step S604 on the display unit 108 or EVF 217 as a reproduced image.

[0077] Next, in step S606, the system control unit 50 accepts an operation input from the user. If the user performs an operation during this time, the content of the operation is identified in the subsequent processing, and the corresponding processing is performed.

[0078] Upon receiving an operation input from the user, the system control unit 50 determines in step S607 whether VR is being played back. If VR is not being played back (No in step S607), the system control unit 50 proceeds to step S612. On the other hand, if VR is being played back (Yes in step S607), the system control unit 50 proceeds to step S608.

[0079] In step S608, the system control unit 50 determines whether the user's operation input is an operation to change the display range. This is determined based on whether a slide operation or a pinch-in / pinch-out operation has been performed on the touch panel 109. If the operation input is an operation to change the display range (Yes in step S608), the system control unit 50 proceeds to step S609. On the other hand, if the operation input is not an operation to change the display range (No in step S608), the system control unit 50 proceeds to step S610.

[0080] In step S609, the system control unit 50 changes the center position and magnification of the display range in response to the touch operation, and then performs perspective projection transformation again to change the content of the VR display. After that, the system control unit 50 returns to step S606 and accepts operation input from the user again.

[0081] In step S610, the system control unit 50 determines whether or not the operation is a display range reset operation. This is determined by whether or not the user has pressed the SET button 111. If the operation is a display range reset operation (Yes in step S610), in step S611 the system control unit 50 changes the center position and magnification of the display range to the same values ​​as those at the time of VRLV display at the time of shooting. To do this, the values ​​at the time of VRLV stored in memory 215 are read, and perspective projection transformation is performed again. Thereafter, the system control unit 50 returns to step S606 and accepts operation input from the user again. On the other hand, if the operation is not a display range reset operation (No in step S610), the system control unit 50 proceeds to step S612.

[0082] In step S612, the system control unit 50 determines whether the user's operation input is an operation to switch the display method. This is determined by whether the user has pressed the up direction key 110. If it is an operation to switch the display method (Yes in step S612), the system control unit 50 performs a display method change process in step S613. Details of this process will be explained later. After the display method change process is completed, the system control unit 50 returns to step S606 and accepts another operation input from the user. On the other hand, if it is not an operation to switch the display method (No in step S612), the system control unit 50 proceeds to step S614.

[0083] In step S614, the system control unit 50 determines whether the user's operation input is an operation to end playback. For example, this is determined based on whether a predetermined key for ending the playback function, such as the playback button 114, menu button 115, or shutter button 101, has been pressed. If the operation input is not an operation to end playback (No in step S614), the system control unit 50 returns to step S606 and accepts another operation input from the user. On the other hand, if the operation input is an operation to end playback (Yes in step S614), the system control unit 50 ends the playback process.

[0084] <Flowchart of display method change process> 7 is a flowchart showing the display method change process of the camera 100 in this embodiment. First, in step S700, the system control unit 50 determines whether VR playback is in progress. If VR playback is in progress (Yes in step S700), in step S701 the system control unit 50 resizes the decoded result to match the output resolution in order to switch from VR playback to normal playback. As with step S602, this resizes the decoded result to match the resolution of the display unit 108 or EVF 217 and presents it to the user.

[0085] Next, in step S702, the system control unit 50 displays a frame indicating the display range during VR playback. This is processing to display the VR playback display range 404 superimposed on the normal playback display 402. The frame display position and size are changed and drawn based on the viewpoint position and magnification ratio of the perspective projection transformation last set during VR playback.

[0086] On the other hand, if VR playback was not in progress (No in step S700), in step S703, the system control unit 50 determines whether or not the normal LV was in effect when the image was captured. If the normal LV was not in effect (No in step S703), in step S704, the system control unit 50 reads out information on the eye and viewpoint positions and magnification ratio in the VRLV display when the image was captured from the memory 215. This information was stored in step S504. On the other hand, if the normal LV was in effect when the image was captured (Yes in step S703), in step S705, the system control unit 50 reads out the eye and viewpoint positions identified from the line-of-sight information when the image was captured from the memory 215. Here, since the magnification ratio information cannot be identified from the line-of-sight information, the system control unit 50 adopts the predetermined initial value stored in the memory 215 in step S505.

[0087] Subsequently, in step S706, the system control unit 50 sets the eye / viewpoint positions and magnification ratio read out in the previous steps S704 and S705 as parameters of perspective projection transformation.

[0088] Then, in step S707, the system control unit 50 performs perspective projection conversion drawing based on the parameters set in the previous step S706, thereby switching the display method to VR display.

[0089] According to the first embodiment of the present invention described above, it is possible to check the shooting results of a VR image in a display format that matches the display method of the VR image at the time of shooting.

[0090] (Variation) (A) In the first embodiment of the present invention described above, an example was shown in which the display format during playback was determined depending on whether normal LV or VRLV was displayed during shooting. As another modification, for example, a configuration may be adopted in which the user can set a desired display format during playback in advance. To achieve this, in step S601 shown in FIG. 6, camera 100 reads the setting value of the display format during playback stored in nonvolatile memory 219, and proceeds to step S601 in the case of VR playback, or to step S602 in the case of normal playback.

[0091] (B) The present invention may be an information processing method including the steps of each process performed by the information processing device described above. The present invention may also be an information processing program that causes each process performed by the information processing device described above to run on a computer. The program can be distributed via various storage media or a network, and can be executed by being installed on a computer having a storage device such as a ROM. In this case, the program and the storage medium storing the program constitute the present invention.

[0092] (Other embodiments) The present invention can also be realized by executing the following process: software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various storage media, and the computer (or control unit, MPU, etc.) of the system or device reads and executes the program code. In this case, the program and the storage medium storing the program constitute the present invention.

[0093] While the present invention has been described in detail above based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

[0094] Note that each functional unit in each of the above embodiments (variations) may or may not be individual hardware. The functions of two or more functional units may be realized by common hardware. Each of multiple functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Furthermore, each functional unit may or may not be realized by hardware such as an ASIC, FPGA, or DSP. For example, an apparatus may have a processor and a memory (storage medium) in which a control program is stored. Then, the functions of at least some of the functional units of the apparatus may be realized by the processor reading and executing the control program from the memory.

[0095] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0096] In addition, in each of the examples described above, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs) and dedicated processors (e.g., GPUs, ASICs, FPGAs, and programmable logic devices, etc.).

[0097] The disclosure of this embodiment includes the following configuration, method, and program.

[0098] [Configuration 1] a determination means for determining, by having a user select, a first display mode for displaying a live view image on a display unit from among a plurality of display modes including a display mode in which distortion is suppressed; a control means for controlling the display unit to display the live view image in the first display format, and for controlling, when an image capture instruction is given while the live view image is being displayed on the display unit in the first display format, the display unit to display, in the first display format, an image that has been recorded in a recording unit in response to the image capture instruction. A display device characterized by:

[0099] [Configuration 2] further comprising an acquisition means for acquiring an image set including a first image and a second image having a parallax with respect to each other; The determining means determines the first display method by having a user select from at least a second display method in which a predetermined region of one of the images in the image set is displayed in a display method in which distortion is suppressed, and a third display method in which the image set is displayed. 2. The display device according to configuration 1,

[0100] [Configuration 3] When the live view image or the captured image is changed from the second display format to the third display format, the control means controls the third display format so that an item indicating the predetermined area is displayed on at least the first image. 3. The display device according to configuration 2.

[0101] [Configuration 4] When the display mode is changed from the third display mode to the second display mode while the user is viewing the first image of the image set in the third display mode, the control means controls the display unit to display the first image in the second display mode. 4. The display device according to configuration 2 or 3.

[0102] [Configuration 5] When the display mode is changed from the third display mode to the second display mode while the user is looking at a predetermined position of the first image in the image set in the third display mode, the control means controls the display unit to display the predetermined area including the predetermined position of the first image in the second display mode. 5. The display device according to any one of configurations 2 to 4.

[0103] [Configuration 6] The first image and the second image have distorted circular image regions. 6. The display device according to any one of configurations 2 to 5, wherein:

[0104] [Configuration 7] a third image is generated by forming the first image and the second image on one image sensor; When the live view image or the captured image is displayed on the display unit in the second display format, the control means controls the display unit to cut out the predetermined area of ​​the first image from the third image, convert it into an image with reduced distortion, and display it. 7. The display device according to any one of configurations 2 to 6, wherein:

[0105] [Configuration 8] When a second display method different from the first display method has been selected in advance as a display method for displaying the captured image, the control means controls the display unit to display the captured image in the second display method, even if the control means has controlled the display unit to display the live view image in the first display method. 8. The display device according to any one of configurations 1 to 7.

[0106] [Configuration 9] The control means controls the display unit to display a predetermined area of ​​the imaging range as the live view image, and when the imaging instruction is given while the predetermined area is being displayed on the display unit, controls the display unit to display the predetermined area of ​​the captured image recorded in the recording unit in response to the imaging instruction. 9. The display device according to any one of configurations 1 to 8.

[0107] [Configuration 10] The display unit further includes an instruction acquisition unit that acquires an instruction to change the display method from the user while the live view image or the captured image is being displayed on the display unit. 10. The display device according to any one of configurations 1 to 9.

[0108] [Configuration 11] The camera further includes a recording unit that, in response to the image capture instruction, records the captured image in the recording unit in association with the display format in which the live view image was displayed. 11. The display device according to any one of configurations 1 to 10.

[0109] [Configuration 12] The control means controls the display unit to display the captured image recorded in the recording unit in response to the image capture instruction instead of the live view image. 12. The display device according to any one of configurations 1 to 11.

[0110] [Configuration 13] The plurality of display methods include at least one of a circular fisheye method, a perspective projection method, and an equirectangular method. 13. The display device according to any one of configurations 1 to 12.

[0111] [Configuration 14] a determination means for determining, by having a user select, a first display mode for displaying a live view image on a display unit from among a plurality of display modes including a display mode in which distortion is suppressed; an acquisition means for acquiring an instruction to display a first captured image recorded in a recording unit in response to an imaging instruction while the live view image is being displayed on the display unit in the first display format; and a control means for controlling the display unit to display the first captured image in the first display format in response to the instruction. A display device characterized by:

[0112] [Control method] a determination step of having a user select a first display mode for displaying a live view image on a display unit from among a plurality of display modes including a display mode in which distortion is suppressed; a control step of controlling the display unit to display the live view image in the first display format, and, when an image capturing instruction is given while the live view image is being displayed on the display unit in the first display format, controlling the display unit to display, in the first display format, an image captured in response to the image capturing instruction and recorded in a recording unit. A method for controlling a display device.

[0113] [Control method] a determination step of having a user select a first display mode for displaying a live view image on a display unit from among a plurality of display modes including a display mode in which distortion is suppressed; an acquiring step of acquiring an instruction to display a first captured image recorded in a recording unit in response to an imaging instruction while the live view image is being displayed on the display unit in the first display format; and a control step of controlling the display unit to display the first captured image in the first display format in response to the instruction. A method for controlling a display device.

[0114] [system] A display device; A recording device; a determination device that determines, by having a user select, a first display mode when a live view image is to be displayed on the display device, from a plurality of display modes including a display mode in which distortion is suppressed; a control device that controls the display device to display the live view image in the first display format, and, when an image capture instruction is given while the live view image is being displayed on the display device in the first display format, controls the display device to display an image that has been recorded in the recording device in response to the image capture instruction in the first display format. A system characterized by:

[0115] [system] A display device; A recording device; a determination device that determines a first display mode for displaying a live view image by having a user select one of a plurality of display modes including a display mode in which distortion is suppressed; an acquisition device that acquires an instruction to display a first captured image recorded in the recording device in response to an imaging instruction while the live view image is being displayed on the display device in the first display format; a control device that controls the display device to display the first captured image in the first display format in response to the instruction. A system characterized by:

[0116] [program] A program for causing a computer to function as each of the means of the display device according to any one of configurations 1 to 14.

Claims

1. a determination means for determining, by having a user select, a first display mode when displaying a live view image on a display unit from among a plurality of display modes including a display mode in which distortion is suppressed; a control means for controlling the display unit to display the live view image in the first display format, and for controlling, when an image capture instruction is given while the live view image is being displayed on the display unit in the first display format, the display unit to display, in the first display format, an image captured in response to the image capture instruction and recorded in a recording unit. A display device characterized by:

2. The method further includes acquiring an image set including a first image and a second image having a parallax with respect to each other, The determining means determines the first display method by having a user select from at least a second display method in which a predetermined area of ​​one of the images in the image set is displayed in a display method in which distortion is suppressed, and a third display method in which the image set is displayed.

2. The display device according to claim 1.

3. When the live view image or the captured image is changed from the second display format to the third display format, the control means controls the display of the item indicating the predetermined area on at least the first image in the third display format.

3. The display device according to claim 2.

4. When the display mode is changed from the third display mode to the second display mode while the user is viewing the first image of the image set in the third display mode, the control means controls the display unit to display the first image in the second display mode.

3. The display device according to claim 2.

5. When the display mode is changed from the third display mode to the second display mode while the user is looking at a predetermined position of the first image in the image set in the third display mode, the control means controls the display unit to display the predetermined area including the predetermined position of the first image in the second display mode.

3. The display device according to claim 2.

6. The first image and the second image have distorted circular image regions.

3. The display device according to claim 2.

7. a third image is generated by forming the first image and the second image on one image sensor; When the live view image or the captured image is displayed on the display unit in the second display format, the control unit controls the display unit to cut out the predetermined area of ​​the first image from the third image, convert it into an image with reduced distortion, and display it on the display unit.

3. The display device according to claim 2.

8. When a second display method different from the first display method has been selected in advance as a display method for displaying the captured image, the control means controls the display unit to display the captured image in the second display method, even if the control means has controlled the display unit to display the live view image in the first display method.

2. The display device according to claim 1.

9. The control means controls the display unit to display a predetermined area of ​​the imaging range as the live view image, and when the imaging instruction is given while the predetermined area is being displayed on the display unit, controls the display unit to display the predetermined area of ​​the captured image recorded in the recording unit in response to the imaging instruction.

2. The display device according to claim 1.

10. The display unit further includes an instruction acquisition unit that acquires an instruction to change the display method from the user while the live view image or the captured image is being displayed on the display unit.

2. The display device according to claim 1.

11. The camera further includes a recording unit that, in response to the image capture instruction, records the captured image in the recording unit in association with the display format in which the live view image was displayed.

2. The display device according to claim 1.

12. The control means controls the display unit to display the captured image recorded in the recording unit in response to the image capture instruction instead of the live view image.

2. The display device according to claim 1.

13. The plurality of display methods include at least one of a circular fisheye method, a perspective projection method, and an equirectangular method.

2. The display device according to claim 1.

14. a determination means for determining, by having a user select, a first display mode when displaying a live view image on a display unit from among a plurality of display modes including a display mode in which distortion is suppressed; an acquisition means for acquiring an instruction to display a first captured image recorded in a recording unit in response to an image capturing instruction while the live view image is being displayed on the display unit in the first display format; and a control means for controlling the display unit to display the first captured image in the first display format in response to the instruction. A display device characterized by:

15. a determining step of determining, by having a user select, a first display mode for displaying a live view image on a display unit from among a plurality of display modes including a display mode in which distortion is suppressed; a control step of controlling the display unit to display the live view image in the first display format, and, when an image capturing instruction is given while the live view image is being displayed on the display unit in the first display format, controlling the display unit to display an image recorded in a recording unit in response to the image capturing instruction in the first display format. A method for controlling a display device.

16. A program for causing a computer to function as each of the means of the display device according to claim 1.

Images