Electronic device, electronic device control method, and program

Abstract

To reduce uncomfortable feeling given to a user when switching between display of a part of an image and display of a part of an image obtained by performing distortion correction of the image.SOLUTION: An electronic device includes: acquisition means which acquires a first image and a second image which is an image obtained by subjecting the first image to geometric conversion processing for distortion reduction; switch means which switches a display mode between a first mode which displays a range of a part of the first image on a display region and a second mode which displays a range of a part of the second image on the display region; and adjustment means which, when the display mode is switched, adjusts a display position of an image to be displayed in the display region after the display mode is switched so that an object which appears at a first position of the display region before the display mode is switched appears at the first position after the display mode is switched.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an electronic device, a control method for an electronic device, and a program. [Background technology]

[0002] There is a known technology that uses a fisheye lens with two optical systems in one lens mount to simultaneously capture two moving images with parallax and display them as a parallax image (VR video), providing users with a three-dimensional VR viewing experience.

[0003] In Patent Document 1, the device cuts out an image within a predetermined area from a fisheye image, and displays an image in which distortion has been corrected for the image within the cut-out area. At that time, the device allows the user to easily confirm the cut-out area within the fisheye image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-58801 A Summary of the Invention [Problem to be solved by the invention]

[0005] Here, a user may wish to switch between an image showing a part of a fisheye image and an image showing a part of an image after distortion correction of the fisheye image. In this case, different subjects are captured at the same coordinates in the two images. For this reason, after switching between the images, the user may feel uncomfortable when viewing the images.

[0006] Therefore, an object of the present invention is to reduce the sense of discomfort felt by a user when switching between displaying a portion of an image and displaying a portion of an image after distortion correction of the image. [Means for solving the problem]

[0007] One aspect of the present invention is a method for producing a composition comprising the steps of: An acquisition means for acquiring a first image and a second image obtained by subjecting the first image to a geometric transformation process for reducing distortion; a switching means for switching a display mode between a first mode in which a partial range of the first image is displayed in a display area and a second mode in which a partial range of the second image is displayed in the display area; an adjustment means for adjusting, when the display mode is switched, a display position of an image displayed in the display area after the display mode is switched, so that an object displayed in a first position in the display area before the display mode is switched is displayed in the first position after the display mode is switched; The electronic device is characterized by having the following features.

[0008] One aspect of the present invention is a method for producing a composition comprising the steps of: acquiring a first image and a second image obtained by subjecting the first image to a geometric transformation process that reduces distortion; a switching step of switching a display mode between a first mode in which a partial range of the first image is displayed in a display area and a second mode in which a partial range of the second image is displayed in the display area; When the display mode is switched, an object that is displayed at a first position in the display area before the display mode is switched is displayed at the first position after the display mode is switched. an adjustment step of adjusting a display position of an image displayed in the display area after the display mode is switched, 13. A method for controlling an electronic device comprising: Effect of the Invention

[0009] According to the present invention, it is possible to reduce the sense of discomfort felt by a user when switching between displaying a portion of an image and displaying a portion of an image after distortion correction of the image. [Brief description of the drawings]

[0010] [Figure 1] 1 is an external view of a camera according to a first embodiment. [Diagram 2] FIG. 1 is a configuration diagram of a camera according to a first embodiment. [Diagram 3] FIG. 1 is a configuration diagram of a lens unit according to a first embodiment. [Figure 4] FIG. 2 is a diagram illustrating the configuration of a PC according to the first embodiment. [Diagram 5] FIG. 2 is a diagram illustrating a fitted display according to the first embodiment. [Figure 6] FIG. 2 is a diagram for explaining a partial display of an image according to the first embodiment. [Figure 7] 4 is a flowchart showing the operation of a PC according to the first embodiment. [Figure 8] 5A to 5C are diagrams illustrating adjustment of a display position according to the first embodiment. [Figure 9] 13A and 13B are diagrams illustrating a display of a circular fisheye image according to the second embodiment. [Figure 10] 10 is a flowchart showing the operation of a PC according to the second embodiment. [Figure 11] 13A to 13C are diagrams illustrating adjustment of a display position according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0012] Applications that handle parallax images (VR images) have, for example, a "circular fisheye display mode" that displays a circular fisheye image after shooting, and an "equirectangular cylindrical projection display mode" that displays a distortion-corrected image that is converted from the circular fisheye image using equirectangular projection. In addition, the above applications are generally capable of switching between a "fit display mode" that reduces the image to fit the entire image in the preview area, and an "enlarged display mode" that displays only a portion of the image. The enlarged display mode also includes a dot-by-dot display.

[0013] Consider a case where a user, while checking a certain object in an image in both circular fisheye display mode and enlarged display mode, switches the display mode from the circular fisheye display mode to the equirectangular projection display mode while keeping the enlarged display. In this case, the object that the user checked at the center position of the image display area before switching the display mode may not be displayed in the same position after switching the display mode. In particular, for an object near the circumference of a circular fisheye image, the position of the object may move significantly due to switching the display mode, and the position of the object may change every time the display mode is switched, which may be annoying.

[0014] <Embodiment 1> The imaging device according to the first embodiment is a digital camera that captures moving images with a twin lens (VR180 lens). Digital camera (hereinafter, camera) 100 that captures images will be described below. FIGS. 1A and 1B are diagrams showing the external appearance of camera 100 according to the first embodiment. FIG. 1A is a perspective view of camera 100 as seen from the front. FIG. 1B is a perspective view of camera 100 as seen from the back. FIG. 2 is a diagram showing an example of the configuration of camera 100.

[0015] The camera 100 has, on its top surface, a shutter button 101, a power switch 102, a mode changeover switch 103, a main electronic dial 104, a sub electronic dial 105, a video button 106, and an outside viewfinder display unit 107. The shutter button 101 is an operation unit for preparing for shooting or issuing shooting instructions. The power switch 102 is used to turn the power of the camera 100 on and The mode switch 103 is an operation unit for switching between on and off. The mode changeover switch 103 is an operation unit for switching between various modes. The main electronic dial 104 is a rotary operation unit for changing settings such as the shutter speed or aperture. The sub electronic dial 105 is a rotary operation unit for moving the selection frame (cursor) or forwarding images. The video button 106 is an operation unit for issuing an instruction to start or stop video shooting (recording). The outside viewfinder display unit 107 displays various settings such as the shutter speed or aperture.

[0016] The camera 100 has a display unit 108, a touch panel 109, direction keys 110, a SET button 111, an AE lock button 112, a magnification button 113, a playback button 114, a menu button 115, an eyepiece unit 116, an eyepiece detection unit 118, and a touch bar 119 on the rear surface.

[0017] The display unit 108 displays images and various information. The touch panel 109 is an operation unit that detects touch operations on the display surface (touch operation surface) of the display unit 108. The directional keys 110 are an operation unit consisting of keys (four-way keys) that can be pressed up, down, left, and right. The directional keys 110 can accept operations according to the position where the directional keys 110 are pressed. The SET button 111 is an operation unit that is pressed mainly to confirm a selected item. The AE lock button 112 is an operation unit that is pressed to fix the exposure state in a shooting standby state.

[0018] The enlargement button 113 is an operation unit for switching the enlargement mode on and off in the live view display (LV display) in the shooting mode. When the enlargement mode is on, the live view image (LV image) is enlarged or reduced by operating the main electronic dial 104. The enlargement button 113 is also used to enlarge a playback image or increase the magnification ratio (display magnification) in the playback mode. The playback button 114 is an operation unit for switching between the shooting mode and the playback mode. When the playback button 114 is pressed in the shooting mode, the mode transitions to the playback mode, and the latest image among images recorded in the recording medium 227 described later can be displayed on the display unit 108.

[0019] Menu button 115 is an operation unit that is pressed when displaying a menu screen on display unit 108 that allows various settings to be made. The user can intuitively make various settings using the menu screen displayed on display unit 108, direction key 110, and SET button 111. Eyepiece unit 116 is a portion for placing an eye on eyepiece finder (peek-in type finder) 117. The user can view an image displayed on an internal EVF 217 (Electronic View Finder) described later through eyepiece unit 116. Eyepiece detection unit 118 is a sensor that detects whether the user places his / her eye on eyepiece unit 116.

[0020] The touch bar 119 is a line-shaped touch operation unit (line touch sensor) capable of receiving a touch operation. The touch bar 119 is disposed at a position where it can be touched (touched) by the thumb of the right hand when the grip unit 120 is held by the right hand (holding it with the little finger, ring finger, and middle finger of the right hand) so that the shutter button 101 can be pressed by the index finger of the right hand. That is, the touch bar 119 can be operated by placing the eyepiece unit 116 close to the eyepiece 116, looking into the eyepiece finder 117, and holding it in a position (shooting posture) so that the shutter button 101 can be pressed at any time. The touch bar 119 can receive a tap operation (operation of touching and releasing without moving within a predetermined period of time) on the touch bar 119, a slide operation to the left and right (operation of touching and then moving the touch position while keeping the touch), and the like. The touch bar 119 is an operation unit different from the touch panel 109, and does not have a display function. The touch bar 119 of this embodiment is a multi-function bar, and functions as, for example, an M-Fn bar.

[0021] The camera 100 also has a grip portion 120, a thumb rest portion 121, a terminal cover 122, a lid 123, a communication terminal 124, and the like.

[0022] Grip section 120 is a holding section formed in a shape that is easy for the user to hold with the right hand when holding camera 100. In a state in which camera 100 is held by gripping grip section 120 with the little finger, ring finger, and middle finger of the right hand, shutter button 101 and main electronic dial 104 are disposed in positions that can be operated with the index finger of the right hand. In a similar state, sub electronic dial 105 and touch bar 119 are disposed in positions that can be operated with the thumb of the right hand. Thumb rest section 121 (thumb standby position) is a grip section provided on the rear side of camera 100 at a position where it is easy to place the thumb of the right hand that is gripping grip section 120 when none of the operation sections are being operated.

[0023] The thumb rest 121 is made of a rubber member or the like for enhancing holding power (grip feeling). The terminal cover 122 protects connectors such as a connection cable that connects the camera 100 to an external device. The lid 123 protects the recording medium 227 and the slot by closing the slot for storing the recording medium 227 described later. The communication terminal 124 is a terminal for communicating with the detachable lens unit 200 side of the camera 100.

[0024] Fig. 2 is a diagram showing an example of the internal configuration of the camera 100. Note that the same components as those in Fig. 1A and Fig. 1B are given the same reference numerals and descriptions thereof will be omitted as appropriate. A lens unit 200 is attached to the camera 100.

[0025] First, a description will be given of the lens unit 200. The lens unit 200 is a type of interchangeable lens that is detachable from the camera 100. The lens unit 200 is a single lens, and is an example of a normal lens.

[0026] The lens unit 200 includes an aperture 201, a lens 202, an aperture drive circuit 203, an AF (autofocus) drive circuit 204, a lens system control circuit 205, a communication terminal 206, and the like.

[0027] The aperture diameter of the diaphragm 201 is adjustable. The lens 202 is made up of a plurality of lenses. The diaphragm drive circuit 203 adjusts the amount of light by controlling the aperture diameter of the diaphragm 201. The AF drive circuit 204 drives the lens 202 to adjust the focus.

[0028] The lens system control circuit 205 controls the aperture drive circuit 203, the AF drive circuit 204, etc., based on instructions from the system control unit 50, which will be described later. The lens system control circuit 205 controls the aperture 201 via the aperture drive circuit 203, and adjusts the focus by displacing the position of the lens 202 via the AF drive circuit 204. The lens system control circuit 205 is capable of communicating with the camera 100. Specifically, the camera 100 and the lens unit 200 communicate with each other via a communication terminal 206 of the lens unit 200 and a communication terminal 124 of the camera 100. The communication terminal 206 is a terminal through which the lens unit 200 communicates with the camera 100.

[0029] Next, a description will be given of the camera 100. The camera 100 has a shutter 210, an imaging unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, an EVF 217, a display unit 108, and a system control unit 50.

[0030] The shutter 210 is a focal plane shutter that can freely control the exposure time of the imaging unit 211 based on instructions from the system control unit 50.

[0031] The imaging unit 211 is an imaging element (image sensor) that converts an optical image into an electrical signal and is composed of a CCD, a CMOS element, etc. The imaging unit 211 may have an imaging surface phase difference sensor that outputs defocus amount information to the system control unit 50.

[0032] The A / D converter 212 converts the analog signal output from the imaging unit 211 into a digital signal.

[0033] The image processing unit 214 performs predetermined processing (pixel interpolation, resizing such as reduction, color conversion, etc.) on the data from the A / D converter 212 or the data from the memory control unit 213. The image processing unit 214 also performs predetermined arithmetic processing using the captured image data. The system control unit 50 performs exposure control and distance measurement control based on the obtained arithmetic results. This processing performs TTL (through-the-lens) AF processing, AE (automatic exposure) processing, EF (flash pre-flash) processing, etc. Furthermore, the image processing unit 214 performs predetermined arithmetic processing using the captured image data, and performs TTL AWB (auto white balance) processing based on the obtained arithmetic results.

[0034] The image data from the A / D converter 212 is written to the memory 215 via the image processing unit 214 and the memory control unit 213. Alternatively, the image data from the A / D converter 212 is written to the memory 215 via the memory control unit 213 without going through the image processing unit 214. The memory 215 stores image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212, and image data to be displayed on the display unit 108 and the EVF 217. The memory 215 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio. The memory 215 also serves as a memory for displaying images (video memory).

[0035] The D / A converter 216 converts the image display data stored in the memory 215 into an analog signal and supplies it to the display unit 108 or the EVF 217. Therefore, the display image data written in the memory 215 is displayed on the display unit 108 or the EVF 217 via the D / A converter 216. The display unit 108 and the EVF 217 perform display according to the analog signal from the D / A converter 216. The display unit 108 and the EVF 217 are, for example, displays such as an LCD or an organic EL. The digital signal that is A / D converted by the A / D converter 212 and stored in the memory 215 is converted into an analog signal by the D / A converter 216. The analog signal is then sequentially transferred to the display unit 108 or the EVF 217, whereby an image based on the analog signal is displayed. This realizes a live view display.

[0036] The system control unit 50 is a control unit consisting of at least one processor and / or at least one circuit. That is, the system control unit 50 may be a processor, a circuit, or a combination of a processor and a circuit. The system control unit 50 controls the entire camera 100. The system control unit 50 executes a program recorded in the non-volatile memory 219 to realize each process of the flowchart described later. The system control unit 50 also performs display control by controlling the memory 215, the D / A converter 216, the display unit 108, the EVF 217, and the like.

[0037] The camera 100 also includes a system memory 218 , a non-volatile memory 219 , a system timer 220 , a communication unit 221 , an attitude detection unit 222 , and an eye proximity detection unit 118 .

[0038] For example, a RAM is used as the system memory 218. In the system memory 218, constants and variables for the operation of the system control unit 50, programs read from the non-volatile memory 219, and the like are developed.

[0039] The non-volatile memory 219 is an electrically erasable and recordable memory. For example, an EEPROM is used as the non-volatile memory 219. Constants, programs, etc. for the operation of the system control unit 50 are recorded in the non-volatile memory 219.

[0040] The system timer 220 is a timing unit that measures the time used for various controls and the time of a built-in clock.

[0041] The communication unit 221 transmits and receives video signals or audio signals to and from an external device connected wirelessly or via a wired cable. The communication unit 221 can also connect to a wireless LAN (Local Area Network) or the Internet. The communication unit 221 can also communicate with an external device via Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 221 can transmit images (including live images) captured by the imaging unit 211 and images recorded in the recording medium 227. The communication unit 221 can receive image data or various other information from an external device.

[0042] The attitude detection unit 222 detects the attitude of the camera 100 with respect to the direction of gravity. Based on the attitude detected by the attitude detection unit 222, it is possible to determine whether the image captured by the imaging unit 211 is an image captured by holding the camera 100 horizontally or vertically. The system control unit 50 can add orientation information corresponding to the attitude detected by the attitude detection unit 222 to the image file of the image captured by the imaging unit 211. The system control unit 50 can also rotate and record the image according to the orientation information. For example, an acceleration sensor or a gyro sensor can be used as the attitude detection unit 222. It is also possible to detect the movement of the camera 100 (panning, tilting, lifting, whether it is stationary, etc.) using the attitude detection unit 222.

[0043] Eyepiece detection unit 118 can detect the approach of some object to eyepiece unit 116 of eyepiece finder 117 incorporating EVF 217. Eyepiece detection unit 118 can use, for example, an infrared proximity sensor. When an object approaches, infrared light projected from a light projecting unit of eyepiece detection unit 118 is reflected by the object and received by a light receiving unit of the infrared proximity sensor. Eyepiece detection unit 118 can determine the distance from eyepiece unit 116 to the object based on the amount of infrared light received. In this way, eyepiece detection unit 118 performs eyepiece detection to detect the proximity of the object to eyepiece unit 116.

[0044] The eye-contact detection unit 118 is an eye-contact detection sensor that detects the approach (eye-contact) and separation (eye-away) of an eye (object) to the eyepiece unit 116 of the eyepiece finder 117. When an object is detected approaching within a predetermined distance from the eyepiece unit 116 from a non-eye-contact state (non-approaching state), it detects that the eye has been placed near the object. On the other hand, when an object that was detected as approaching from the eye-contact state (approaching state) moves away by more than a predetermined distance, it detects that the eye has been removed. The threshold value for detecting the approach of the eye and the threshold value for detecting the removal of the eye may be different, for example, by providing a hysteresis. In addition, after the approach of the eye is detected, the eye is considered to remain in the eye-contact state until the removal of the eye is detected. After the removal of the eye is detected, the non-eye-contact state is considered to remain until the eye is detected.

[0045] The system control unit 50 switches between display (display state) / non-display (non-display state) of the display unit 108 and the EVF 217 according to the state detected by the eye proximity detection unit 118. Specifically, when at least the shooting standby state is present and the display destination switching setting is automatic switching, the display destination is set to the display unit 108 (the display of the display unit 108 is turned on) and the EVF 217 is hidden when the eye is not placed in contact with the camera. Furthermore, the system control unit 50 switches the display destination to the EVF 217 (the display of the EVF 217 is turned on) and the display unit 108 is hidden when the eye is placed in contact with the camera. Note that the eye proximity detection unit 118 is not limited to an infrared proximity sensor, and other sensors may be used as long as they can detect a state that can be regarded as being in contact with the eye.

[0046] The camera 100 also includes an outside-finder display unit 107, an outside-finder display drive circuit 223, a power supply control unit 224, a power supply unit 225, a recording medium I / F 226, an operation unit 228, and a video signal output unit 229. It has power I / F 240 etc.

[0047] The outside viewfinder display unit 107 displays various settings of the camera 100 such as the shutter speed and aperture via an outside viewfinder display drive circuit 223 .

[0048] The power supply control unit 224 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to be energized, etc. The power supply control unit 224 detects whether a battery is installed, the type of battery, and the remaining battery level. The power supply control unit 224 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50. In this way, the power supply control unit 224 supplies the necessary voltage to each unit including the recording medium 227 for the necessary period.

[0049] The power supply unit 225 is a primary battery (such as an alkaline battery or a lithium battery), a secondary battery (such as a NiCd battery, a NiMH battery, or a Li battery), an AC adapter, or the like.

[0050] The recording medium I / F 226 is an interface for connecting to a recording medium 227 (such as a memory card or a hard disk). The recording medium 227 is, for example, a memory card for recording captured images. The recording medium 227 is configured from a semiconductor memory, a magnetic disk, or the like. The recording medium 227 may be detachable from the camera 100, or may be built into the camera 100.

[0051] The operation unit 228 is an input unit that accepts operations from a user (user operations) and is used to input various instructions to the system control unit 50.

[0052] The operation unit 228 includes the shutter button 101, the power switch 102, the mode switching switch 103, the touch panel 109, and other operation units 229. The other operation units 229 include the main electronic dial 104, the sub electronic dial 105, the video button 106, the direction keys 110, the SET button 111, the AE lock button 112, the enlargement button 113, the playback button 114, the menu button 115, and the touch bar 119.

[0053] The shutter button 101 has a first shutter switch 230 and a second shutter switch 231. The first shutter switch 230 is turned on when the shutter button 101 is pressed halfway (instruction to prepare for shooting) during operation, and generates a first shutter switch signal SW1. In response to the generation of the first shutter switch signal SW1, the system control unit 50 starts preparation processing for shooting, such as AF processing, AE processing, AWB processing, and EF processing. The second shutter switch 231 is turned on when the shutter button 101 is pressed fully (instruction to shoot) and generates a second shutter switch signal SW2. In response to the generation of the second shutter switch signal SW2, the system control unit 50 starts a series of shooting processes (processing from reading out a signal from the imaging unit 211 to generating an image file including a shot image and writing it to the recording medium 227).

[0054] The mode changeover switch 103 changes the operation mode of the system control unit 50 to one of still image shooting mode, video shooting mode, playback mode, etc. Modes included in the still image shooting mode include auto shooting mode, auto scene determination mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). Modes included in the still image shooting mode include various scene modes and custom modes that provide shooting settings according to shooting scenes. The user can directly switch to one of the above-mentioned shooting modes using the mode changeover switch 103. Alternatively, the user can switch to a list screen of shooting modes using the mode changeover switch 103, and then selectively switch to one of the displayed modes using the operation unit 228. Similarly, the video shooting mode may include multiple modes.

[0055] Touch panel 109 is a touch sensor that detects various touch operations on the display surface of display unit 108 (operation surface of touch panel 109). Touch panel 109 and display unit 108 can be configured integrally. For example, touch panel 109 is attached to the upper layer of the display surface of display unit 108 so that light transmittance does not interfere with the display of display unit 108. Then, by associating input coordinates on touch panel 109 with display coordinates on the display surface of display unit 108, a GUI (Graphical User Interface) can be realized as if the user could directly operate the screen displayed on display unit 108.

[0056] The touch panel 109 can use any of various types such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, an optical sensor type, etc. Depending on the type, there are types that detect a touch by contact with the touch panel 109, and types that detect a touch by the approach of a finger or a pen to the touch panel 109, but any type may be used.

[0057] The system control unit 50 can detect the following operations or states on the touch panel 109. A finger or pen that has not been touching the touch panel 109 touches the touch panel 109 again, that is, the start of touching (hereinafter referred to as touch-down). A state in which the touch panel 109 is touched with a finger or a pen (hereinafter referred to as Touch-On). The touch panel 109 is moved while being touched by a finger or a pen (hereinafter referred to as Touch-Move). The finger or pen that has been touching the touch panel 109 is removed (released) from the touch panel 109, that is, the touch ends (hereinafter, referred to as "touch-up"). A state in which nothing is being touched on the touch panel 109 (hereinafter referred to as Touch-Off).

[0058] When touch-down is detected, touch-on is also detected at the same time. After touch-down, touch-on will usually continue to be detected unless touch-up is detected. If touch-move is detected, touch-on will also be detected at the same time. Even if touch-on is detected, touch-move will not be detected if the touch position does not move. After it is detected that all fingers or pens that were touching have touched up, touch-off will occur.

[0059] These operation / states and the position coordinates of the touch panel 109 touched by a finger or pen are notified to the system control unit 50 via the internal bus. The system control unit 50 determines what kind of operation (touch operation) has been performed on the touch panel 109 based on the notified information. Regarding touch-move, the moving direction of the finger or pen moving on the touch panel 109 can also be determined for each vertical and horizontal component on the touch panel 109 based on changes in the position coordinates. If a touch-move of a predetermined distance or more is detected, it is determined that a slide operation has been performed.

[0060] An operation in which a finger is touched on the touch panel 109, quickly moved a certain distance, and then released is called a flick. In other words, a flick is an operation in which a finger is quickly traced on the touch panel 109 as if flicking. When a touch-move of a certain distance or more at a certain speed or more is detected, and a touch-up is detected immediately after that, it is determined that a flick has been performed (it can be determined that a flick has occurred following a slide operation). Furthermore, when multiple A touch operation in which two points (for example, two points) are touched together (multi-touch) and the touch positions are brought closer together is called a pinch in, and a touch operation in which the touch positions are moved farther apart is called a pinch out. Pinch out and pinch in are collectively called pinch operations (or simply pinch).

[0061] Fig. 3 is a schematic diagram showing an example of the configuration of lens unit 300. Fig. 3 shows a state in which lens unit 300 is attached to camera 100. Note that, among the camera 100 shown in Fig. 3, the same configurations as those explained in Fig. 2 are given the same reference numerals and explanations thereof will be omitted as appropriate.

[0062] Lens unit 300 is a type of interchangeable lens that can be attached to and detached from camera 100. Lens unit 300 is a twin lens that can capture images with parallax between left and right images. Lens unit 300 has two optical systems, each with a wide viewing angle of approximately 180 degrees, and can capture the range of the front hemisphere. Specifically, the two optical systems of lens unit 300 can capture an object with a viewing field (angle of view) of 180 degrees in the left-right direction (horizontal angle, azimuth angle, yaw angle) and 180 degrees in the up-down direction (vertical angle, elevation angle, pitch angle).

[0063] The lens unit 300 has a right-eye optical system 301R having a plurality of lenses and a reflecting mirror, a left-eye optical system 301L having a plurality of lenses and a reflecting mirror, and a lens system control circuit 303. The right-eye optical system 301R corresponds to an example of a first optical system, and the left-eye optical system 301L corresponds to an example of a second optical system.

[0064] The right-eye optical system 301R and the left-eye optical system 301L have lenses 302R and 302L located on the subject side facing the same direction, and their optical axes are approximately parallel. The lens unit 300 of this embodiment is a lens for VR180 for capturing images for so-called VR180, which is a VR image format that allows two-eye stereoscopic viewing. The lens for VR180 has a fisheye lens that allows the right-eye optical system 301R and the left-eye optical system 301L to capture a range of approximately 180 degrees. The lens for VR180 may be a lens that can capture a wide viewing angle range of about 160 degrees, which is narrower than the range of 180 degrees, as long as the right-eye optical system 301R and the left-eye optical system 301L can acquire images that allow two-eye VR display as VR180. The VR180 lens can form a right image (first image) formed via the right eye optical system 301R and a left image (second image) formed via the left eye optical system 301L, which has parallax from the right image, on one or two imaging elements of the attached camera.

[0065] Furthermore, lens unit 300 is attached to camera 100 via lens mount portion 304 and camera mount portion 305 of camera 100. By attaching lens unit 300 to camera 100, system control unit 50 of camera 100 and lens system control circuit 303 of lens unit 300 are electrically connected to each other via communication terminal 124 and communication terminal 306.

[0066] In this embodiment, a right image formed through the right eye optical system 301R and a left image formed through the left eye optical system 301L having a parallax from the right image are imaged side by side on the imaging unit 211 of the camera 100. That is, two optical images formed by the right eye optical system 301R and the left eye optical system 301L are formed on one imaging element. The imaging unit 211 converts the imaged subject image (optical signal) into an analog electrical signal. In this way, by using the lens unit 300, two images having parallax can be simultaneously acquired (as a set) from two locations (optical systems) of the right eye optical system 301R and the left eye optical system 301L. In addition, by dividing the acquired image into an image for the left eye and an image for the right eye and displaying them in VR, the user can view a stereoscopic VR image in a range of approximately 180 degrees, so-called VR180.

[0067] Here, the VR image is an image that can be displayed in VR, which will be described later. , omnidirectional images (spherical images) captured by an omnidirectional camera (spherical camera), panoramic images having a wider image range (effective image range) than the display range that can be displayed on the display unit at one time, and the like. In addition, VR images are not limited to still images, but also include videos and live images (images acquired from a camera almost in real time). A VR image has an image range (effective image range) of a maximum field of view of 360 degrees in the left and right directions and 360 degrees in the up and down directions. In addition, VR images also include images having a wider angle of view than the angle of view that can be captured by a normal camera, or an image range that is wider than the display range that can be displayed on the display unit at one time, even if the field of view is less than 360 degrees in the left and right directions and less than 360 degrees in the up and down directions. An image captured by the camera 100 using the above-mentioned lens unit 300 is a type of VR image.

[0068] VR images can be displayed in VR by, for example, setting the display mode of a display device (a display device capable of displaying VR images) to "VR view." By displaying a VR image with a 360-degree angle of view and changing the position of the display device left and right (horizontal rotation direction), the user can view seamless omnidirectional images left and right.

[0069] Here, VR display (VR view) is a display method (display mode) capable of changing the display range, which displays an image of a VR image with a field of view according to the posture of the display device. VR display includes "single-eye VR display (single-eye VR view)" that displays one image by performing a transformation (transformation in which distortion correction is performed) that maps the VR image onto a virtual sphere. VR display also includes "two-eye VR display (two-eye VR view)" that displays a VR image for the left eye and a VR image for the right eye side by side in the left and right regions by performing a transformation that maps the VR image for the left eye and the VR image for the right eye, respectively, onto a virtual sphere. Stereoscopic vision is possible by performing "two-eye VR display" using a VR image for the left eye and a VR image for the right eye that have a parallax from each other. In any VR display, for example, when a user wears a display device such as an HMD (head-mounted display), an image with a field of view according to the orientation of the user's face is displayed.

[0070] For example, let us assume that, at a certain point in time, an image of a visual field range centered on 0 degrees in the left-right direction (a specific direction, for example, north) and 90 degrees in the up-down direction (90 degrees from the zenith, i.e. horizontal) is displayed in the VR image. When the attitude of the display device is inverted from this state (for example, the display surface is changed from facing south to facing north), the display range is changed to an image of the same VR image with a visual field range centered on 180 degrees in the left-right direction (opposite direction, for example, south) and 90 degrees in the up-down direction. That is, when the user turns his face from north to south (i.e. turns backwards) while wearing the HMD, the image displayed on the HMD is also changed from a north image to a south image. Note that the VR image captured using the lens unit 300 of this embodiment is a VR180 image captured in a range of approximately 180 degrees forward, and no image of a range of approximately 180 degrees backward exists. When such an image of VR180 is VR displayed and the attitude of the display device is changed to a side where no image exists, a blank area is displayed.

[0071] By displaying the VR image in this way, the user visually feels as if he or she is inside the VR image (in the VR space). The method of displaying the VR image is not limited to changing the posture of the display device. For example, the display range may be moved (scrolled) in response to user operation via a touch panel or directional buttons. Furthermore, during VR display (when in the "VR view" display mode), in addition to changing the display range due to posture changes, the display range may also be changed in response to touch-move on the touch panel, dragging with a mouse, pressing a directional button, etc. A smartphone attached to VR goggles (head-mounted adapter) is a type of HMD.

[0072] 4 is a block diagram showing the configuration of a personal computer (PC) 400, which is an electronic device according to the first embodiment. The PC 400 includes a control unit 401, a ROM 402, a RAM 403, an external storage device 404, an operation unit 405, a display unit 406, and a communication unit 407. Instead of the C400, any electronic device such as an imaging device (camera), a smartphone, or a tablet terminal may be used.

[0073] The control unit 401 controls the entire PC 400. The control unit 401 is, for example, a Central Processing Unit (CPU). In the first embodiment, the control unit 401 can operate as a display control unit that controls the display of the display unit 406, a switching unit that switches the display mode, and an adjustment unit that adjusts the display position of an image.

[0074] The ROM 402 is a Read Only Memory (ROM) that stores information (programs and parameters) that does not require modification. The ROM 402 stores a predetermined information processing program as program code that can be read by the control unit 401. The program code is executed by the control unit 401.

[0075] The RAM 403 is a random access memory (RAM) that temporarily stores information (programs and data) supplied from an external device or the like.

[0076] The external storage device 404 includes a hard disk or a flash memory that is fixedly installed in the PC 400. Alternatively, the external storage device 404 includes a memory card or the like that is detachable from the PC 400. Images and video files captured by the camera 100 are written to the external storage device 404.

[0077] The operation unit 405 receives operations from a user and includes operation members such as a mouse, a keyboard, or a touch panel.

[0078] The display unit 406 performs display based on data held in the PC 400 or data supplied to the PC 400. The display unit 406 may perform display based on data on an external display device by connecting to the external display device.

[0079] The communication unit 407 connects to the camera 100 or an external network.

[0080] The system bus 408 connects each component of the PC 400 to each other.

[0081] 5A to 6B, a process for displaying "an image based on a captured image of a real space" will be described. In the following, a display application is started, and a screen 500 (a screen of the display application) is displayed on the display unit 406.

[0082] The screen 500 includes a clip selection area 510 , an image display area 520 , a playback control area 530 , a check box 540 , a check box 550 , and an export button 560 .

[0083] A list of moving image files loaded into the display application is displayed in clip selection area 510. The user selects a moving image file to be played (confirmed) from the list of moving image files (multiple moving image files) in clip selection area 510. Here, each moving image file is a moving image file (moving image) acquired by imaging with camera 100.

[0084] In the image display area 520, an arbitrary frame (one still image) of the moving image file selected in the clip selection area 510 is displayed in a display form according to the display mode.

[0085] Here, when the display application is activated, the display mode is switched among a plurality of modes in response to a user's operation. The plurality of modes includes a "fisheye fit mode" which displays a binocular image including two circular fisheye images in a fitted manner, as shown in FIG. 5A. The mode includes an "equirectangular fit mode" that displays two images (hereinafter referred to as "equirectangular images") in a fitted manner, which are obtained by performing a geometric transformation process (a process for reducing distortion) on two circular fisheye images using the equirectangular projection method, as shown in Fig. 5B. The "fit display" refers to displaying an image that has been reduced so that the entire image fits within the image display area 520.

[0086] The multiple modes include a "fisheye magnification mode" that displays a partial range of a twin-eye image as shown in Fig. 6A, and a "equidistance magnification mode" that displays a partial range of a side-by-side image (equidistance cylindrical image) as shown in Fig. 6B. When a partial range of an image is displayed, for example, the partial range of the image is displayed dot-by-dot (at 100% magnification). Dot-by-dot refers to a format in which an image is displayed by matching one pixel of the image to one pixel of the display without thinning out or compressing the image information.

[0087] In the first embodiment, the image size of the circular fisheye image and the image size of the equirectangular image are the same. For example, if the image size of one circular fisheye image is 4096 x 4096, the image size of one equirectangular image is also 4096 x 4096. On the other hand, the image size of the entire twin-lens image is twice the size of one circular fisheye image (=8192 x 4096).

[0088] In addition, the two circular fisheye images (left eye image and right eye image) are recorded in a state where the left and right are swapped in the camera 100. Therefore, when displaying the two-eye image, the two circular fisheye images are displayed in the image display area 520 after the left and right are swapped so that the left eye image and the right eye image are correctly arranged.

[0089] Also, a drag operation by the user is accepted in image display area 520. When a part of an image is displayed (when a check box 550, which will be described later, is checked), the user can specify the range (range of the image) to be displayed in image display area 520 by a drag operation.

[0090] In the playback control area 530, operations for controlling the playback of the video file selected in the clip selection area 510 (such as starting playback, pausing, forwarding frame, reversing frame, and seeking to any frame) are possible.

[0091] Depending on whether or not the user checks the check box 540, it is possible to switch between a circular fisheye image and an equirectangular image to be displayed in the image display area 520.

[0092] The display form of the image can be changed depending on whether or not the user checks the check box 550. Depending on whether or not the user checks the check box 550, it is possible to switch between displaying a part of the image in the image display area 520 or displaying the image in a fitted manner.

[0093] Specifically, if check box 540 is checked and check box 550 is not checked, the display mode is set to "fisheye fit mode" as shown in Fig. 5A. If check box 540 is not checked and check box 550 is not checked, the display mode is set to "equal distance fit mode" as shown in Fig. 5B.

[0094] If the check box 540 is checked and the check box 550 is checked, the display mode is set to "fisheye magnification mode" as shown in FIG. 6A. If the check box 540 is not checked and the check box If check box 550 is checked, the display mode is set to "equal distance expansion mode" as shown in Fig. 6B. In the following, the state in which the check box is checked will be simply referred to as "the check box is in the ON state." The state in which the check box is not checked will be simply referred to as "the check box is in the OFF state."

[0095] When the export button 560 is pressed, an equirectangular image (an image in which correction has been applied to the circular fisheye image to reduce distortion due to equirectangular projection conversion) is saved as a separate file.

[0096] On screen 500, a display item (such as a controller) that allows the user to specify an arbitrary magnification ratio may be displayed in addition to check box 550. Also, an image that is enlarged (an image with a higher display magnification ratio) than that displayed dot-by-dot may be displayed in image display area 520.

[0097] In FIG. 6A, the display position of the binocular image is adjusted by the user so that the subject 610 of the right eye image is displayed (projected) at the center of the image display area 520. Here, the upper left vertex of the entire binocular image (i.e., an image of 8192×4096) is set as the origin (0,0), and the coordinates of the center of the subject 610 are coordinates (X1, Y1). In this state, when the user switches the check box 540 to the ON state, if the display position of the parallel image (subject 610) is not adjusted, the range of the coordinates (X1, Y1) in the parallel image will be displayed at the center of the image display area 520. Therefore, as shown in FIG. 6B, the subject 610 is displayed (projected) shifted from the center of the image display area 520.

[0098] Next, the operation of the PC 400 according to the first embodiment will be described with reference to the flowchart of Fig. 7. The processing of the flowchart of Fig. 7 can be realized by the control unit 401 executing a program.

[0099] In step S701, the control unit 401 determines whether or not one of the one or more video files stored in the external storage device 404 has been selected by the user. If it is determined that one video file has been selected, the process proceeds to step S702. If it is determined that one video file has not been selected, the process of step S701 is repeated.

[0100] In step S702, control unit 401 displays an image included in the moving image file selected in step S701 (hereinafter referred to as a "selected image") in image display area 520. At this time, control unit 401 displays all or a part of the selected image in image display area 520 in a display mode according to the states of check boxes 540 and 550. Note that in the first embodiment, the selected image is a twin-lens image or a side-by-side image.

[0101] In step S703, the control unit 401 determines whether the check box 550 is ON (=part of the image is being displayed). If it is determined that the check box 550 is ON, the process proceeds to step S704. If it is determined that the check box 550 is OFF, the process proceeds to step S710.

[0102] In step S704, the control unit 401 determines whether or not a command to move the display position of the selected image has been issued by a drag operation by the user. If it is determined that a command to move has been issued, the process proceeds to step S705. If it is determined that a command to move has not been issued, the process proceeds to step S706.

[0103] In step S705, the control unit 401 moves the display position of the selected image in response to a movement instruction (a drag operation by the user).

[0104] In step S706, the control unit 401 acquires the coordinates (X1, Y1) of the selected image corresponding to the center position of the image display area 520.

[0105] In step S707, the control unit 401 determines whether the check box 540 has been switched between ON / OFF (i.e., an instruction to switch the display mode between "fisheye magnification mode" and "equal distance magnification mode") or not. If it is determined that the check box 540 has been switched between ON / OFF, the process proceeds to step S708. If it is determined that the check box 540 has not been switched between ON / OFF, the process proceeds to step S710.

[0106] In step S708, the control unit 401 calculates the coordinates (X2, Y2) of the selected image after the display mode is switched, which correspond to the coordinates (X1, Y1) of the selected image before the display mode is switched. Here, the selected image before the display mode is switched is the image (two-eye image or side-by-side image) displayed in the image display area 520 before the display mode is switched. The selected image after the display mode is switched is the image (two-eye image or side-by-side image) displayed in the image display area 520 after the display mode is switched.

[0107] For example, coordinates (X2, Y2) in the equirectangular image corresponding to coordinates (X1, Y1) in the circular fisheye image can be found by a known geometric transformation technique of the equirectangular projection. When calculating coordinates in the circular fisheye image corresponding to coordinates in the equirectangular image, a technique of inverse transformation of the geometric transformation of the equirectangular projection can be used. Note that a two-dimensional table indicating the corresponding coordinates between the circular fisheye image and the equirectangular image may be generated in advance, and the two-dimensional table may be stored in the RAM 403 or the external storage device 404. The two-dimensional table may be referenced when calculating coordinates when switching the display mode.

[0108] In step S709, the control unit 401 adjusts (determines) the display position of the selected image after the display mode is switched so that the coordinates (X2, Y2) calculated in step S708 are positioned at the center position of the image display area 520. Then, the control unit 401 displays the selected image in a form according to the switched display mode at the adjusted display position. This allows the user to confirm the subject that the user confirmed at the center of the image display area 520 before the display mode was switched, at the center of the image display area 520 even after the display mode is switched between the fisheye magnification mode and the equidistant magnification mode.

[0109] For example, as shown in Fig. 6A, subject 610 that appears at the center of image display area 520 in fisheye magnification mode remains at the center of image display area 520 even after switching to equidistance magnification mode, as shown in Fig. 8. In other words, the display positions of the parallel images are controlled (adjusted) so that the center coordinates (X2, Y2) of subject 610 in the parallel images are at the center of image display area 520.

[0110] Note that if the display position of the selected image in the switched display mode is not adjusted (if the process of step S709 is not performed), the coordinates (X1, Y1) of the selected image in the switched display mode are placed at the center position of the image display area 520 (see FIG. 6B). In other words, the same coordinates of the image are placed at the center position of the image display area 520 both before and after the display mode is switched. For this reason, there is a possibility that a different object (subject) will be displayed at the center position of the image display area 520 before and after the display mode is switched.

[0111] In step S710, the control unit 401 determines whether or not a display end instruction (such as an operation to close the screen 500) has been issued by the user. If it is determined that a display end instruction has been issued, the process of this flowchart ends. If it is determined that a display end instruction has not been issued, If so, the process returns to step S703.

[0112] According to embodiment 1, when switching the display mode between fisheye magnification mode and equidistant magnification mode, the subject (target object) that was displayed at the center of image display area 520 before switching the display mode can be displayed at the center position of image display area 520 even after switching the display mode.

[0113] Note that the first embodiment is applicable not only to the case where the display mode is switched from the fisheye magnification mode to the equidistant magnification mode, but also to the case where the display mode is switched from the equidistant magnification mode to the fisheye magnification mode.

[0114] <Embodiment 2> In the second embodiment, a case will be described in which the image display area 520 includes areas of two images in the fisheye magnification mode and the equidistant magnification mode. The imaging device (camera and lens unit) according to the second embodiment has the same configuration as the imaging device according to the first embodiment, and records the same video file as in the first embodiment. Also, the configuration of the PC 400 according to the second embodiment is the same as the configuration of the PC 400 according to the first embodiment.

[0115] Fig. 9A shows the screen 500 in fisheye fit mode, and Fig. 9B shows the screen 500 in fisheye magnification mode.

[0116] In the following, the area of ​​the right eye image in the twin image (image captured by imaging through the right eye optical system 301R) and the area of ​​the image corresponding to the right eye image in the parallel image (right equirectangular image) are both referred to as the "right eye area." The area of ​​the left eye image in the twin image (image captured by imaging through the left eye optical system 301L) and the area of ​​the image corresponding to the left eye image in the parallel image (left equirectangular image) are both referred to as the "left eye area."

[0117] In the image display area 520 of FIG. 9B, the range inside the frame 910 shown in FIG. 9A is displayed dot by dot. Inside the frame 910, there are included a subject located to the right of the left eye image and a subject located to the left of the right eye image. These subjects have low correlation. For this reason, in the state of FIG. 9B, the user usually focuses on the subject in the right eye area, which is a larger area in the image display area 520.

[0118] The operation of the PC 400 according to the second embodiment will be described with reference to the flowchart of Fig. 10. Steps S1001 and S1002 in Fig. 10 are similar to steps S701 and S702 in Fig. 7 according to the first embodiment, and therefore description thereof will be omitted.

[0119] In step S1003, PC 400 determines whether check box 550 is ON (part of the image is being displayed). If it is determined that check box 550 is ON, the process proceeds to step S1004. If it is determined that check box 550 is not ON, the process proceeds to step S1013.

[0120] In step S1004, the control unit 401 determines whether or not an instruction to move the display position of the selected image has been issued by a user's drag operation on the image display area 520. If it is determined that an instruction to move the display position has been issued, the process proceeds to step S1005. If it is determined that an instruction to move the display position has not been issued, the process proceeds to step S1006.

[0121] In step S1005, the control unit 401 moves the display position of the selected image in response to a movement instruction from the user (the amount of operation of the drag operation).

[0122] In step S1006, the control unit 401 displays the left eye region and the right eye region in the image display area 520. It is determined whether or not both the left eye region and the right eye region are included in the image display area 520. If it is determined that both the left eye region and the right eye region are included in the image display area 520, the process proceeds to step S1007. If it is determined that only one of the left eye region and the right eye region is included in the image display area 520, the process proceeds to step S1013. Note that if only one of the left eye region and the right eye region is included in the image display area 520, the display position can be controlled by the process described in the first embodiment (the process of steps S706 to S709).

[0123] In step S1007, the control unit 401 calculates the area of ​​the left eye region in the image display area 520 and the area of ​​the right eye region in the image display area 520.

[0124] In step S1008, the control unit 401 determines which of the left eye region and the right eye region has the larger area calculated in step S1007 (maximum display region). The control unit 401 acquires the coordinates (xd, yd) of the center position of the range of the maximum display region displayed in the image display region 520. The coordinates (xd, yd) are coordinates in the image display region 520.

[0125] In step S1009, control unit 401 acquires coordinates (X3, Y3) in the entire image displayed in image display area 520, which correspond to the position of coordinates (xd, yd).

[0126] FIG. 11A is a diagram for explaining step S1008 and step S1009 in detail. FIG. 11A shows a screen 500 in fisheye magnification mode. In FIG. 11A, coordinates (x0, y0) are coordinates (0, 0) of the origin of a coordinate system in the image display area 520 (hereinafter referred to as the "display coordinate system"). The x coordinate xa is the x coordinate of the boundary between the left eye image and the right eye image in the display coordinate system. The x coordinate xb is the maximum value of the x coordinate in the display coordinate system, and the y coordinate yb is the maximum value of the y coordinate in the display coordinate system. Here, the coordinates (xd, yd) can be calculated by the following formula. xd=(xa+xb) / 2 yd=(y0+yb) / 2

[0127] 11A, a subject 1110 is displayed at a position of coordinates (xd, yd) in the image display area 520. Then, coordinates (X3, Y3) of the position of the subject 1110 in the coordinate system (image coordinate system) of the entire twin-eye image including the left eye image and the right eye image are acquired.

[0128] In step S1010, control unit 401 determines whether an instruction to switch the ON / OFF state of check box 540 (i.e., an instruction to switch between fisheye magnification mode and regular distance magnification mode) has been given. If an instruction to switch the ON / OFF state of check box 540 has been given, the process proceeds to step S1011. If an instruction to switch the ON / OFF state of check box 540 has not been given, the process proceeds to step S1013.

[0129] In step S1011, the control unit 401 calculates the coordinates (X4, Y4) of the selected image after the display mode is switched, which correspond to the coordinates (X3, Y3) of the selected image before the display mode is switched.

[0130] In step S1012, control unit 401 adjusts the display position of the selected image after the display mode is switched so that the coordinates (X4, Y4) of the selected image after the display mode is switched are positioned at the position of coordinates (xd, yd) in image display area 520. Then, control unit 401 displays the selected image after the display mode is switched at the adjusted display position.

[0131] This allows the user to check the same subject at the same position in the image display area 520 even when the display mode is switched between the fisheye magnification mode and the equidistant magnification mode. The subject that was confirmed at the center position of the range shown in one region (the region having the larger area of ​​the two regions) can be confirmed at the same position in image display region 520 even after the display mode is switched.

[0132] Fig. 11B shows screen 500 after the display mode is switched to the equidistant magnification mode from the state shown in Fig. 11A. In Fig. 11B, subject 1110 at coordinates (X4, Y4) in the entire parallel image is displayed at a position of coordinates (xd, yd) in image display area 520.

[0133] In step S1013, the control unit 401 determines whether or not a display end instruction (such as an operation to end the screen 500) has been given by the user. If it is determined that a display end instruction has been given, the process of this flowchart ends. If it is determined that a display end instruction has not been given, the process returns to step S1003.

[0134] In the second embodiment, it is determined whether or not both the left eye region and the right eye region of the image before switching the display mode between the fisheye magnification mode and the equidistant magnification mode are included in the image display area 520. Then, if both the left eye region and the right eye region are included in the image display area 520, the subject at the center position of the range in which one region is shown in the image display area 520 before switching is displayed at the same position in the image display area 520 after switching.

[0135] If the user does not move the display position in step S1005, the processes in steps S1007 to S1008 may not be performed, and previously acquired coordinate (xd, yd) information may be used in steps S1009 and after. This makes it possible to fix the display position of the subject.

[0136] <Variation 1> In each embodiment, an example of handling a moving image file has been described, but instead of the moving image file, a still image file in a format such as JPEG or TIFF can also be used.

[0137] Also, in the first embodiment, an example has been shown in which a subject that was displayed at the center position of the image display area 520 before the display mode was switched is still displayed at the center position of the image display area 520 after the display mode is switched. However, the position of the image display area 520 where the subject is fixed is not limited to the center of the image display area 520 and may be another position. For example, the position of the image display area 520 where the subject is fixed may be a position designated by a user by a mouse operation or a touch panel operation. The position of the image display area 520 where the subject is fixed may be the position of a specific object (subject) detected by known object detection or object recognition.

[0138] In addition, in the fisheye magnification mode, when the area outside the circular fisheye is captured in a range that is greater than a certain percentage of the image display area 520, the control unit 401 may not adjust the display position as described in each embodiment. This is because when the display mode is switched and display is performed in the equirectangular magnification mode, coordinates corresponding to the coordinates before the display mode is switched may not exist in the equirectangular image. Alternatively, in this case, the control unit 401 may display a warning message (for example, a message that prompts the user to move the display position of the image). Here, the area outside the circular fisheye is an area other than the "area captured via the optical system." The area outside the circular fisheye is, for example, an area in which no subject is captured, such as the black area 570 in FIG. 9.

[0139] In addition, in the fisheye magnification mode, most of the area of ​​the twin-eye image (area larger than a certain percentage) may be displayed in the image display area 520. For example, when a high-resolution display device (such as a display with 8K resolution) is used and the screen 500 is displayed as large as possible, In this case, even if the display mode is subsequently switched, the position of the subject is unlikely to change significantly. Therefore, even if the display mode is switched from the fisheye magnification mode to the equidistant magnification mode, the control unit 401 may not adjust the display position of the parallel images after the display mode is switched.

[0140] Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

[0141] Also, in the above, "If A is equal to or greater than B, proceed to step S1, and if A is smaller (lower) than B, proceed to step S2" may be read as "If A is greater (higher) than B, proceed to step S1, and if A is equal to or less than B, proceed to step S2." Conversely, "If A is greater (higher) than B, proceed to step S1, and if A is equal to or less than B, proceed to step S2" may be read as "If A is greater (higher) than B, proceed to step S1, and if A is smaller (lower) than B, proceed to step S2." Therefore, unless a contradiction occurs, "equal to or greater than A" may be read as "equal to or greater than A (high; long; many)," and "equal to or less than A" may be read as "equal to or less than A (low; short; few)." And, "equal to or greater than A" may be read as "equal to or greater than A," and "equal to or less than A" may be read as "equal to or less than A."

[0142] Each functional unit in each of the above embodiments (variations) may or may not be individual hardware. The functions of two or more functional units may be realized by common hardware. Each of a plurality of functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Furthermore, each functional unit may or may not be realized by hardware such as an ASIC, FPGA, or DSP. For example, the device may have a processor and a memory (storage medium) in which a control program is stored. Then, the functions of at least some of the functional units of the device may be realized by the processor reading and executing the control program from the memory.

[0143] (Other embodiments) The present invention can also be realized by supplying a program for implementing one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0144] The disclosure of the above embodiments includes the following configurations, methods, and programs. (Configuration 1) An acquisition means for acquiring a first image and a second image obtained by subjecting the first image to a geometric transformation process for reducing distortion; a switching means for switching a display mode between a first mode in which a partial range of the first image is displayed in a display area and a second mode in which a partial range of the second image is displayed in the display area; an adjustment means for adjusting, when the display mode is switched, a display position of an image displayed in the display area after the display mode is switched, so that an object displayed in a first position in the display area before the display mode is switched is displayed in the first position after the display mode is switched; 1. An electronic device comprising: (Configuration 2) When the display mode is switched, the adjustment means acquiring first coordinates that are coordinates of the object in an image displayed in the display area before the display mode is switched; acquiring second coordinates, which are coordinates of the object in an image displayed in the display area after the display mode is switched, based on the first coordinates; adjusting a display position of the image displayed in the display area after the switching of the display mode so that the second coordinate is located at the first position; 2. The electronic device according to configuration 1. (Configuration 3) In the first mode, the first image is displayed dot-by-dot; In the second mode, the second image is displayed dot-by-dot. 3. The electronic device according to configuration 1 or 2. (Configuration 4) The first position is a center position of the display area. 4. The electronic device according to any one of configurations 1 to 3. (Configuration 5) The first location is a location specified by a user. 4. The electronic device according to any one of configurations 1 to 3. (Configuration 6) the first position is a position where a specific subject is captured before the display mode is switched; 4. The electronic device according to any one of configurations 1 to 3. (Configuration 7) the first image includes a first region acquired by imaging via a first optical system and a second region acquired by imaging via a second optical system; the second image includes a third region which is an area obtained by performing the geometric transformation process on the first region, and a fourth region which is an area obtained by performing the geometric transformation process on the second region; 7. The electronic device according to any one of configurations 1 to 6. (Configuration 8) When the display mode is switched from the first mode to the second mode, and the first region and the second region are displayed in the display area before the display mode is switched, the first position is a center position of a range in which one of the first region and the second region, which has a larger range included in the display area before the display mode is switched, is displayed. 8. The electronic device according to configuration 7. (Configuration 9) when the display mode is switched from the first mode to the second mode, and the display area before the display mode is switched includes an area that is neither the first area nor the second area more than a certain ratio of the display area, the adjustment means does not adjust a display position of an image displayed in the display area after the display mode is switched. 9. The electronic device according to configuration 7 or 8. (Configuration 10) when a range displayed in the display area before the switching of the display mode is a range larger than a predetermined ratio of the entire image including the range, the adjustment means does not adjust a display position of the image displayed in the display area after the switching of the display mode. 10. The electronic device according to any one of configurations 1 to 9. (Configuration 11) In a case where the adjustment means does not adjust the display position of the image displayed in the display area after the display mode is switched, if the coordinates of the object in the image displayed in the display area before the display mode is switched are third coordinates, the third coordinates in the image displayed in the display area are placed at the first position, 11. The electronic device according to any one of configurations 1 to 10. (method) acquiring a first image and a second image obtained by subjecting the first image to a geometric transformation process that reduces distortion; a switching step of switching a display mode between a first mode in which a partial range of the first image is displayed in a display area and a second mode in which a partial range of the second image is displayed in the display area; an adjustment step of adjusting, when the display mode is switched, a display position of an image displayed in the display area after the display mode is switched so that an object displayed in a first position in the display area before the display mode is switched is displayed in the first position after the display mode is switched; 13. A method for controlling an electronic device comprising: (program) A program for causing a computer to function as each of the means of the electronic device described in any one of configurations 1 to 11. [Explanation of symbols]

[0145] 400: PC, 401: control unit, 406: display unit

Claims

1. An acquisition means for acquiring a first image and a second image obtained by subjecting the first image to a geometric transformation process for reducing distortion; a switching means for switching a display mode between a first mode in which a partial range of the first image is displayed in a display area and a second mode in which a partial range of the second image is displayed in the display area; an adjustment means for adjusting, when the display mode is switched, a display position of an image displayed in the display area after the display mode is switched, so that an object displayed in a first position in the display area before the display mode is switched is displayed in the first position after the display mode is switched; 1. An electronic device comprising:

2. When the display mode is switched, the adjustment means acquiring first coordinates that are coordinates of the object in an image displayed in the display area before the display mode is switched; acquiring second coordinates, which are coordinates of the object in an image displayed in the display area after the display mode is switched, based on the first coordinates; adjusting a display position of an image displayed in the display area after the switching of the display mode so that the second coordinate is located at the first position; 2. The electronic device according to claim 1 .

3. In the first mode, the first image is displayed dot-by-dot; In the second mode, the second image is displayed dot by dot.

3. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes.

4. The first position is a center position of the display area.

3. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes.

5. The first location is a location specified by a user.

3. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes.

6. the first position is a position at which a specific subject is captured before the display mode is switched; 3. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes.

7. the first image includes a first region acquired by imaging via a first optical system and a second region acquired by imaging via a second optical system; the second image includes a third region which is an area obtained by performing the geometric transformation process on the first region, and a fourth region which is an area obtained by performing the geometric transformation process on the second region; 3. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes.

8. When the display mode is switched from the first mode to the second mode, and the first region and the second region are displayed in the display area before the display mode is switched, the first position is a center position of a region, which is larger than the first region and the second region and is included in the display area before the display mode is switched, displayed in the display area.

8. The electronic device according to claim 7,

9. the adjustment means, when switching from the first mode to the second mode, if the display area before the display mode switching includes an area that is neither the first area nor the second area more than a certain ratio of the display area, does not adjust a display position of an image displayed in the display area after the display mode switching; 8. The electronic device according to claim 7,

10. when a range displayed in the display area before the switching of the display mode is a range larger than a predetermined ratio of the entire image including the range, the adjustment means does not adjust a display position of the image displayed in the display area after the switching of the display mode.

3. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes.

11. when the adjustment means does not adjust the display position of the image displayed in the display area after the display mode is switched, if the coordinates of the object in the image displayed in the display area before the display mode is switched are third coordinates, the third coordinates in the image displayed in the display area after the display mode is switched are arranged at the first position.

3. The electronic device according to claim 1, wherein the first and second electrodes are electrically connected to the first and second electrodes.

12. acquiring a first image and a second image, the second image being an image obtained by performing a geometric transformation process on the first image to reduce distortion; a switching step of switching a display mode between a first mode in which a partial range of the first image is displayed in a display area and a second mode in which a partial range of the second image is displayed in the display area; an adjustment step of adjusting, when the display mode is switched, a display position of an image displayed in the display area after the display mode is switched so that an object displayed in a first position in the display area before the display mode is switched is displayed in the first position after the display mode is switched; 13. A method for controlling an electronic device comprising:

13. A program for causing a computer to function as each of the means of the electronic device according to claim 1 or 2.

Images