Electronic apparatus

Abstract

To provide a technique capable of facilitating a setting of a camera while mounting a HMD by a user.SOLUTION: An electronic apparatus of the present invention includes: acquisition means of acquiring a position coordinate designated by a user in a display image in which first conversion processing is executed to a captured image obtained by an imaging device; conversion means of converting the position coordination in the display image obtained by the acquisition means into a position coordination in the captured image by second conversion processing that is inverted to the first conversion processing; and control means of controlling the setting of the imaging device on the basis of the position coordination in the captured image obtained by the conversion means.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to electronic devices, and more particularly to a technique for setting an imaging device. [Background technology]

[0002] A technology has been proposed in which two images with parallax are acquired using two optical systems, and the two images are displayed in a stereoscopic manner on a head-mounted display device (head-mounted display (HMD)) such as VR goggles or AR glasses. A technology has also been proposed in which a digital camera (camera) is connected to an external display device, and an image captured by the digital camera (live view image) is displayed on the display device (Patent Document 1). A technology (xR live view) has also been proposed in which the live view image is displayed on an HMD (Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-183845 [Patent Document 2] Patent Publication No. 2022-082127 Summary of the Invention [Problem to be solved by the invention]

[0004] A user checking the live view video on the HMD needs to remove the HMD in order to operate the camera (to view the camera). Therefore, the user cannot easily set the camera. Patent Document 2 describes that the user operates the camera while wearing the AR glasses. In the case of the AR glasses, the user can see the real space (the camera in the real space) through the live view video. However, the visibility of the real space is reduced by the live view video. Therefore, in the case of the AR glasses, the user cannot easily set the camera.

[0005] An object of the present invention is to provide a technique that enables a user to easily configure camera settings while wearing an HMD. [Means for solving the problem]

[0006] The electronic device of the present invention is characterized by having an acquisition means for acquiring position coordinates specified by a user in a display image obtained by applying a first conversion process to an image obtained by an imaging device, a conversion means for converting the position coordinates in the display image obtained by the acquisition means into position coordinates in the imaged image by a second conversion process that is the opposite of the first conversion process, and a control means for controlling the settings of the imaging device based on the position coordinates in the imaged image obtained by the conversion means. Effect of the Invention

[0007] According to the present invention, the user can easily configure the camera settings while wearing the HMD. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram showing a system configuration. [Diagram 2] FIG. [Diagram 3] FIG. 2 is a block diagram of the camera. [Figure 4] FIG. 2 is a schematic diagram showing a configuration of a lens unit. [Diagram 5] 1 is an external view of an electronic device. [Figure 6] 11 is a flowchart showing the processing of the camera. [Figure 7] 13 is a flowchart showing the processing of the VR goggles. [Figure 8] 11A and 11B are schematic diagrams showing left-right swapping processing; [Figure 9] 13 is a flowchart showing the processing of the camera. [Figure 10] 13 is a flowchart showing the processing of the VR goggles. [Figure 11]FIG. 2 is a schematic diagram showing a display screen. [Figure 12] FIG. 11 is a schematic diagram for explaining a conversion process of position coordinates. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0010] <System configuration> FIG. 1 is a schematic diagram showing an example of the configuration of a system according to this embodiment. The system of FIG. 1 includes a digital camera (camera) 100 and VR goggles 530. A lens unit 300 is attached to the camera 100. The camera 100 and the lens unit 300 are connected to each other so that they can communicate with each other, and the lens unit 300 transmits lens information (described later) to the digital camera 100. The camera 100 and the VR goggles 530 are connected to each other so that they can communicate with each other wirelessly or by wire, and transmit and receive various information and data such as lens information and live view images (live view image data). When the camera 100 transmits a live view image to the VR goggles 530, the live view image is displayed on a display 505 (described later) of the VR goggles 530 (VR live view). At this time, in response to an instruction from a user, a CPU 501 (described later) of the VR goggles 530 performs various processes such as image processing on the live view image, and displays the processing results on the display 505 of the VR goggles 530.

[0011] The display device connected to the camera 100 is not limited to the VR goggles 530, and may be, for example, another head-mounted display device (head-mounted display (HMD)) such as AR glasses. Any display device capable of VR display (described later) may be connected to the camera 100. The display device may be directly connected to the camera 100, or may be connected to the camera 100 via another electronic device (for example, an information processing device such as a personal computer).

[0012] <External configuration of the camera> 2(A) and 2(B) are external views showing an example of the external configuration of camera 100. Fig. 2(A) is a perspective view of camera 100 seen from the front side, and Fig. 2(B) is a perspective view of camera 100 seen from the rear side.

[0013] The camera 100 has a shutter button 101, a power switch 102, a mode switch 103, a main electronic dial 104, a sub electronic dial 105, a movie button 106, and a viewfinder display 107 on the top surface. The shutter button 101 is an operation member for issuing a shooting preparation instruction or a shooting instruction. The power switch 102 is an operation member for switching the power of the camera 100 on and off. The mode switch 103 is an operation member for switching between various modes. The main electronic dial 104 is a rotary operation member for changing settings such as shutter speed and aperture. The sub electronic dial 105 is a rotary operation member for moving a selection frame (cursor), forwarding images, and the like. The movie button 106 is an operation member for issuing an instruction to start or stop movie shooting (recording). The viewfinder display 107 displays various settings such as shutter speed and aperture.

[0014] The camera 100 has, on the back thereof, a display unit 108, a touch panel 109, direction keys 110, a SET button 111, an AE lock button 112, a magnification button 113, a playback button 114, a menu The camera includes a menu button 115, an eyepiece unit 116, an eyepiece detection unit 118, and a touch bar 119. The display unit 108 displays various images and various information. The touch panel 109 is an operation member that detects a touch operation on the display surface (touch operation surface) of the display unit 108. The direction key 110 is an operation member that is composed of keys (four-way keys) that can be pressed up, down, left, and right. Processing according to the position where the direction key 110 is pressed can be performed. The SET button 111 is an operation member that is pressed mainly when deciding a selection item. The AE lock button 112 is an operation member that is pressed when fixing an exposure state in a shooting standby state. The enlargement button 113 is an operation member for switching the enlargement mode on and off in the live view display (LV display) in the shooting mode. When the enlargement mode is on, the live view image (LV image) is enlarged or reduced by operating the main electronic dial 104. The enlargement button 113 is also used when enlarging a playback image or increasing the magnification ratio in the playback mode. The playback button 114 is an operating member for switching between a shooting mode and a playback mode. When the playback button 114 is pressed in the shooting mode, the mode changes to the playback mode, and the latest image among the images recorded in the recording medium 227 described later can be displayed on the display unit 108.

[0015] Menu button 115 is an operation member that is pressed to display a menu screen on display unit 108 that allows various settings to be made. A user can intuitively make various settings using the menu screen displayed on display unit 108, direction key 110, and SET button 111. Eyepiece unit 116 is a portion that is used to put one's eye close to eyepiece finder (peek-in type finder) 117 and peer into it. Through eyepiece unit 116, a user can view an image displayed on an EVF 217 (Electronic View Finder) (described later) inside camera 100. Eyepiece detection unit 118 is a sensor that detects whether or not the user has put his / her eye close to eyepiece unit 116 (eyepiece finder 117).

[0016] The touch bar 119 is a line-shaped touch operation member (line touch sensor) capable of receiving a touch operation. The touch bar 119 is disposed at a position where it can be touched (touched) by the thumb of the right hand when the grip unit 120 is held in the right hand (held with the little finger, ring finger, and middle finger of the right hand) so that the shutter button 101 can be pressed with the index finger of the right hand. That is, the touch bar 119 can be operated in a state (shooting posture) where the eyepiece unit 116 is looked into by putting the eyepiece 117 close to the eyepiece finder 117 and the user is ready to press the shutter button 101 at any time. The touch bar 119 can receive a tap operation (operation of touching and releasing the touch position without moving it within a predetermined period of time) on the touch bar 119, a slide operation to the left and right (operation of touching and then moving the touch position while keeping the touch) and the like. The touch bar 119 is an operation member different from the touch panel 109, and does not have a display function. The touch bar 119 functions as, for example, a multi-function bar (M-Fn bar) to which various functions can be assigned.

[0017] The camera 100 also includes a grip section 120, a thumb rest section 121, a terminal cover 122, a lid 123, a communication terminal 124, and the like. The grip section 120 is a holding section formed in a shape that is easy to hold with the right hand when the user holds the camera 100. The shutter button 101 and the main electronic dial 104 are arranged at positions that can be operated with the index finger of the right hand when the camera 100 is held by gripping the grip section 120 with the little finger, ring finger, and middle finger of the right hand. In the same state, the sub electronic dial 105 and the touch bar 119 are arranged at positions that can be operated with the thumb of the right hand. The thumb rest section 121 (thumb standby position) is a grip section provided on the rear side of the camera 100 at a position where it is easy to place the thumb of the right hand that is gripping the grip section 120 when none of the operation members are being operated. The thumb rest section 121 is made of a rubber member or the like for increasing the holding force (grip feeling). The terminal cover 122 protects connectors such as a connection cable that connects the camera 100 to an external device (external apparatus). The lid 123 closes a slot for storing a recording medium 227 (described later) to protect the recording medium 227 and the slot. The communication terminal 124 is a terminal for communicating with a lens unit that is detachable from the camera 100 (lens unit 200 or lens unit 300, described later).

[0018] <Internal structure of the camera> Fig. 3 is a block diagram showing an example of the configuration of the camera 100. Note that the same components as those in Fig. 2(A) and 2(B) are given the same reference numerals as those in Fig. 2(A) and 2(B), and the description of those components will be omitted as appropriate. In Fig. 3, the lens unit 200 is attached to the camera 100.

[0019] First, the lens unit 200 will be described. The lens unit 200 is a type of interchangeable lens unit that is detachable from the camera 100. The lens unit 200 is a single lens unit (single lens unit) and is an example of a normal lens unit. The lens unit 200 has an aperture 201, a lens 202, an aperture drive circuit 203, an AF (autofocus) drive circuit 204, a lens system control circuit 205, a communication terminal 206, and the like.

[0020] The aperture 201 is configured to have an adjustable aperture diameter. The lens 202 is composed of a plurality of lenses. The aperture drive circuit 203 adjusts the amount of light by controlling the aperture diameter of the aperture 201. The AF drive circuit 204 drives the lens 202 to adjust the focus. The lens system control circuit 205 controls the aperture drive circuit 203 and the AF drive circuit 204 based on an instruction from a system control unit 50, which will be described later. The lens system control circuit 205 controls the aperture 201 via the aperture drive circuit 203. In addition, the lens system control circuit 205 adjusts the focus by changing the position of the lens 202 via the AF drive circuit 204. The lens system control circuit 205 can communicate with the camera 100. Specifically, communication is performed via a communication terminal 206 of the lens unit 200 and a communication terminal 124 of the camera 100. The communication terminal 206 is a terminal through which the lens unit 200 communicates with the camera 100.

[0021] Next, a description will be given of the camera 100. The camera 100 has a shutter 210, an imaging unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, an EVF 217, a display unit 108, and a system control unit 50.

[0022] The shutter 210 is a focal plane shutter that can freely control the exposure time of the imaging unit 211 based on an instruction from the system control unit 50. The imaging unit 211 is an imaging element (image sensor) composed of a CCD or CMOS element that converts an optical image into an electric signal. The imaging unit 211 may have an imaging surface phase difference sensor that outputs defocus amount information to the system control unit 50. The A / D converter 212 converts an analog signal output from the imaging unit 211 into a digital signal. The image processing unit 214 performs predetermined image processing (resizing processing such as pixel interpolation and reduction, color conversion processing, etc.) on data from the A / D converter 212 or data from the memory control unit 213. In addition, the image processing unit 214 performs predetermined calculation processing using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the obtained calculation result. Through this processing, TTL (through-the-lens) type AF processing, AE (automatic exposure) processing, EF (flash pre-emission) processing, etc. are performed. Furthermore, the image processing unit 214 performs a predetermined calculation process using the captured image data, and the system control unit 50 performs TTL type AWB (auto white balance) processing based on the obtained calculation results.

[0023] The image data from the A / D converter 212 is written into the memory 215 via the image processing unit 214 and the memory control unit 213. Alternatively, the image data from the A / D converter 212 is written into the memory 215 via the memory control unit 213 without passing through the image processing unit 214. The memory 215 stores image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212, and image data to be displayed on the display unit 108 and EVF 217. The memory 215 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio. The memory 215 also serves as a memory for displaying images (video memory).

[0024] The D / A converter 216 converts the image data for display stored in the memory 215 into an analog signal and supplies it to the display unit 108 or the EVF 217. Therefore, the image data for display written in the memory 215 is displayed on the display unit 108 or the EVF 217 via the D / A converter 216. The display unit 108 or the EVF 217 performs display according to the analog signal from the D / A converter 216. The display unit 108 or the EVF 217 is, for example, an LCD or an organic EL display. A digital signal that has been A / D converted by the A / D converter 212 and stored in the memory 215 is converted into an analog signal by the D / A converter 216, and the analog signal is sequentially transferred to and displayed on the display unit 108 or the EVF 217, thereby performing live view display.

[0025] The system control unit 50 is a control unit consisting of at least one processor and / or at least one circuit. That is, the system control unit 50 may be a processor, a circuit, or a combination of a processor and a circuit. The system control unit 50 controls the entire camera 100. The system control unit 50 executes a program recorded in the non-volatile memory 219 to realize each process of a flowchart described later. The system control unit 50 also performs display control by controlling the memory 215, the D / A converter 216, the display unit 108, the EVF 217, and the like. The system control unit 50 can identify the type of the lens unit attached to the camera 100 by communicating via the communication terminal 124 and the communication terminal 206.

[0026] The camera 100 also includes a system memory 218 , a non-volatile memory 219 , a system timer 220 , a communication unit 221 , an attitude detection unit 222 , and an eye proximity detection unit 118 .

[0027] For example, a RAM is used as the system memory 218. Constants and variables for the operation of the system control unit 50, programs read from the nonvolatile memory 219, and the like are deployed in the system memory 218. The nonvolatile memory 219 is an electrically erasable and recordable memory, and for example, an EEPROM is used as the nonvolatile memory 219. Constants and programs for the operation of the system control unit 50 are recorded in the nonvolatile memory 219. The programs here are programs for executing a flowchart described later. The system timer 220 is a clock unit that measures the time used for various controls and the time of a built-in clock. The communication unit 221 transmits and receives video signals and audio signals to and from an external device connected wirelessly or by a wired cable. The communication unit 221 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 221 can also communicate with an external device via Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 221 can transmit images (including live images) captured by the imaging unit 211 and images recorded in the recording medium 227, and can receive images and various other information from external devices. The attitude detection unit 222 detects the attitude (tilt) of the camera 100 with respect to the direction of gravity. Based on the attitude detected by the attitude detection unit 222, the tilt angle of the camera 100 in the horizontal direction (left and right direction) or vertical direction (up and down direction; front and back direction) can be detected. Based on the attitude detected by the attitude detection unit 222, it is also possible to determine whether the image captured by the imaging unit 211 is an image captured with the camera 100 held horizontally or held vertically. The system control unit 50 can add orientation information corresponding to the attitude detected by the attitude detection unit 222 to the image file of the image captured by the imaging unit 211, or rotate the image according to the detected attitude. Using the attitude detection unit 222, the movement of the camera 100 (panning, tilting, lifting, whether it is stationary, etc.) can be detected. The attitude detection unit 222 can use, for example, an acceleration sensor or a gyro sensor.

[0028] The eyepiece detection unit 118 can detect the approach of some object to the eyepiece unit 116 (eyepiece finder 117). For example, an infrared proximity sensor can be used for the eyepiece detection unit 118. When an object approaches, infrared rays projected from a light projecting unit of the eyepiece detection unit 118 are reflected by the object and received by a light receiving unit of the infrared proximity sensor. The distance from the eyepiece unit 116 to the object can be determined based on the amount of infrared rays received. In this way, the eyepiece detection unit 118 performs eyepiece detection to detect the proximity of the object to the eyepiece unit 116. The eyepiece detection unit 118 is an eyepiece detection sensor that detects the approach (approach) and departure (away) of the eye (object) to the eyepiece unit 116. When an object is detected approaching within a predetermined distance from the non-eyepiece state (non-approach state), it detects that the eye has been placed near the eyepiece unit 116. On the other hand, when an object that has been detected as approaching moves away from the eye-closed state (approaching state) by a distance greater than a predetermined distance, the system detects that the object has been moved away. The threshold for detecting the eye-closed state and the threshold for detecting the eye-moving state may be different, for example, by providing a hysteresis. After detecting the eye-closed state, the system is assumed to be in the eye-closed state until the eye-moving state is detected. After detecting the eye-moving state, the system is assumed to be in the non-eye-closed state until the eye-closed state is detected. The system control unit 50 switches between display (display state) / non-display (non-display state) of the display unit 108 and the EVF 217 according to the state detected by the eye-closed state detection unit 118. Specifically, when at least in the shooting standby state and the display destination switching setting is automatic switching, the display destination is set to the display unit 108 and the display is turned on while the eye is not placed in the eye-closed state, and the EVF 217 is turned off. Moreover, during the eye-closed state, the display destination is set to the EVF 217 and the display is turned on while the display unit 108 is turned off. Note that eye proximity detection unit 118 is not limited to an infrared proximity sensor, and other sensors may be used as eye proximity detection unit 118 as long as they can detect a state that can be regarded as eye proximity.

[0029] The camera 100 also has an outside-finder display unit 107, an outside-finder display drive circuit 223, a power supply control unit 224, a power supply unit 225, a recording medium I / F 226, an operation unit 228, and the like.

[0030] The outside viewfinder display unit 107 is driven by an outside viewfinder display drive circuit 223, and displays various settings of the camera 100 such as the shutter speed and aperture. The power supply control unit 224 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between blocks to be energized, and the like, and detects whether a battery is attached, the type of battery, and the remaining battery level. The power supply control unit 224 also controls the DC-DC converter based on the detection result and an instruction from the system control unit 50, and supplies the required voltage to each unit including the recording medium 227 for the required period. The power supply unit 225 is a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like. The recording medium I / F 226 is an interface with the recording medium 227 such as a memory card or a hard disk. The recording medium 227 is a memory card or the like for recording captured images, and is composed of a semiconductor memory, a magnetic disk, or the like. The recording medium 227 may be detachable from the camera 100, or may be built into the camera 100.

[0031] The operation unit 228 is an input unit that accepts operations from the user (user operations) and is used to input various instructions to the system control unit 50. The operation unit 228 includes the shutter button 101, the power switch 102, the mode changeover switch 103, the touch panel 109, and other operation units 229. The other operation units 229 include the main electronic dial 104, the sub electronic dial 105, the video button 106, the direction keys 110, the SET button 111, the AE lock button 112, the enlargement button 113, the playback button 114, the menu button 115, and the touch bar 119.

[0032] The shutter button 101 includes a first shutter switch 230 and a second shutter switch 231. The first shutter switch 230 is turned on when the shutter button 101 is pressed halfway (instruction to prepare for shooting) during operation, and outputs a first shutter switch signal SW1. In response to the first shutter switch signal SW1, the system control unit 50 starts preparation processes for shooting, such as AF processing, AE processing, AWB processing, and EF processing. The second shutter switch 231 is turned on when the shutter button 101 is pressed fully (instruction to shoot), and outputs a second shutter switch signal SW2. In response to the second shutter switch signal SW2, the system control unit 50 starts a series of shooting processes, from reading out a signal from the imaging unit 211 to generating an image file including a shot image and writing it to the recording medium 227.

[0033] The mode changeover switch 103 changes the operation mode of the system control unit 50 to one of a still image shooting mode, a video shooting mode, a playback mode, etc. Modes included in the still image shooting mode include an auto shooting mode, an auto scene determination mode, a manual mode, an aperture priority mode (Av mode), a shutter speed priority mode (Tv mode), and a program AE mode (P mode). There are also various scene modes and custom modes that are shooting settings according to shooting scenes. The user can directly switch to one of the above-mentioned shooting modes using the mode changeover switch 103. Alternatively, the user can selectively switch to one of the displayed modes using the operation unit 228 after once switching to a list screen of shooting modes using the mode changeover switch 103. Similarly, the video shooting mode may also include a plurality of modes.

[0034] The touch panel 109 is a touch sensor that detects various touch operations on the display surface of the display unit 108 (the operation surface of the touch panel 109). The touch panel 109 and the display unit 108 can be configured as one unit. For example, the touch panel 109 has a light transmittance that does not interfere with the display of the display unit 108, and is attached to the upper layer of the display surface of the display unit 108. Then, the input coordinates on the touch panel 109 are associated with the display coordinates on the display surface of the display unit 108. By doing so, it is possible to configure a GUI (Graphical User Interface) as if the user could directly operate the screen displayed on the display unit 108. The touch panel 109 can be any of various types such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type. Depending on the type, there is a type that detects a touch by contact with the touch panel 109, and a type that detects a touch by approaching a finger or a pen to the touch panel 109, but any type may be used.

[0035] The system control unit 50 can detect the following operations or states on the touch panel 109. A finger or pen that has not been touching the touch panel 109 touches the touch panel 109 again, that is, the start of touching (hereinafter referred to as Touch-Down). A state in which the touch panel 109 is touched with a finger or a pen (hereinafter referred to as Touch-On). The touch panel 109 is moved while being touched by a finger or a pen (hereinafter referred to as Touch-Move). The finger or pen that has been touching the touch panel 109 is removed (released) from the touch panel 109, that is, the touch ends (hereinafter, referred to as "touch-up"). A state in which nothing is being touched on the touch panel 109 (hereinafter referred to as Touch-Off).

[0036] When touch down is detected, touch on is also detected at the same time. Normally, touch-on continues to be detected unless a touch-up is detected. Touch-on continues to be detected even if a touch-move is detected. Even if touch-on is detected, if the touch position does not move, touch-move is not detected. After it is detected that all fingers or pens that were touching have touched up, touch-off occurs.

[0037] These operations and states, and the position coordinates of the touch panel 109 touched by a finger or pen are notified to the system control unit 50 via the internal bus. The system control unit 50 determines what kind of operation (touch operation) has been performed on the touch panel 109 based on the notified information. For touch-move, the moving direction of the finger or pen moving on the touch panel 109 can also be determined for each vertical component and horizontal component on the touch panel 109 based on the change in the position coordinates. If a touch-move of a predetermined distance or more is detected, it is determined that a slide operation has been performed. An operation in which a finger is touched on the touch panel 109, quickly moved for a certain distance, and then released is called a flick. In other words, a flick is an operation in which a finger is quickly traced on the touch panel 109 as if flicking it. If a touch-move of a predetermined distance or more at a predetermined speed or more is detected and a touch-up is detected as it is, it is determined that a flick has been performed (it can be determined that a flick has occurred following a slide operation). Furthermore, a touch operation in which multiple points (for example, two points) are touched together (multi-touch) and the touch positions are brought closer together is called a pinch in, and a touch operation in which the touch positions are moved farther apart is called a pinch out. Pinch out and pinch in are collectively called a pinch operation (or simply pinch).

[0038] <Lens unit configuration> Fig. 4 is a schematic diagram showing an example of the configuration of lens unit 300. Fig. 4 shows a state in which lens unit 300 is attached to camera 100. By attaching lens unit 300, camera 100 becomes able to capture one image (still image or video) including two image areas having a predetermined parallax. Note that, among the camera 100 shown in Fig. 4, the same components as those explained in Fig. 3 are assigned the same reference numerals as in Fig. 3, and explanations of those components will be omitted as appropriate.

[0039] The lens unit 300 is a type of interchangeable lens unit that can be attached to and detached from the camera 100. The lens unit 300 is a twin lens unit that can capture right and left images with parallax. The lens unit 300 has two optical systems (photographing lenses), and each of the two optical systems can capture an image over a wide viewing angle range of approximately 180 degrees. Specifically, each of the two optical systems of the lens unit 300 can capture an image of a subject over a viewing field (angle of view) of 180 degrees in the left-right direction (horizontal angle, azimuth angle, yaw angle) and 180 degrees in the up-down direction (vertical angle, elevation angle, pitch angle). In other words, each of the two optical systems can capture an image over the range of the front hemisphere.

[0040] The lens unit 300 includes a right-eye optical system 301R having a plurality of lenses and a reflecting mirror, a left-eye optical system 301L having a plurality of lenses and a reflecting mirror, and a lens system control circuit 303. The right-eye optical system 301R includes a lens 302R arranged on the subject side, and the left-eye optical system 301L includes a lens 302L arranged on the subject side. The lenses 302R and 302L face in the same direction, and their optical axes are approximately parallel. Each of the right-eye optical system 301R and the left-eye optical system 301L includes a fisheye lens, and forms a circular optical image on the imaging unit 211. The optical image (right image) formed via the right-eye optical system 301R and the optical image (left image) formed via the left-eye optical system 301L are formed on the imaging surface of one imaging unit 211, and the imaging unit 211 acquires one image including the image areas of each optical image.

[0041] The lens unit 300 is a twin lens unit (VR180 lens unit) for obtaining an image in VR180, which is one of the formats of VR images that allow two-eye stereoscopic viewing. The lens unit 300 has a right-eye optical system 301R and a left-eye optical system 301L each having: The right-eye optical system 301R and the left-eye optical system 301L have a fisheye lens capable of capturing a range of approximately 180 degrees. The range that can be captured by the lenses of each of the right-eye optical system 301R and the left-eye optical system 301L may be approximately 160 degrees, which is narrower than the range of 180 degrees. The lens unit 300 can form a right image formed through the right-eye optical system 301R and a left image formed through the left-eye optical system 301L on one or two image pickup elements of a camera to which the lens unit 300 is attached. In the camera 100, the right image and the left image are formed on one image pickup element (image pickup sensor), and one image (two-eye image) in which a right image area corresponding to the right image and a left image area corresponding to the left image are arranged side by side is generated. The two-eye image includes a right image area, a left image area, and an area that does not correspond to an optical image (a non-image area, for example, a black area).

[0042] The lens unit 300 is attached to the camera 100 via the lens mount portion 304 and the camera mount portion 305 of the camera 100. In this manner, the system control unit 50 of the camera 100 and the lens system control circuit 303 of the lens unit 300 are electrically connected via the communication terminal 124 of the camera 100 and the communication terminal 306 of the lens unit 300.

[0043] In FIG. 4, a right image formed through the right eye optical system 301R and a left image formed through the left eye optical system 301L are formed side by side on the imaging unit 211 of the camera 100. That is, two optical images are formed in two areas of one imaging element (imaging sensor) by the right eye optical system 301R and the left eye optical system 301L. The imaging unit 211 converts the formed subject image (optical signal) into an analog electrical signal. By using the lens unit 300 (the right eye optical system 301R and the left eye optical system 301L) in this way, it is possible to obtain one image (two-eye image) including two image areas with parallax. By dividing the obtained image into an image for the left eye and an image for the right eye and performing VR display, the user can view a stereoscopic VR image in a range of approximately 180 degrees. That is, the user can stereoscopically view a VR180 image.

[0044] In the case of a normal single lens unit, an image (optical image) incident on the lens unit is inverted point-symmetrically around the optical axis of the lens unit and input to an image sensor. An image capture device such as the camera 100 can generate a natural (non-inverted) image by controlling the order in which signals are read from the image sensor and by inverting the read signal (image). In the case of a twin lens unit, an image is inverted vertically and input to the image sensor, but is not inverted horizontally. Therefore, the left image incident via the left eye optical system is placed on the left side, and the right image incident via the right eye optical system is placed on the right side, and the left and right images are input to the image sensor. Therefore, when the same inversion process as in the case of the single lens unit is performed, the left and right in the camera 100 and the left and right of the image after the inversion process are reversed. In other words, an image is generated in which the left image area corresponding to the left image is placed on the right side, and the right image area corresponding to the right image is placed on the left side.

[0045] Here, the VR image is an image that can be displayed in VR, which will be described later. The VR image includes an omnidirectional image (spherical image) captured by an omnidirectional camera (spherical camera) and a panoramic image having a wider image range (effective image range) than the display range that can be displayed at one time on a display unit. In addition, the VR image is not limited to a still image, but also includes a video and a live image (image acquired from a camera in almost real time). The VR image has an image range (effective image range) of a field of view of 360 degrees in the left and right directions and 360 degrees in the up and down directions at maximum. In addition, the VR image includes an image having a wider angle of view than the angle of view that can be captured by a normal camera, or a wider image range than the display range that can be displayed at one time on a display unit, even if the field of view is less than 360 degrees in the left and right directions or less than 360 degrees in the up and down directions. The image captured by the camera 100 using the lens unit 300 described above is a type of VR image. The VR image can be displayed in VR by, for example, setting the display mode of a display device (a display device that can display a VR image) to "VR view". 360 A portion of a VR image with a field of view of 350 degrees is displayed, and the user can change the attitude of the display device left or right (horizontal rotation direction) to move the displayed area and view seamless omnidirectional images in the left or right direction.

[0046] VR display (VR view) is a display method (display mode) that can change the display range and displays an image of a VR image with a field of view according to the posture of the display device. VR display includes "single-eye VR display (single-eye VR view)" that displays one image by performing a transformation (distortion correction) to map a VR image onto a virtual sphere. VR display also includes "two-eye VR display (two-eye VR view)" that displays a VR image for the left eye and a VR image for the right eye side by side in the left and right regions by performing a transformation to map the VR image for the left eye and the VR image for the right eye, which have a parallax from each other, in a "two-eye VR display" that allows the VR images to be viewed in stereoscopic view. In any VR display, for example, when a user wears a display device such as an HMD (head-mounted display), an image with a field of view according to the direction of the user's face is displayed. For example, assume that a VR image with a field of view centered on 0 degrees left and right (a specific direction, for example, north) and 90 degrees up and down (90 degrees from the zenith, i.e. horizontal) is displayed at a certain point in time. When the orientation of the display device is flipped from this state (for example, the display surface is changed from facing south to facing north), the display range of the same VR image is changed to an image with a viewing range centered on 180 degrees left and right (the opposite direction, for example, south) and 90 degrees up and down. That is, when the user faces from north to south (i.e. turns around) while wearing the HMD, the image displayed on the HMD is also changed from a north image to a south image. Note that the VR image captured using the lens unit 300 is a VR180 image (180° image) capturing an approximately 180° range in front, and no image exists in an approximately 180° range behind. When such a VR180 image is displayed in VR and the orientation of the display device is changed to the side where no image exists, a blank area is displayed.

[0047] By performing VR display of VR images in this way, the user can visually obtain a feeling (sense of immersion) as if they were inside the VR image (VR space). Note that the method of displaying VR images is not limited to the method of changing the posture of the display device. For example, the display range may be moved (scrolled) according to user operations via a touch panel, direction buttons, etc. Also, during VR display (when in the display mode "VR view"), in addition to changing the display range due to a change in posture, the display range may be moved according to a touch move on the touch panel, a drag operation with a mouse, etc., or pressing a direction button. Note that a smartphone mounted on a VR goggle (head-mounted adapter) is a type of HMD.

[0048] <Configuration of VR Goggles> FIG. 5(A) is a front perspective view of the electronic device 500. The electronic device 500 is a display device, for example, a smartphone. The display 505 is a display unit that displays images and various information. The display 505 is integrally configured with a touch panel 506a and is capable of detecting a touch operation on the display surface of the display 505. The electronic device 500 is capable of performing VR display of VR images (VR content) on the display 505. The operation member 506b is a power button that accepts an operation to switch the power of the electronic device 500 on and off. The operation members 506c and 506d are volume buttons for increasing and decreasing the volume of the sound output from the speaker 512b or earphones or an external speaker connected to the audio output terminal 512a. The operation member 506e is a home button for displaying a home screen on the display 505. The audio output terminal 512a is an earphone jack and is a terminal for outputting an audio signal to earphones or an external speaker. The speaker 512b is a built-in speaker for outputting sound.

[0049] FIG. 5(B) is a rear perspective view of the electronic device 500. The imaging unit 515 is a camera capable of imaging the real space.

[0050] 5(C) is a block diagram showing the configuration of an electronic device 500. A CPU 501, a memory 502, a non-volatile memory 503, an image processing unit 504, a display 505, an operation unit 506, a storage medium I / F 507, an external I / F 509, and a communication I / F 510 are connected to an internal bus 520. Also connected to the internal bus 520 are an audio output unit 512, an attitude detection unit 513, a self-position / surrounding environment estimation unit 514, and an imaging unit 515. The units connected to the internal bus 520 are configured to be able to exchange data with each other via the internal bus 520.

[0051] CPU 501 is a control unit that controls the entire electronic device 500, and is composed of at least one processor or circuit. Memory 502 is, for example, a RAM (a volatile memory using a semiconductor element). CPU 501 controls each unit of electronic device 500, for example, using memory 502 as a work memory according to a program stored in nonvolatile memory 503. Various information such as image data, audio data, other data, and various programs for CPU 501 to operate are stored in nonvolatile memory 503. Nonvolatile memory 503 is, for example, a flash memory or a ROM.

[0052] The image processing unit 504 performs various image processing on the image stored in the non-volatile memory 503 or the storage medium 508, the video signal acquired via the external I / F 509, or the image acquired via the communication I / F 510 under the control of the CPU 501. The various image processing includes A / D conversion processing, D / A conversion processing, image data encoding processing, image data compression processing, image data decoding processing, image data enlargement / reduction processing (resizing), image data noise reduction processing, and image data color conversion processing. The various image processing also includes panoramic development, mapping processing, and conversion processing of VR images, which are omnidirectional images or wide-range images having a wide range of images even if they are not omnidirectional. The image processing unit 504 may be a dedicated circuit block for performing specific image processing. Depending on the type of image processing, the CPU 501 can perform image processing according to a program without using the image processing unit 504.

[0053] Display 505 displays images and a GUI screen constituting a GUI (Graphical User Interface) under the control of CPU 501. CPU 501 generates a display control signal according to a program, and controls each unit of electronic device 500 to generate a video signal for display on display 505 and output it to display 505. Display 505 displays an image based on the generated and output video signal. Note that the configuration of electronic device 500 itself is limited to an interface for outputting a video signal for display on display 505, and display 505 may be an external monitor (for example, a television or a head-mounted display).

[0054] The operation unit 506 includes various input units for receiving user operations. For example, the operation unit 506 includes a character information input device (e.g., a keyboard), a pointing device (e.g., a mouse or a touch panel), a button, a dial, a joystick, a touch sensor, and a touch pad. In this embodiment, the operation unit 506 includes a touch panel 506a, operation members 506b, 506c, 506d, and 506e, and a gesture detection unit 506f.

[0055] A storage medium 508 such as a memory card, CD, or DVD can be attached to the storage medium I / F 507. The storage medium I / F 507 reads data from the attached storage medium 508 and writes data to the storage medium 508 under the control of the CPU 501. The storage medium 508 is a storage unit that stores various data including images to be displayed on the display 505. The external I / F 509 is connected to an external device via a wired or wireless connection. The communication I / F 510 is an interface for communicating (wirelessly communicating) with an external device or the Internet 511 and transmitting and receiving (data communicating) various data such as files and commands. The communication I / F 510 is also capable of communicating (wirelessly communicating) with a controller 516.

[0056] Audio output unit 512 outputs the sound of video or music data played back by electronic device 500, operation sounds, ringtones, and various notification sounds. Audio output unit 512 includes audio output terminal 512a for connecting earphones or an external speaker, and speaker 512b, but audio output unit 512 may output audio data to an external speaker via wireless communication.

[0057] The attitude detection unit 513 detects the attitude (tilt) of the electronic device 500 with respect to the direction of gravity, or the attitude of the electronic device 500 with respect to each axis of the yaw direction, pitch direction, and roll direction, and notifies the CPU 501 of the attitude information. Based on the attitude detected by the attitude detection unit 513, it is possible to determine whether the electronic device 500 is held horizontally, held vertically, facing up, facing down, or in an oblique attitude. It is also possible to determine the presence or absence and the magnitude of the inclination of the electronic device 500 in the rotational directions such as the yaw direction, pitch direction, and roll direction, and whether the electronic device 500 has rotated in the rotational direction. One sensor or a combination of multiple sensors from among an acceleration sensor, a gyro sensor, a geomagnetic sensor, a direction sensor, and an altitude sensor can be used as the attitude detection unit 513.

[0058] The self-position / surrounding environment estimation unit 514 estimates the self-position and surrounding environment of the electronic device 500 or the VR goggles 530 .

[0059] The self-position is the position of the electronic device 500 or the VR goggles 530 in a space of a predetermined range. For example, the self-position is expressed by three parameters that represent a position in a coordinate system defined by three axes, an X axis, a Y axis, and a Z axis, that are mutually orthogonal at a predetermined position in the space of a predetermined range so that the predetermined position is the origin. The self-position may be further expressed using three parameters that represent the attitude (orientation).

[0060] The surrounding environment includes an obstacle region. The obstacle region is a region of objects that are present around the electronic device 500 or the VR goggles 530 and that are obstacles to a user carrying the electronic device 500 or a user wearing the VR goggles 530. For example, the obstacle region is expressed by a plurality of sets of three parameters that represent a position in a coordinate system defined by three axes, an X axis, a Y axis, and a Z axis, which are mutually orthogonal at a predetermined position within a predetermined range of space so that the predetermined position is the origin.

[0061] The imaging unit 515 is a camera capable of capturing an image of the real space. The image captured of the real space can be used for various detection processes, for example, by the gesture detection unit 506f and the self-position / surrounding environment estimation unit 514. The image captured of the real space can be displayed on the display 505.

[0062] As described above, operation unit 506 includes touch panel 506a. Touch panel 506a is an input device that is configured to be planar and overlaid on display 505, and outputs coordinate information according to the touched position. CPU 501 can detect the following operations or states on touch panel 506a. An operation object (e.g., a finger or a pen) that has not been touching the touch panel 506a touches the touch panel 506a again, that is, the start of touching (hereinafter referred to as Touch-Down). A state where the operating object is touching the touch panel 506a (hereinafter, “touch-on”) h-On) The operating object is moved while touching the touch panel 506a (hereinafter referred to as Touch-Move). The operation object that was touching the touch panel 506a is removed from the touch panel 506a, that is, the touch is ended (hereinafter referred to as "touch-up"). A state in which nothing is touching the touch panel 506a (hereinafter referred to as Touch-Off)

[0063] When touch-down is detected, touch-on is also detected at the same time. After touch-down, touch-on will usually continue to be detected unless touch-up is detected. If touch-move is detected, touch-on is also detected at the same time. Even if touch-on is detected, touch-move will not be detected if the touch position does not move. When it is detected that all operating objects that were touching have touched up, touch-off is detected.

[0064] These operations and states, as well as the position coordinates of the position where the operating object touches the touch panel 506a, are notified to the CPU 501 via an internal bus. The CPU 501 determines what kind of operation (touch operation) has been performed on the touch panel 506a based on the notified information. Regarding touch-move, the moving direction of the operating object moving on the touch panel 506a can also be determined for each of the vertical and horizontal components on the touch panel 506a based on the change in the position coordinates. If a touch-move of a predetermined distance or more is detected, it is determined that a slide operation has been performed.

[0065] An operation in which the operating object is touched on the touch panel 506a, moved quickly for a certain distance, and then released is called a flick. In other words, a flick is an operation in which the operating object is quickly traced on the touch panel 506a as if flicking it. When a touch-move of a predetermined distance or more at a predetermined speed or more is detected and a touch-up is detected immediately after that, it can be determined that a flick has been performed (it can be determined that a flick has occurred following a slide operation).

[0066] Furthermore, a touch operation in which multiple points (for example, two points) are touched simultaneously and the touch positions are brought closer to each other is called pinch in, and a touch operation in which the touch positions are moved away from each other is called pinch out. Pinch out and pinch in are collectively called pinch operation (or simply pinch). The touch panel 506a may be of any of various types such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type. There are types that detect a touch by contact with the touch panel, and types that detect a touch by the approach of an operating object to the touch panel, and either type may be used.

[0067] As described above, the operation unit 506 includes the gesture detection unit 506f. The gesture detection unit 506f detects a gesture of a user (for example, a user's hand) from an image (an image of a real space) obtained by the imaging unit 515 under the control of the CPU 501. The CPU 501 performs various processes (controls) according to the detected gesture.

[0068] FIG. 5(D) is an external view of VR goggles (head-mounted adapter) 530 to which electronic device 500 can be attached. By attaching electronic device 500 to VR goggles 530, it is also possible to use electronic device 500 as a head-mounted display. Insertion port 531 is an insertion port for inserting electronic device 500. The entire electronic device 500 can be inserted into VR goggles 530 with the display surface of display 505 facing headband 532 side (i.e., user side) for fixing VR goggles 530 to the user's head. With VR goggles 530 with electronic device 500 attached attached attached to the head, the user can view display 505 without holding electronic device 500 in hands. In this case, When the user moves his / her head or entire body, the posture of the electronic device 500 also changes. The posture detection unit 513 detects the posture change of the electronic device 500 at this time, and the CPU 501 performs processing for VR display (display in the display mode "VR view") based on this posture change. In this case, the detection of the posture of the electronic device 500 by the posture detection unit 513 is equivalent to detecting the posture of the user's head (the direction in which the user's line of sight is facing). VR display (VR view) is a display method (display mode) in which an image of a VR image with a field of view range corresponding to the posture of the display device is displayed, and the display range can be changed.

[0069] The electronic device 500 itself may be a head-mounted display that can reach the head without VR goggles. The electronic device 500 may be capable of detecting the user's line of sight or facial expression, and the user may be able to operate the electronic device 500 by the line of sight or facial expression.

[0070] FIG. 5(E) is an external view of controllers 540 and 550, which are examples of a controller 516 capable of communicating with the electronic device 500. The controller 540 is a grip-type controller that is operated by the user with the left hand. The user holds the hold portion 541 of the controller 540 with the left hand and operates the operating member disposed on the operation surface 542 with a finger (e.g., the thumb) of the left hand. The controller 540 notifies the electronic device 500 of the operation performed by the user. The controller 550 is a grip-type controller that is operated by the user with the right hand and has the same configuration as the controller 540 (e.g., a form in which the controller 540 is reversed left and right).

[0071] Note that, as the controller 516, the controller 560 shown in FIG. 5(F) may be used. The controller 560 is a ring-type controller that the user wears on a finger and operates. The controller 560 has a ring portion 561 for wearing on the user's finger 563 and an operating member 562. The operating member 562 is, for example, a push-type button, a rotary dial, or an optical track pad. The optical track pad is a type of operating member capable of detecting contact or approach of a finger.

[0072] Hereinafter, the description will be made on the assumption that the user is wearing the VR goggles 530 on which the electronic device 500 is mounted.

[0073] <Processing of the Camera for VR Live View> FIG. 6 is a flowchart showing an example of the processing of the camera 100 for VR live view. The processing in FIG. 6 is realized by the system control unit 50 expanding and executing the program recorded in the non-volatile memory 219 in the system memory 218. For example, when the camera 100 is activated, the processing in FIG. 6 starts.

[0074] In step S601, the system control unit 50 determines whether the camera 100 is compatible with a twin lens unit (e.g., the lens unit 300). For example, the system control unit 50 determines whether the firmware version of the system control unit 50 is compatible with a twin lens unit. If the system control unit 50 determines that the camera 100 is compatible with a twin lens unit, the process proceeds to step S602, and if not, the process of FIG. 6 ends.

[0075] In step S602, the system control unit 50 determines whether or not a twin lens unit is attached to the camera 100. If the system control unit 50 determines that a twin lens unit is attached, the process proceeds to step S603; otherwise, the process in FIG. 6 ends.

[0076] In step S603, the system control unit 50 acquires design information (lens design information) of the attached (connected) twin lens unit from the twin lens unit. The design information includes design parameters and is used for image processing such as left-right swapping and equirectangular conversion, which will be described later.

[0077] For example, the lens design information includes the following information: 1. Image circle position 2. Image circle diameter 3. Angle of view 4. Distortion correction coefficient

[0078] The image circle position is the optical axis center coordinate of the optical system in the captured image (image including the right image area and the left image area), and is prepared for each of the two optical systems (the left eye optical system 301L and the right eye optical system 301R) of the twin lens unit. In other words, the image circle position is the center coordinate of the image circle (circular fisheye image) formed on the imaging element, and is prepared for each of the right image and the left image. The origin of the coordinates is, for example, the center of the imaging element (the center of the captured image). The image circle position includes horizontal coordinates and vertical coordinates. Note that various information related to the optical axis center of the optical system in the captured image can be used as the image circle position. For example, the distance from a predetermined position (such as the center or the upper left corner) in the captured image to the optical axis center can be used.

[0079] The image circle diameter is the diameter of the image circle (circular fisheye image) formed on the imaging element. The angle of view is the angle of view of the image circle (circular fisheye image) formed on the imaging element. The distortion correction coefficient is the ratio of the design image height to the ideal image height of the lens. A distortion correction coefficient may be set for each image height, and for an image height for which a distortion correction coefficient is not set, a distortion correction coefficient may be calculated by an interpolation calculation using multiple distortion correction coefficients. A polynomial that approximates the relationship between the image height and the distortion correction coefficient may be set. The image circle diameter, angle of view, and distortion correction coefficient may be parameters common to the two optical systems (left eye optical system 301L and right eye optical system 301R) of the twin lens unit, or may not be parameters common to them.

[0080] In step S604, the system control unit 50 acquires individual information (lens individual information) of the attached (connected) twin lens unit from the twin lens unit. The lens individual information includes parameters specific to the lens unit, such as manufacturing errors. By using the lens individual information, the image processing can be performed with higher accuracy than when only lens design information is used.

[0081] For example, the lens individual information includes the following information: The following information is prepared by measuring each of the two optical systems (the left eye optical system 301L and the right eye optical system 301R) of the twin lens unit. 5. Image circle misalignment 6. Optical axis tilt 7. Image Magnification Misalignment

[0082] The image circle position deviation is the deviation of the center coordinates of the image circle (circular fisheye image) formed on the imaging element from the design value. For example, the image circle position deviation includes horizontal deviation and vertical deviation. With the coordinates (two-dimensional coordinates including horizontal and vertical coordinates) of the design value (design information) as the origin, the horizontal coordinate indicates the horizontal deviation, and the vertical coordinate indicates the vertical deviation. The optical axis tilt is the deviation of the direction of the optical axis on the subject side from the design value. For example, the optical axis tilt includes horizontal deviation and vertical deviation. Deviations in each direction are expressed as angles. The image magnification deviation is the deviation of the size of the image circle (circular fisheye image) formed on the imaging element from the design value. This deviation is expressed, for example, as a ratio to the design value.

[0083] In step S605, the camera 100 is connected to the VR goggles 530 (electronic device 500), and the system control unit 50 detects that the camera 100 is connected to the VR goggles 530. In step S606, the system control unit 50 receives a VR live view start request from the VR goggles 530. In step S607, the system control unit 50 receives a live view image request from the VR goggles 530.

[0084] In step S608, the system control unit 50 converts the information acquired in steps S603 and S604 (lens information related to the twin lens unit) so that it matches the coordinate system of the live view image to be transmitted. If the information acquired in steps S603 and S604 is defined based on a coordinate system different from the coordinate system of the live view image, the information acquired in steps S603 and S604 cannot be used as is for image processing of the live view image. Therefore, the lens information is converted to information that matches the coordinate system of the live view image.

[0085] The information included in the lens information is not limited to the above-mentioned information. For example, the lens information may include the boundary positions of the right image area and the left image area in the captured image (the position of the edge of the circular fisheye image). The lens information may include the midpoint coordinate between the right image area and the left image area in the captured image. In many cases, the midpoint coordinate coincides with the center coordinate of the captured image. The lens information may include information indicating a magic window area, which is an area to be cut out (first) for VR display (for example, the coordinate of the upper left corner of the magic window area, the width of the magic window area, and the height of the magic window area). The lens information may include correction data (for example, correction values ​​obtained by calibration of the twin lens unit) for improving the accuracy of left-right swapping processing, equirectangular conversion processing, and the like.

[0086] In step S609, the system control unit 50 transmits the lens information converted in step S608 and the live view image to the VR goggles 530. In this embodiment, the system control unit 50 of the camera 100 performs the conversion process of the lens information, but the CPU 501 of the VR goggles 530 may perform the conversion process of the lens information. In this case, the system control unit 50 transmits to the VR goggles 530 the lens information before the conversion process and parameters necessary for the conversion process of the lens information.

[0087] In step S610, the system control unit 50 determines whether to end the VR live view. For example, when the connection between the camera 100 and the VR goggles 530 is released, or when the user instructs the camera 100 or the VR goggles 530 to end the VR live view, it is determined to end the VR live view. If the system control unit 50 determines to end the VR live view, it ends the process of FIG. 6; otherwise, it proceeds to step S607.

[0088] <VR Goggles Processing for VR Live View> FIG. 7 is a flowchart showing an example of the processing of the VR goggles 530 (electronic device 500) for the VR live view. The processing of FIG. 7 is realized by the CPU 501 expanding and executing a program (application program) recorded in the non-volatile memory 503 in the memory 502. For example, when the user instructs the VR goggles 530 to start a specific application, the processing of FIG. 7 starts.

[0089] In step S701, a camera (for example, camera 100) is connected to the VR goggles 530, and the CPU 501 detects that a camera is connected to the VR goggles 530.

[0090] In step S702, the CPU 501 determines whether the camera connected in step S701 is a camera capable of corresponding to a binocular lens unit (for example, lens unit 300). For example, the CPU 501 acquires the model information of the connected camera from the connected camera, and based on the acquired model information, determines whether it is a camera capable of corresponding to a binocular lens unit. If the CPU 501 determines that it is a camera capable of corresponding to a binocular lens unit, it proceeds to step S703; otherwise, it ends the processing of FIG. 7. A camera capable of corresponding to a binocular lens unit is, for example, a camera on which a binocular lens unit can be mounted.

[0091] In step S703, CPU 501 determines whether the firmware of the camera connected in step S701 is compatible with the twin lens unit. For example, CPU 501 obtains firmware version information of the connected camera from the connected camera, and determines whether the firmware version of the connected camera is compatible with the twin lens unit based on the obtained information. If CPU 501 determines that the firmware is compatible with the twin lens unit, it proceeds to step S704, and if not, ends the process of FIG. 7.

[0092] Even if a camera compatible with a twin lens unit is connected to the VR goggles 530, the connected camera may not be compatible with the twin lens unit due to reasons such as an old version of the firmware of the connected camera. For this reason, the processing of step S703 is required. Also, various cameras can be connected to the VR goggles 530, and a camera that is not compatible with the twin lens unit may be connected regardless of the firmware version. For this reason, the processing of step S702 is required before the processing of step S703.

[0093] In step S704, CPU 501 determines whether or not a twin lens unit is attached to the camera connected in step S701. If CPU 501 determines that a twin lens unit is attached, it proceeds to step S705, and if not, it ends the process in FIG.

[0094] In step S705, the CPU 501 transmits a VR live view start request to the camera connected in step S701.

[0095] In step S706, the CPU 501 transmits a live view image request to the camera connected in step S701.

[0096] In step S707, CPU 501 receives, from the camera connected in step S701, a live view image captured by the camera and lens information of the twin lens unit attached to the camera. The lens information received in step S708 is information converted to match the live view image to be received (for example, the lens information converted in step S608 in FIG. 6). The live view image and lens information received in step S707 are, for example, the live view image and lens information transmitted from camera 100 in step S609.

[0097] In step S708, the CPU 501 performs left-right swapping processing to swap the positions of the right image area and the left image area in the live view image acquired in step S707, based on the lens information acquired in step S707. The CPU 501 swaps the positions of the right image area and the left image area in the live view image, based on center coordinates (the optical axis centers of the left eye optical system 301L and the right eye optical system 301R) included in the lens information received together with the live view image. For example, the CPU 501 identifies the right image area based on the center coordinates of the right image area, and identifies the left image area based on the center coordinates of the left image area. The CPU 501 then swaps the positions of the identified right image area and left image area. In this embodiment, the live view In an image (captured image), the right image area and the left image area are arranged side by side, and the left / right positional relationship between the right image area and the left image area is reversed by the left / right swapping process. In order to specify the right image area and the left image area with higher accuracy, the diameter or radius of each of the right image area and the left image area may be obtained and used from the lens information of the twin lens unit.

[0098] The method of left-right swapping is not limited to the above method. For example, the shift amounts 805, 806, 809, and 810 in FIG. 8A are acquired from the lens information of the twin lens unit. Then, when swapping the positions of the right image area 803 and the left image area 807, the right image area and the left image area may be arranged so that the acquired shift amount is maintained, and the remaining area may be filled with black or the like. The shift amount 805 is the distance from the left edge of the live view image to the left edge of the right image area 803, and the shift amount 806 is the distance from the center of the live view image to the right edge of the right image area 803. When performing left-right swapping, the shift amount 805 is the distance from the left edge of the live view image to the left edge of the left image area, and the shift amount 806 is the distance from the center of the live view image to the right edge of the left image area. Similarly, the shift amount 809 is the distance from the right edge of the live view image to the right edge of the left image area 807, and the shift amount 810 is the distance from the center of the live view image to the left edge of the left image area 807. When performing left-right swapping processing, the shift amount 809 is the distance from the right edge of the live view image to the right edge of the right image area, and the shift amount 810 is the distance from the center of the live view image to the left edge of the right image area.

[0099] In step S709, the CPU 501 performs equirectangular conversion processing on the image after the left-right swap processing in step S708, converting each of the right and left image regions from a circular fisheye image region (region of a circular fisheye image) to an equirectangular image region (region of an equirectangular image). The equirectangular conversion processing is a conversion processing in which, like the equirectangular projection of a map, a circular fisheye image is considered as a sphere and latitudes (horizontal lines) and meridians (vertical lines) intersect at right angles. The equirectangular conversion processing converts the circular circular fisheye image region into a rectangular equirectangular image region. The CPU 501 adjusts the lens design information acquired in step S707 based on the lens individual information acquired in step S707. For example, the image circle position (center coordinates of the right image region and the left image region in the live view image) is adjusted based on the image circle position deviation. If the lens individual information indicates a difference from the lens design information, the value of the lens individual information is added to the value of the lens design information. When the lens individual information indicates the same absolute value as the lens design information, the value of the lens design information is replaced with the value of the lens individual information. The CPU 501 generates a map for equirectangular conversion based on the adjusted center coordinates (the optical axis centers of the left-eye optical system 301L and the right-eye optical system 301R). The map indicates which position in the image before conversion each pixel after conversion corresponds to. The map for equirectangular conversion is generated so that the circular fisheye image area can be converted into an equirectangular image area as well as the positions of the right image area and the left image area can be corrected (adjusted). By using the adjusted center coordinates, more accurate equirectangular conversion is possible. In this way, the CPU 501 may correct (adjust) the lens design information using the lens individual information included in the lens information received together with the live view image, generate a map based on the corrected lens design information, and perform equirectangular conversion processing based on the map.

[0100] The order of the left-right swapping process in step S708 and the equirectangular conversion process in step S709 may be reversed. The left-right swapping process and the equirectangular conversion process may not be performed separately, but may be performed together as one image process.

[0101] In step S710, CPU 501 displays on display 505 (VR display) a part of the live view image (display image, image for display) after performing the left-right swap process in step S708 and the equirectangular conversion process in step S709. In this embodiment, the process of step S710 is performed when a twin lens unit is attached to the camera connected in step S701. If the check is not performed, the check will not be performed.

[0102] In step S711, the CPU 501 determines whether or not to end the VR live view. If the CPU 501 determines that the VR live view is to be ended, the CPU 501 ends the process in FIG. 7, otherwise, the process proceeds to step S706.

[0103] <Left / right swap processing> 8(A) and 8(B) are schematic diagrams showing left-right interchange processing. Fig. 8(A) shows left-right interchange processing without using lens information of the twin lens unit. Fig. 8(B) shows left-right interchange processing with use of lens information of the twin lens unit.

[0104] As shown in Figures 8(A) and 8(B), in an image 801 before left-right swapping processing, a right image area 803, which is a circular fisheye image area, is positioned on the left side, and a left image area 807, which is also a circular fisheye image area, is positioned on the right side.

[0105] In Fig. 8(A), an image 801 is divided into a left half image area and a right half image area at a center coordinate 802 of the image 801, and the left half image area and the right half image area are swapped. In other words, the left half image area is moved to the right of the right half image area. An image 811 is an image after such left-right swapping processing.

[0106] In FIG. 8(A), the deviation amount 806 is smaller than the deviation amount 805. That is, in the image 801, the right image region 803 is closer to the center of the image 801 from the center of the left half of the image 801. Similarly, the deviation amount 810 is smaller than the deviation amount 809. That is, in the image 801, the left image region 807 is closer to the center of the image 801 from the center of the right half of the image 801. Therefore, in the image 811, the center coordinate 813 of the left image region 807 in the left - right direction is also deviated by a distance 814 from the center coordinate 804 (the center coordinate of the right image region 803 before the left - right swapping process). Similarly, the center coordinate 816 of the right image region 803 in the left - right direction is also deviated by a distance 817 from the center coordinate 808 (the center coordinate of the left image region 807 before the left - right swapping process).

[0107] By using the lens information, in the image 837 (FIG. 8(B)) after the left - right swapping process, the center of the left image region in the left - right direction can be made to coincide with the center coordinate 804, and the center of the right image region in the left - right direction can be made to coincide with the center coordinate 808.

[0108] <Processing of the camera for performing camera settings during VR live view> FIG. 9 is a flowchart showing an example of the processing of the camera 100 for performing camera settings (settings change of the camera 100) during VR live view. The processing in FIG. 9 is realized by the system control unit 50 expanding and executing the program recorded in the non - volatile memory 219 into the system memory 218. For example, the processing in FIG. 9 starts after the processing in FIG. 6.

[0109] In step S901, the system control unit 50 determines whether it has received a setting instruction (setting change instruction) of the digital camera 100 from the VR goggles 530. If the system control unit 50 determines that it has received the setting instruction, the process proceeds to step S902; otherwise, the processing in FIG. 9 ends.

[0110] In step S902, the system control unit 50 determines whether the setting instruction received in step S901 includes position coordinates. If the system control unit 50 determines that the setting instruction includes position coordinates, the process proceeds to step S903; otherwise, the process proceeds to step S904.

[0111] In step S903, the system control unit 50 acquires the position coordinates from the setting instruction received in step S901.

[0112] In step S904, the system control unit 50 performs the setting (setting change) of the camera 100 according to the setting instruction received in step S901. If the system control unit 50 acquires the position coordinates in step S903, it performs the setting of the camera 100 based on the position coordinates. The camera setting based on the position coordinates is, for example, the setting of the focus position (focus target position) or the setting of the reference position of the white balance.

[0113] <Processing of the VR goggles for performing camera settings during VR live view> FIG. 10 is a flowchart showing an example of the processing of the VR goggles 530 (electronic device 500) for performing camera settings during VR live view. The processing in FIG. 10 is realized by the CPU 501 expanding and executing a program (application program) recorded in the non-volatile memory 503 in the memory 502. For example, the processing in FIG. 10 starts after the processing in FIG. 7.

[0114] In step S1001, the CPU 501 determines whether a user operation (setting operation, setting change operation) for camera settings has been performed on the VR goggles 530. If the CPU 501 determines that the setting operation has been performed, the process proceeds to step S1002; otherwise, the processing in FIG. 10 ends.

[0115] In step S1002, CPU 501 determines whether the setting operation in step S1001 includes the designation of position coordinates. If CPU 501 determines that the setting operation includes the designation of position coordinates, it proceeds to step S1003, otherwise it proceeds to step S1008.

[0116] In step S1003, the CPU 501 acquires position coordinates designated by the user. The user designates position coordinates in a part (display range) of the live view image displayed on the display 505. In step S1003, position coordinates in the coordinate system of the display range may be obtained. In this embodiment, it is assumed that the two-eye VR display is performed so that a part of the live view image displayed on the display 505 is stereoscopically viewed, and the user designates three-dimensional position coordinates in a three-dimensional space of the stereoscopically viewed live view image. Then, in step S1003, it is assumed that three-dimensional position coordinates in a three-dimensional coordinate system (world coordinate system) representing the three-dimensional space are obtained. Here, the live view image displayed on the display 505 is a live view image after left-right swapping processing and equirectangular conversion processing have been performed.

[0117] In step S1004, CPU 501 converts the position coordinates acquired in step S1003 into position coordinates in (the entire) live view image displayed on display 505. As described above, the live view image displayed on display 505 is a live view image after left / right swapping processing and equirectangular conversion processing. In this embodiment, in step S1003, three-dimensional position coordinates in a three-dimensional space of the stereoscopically viewed live view image are specified. Then, in step S1004, two position coordinates (position coordinates in the right image area which is an equirectangular image area and position coordinates in the left image area which is an equirectangular image area) corresponding to two equirectangular image areas respectively are obtained from one three-dimensional coordinate.

[0118] The captured image obtained by the imaging unit of the camera connected to the VR goggles 530 (for example, the imaging unit 211 of the camera 100) is an image before conversion processing such as left-right swapping processing and equirectangular conversion processing is performed. Therefore, the position coordinates in the live view image after such conversion processing cannot be used as is for camera settings. Therefore, in this embodiment, performs a conversion process that is the inverse of the conversion process such as left-right swapping or equirectangular conversion on the position coordinates acquired in step S1004.

[0119] In step S1005, the CPU 501 performs an inverse conversion process of the equirectangular conversion process to convert the position coordinates acquired in step S1004 (position coordinates in the equirectangular image area) into position coordinates in the circular fisheye image.

[0120] In step S1006, CPU 501 performs the inverse process (inverse transformation process) of the left-right swap process on the position coordinates acquired in step S1005. This allows obtaining position coordinates in the live view image before the left-right swap process and the equirectangular transformation process are performed. In this embodiment, two position coordinates (position coordinates in the right image area, which is a circular fisheye image area, and position coordinates in the left image area, which is a circular fisheye image area) corresponding to the two circular fisheye image areas are obtained.

[0121] The inverse conversion process of steps S1005 and S1006 is performed based on the lens information received together with the live view image, for example, in the same manner as the left-right swap process and the equirectangular conversion process. When the twin lens unit attached to the camera connected to the VR goggles 530 is changed to another twin lens unit, the lens information of the changed twin lens unit is obtained, and various processes are performed using the lens information. The order of the inverse conversion process of step S1005 and the inverse conversion process of step S1006 may be reversed. Multiple inverse conversion processes may not be performed individually, but may be performed collectively as one image process.

[0122] In step S1007, the CPU 501 selects the position coordinates in the image region on a predetermined side out of the position coordinates in the right image region which is the circumferential fisheye image region and the position coordinates in the left image region which is the circumferential fisheye image region. For example, the user's dominant eye is set in advance, and the position coordinates in the image region corresponding to the dominant eye are selected. When the dominant eye is the right eye, the position coordinates in the right image region are selected, and when the dominant eye is the left eye, the position coordinates in the left image region are selected.

[0123] In addition, in the case of monocular VR display based on only one of the right image region and the left image region, for one position coordinate specified by the user, one position coordinate is obtained in step S1006. Therefore, in such a case, the process of step S1007 may be omitted. Also, the process of step S1007 (selection of position coordinates) may be performed after step S1004, and the inverse transformation process of steps S1005 and S1006 may be performed on the selected position coordinates.

[0124] In step S1008, the CPU 501 transmits a setting instruction corresponding to the setting operation in step S1001 to the camera connected to the VR goggles 530. When the setting operation in step S1001 includes specification of position coordinates, the CPU 501 includes the position coordinates (the position coordinates selected in step S1007) after performing the inverse transformation processes of steps S1005 and S1006 in the setting instruction. In the camera connected to the VR goggles 530, camera settings are performed according to the setting instruction (step S904 in FIG. 9). Therefore, the process of step S1008 can also be regarded as control for performing camera settings. In the present embodiment, the CPU 501 of the VR goggles 530 performs the inverse transformation process of the position coordinates (the inverse transformation process of the transformation processes such as the left-right swapping process and the orthographic cylindrical transformation process), but the system control unit 50 of the camera 100 may perform the inverse transformation process of the position coordinates. In that case, the CPU 501 transmits the position coordinates before the transformation process and the parameters necessary for the inverse transformation process of the position coordinates to the camera 100.

[0125] <VR live view display screen> 11(A) to 11(D) are displayed on the display 505 during VR live view. 11(A) to 11(D) are schematic diagrams showing an example of a screen (display screen), and show a display screen perceived by a user.

[0126] Fig. 11(A) shows a display screen before starting camera setting. In Fig. 11(A), an arrow 1101 and a button 1102 are superimposed on a live view image 1100. The arrow 1101 indicates the direction in which the controller 516 is pointed. The direction and length of the arrow change according to the movement of the controller 516. The button 1102 is a button for starting camera setting. The user can instruct the disclosure of camera setting (press the button 1102) by performing a specific operation (for example, pressing a specific button) on the controller 516 while aligning the tip of the arrow 1101 with the button 1102. The user can operate other displayed buttons in the same manner, not limited to the button 1102.

[0127] When button 1102 in Fig. 11(A) is pressed, the display screen transitions from the display screen in Fig. 11(A) to the display screen in Fig. 11(B) (buttons 1103 and 1104 are displayed). Button 1103 is a button for starting to set the focus position. Button 1104 is a button for starting to set the image quality of the live view display. The appearance of button 1102 differs between Fig. 11(A) and Fig. 11(B), making it possible to distinguish whether button 1102 has been pressed or not.

[0128] When the button 1103 in FIG. 11(B) is pressed, the display screen transitions from the display screen in FIG. 11(B) to the display screen in FIG. 11(C). In FIG. 11(C), an arrow 1101, a message 1105, and a pointer 1106 are superimposed on a live view image 1100. The message 1105 indicates an operation instruction. The pointer 1106 is an item indicating a specified position, and is displayed at the tip of the arrow 1101. The position coordinates of the pointer 1106 are obtained in step S1003 in FIG. 10. When the user performs a specific operation (for example, pressing a specific button) on the controller 516 while the display screen in FIG. 11(C) is displayed, the focus position is set based on the position coordinates of the pointer 1106. Note that the setting based on the position coordinates is not limited to the setting of the focus position (focus target position), and may be, for example, the setting of a reference position of white balance.

[0129] When the button 1104 in FIG. 11(B) is pressed, the display screen transitions from the display screen in FIG. 11(B) to the display screen in FIG. 11(D). In FIG. 11(C), an arrow 1101, a button 1102, a button 1104, a button 1107, and a button 1108 are superimposed on a live view image 1100. The button 1107 is a button for setting a NORMAL mode for performing a live view display in normal image quality. The button 1108 is a button for setting a FINE mode for performing a live view display in high image quality. When the button 1107 is pressed, the NORMAL mode is set, and when the button 1108 is pressed, the FINE mode is set. These settings are not based on position coordinates. The state of the button 1104 is different between FIG. 11(B) and FIG. 11(D), and it is possible to distinguish whether the button 1104 has been pressed or not.

[0130] <Position coordinate conversion> 12(A) to 12(C) are schematic diagrams for explaining the conversion process of the position coordinates designated by the user.

[0131] Fig. 12A shows an example of a display screen perceived by a user. A position 1201 is a position designated by the user. In step S1003 in Fig. 10, the coordinates of the position 1201 are obtained.

[0132] FIG. 12B shows one example of a live view image after left-right swapping and equirectangular conversion. An example is shown below. The display screen in Fig. 12(A) is perceived by viewing area 1211 in Fig. 12(B) with the left eye and area 1213 in Fig. 12(B) with the right eye. In step S1004 in Fig. 10, the coordinates of positions 1212 and 1214 in Fig. 12(B) are obtained from the coordinates of position 1201 in Fig. 12(A). Area 1211 and position 1212 correspond to the left image area, which is an equirectangular image area, and area 1213 and position 1214 correspond to the right image area, which is an equirectangular image area.

[0133] Fig. 12C shows an example of a live view image before the left-right swapping process and the equirectangular conversion process are performed. By the processes of steps S1005 and S1006 in Fig. 10, the coordinates of position 1212 in Fig. 12B are converted to the coordinates of position 1216 in Fig. 12C, and the coordinates of position 1214 in Fig. 12B are converted to the coordinates of position 1215 in Fig. 12C. Position 1215 corresponds to the right image area which is a circular fisheye image area, and position 1216 corresponds to the left image area which is a circular fisheye image area.

[0134] As described above, according to this embodiment, the user, while wearing the VR goggles (HMD), specifies position coordinates in an image displayed after conversion processing such as left-right swapping processing and equirectangular conversion processing is performed. These position coordinates are used for camera settings after conversion processing reverse to the conversion processing such as left-right swapping processing and equirectangular conversion processing, so that correct camera settings (camera settings that match the user's intention) can be performed. In this way, according to this embodiment, the user can easily perform camera settings while wearing the HMD.

[0135] The various controls described above as being performed by the system control unit 50 may be performed by a single piece of hardware, or the entire device may be controlled by a plurality of pieces of hardware (e.g., a plurality of processors or circuits) sharing the processing. Similarly, the various controls described above as being performed by the CPU 501 may be performed by a single piece of hardware, or the entire device may be controlled by a plurality of pieces of hardware (e.g., a plurality of processors or circuits) sharing the processing.

[0136] In addition, although the embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-mentioned embodiments merely shows one embodiment of the present invention, and each embodiment can be appropriately combined.

[0137] The present invention is not limited to cameras and VR goggles, and can be applied to any electronic device capable of converting position coordinates. For example, the present invention can be applied to PDAs, mobile phone terminals, portable image viewers, printers, digital photo frames, music players, game consoles, electronic book readers, etc. The present invention can also be applied to video players, display devices (including projection devices), tablet terminals, smartphones, AI speakers, home appliances, and in-vehicle devices.

[0138] <Other embodiments> The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0139] The disclosure of the present embodiment includes the following configuration, method, program, and medium. (Configuration 1) an acquisition means for acquiring position coordinates designated by a user in a display image obtained by performing a first conversion process on a captured image obtained by an imaging device; The position coordinates in the display image obtained by the acquisition means are subjected to the first conversion process. a conversion means for converting the position coordinates of the captured image into position coordinates in the captured image by a second conversion process that is the inverse of the first conversion process; a control means for controlling the imaging device to perform settings based on the position coordinates in the captured image obtained by the conversion means; 1. An electronic device comprising: (Configuration 2) The display device further includes a display control means for controlling the display device to display a part of the display image, The area of ​​the display image to be displayed on the display device is changeable. 2. The electronic device according to configuration 1. (Configuration 3) The display device is a head-mounted display device. 3. The electronic device according to configuration 2. (Configuration 4) the captured image is an image obtained in a state where a specific lens unit is attached to the imaging device, When the specific lens unit is not attached to the imaging device, the display control means does not control the display device to display a part of the display image. 4. The electronic device according to configuration 2 or 3. (Configuration 5) The captured image is an image in which two fisheye image areas are arranged side by side, The first conversion process includes converting each of the two fisheye image regions into an equirectangular image region. 5. The electronic device according to any one of configurations 1 to 4. (Configuration 6) The first conversion process includes a process of exchanging the two fisheye image regions, or a process of exchanging two equirectangular image regions obtained by converting the two fisheye image regions. 6. The electronic device according to configuration 5. (Configuration 7) the acquiring means acquires, as the position coordinates designated by the user, two position coordinates corresponding to two equirectangular image areas converted from the two fisheye image areas, respectively; The control means controls to set the imaging device based on the position coordinates corresponding to the fisheye image area on a predetermined side out of two position coordinates corresponding to the two fisheye image areas respectively obtained by the conversion means. 7. The electronic device according to configuration 5 or 6. (Configuration 8) The fisheye image area on the predetermined side is an area corresponding to a dominant eye that is set in advance. 8. The electronic device according to configuration 7. (Configuration 9) the captured image is an image obtained in a state where a specific lens unit is attached to the imaging device, the electronic device further includes a second acquisition means for acquiring information regarding the specific lens unit attached to the imaging device, The conversion means performs the second conversion process based on the information obtained by the second acquisition means. 9. The electronic device according to claim 1, wherein the first and second electrodes are arranged in a first direction. (Configuration 10) When the lens unit attached to the imaging device is changed from a first lens unit, which is a specific lens unit, to a second lens unit, which is a specific lens unit different from the first lens unit, the conversion means changes information used in the second conversion process from information about the first lens unit to information about the second lens unit. 10. The electronic device according to configuration 9. (Configuration 11) When a user operation that does not specify a position coordinate in the display image is performed as a user operation for setting the imaging device, The conversion means does not perform the second conversion process, 11. The electronic device according to any one of configurations 1 to 10, wherein the control means controls the imaging device to perform settings in response to the user operation. (Configuration 12) The control means controls to set a focus position or a reference position for white balance based on the position coordinates in the captured image obtained by the conversion means. 12. The electronic device according to any one of configurations 1 to 11. (method) an acquisition step of acquiring position coordinates designated by a user in a display image obtained by performing a first conversion process on a captured image obtained by an imaging device; a conversion step of converting the position coordinates in the display image obtained by the acquisition step into position coordinates in the captured image by a second conversion process that is the inverse of the first conversion process; a control step of controlling the imaging device so as to set the imaging device based on the position coordinates in the captured image obtained by the conversion step; 13. A method for controlling an electronic device comprising: (program) 13. A program for causing a computer to function as each of the means of the electronic device according to any one of configurations 1 to 12. (medium) 13. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the electronic device according to any one of configurations 1 to 12. [Explanation of symbols]

[0140] 500: Electronic equipment 501: CPU 530: VR goggles

Claims

1. A display control means for controlling a display device to display a part of a display image obtained by performing a first conversion process on an image captured by an imaging device, on a display device worn on a user's head; an acquisition means for acquiring position coordinates designated by the user in the display image; a conversion means for converting the position coordinates in the display image obtained by the acquisition means into position coordinates in the captured image by a second conversion process that is the inverse of the first conversion process; a control means for controlling the imaging device to be set based on the position coordinates in the captured image obtained by the conversion means; An electronic device comprising:

2. The area of ​​the display image to be displayed on the display device is changeable.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

3. the captured image is an image obtained with a specific lens unit attached to the imaging device, When the specific lens unit is not attached to the imaging device, the display control means does not control the display device to display a part of the display image.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

4. the captured image is an image in which two fisheye image areas are arranged side by side, The first conversion process includes converting each of the two fisheye image regions into an equirectangular image region.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

5. The first conversion process includes a process of exchanging the two fisheye image regions, or a process of exchanging two equirectangular image regions obtained by converting the two fisheye image regions.

5. The electronic device according to claim 4.

6. the acquiring means acquires, as the position coordinates designated by the user, two position coordinates corresponding to two equirectangular image areas converted from the two fisheye image areas, respectively; The control means controls the imaging device to be set based on the position coordinates corresponding to the fisheye image area on a predetermined side out of two position coordinates corresponding to the two fisheye image areas respectively obtained by the conversion means.

5. The electronic device according to claim 4.

7. The fisheye image area on the predetermined side is an area corresponding to a predetermined dominant eye.

7. The electronic device according to claim 6, wherein the electronic device is a semiconductor device.

8. the captured image is an image obtained with a specific lens unit attached to the imaging device, the electronic device further includes a second acquisition unit that acquires information about the specific lens unit attached to the imaging device; The conversion means performs the second conversion process based on the information obtained by the second acquisition means.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

9. When the lens unit attached to the imaging device is changed from a first lens unit, which is a specific lens unit, to a second lens unit, which is a specific lens unit different from the first lens unit, the conversion means changes the information used in the second conversion process from information about the first lens unit to information about the second lens unit.

9. The electronic device according to claim 8.

10. When a user operation that does not specify position coordinates in the display image is performed as a user operation for setting the imaging device, The conversion means does not perform the second conversion process, 2. The electronic device according to claim 1, wherein the control means controls the imaging device so as to perform settings in response to the user's operation.

11. The control means controls to set a focus position or a reference position for white balance based on the position coordinates in the captured image obtained by the conversion means.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

12. A display control step of controlling a display device mounted on a user's head to display a part of a display image obtained by performing a first conversion process on an image captured by the imaging device; an acquisition step of acquiring position coordinates designated by the user in the display image; a conversion step of converting the position coordinates in the display image obtained by the acquisition step into position coordinates in the captured image by a second conversion process that is the inverse of the first conversion process; a control step of controlling the imaging device so as to set the imaging device based on the position coordinates in the captured image obtained by the conversion step; 1. A method for controlling an electronic device, comprising:

13. A program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 11.

Images