Electronic device, control method of an electronic device, program and computer-readable medium

Abstract

An electronic device comprises an acquisition unit that acquires an image having a plurality of image regions captured through a plurality of optical systems by an imaging device, a display control unit that performs control to display a display image based on the acquired image and control to display an element indicating a position in the display image, and a receiving unit that receives a user operation for changing a position of the element, wherein the display control unit performs control to display the element in a first form in a case where the position of the element corresponds to a position in a predetermined region of the image, and control to display the element in a second form in a casein which the position of the element corresponds to a position outside the predetermined range.,

Description

BACKGROUNDArea of ​​Revelation

[0001] The present disclosure relates to an electronic device, a control method of the electronic device, a program, and a computer-readable medium. Description of the related art

[0002] A technique for obtaining an image having two image areas with parallax is known. Two optical systems facing the same direction are used to display the two image areas in a manner that enables stereoscopic viewing. When a circular fisheye lens is used as each optical system, an image area that covers a wide range of 180 degrees (hemispherical, 90 degrees in all directions from an image center) or more vertically and horizontally can be obtained as each image area.

[0003] Additionally, a function (PC Live View) for connecting a camera (a digital camera) and a PC (personal computer) and displaying an image captured by the camera on a display device of the PC in real time has been proposed (JP 2022-183845 A). In PC Live View, when the user designates any point on a displayed image (live view image), a specific instruction (AF instruction, photometry instruction, or the like) related to the position is transmitted to the camera. When a specific instruction is received, the camera performs a specific operation (AF, photometry, or the like) based on a position designated by the user.

[0004] However, depending on the lens attached to the camera, a range in which a specific operation (AF, photometry, or the like) can be performed appropriately (stably) may be limited to a specific range in the live view image (JP 2019-144401 A). SUMMARY

[0005] The present disclosure provides a technique for enabling a user to easily designate a position in an area in which a specific operation (predetermined processing) can be appropriately performed.

[0006] An electronic device according to the present disclosure includes an acquisition unit configured to acquire an image having a plurality of image regions each captured by an imaging device via a plurality of optical systems, a display control unit configured to perform control to display a display image based on the acquired image and to perform control to display an element indicating a position in the display image, and a receiving unit configured to receive a user operation to change a position of the element, wherein the display control unit performs control to display the element in a first form in a case where the position of the element corresponds to a position in a predetermined region of the image, and performs control tothat the element is displayed in a second form in a case where the position of the element corresponds to a position outside the predetermined range.,

[0007] Further features of the present disclosure will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWING Fig. 1A and Fig. 1B show external views of a camera; Fig. 2 shows a block diagram of the camera; Fig. 3 is a schematic diagram illustrating a configuration of a lens unit; Fig. 4 shows a cross-sectional view of the lens unit; Fig. 5A and Fig. 5B show exploded perspective views of the lens unit; Fig. 6 is a schematic diagram illustrating a positional relationship between respective optical axes and an image circle; Fig. 7 shows a block diagram of the camera system; Fig. 8 is a schematic view illustrating a configuration of a PC live view system; Fig. 9 shows a block diagram of a PC; Fig. 10 is a flowchart illustrating an operation of the camera; Fig. 11 shows a flowchart illustrating an operation of the PC; and Fig. 12A and Fig. 12B show schematic diagrams of an application screen. DESCRIPTION OF THE EMBODIMENTS

[0008] In the following description, embodiments of the present disclosure are described with reference to the accompanying drawings.

[0009] The Fig. 1A and Fig. 1B show external views illustrating an example of an external appearance of a digital camera (a camera) 100 according to the present embodiment. Fig. 1A shows a perspective view of the camera 100 when viewed from the front, and Fig. Figure 1B shows a perspective view of the camera 100 when viewed from the rear surface.

[0010] The camera 100 includes, on an upper surface thereof, a shutter button 101, a power switch 102, a mode selection switch 103, an electronic main dial 104, an electronic sub-dial 105, a movie button 106, and an external viewfinder display unit 107. The shutter button 101 is an operation member for providing a shooting preparation instruction or a shooting instruction. The power switch 102 is an operation member for turning the power of the camera 100 on or off. The mode selection switch 103 is an operation member for switching between various operation modes. The electronic main dial 104 is a rotary operation member for changing setting values ​​such as a shutter speed and an aperture value. The electronic sub-dial 105 is a rotary operation member for moving a selection frame (a pointer or a cursor).Cursors) and for feeding images. The movie button 106 is an operating element for providing an instruction to start or stop movie shooting (recording). The external viewfinder display unit 107 displays various setting values, such as a shutter speed and an aperture value.

[0011] The camera 100 includes, on the rear surface, a display unit 108, a touch panel 109, a direction button 110, a SET button 111, an AE lock button 112, a magnification button 113, a playback button 114, a menu button 115, an eyepiece section 116, an eyepiece detection unit 118, and a touch bar 119. The display unit 108 displays images and various types of information. The touch panel 109 is an operation member for detecting a touch operation on a display surface (touch operation surface) of the display unit 108. The direction button 110 is an operation member configured with a button (four-directional button) that can be pressed in the up, down, left, and right directions. Processing corresponding to the pressed position of the direction key 110 can be executed.The SET button 111 is an operation member to be pressed mainly when a selected subject is determined. The AE lock button 112 is an operation member to be pressed when an exposure state is fixed in a shooting standby state. The magnification button 113 is an operation member for turning on or off a magnification mode in a live view (LV) display in a shooting mode. When the magnification mode is turned on, a live view (LV) image is magnified or reduced by operating the electronic main dial 104. In addition, the magnification button 113 is used for enlarging a playback image or increasing a magnification ratio in a playback mode. The playback button 114 is an operation member for switching between the shooting mode and the playback mode.In the case of the recording mode, according to the pressing of the playback button 114, the mode switches to the playback mode, so that it is possible to display the last of images recorded in a recording medium 227, which will be described later, on the display unit 108.

[0012] The menu button 115 is an operation member to be pressed on the display unit 108 to display a menu screen enabling various settings. A user can perform various settings instinctively by using the menu screen displayed on the display unit 108, the direction button 110, and the SET button 111. The eyepiece section 116 is a section to which the user approaches and looks through an eyepiece viewfinder (look-through type) 117. The user can visually recognize an image displayed in an EVF 217 (Electronic View Finder), described below, positioned inside the camera 100, through the eyepiece section 116. The eyepiece detection unit 118 is a sensor that detects whether the user approaches the eyepiece section 116 (the eyepiece viewfinder 117) with his eyes.

[0013] The touch bar 119 is a linear touch operation element (a line touch sensor) that can receive a touch operation. The touch bar 119 is arranged at a position that allows a touch operation (touchable) with the thumb of the right hand in a state where a grip portion 120 is gripped with the right hand (a state where the grip portion 120 is gripped with the little finger, ring finger, and middle finger of the right hand), so that the shutter button 101 can be pressed by the index finger of the right hand. That is, the touch bar 119 can be operated in a state where the user approaches the eyepiece viewfinder 117 with their eyes, looks through the eyepiece portion 116, and holds the camera 100 upright to be able to press the shutter button 101 at any time (shooting orientation).The touch bar 119 can receive a tap operation on the touch bar 119 (an operation involving touching the touch bar and releasing the touch bar without moving the touch position within a predetermined period of time), a left or right slide operation (an operation involving touching the touch bar and thereby moving the touch position while maintaining the touch), and the like. The touch bar 119 is an operation element different from the touch panel 109 and does not have a display function. The touch bar 119 functions, for example, as a multi-function bar (M-Fn bar) to which various functions can be assigned.

[0014] In addition, the camera 100 also includes the grip portion 120, a thumb support portion 121, a terminal cover 122, a lid 123, a communication terminal 124, and the like. The grip portion 120 is a holding portion formed into a shape that the user can easily grasp with the right hand when holding the camera 100. The shutter button 101 and the main electronic dial 104 are arranged at positions where the user can operate the shutter button 101 and the main electronic dial 104 with the index finger of the right hand in a state where the user holds the camera 100 while gripping the grip portion 120 with the little finger, the ring finger, and the middle finger of the right hand.Likewise, in the same state, the electronic sub-dial 105 and the touch bar 119 are arranged at positions where the user can operate the electronic sub-dial 105 and the touch bar 119 with the thumb of the right hand. The thumb support portion 121 (thumb standby position) is a grip portion provided at a location where it is easy for the user to place the thumb of the right hand, which grips the grip portion 120, on the back of the camera 100 in a state where none of the operation members is operated. The thumb support portion 121 is configured with a rubber member to improve the gripping performance (the grip feeling). The terminal cover 122 protects connecting devices, such as connecting cables, for connecting the camera 100 to external devices (an external device).The lid 123 closes a slot for storing the recording medium 227, which will be described below, to protect the recording medium 227 and the slot. The communication port 124 is a port for communicating with a lens unit (a lens unit 200, a lens unit 300, or the like, which will be described below) that is attachable to and detachable from the camera 100.

[0015] Fig. 2 shows a block diagram illustrating an example of the configuration of the camera 100. In Fig. 2 the same components as those in the Fig. 1A and Fig. 1B by the same reference numerals as in the Fig. 1A and Fig. 1B, with a description of the components being appropriately omitted. In Fig. 2, the lens unit 200 is attached to the camera 100.

[0016] First, the lens unit 200 will be described. The lens unit 200 is a type of interchangeable lens unit (interchangeable lens) that can be attached to and detached from the camera 100. The lens unit 200 is a single-lens unit (single lens) and is an example of a normal lens unit. The lens unit 200 includes a diaphragm 201, a lens 202, a diaphragm drive circuit 203, an autofocus (AF) drive circuit 204, a lens system control circuit 205, and a communication terminal 206, among others.

[0017] The aperture 201 is configured so that an aperture diameter is adjustable. The lens 202 is configured with a plurality of lenses. The aperture control circuit 203 adjusts an amount of light by controlling the aperture diameter of the aperture 201. The AF control circuit 204 adjusts the focus by driving the lens 202. The lens system control circuit 205 controls the aperture control circuit 203, the AF control circuit 204, and the like based on instructions from a system control unit 50, which will be described below. The lens system control circuit 205 controls the aperture 201 via the aperture control circuit 203 and adjusts the focus by changing the position of the lens 202 via the AF control circuit 204. The lens system control circuit 205 can communicate with the camera 100.Specifically, communication is performed via the communication port 206 of the lens unit 200 and the communication port 124 of the camera 100. The communication port 206 is a port that allows the lens unit 200 to communicate with the camera 100 side.

[0018] Next, the camera 100 will be described. The camera 100 includes a shutter 210, an imaging unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, the EVF 217, the display unit 108, and the system control unit 50.

[0019] The shutter 210 is a focal plane shutter that can freely control the exposure time of the imaging unit 211 based on an instruction from the system control unit 50. The imaging unit 211 is an imaging element (an image sensor) configured with a CCD, a CMOS element, or the like, which converts an optical image into an electrical signal. The imaging unit 211 may include an imaging surface phase difference sensor for outputting defocus amount information to the system control unit 50. The A / D converter 212 converts an analog signal output from the imaging unit 211 into a digital signal. The image processing unit 214 performs predetermined processing (pixel interpolation, resizing processing such as downsizing, color conversion processing, and the like) on data from the A / D converter 212 or data from the memory control unit 213.In addition, the image processing unit 214 performs predetermined arithmetic processing using captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the obtained calculation result. Through this processing, through-the-lens (TTL) type AF processing, automatic exposure (AE) processing, EF (flash pre-flash) processing, and the like are performed. Furthermore, the image processing unit 214 performs predetermined arithmetic processing using the captured image data, and the system control unit 50 performs TTL type auto white balance (ABB) processing based on the obtained calculation result.

[0020] The image data from the A / D converter 212 is written into the memory 215 via the image processing unit 214 and the memory control unit 213. Alternatively, the image data from the A / D converter 212 is written into the memory 215 via the memory control unit 213 without the intervention of the image processing unit 214. The memory 215 stores the image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212, and image data to be displayed on the display unit 108 or the EVF 217. The memory 215 includes a storage capacity sufficient to store a predetermined number of still images and a predetermined length of moving images and voice. Also, the memory 215 serves as a memory for displaying an image (video memory).

[0021] The D / A converter 216 converts image data for display stored in the memory 215 into an analog signal and supplies the analog signal to the display unit 108 or the EVF 217. Accordingly, the image data for display written in the memory 215 is displayed on the display unit 108 or the EVF 217 via the D / A converter 216. The display unit 108 and the EVF 217 provide a display in response to the analog signal from the D / A converter 216. The display unit 108 and the EVF 217 are, for example, LCD or organic EL displays. The digital signals A / D converted by the A / D converter 212 and accumulated in the memory 215 are converted into the analog signals by the D / A converter 216, and the analog signals are sequentially transmitted to and displayed on the display unit 108 or the EVF 217, so that a live view display is performed.

[0022] The system control unit 50 is a control unit that includes at least one processor and / or at least one circuit. That is, the system control unit 50 may be a processor, a circuit, or a combination of a processor and a circuit. The system control unit 50 controls the entire camera 100. The system control unit 50 implements the processing of flowcharts described below by executing programs recorded in a non-volatile memory 219. In addition, the system control unit 50 also performs display control by controlling the memory 215, the D / A converter 216, the display unit 108, the EVF 217, and the like.

[0023] The camera 100 also includes a system memory 218, the non-volatile memory 219, a system timer 220, a communication unit 221, an alignment detection unit 222, and an eyepiece detection unit 118.

[0024] For example, a RAM is used as the system memory 218. Constants, variables, and programs read from the non-volatile memory 219 for the operation of the system control unit 50 are loaded into the system memory 218. The non-volatile memory 219 is an electrically erasable and writable memory. For example, an EEPROM is used as the non-volatile memory 219. Constants, programs, and the like for the operation of the system control unit 50 are recorded in the non-volatile memory 219. The program used here includes programs for executing the flowcharts described below. The system timer 220 is a timer unit that counts a time used for various types of control and a time of a built-in clock.The communication unit 221 transmits and receives a video signal and a voice signal to and from an external device connected wirelessly or via a wired cable. The communication unit 221 is also connectable to a wireless local area network (LAN) and the Internet. Furthermore, the communication unit 221 can also communicate with an external device via Bluetooth (registered trademark) and Bluetooth Low Energy. The communication unit 221 can transmit an image captured by the imaging unit 211 (including a live image) and an image recorded in the recording medium 227, and can receive an image and other various types of information from an external device. The orientation detection unit 222 is an orientation detection sensor that detects the orientation of the camera 100 with respect to the direction of gravity.Based on the orientation detected by the orientation detection unit 222, it can be determined whether an image captured by the imaging unit 211 is an image captured with the camera 100 held in a horizontal position or held in a vertical position. The system control unit 50 can add orientation information corresponding to the orientation detected by the orientation detection unit 222 to an image file of the image captured by the imaging unit 211 and can rotate the image according to the detected orientation. For example, an acceleration sensor or a gyro sensor can be used for the orientation detection unit 222. It is also possible to detect the movement of the camera 100 (whether it is panning, tilting, lifting, stationary, or the like) using the orientation detection unit 222.

[0025] The eyepiece detection unit 118 can detect an object approaching the eyepiece portion 116 (the eyepiece image viewfinder 117). For example, an infrared environment sensor can be used as the eyepiece detection unit 118. When an object approaches, infrared light emitted from a light-emitting portion of the eyepiece detection unit 118 is reflected by the object and received by a light-receiving portion of the infrared environment sensor. A distance from the eyepiece portion 116 to the object can be determined according to the amount of received infrared light. In this way, the eyepiece detection unit 118 performs eye proximity detection for detecting a distance between the eyepiece portion 116 and the object approaching the eyepiece portion 116.The eyepiece detection unit 118 is an eyepiece detection sensor that detects an approach (eye approach) and a separation (eye separation) of an eye (object) to and from the eyepiece portion 116. In a case where an object approaching the eyepiece portion 116 within a predetermined distance is detected in a non-eye approach state (non-approach state), the eyepiece detection unit 118 detects that an eye is approaching. Meanwhile, in a case where the object whose approach is detected is separated by a predetermined distance or further in an eye approach state (approach state), the eyepiece detection unit 118 detects that an eye is separated. A threshold value for detecting eye approach and a threshold value for detecting eye separation may be different, for example, to provide hysteresis.In addition, after eye approach is detected, the eye approach state is assumed until eye separation is detected. After eye separation is detected, the non-eye approach state is assumed until eye approach is detected. The system control unit 50 switches between a display (display state) and a non-display (non-display state) of each of the display unit 108 and the EVF 217 according to the state detected by the eyepiece detection unit 118. Specifically, in a case where at least the shooting standby state is established and a switching setting for a display target is set to automatic switching, the display target is set as the display unit 108, and the display is turned on while the EVF 217 is set to non-display during the non-eye approach state.In addition, the display target is set as the EVF 217, and the display is turned on, while the display unit 108 is set to not display during the eye approach state. Note that the eyepiece detection unit 118 is not limited to the infrared environment sensor; sensors other than the eyepiece detection unit 118 can be used as long as the sensors can detect the state that can be considered eye approach.

[0026] Also, the camera 100 includes the external viewfinder display unit 107, an external viewfinder display unit driving circuit 223, a power supply control unit 224, a power supply unit 225, a recording medium I / F 226, and an operation unit 228.

[0027] The external viewfinder display unit 107 is driven by the external viewfinder display unit drive circuit 223 and displays various setting values ​​for the camera 100, such as a shutter speed and an aperture value. The power supply control unit 224 is configured with a battery detection circuit, a DC-DC converter, a switching circuit that switches the power supply block, and the like, and detects whether a battery is attached, the type of battery, the remaining battery level, and the like. Furthermore, the power supply control unit 224 controls the DC-DC converter based on the detection result and an instruction from the system control unit 50, supplying a required voltage to portions including the recording medium 227 for a required period of time.The power supply unit 225 is a primary battery such as alkaline batteries and lithium batteries, a secondary battery such as NiCd, NiMH, and Li batteries, an AC adapter, or the like. The recording medium I / F 226 is an interface for the recording medium 227, such as a memory card and a hard disk. The recording medium 227 is a memory card for recording captured images and the like, and is configured with a semiconductor memory, a magnetic disk, and the like. The recording medium 227 can be attachable to and detachable from the camera 100, and can also be embedded in the camera 100.

[0028] The operation unit 228 is an input unit (receiving unit) that can receive operations from the user (user operations) and is used to input various instructions to the system control unit 50. The operation unit 228 includes the shutter button 101, the power switch 102, the mode selection switch 103, the touch panel 109, another operation unit 229, and the like. The other operation unit 229 includes the electronic main dial 104, the electronic sub dial 105, the movie button 106, the direction button 110, the SET button 111, the AE lock button 112, the magnification button 113, the playback button 114, the menu button 115, and the touch bar 119.

[0029] The shutter button 101 includes a first shutter switch 230 and a second shutter switch 231. The first shutter switch 230 is turned on in the middle of the operation of the shutter button 101 in response to a so-called half-press (shooting preparation instruction) and outputs a first shutter switch signal SW1. The system control unit 50 starts shooting preparation processing, such as AF processing, AE processing, ABB processing, and EF processing, in response to the first shutter switch signal SW1. The second shutter switch 230 is turned on at the completion of the operation of the shutter button 101 in response to a so-called full press (shooting instruction) and outputs a second shutter switch signal SW2. In response to the second shutter switch signal SW2, the system control unit 50 starts a sequence of shooting processing that includes reading a signal from the Fig. , creating an image file comprising the captured image, and writing the created image file to the recording medium 227.

[0030] The mode selection switch 103 switches the operation mode of the system control unit 50 to any one of a still image shooting mode, a movie shooting mode, and a playback mode. Examples of the still image shooting mode include an automatic shooting mode, an automatic scene determination mode, a manual mode, an aperture priority (Av) mode, a shutter speed priority (Tv) mode, and a program AE (P) mode. Examples of the modes also include various scene modes having shooting settings for different shooting scenes, a user-defined mode, and the like. The user can directly switch the mode to any of the above-described shooting modes with the mode selection switch 103.Alternatively, the user may once switch a screen to a list screen of recording modes using the mode selection switch 103, and then selectively switch the mode to any of a plurality of displayed modes using the operation unit 228. Similarly, the movie recording mode may include a plurality of modes.

[0031] The touch panel 109 is a touch sensor for detecting various touch operations on the display surface of the display unit 108 (the operation surface of the touch panel 109). The touch panel 109 and the display unit 108 may be integrally configured. For example, the touch panel 109 is attached to an upper layer of the display surface of the display unit 108 so that light transmittance does not obstruct the display on the display unit 108. Furthermore, input coordinates on the touch panel 109 and display coordinates on the display surface of the display unit 108 are connected to each other, thereby configuring a graphical user interface (GUI) that allows the user to operate a screen displayed on the display unit 108 as if the user were directly operating the screen.The touch panel 109 may use any of various methods, including a film-like resistive layer, capacitive, surface acoustic wave, infrared, electromagnetic induction, image recognition, optical sensor methods, and the like. Depending on the methods, there is a method for detecting a touch based on contact with the touch panel 109 and a method for detecting a touch based on approaching a finger or stylus to the touch panel 109, but any method may be used.

[0032] For the touch panel 109, the system control unit 50 can detect the following operations or states: - An operation in which a finger or a stylus not in contact with the touch pad 109 re-touches the touch pad 109, ie, a start of a touch (hereinafter referred to as touch-down). - A state in which the finger or the stylus is in contact with the touch panel (109) (hereinafter referred to as touch-on). - An operation in which the finger or the pen moves while in contact with the touch pad 109 (hereinafter referred to as touch move). - An operation in which the finger or the stylus in contact with the touch pad 109 is separated (released) from the touch pad 109, that is, an end of touch (hereinafter referred to as touch-up). - A state in which nothing is in contact with the touch pad 109 (hereinafter referred to as touch-off).

[0033] When the touch-down is detected, the touch-on is detected at the same time. After the touch-down, the touch-on is normally continuously detected as long as the touch-up is not detected. Likewise, when the touch-move is detected, the touch-on is continuously detected. Even if the touch-on is detected, the touch-move is not detected as long as the touch position is not moved. After a touch-up of both the finger and the stylus that have been in contact with the touch pad 109 is detected, the touch-off is established.

[0034] These operations and states, and the position coordinates of the finger or stylus in contact with the touch panel 109, are reported to the system control unit 50 through an internal bus. The system control unit 50 determines what type of operation (touch operation) is being performed on the touch panel 109 based on the reported information. Regarding the touch move, a movement direction of the finger or stylus moving on the touch panel 109 can be determined for each vertical component and for each horizontal component on the touch panel 109 based on a change in the position coordinates. When the touch move is detected for a predetermined distance or longer, it is determined that a sliding operation is being performed.An operation in which a finger is quickly moved a certain distance while in contact with the touch pad 109 and separated is called a flick. In other words, a flick is an operation in which the finger is quickly slid on the touch pad 109 to flick the touch pad 109. When the touch move is detected for a predetermined distance or longer at a predetermined speed or higher, and then the touch-up is detected without change, it is determined that the flick is being performed (it may be determined that the flick is being performed subsequent to the same operation). Furthermore, a touch operation in which a plurality of locations (e.g., two points) are both touched (multi-touch) and the touch positions are brought close to each other is called a pinch-in.A touch operation that moves the touch positions away from each other is called a pinch-out. Pinch-out and pinch-in are collectively referred to as a pinch operation (or, for simplicity, as a pinch or push).

[0035] Fig. 3 is a schematic diagram illustrating an example of the configuration of the lens unit 300. Fig. 3 illustrates a state in which the lens unit 300 is attached to the camera 100. In the camera 100 shown in Fig. 3, the same components as those shown in Fig. 2 by the same reference numerals as in Fig. 2, a description thereof is appropriately omitted. Components relating to the right eye are denoted by R at the end of the reference numeral, components relating to the left eye are denoted by L at the end of the reference numeral, and components relating to both the right eye and the left eye are denoted by neither R nor L at the end.

[0036] The lens unit 300 is a type of interchangeable lens unit that is attachable to and detachable from the camera 100. The lens unit 300 is a dual-lens unit capable of capturing a right image and a left image with parallax. The lens unit 300 includes two optical systems, each of which can capture an image within a wide field of view of approximately 180 degrees. Specifically, each of the two optical systems of the lens unit 300 can capture an image of an object corresponding to a field of view (angle of view) of 180 degrees in the left-right direction (horizontal angle, azimuth angle, yaw angle) and 180 degrees in the up-down direction (vertical angle, elevation angle, pitch angle). That is, each of the two optical systems can capture an image within a forward hemispherical area.

[0037] The lens unit 300 includes an optical system 301R including a plurality of lenses, reflection mirrors, and the like, an optical system 301L including a plurality of lenses, reflection mirrors, and the like, and a lens system control circuit 303. The optical system 301R includes a lens 302R arranged near the object, and the optical system 301L includes a lens 302L arranged near the object. That is, the lens 302R and the lens 302L are arranged on the object side of the lens unit 300. The lens 302R and the lens 302L are aligned in the same direction, with their respective optical axes substantially parallel to each other.

[0038] The lens unit 300 is a dual lens unit (VR180 lens unit) for obtaining a VR180 image, which is one of virtual reality (VR) image formats capable of binocular stereoscopic vision. In the lens unit 300, each of the optical system 301R and the optical system 301L includes a fisheye lens capable of capturing a range of approximately 180 degrees. Note that the range that can be captured by the lens of each of the optical system 301R and the optical system 301L may be a range of approximately 120 degrees or 160 degrees, which is narrower than the range of 180 degrees. In other words, the optical systems 301R and 301L may be in the range of approximately 120° to approximately 180°.The lens unit 300 can form a right image formed by the optical system 301R and a left image formed by the optical system 301L at one or two imaging elements of the camera to which the lens unit 300 is attached. In the camera 100, the right image and the left image are formed on one imaging element (imaging sensor), forming an image (binocular image) in which a right image area (right image area) and a left image area (left image area) are arranged side by side.

[0039] The lens unit 300 is attached to the camera 100 via a lens attachment portion 304 of the lens unit 300 and a camera attachment portion 305 of the camera 100. Thus, the system control unit 50 of the camera 100 and the lens system control circuit 303 of the lens unit 300 are electrically connected to each other via the communication port 124 of the camera 100 and a communication port 306 of the lens unit 300.

[0040] In Fig. 3, the right image formed by the optical system 301R and the left image formed by the optical system 301L are formed side by side in the imaging unit 211 of the camera 100. In other words, the optical system 301R and the optical system 301L form two optical images (object images) in the two areas of one imaging element (imaging sensor). The imaging unit 211 converts the formed optical image (optical signal) into an analog electrical signal. By using the lens unit 300 in this way, an image including two image areas with a parallax can be acquired from two locations (optical systems) of the optical system 301R and the optical system 301L. By dividing the acquired image into a left-eye image and a right-eye image and performing VR display of the images, the user can see a three-dimensional VR image in a range of about 180 degrees.This means that the user can see the VR180 image stereoscopically.

[0041] Here, a VR image is an image that can be viewed in a VR display, which is described below. Examples of VR images include an omnidirectional image (complete spherical image) captured by an omnidirectional camera (complete spherical camera), and a panoramic image, which has a wider video area (effective video area) than a display area that can be displayed at one time on a display unit. Examples of VR images also include a moving image, a live image (an image acquired from a camera in essentially real time), and a still image. The VR image has a maximum video area (effective video area) corresponding to a field of view of 360 degrees in a left-right direction and 360 degrees in a top-down direction.Examples of the VR image also include images having a view angle wider than a view angle that can be captured by a normal camera or a video range wider than a display range that can be displayed at one time on the display unit, even if the view angle or video range is smaller than 360 degrees in the left-right direction and 360 degrees in the up-down direction. An image captured by the camera 100 having the lens unit 300 described above is one type of VR image. The VR image can be viewed in a VR display by, for example, setting a display mode of a display device (a display device capable of displaying a VR image) to "VR View."A specific area of ​​a VR image with a 360-degree viewing angle is displayed in such a way that the user can watch a seamless omnidirectional video in the left-right direction by changing the orientation of the display device to the left-right direction (horizontal rotation direction) to move the displayed area.

[0042] VR display (VR view) is a display method (a display mode) for displaying, from VR images, a video in a field of view depending on the orientation of the display device, the display method being capable of changing a corresponding display area. Examples of VR display include "single-lens VR display (single-lens VR view)" in which an image is displayed after deformation (distortion correction) to fit a VR image to a virtual sphere. Examples of VR display include "dual-lens VR display (dual-lens VR view)" in which a left-eye VR image and a right-eye VR image are displayed side by side in left and right areas after deformation to fit the VR images to a virtual sphere.The "dual-lens VR display" is implemented by using the left-eye VR image and the right-eye VR image with parallax, thereby achieving stereoscopic vision of the VR images. For each type of VR display, for example, when the user wears a display device such as a head-mounted display (HMD), a video is displayed in the field of view corresponding to the user's facial orientation. For example, it is assumed that among the VR images, a video is displayed in a field of view corresponding to a center of 0 degrees in the left-right direction (a specific orientation, such as north) and 90 degrees in the up-down direction (90 degrees from the zenith, which is the horizon) at a given time.In this state, when the orientation of the display device is reversed (for example, the display surface is changed from a south direction to a north direction), from the same VR images, the display area is changed to a video in a field of view area having the corresponding center at 180 degrees in the left-right direction (the opposite orientation, such as south) and 90 degrees in the up-down direction. That is, when the user wearing the HMD faces south from north (or looks backward), the video displayed on the HMD is changed from a video of the north to a video of the south. Note that the VR image captured with the lens unit 300 is an image (180-degree image) obtained by capturing the area of ​​approximately 180 degrees forward, with no video present in the area of ​​approximately 180 degrees backward.In the VR display of such an image, if the orientation of the display device is changed to a side where there is no video image, a blank area is displayed.

[0043] Such VR display of a VR image makes the user visually feel as if they exist in the VR image (in a VR space) (a sense of immersion). Note that the VR image display method is not limited to the method of changing the orientation of the display device. For example, the display area can be moved (scrolled) in response to user operation via a touch panel, directional buttons, or the like. In addition to changing the display area by changing the orientation during VR display (in the "VR View" display mode), the display area can be changed in response to a touch move on the touch panel, a drag operation with a mouse device or the like, or pressing the directional buttons. In addition, a smartphone attached to VR glasses (a head-mounted adapter) is one type of HMD.

[0044] The configuration of the lens unit 300 is described in more detail. Fig. Fig. 4 is a cross-sectional view illustrating an example of a configuration of the lens unit 300, wherein the Fig. 5A and Fig. 5B show exploded perspective views illustrating an example of a configuration of the lens unit 300. Fig. 5A shows a perspective view of the lens unit 300 when viewed from the front, and Fig. 5B shows a perspective view of the lens unit 300 when viewed from the back.

[0045] Each of the optical system 301R and the optical system 301L is fixed to a lens top base 310 by screw fastening or the like. The optical axes of the optical system 301R, from the object side, include a first optical axis OA1R, a second optical axis OA2R substantially orthogonal to the first optical axis OA1R, and a third optical axis OA3R substantially parallel to the first optical axis OA1R. Similarly, the optical axes of the optical system 301L include a first optical axis OA1L, a second optical axis OA2L, and a third optical axis OA3L.

[0046] The optical system 301R includes a first lens 311R, a second lens 321R, a third lens 331R, and a fourth lens 341R. The first lens 311R is arranged on the first optical axis OA1R, with a surface 311AR of the first lens 311R on the object side having a convex shape. The second lens 321R is arranged on the second optical axis OA2R. The third lens 331R and the fourth lens 341R are arranged on the third optical axis OA3R. Similarly, the optical system 301L includes a first lens 311L, a second lens 321L, a third lens 331L, and a fourth lens 341L.

[0047] Furthermore, the optical system 301R includes a first prism 320R and a second prism 330R. The first prism 320R diffracts the light flux entering the first lens 311R from the object side in a direction substantially parallel to the second optical axis OA2R from a direction substantially parallel to the first optical axis OA1R, and guides the light flux to the second lens 321R. The second prism 330R diffracts the light flux entering the second lens 321R from a direction substantially parallel to the second optical axis OA2R into a direction substantially parallel to the third optical axis OA3R, and guides the light flux to the third lens 331R (and the fourth lens 341R). Similarly, the optical system 301L includes a first prism 320L and a second prism 330L.

[0048] Fig. 6 is a schematic diagram illustrating a positional relationship between each optical axis and an image circle on the imaging unit 211. An image circle ICR corresponding to the effective angle of view of the optical system 301R and an image circle ICL corresponding to the effective angle of view of the optical system 301L are formed in parallel on the imaging unit 211 of the camera 100. Diameters ΦD2 of the image circles ICR and ICL and the distance between the image circle ICR and the image circle ICL are preferably set such that the image circle ICR and the image circle ICL do not overlap each other.For example, the arrangement of the image circles ICR and ICL is set such that the center of the image circle ICR is located substantially at the center of the right area, and the center of the image circle ICL is located substantially at the center of the left area among two areas obtained by dividing the light receiving area of ​​the imaging unit 211 into the left and right directions. The size and arrangement of the image circles ICR and ICL are determined, for example, by the configuration of the lens unit 300 (and the camera 100).

[0049] In Fig. 6, a distance L1 is a distance (a baseline length) between the first optical axis OA1R and the first optical axis OA1L. In stereoscopic viewing of the image obtained by using the lens unit 300, a higher stereoscopic effect can be obtained when the baseline length L1 is longer. For example, assume that the sensor size (size of an imaging surface (light-receiving surface, light-receiving area)) of the imaging unit 211 is 24 mm in length × 36 mm in width, where the diameters ΦD2 of the image circles ICR and ICL are 17 mm. Similarly, a distance L2 between the third optical axis OA3R and the third optical axis OA3L is 18 mm, where the lengths of the second optical axes OA2R and OA2L are 21 mm.

[0050] Assuming that the second optical axes OA2R and OA2L extend in the horizontal direction, the baseline length L1 is 60 mm, which is essentially equal to the eye width of an adult (the distance between the right eye and the left eye).

[0051] A diameter ΦD of the lens mounting portion 304 may be longer or shorter than the base length L1. When a distance L2 between the third optical axis OA3R and the third optical axis OA3L is shorter than the diameter ΦD of the lens mounting portion 304, the third lenses 331R and 331L and the fourth lenses 341R and 341L may be arranged within the lens mounting portion 304. Fig. 6 a relationship according to L1>ΦD>L2 is established.

[0052] When a dual-lens VR display with a field of view (angle of view) of approximately 120 degrees is implemented, a sufficient stereoscopic effect can be obtained. However, since an uncomfortable feeling remains when the field of view is approximately 120 degrees, the angle of view (effective angle of view) of the optical systems 301R and 301L is often approximately 180 degrees. Fig. 6, the angle of view (the effective angle of view) of the optical systems 301R and 301L is greater than 180 degrees, with a diameter ΦD3 of the image circle in the range of 180 degrees being smaller than the diameter ΦD2 of the image circles ICR and ICL.

[0053] Fig. Fig. 7 is a block diagram illustrating an example of a configuration of a camera system according to the present embodiment. The camera system in Fig. 7 includes the camera 100 and the lens unit 300.

[0054] The lens unit 300 includes the optical systems 301R and 301L, driving units 363R and 363L, and a lens information storage unit 350. The optical systems 301R and 301L are as described above. The driving unit 363R drives the optical system 301R, and the driving unit 363L drives the optical system 301L. The lens information storage unit 350 stores lens information related to the lens unit 300. The lens information includes, for example, configuration information of the optical systems 301R and 301L. The lens information may include information (identifiers) indicating whether the lens unit 300 is a dual lens unit (a lens unit for obtaining a VR image capable of binocular stereoscopic vision).

[0055] As described above, the camera 100 includes the imaging unit 211, the operation unit 228, and the system control unit 50. The system control unit 50 includes a parallax calculation unit 152, a focus detection unit 153, and a drive amount determination unit 154. It should be noted that the parallax calculation unit 152, the focus detection unit 153, and the drive amount determination unit 154 may be included in a device separate from the camera 100. For example, these components may be included in the lens system control circuit 303 (in Fig. 7 not illustrated) of the lens unit 300 or may be included in a personal computer (PC) 500 described below.

[0056] As described above, the imaging unit 211 is configured with an imaging element, and the right image formed via the optical system 301R and the left image formed via the optical system 301L are formed on the imaging surface of the imaging unit 211. The operation unit 228 includes, for example, a touch panel or a joystick and is used by the user to determine an AF position (a focus detection position) in the AF processing.

[0057] The parallax calculation unit 152 calculates a parallax amount between the right image formed via the optical system 301R and the left image formed via the optical system 301L based on the lens information stored in the lens information storage unit 350. Based on the calculated parallax amount and the AF position (AF position in the right image) corresponding to the optical system 301R, the parallax calculation unit 152 determines the AF position (AF position in the left image) corresponding to the optical system 301L.

[0058] These two AF positions are image formation positions of the same subject. The parallax calculation unit 152 can determine the AF position corresponding to the optical system 301R based on the calculated parallax amount and the AF position corresponding to the optical system 301L.

[0059] The focus detection unit 153 acquires an AF evaluation value (a focus detection evaluation value) for the AF position designated by the user or the AF position designated by the parallax calculation unit 152. For example, when the AF position corresponding to the optical system 301R is designated by the user, the AF position corresponding to the optical system 301L is designated by the parallax calculation unit 152. Then, two AF evaluation values ​​corresponding to the two AF positions are acquired by the focus detection unit 153.

[0060] The drive amount determination unit 154 determines the drive amount of the optical system 301R and the drive amount of the optical system 301L based on the AF evaluation value acquired by the focus detection unit 153, outputs the drive amount of the optical system 301R to the drive unit 363R, and outputs the drive amount of the optical system 301L to the drive unit 363L. The drive units 363R and 363L drive the optical systems 301R and 301L with the drive amount determined by the drive amount determination unit 154.

[0061] Fig. Fig. 8 is a schematic view illustrating an example of an overall configuration of the PC live view system according to the present embodiment. The PC live view system in Fig. 8 includes the camera 100 and the PC 500. The lens unit 300 is attached to (connected to) the camera 100. As described above, by attaching the lens unit 300, the camera 100 can capture a single image (a still image or a movie) including two image areas having a prescribed parallax. The PC 500 is an information processing device that handles an image captured by the imaging device such as the camera 100. Fig. 8 illustrates a configuration in which the camera 100 and the PC 500 are communicatively connected to each other wirelessly or by a wire.

[0062] Fig. 9 is a block diagram illustrating an example of a configuration of the PC 500. The PC 500 includes a CPU 501, a memory 502, a non-volatile memory 503, an operation unit 504, a display unit 505, and an external I / F 506.

[0063] For example, the CPU 501 controls each unit of the PC 500 by using the RAM 502 as a working memory according to a program stored in the non-volatile memory 503. The RAM 502 is configured, for example, with a RAM (volatile memory using a semiconductor element or the like). The non-volatile memory 503 stores image data, audio data, other data, various programs for operating the CPU 501, and the like. The non-volatile memory 503 is configured, for example, with a hard disk (HD), a ROM, and the like.

[0064] The operation unit 504 is an input device (receiving unit) capable of receiving a user operation. For example, the operation unit 504 includes a character information input device such as a keyboard, a pointing device such as a mouse or a touch pad, a knob, a dial, a joystick, a touch sensor, and a touch panel. The operation unit 504 is used, for example, by a user to determine an AF position during AF processing.

[0065] The display unit 505 displays various images, screens, and the like under the control of the CPU 501. For example, the display unit 505 displays a live view image obtained by the camera 100 or displays a GUI screen that configures a graphical user interface (GUI). The CPU 501 controls each unit of the PC 500 to generate a display control signal according to a program, generate a video signal to be displayed on the display unit 505, and output the video signal to the display unit 505. Note that the display unit 505 may be configured with an external monitor (a television or the like).

[0066] The external I / F 506 is an interface for connecting to an external device (for example, the camera 100) through a hard line cable or wirelessly and for performing input / output (data communication) of a video signal or an audio signal.

[0067] Fig. Fig. 10 shows a flowchart illustrating an example of the operation of the camera 100. These operations are implemented by loading a program recorded in the non-volatile memory 219 into the system memory 218 and executing the program by the system control unit 50. For example, when the camera 100 is started, the operations according to Fig. 10. The companies according to Fig. 10 are operations for a function for displaying a live view image captured by the camera on the display unit of the PC (PC Live View). The operations according to Fig. 10 are executed in a case where the camera 100 is in a recording standby state. If an instruction to start recording is input from the PC 500 during PC live view operation, still image capture or movie recording is executed. At this point, the PC live view can be resumed.

[0068] In step S1001, the system control unit 50 determines whether the camera 100 is compatible with a dual lens unit (a lens unit for obtaining a VR image capable of binocular stereoscopic vision, for example, the lens unit 300). For example, the system control unit 50 determines whether the firmware version of the system control unit 50 is compatible with the dual lens unit. If it is determined that the version is compatible with the dual lens unit, the processing proceeds to step S1002; otherwise, the operation according to Fig. 10 is terminated. In the operation according to Fig. 10, a dual lens unit with an AF function (a dual lens unit capable of performing AF processing of each optical system) is adopted as the dual lens unit.

[0069] In step S1002, the system control unit 50 determines whether the dual lens unit is attached to the camera 100. The system control unit 50 acquires, for example, lens unit identification information from the lens unit attached to the camera 100 and determines whether the lens unit is a dual lens unit according to the acquired identification information. This identification information is a part of the lens information described above. If it is determined that the dual lens unit is attached, the processing proceeds to step S1003; otherwise, the operation according to Fig. 10 is terminated.

[0070] In step S1003, the system control unit 50 acquires a design value of the dual lens unit from the attached (connected) dual lens unit. This design value is a piece of the lens information described above. One or more design values ​​as described above may be acquired.

[0071] In step S1004, the system control unit 50 acquires a dual-lens unit individual value from the attached (connected) dual-lens unit. The individual value is a parameter specific to the lens unit, such as a defect during manufacturing. The individual value is a piece of the lens information described above. One or more individual values ​​may be acquired, and each individual value may be a parameter specific to the lens unit.

[0072] In step S1005, the camera 100 is connected to the PC 500, and the system control unit 50 detects the connection of the camera 100 to the PC 500.

[0073] In step S1006, the system control unit 50 receives a request from the PC 500 to start a PC live view.

[0074] In step S1007, the system control unit 50 receives a request for a live view image from the PC 500.

[0075] In step S1008, the system control unit 50 converts the information obtained in steps S1003 and S1004 (lens information) to conform to the coordinate system of the live view image to be transmitted. The information obtained in steps S1003 and S1004 cannot be used for image processing of the live view image as it is. Thus, the lens information is converted into information that conforms to the coordinate system of the live view image.

[0076] In step S1009, the system control unit 50 transmits the lens information converted in step S1008 and the live view image to the PC 500. In the present embodiment, the system control unit 50 of the camera 100 converts the lens information. However, the CPU 501 of the PC 500 may convert the lens information. In such embodiments, the lens information before conversion and a parameter required for converting the lens information are transmitted to the PC 500.

[0077] In step S1010, the system control unit 50 determines whether there is an AF instruction from the PC 500. If it is determined that there is an AF instruction, the processing proceeds to step S1011; otherwise, the process proceeds to step S1012.

[0078] In step S1011, the system control unit 50 executes the AF processing according to the AF instruction. This AF instruction is, for example, the AF instruction sent from the PC 500 in step S1124 in Fig. 11. The AF instruction includes coordinates (coordinate information) of the AF position designated by the user. The system control unit 50 performs AF processing of the optical systems 301R and 301L based on the designated AF positions. The parallax calculation unit 152 acquires two AF positions corresponding respectively to the two optical systems 301R and 301L, and the focus detection unit 153 determines two AF evaluation values ​​corresponding respectively to the two AF positions. Then, the drive amount determination unit 154 determines the drive amount of the optical system 301R, outputs the drive amount to the drive unit 363R, determines the drive amount of the optical system 301L, and outputs the drive amount to the drive unit 363L.

[0079] In step 1012, the system control unit 50 determines whether to terminate the PC live view. For example, if the camera 100 and the PC 500 are disconnected or the user instructs the camera 100 or the PC 500 to terminate the PC live view, it is determined that the PC live view is to be terminated. If it is determined that the PC live view is to be terminated, the operations are performed according to Fig. 10 is terminated, otherwise the process proceeds to step S1007.

[0080] Fig. Fig. 11 is a flowchart illustrating an example of the operations of the PC 500. These operations are implemented by loading programs (application programs) recorded in the non-volatile memory 503 into the main memory 502 by the CPU 501. For example, when the user instructs the PC 500 to start a specific application, the operations are performed according to Fig. 11. The companies according to Fig. 11 are operations for a function of displaying a live view image captured by the camera on the display unit of the PC (PC live view).

[0081] In step S1101, the camera (for example, the camera 100) is connected to the PC 500, and the CPU 501 detects the connection of the camera to the PC 500.

[0082] In step S1102, the CPU 501 determines whether the camera connected in step S1101 is a camera capable of being compatible with a dual lens unit (a lens unit for obtaining a VR image capable of binocular stereoscopic vision, for example, the lens unit 300). The CPU 501 acquires, for example, model information about the camera from the connected camera and determines whether the camera is compatible with the dual lens unit based on the acquired model information. If it is determined that the camera is compatible with the dual lens unit, the process proceeds to step S1103; otherwise, the operation according to Fig. 11 is terminated. The camera compatible with the dual lens unit is, for example, a camera to which a dual lens unit can be attached. In the operation according to Fig. 11, a dual lens unit with an AF function (a dual lens unit capable of performing AF processing of each optical system) is adopted as the dual lens unit.

[0083] In step S1103, the CPU 501 determines whether the firmware of the camera connected in step S1101 is compatible with the dual lens unit. For example, the CPU 501 acquires version information about the camera firmware from the connected camera and determines, based on the acquired information, whether the version of the firmware of the connected camera is compatible with the dual lens unit. If it is determined that the version is compatible with the dual lens unit, the process proceeds to step S1104; otherwise, the operation according to Fig. 11 is terminated.

[0084] In step S1104, the CPU 501 determines whether the dual lens unit is attached to the camera connected in step S1101. For example, the CPU 501 acquires information from the connected camera indicating whether the dual lens unit is attached to the camera and determines whether the dual lens unit is attached to the connected camera based on the acquired information. If it is determined that the dual lens unit is attached, the process proceeds to step S1105; otherwise, the operation according to Fig. 11 is terminated.

[0085] In step S1105, the CPU 501 transmits a request to start a PC live view to the camera connected in step S1101.

[0086] In step S1106, the CPU 501 transmits a request for the live view image to the camera connected in step S1101.

[0087] In step S1107, the CPU 501 receives from the camera connected in step S1101 the live view image captured by the camera and lens information of the dual lens unit attached to the camera. The received lens information is information that is converted to conform to the received live view image (for example, the lens information acquired in step S1008 according to Fig. 10). The received live view image and lens information are, for example, the live view image and lens information transmitted from the camera 100 in step S1009.

[0088] Assume that the lens information received in step S1107 includes the following information. The AF-capable area information is area information indicating the AF-capable area. The AF-capable area is a predetermined area in which AF processing can be appropriately performed. AF processing outside the AF-capable area (AF processing in which the position outside the AF-capable area is set as the AF position) may not be performed or may be performed. The AF-capable area may be regarded as a recommended AF area. The AF-capable area may be regarded as an area in which the AF position can be set, or may be regarded as an area including a plurality of recommended AF positions (candidates). The lens information may include at least one of the following: - AF-capable area information; - Center position of the image circle; - Limit position of the image circle; and - Diameter of the image circle.

[0089] Note that the lens information received in step S1107 may include the following information. The magic window area is an area (e.g., the central portion) at a predetermined position defined in advance, and is an area that is (first) cut out for VR display. That is, the magic window area is cut out from the captured image, undergoes perspective projection conversion, and is displayed on the display device, such as the head-mounted display. The lens information may include at least one of the following: - Information specifying a Magic Window area; - Information indicating a manufacturing defect of the dual lens unit; and - Correction data for improving the accuracy of image processing of a live view image (for example, a correction value obtained by calibrating the dual lens unit).

[0090] In step S1108, the CPU 501 determines whether to perform an equal angle conversion display. If it is determined that an equal angle conversion display is to be performed, the process proceeds to step S1109; otherwise (if a circular fisheye display is to be performed), the process proceeds to step S1116. For example, the CPU 501 determines whether to perform an equal angle conversion display based on states (selected state / unselected state) of radio buttons 1205 and 1206 in the Fig. 12A and Fig. 12B. Details of the Fig. 12A and Fig. 12B are described below.

[0091] In step S1109, the CPU 501 determines whether to perform arrangement conversion of a plurality of image regions (a plurality of image areas captured via a plurality of optical systems) in the live view image received (acquired) in step S1107. In the present embodiment, it is assumed that a live view image in which two image areas (a right image area and a left image area) are arranged side by side is received, and conversion (left-right exchange) for exchanging positions of the two image areas is performed as the arrangement conversion. If it is determined to perform left-right exchange, the process proceeds to step S1110; otherwise, the process proceeds to step S1113. For example, the CPU 501 determines whether to perform left-right exchange based on whether a check box 1207 in the Fig. 12A and Fig. 12B is checked.

[0092] In step S1110, the CPU 501 performs left-right interchange and equiangular conversion on the live view image received in step S1107 based on the lens information received (acquired) in step S1107, such as the center position of each image circle. As a result, an image obtained by performing left-right interchange and equiangular conversion on the received live view image is obtained as a display image (an image to be displayed). For example, the left-right interchange is performed such that the right image region is arranged around the center of the left image region before the left-right interchange, and the left image region is arranged around the center of the right image region before the left-right interchange. Then, equiangular conversion is performed on each of the right image region and the left image region.In the present embodiment, it is assumed that each of the right image area and the left image area in the received live view image is a region of the circular fisheye image. Equiangular conversion is a conversion process in which a circular fisheye image is assumed to be a sphere and converted to have latitude lines (horizontal lines) and longitude lines (vertical lines) intersecting each other at right angles, as in an equiangular projection for a map. Through the equiangular conversion, the circular fisheye image is converted into a rectangular equiangular conversion image.

[0093] In step S1111, the CPU 501 displays the display image generated in step S1110 (the live view image after the left-right exchange and the equal angle conversion) on the display unit 505.

[0094] In step S1112, the CPU 501 performs image processing (left-right exchange and equiangular conversion) similar to that in step S1110 on the AF-enabled area information included in the lens information received in step S1107.

[0095] In step S1113, as in step S1110, the CPU 501 performs equiangular conversion on the live view image received in step S1107. Left-right exchange is not performed. As a result, the image obtained by performing equiangular conversion on the received live view image is obtained as the display image.

[0096] In step S1114, the CPU 501 displays the display image generated in step S1113 (the live view image after the equiangular conversion) on the display unit 505.

[0097] In step S1115, the CPU 501 performs image processing (equiangular conversion) similar to that in step S1113 on the AF-enabled area information included in the lens information received in step S1107.

[0098] In step S1116, as in step S1109, the CPU 501 determines whether the arrangement conversion (left-right exchange) is to be performed. If it is determined that the left-right exchange is to be performed, the process proceeds to step S1117; otherwise, the process proceeds to step S1120.

[0099] In step S1117, as in step S1110, the CPU 501 performs left-right exchange on the live view image received in step S1107. Equiangular conversion is not performed. As a result, the image obtained by performing left-right exchange on the received live view image is obtained as the display image.

[0100] In step S1118, the CPU 501 displays the display image generated in step S1117 (the live view image after the left-right exchange) on the display unit 505.

[0101] In step S1119, the CPU 501 performs image processing (left-right exchange) similar to that in step S1118 on the AF-enabled area information included in the lens information received in step S1107.

[0102] In step S1120, the CPU 501 displays the live view image received in step S1107 on the display unit 505 (as a display image).

[0103] The CPU 501 displays an element (a position specification element, for example, a cursor 1210, which is inserted in the Fig. 12A and Fig. 12B) indicating a position in the display image on the display unit 505, along with the display image described above. The operation unit 504 can receive a user operation for changing the position (a position indicated by the element, a display position of the element) of the element.

[0104] In step S1121, the CPU 501 determines whether the position of the position specification item is outside the AF-capable area based on the AF-capable area information. If it is determined that the position of the position specification item is outside the AF-capable area, the process proceeds to step S1122; otherwise, the process proceeds to step S1123. If neither equiangular conversion nor left-right interchange is performed, the AF-capable area information received in step S1107 is used. If left-right interchange is performed, the AF-capable area information after the left-right interchange (after the array conversion) is used. If equiangular conversion is performed, the AF-capable area information after the equiangular conversion is used.When left-right swapping and equal angle conversion are performed, the AF-capable area information after left-right swapping and equal angle conversion is used.

[0105] In step S1122, the CPU 501 changes the form (a display format; a display form) of the position specification item. Through this processing, when the position of the position specification item is within the AF-capable area, the position specification item is displayed in a first form, and when the position of the position specification item is outside the AF-capable area, the position specification item is displayed in a second form.

[0106] In step S1123, the CPU 501 determines whether a user operation (an AF execution operation) for executing AF processing is being executed. The AF execution operation is executed using the operation unit 504. If it is determined that the AF execution operation is being executed, the process proceeds to step S1124; otherwise, the process proceeds to step S1125.

[0107] In step S1124, the CPU 501 transmits an AF instruction including the current coordinates of the position specification item as the coordinates of the AF position (an AF position specified by the user) to the camera connected in step S1101. As a result, the camera is controlled to execute AF processing based on the position of the position specification item.

[0108] In the present embodiment, steps S1123 and S1124 are executed only in a case where the position of the position specification item is within the AF-capable area. Therefore, the AF processing is not executed in a case where the position of the position specification item is outside the AF-capable area, and the AF processing is executed in a case where the position of the position specification item is within the AF-capable area. However, steps S1123 and S1124 may be executed regardless of whether the position of the position specification item is within the AF-capable area. That is, the AF processing may be executed regardless of whether the position of the position specification item is within the AF-capable area. For example, the process may proceed to step S1123 after step S1122.

[0109] In step S1125, the CPU 501 determines whether to terminate the PC live view. If it is determined to terminate the PC live view, the operations are performed according to Fig. 11 is terminated, otherwise the process proceeds to step S1106.

[0110] The Fig. 12A and Fig. 12B shows schematic diagrams illustrating display examples of an application screen displayed on the display unit 505 by the CPU 501 during PC live view when the camera 100 is connected to the PC 500. A screen 1200 is the application screen (remote live view screen). The screen 1200 includes a live view display area 1201, a guide display area 1202, a guide display area 1203, an operation area 1204, and an exit button 1208.

[0111] The live view display area 1201 is an area for displaying a live view image. The live view display area 1201 includes a left-side display area 1201A and a right-side display area 1201B.

[0112] The guide display area 1202 is an area for displaying a character string indicating whether the image displayed in the left-side display area 1201A is an image captured via a corresponding one of the two optical systems 301L and 301R in the dual lens unit. The guide display area 1203 is an area for displaying a character string indicating whether the image displayed in the right-side display area 1201B is an image captured via a corresponding one of the two optical systems 301L and 301R in the dual lens unit.

[0113] The operation area 1204 is an area for receiving an operation for a PC live view. The radio buttons 1205 and 1206 and the checkbox 1207 are displayed in the operation area 1204.

[0114] Radio button 1205 is selected in a case of executing the circular fisheye display, and radio button 1206 is selected in a case of executing the equal angle conversion display. When radio button 1205 is selected, radio button 1206 is not selected. When radio button 1205 is not selected, radio button 1206 is selected.

[0115] In Fig. 12A, radio button 1205 is selected and radio button 1206 is not selected. Consequently, a circular fisheye display is executed (a circular fisheye image is displayed in the display areas 1201A and 1202B). Fig. 12B, radio button 1205 is not selected, and radio button 1206 is selected. Consequently, an equal angle conversion display is executed (an equal angle conversion image is displayed in display areas 1201A and 1202B).

[0116] Checkbox 1207 is a checkbox to be checked when performing left-right swapping. When checkbox 1207 is checked, display positions of a right image area (a right-eye video area; a right-eye video area) in the live view image and a left image area (a left-eye video area; a left-eye video area) can be swapped. Accordingly, the character strings displayed in the guide display areas 1202 and 1203 are also swapped.

[0117] In Fig. 12A, checkbox 1207 is not checked. Consequently, the left-right exchange is not performed, the right image (the right-eye video) captured via the optical system 301R is displayed in the display area 1201A, and the left image (left-eye video) captured via the optical system 301L is displayed in the display area 1201B. Fig. 12B, checkbox 1207 is checked. Consequently, the left-right swap is performed, the right image is displayed in the display area 1201B, and the left image is displayed in the display area 1201B.

[0118] The Exit button 1208 is a button to exit the PC live view.

[0119] A frame 1209 is a frame (element) indicating the AF-capable area. As described above, in a case where left-right interchange is performed on the live view image, the left-right interchange is also performed on the AF-capable area, and in a case where equiangular conversion is performed on the live view image, the equiangular conversion is also performed on the AF-capable area. The frame 1209 in Fig. 12A illustrates an AF-capable area in which neither left-right exchange nor equal angle conversion is performed, with frame 1209 in Fig. 12B illustrates an AF-capable area after the left-right exchange and equal angle conversion have been performed. Note that the frame 1209 may or may not be displayed. The element indicating the AF-capable area is not limited to the frame 1209 and may be, for example, a mask that covers the AF-capable area with a predetermined transparency.

[0120] The cursor 1210 is an element (position specification element) that specifies a position in the display image. Two cursors 1210 are provided in an application screen (in each of the Fig. 12A and Fig. 12B), but with a cursor 1210 displayed on an application screen.

[0121] The user can move the cursor 1210 through a user operation (e.g., moving a mouse) using the operation unit 504. Furthermore, the user can give an instruction to execute the AF processing through a user operation (e.g., clicking a mouse button) using the operation unit 504. When instructing to execute the AF processing, an AF instruction including the current coordinates of the cursor 1210 is transmitted to the camera 100. Then, the camera 100 executes the AF processing based on the coordinates of the cursor 1210.

[0122] As it is in the Fig. 12A and Fig.12B, the shape of the cursor 1210 is changed according to whether the cursor 1210 is within the frame 1290 (within the AF-capable area). As a result, the user can easily grasp whether the cursor 1210 is within the AF-capable area (even when the frame 1209 is not displayed), whether AF processing can be properly performed, and the like. Accordingly, the user can easily determine the inside of the AF-capable area. Changing the shape of the cursor 1210 may include changing a type of the cursor 1210, changing a color of the cursor 1210, changing a size of the cursor 1210, and changing other parameters of the cursor 1210.

[0123] It should be noted that the various types of control described above may be processing performed by a piece of hardware (e.g., a processor or a circuit) or in another way. The processing may be shared among a plurality of pieces of hardware (e.g., a plurality of processors, a plurality of circuits, or a combination of one or more processors and one or more circuits), thereby performing control of the entire device.

[0124] Likewise, the aforementioned processor is a processor in the broad sense, encompassing general-purpose processors and dedicated processors. Examples of general-purpose processors include a central processing unit (CPU), a microprocessing unit (MPU), a digital signal processor (DSP), etc. Examples of dedicated processors include a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), etc. Examples of PLDs include a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), etc.

[0125] The above-described embodiment (including variation examples) is merely an example. Any configurations obtained by appropriately modifying or changing some configurations of the embodiment within the scope of the subject matter of the present disclosure are also included in the present disclosure. The present disclosure also includes other configurations obtained by appropriately combining various features of the embodiment.

[0126] For example, it has been described that an image in which two image areas having parallax are arranged side by side is acquired, but the number of image areas, that is, the number of optical systems, may be larger than two, and the arrangement of the plurality of image areas is not specifically limited.

[0127] The shape of the frame 1209 can also be changed according to whether the cursor 1210 is in the AF-capable area. Whether the shape of the cursor 1210 is to be changed according to whether the cursor 1210 is in the AF-capable area can be set according to an instruction from the user. Similarly, according to an instruction from the user, whether the shape of the frame 1209 is to be changed according to whether the cursor 1210 is in the AF-capable area can be set. Whether the frame 1209 is to be displayed can be set according to an instruction from the user. The shape of the cursor 1210 can be changed without a

[0128] Changing the shape of the frame 1209 can be changed according to whether the cursor 1210 is in the AF-capable area. The shape of the frame 1209 can be changed without changing the shape of the cursor 1210 according to whether the cursor 1210 is in the AF-capable area.

[0129] From a variety of operating elements, such as a mouse, a keyboard, and a touch pad, the cursor 1210 and the frame 1209 can be displayed in a shape corresponding to an operating element used for a user operation (e.g., moving the cursor 1210). This allows the user to easily grasp which operating element is being used.

[0130] An example of considering (using) an area suitable for AF processing (focus adjustment) has been described, but an area suitable for white balance adjustment, magnification display, photometry (exposure adjustment), or the like may be considered. For example, a plurality of different types of areas, such as an AF-enabled area (an area suitable for AF processing) and a WB-enabled area (an area suitable for white balance adjustment), may be considered. The cursor 1210 and the frame 1209 may be displayed in shapes corresponding to the positional relationship between the plurality of areas and the cursor 1210. This allows the user to easily grasp which processing an area in which the cursor 1210 is located is suitable for.The user may be able to give an instruction to perform white balance adjustment, enlarged display, photometry (exposure adjustment), and the like by user operation using the operation unit 504.

[0131] At least some of the processing described as being performed by PC 500 may be performed by camera 100 or another external device (e.g., a cloud server). At least some of the processing described as being performed by camera 100 may be performed by PC 500 or another external device (e.g., a cloud server).

[0132] In addition, the present disclosure is not limited to a camera and a PC, and is applicable to any electronic device that can handle an image having a plurality of image areas corresponding to a plurality of optical systems. For example, the present disclosure is applicable to a PDA, a mobile phone terminal or a portable image viewer, a printer device, a digital photo frame, a music player, a video game device, an electronic book reader, a cloud server, and the like. Likewise, the present disclosure is further applicable to, for example, a video player, a display device (including a projector), a tablet terminal, a smartphone, an AI speaker, a home electrical appliance device, and an in-vehicle device.The disclosure is also applicable to a multi-view smartphone or the like having a plurality of optical systems of different types, such as a standard lens, a wide-angle lens, and a zoom lens.

[0133] According to the present disclosure, a user can easily determine a range in which a specific operation (predetermined processing) can be appropriately performed. Other embodiments

[0134] Embodiments of the present disclosure may also be implemented by a computer of a system or device that retrieves and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (more fully referred to as a "non-transitory computer-readable storage medium") to perform the functions of one or more of the above-described embodiments, and / or that includes one or more circuits (e.g., application-specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiments, and by a method performed by the computer of the system or device, for example, by retrieving and executing the computer-executable instructions from the storage medium,to perform the functions of one or more of the embodiments described above, and / or by controlling the one or more circuits to perform the functions of one or more of the embodiments described above. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)) and may have a network of separate computers or separate processors to retrieve and execute the computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may, for example, be one or more of a hard disk, random access memory (RAM), read-only memory (ROM), distributed computer system memory, an optical disk (such as a compact disc (CD),a Digital Versatile Disk (DVD) or a Blu-Ray Disk (BD)™), a flash memory device, a memory card, and the like.

[0135] While the present disclosure has been described with reference to exemplary embodiments, it should be understood that the disclosure is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation to encompass all such modifications and equivalent structures and functions. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2022-183845 A

[0003] JP 2019-144401 A

[0004]

Claims

[1] Electronic device comprising: an acquisition unit configured to acquire an image having a plurality of image regions each captured by an imaging device via a plurality of optical systems; a display control unit configured to perform control such that a display image based on the acquired image is displayed and to perform control such that an element indicating a position in the display image is displayed; and a receiving unit configured to receive a user operation to change a position of the element, wherein the display control unit executes control such that the element is displayed in a first form in a case where the position of the element corresponds to a position in a predetermined area of the image, and executes control such that the element is displayed in a second form in a case where the position of the element corresponds to a position outside the predetermined range. [2] The electronic device of claim 1, further comprising: an information acquisition unit configured to acquire area information indicating the predetermined area from the imaging device, wherein the display control unit determines whether the position of the element corresponds to a position in the predetermined range based on the range information. [3] Electronic device according to claim 2, wherein the display image is an image obtained by performing equiangular conversion on the image, and the display control unit performs the equiangular conversion on the range information and determines whether the position of the element corresponds to a position in the predetermined range based on the range information after the equiangular conversion. [4] Electronic device according to claim 2 or 3, wherein the display image is an image obtained by performing arrangement conversion of the plurality of image areas, and the display control unit performs the arrangement conversion on the area information and determines whether the position of the element corresponds to a position in the predetermined area based on the area information after the arrangement conversion. [5] Electronic device according to claim 4, wherein the image is an image in which two image areas are arranged side by side, and the arrangement conversion is a conversion for exchanging positions of the two image areas. [6] Electronic device according to one of claims 1 to 5, further comprising: a control unit configured to perform predetermined processing based on the position of the element, wherein the control unit executes a control, in a case where the position of the element corresponds to a position outside the predetermined range, not to execute the predetermined processing, and in a case where the position of the element corresponds to a position in the predetermined range, to execute the predetermined processing. [7] The electronic device according to any one of claims 1 to 6, wherein the display control unit further executes control to display a second element indicating an area corresponding to the predetermined area in the display image. [8] Electronic device according to one of claims 1 to 7, further comprising: a setting unit configured to set, according to an instruction from the user, whether to change a shape of the element according to whether the position of the element corresponds to a position in the predetermined range. [9] The electronic device according to any one of claims 1 to 8, wherein the display control unit performs control such that the item is displayed in a form corresponding to an operation element used for user operation among a plurality of operation elements. [10] Electronic device according to one of claims 1 to 9, wherein a plurality of areas of different types are used as the predetermined area, and the display control unit performs control such that the element is displayed in a form corresponding to a positional relationship between the plurality of regions and the element. [11] The electronic device according to any one of claims 1 to 10, wherein the acquiring unit acquires a live view image as the image. [12] The electronic device according to any one of claims 1 to 11, wherein the predetermined range corresponds to a range suitable for white balance adjustment, magnification display, exposure adjustment and focus adjustment in the imaging device. [13] The electronic device according to any one of claims 1 to 12, wherein the image is an image in which two optical images are formed in two regions of an imaging element, respectively. [14] A control method of an electronic device, comprising: an acquiring step of acquiring an image having a plurality of image regions each captured by an imaging device through a plurality of optical systems; a display control step for executing control such that a display image based on the acquired image is displayed, and for executing control such that an element indicating a position in the display image is displayed; and a receiving step for receiving a user operation to change a position of the element, wherein in the display control step a control is carried out such that the element is displayed in a first form in a case where the position of the element corresponds to a position in a predetermined area of the image, and control is performed such that the element is displayed in a second form in a case where the position of the element corresponds to a position outside the predetermined range. [15] A computer program comprising instructions which, when executed by one or more processors, are configured to carry out the control method according to claim 14. [16] A program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 13. [17] A computer-readable medium storing a program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 13.

Images