Electronic device

Abstract

To provide a technique capable of making it difficult for a user to lose sight of a part that the user paid attention to (noticed) in the past.SOLUTION: An electronic device of an embodiment of the present invention is provided with: acquisition means for acquiring information related to a visual line position of a user; and control means for controlling a screen to be displayed and an item corresponding to a focused position of the user on the screen to be identifiably displayed based on the information related to the visual line position in which, when a layout of the screen is switched, the control means controls the item that was identifiably displayed before the switching to be identifiably displayed on the screen after the switching, regardless of the focused position of the user displayed on the screen after the switching.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to electronic devices, and more particularly to screen displays. [Background technology]

[0002] Patent Document 1 discloses a technology for detecting the user's line of sight on the screen of an electronic book and highlighting and displaying the part of the electronic book that the user pays attention to. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-102360 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technology disclosed in Patent Document 1, when the user looks away from (avoids) the focused portion, the emphasis is released and the focused portion may be lost.

[0005] The present invention provides a technique that makes it more difficult for a user to lose sight of an area that the user previously focused on (attention to). [Means for solving the problem]

[0006] The electronic device of the present invention has an acquisition means for acquiring information regarding the user's gaze position, and a control means for controlling the display of a screen and, based on the information regarding the gaze position, controlling the display of items corresponding to the user's focus position on the screen in an identifiable manner, and is characterized in that when the layout of the screen is switched, the control means controls the display of items that were identifiable before the switching to be identifiable on the screen after the switching, regardless of the user's focus position on the screen after the switching. [Effects of the Invention]

[0007] According to the present invention, it is possible to make it more difficult for the user to lose sight of the part that he or she previously focused on (attention to). [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an external view of a digital camera. [Figure 2] FIG. 1 is a block diagram of a digital camera. [Figure 3] FIG. 2 is a schematic diagram of a display screen. [Figure 4] 10 is a flowchart of a photographing mode process according to the first embodiment. [Figure 5] 10 is a flowchart of a photographing mode process according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, it is made more difficult than ever for a user to lose sight of an area that the user previously focused on (attention to).

[0010] Due to the popularity of vertical videos on SNS (Social Networking Service) that are adapted to smartphone screens, the number of situations where the camera is taken in a vertical position is increasing, and the number of situations where the camera is frequently switched between vertical and horizontal positions is also increasing. Therefore, it is important to create a comfortable UX (User Experience) in both vertical and horizontal camera positions. It is important to provide

[0011] However, when the screen layout changes, such as when the displayed screen switches from portrait to landscape (or from landscape to portrait), the user may lose track of (or be unable to immediately find) the item that was on their attention before the switch. In this embodiment, in such a case, it is made difficult for the user to lose track of the item that they were focusing on before the switch.

[0012] 1A and 1B are external views of a digital camera 100 (image capture device) as an example of a device (electronic device) to which the present invention can be applied. Fig. 1A is a front perspective view of the digital camera 100, and Fig. 1B is a rear perspective view of the digital camera 100.

[0013] The display unit 28 is a display unit provided on the back surface of the digital camera 100, and displays images and various information. The touch panel 70a can detect touch operations on the display surface (touch operation surface) of the display unit 28. The out-of-viewfinder display unit 43 is a display unit provided on the top surface of the digital camera 100, and displays various settings of the digital camera 100, including the shutter speed and aperture. The shutter button 61 is an operation member for issuing shooting instructions. The mode selector switch 60 is an operation member for switching between various modes. The terminal cover 40 is a cover that protects a connector (not shown) for connecting a connection cable or the like that connects the digital camera 100 to an external device.

[0014] The main electronic dial 71 is a rotary operation member, and by turning the main electronic dial 71, settings such as shutter speed and aperture can be changed. The power switch 72 is an operation member that switches the power of the digital camera 100 on and off. The sub electronic dial 73 is a rotary operation member, and by turning the sub electronic dial 73, the selection frame (cursor) can be moved, images can be forwarded, etc. The four-way key 74 is configured so that the up, down, left, and right parts can each be pressed, and processing can be performed according to the part of the four-way key 74 that is pressed. The SET button 75 is a push button, and is mainly used to confirm a selection item, etc.

[0015] The movie button 76 is used to start or stop movie shooting (recording). The AE lock button 77 is a push button, and pressing the AE lock button 77 in shooting standby mode fixes the exposure state. The enlarge button 78 is an operation button for switching the enlargement mode on and off in the live view display (LV display) of the shooting mode. By turning the enlargement mode on and operating the main electronic dial 71, the live view image (LV image) can be enlarged or reduced. In playback mode, the enlargement button 78 functions as an operation button for enlarging the playback image or increasing its magnification. The playback button 79 is an operation button for switching between shooting mode and playback mode. Pressing the playback button 79 in shooting mode switches to playback mode, and the most recent image recorded on the recording medium 200 (described below) can be displayed on the display unit 28. The menu button 81 is a push button used to issue an instruction to display a menu screen. Pressing the menu button 81 displays a menu screen on the display unit 28 that allows various settings to be made. The user can intuitively make various settings using the menu screen displayed on the display unit 28, the four-way key 74, and the SET button 75.

[0016] The touch bar 82 is a line-shaped touch operation member (line touch sensor) that can receive touch operations. The touch bar 82 is located in a position that allows touch operation (touchable) with the thumb of the right hand when the grip portion 90 is held in the right hand (held with the little finger, ring finger, and middle finger of the right hand) so that the shutter button 61 can be pressed with the index finger of the right hand. In other words, the touch bar 82 is located in a position that allows operation when the user places his / her eye on the eyepiece viewfinder 17, looks through the eyepiece portion 16, and is in a position that allows operation when the user is ready to press the shutter button 61 at any time (shooting posture). The touch bar 82 can accept tap operations on the touch bar 82 (operation of touching and then releasing the touched position within a predetermined period without moving it), left and right slide operations (operation of touching and then moving the touched position while still touching), etc. The touch bar 82 is an operating member different from the touch panel 70a, and does not have a display function. The touch bar 82 functions as a multi-function bar (M-Fn bar) to which various functions can be assigned, for example.

[0017] The communication terminal 10 is a communication terminal through which the digital camera 100 communicates with the lens unit 150 (described later; detachable). The eyepiece 16 is the eyepiece of the eyepiece finder 17 (a peer-type finder), and the user can view an image displayed on the internal EVF 29 (described later; electronic viewfinder) through the eyepiece 16. The eyepiece detection unit 57 is an eyepiece detection sensor that detects whether the user (photographer) has placed their eye on the eyepiece 16. The cover 202 is a cover for a slot that stores a recording medium 200 (described later). The grip unit 90 is a holding unit shaped to be easily held with the user's right hand when holding the digital camera 100. The shutter button 61 and main electronic dial 71 are positioned so that they can be operated with the index finger of the right hand when the digital camera 100 is held by gripping the grip unit 90 with the little finger, ring finger, and middle finger of the right hand. In the same state, the sub electronic dial 73 and touch bar 82 are also located at positions that can be operated with the thumb of the right hand. The thumb rest 91 (thumb standby position) is a grip section provided on the rear side of the digital camera 100, in a position where it is easy to place the thumb of the right hand that is holding the grip section 90 when none of the operation members are being operated. The thumb rest 91 is made of a rubber member or the like to increase the holding strength (grip feeling).

[0018] FIG. 2 is a block diagram showing an example configuration of a digital camera 100. The lens unit 150 is a lens unit equipped with an interchangeable photographic lens. The lens 103 is usually composed of multiple lenses, but FIG. 2 shows only one lens for simplicity's sake. The communication terminal 6 is a communication terminal through which the lens unit 150 communicates with the digital camera 100, and the communication terminal 10 is a communication terminal through which the digital camera 100 communicates with the lens unit 150. The lens unit 150 communicates with the system controller 50 via these communication terminals 6 and 10. The lens unit 150 controls the aperture 1 via the aperture drive circuit 2 by the internal lens system control circuit 4. The lens unit 150 also adjusts the focus by displacing the position of the lens 103 via the AF drive circuit 3 by the lens system control circuit 4.

[0019] The shutter 101 is a focal plane shutter that can freely control the exposure time of the imaging unit 22 under the control of the system control unit 50.

[0020] The imaging unit 22 is an imaging element (image sensor) configured with a CCD, CMOS element, or the like that converts an optical image into an electrical signal. The imaging unit 22 may have an imaging surface phase difference sensor that outputs defocus amount information to the system control unit 50. The A / D converter 23 converts the analog signal output from the imaging unit 22 into a digital signal.

[0021] The image processing unit 24 performs predetermined processing (pixel interpolation, resizing such as reduction, color conversion, etc.) on data from the A / D converter 23 or data from the memory control unit 15. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the arithmetic results obtained by the image processing unit 24. This allows for TTL (through-the-lens) type AF (autofocus) processing, AE (autoexposure) processing, EF (flash pre-flash) processing, etc. to be performed. The image processing unit 24 also performs predetermined arithmetic processing using the captured image data, and performs TTL type AWB (auto white balance) processing based on the arithmetic results obtained.

[0022] The memory control unit 15 controls the transmission and reception of data between the A / D converter 23, the image processing unit 24, and the memory 32. The output data from the A / D converter 23 is written to the memory 32 via the image processing unit 24 and the memory control unit 15. Alternatively, the output data from the A / D converter 23 is written to the memory 32 via the memory control unit 15 without going through the image processing unit 24. The memory 32 stores image data obtained by the imaging unit 22 and converted into digital data by the A / D converter 23, as well as image data to be displayed on the display unit 28 and the EVF 29. The memory 32 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio.

[0023] The memory 32 also serves as a memory (video memory) for image display. The D / A converter 19 converts the display image data stored in the memory 32 into an analog signal and supplies it to the display unit 28 or the EVF 29. In this way, the display image data written to the memory 32 is displayed on the display unit 28 or the EVF 29 via the D / A converter 19. The display unit 28 and the EVF 29 are each a display such as an LCD or an organic EL, and perform display according to the analog signal from the D / A converter 19. The digital signal that has been A / D converted by the A / D converter 23 and stored in the memory 32 is converted into an analog signal by the D / A converter 19, and the analog signal is sequentially transferred to and displayed on the display unit 28 or the EVF 29, thereby performing a live view display (LV). Hereinafter, an image displayed in live view display will be referred to as a live view image (LV image).

[0024] The system control unit 50 is a control unit made up of at least one processor and / or at least one circuit, and controls the entire digital camera 100. The system control unit 50 is both a processor and a circuit. The system control unit 50 executes programs recorded in nonvolatile memory 56 to realize each process of this embodiment, which will be described later. The system control unit 50 also performs display control by controlling the memory 32, D / A converter 19, display unit 28, EVF 29, etc.

[0025] The system memory 52 is, for example, a RAM, and the system control unit 50 loads constants and variables for the operation of the system control unit 50, programs read from the nonvolatile memory 56, and the like into the system memory 52.

[0026] The nonvolatile memory 56 is an electrically erasable and recordable memory, such as an EEPROM. Constants, programs, etc. for the operation of the system control unit 50 are recorded in the nonvolatile memory 56. The programs referred to here are programs for executing various flowcharts described later in this embodiment.

[0027] The system timer 53 is a timekeeping unit that measures the time used for various controls and the time of a built-in clock.

[0028] The communication unit 54 transmits and receives video signals and audio signals to and from external devices connected wirelessly or via a wired cable. The communication unit 54 can also be connected to a wireless LAN (Local Area Network) or the Internet. The communication unit 54 can also communicate with external devices via Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 54 can transmit images (including LV images) captured by the imaging unit 22 and images recorded on the recording medium 200, and can receive image data and various other information from external devices.

[0029] The orientation detection unit 55 detects the orientation of the digital camera 100 relative to the direction of gravity. Based on the orientation detected by the orientation detection unit 55, it is determined whether the image captured by the imaging unit 22 was captured with the digital camera 100 held horizontally or vertically. It is possible. The system control unit 50 can add orientation information corresponding to the attitude detected by the attitude detection unit 55 to the image file of the image captured by the imaging unit 22, or rotate and record the image. An acceleration sensor, a gyro sensor, or the like can be used as the attitude detection unit 55. It is also possible to detect the movement of the digital camera 100 (panning, tilting, lifting, whether it is stationary, etc.) using the acceleration sensor or gyro sensor that is the attitude detection unit 55.

[0030] The eyepiece detection unit 57 is an eyepiece detection sensor that detects (approach detection) whether the eye (object) 161 approaches (approach) or moves away (away) from (moves away) the eyepiece 16 of the eyepiece viewfinder 17 (hereinafter simply referred to as the "viewfinder"). The system control unit 50 switches the display unit 28 and the EVF 29 between on (display state) and off (non-display state) depending on the state detected by the eyepiece detection unit 57. More specifically, at least in a shooting standby state and when the display destination switching setting is automatic switching, when the eye is not in contact with the camera, the display is turned on with the display on the display unit 28 and the EVF 29 is hidden. When the eye is in contact with the camera, the display is turned on with the display on the EVF 29 and the display unit 28 is hidden. For example, an infrared proximity sensor can be used as the eyepiece detection unit 57, and it can detect the approach of an object to the eyepiece 16 of the viewfinder 17 that incorporates the EVF 29. When an object approaches, infrared light emitted from a light-emitting unit (not shown) of the eyepiece detection unit 57 is reflected by the object and received by a light-receiving unit (not shown) of the infrared proximity sensor. The amount of received infrared light can also determine the distance the object is approaching the eyepiece 16 (eyepiece distance). In this way, the eyepiece detection unit 57 performs eyepiece detection, which detects the proximity of an object to the eyepiece 16. When an object approaching within a predetermined distance from the eyepiece 16 is detected from a non-eyepiece state (non-approach state), it is detected as being in eye contact. When an object detected as approaching moves away from the eyepiece state (approach state) by more than a predetermined distance, it is detected as being away from the eye. The threshold for detecting eye contact and the threshold for detecting eye separation may be different, for example, by providing hysteresis. Furthermore, after eye contact is detected, the eyepiece remains in the eye contact state until eye separation is detected. After eye separation is detected, the eyepiece remains in the non-eye contact state until eye contact is detected. The infrared proximity sensor is just an example, and other sensors may be used for the eye proximity detector 57 as long as they can detect a state that can be considered as eye proximity.

[0031] Various camera settings such as shutter speed and aperture are displayed on the outside viewfinder display 43 via an outside viewfinder display drive circuit 44 .

[0032] The power supply control unit 80 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between powered blocks, etc., and detects whether a battery is installed, the type of battery, and the remaining battery power. The power supply control unit 80 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, and supplies the required voltage for the required period to each unit, including the recording medium 200. The power supply unit 30 is composed of primary batteries such as alkaline batteries or lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, or lithium-ion batteries, an AC adapter, etc.

[0033] The recording medium I / F 18 is an interface with a recording medium 200 such as a memory card or a hard disk. The recording medium 200 is a recording medium such as a memory card for recording captured images, and is composed of a semiconductor memory, a magnetic disk, or the like.

[0034] The operation unit 70 is an input unit that accepts operations from the user (user operations) and is used to input various operational instructions to the system control unit 50. As shown in Fig. 2, the operation unit 70 includes a shutter button 61, a mode selector switch 60, a power switch 72, a touch panel 70a, and other operation members 70b. The other operation members 70b include a main electronic dial 71, a sub electronic dial 73, a four-way key 74, a SET button 75, a video button 76, an AE lock button 77, a magnification button 78, a playback button 79, a menu button 80, and a touch panel 70a. 1, Touch Bar 82, etc.

[0035] The shutter button 61 includes a first shutter switch 62 and a second shutter switch 64. The first shutter switch 62 is turned on when the shutter button 61 is pressed halfway (a shooting preparation command) and generates a first shutter switch signal SW1. The system control unit 50 starts shooting preparation operations such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (pre-flash) processing in response to the first shutter switch signal SW1. The second shutter switch 64 is turned on when the shutter button 61 is pressed fully (a shooting command) and generates a second shutter switch signal SW2. The system control unit 50 starts a series of shooting processing operations in response to the second shutter switch signal SW2, from reading out a signal from the imaging unit 22 to writing the captured image to the recording medium 200 as an image file.

[0036] The mode selector switch 60 switches the operating mode of the system control unit 50 to one of still image capture mode, video capture mode, playback mode, etc. Modes included in the still image capture mode include auto capture mode, auto scene determination mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). There are also various scene modes and custom modes that provide capture settings for specific capture scenes. The mode selector switch 60 allows the user to directly switch to one of these modes. Alternatively, after first switching to a list screen of capture modes with the mode selector switch 60, the user may selectively switch to one of the displayed modes using another operating member. Similarly, the video capture mode may also include multiple modes.

[0037] The touch panel 70a is a touch sensor that detects various touch operations on the display surface of the display unit 28 (the operation surface of the touch panel 70a). The touch panel 70a and the display unit 28 can be configured as an integrated unit. For example, the touch panel 70a is configured so that its light transmittance does not interfere with the display of the display unit 28, and is attached to the upper layer of the display surface of the display unit 28. Input coordinates on the touch panel 70a are associated with display coordinates on the display surface of the display unit 28. This makes it possible to provide a GUI (Graphical User Interface) that allows the user to directly operate the screen displayed on the display unit 28.

[0038] The gaze detection block 160 (eye tracking unit, eye tracker) is a block for detecting the gaze of a user who has placed their eye in the eyepiece unit 16, whether they are looking at the EVF 29, and if so, where they are looking. The gaze detection block 160 includes a dichroic mirror 162, an imaging lens 163, a gaze detection sensor 164, an infrared light emitting diode 166, and a gaze detection circuit 165.

[0039] The infrared light emitting diode 166 is a light emitting element that emits infrared light onto the user's eyeball (eye) 161 placed in the eyepiece 16. The infrared light emitted from the infrared light emitting diode 166 is reflected by the eyeball (eye) 161, and the reflected infrared light reaches the dichroic mirror 162. The dichroic mirror 162 reflects only the infrared light and transmits visible light. The reflected infrared light, whose optical path has been changed, forms an image on the imaging surface of the line of sight detection sensor 164 via the imaging lens 163. The imaging lens 163 is an optical component that constitutes the line of sight detection optical system. The line of sight detection sensor 164 is composed of an imaging device such as a CCD image sensor. The line of sight detection sensor 164 photoelectrically converts the incident reflected infrared light into an electrical signal and outputs it to the line of sight detection circuit 165. The gaze detection circuit 165 includes at least one processor, and detects the gaze position of the user from the image or movement of the user's eyeball (eye) 161 based on the output signal of the gaze detection sensor 164, and outputs the detected information to the system control unit 50.

[0040] In this embodiment, the gaze detection block 160 is used to detect the gaze using a method called the corneal reflex method. The corneal reflex method detects the gaze direction and position based on the positional relationship between the pupil of the eyeball (eye) 161 and the light reflected from the eyeball (eye) 161 (particularly the cornea) when infrared light emitted from the infrared light-emitting diode 166 is reflected. There are various other methods for detecting the gaze direction and position, such as the scleral reflex method, which utilizes the difference in light reflectance between the black and white of the eye. Note that other gaze detection methods may be used as long as they can detect the gaze direction and position. In this embodiment, the light-emitting unit and light-receiving unit of the eyepiece detection unit 57 are described as being separate devices from the infrared light-emitting diode 166 and the gaze detection sensor 164. However, this is not a limitation. The infrared light-emitting diode 166 may also serve as the light-emitting unit of the eyepiece detection unit 57, and the gaze detection sensor 164 may also serve as the light-receiving unit.

[0041] The system control unit 50 can detect the following operations or states based on the output from the line-of-sight detection block 160. The gaze of the user who has placed his / her eye on the eyepiece unit 16 is newly input (detected). In other words, gaze input has started. A state in which there is gaze input from a user who has placed an eyepiece 16 in front of the eye. A state in which a user is gazing at the eyepiece 16. The user who has placed his / her eye on the eyepiece unit 16 moves his / her gaze away from the eyepiece unit 16. In other words, the end of the gaze input. A state in which the user has placed their eye on the eyepiece 16 but is not inputting any gaze.

[0042] The term "gazing" as used herein means that the user continues to look at approximately the same position for a certain period of time. In determining whether or not a user is gazing, for example, a determination is made that the user is gazing if the user's gaze position does not exceed a predetermined amount of movement for a predetermined period of time (e.g., approximately 0.5 seconds). The predetermined period of time may be user-settable, a fixed period of time, or a period that varies depending on the distance between the previous and current gaze positions. For example, the system control unit 50 determines that the user is gazing if the duration of a state in which the user's gaze is detected at approximately the same position (no gaze movement state) exceeds a predetermined period of time (threshold period) based on detection information acquired from the gaze detection circuit 165. Furthermore, the system control unit 50 determines that the user is gazing if, for example, the average position of the gaze detection position over a short period of time (≦the aforementioned threshold period) including the most recent detection timing falls within a predetermined range and the variance (dispersion) is less than a predetermined value.

[0043] 3(A) to 3(C) are schematic diagrams showing examples of screens displayed on the EVF 29. Fig. 3(A) shows a screen when the digital camera 100 is in a horizontal position (when the user holds the digital camera 100 horizontally). Figs. 3(B) and 3(C) show a screen when the digital camera 100 is in a vertical position (when the user holds the digital camera 100 vertically).

[0044] Screen 301A (landscape screen) in Fig. 3(A) includes icons 302 and 303. Similarly, screens 301B and 301C (portrait screen) in Fig. 3(B) and 3(C) also include icons 302 and 303. The screen layout is different between landscape screen 301A and portrait screens 301B and 301C, and the display positions of icons 302 and 303 are different. Therefore, when switching from landscape screen to portrait screen (or from portrait screen to landscape screen), the user may lose track of (or be unable to immediately find) an item that they were focusing on before the switch.

[0045] Therefore, in this embodiment, the system control unit 50 controls the display so that an item corresponding to the user's focused position on the screen is identifiable based on the gaze information (information about the gaze position, information detected by the gaze detection circuit 165). Then, the screen layout is switched. In this case, the system control unit 50 performs control so that the items that were identifiably displayed before the switching are identifiably displayed on the screen after the switching, regardless of the position of the user's attention on the screen after the switching.

[0046] By displaying the items that were identifiably displayed before the switch in an identifiably manner on the screen after the switch, it is possible to make it difficult for the user to lose sight of the items that they were focusing on before the switch.

[0047] The system control unit 50 may control the screen after switching so that the first item that was identifiably displayed before the switching is identifiably displayed, but the second item corresponding to the user's new focus position is not identifiably displayed. The first item and the second item are different.

[0048] For example, when the user holds the digital camera 100 in the landscape orientation and focuses on icon 302 in landscape screen 301A, highlighting 304 of icon 302 begins under the control of the system control unit 50. Then, when the user changes the orientation of the digital camera 100 from the landscape orientation to the portrait orientation, the system control unit 50 controls the screen displayed on the EVF 29 to transition from landscape screen 301A to portrait screen 301B. In portrait screen 301B, highlighting 304 of icon 302 continues even if the user looks away from icon 302. At this time, even if the user directs their gaze toward icon 303, highlighting of icon 303 is not performed.

[0049] By not identifiably displaying the item at the new focus position, the screen becomes simpler (the clutter on the screen is reduced), and it becomes easier for the user to find the item that was on the user's focus before switching.

[0050] The system control unit 50 may perform control so that, on the screen after switching, a first item that was identifiably displayed before switching is identifiably displayed in a first manner, and a second item corresponding to the user's new focus position is identifiably displayed in a second manner. The first item and the second item are different, and the first manner and the second manner are also different.

[0051] For example, when the user changes the orientation of the digital camera 100 from landscape to portrait, the screen displayed on the EVF 29 transitions from landscape screen 301A to portrait screen 301C under the control of the system control unit 50. In portrait screen 301C, the highlighting of the icon 302 that was focused on before the change is changed from highlighting 304 to highlighting 305, and highlighting 304 is displayed as the newly focused icon 302. Note that in portrait screen, highlighting 304 of icon 302 may continue, and highlighting 305 of icon 303 may begin.

[0052] By identifiably displaying the item that was identifiably displayed before switching and the item at the new focus position in different modes, the user can easily find the item that was focused on before switching and can easily recognize the item that is currently being focused on.

[0053] The control for identifiably displaying items includes, for example, at least one of the following controls: -Control to enlarge items Controls that change the color of an item (for example, adding a certain color tint, inverting the color, or changing the color intensity) - Control blinking of items -Control to change the speed of item display changes (changes (movement, blinking, size change, color change) ) Change in the speed of change in icons (animated icons)

[0054] To reduce screen clutter, items may be hidden when the user is not looking and displayed when the user is looking, so controls for making items identifiable may include controls for displaying hidden items.

[0055] The system control unit 50 may determine whether to display an item that was identifiably displayed before switching in an identifiably manner on the switched screen, taking into account the difficulty of finding the item based on the size or display position of the item. For example, if the size of an item that was identifiably displayed before switching is smaller than a threshold on the switched screen, the system control unit 50 may determine not to display the item identifiably on the switched screen. If the position on the switched screen corresponding to an item that was identifiably displayed before switching is within a predetermined area (e.g., an edge of the screen), the system control unit 50 may determine not to display the item identifiably on the switched screen.

[0056] The above-mentioned position of interest may be a gaze position (instantaneous value) or a position of interest (statistical value). For example, the position of interest may be a position where the gaze position is maintained for a predetermined time.

[0057] Example 1 A first embodiment of the present invention will be described. FIG. 4 is a flowchart showing a shooting mode process according to the first embodiment. The shooting mode process of FIG. 4 is realized by the system control unit 50 loading a program stored in the nonvolatile memory 56 into the system memory 52 and executing the program. For example, when the shooting mode is set, the system control unit 50 starts the shooting mode process of FIG. 4. Note that FIG. 4 omits several operations such as shooting preparation operations and shooting processing operations. Operations similar to those of FIG. 4 may be performed in various situations, not just in the shooting mode.

[0058] In S401, the system control unit 50 determines whether or not the camera position has been switched (from landscape to portrait position, or from portrait to landscape position) based on the output signal of the orientation detection unit 55. If it is determined that the camera position has been switched, the process proceeds to S411; otherwise, the process proceeds to S402.

[0059] In S402, the system control unit 50 determines whether or not the user's gaze position has been detected based on the gaze information (information related to the gaze position, information detected by the gaze detection circuit 165). If it is determined that the gaze position has been detected, the system control unit 50 proceeds to S403, and if not, the system control unit 50 proceeds to S407.

[0060] In S403, the system control unit 50 determines the user's focus position based on the gaze information, and detects an icon corresponding to the focus position (an icon displayed at the focus position or an icon to be displayed at the focus position). Hereinafter, the icon corresponding to the focus position will be referred to as a focus icon.

[0061] In S404, the system control unit 50 starts highlighting the icon of interest detected in S403. If there is an icon that is already being highlighted, the highlighting of that icon is stopped and highlighting of a new icon of interest is started.

[0062] In S405, the system control unit 50 stores information about the icon of interest (icon of interest information) that has started to be highlighted in S404 in the system memory 52. ​​If the icon of interest information has already been stored in the system memory 52, the icon of interest information stored in the system memory 52 is updated.

[0063] In S406, system controller 50 starts measuring time T1 using system timer 53 in order to continue highlighting the icon for a predetermined time T1.

[0064] If it is determined in S402 that the gaze position has been detected, the system control unit 50 determines in S407 whether or not the time T1 has elapsed since the highlighting in S404. If it is determined that the time T1 has elapsed, the process proceeds to S408, and if not, the process proceeds to S410.

[0065] In S408, the system control unit 50 ends the highlighting of the icon. In S409, the system control unit 50 deletes the icon information of the target icon from the system memory 52.

[0066] In S410, system controller 50 continues counting time T1.

[0067] If it is determined in S401 that the camera position has been switched, in S411 the system control unit 50 switches the screen orientation (screen layout) (from portrait to landscape, or from landscape to portrait).

[0068] In S412, the system control unit 50 reads out the icon information of interest from the system memory 52.

[0069] In S413, the system control unit 50 starts highlighting the icon indicated by the icon information of interest read in S412 (the icon that was highlighted before the switching in S411). At this time, the system control unit 50 starts measuring time T2 using the system timer 53 so that the icon will continue to be highlighted for a predetermined time T2. Time T2 may be equal to time T1, or may be longer or shorter than time T1.

[0070] In S414, the system control unit 50 determines whether or not time T2 has elapsed since the highlighting in S413. After waiting for time T2 to elapse, if it determines that time T2 has elapsed, the process proceeds to S415. Note that if the camera position is switched while measuring time T2, the process proceeds to S411.

[0071] In S415, the system controller 50 ends the highlighting of the icon.

[0072] In S416, the system control unit 50 determines whether an instruction to end the shooting mode process has been issued. If it is determined that an instruction to end the shooting mode process has been issued, the system control unit 50 terminates the shooting mode process; if not, the system control unit 50 proceeds to S401. For example, if the mode selector switch 60 has been operated, that is, if an instruction to switch to another operating mode has been issued, the system control unit 50 determines that an instruction to end the shooting mode process has been issued. Also, if the power switch 72 has been operated, that is, if an instruction to power off the digital camera 100 has been issued, the system control unit 50 determines that an instruction to end the shooting mode process has been issued.

[0073] <Example 2> A second embodiment of the present invention will be described. FIG. 5 is a flowchart showing a shooting mode process according to the second embodiment. The shooting mode process of FIG. 5 is realized by the system control unit 50 loading a program stored in the nonvolatile memory 56 into the system memory 52 and executing it. For example, when the shooting mode is set, the system control unit 50 starts the shooting mode process of FIG. 5. Note that FIG. 5 omits several operations such as shooting preparation operations and shooting processing operations. Operations similar to those of FIG. 5 may be performed in various situations, not just in the shooting mode.

[0074] In S501, the system control unit 50 starts displaying a shooting standby screen.

[0075] In S502, the system control unit 50 sets the highlight fixation flag, which is ON when highlighting continues from before the layout change and OFF in other cases, to OFF after the layout change.

[0076] In S503, the system control unit 50 sets the re-eyepiece flag, which is ON while waiting for the user to put the eyepiece on the camera and OFF in other cases, to OFF.

[0077] In S504, the system control unit 50 determines whether or not the camera position has been switched (from landscape to portrait position, or from portrait to landscape position) based on the output signal of the orientation detection unit 55. If it is determined that the camera position has been switched, the process proceeds to S527; otherwise, the process proceeds to S505.

[0078] In S505, the system control unit 50 determines whether or not the user has placed their eye on the eyepiece unit 16 based on the output signal of the eyepiece detection unit 57. If it is determined that the user has placed their eye on the eyepiece unit 16, the system control unit 50 proceeds to S506; otherwise, the system control unit 50 proceeds to S524.

[0079] In S506, the system control unit 50 determines whether the highlight fixation flag is OFF. If it is determined to be OFF, the process proceeds to S507, and if not, the process proceeds to S521.

[0080] In S507, the system control unit 50 determines whether the re-eyepiece flag is OFF. If it is determined to be OFF, the process proceeds to S508, and if not, the process proceeds to S517.

[0081] In S508, the system control unit 50 determines whether or not the user's gaze position has been detected based on the gaze information (information related to the gaze position, information detected by the gaze detection circuit 165). If it is determined that the gaze position has been detected, the system control unit 50 proceeds to S509, and if not, the system control unit 50 proceeds to S514.

[0082] In S509, the system control unit 50 determines the user's focus position based on the line-of-sight information and determines whether or not there is a focus icon. If it is determined that there is a focus icon, the system control unit 50 proceeds to S510; if not, the system control unit 50 proceeds to S512.

[0083] In S510, the system control unit 50 starts highlighting the icon of interest detected in S509. In S511, the system control unit 50 stores information about the icon of interest (icon of interest information) that started to be highlighted in S510 in the system memory 52.

[0084] In S512, the system control unit 50 ends the highlighting of the icon. In S513, the system control unit 50 deletes the icon information of the target icon from the system memory 52.

[0085] If it is determined in S508 that the gaze position has been detected, then in S514 the system control unit 50 determines whether a predetermined time T3 has elapsed since the start of the currently performed highlighting. Time T3 corresponds to times T1 and T2 in the first embodiment. If it is determined that time T3 has elapsed, the process proceeds to S515; otherwise, the process proceeds to S538.

[0086] In S515, the system control unit 50 ends the highlighting of the icon. In S516, the system control unit 50 deletes the icon information of the target icon from the system memory 52.

[0087] If it is determined in S507 that the re-eyepiece flag is ON, the system control The unit 50 determines whether or not the time T3 has elapsed since the start of the current highlighting. If it is determined that the time T3 has elapsed, the process proceeds to S518, and if not, the process proceeds to S520.

[0088] In S518, the system control unit 50 ends the highlighting of the icon. In S519, the system control unit 50 sets the re-eyepiece flag to OFF.

[0089] In S520, the system control unit 50 reads the icon information of interest from the system memory 52 and starts highlighting the icon indicated by the icon information of interest. If the icon indicated by the icon information of interest is already being highlighted, the system control unit 50 continues highlighting the icon.

[0090] If it is determined in S506 that the highlight fixation flag is ON, the system control unit 50 determines in S521 whether or not the time T3 has elapsed since the start of the current highlighting. If it is determined that the time T3 has elapsed, the process proceeds to S522; otherwise, the process proceeds to S538.

[0091] In S522, the system control unit 50 ends the highlighting of the icon. In S523, the system control unit 50 sets the highlight fixation flag to OFF.

[0092] If it is determined in S505 that the eyepiece is not in contact with the eye, then in S524 the system controller 50 determines whether a predetermined time T4 has elapsed since the eyepiece 16 was removed. If it is determined that time T4 has elapsed, the process proceeds to S525; otherwise, the process proceeds to S538. Time T4 may be equal to time T3, or may be longer or shorter than time T3.

[0093] In S525, the system control unit 50 sets the highlight fixation flag to OFF, and in S526, the system control unit 50 sets the re-eyepiece flag to OFF.

[0094] If the eye-away time is long, S524 to S526 cause the process to proceed to S508 after the eye is away from the eyepiece unit 16. As a result, the previously highlighted icon is no longer highlighted, and the new icon of interest is highlighted.

[0095] If it is determined in S504 that the camera position has been switched, in S527 the system control unit 50 switches the screen orientation (screen layout) (from portrait to landscape, or from landscape to portrait).

[0096] In S528, the system control unit 50 determines whether or not the user has placed their eye on the eyepiece unit 16 based on the output signal of the eyepiece detection unit 57. If it is determined that the user has placed their eye on the eyepiece unit 16, the system control unit 50 proceeds to S529; otherwise, the system control unit 50 proceeds to S534.

[0097] In S529, the system control unit 50 determines whether or not the icon information of interest is stored in the system memory 52. ​​If the icon information of interest is stored, the process proceeds to S530, and if not, the process proceeds to S532.

[0098] In S530, the system control unit 50 reads the icon information of interest from the system memory 52 and starts highlighting the icon indicated by the icon information of interest. In S531, the system control unit 50 sets the highlight fixation flag to ON.

[0099] By S504, S527 to S531, S506, and S521 to S523, when the user switches the camera position while keeping his / her eye close to the eyepiece 16, the icon is highlighted for a period of time T3.

[0100] In S532, the system control unit 50 sets the highlight fixation flag to OFF.

[0101] In S533, the system control unit 50 sets the re-eyepiece flag to OFF.

[0102] If it is determined in S528 that the eye is not placed in close contact, it is determined in S534 whether or not the icon of interest information is stored in the system memory 52. ​​If the icon of interest information is stored, the process proceeds to S535; otherwise, the process proceeds to S536.

[0103] In S535, the system control unit 50 sets the re-eyepiece flag to ON.

[0104] S504, S527, S528, S534, S535, S507, and S517 to S520 cause the icon to be highlighted for a time T3 after the user places their eye on the eyepiece 16 when they switch the camera position without placing their eye on the eyepiece 16.

[0105] In S536, the system control unit 50 sets the re-eyepiece flag to OFF.

[0106] In S537, the system control unit 50 sets the highlight fixation flag to OFF.

[0107] In S538, the system control unit 50 determines whether an instruction to end the shooting mode process has been issued. If it is determined that an instruction to end the shooting mode process has been issued, the system control unit 50 ends the shooting mode process; if not, the system control unit 50 proceeds to S504.

[0108] The various controls described above as being performed by the system control unit 50 may be performed by a single piece of hardware, or the entire device may be controlled by multiple pieces of hardware (e.g., multiple processors or circuits) sharing the processing.

[0109] Although the embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

[0110] For example, highlighting of an item may be stopped when the digital camera is not in eye contact and resumed when the digital camera is re-contacted. In this case, the screen layout may be switched while highlighting is stopped. Although an example has been described in which the layout of the shooting standby screen is switched in response to a change in the digital camera's posture, the type of screen and the method of switching the screen are not particularly limited. For example, the layout of an application such as a word processing application or a spreadsheet application may be switched by maximizing or canceling the maximization. The present invention is also applicable to such layout switching.

[0111] Furthermore, in the above-described embodiment, the present invention has been described as being applied to a digital camera, but this is not limited to this example and can be applied to any electronic device (imaging device) capable of capturing images. For example, the present invention can be applied to personal computers, PDAs, mobile phone terminals, portable image viewers, printers, digital photo frames, music players, game consoles, e-book readers, etc. Furthermore, the present invention can be applied to video players, display devices (including projection devices), tablet terminals, smartphones, AI speakers, home appliances, in-vehicle devices, etc.

[0112] <Other embodiments> The present invention provides a program that realizes one or more functions of the above-described embodiments, over a network or The program may be provided to a system or device via a storage medium, and may be read and executed by one or more processors in the computer of the system or device. It may also be implemented by a circuit (e.g., ASIC) that implements one or more functions.

[0113] The disclosure of this embodiment includes the following configuration, method, program, and medium. (Configuration 1) An acquisition means for acquiring information regarding a user's gaze position; a control means for controlling the display of a screen and for controlling the display of an item corresponding to a position of focus of the user on the screen in a distinguishable manner based on information regarding the gaze position; and When switching the layout of the screen, the control means controls so that the items that were identifiably displayed before the switching are identifiably displayed on the screen after the switching, regardless of the position of the user's attention on the screen after the switching. An electronic device characterized by: (Configuration 2) further comprising a detection means for detecting the attitude of the electronic device; The control means switches the layout of the screen in response to a change in the attitude of the electronic device from a first attitude to a second attitude. 2. The electronic device according to configuration 1. (Configuration 3) The attention position is the gaze position. 3. The electronic device according to configuration 1 or 2. (Configuration 4) The focus position is a position where the gaze position is maintained for a predetermined period of time. 3. The electronic device according to configuration 1 or 2. (Configuration 5) The control for identifiably displaying the items includes: A control to enlarge the item; A control to change the color of the item in question, A control that causes the item to blink, or Control to change the speed of the display change of the item Contains 5. The electronic device according to any one of configurations 1 to 4. (Configuration 6) The control for identifiably displaying the item is a control for displaying the hidden item. 5. The electronic device according to any one of configurations 1 to 4. (Configuration 7) The control means controls the screen after the switching so that the first item that was identifiably displayed before the switching is identifiably displayed, but the second item corresponding to the user's new focus position is not identifiably displayed. 7. The electronic device according to any one of configurations 1 to 6. (Configuration 8) The electronic device described in any one of configurations 1 to 6, characterized in that the control means controls the screen after the switching so that a first item that was identifiably displayed before the switching is identifiably displayed in a first manner, and a second item corresponding to the user's new focus position is identifiably displayed in a second manner. (Configuration 9) On the screen after the switching, the items that were identifiably displayed before the switching are When the size is smaller than the threshold value, the control means does not control the item that was identifiably displayed before the switching to be identifiably displayed on the screen after the switching. 9. The electronic device according to any one of configurations 1 to 8. (Configuration 10) When a position on the screen after the switching that corresponds to an item that was identifiably displayed before the switching is within a predetermined area, the control means does not perform control to identifiably display the item that was identifiably displayed before the switching on the screen after the switching. 10. The electronic device according to any one of configurations 1 to 9. (Configuration 11) The control means controls the display of the item in a distinguishable manner for a first time based on the information about the gaze position. 11. The electronic device according to any one of configurations 1 to 10. (Configuration 12) The control means controls the items that were identifiably displayed before the switching to be identifiably displayed on the screen after the switching for a second period of time. 12. The electronic device according to any one of configurations 1 to 11. (Configuration 13) the electronic device is an imaging device, When the layout of the screen is switched while the eyepiece for viewing the screen is not placed on the eyepiece, the control means controls the screen so that, after the eyepiece is placed on the eyepiece, the item that was identifiably displayed before the switching is identifiably displayed on the screen after the switching for a second period of time. 13. The electronic device according to any one of configurations 1 to 12. (method) an acquisition step of acquiring information about a user's gaze position; a control step of controlling to display a screen, and controlling to identifiably display an item corresponding to a position of focus of the user on the screen based on information regarding the gaze position; and When the layout of the screen is switched, the control step controls so that the items that were identifiably displayed before the switching are identifiably displayed on the screen after the switching, regardless of the position of the user's attention on the screen after the switching. A method for controlling an electronic device. (program) 14. A program for causing a computer to function as each means of the electronic device according to any one of configurations 1 to 13. (medium) 14. A computer-readable storage medium storing a program for causing a computer to function as each means of the electronic device according to any one of configurations 1 to 13. [Explanation of symbols]

[0114] 100: Digital camera 50: System control unit

Claims

1. An acquisition means for acquiring information regarding a user's gaze position; a control means for controlling the display of a screen and for controlling the display of an item corresponding to a position of focus of the user on the screen in a distinguishable manner based on information regarding the gaze position; and When switching the layout of the screen, the control means controls so that the items that were identifiably displayed before the switching are identifiably displayed on the screen after the switching, regardless of the position of the user's attention on the screen after the switching. An electronic device characterized by:

2. further comprising a detection means for detecting the attitude of the electronic device; The control means switches the layout of the screen in response to a change in the orientation of the electronic device from a first orientation to a second orientation.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

3. The attention position is the gaze position.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

4. The focus position is a position where the gaze position is maintained for a predetermined period of time.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

5. The control for identifiably displaying the items includes: A control to enlarge the item; A control to change the color of the item in question, A control that causes the item to blink, or Control to change the speed of the display change of the item Contains 2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

6. The control for identifiably displaying the item is a control for displaying the hidden item.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

7. The control means controls the screen after the switching so that the first item that was identifiably displayed before the switching is identifiably displayed, but the second item corresponding to the new focus position of the user is not identifiably displayed.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

8. The electronic device described in claim 1, characterized in that the control means controls the screen after the switching so that a first item that was identifiably displayed before the switching is displayed identifiably in a first manner, and a second item corresponding to the user's new focus position is displayed identifiably in a second manner.

9. When the size of the item that was identifiably displayed before the switching is smaller than a threshold value on the screen after the switching, the control means does not control the item that was identifiably displayed before the switching to be identifiably displayed on the screen after the switching.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

10. When a position on the screen after the switching that corresponds to an item that was identifiably displayed before the switching is within a predetermined area, the control means does not perform control to identifiably display the item that was identifiably displayed before the switching on the screen after the switching.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

11. The control means controls the display of the item in a identifiable manner for a first time based on the information about the gaze position.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

12. The control means controls the display so that the item that was identifiably displayed before the switching is identifiably displayed on the screen after the switching for a second period of time.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

13. the electronic device is an imaging device, When the layout of the screen is switched while the eyepiece for viewing the screen is not placed close to the eye, the control means controls the item that was identifiably displayed before the switching to be identifiably displayed for a second period of time on the screen after the switching, after the eyepiece is placed close to the eye.

2. The electronic device according to claim 1, wherein the electronic device is a semiconductor device.

14. an acquisition step of acquiring information about a user's gaze position; a control step of controlling to display a screen, and controlling to identifiably display an item corresponding to a position of focus of the user on the screen based on information regarding the gaze position; and When the layout of the screen is switched, the control step controls so that the items that were identifiably displayed before the switching are identifiably displayed on the screen after the switching, regardless of the position of the user's attention on the screen after the switching. A method for controlling an electronic device.

15. A program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 13.

16. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 13.

Images