Display of related user interface object

The user interface system on small devices optimizes icon display by prioritizing relevance based on user context and movement, enhancing interaction efficiency by minimizing required user input.

JP2024164027AActive Publication Date: 2024-11-26APPLE INC
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2024122504
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-26
Estimated Expiration
2033-10-30

AI Technical Summary

Technical Problem

Existing user interfaces on small personal electronic devices, such as tablets and smartphones, require multiple operations before presenting relevant information, leading to inefficiency.

Method used

A user interface system that automatically displays a prioritized subset of relevant icons based on user context and movement, using sensors and algorithms to determine relevance, allowing navigation through multiple screens with minimal user input.

Benefits of technology

Enhances user interaction efficiency by reducing the number of operations needed to access relevant information, optimizing display based on user context and device orientation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024164027000001_ABST
    Figure 2024164027000001_ABST
Patent Text Reader

Abstract

To provide a technology for displaying a related user interface object when a device is placed in a viewing position.SOLUTION: A device can update display according to user input. The update of display can be based on a logical configuration of user interface information along a z-axis.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The disclosed embodiments relate generally to user interfaces for electronic devices. [Background technology]

[0002] Advanced personal electronic devices may have small form factors. Exemplary personal electronic devices include, but are not limited to, tablets and smartphones. Use of such personal electronic devices includes viewing and manipulating user interface objects on display screens that are small and designed to complement the personal electronic device.

[0003] Exemplary user interface objects include digital images, video, text, icons, control elements such as buttons, and other graphics. As used herein, the term "icon" refers to an image used to represent and launch an application, consistent with its ordinary meaning in the art. Additionally, "widgets," as used in the art to refer to simplified diagrams of applications, constitute icons for purposes of this disclosure.

[0004] Existing user interfaces on reduced-size personal electronic devices can be inefficient as they may require multiple actions by the user before appropriate information is presented. Summary of the Invention

[0005] SUMMARY Techniques for presenting user interface objects on a personal electronic device are disclosed. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 illustrates an exemplary personal electronic device.

[0007] [Diagram 2] FIG. 2 illustrates an exemplary user interface.

[0008] [Diagram 3] FIG. 2 illustrates an exemplary user interface.

[0009] [Figure 4] FIG. 2 illustrates an exemplary logical structure of a user interface.

[0010] [Diagram 5] FIG. 2 illustrates an exemplary user interface.

[0011] [Figure 6] FIG. 2 illustrates an exemplary user interface.

[0012] [Figure 7] FIG. 1 illustrates an exemplary computing system.

[0013] [Figure 8] FIG. 2 illustrates an exemplary user interface.

[0014] [Figure 9] FIG. 2 illustrates an exemplary user interface.

[0015] [Figure 10] FIG. 2 illustrates an exemplary user interface.

[0016] [Figure 11] FIG. 2 illustrates an exemplary user interface.

[0017] [Figure 12] FIG. 2 illustrates an exemplary user interface.

[0018] [Figure 13]FIG. 2 illustrates an exemplary user interface.

[0019] [Figure 14] FIG. 2 illustrates an exemplary user interface.

[0020] [Figure 15] FIG. 2 illustrates an exemplary user interface.

[0021] [Figure 16] FIG. 2 illustrates an exemplary user interface.

[0022] [Figure 17] FIG. 2 illustrates an exemplary user interface.

[0023] [Figure 18] FIG. 2 illustrates an example process for displaying a user interface object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] In the following disclosure and description of the embodiments, reference is made to the accompanying drawings, which show, by way of illustration, specific embodiments in which the invention may be practiced. It is understood that other embodiments may be implemented and structural changes may be made without departing from the scope of the present disclosure.

[0025] 1 is a diagram illustrating an exemplary personal electronic device 100 (hereinafter, device 100). In the illustrated example, device 100 includes a body 102. Device 100 may have a touch-sensitive display screen (hereinafter, touch screen) 104.

[0026] Touch screen 104 may include a display device such as a liquid crystal display (LCD), light emitting diode (LED) display, organic light emitting diode (OLED) display, etc., disposed partially or completely behind or in front of a touch sensor panel implemented with any desired touch sensing technology such as mutual capacitance touch sensing, self-capacitance touch sensing, resistive touch sensing, projected scanning touch sensing, etc. Touch screen 104 allows a user to perform various functions by hovering near and touching the touch sensor panel with one or more fingers or other objects.

[0027] In some embodiments, device 100 can have one or more input mechanisms 106 and 108. Input mechanisms 106 and 108, if included, can be touch-sensitive. Examples of touch-sensitive input mechanisms include touch-sensitive buttons and touch-sensitive surfaces. Input mechanisms 106 and 108, if included, can be physical. Examples of physical input mechanisms include push buttons and rotatable mechanisms. Body 102, which may include a bezel, can have predefined areas on the bezel that act as input mechanisms. In some embodiments, device 100 can have attachment mechanisms. If included, such attachment mechanisms can allow device 100 to be attached to, for example, clothing, jewelry, and other wearable accessories. For example, attachment mechanisms can be attached to hats, glasses, earrings, necklaces, shirts, jackets, bracelets, watch straps, chains, pants, belts, shoes, wallets, backpacks, and the like.

[0028] In some embodiments, device 100 may have one or more pressure sensors (not shown) for detecting a force or pressure applied to touchscreen 104. The force or pressure applied to touchscreen 104 may be used as an input to device 100 to perform any desired action, such as making a selection, entering or exiting a menu, displaying additional options / actions, etc. Different actions may be taken based on the amount of force or pressure applied to touchscreen 104. The one or more pressure sensors may further be used to determine the location at which force is being applied to touchscreen 104. 1. Displaying related user interface objects

[0029] 2 illustrates an exemplary device 100 worn by a user 201 walking to his car 202. When the user 201 moves the device 100 to a viewing position, the device 100 automatically displays a user interface screen 203 on the touchscreen 104. In some embodiments, the display elements of the touchscreen 104 are inactive, meaning that they are off or appear to be off, until the user 201 moves the device 100 to a viewing position. In some embodiments, the device 100 can rotate the display content of the touchscreen 104 (e.g., between landscape and portrait modes) so that the displayed information is in the appropriate viewing orientation, regardless of whether the device 100 is held face-up, face-down, or sideways by the user 201.

[0030] The user interface screen 203 includes the user interface objects that the device 100 has determined to be most relevant to the user at this time. In particular, the screen 203 includes an icon 204 for unlocking the vehicle 202, which is useful to the user 201 as the user approaches his or her car. The screen 203 also includes a map icon 205 for accessing traffic information, which may be useful as the user 201 begins a journey. The screen 203 may also include an icon 206, which is useful for referencing upcoming calendar events and providing destination information. The size of the icons displayed may relate to their relevance. On the screen 203, the icon 204 is larger than the icons 205 and 206 because the device 100 has determined that the unlocking of the car 202, provided via the icon 204, is more relevant.

[0031] This user interface display is notable in that it prioritizes and displays to the user 201 a manageable subset of icons even if many more user interface objects are available for display. Additionally, this user interface is available to the user 201 without any user interface navigation input from the user other than a lift of the arm (e.g., without the user 201 having to press a power on or equivalent button). In this manner, the device 100 reduces the amount of user input required to initiate appropriate user interface actions. This advantage is not trivial, especially since the device 100 has a relatively small display screen size compared to smartphones and other electronic devices that may impede a user's navigation of a larger user interface environment.

[0032] It is possible that the number of related user interface objects in a given situation may exceed the number that can reasonably be displayed together on touch screen 104 (such as three as shown in FIG. 2). In this case, device 100 may prioritize the most related icons (as determined by a computer-based relevance algorithm on device 100) for initial display. In some embodiments, a user may cause the remaining related icons to be displayed on the display using input mechanism 106 or 108. In some embodiments, a user may cause the remaining related icons to be displayed on the display using touch screen 104 (such as by swiping touch screen 104 with a touch object).

[0033] FIG. 3 is a diagram illustrating the display of related icons on multiple user interface screens. In the illustrated example, a user interface screen 301 is displayed on the device 100 in response to an upward motion of the device. The screen 301 includes icons 302-304 (which may be icons 204-206 (FIG. 2) in some examples) representing related applications. In response to a rotation of the input mechanism 108 in a direction 306, a user interface screen 311 is displayed on the device 100. The screen 311 may display several additional related icons 312-314 that are less relevant than those shown on the screen 301. In response to a further rotation of the input mechanism 108 in the same direction 306, the device 100 may display a user interface screen 321. The screen 321 includes another set of related icons 322-324 that are less relevant than those shown on the screen 311, and thus less relevant than those on the screen 301. The input mechanism 108 may be a rotatable crown. In this manner, a user can navigate among multiple sets of related user interface objects (eg, icons) on device 100.

[0034] A user can touch (e.g., with a finger tap) a displayed icon to launch an application corresponding to the displayed icon. As used herein, launching an application means that the application runs in the foreground of device 100 and is displayed on the screen. FIG. 4 illustrates this aspect. The illustrated example shows a user interface screen 401 displayed on device 100 in response to movement of the device to a viewing position. Screen 401 includes an icon 402 representing a messaging application (e.g., supporting short message service (SMS)) having five unread messages, as well as icons 403 and 404 representing other applications. In response to tapping icon 402 with a touch object (e.g., finger 405), device 100 launches the corresponding messaging application and displays unread messages 412 on user interface screen 411.

[0035] Under some use conditions, a user may wish to navigate from a messaging application to another related application. For example, a user may wish to navigate to a music and maps application, previously represented by icons 403 and 404 on screen 401. Device 100 may enable navigation between these applications directly without first returning to screen 401. Specifically, rotating input mechanism 108 in direction 414 while screen 411 is displayed causes device 100 to display a music player, represented by icon 403, on screen 421. Screen 421 may include music playback controls 423. Further rotating input mechanism 108 in direction 414 while screen 421 is displayed causes device 100 to display a maps application, represented by icon 404, on screen 431. Screen 431 may include traffic information 432.

[0036] In some embodiments, screens 411, 421, and 431 each include a visual aid (such as paging dots 415, 425, and 435) that identifies the relative position of the currently displayed application along the sequence of applications accessible by input mechanism 108. Other visual aids (such as scroll bars and transitions between screens) may also be used to aid the user in identifying the currently displayed user interface screen relative to the larger set of available user interface screens.

[0037] The exemplary user interface screens shown in Figures 2-4 are primarily concerned with the efficient display of related user interface objects. However, it should be noted that device 100 may include many more user interface objects that should be accessible to the user even if their current relevance is not readily identifiable. For example, a user may impulsively want to play a game. Device 100 allows the user to navigate beyond related user interface objects to other user interface objects. Figure 5 illustrates this aspect.

[0038] In FIG. 5, user interface screen 501 is displayed on device 100 in response to movement of the device to a viewing position. Screen 501 includes icons 502-504 (which may be icons 203-205 (FIG. 2) in some examples) representing relevant applications. In the illustrated example, device 100 has determined that only three user interface objects (i.e., icons 502-504) are currently relevant. Thus, in response to rotation of input mechanism 108 in direction 505, device 100 displays user interface screen 511, which has other user interface objects available for user selection on device 100. The icons displayed on screen 511 may be the user's favorite icons, meaning that the icons on screen 511 are a predefined subset of the user interface objects available on device 100. In response to further rotation of input mechanism 108 in direction 505, device 100 displays user interface screen 521, which includes icons representing all applications available on device 100. The size of the icons displayed on screen 521 may be too small for a user to navigate, so in response to further rotation of input mechanism 108 in direction 505, device 100 displays screen 531. This has the effect of enlarging a subset of the icons on screen 521, so that they are displayed at a larger size for user interaction.

[0039] Navigation of the user interface described with reference to Fig. 5 can be logically organized according to a logical structure 600 shown in Fig. 6. In the example shown in Fig. 6, x-axis 601 and y-axis 602 form the same plane as the surface of the touchscreen display of device 100 (Fig. 1), and z-axis 603 is perpendicular to the x / y plane formed by axes 601 and 602. Plane 604 corresponds, in one example, to user interface screen 501 (Fig. 5), plane 605 to user interface screen 511 (Fig. 5), and plane 607 to user interface screens 521 and 531 (Fig. 5). More specifically, screen 521 (Fig. 5) can correspond to a perspective of the entire content of plane 607, and screen 531 (Fig. 5) can correspond to a magnified perspective (i.e., a magnified subset) of the content of plane 607. 3. In another example, planes 604, 607, 608 can correspond to user interface screens 301, 311, and 321, respectively. Movement of the input mechanism can be used to select a particular plane of information (i.e., a screen of icons) to display on device 100. For example, rotation of input mechanism 108 can cause display device 100 to display different screens of icons, for example, in a manner similar to that shown in FIG. 2. Determine the relevant user interface objects

[0040] Consistent with its ordinary meaning, the phrase "relevant icon" is used herein to refer to a user interface icon that is relevant or applicable to the matter at hand. In the example of FIG. 2, as the user approaches his or her car, the icon for unlocking the vehicle application is relevant because the user may want to drive the car. Device 100 may determine relevance using computer instructions (e.g., algorithms) that consider different inputs (including sensor inputs, application data, and operating system data).

[0041] 7 illustrates an exemplary computing system 700 that, in some embodiments, forms device 100. Computing system 700 includes components for determining and displaying associated user interface objects. In the illustrated example, computing system 700 includes an I / O section 704 that can be operatively coupled to various sensors, such as a GPS sensor 720, an acceleration sensor 722, an orientation sensor 724, a gyroscope 726, a light sensor 728, and / or combinations thereof. I / O section 704 can also be connected to a communication unit 718 via Wi-Fi, Bluetooth, near field communication ("NFC"), cellular, and other wireless communication technologies, for receiving application and operating system data. Additionally, computing system 700 can include a bus 702 that couples I / O section 704 together with one or more computer processors 706 and a memory section 708. Memory section 708 can store computer executable instructions (e.g., representing algorithms) and / or data for determining and displaying associated user interface objects. One or more of these components may be part of an integrated chip or a so-called system-on-chip. Furthermore, the I / O section 704 may be connected to an input mechanism 714. The I / O section 704 may be connected to one or more input buttons 716. The I / O section 704 may be connected to a display 710, which may have a touch-sensitive component 712 and, optionally, a touch-sensitive component 713.

[0042] The sensors and communication units of the computing system 700 can provide information for identifying relevant user interface objects. For example, the GPS sensor 720 can determine the location and movement of the user, while the communication unit 718 can receive information about the location and identification of nearby vehicles (e.g., vehicle 202 in FIG. 2). The acceleration sensor 722, the orientation sensor 724, and the gyroscope 726 can further detect device movement. Optionally, the output of the GPS sensor 720, the accelerometer 722, the orientation sensor 724, and / or the gyroscope 726 can be interpreted by the motion processor 730. The computer-executable instructions in the processor 706 and the memory portion 708 can use some or all of this information to determine that the user is approaching his / her vehicle. The instructions in the processor 706 and the memory 708 can also determine that an application is installed for interacting with the user's vehicle based on application data and / or operating system data (including metadata) stored in the memory 708. In this manner, the relevance algorithm of device 100 can determine that a car interaction application is currently relevant to the user. Additionally, device 100 can conclude, based on the same data, that a map application would also be relevant to the user.

[0043] The communication unit 718 can also receive other information that affects the relevance of user interface objects. For example, the communication unit can detect nearby devices that are identical or similar, such as other wearable devices of the same design. The communication unit can also detect non-identical units (such as smartphones and tablets of the same brand) running the same operating system as the device 100. The communication unit can also identify different devices that support communication over a common protocol. These protocols can include wireless protocols such as Wi-Fi, Bluetooth, NFC, and others. These protocols can also be software-based service protocols, such as operating environment service protocols (Apple® AirPlay® and AirDrop®), home automation service protocols (e.g., those offered by Phillips® Lighting and Nest®), authentication service protocols (e.g., airport clearance and subway fares), point of sale service protocols (e.g., grocery store checkout), and the like. The algorithms used by the device 100 to identify related user interface objects can take into account these inputs provided by the communication unit 718.

[0044] Additionally, the communications unit 718 can receive application and operating system data that signals relevance. For example, a messaging application can receive an incoming message by SMS or Wi-Fi service, thereby becoming relevant. As another example, the relevance algorithm of the device 100 can use calendar data and cellular system time to determine that an event reminder is relevant. Additionally, the relevance algorithm of the device 100 can consider the content of the application and operating system data when determining relevance. For example, the algorithm can consider that an incoming message that includes a reference to a particular time (e.g., "Let's meet at 3 p.m.") becomes more relevant as the time (i.e., 3 p.m.) approaches.

[0045] In some embodiments, user interface objects may be related in groups. That is, application data (including metadata) may specify that whenever user interface object A is related, user interface object B is also related. For example, a music application may be linked in this manner to a car interaction application because drivers typically enjoy music. A map application may also be linked in this manner to a car interaction application because drivers typically want traffic and / or route information.

[0046] In some embodiments, the relevance algorithm used by device 100 can be adaptive, meaning that the results of the algorithm can change based on past user behavior. For example, the algorithm can recognize a user's commute based on the user's driving patterns during the weekday morning. In this manner, device 100 can prioritize certain traffic information to display in the morning. As another example, if a user repeatedly launches one particular radio application over other available radio applications during their commute, device 100 can identify that radio application as being more relevant and display its icon whenever the user unlocks their car.

[0047] In some embodiments, the computing system 700 can include biometric sensors, such as health-related sensors, such as a photoplethysmograph (PPG) sensor, an electrocardiogram (ECG) sensor, and / or a galvanic skin response (GSR) sensor. The device 100 can receive input from one or more of these sensors that provide health-related information. For example, the device 100 can use information from a PPG sensor to alert a user to abnormal respiratory rate, blood pressure, and / or oxygen saturation. As another example, the device 100 can use an ECG sensor to alert a user to an irregular heartbeat. As yet another example, the device 100 can use a GSR sensor to detect moisture on the user's skin, indicative of sweating, and prioritize a thermostat application for display on the device 100. These sensors can also be used to facilitate biometric identification and authentication of the user.

[0048] Sensors in the computing system 700 can detect when the system (e.g., device 100) is placed in a viewing position. For example, the accelerometer 722 and / or the motion sensor 730 can detect when the computing system 700 is lifted, lowered, or shaken. These sensors can detect forward and backward rotation of the wrist. In some embodiments, lifting of the computing device 700 is interpreted as placing the device in the viewing position. In some embodiments, lifting and rotating of the computing device 700 is interpreted as placing the device in the viewing position. In some embodiments, the duration between lifting and lowering of the computing device 700 is interpreted as placing the device in the viewing position.

[0049] The algorithms used by device 100 to identify relevant user interface objects for display may use one or more of the aforementioned aspects of the device (e.g., computing system 700) in determining relevance, that is, the algorithms may consider a combination of inputs including: position, motion (including posture, orientation, tilt, acceleration, and speed), ambient conditions (including light, time, temperature, and the user's health), and application data (including incoming calls, incoming messages, and upcoming calendar events).

[0050] For example, if the device 100 is moving at a speed above a threshold (e.g., 10 miles per hour, 20 miles per hour, 25 miles per hour, 30 miles per hour, 40 miles per hour, 50 miles per hour, 55 miles per hour, 60 miles per hour, 65 miles per hour, etc.), the device 100 may determine that the device user is commuting to work and an icon corresponding to a navigation application is more relevant. In such a situation, if an available in-vehicle device is in communication with the communication unit of the device 100, the device 100 may also determine that an icon representing an in-vehicle entertainment application is more relevant. As another example, the device 100 may determine that an icon representing a health-related application is more relevant if the biometric sensor and the motion sensor detect motion representative of exercise. As another example, the device 100 may determine that a calendar event scheduled to occur in a particular amount of time (e.g., 15 minutes, 30 minutes, 1 hour, 1 day, 1 week, etc.) is more relevant. Optionally, device 100 may take into account other variables in determining the relevance of an event, such as the distance between the device's current location and the location of the event, as well as the current weather, etc. That is, device 100 may determine, for example, that a nearby event scheduled in 15 minutes is less relevant than an event scheduled in an hour but 30 miles away. 3. Example User Interactions

[0051] A user can interact with the user interface of device 100. These interactions can include shortcuts for invoking application functions. This aspect will be described with reference to Figures 8-9.

[0052] In the example of FIG. 8, device 100 has just received an incoming SMS message and provided haptic feedback to the user. In response to the haptic feedback, the user raises device 100 to a viewing position, causing device 100 to display user interface screen 801. Screen 801 includes icons 802-804 that represent applications determined to be relevant to the user at that time. Icon 802 represents an unread SMS message. Icon 803 represents an upcoming calendar event. Icon 804 represents available traffic information. Icon 802 is displayed in a large format because recently received SMS messages are ranked highest in relevance.

[0053] Because message icon 802 has the highest relevance, when the user rotates input mechanism 108 in direction 805, device 100 launches a corresponding messaging application and displays unread SMS messages 812 on user interface screen 811. In response to further rotation of input mechanism 108 in direction 805, device 100 displays calendar events 822 on user interface screen 821 in a calendar application represented by icon 803. In response to further rotation of input mechanism 108 in direction 805, device 100 displays traffic information provided by a map application (corresponding to icon 804) on user interface screen 831.

[0054] From screen 811, the user may tap SMS message 812 to invoke user interface screen 901 shown in FIG. 9. Referring to FIG. 9, screen 901 includes an icon 902 for replying to SMS message 812. Screen 901 also includes an icon 903 for generating an alarm at 3:00 PM suggested by SMS message 812. Similarly, when screen 821 (FIG. 8) is displayed, the user can tap calendar event 822 to invoke user interface screen 911 shown in FIG. 9. Screen 911 includes an icon 912 for messaging an event participant (e.g., Larry). Screen 911 also includes an icon 913 for obtaining navigation to the event location. Finally, when screen 831 (FIG. 8) is displayed, the user can tap map 832 to invoke user interface screen 921 shown in FIG. 9. Screen 921 includes an icon 922 for setting a navigation waypoint and an icon 923 for obtaining turn-by-turn navigation instructions.

[0055] In some embodiments, device 100 can distinguish between short and long taps on touch screen 104 (FIG. 1), e.g., can call up screen 901 only after a long tap on screen 811 (FIG. 8). For purposes of this disclosure, a short tap refers to a short touch on touch screen 104 (FIG. 1) followed by a release of the touch. A long tap refers to a longer touch on touch screen 104 (FIG. 1) before the release of the touch. Device 100 can consider a touch that exceeds a predefined duration as a long tap (and a touch of a shorter duration as a short tap). In some embodiments, device 100 can identify the degree of pressure on touch screen 104. That is, device 100 can detect the strength of a touch object (e.g., a user's finger) on touch screen 104. Thus, device 100 can call up screen 901 only after a user taps screen 811 (FIG. 8) with sufficient pressure.

[0056] In some embodiments, device 100 can distinguish between short glances and longer gazes at touch screen 104 (FIG. 1). A short glance can be characterized by a short duration between lifting the device into the viewing position and the subsequent lowering of the device. A longer gaze can be characterized by a period of relative stationary movement of the device in the viewing position. Device 100 can respond differently to short glances and longer gazes. This aspect is illustrated in FIG. 10. In the example of FIG. 10, a user interface screen 1001 is displayed in response to a user's movement of device 100 into a viewing position. However, because message 1002 arrived just before device 100 was lifted into the viewing position, user interface screen 1001 highlights a display of an unread SMS message 1002 from a contact instead of displaying multiple related user interface objects. If the user maintains device 100 in the viewing position for more than a predetermined amount of time, device 100 replaces screen 1001 with user interface screen 1011 (showing a number of icons representing related user interface objects available on device 100). From screen 1011, the user can tap icon 1012 with finger 1013 to return to SMS messages 1002. In this manner, device 100 allows the user to easily glance at incoming messages. 4. Exemplary User Interface.

[0057] 11-16 illustrate exemplary user interfaces that device 100 may display over the course of a day based on relevance. In FIG. 11, device 100 determines that the user has just woken up and displays an appropriate greeting 1102 saying "Good morning." Device 100 may make this determination based on the time of day, the user's interaction with an alarm clock application (e.g., the user has just turned off the alarm), and / or device movement indicating, for example, that the user is walking after a period of being motionless. Device 100 may rank greeting 1102 as the most relevant icon to display to the user when the user wakes up. Because of its high relevance, greeting 1102 is highlighted in user interface screen 1101, meaning that greeting 1102 may be the largest icon displayed, or the only icon displayed. Note, however, that if greeting 1102 is the only icon displayed, other non-icon user interface elements (such as the current time) may still be displayed on-screen.

[0058] User interface screen 1111 illustrates another exemplary user interface that device 100 may display when the user wakes up. Screen 1111 includes an icon 1112 that indicates the current time. Icon 1113 may have a circumferential outline 1113 that indicates the time remaining for snooze. Optionally, icon 1112 has a background that indicates the current weather, e.g., blue represents warm weather and gray represents inclement weather. Screen 1111 may also include an icon 1115 that indicates unread messages that the user should pay attention to.

[0059] 12 illustrates a user interface screen 1201 that may show additional related user interface objects after the user wakes up. Screen 1201 includes related icons 1202-1204. Icon 1202 may correspond to a health application and may show sleep information, such as the duration of sleep by the user. Icon 1203 may correspond to calendar information, such as the time remaining before the next calendar event. Icon 1204 may correspond to additional calendar information, such as all-day events.

[0060] User interface screen 1211 illustrates additional relevant user interface objects that device 100 may display after the user wakes up. Screen 1211 includes associated icons 1212 and 1213. Icon 1212 may correspond to a weather application that shows the weather at the device's current location. Optionally, icon 1212 may show the weather at a location the user has traveled to so far in the morning, such as the weather at the user's work location. Additionally, icon 1213 may indicate that the user should begin their morning commute in 45 minutes. Device 100 may make this determination based, for example, on the first event on today's calendar, the user's usual destination during the weekday morning, and the expected travel time to that destination based on distance and traffic information.

[0061] User interface screen 1221 illustrates additional relevant user interface objects that device 100 may display later in the morning. Exemplary user interface screen 1221 includes relevant icons 1222-1224. Icon 1222 indicating weather conditions may display the same information previously displayed by icon 1212. However, whereas in screen 1211 icon 1212 was the most relevant icon, in screen 1221 its relevance has been superseded by traffic icon 1223. Traffic icon 1223 indicates a traffic alert and is displayed as the largest icon because device 100 has determined that information regarding an accident along the user's normal morning commute is highly relevant at that time. Screen 1221 also includes icon 1224 indicating that the user's commute should begin in 10 minutes given the traffic information received by device 100 (caused by an accident) rather than the 45 minute instruction previously given by icon 1213.

[0062] Referring to FIG. 13, screen 1301 shows an icon 1302 for unlocking the user's car when the user approaches his / her car. Based on a decrease in distance between device 100 and the user's nearby car, device 100 may display icon 1302. Optionally, screen 1301 may include additional relevant icons, such as those described in connection with FIG. 1. While the user is in his / her car, device 100 may display user interface screen 1311 when raised to a viewing position. Screen 1311 includes information regarding estimated time of arrival ("ETA") to work (i.e., icon 1312), time to the next meeting on the user's calendar (i.e., icon 1313), and a music player (i.e., as represented by icon 1314), which are relevant to the user as he / she is on his / her way to work. The device 100 may determine that the user is driving based on GPS movement and / or by communication with an in-vehicle telematics system (e.g., via Bluetooth or cable connection). The device 100 may determine that the user is driving to work based on historical information about the user's commuting patterns. As the user gets closer to work, the estimated time of arrival becomes less relevant and the information will be displayed with less importance. For example, in user interface screen 1321, music icon 1322 is displayed in a larger format than ETA icon 1324. The information of the time until the next meeting on the calendar remains relevant, so icon 1323 may continue to display this time. If the meeting is off-site (i.e., physically far from the user's work place), the device 100 may mark the reminder as relevant based on the GPS sensor and calendar information.

[0063] Referring to FIG. 14, later in the day, the user of the device 100 may visit a store, such as a coffee shop. In screen 1401, the device 100 may display an electronic payment icon 1402 that allows the user to approve a purchase at the coffee shop. The device 100 may determine its proximity to the coffee shop based on GPS information and application data provided by a map application or a third party application, such as a Starbucks® application. The device 100 may also determine its proximity to the coffee shop based on wireless communication with the store's point of sale system, such as via near-field communication with a payment reader. Additionally, in screen 1401, the device 100 may display an icon 1403 indicating the proximity of a contact (e.g., a friend) at the coffee shop. In screen 1411, the device 100 may display an icon 1412 indicating a new incoming message, an icon 1413 counting down to an upcoming meeting, and an icon 1414 suggesting that the user should take the stairs to the meeting location for extra exercise. If the user is late for a meeting, the device 100 can remind the user. For example, in screen 1421, the device 100 can display icon 1422 that alerts the user that the user is eight minutes late to a meeting listed on the calendar, and icons 1422 and 1424 that inform the user of new incoming messages, some of which may be triggered by the user's missed meeting.

[0064] 15, the device 100 can display relevant information as the workday draws to a close. In user interface screen 1501, the device 100 can display the ETA to the user's home (i.e., icon 1502) and the ETA to the user's spouse's home (i.e., icon 1503). In user interface screen 1511, the device 100 can continue to display the ETA to the user's home (i.e., icon 1512) and display a music application (i.e., icon 1513) to change the music in the user's car, and a stress level indicator (i.e., icon 1514). The device 100 can calculate the user's stress level based on sensor inputs including, for example, PPG, ECG, and GSR sensor readings. When the user arrives home and looks at the device 100, the device 100 can display icon 1522 to unlock the wirelessly enabled front door. The device 100 may also display icons 1524 for controlling home electronics, such as light and fireplace settings with Wi-Fi enabled lighting and HVAC controllers. The device 100 may also display an icon 1523 indicating a dinner event.

[0065] Referring to FIG. 16, the device 100 can display relevant information when the day is ending. In user interface screen 1601, the device 100 can display, for example, an icon 1602 suggesting that the user should go to sleep soon, based on the user's usual sleep time and the activity listed on the calendar for the next morning. The device 100 can also display an icon 1604 for controlling the television, based on the user's habit of watching television in the evening. The device 100 can also display an icon 1603 for lighting control, based on the user's usual end-of-day routine. As the user's usual bedtime continues to approach, the device 100 can display a summary of the user's physical activity for the day (i.e., icon 1612 indicating that the user has met 75% of the goal for the day), and an alarm clock icon 1613 for setting an alarm for the next morning. The device 100 can also reduce the amount of user interface objects displayed at the end of the day. For example, as shown in screen 1621, the device 100 can display a single icon 1622 suggesting sleep. Additionally, icon 1622 may be displayed using wavelengths of light that are less likely to disrupt the user's sleep patterns. In this manner, device 100 may maintain the user's wakefulness and / or avoid waking a sleeping user.

[0066] Optionally, device 100 may be configured to persistently display a clock face. This aspect is described with respect to FIG. 17. In the illustrated example, device 100 displays user interface screen 1701 in response to lifting the device to a viewing position. In screen 1701, clock 1702 is displayed along with associated icon 1703. As device 100 identifies additional associated user interface objects, they may be displayed on top of touch screen 104 (FIG. 1) around clock 1702 (as indicated by additional associated icons 1712 and 1703 on screens 1711 and 1721). In this manner, a user may configure device 100 to emphasize timekeeping functionality.

[0067] FIG. 18 illustrates an example process 1800 that may be performed by device 100 to display relevant user interface objects. At block 1810, device 100 obtains input from a motion sensor indicative of a movement of the device toward a viewing position. In some embodiments, the movement may be an upward movement. At block 1820, device 100 obtains additional sensor data. Such sensor data may include GPS location information, light information, motion information, and / or accelerometer information. At block 1830, device 100 obtains application or operating system data. Such data may be obtained by a communication link such as Wi-Fi, Bluetooth, or NFC. At block 1840, device 100 identifies relevant user interface objects for display to the user based on the sensor data and the application / OS data. Device 100 may also rank the relevant user interface objects. At block 1850, the most relevant user interface objects are displayed to the user. At block 1860, device 100 receives input representing a movement of an input mechanism. In response, at block 1870, device 100 displays icons representing the user's favorite applications available on the device. At block 1880, device 100 receives additional input representing movement of an input mechanism. In response, at block 1890, device 100 displays icons representing all applications available on the device.

[0068] Returning to FIG. 7, the memory portion 708 of the computing system 700 may be a non-transitory computer readable storage medium that stores computer executable instructions that, when executed by one or more computer processors 706, can cause the computer processors to execute the user interface techniques described above, including process 1800 (FIG. 18). The computer executable instructions may also be stored and / or transferred in any non-transitory computer readable storage medium for use by or in association with an instruction execution system, device, or appliance, such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, device, or appliance. For purposes of this document, a "non-transitory computer readable storage medium" may be any medium that can store or store computer executable instructions for use by or in association with an instruction execution system, device, or appliance. Non-transitory computer readable storage media may include, but are not limited to, magnetic, optical, and / or semiconductor storage. Examples of such storage may include magnetic disks, CDs, DVDs, or optical disks based on Blu-ray technology, as well as RAM, ROM, EPROM, flash memory, and solid-state memory. Computing system 700 is not limited to the components and configuration of FIG. 7, but may include other or additional components in multiple configurations.

[0069] Although the present disclosure and examples have been fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will be apparent to those skilled in the art, and such changes and modifications are to be understood as being included within the scope of the present disclosure and examples, as defined by the appended claims.

Claims

1. An electronic device, One or more processors, the processor comprising: a motion sensor configured to detect motion of the electronic device; Memory, A touch-sensitive display; operatively connected to The one or more processors: receiving an input from the motion sensor based on a motion of the electronic device; displaying a first plurality of icons on the touch-sensitive display; It is configured as follows: the display is responsive to the received motion sensor input; the first plurality of icons being a subset of a larger plurality of icons available for display; the first plurality of icons is selected from the larger plurality of icons using a relevance algorithm; electronic equipment.

2. The one or more processors: further operatively coupled to the rotatable input mechanism; receiving an input based on a movement of the rotatable input mechanism; replacing the first plurality of icons with a second plurality of icons on the touch-sensitive display in response to the received rotatable input mechanism input; The electronic device of claim 1 , further configured as follows:

3. The one or more processors: receiving input from the touch-sensitive display based on movement of a touch object; replacing the first plurality of icons with a second plurality of icons on the touch-sensitive display in response to the received touch-sensitive display input; The electronic device according to claim 1 or 2, further configured as follows.

4. The electronic device according to claim 1 , wherein the electronic device is a wearable device.

5. The electronic device of claim 1 , further comprising a GPS sensor operably coupled to the one or more processors.

6. The electronic device of claim 1 , further comprising an accelerometer operably coupled to the one or more processors.

7. The electronic device of claim 1 , further comprising a direction sensor operably coupled to the one or more processors.

8. The electronic device of claim 1 , further comprising a gyroscope operably coupled to the one or more processors.

9. The electronic device of claim 1 , further comprising a light sensor operably coupled to the one or more processors.

10. The electronic device of claim 1 , further comprising a biometric sensor operably coupled to the one or more processors.

11. The electronic device according to claim 1 , further comprising a wireless communication unit configured to communicate with an external device.

12. The electronic device according to claim 11 , wherein the wireless communication unit performs Bluetooth communication.

13. The electronic device according to claim 11 or 12, wherein the wireless communication unit performs short-range communication.

14. The one or more processors: receiving an input from the GPS sensor representative of the location of the electronic device; Obtaining application data that identifies the external device; The method further comprises:

14. The electronic device of claim 5, wherein the relevance algorithm uses the location of the electronic device and the location of the external device as inputs.

15. 15. The electronic device of claim 14, wherein as the distance between the electronic device and the location of the external device decreases, the relevance algorithm increases the relevance of an icon associated with the external device from among the larger plurality of icons.

16. The one or more processors: receiving an input from the GPS sensor; determining a moving speed of the electronic device based on the received GPS sensor input; The method further comprises:

16. The electronic device of claim 5, wherein the relevance algorithm uses the location of the electronic device and the location of the external device as inputs.

17. 17. The electronic device of claim 5, wherein the relevance algorithm uses the current time as an input.

18. The one or more processors: receiving an input from the external device by the wireless communication unit; Identifying the external device; The method further comprises:

18. An electronic device as claimed in any one of claims 5 to 17, wherein the relevance algorithm uses an identity of the external device as an input.

19. The relevance algorithm uses metadata as input; The electronic device of claim 5 , wherein the metadata identifies at least one application that corresponds to the icon of the first plurality of icons.

20. 20. The electronic device of claim 5, wherein the relevance algorithm uses application data as input.

21. the application data represents a message; The electronic device of claim 20 , wherein the relevance algorithm increases the relevance of icons associated with messaging from among the larger plurality of icons.

22. the application data represents map information; 21. The electronic device of claim 20, wherein the relevance algorithm increases the relevance of an icon associated with map information from the larger plurality of icons when the electronic device is moving faster than a predetermined speed.

23. the application data represents upcoming calendar events; 21. The electronic device of claim 20, wherein the relevance algorithm increases the relevance of an icon from the larger plurality of icons associated with an upcoming calendar event.

24. The one or more processors: receiving an input from the biometric sensor representative of health information; It is further constructed, The relevance algorithm comprises: Using the health information as an input; increasing the relevance of an icon from the larger plurality of icons associated with an application for displaying the health information when the biometric sensor input indicates that a user of the electronic device is exercising.

24. The electronic device according to claim 5.

25. the one or more processors are further configured to store in the memory data determined using input from at least one day ago; 25. The electronic device of claim 5, wherein the relevance algorithm uses the stored data from the previous day as input.

26. The input from at least one day prior to the day 26. The electronic device of claim 25, including input from at least one of the GPS sensor, the acceleration sensor, the orientation sensor, the gyroscope, and / or the light sensor.

27. The second plurality of icons include: a further subset of the larger plurality of icons; selecting from among the greater number of icons using the relevance algorithm; The electronic device of claim 1 , wherein the first icon is different from the first plurality of icons.

28. The electronic device of claim 1 , wherein the movement is an upward movement.

29. 29. The electronic device of claim 1, wherein the second plurality of icons is a subset of the larger plurality of icons selected by a user.

30. 30. The electronic device of claim 1, wherein the touchscreen was inactive immediately prior to the movement.

31. 1. A computer-implemented method comprising: receiving an input from the motion sensor based on a motion of the electronic device; Displaying a first plurality of icons on the touch-sensitive display; Including, the display is responsive to the received motion sensor input; the first plurality of icons being a subset of a larger plurality of icons available for display; the first plurality of icons is selected from the larger plurality of icons using a relevance algorithm; method.

32. Receiving an input based on a movement of a rotatable input mechanism; replacing the first plurality of icons with a second plurality of icons on the touch-sensitive display in response to the received rotatable input mechanism input; 32. The method of claim 31 further comprising:

33. receiving input from the touch-sensitive display based on movement of a touch object; replacing the first plurality of icons with a second plurality of icons on the touch-sensitive display in response to the received touch-sensitive display input; 33. The method of claim 31 or 32, further comprising:

34. 34. The method of any one of claims 31 to 33, wherein the electronic device is a wearable device.

35. receiving input from a GPS sensor indicative of a location of the electronic device; obtaining application data identifying the external device; Further comprising:

35. The method of any one of claims 31 to 34, wherein the relevance algorithm uses the location of the electronic device and the location of the external device as inputs.

36. 36. The method of claim 35, wherein as the distance between the electronic device and the location of the external device decreases, the relevance algorithm increases the relevance of an icon associated with the external device from among the larger plurality of icons.

37. receiving an input from a GPS sensor; determining a moving speed of the electronic device based on the received GPS sensor input; Further comprising:

37. A method according to any one of claims 31 to 36, wherein the relevance algorithm uses the location of the electronic device and the location of the external device as inputs.

38. 38. A method according to any one of claims 31 to 37, wherein the relevance algorithm uses the current time as an input.

39. receiving an input from the external device by the wireless communication unit; identifying the external device; Further comprising:

39. A method according to any one of claims 31 to 38, wherein the relevance algorithm uses an identification of the external device as an input.

40. The relevance algorithm uses metadata as input; 40. The method of any one of claims 31 to 39, wherein the metadata identifies at least one application that corresponds to the icon of the first plurality of icons.

41. 41. The method of any one of claims 31 to 40, wherein the relevance algorithm uses application data as input.

42. the application data represents a message; 42. The method of claim 41, wherein the relevance algorithm increases the relevance of icons associated with messaging from among the larger plurality of icons.

43. the application data represents upcoming calendar events; 42. The method of claim 41, wherein the relevance algorithm increases the relevance of an icon from the larger plurality of icons that is associated with an upcoming calendar event.

44. the application data represents map information; 42. The method of claim 41, wherein the relevance algorithm increases the relevance of an icon from the larger plurality of icons associated with map information when the electronic device is moving faster than a predetermined speed.

45. receiving an input from the biometric sensor representative of health information; The relevance algorithm comprises: Using the health information as an input; increasing the relevance of an icon from the larger plurality of icons associated with an application for displaying the health information when the biometric sensor input indicates that a user of the electronic device is exercising.

45. The method of any one of claims 31 to 44.

46. storing in a memory the data determined using input from at least one day prior; 46. ​​A method according to any one of claims 31 to 45, wherein the relevance algorithm uses data stored from the previous day as input.

47. The input from at least one day prior to the day 47. The method of claim 46, comprising input from at least one of the GPS sensor, the acceleration sensor, the orientation sensor, the gyroscope, and / or the light sensor.

48. The second plurality of icons include: a further subset of the larger plurality of icons; selecting from among the greater number of icons using the relevance algorithm; 48. The method of any one of claims 31 to 47, wherein the first plurality of icons is different.

49. 49. The method of any one of claims 31 to 48, wherein the movement is an upward movement.

50. 50. The method of any one of claims 31 to 49, wherein the second plurality of icons is a subset of the larger plurality of icons selected by a user.

51. 51. The method of any one of claims 31 to 50, wherein the touchscreen was inactive immediately prior to the movement.

52. 1. A non-transitory computer-readable storage medium having computer-executable instructions that, when executed by one or more computer processors, cause the one or more computer processors to display a user interface, the computer-executable instructions comprising: receiving an input from the motion sensor based on a motion of the electronic device; Displaying a first plurality of icons on the touch-sensitive display; Contains a set of instructions, the display is responsive to the received motion sensor input; the first plurality of icons being a subset of a larger plurality of icons available for display; the first plurality of icons is selected from the larger plurality of icons using a relevance algorithm; A non-transitory computer-readable storage medium.

53. The computer executable instructions include: receiving input based on movement of a rotatable input mechanism; replacing the first plurality of icons with a second plurality of icons on the touch-sensitive display in response to the received rotatable input mechanism input; 53. The non-transitory computer readable storage medium of claim 52, further comprising instructions.

54. The computer executable instructions include: receiving input from the touch-sensitive display based on movement of a touch object; replacing the first plurality of icons with a second plurality of icons on the touch-sensitive display in response to the received touch-sensitive display input; 54. The non-transitory computer readable storage medium of claim 52 or 53, further comprising instructions.

55. 55. The non-transitory computer-readable storage medium of any one of claims 52 to 54, wherein the electronic device is a wearable device.

56. The computer executable instructions include: receiving an input from a GPS sensor indicative of a location of the electronic device; Obtaining application data that identifies the external device; Further comprising a set of instructions:

56. The non-transitory computer-readable storage medium of any one of claims 52 to 55, wherein the relevance algorithm uses the location of the electronic device and the location of the external device as inputs.

57. 57. The non-transitory computer-readable storage medium of claim 56, wherein as the distance between the electronic device and the location of the external device decreases, the relevance algorithm increases the relevance of an icon associated with the external device from among the larger plurality of icons.

58. The computer executable instructions include: Receives input from a GPS sensor; determining a moving speed of the electronic device based on the received GPS sensor input; Further comprising a set of instructions:

58. The non-transitory computer-readable storage medium of any one of claims 52 to 57, wherein the relevance algorithm uses the location of the electronic device and the location of the external device as inputs.

59. 59. The non-transitory computer readable storage medium of any one of claims 52 to 58, wherein the relevance algorithm uses a current time as an input.

60. The computer executable instructions include: receiving an input from the external device via the wireless communication unit; Identifying the external device; Further comprising a set of instructions:

60. The non-transitory computer readable storage medium of any one of claims 52 to 59, wherein the relevance algorithm uses the identification of the external device as an input.

61. The relevance algorithm uses metadata as input; 61. The non-transitory computer-readable storage medium of any one of claims 52 to 60, wherein the metadata identifies at least one application that corresponds to the icon of the first plurality of icons.

62. 62. The non-transitory computer-readable storage medium of any one of claims 52 to 61, wherein the relevance algorithm uses application data as input.

63. the application data represents a message; 63. The non-transitory computer readable storage medium of claim 62, wherein the relevance algorithm increases the relevance of an icon associated with messaging from among the larger plurality of icons.

64. the application data represents a calendar event; 63. The non-transitory computer-readable storage medium of claim 62, wherein the relevance algorithm increases the relevance of an icon associated with the event from among the larger plurality of icons.

65. the application data represents map information; 63. The non-transitory computer-readable storage medium of claim 62, wherein the relevance algorithm increases the relevance of an icon associated with map information from the larger plurality of icons when the electronic device is moving faster than a predetermined speed.

66. The computer executable instructions include: instructions for receiving input from the biometric sensor representative of health information; The relevance algorithm comprises: Using the health information as an input; increasing the relevance of an icon from the larger plurality of icons associated with an application for displaying the health information when the biometric sensor input indicates that a user of the electronic device is exercising.

66. The non-transitory computer readable storage medium of any one of claims 52 to 65.

67. The computer executable instructions include: further comprising instructions for storing in a memory the data determined using input from at least one day prior; 67. The non-transitory computer readable storage medium of any one of claims 52 to 66, wherein the relevance algorithm uses data stored from the previous day as input.

68. The input from at least one day prior to the day 68. The non-transitory computer-readable storage medium of claim 67, comprising input from at least one of the GPS sensor, the acceleration sensor, the orientation sensor, the gyroscope, and / or the optical sensor.

69. The second plurality of icons include: a further subset of the larger plurality of icons; selecting from among the greater number of icons using the relevance algorithm; 69. The non-transitory computer readable storage medium of any one of claims 52 to 68, different from the first plurality of icons.

70. 70. The non-transitory computer readable storage medium of any one of claims 52 to 69, wherein the movement is an upward movement.

71. 71. The non-transitory computer-readable storage medium of any one of claims 52 to 70, wherein the second plurality of icons is a user-selected subset of the larger plurality of icons.

72. 72. The non-transitory computer-readable storage medium of claim 52, wherein the touch screen was inactive immediately prior to the movement.

Citation Information

Patent Citations

  • Portable terminal equipment, information display method and information processing method

    JP1998340178A

  • Portable information terminal device

    JP2002007016A

  • Processor and method for information processing, and recording medium

    JP2002157078A

  • Portable electronic apparatus

    JP2009290847A

  • Electronic apparatus in which navigation application is installed, control method of electronic apparatus and computer program

    JP2013200265A