Device and method for manipulating user interface with stylus

Abstract

To provide an electronic device and method with faster or more efficient methods for manipulating user interfaces with a stylus.SOLUTION: An electronic device includes a touch-sensitive surface, a display, and one or more sensors to detect signals from a stylus. The electronic device is configured to: detect an input from the stylus; determine the characteristics of the input including a distance of the stylus relative to the touch-sensitive display and a direction of movement of the stylus, on the basis of the input; in response to detecting the input, displaying a mark on the display; and adjusting one or more characteristics of the mark, including opacity, width, and / or color, in accordance with virtual drawing means being emulated by the stylus, and changes in the characteristics of the input from the stylus.SELECTED DRAWING: Figure 5A

Description

[Technical field]

[0001] This application relates generally to electronic devices with touch-sensitive surfaces, including, but not limited to, electronic devices with touch-sensitive surfaces that operate a user interface with a stylus. [Background technology]

[0002] The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has grown significantly in recent years. Exemplary touch-sensitive surfaces include touchpads and touch screen displays. Such surfaces are widely used to manipulate user interface objects on the display.

[0003] A user interface can be operated with either finger or stylus input. Finger input is more common than stylus input, in part because existing methods of using a stylus are cumbersome and inefficient. Summary of the Invention

[0004] Thus, disclosed herein is an electronic device with a faster, more efficient method for manipulating a user interface with a stylus. Such a method, optionally, complements or replaces existing methods for manipulating a user interface with a stylus. Such a method reduces the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such a method conserves power and increases the time between battery charges.

[0005] The above deficiencies and other problems associated with user interfaces for electronic devices with touch-sensitive surfaces are reduced or eliminated by the disclosed device. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (e.g., a notebook computer, a tablet computer, or a handheld device). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also known as a "touch screen" or "touch screen display"). In some embodiments, the device has a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or sets of instructions stored in the memory for performing a number of functions. In some embodiments, a user interacts with the GUI primarily through stylus and / or finger contacts and gestures on the touch-sensitive surface. In some embodiments, the functionality optionally includes image editing, drawing, presenting, word processing, website creation, disk authoring, spreadsheet creation, game playing, making phone calls, video conferencing, emailing, instant messaging, training support, digital photography, digital videography, web browsing, digital music playback, and / or digital video playback. Executable instructions for performing these functionality are optionally included in a non-transitory computer-readable storage medium or other computer program product configured to be executed by one or more processors. Executable instructions for performing these functionality are optionally included in a transitory computer-readable medium or other computer program product configured to be executed by one or more processors.

[0006] Disclosed herein is an electronic device with an improved method for displaying and updating indicia corresponding to the pose state of a stylus. Such a method optionally complements or replaces conventional methods for displaying indicia. Such a method reduces the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such a method conserves power and increases the time between battery charges.

[0007] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display, the device including one or more sensors that detect signals from a stylus associated with the device, the stylus including a representative portion (e.g., a tip of the stylus). The method includes detecting a pose state of the stylus corresponding to a distance of the stylus relative to the touch-sensitive display, a tilt of the stylus relative to the touch-sensitive display, and / or an orientation of the stylus relative to the touch-sensitive display, determining a location on the touch-sensitive display corresponding to the detected pose state of the stylus, displaying indicia on the touch-sensitive display at the determined location before the stylus touched the touch-sensitive display according to the pose state of the stylus, detecting a change in the distance, tilt, and / or orientation of the stylus before the stylus touched the touch-sensitive display, and updating the displayed indicia on the touch-sensitive display in response to detecting the change.

[0008] According to some embodiments, an electronic device includes a touch-sensitive display unit configured to display a user interface and receive user contacts (including stylus contacts), one or more sensor units configured to detect signals from a stylus associated with the device, and a processing unit coupled to the touch-sensitive display unit and the one or more sensor units, The processing unit is configured to: detect a pose state of the stylus corresponding to a distance of the stylus relative to the touch-sensitive display unit, a tilt of the stylus relative to the touch-sensitive display unit, and / or an orientation of the stylus relative to the touch-sensitive display unit, determine a location on the touch-sensitive display unit corresponding to the detected pose state of the stylus, enable display of indicia on the touch-sensitive display unit at the determined location before the stylus touched the touch-sensitive display unit according to the pose state of the stylus, detect a change in the distance, tilt, and / or orientation of the stylus before the stylus touched the touch-sensitive display unit, and update the displayed indicia on the touch-sensitive display unit in response to detecting the change.

[0009] Thus, electronic devices having a display, a touch-sensitive surface, and optionally one or more sensors for detecting signals from a stylus associated with the device are provided with a faster and more efficient method for displaying and updating indicia corresponding to pose states of the stylus, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such a method can complement or replace conventional methods for displaying indicia.

[0010] Disclosed herein is an electronic device with an improved method for displaying and updating indicia corresponding to the pose state of a stylus while the stylus is in contact with the device (e.g., with the device's touch-sensitive display). Such a method optionally complements or replaces conventional methods for displaying indicia. Such a method reduces the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such a method conserves power and increases the time between battery charges.

[0011] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display, the device including one or more sensors that detect signals from a stylus associated with the device, the stylus including a tip (or other representative portion). The method includes detecting a pose state of the stylus corresponding to a position of the stylus tip on the touch-sensitive display, a tilt of the stylus relative to the touch-sensitive display, and an orientation of the stylus relative to the touch-sensitive display while the tip of the stylus is in contact with the touch-sensitive display, determining a first location on the touch-sensitive display corresponding to the detected pose state of the stylus, displaying indicia on the touch-sensitive display at the first location corresponding to the detected pose state of the stylus that is a first offset from a position of the stylus tip on the touch-sensitive display, detecting a change in stylus orientation, and in response to detecting the change in stylus orientation, determining a second location on the touch-sensitive display corresponding to the change in stylus orientation that is a second offset from the position of the stylus tip on the touch-sensitive display, and displaying indicia at the second location on the touch-sensitive display.

[0012] In accordance with some embodiments, an electronic device includes a touch-sensitive display unit configured to display a user interface and receive user contacts (including stylus contacts), one or more sensor units associated with the device and configured to detect signals from a stylus including a tip, and a processing unit coupled to the touch-sensitive display unit and the one or more sensor units. The processing unit is configured to: detect a stylus attitude state corresponding to a position of the stylus tip on the touch-sensitive display unit, a tilt of the stylus relative to the touch-sensitive display unit, and an orientation of the stylus relative to the touch-sensitive display unit while the stylus tip is in contact with the touch-sensitive display unit; determine a first position on the touch-sensitive display unit corresponding to the detected pose state of the stylus; enable display on the touch-sensitive display unit of a first position indicia corresponding to the detected pose state of the stylus that is at a first offset from the position of the stylus tip on the touch-sensitive display unit; detect a change in stylus orientation; and in response to detecting the change in stylus orientation, determine a second position on the touch-sensitive display unit corresponding to the change in stylus orientation that is at a second offset from the position of the stylus tip on the touch-sensitive display unit; and enable display of the second position indicia on the touch-sensitive display unit.

[0013] Thus, electronic devices with a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the device have a faster and more efficient method for displaying and updating indicia corresponding to the pose state of the stylus while the stylus is in contact with the device (e.g., with the device's touch-sensitive display), thereby increasing the effectiveness, efficiency and user satisfaction of such devices. Such a method can complement or replace conventional methods for displaying indicia.

[0014] Disclosed herein is an electronic device with an improved method for adjusting one or more characteristics of a mark according to characteristics of input from a stylus. Such a method optionally complements or replaces conventional methods for displaying marks. Such a method reduces the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such a method conserves power and increases the time between battery charges.

[0015] According to some embodiments, a method is performed on an electronic device having a touch-sensitive surface and a display, the device including one or more sensors that detect signals from a stylus associated with the device. The method includes detecting input from the stylus, determining a number of characteristics of the input from the stylus including an orientation of the stylus relative to the touch-sensitive surface during input and a direction of movement of the stylus across the touch-sensitive surface during input ("directionality"), generating a mark to be displayed on the display in response to detecting the input from the stylus, the mark having characteristics including opacity, width, and / or color, and adjusting one or more characteristics of the mark in accordance with changes in the number of characteristics of the virtual drawing instrument emulated by the stylus and the input from the stylus during input.

[0016] According to some embodiments, an electronic device includes a display unit configured to display a user interface, a touch-sensitive surface unit configured to receive user contacts (including stylus contacts), one or more sensor units configured to detect signals from a stylus associated with the device, and a processing unit coupled to the display unit, the touch-sensitive surface unit, and the one or more sensor units. The processing unit is configured to detect input from the stylus, determine a number of characteristics of the input from the stylus including an orientation of the stylus relative to the touch-sensitive surface unit during input and a direction of movement of the stylus across the touch-sensitive surface unit during input ("directionality"), generate a mark to be displayed on the display unit having characteristics including opacity, width, and / or color in response to detecting the input from the stylus, and adjust one or more characteristics of the mark in accordance with changes in the virtual drawing instrument emulated by the stylus and the number of characteristics of the input from the stylus during input.

[0017] Thus, electronic devices having a display, a touch-sensitive surface, and one or more sensors that detect signals from a stylus associated with the device have a faster and more efficient method for adjusting one or more characteristics of marks according to characteristics of input from the stylus, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods can complement or replace conventional methods for displaying marks.

[0018] Disclosed herein is an electronic device with an improved method for creating messages with stylus and finger input. Such a method optionally complements or replaces conventional methods for creating messages with stylus and finger input. Such a method reduces the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such a method conserves power and increases the time between battery charges.

[0019] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display, the device including one or more sensors that detect signals from a stylus associated with the device, the method including: displaying a user interface of a communications application on the touch-sensitive display, detecting input corresponding to a request to create a communication, and in response to detecting the input corresponding to the request to create a communication, in accordance with a determination that the input is finger input, displaying a keyboard for keying a text message, and in accordance with a determination that the input is stylus input, displaying a drawing canvas for generating a drawing message.

[0020] According to some embodiments, an electronic device includes a touch-sensitive display unit configured to display a user interface and receive user contacts (including stylus contacts), one or more sensor units configured to detect signals from a stylus associated with the device, and a processing unit coupled to the touch-sensitive display unit and the one or more sensor units. The processing unit is configured to: enable display of a user interface of a communications application on the touch-sensitive display unit, detect input corresponding to a request to create a communication, and in response to detecting the input corresponding to the request to create a communication, in accordance with a determination that the input is finger input, enable display of a keyboard for keying a text message, and in accordance with a determination that the input is stylus input, enable display of a drawing canvas for generating a drawing message.

[0021] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display, the device being associated with a first individual. The method includes displaying on the touch-sensitive display an instant messaging conversation between the first individual and the second individual, the instant messaging conversation between the first individual and the second individual, and a user interface of a communications application including an input area, detecting a selection of a digital image including a plurality of colors, in response to detecting the selection of the digital image, displaying the digital image in the input area, displaying a color palette including colors based on the plurality of colors contained in the digital image for painting on the digital image in the input area, detecting an input selecting a first color in the color palette, and selecting the first color in the color palette when the first color in the color palette is selected. and, during which, detecting one or more drawing inputs on the digital image in the input area, drawing on the digital image in a first color according to the one or more drawing inputs, detecting a request to send the digital image in the state depicted above as an instant message to a second individual in the instant messaging conversation, and in response to detecting the request to send the digital image in the state depicted above as an instant message to the second individual in the instant messaging conversation, and displaying the digital image in the state depicted above as an instant message in the instant messaging conversation on the touch-sensitive display.

[0022] According to some embodiments, an electronic device associated with a first individual includes a touch-sensitive display unit configured to display a user interface and receive user contacts (including stylus contacts), and a processing unit coupled to the touch-sensitive display unit. The processing unit is configured to: enable display of an instant messaging conversation between the first person and the second person and a user interface of a communication application including an input area, displaying on the touch-sensitive display unit a content balloon having instant messages exchanged between the first person and the second person; detect a selection of a digital image including a plurality of colors; in response to detecting the selection of the digital image, enable display of the digital image in the input area, the color palette including a color based on the plurality of colors contained in the digital image, for drawing on the digital image in the input area; detect an input selecting a first color in the color palette; detect one or more drawing inputs on the digital image in the input area while the first color in the color palette is selected; draw on the digital image with the first color in accordance with the one or more drawing inputs; detect a request to send the digital image in the state depicted above as an instant message to the second person in the instant messaging conversation; in response to detecting the request to send the digital image in the state depicted above as an instant message to the second person in the instant messaging conversation; and enable display of the digital image in the state depicted above as an instant message in the instant messaging conversation on the touch-sensitive display.

[0023] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display, the device being associated with a first person, the method including: displaying on the touch-sensitive display a user interface of a communication application including an instant messaging conversation between the first person and the second person, displaying a content balloon with instant messages exchanged between the first person and the second person, and a typing area; receiving a signal indicating that the second person is composing an instant message; in response to receiving the signal indicating that the second person is composing an instant message, displaying a keyed message composition indicator within the user interface of the communication application in accordance with a determination that the second person is keying an instant message; displaying a drawing message composition indicator separate from the keyed message composition indicator within the user interface of the communication application in accordance with a determination that the second person is drawing an instant message; receiving an instant message created by the second person; in response to receiving an instant message created by the second person that corresponds to the displayed message composition indicator, ceasing to display the keyed message composition indicator or the drawing message composition indicator; and displaying a content balloon including the instant message created by the second person within the instant messaging conversation.

[0024] According to some embodiments, an electronic device associated with a first individual includes a touch-sensitive display unit configured to display a user interface and receive user contacts (including stylus contacts), and a processing unit coupled to the touch-sensitive display unit. The processing unit is configured to: display on the touch-sensitive display unit a content balloon having instant messages exchanged between the first person and the second person; enable display of a user interface of the communications application including an instant messaging conversation between the first person and the second person and an input area; receive a signal indicating that the second person is composing an instant message; in response to receiving the signal indicating that the second person is composing an instant message, enable display of a keyed message composition indicator within the user interface of the communications application in accordance with a determination that the second person is keying an instant message; enable display of a drawing message composition indicator separate from the keyed message composition indicator within the user interface of the communications application in accordance with a determination that the second person is drawing an instant message; receive an instant message created by the second person; and in response to receiving an instant message created by the second person that corresponds to the displayed message composition indicator, cease enabling display of the keyed message composition indicator or the drawing message composition indicator; and enable display of a content balloon including an instant message created by the second person within the instant messaging conversation.

[0025] Thus, electronic devices with a touch-sensitive display, and optionally one or more sensors for detecting signals from a stylus associated with the device, provide faster and more efficient methods for composing messages with stylus and finger input, thereby increasing the effectiveness, efficiency and user satisfaction with such devices. Such methods can complement or replace conventional methods for composing messages with stylus and finger input.

[0026] An electronic device is disclosed with an improved method for accessing a drawing application of a locked device. Such a method optionally complements or replaces conventional methods for accessing a drawing application of a locked device. Such a method reduces the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such a method conserves power and increases the time between battery charges.

[0027] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display, the device including one or more sensors that detect signals from a stylus associated with the device, the method including: displaying a lock screen user interface on the touch-sensitive display while the electronic device is in a locked state, detecting a first input from a stylus drawing on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display, displaying, on the lock screen user interface, one or more marks of a drawing corresponding to the first input from the stylus, detecting a second input from the stylus displaying a drawing application in a restricted mode while displaying, on the lock screen user interface, one or more marks of a drawing corresponding to the first input from the stylus, and running the drawing application in the restricted mode to display one or more marks of a drawing in the drawing application, in response to detecting the second input from the stylus displaying the drawing application.

[0028] According to some embodiments, an electronic device includes a touch-sensitive display unit configured to display a user interface and receive user contacts (including stylus contacts), one or more sensor units configured to detect signals from a stylus associated with the device, and a processing unit coupled to the touch-sensitive display unit and the one or more sensor units, The processing unit is configured to: enable display of a lock screen user interface on the touch-sensitive display unit while the electronic device is in a locked state, detect a first input from a stylus drawing on the touch-sensitive display unit while enabling display of the lock screen user interface on the touch-sensitive display unit, and in response to detecting the first input from the drawing stylus, enable display of one or more marks of a drawing corresponding to the first input from the stylus on the lock screen user interface, detect a second input from the stylus displaying a drawing application in a restricted mode while enabling display of the one or more marks of a drawing corresponding to the first input from the stylus on the lock screen user interface, and in response to detecting the second input from the stylus displaying the drawing application, run the drawing application in the restricted mode and enable display of the one or more marks in the drawing application.

[0029] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display, the device including one or more sensors that detect signals from a stylus associated with the device. The method includes displaying a lock screen user interface on the touch-sensitive display while the electronic device is in a locked state, detecting user input while displaying the lock screen user interface on the touch-sensitive display, replacing the display of the lock screen user interface with a display of a drawing application in a restricted session pursuant to a determination that the user input is a default input with a stylus, and maintaining the display of the lock screen user interface and ceasing the display of the drawing application pursuant to a determination that the user input is a finger touch input.

[0030] According to some embodiments, an electronic device includes a touch-sensitive display unit configured to display a user interface and receive user contacts (including stylus contacts), one or more sensor units configured to detect signals from a stylus associated with the device, and a processing unit coupled to the touch-sensitive display unit and the one or more sensor units. The processing unit is configured to: enable display of a lock screen user interface on the touch-sensitive display unit while the electronic device is in a locked state, detect user input while enabling display of the lock screen user interface on the touch-sensitive display unit, and pursuant to a determination that the user input is a default input with a stylus, replace the display of the lock screen user interface with a display of a drawing application in a restricted session, and pursuant to a determination that the user input is a finger touch input, maintain the display of the lock screen user interface and discontinue the display of the drawing application.

[0031] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display, the device including one or more sensors that detect signals from a stylus associated with the device. The method includes displaying a lock screen user interface on the touch-sensitive display while the electronic device is in a locked state, detecting a first input by the stylus on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display, including detecting an initial contact by the stylus on the touch-sensitive display at the initiation of the first input, detecting movement of the stylus across the touch-sensitive display during the first input, detecting lift-off of the stylus from the touch-sensitive display at the termination of the first input, and in response to detecting movement of the stylus across the touch-sensitive display, drawing a first mark that is displayed on the lock screen image according to movement of the stylus across the touch-sensitive display, and detecting a first indicia that is displayed on the lock screen image while the electronic device is in a locked state. in response to detecting lift-off of the stylus from the ray, ceasing to display the lock screen user interface, including ceasing to display the lock screen image, displaying a user interface for a drawing application in the restricted session, and displaying in the user interface for the drawing application in the restricted session a first mark drawn in accordance with movement of the stylus across the touch-sensitive display during the first input; detecting a second input by the stylus on the touch-sensitive display while displaying the user interface for the drawing application in the restricted session, and in response to detecting the second input by the stylus on the touch-sensitive display, drawing a second mark that is displayed with the first mark in the user interface for the drawing application in the restricted session.

[0032] According to some embodiments, an electronic device includes a touch-sensitive display unit configured to display a user interface and receive user contacts (including stylus contacts), one or more sensor units configured to detect signals from a stylus associated with the device, and a processing unit coupled to the touch-sensitive display unit and the one or more sensor units, the processing unit is configured to: enable display of a lock screen user interface including a lock screen image on the touch-sensitive display unit while the electronic device is in a locked state, detect a first input by the stylus on the touch-sensitive display unit while enabling display of the lock screen user interface on the touch-sensitive display unit, including detecting an initial contact by the stylus on the touch-sensitive display unit at a start of a first input, detect movement of the stylus across the touch-sensitive display unit during the first input, detect lift-off of the stylus from the touch-sensitive display unit at an end of the first input, and in response to detecting movement of the stylus across the touch-sensitive display unit, draw a first mark in accordance with movement of the stylus across the touch-sensitive display unit for display on the lock screen image, and display a first mark on the touch-sensitive display unit in response to detecting movement of the stylus across the touch-sensitive display unit. in response to detecting lift-off of the stylus from the touch-sensitive display unit, cease enabling display of the lock screen user interface, including ceasing to display the lock screen image, enable display of a user interface for a drawing application in the restricted session, enable display of a first mark in the user interface for the drawing application in the restricted session that is drawn in accordance with movement of the stylus across the touch-sensitive display unit during the first input, detect a second input by the stylus on the touch-sensitive display unit while enabling display of the user interface for the drawing application in the restricted session, and in response to detecting the second input by the stylus on the touch-sensitive display unit, draw a second mark that is displayed together with the first mark in the user interface for the drawing application in the restricted session.

[0033] Thus, an electronic device with a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the device provides a faster and more efficient method for accessing drawing applications on a locked device, thereby increasing the effectiveness, efficiency and user satisfaction with such devices. Such a method can complement or replace conventional methods for accessing drawing applications on a locked device.

[0034] Disclosed herein is an electronic device with a faster, more efficient method for selecting and using a virtual drawing instrument using a stylus. Such a method optionally complements or replaces conventional methods for emulating a virtual drawing instrument using a stylus. Such a method reduces the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such a method conserves power and increases the time between battery charges.

[0035] According to some embodiments, the method is performed on an electronic device that includes a display, a touch-sensitive surface, and one or more sensors that detect signals from a stylus associated with the device. The method includes displaying an electronic document on a display; detecting a first input from the stylus while displaying the electronic document on the display, including detecting an initial contact by the stylus on the touch-sensitive surface; determining a plurality of characteristics of the first input from the stylus including a tilt of the stylus which is an angle with respect to a normal to a surface of the touch-sensitive surface; selecting a first virtual drawing vehicle for the stylus to emulate in accordance with a determination that the tilt satisfies one or more selection criteria for the first virtual drawing vehicle; selecting a second virtual drawing vehicle, distinct from the first virtual drawing vehicle, for the stylus to emulate in accordance with a determination that the tilt satisfies one or more selection criteria for the second virtual drawing vehicle; and after selecting one of the first virtual drawing vehicle and the second virtual drawing vehicle for the stylus to emulate, generating a mark in the electronic document with the selected virtual drawing vehicle in response to detecting the first input from the stylus.

[0036] In accordance with some embodiments, an electronic device includes a display unit configured to display a user interface, a touch-sensitive surface unit configured to receive user contacts (including stylus contacts), one or more sensor units configured to detect signals from a stylus associated with the device, and a processing unit coupled to the display unit, the touch-sensitive surface unit, and the one or more sensor units. The processing unit is configured to: enable display of the electronic document on the display unit; detect a first input from the stylus while enabling display of the electronic document on the display unit, including detecting an initial contact by the stylus on the touch-sensitive surface unit; determine a plurality of characteristics of the first input from the stylus including a tilt of the stylus which is an angle with respect to a normal to a surface of the touch-sensitive surface unit; select a first virtual drawing means for the stylus to emulate in accordance with a determination that the tilt satisfies one or more selection criteria for the first virtual drawing means; select a second virtual drawing means, distinct from the first virtual drawing means, for the stylus to emulate in accordance with a determination that the tilt satisfies one or more selection criteria for the second virtual drawing means; and after selecting one of the first virtual drawing means and the second virtual drawing means for the stylus to emulate, generate a mark in the electronic document with the selected virtual drawing means in response to detecting the first input from the stylus.

[0037] Thus, an electronic device comprising a display, a touch-sensitive surface, one or more sensors that detect signals from a stylus associated with the device, and optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface, provides a faster and more efficient method for emulating a virtual drawing implement using a stylus, thereby increasing the effectiveness, efficiency and user satisfaction with such a device. Such a method can complement or replace conventional methods for emulating a virtual drawing implement using a stylus.

[0038] Disclosed herein is an electronic device with an improved method for creating events in a calendar using handwriting input. Such a method optionally complements or replaces conventional methods for creating calendar events. Such a method reduces the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such a method conserves power and increases the time between battery charges.

[0039] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display. The method includes displaying an electronic calendar on the touch-sensitive display, the displayed calendar including an area corresponding to a first day. The method further includes detecting handwritten input on the touch-sensitive display, the handwritten input extending over less than all of a vertical portion of the first day in the displayed calendar. The method further includes, in response to detecting the handwritten input, displaying a graphic corresponding to the handwritten input over the vertical portion of the first day. The method further includes detecting an input corresponding to a request to create an event in the electronic calendar while displaying the graphic corresponding to the handwritten input over the vertical portion of the first day, and, in response to detecting the input corresponding to the request to create the event in the electronic calendar, creating an event in the first day in the electronic calendar having a start time and an end time, the start time and end time of the event being in accordance with the vertical portion of the first day.

[0040] According to some embodiments, an electronic device includes a touch-sensitive display unit and a processing unit coupled to the touch-sensitive display unit. The touch-sensitive display unit is configured to display an electronic calendar on the touch-sensitive display unit, the displayed calendar including an area corresponding to a first day. The touch-sensitive display unit is further configured to detect handwritten input on the touch-sensitive display unit, the handwritten input extending over less than all of a vertical portion of the first day in the displayed calendar. The touch-sensitive display unit is further configured to, in response to detecting the handwritten input, display a graphic corresponding to the handwritten input over the vertical portion of the first day, and detect an input corresponding to a request to create an event in the electronic calendar while displaying the graphic corresponding to the handwritten input over the vertical portion of the first day. The processing unit is configured to, in response to detecting the input corresponding to the request to create an event in the electronic calendar, create an event in the first day in the electronic calendar having a start time and an end time, the start time and end time of the event being in accordance with the vertical portion of the first day.

[0041] Thus, electronic devices with a touch-sensitive display, and optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface, provide a faster and more efficient method for creating calendar events, thereby increasing the effectiveness, efficiency and user satisfaction with such devices. Such a method can complement or replace conventional methods for creating calendar events.

[0042] An electronic device is disclosed with an improved method for selecting a portion of a video. Such a method optionally complements or replaces conventional methods for selecting a portion of a video. Such a method reduces the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such a method conserves power and increases the time between battery charges.

[0043] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display. The method includes displaying a video editing application including a timeline area for a video being edited. The method further includes detecting a gesture by contact on the touch-sensitive display. The gesture includes an initial contact of the contact with the timeline area occurring at a first time location within the timeline area, movement of the contact across the touch-sensitive display after the initial contact with the timeline area, and lift-off of the contact at a location on the touch-sensitive display corresponding to a second time location within the timeline area. The method additionally includes selecting a portion of the video being edited in response to detecting the gesture by contact on the touch-sensitive display. The selected portion of the video begins at the first time location and ends at the second time location.

[0044] According to some embodiments, an electronic device includes a touch-sensitive display unit and a processing unit coupled to the touch-sensitive display unit. The touch-sensitive display unit is configured to display a video application, the displayed video application including a timeline area for a video being edited. The processing unit is configured to detect a contact gesture on the touch-sensitive display unit. The gesture includes an initial contact of the contact with the timeline area occurring at a first time location within the timeline area, a movement of the contact across the touch-sensitive display unit after the initial contact with the timeline area, and a lift-off of the contact at a location on the touch-sensitive display unit corresponding to a second time location within the timeline area. The processing unit is also configured to select a portion of the video being edited in response to detecting the contact gesture on the touch-sensitive display unit, the selected portion of the video beginning at the first time location and ending at the second time location.

[0045] Thus, electronic devices with a display, a touch-sensitive surface, and optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface, are provided with faster and more efficient methods for selecting portions of a video, thereby increasing the effectiveness, efficiency, and user satisfaction with such devices. Such methods can complement or replace conventional methods for selecting portions of a video.

[0046] Disclosed herein is an electronic device with an improved method for displaying and using menus with a stylus. Such a method optionally complements or replaces conventional methods for displaying and using menus. Such a method reduces the number, range, and / or type of inputs from a user, creating a more efficient human-machine interface. For battery-operated devices, such a method conserves power and increases the time between battery charges.

[0047] According to some embodiments, a method is performed on an electronic device with a touch-sensitive display, the device including one or more sensors that detect signals from a stylus associated with the device, the method including: displaying a user interface on the touch-sensitive display, detecting movement of the stylus toward the touch-sensitive display without contacting the touch-sensitive display while displaying the user interface on the touch-sensitive display, determining whether the detected movement of the stylus toward the touch-sensitive display without contacting the touch-sensitive display satisfies one or more stylus movement criteria, in accordance with a determination that the detected movement of the stylus satisfies the one or more stylus movement criteria, overlaying a menu on the user interface, the menu including a plurality of selectable menu options, detecting a selection of a first menu option within the plurality of selectable menu options, and in response to detecting a selection of the first menu option within the plurality of selectable menu options, performing an operation corresponding to the first menu option to cease displaying the menu.

[0048] According to some embodiments, an electronic device includes a touch-sensitive display unit configured to display a user interface and receive user contacts (including stylus contacts), one or more sensor units configured to detect signals from a stylus associated with the device, and a processing unit coupled to the touch-sensitive display unit and the one or more sensor units, wherein the processing unit is configured to: enable display of the user interface on the touch-sensitive display unit, detect movement of the stylus towards the touch-sensitive display unit without contacting the touch-sensitive display unit while enabling display of the user interface on the touch-sensitive display unit, determine whether the detected movement of the stylus towards the touch-sensitive display unit without contacting the touch-sensitive display unit satisfies one or more stylus movement criteria, enable display overlaid on the user interface of a menu including a plurality of selectable menu options in accordance with a determination that the detected movement of the stylus satisfies the one or more stylus movement criteria, detect a selection of a first menu option within the plurality of selectable menu options, and in response to detecting selection of the first menu option within the plurality of selectable menu options, perform an operation corresponding to the first menu option to cease displaying the menu.

[0049] Thus, electronic devices with touch-sensitive displays and one or more sensors that detect signals from a stylus associated with the device are provided with a faster and more efficient method for displaying and using menus with a stylus, thereby increasing the effectiveness, efficiency and user satisfaction of such devices. Such a method can complement or replace conventional methods for displaying menus.

[0050] According to some embodiments, an electronic device includes a display, a touch-sensitive surface, optionally one or more sensors that detect an intensity of contact with the touch-sensitive surface, one or more processors, a memory, and one or more programs, the one or more programs being stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions to perform or cause to be performed any of the operations of the methods described herein. According to some embodiments, a computer-readable storage medium has instructions stored therein that, when executed by an electronic device including a display, a touch-sensitive surface, and optionally one or more sensors that detect an intensity of contact with the touch-sensitive surface, cause the device to perform or cause to be performed any of the operations of the methods described herein. According to some embodiments, a graphical user interface on an electronic device including a display, a touch-sensitive surface, optionally one or more sensors that detect an intensity of contact with the touch-sensitive surface, a memory, and one or more processors that execute the one or more programs stored in the memory includes one or more of the elements displayed in any of the methods described above, which elements are updated in response to an input as described in any of the methods described herein. According to some embodiments, an electronic device includes a display, a touch-sensitive surface, optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface, and means for performing or causing to be performed any of the operations of the methods described herein. According to some embodiments, an information processing apparatus for use in an electronic device comprising a display, a touch-sensitive surface, and optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface includes means for performing or causing to be performed any of the operations of the methods described herein.

[0051] Thus, electronic devices with a display, a touch-sensitive surface, and optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface, provide faster and more efficient methods for navigating a user interface with a stylus, thereby increasing the effectiveness, efficiency, and user satisfaction of such devices. Such methods can complement or replace conventional methods for navigating a user interface with a stylus. [Brief description of the drawings]

[0052] For a better understanding of the various described embodiments, please refer to the following Detailed Description in conjunction with the following drawings, in which like reference numerals refer to corresponding parts throughout the drawings, in which:

[0053] [Figure 1A] FIG. 1 is a block diagram illustrating a portable multifunction device with a touch-sensitive display in accordance with some embodiments.

[0054] [Figure 1B] FIG. 2 is a block diagram illustrating example components for event processing according to some embodiments.

[0055] [Diagram 2] 1 illustrates a portable multifunction device with a touch screen in accordance with some embodiments.

[0056] [Diagram 3] 1 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface in accordance with some embodiments.

[0057] [Figure 4] FIG. 1 is a block diagram of an exemplary electronic stylus according to some embodiments.

[0058] [Figure 5A]4 illustrates pose states of a stylus relative to a touch-sensitive surface in accordance with some embodiments. [Figure 5B] 4 illustrates pose states of a stylus relative to a touch-sensitive surface in accordance with some embodiments.

[0059] [Figure 6A] 1 illustrates an exemplary user interface for a menu of applications on a portable multifunction device in accordance with some embodiments.

[0060] [Figure 6B] 1 illustrates an exemplary user interface for a multifunction device with a touch-sensitive surface that is separate from a display in accordance with some embodiments.

[0061] [Figure 7A] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 7B] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 7C] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 7D] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 7E] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 7F] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 7G] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 7H] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 7I] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 7J] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments.

[0062] [Figure 8A] 4 is a flow diagram illustrating a method for displaying and updating indicia corresponding to a stylus pose state according to some embodiments. [Figure 8B] 4 is a flow diagram illustrating a method for displaying and updating indicia corresponding to a stylus pose state according to some embodiments. [Figure 8C] 4 is a flow diagram illustrating a method for displaying and updating indicia corresponding to a stylus pose state according to some embodiments. [Figure 8D] 4 is a flow diagram illustrating a method for displaying and updating indicia corresponding to a stylus pose state according to some embodiments.

[0063] [Figure 9] FIG. 2 is a functional block diagram of an electronic device according to some embodiments.

[0064] [Figure 10A] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 10B] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 10C] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 10D] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 10E] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 10F] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 10G] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 10H] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 10I] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 10J] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments. [Figure 10K] 1 illustrates an example user interface for displaying and updating indicia corresponding to stylus pose states according to some embodiments.

[0065] [Figure 11A] 4 is a flow diagram illustrating a method for displaying and updating indicia corresponding to a stylus pose state according to some embodiments. [Figure 11B] 4 is a flow diagram illustrating a method for displaying and updating indicia corresponding to a stylus pose state according to some embodiments. [Figure 11C] 4 is a flow diagram illustrating a method for displaying and updating indicia corresponding to a stylus pose state according to some embodiments. [Figure 11D]4 is a flow diagram illustrating a method for displaying and updating indicia corresponding to a stylus pose state according to some embodiments.

[0066] [Figure 12] FIG. 2 is a functional block diagram of an electronic device according to some embodiments.

[0067] [Figure 13A] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13B] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13C] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13D] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13E] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13F] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13G] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13H] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13I]1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13J] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13K] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13L] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13M] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13N] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 13O] 1 illustrates an exemplary user interface for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments.

[0068] [Figure 14A] 1 is a flow diagram illustrating a method for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 14B] 1 is a flow diagram illustrating a method for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 14C] 1 is a flow diagram illustrating a method for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 14D] 1 is a flow diagram illustrating a method for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments. [Figure 14E] 1 is a flow diagram illustrating a method for adjusting one or more characteristics of a mark according to characteristics of an input from a stylus, according to some embodiments.

[0069] [Figure 15] FIG. 2 is a functional block diagram of an electronic device according to some embodiments.

[0070] [Figure 16A] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16B] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16C] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16D] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16E] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16F] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16G] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16H] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16I] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16J]1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16K] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16L] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16M] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments. [Figure 16N] 1 illustrates an exemplary user interface for composing a message with stylus and finger input according to some embodiments.

[0071] [Figure 17A] 4 is a flow diagram illustrating a method for creating a message with stylus and finger input according to some embodiments. [Figure 17B] 4 is a flow diagram illustrating a method for creating a message with stylus and finger input according to some embodiments. [Figure 17C] 4 is a flow diagram illustrating a method for creating a message with stylus and finger input according to some embodiments. [Figure 18A] 4 is a flow diagram illustrating a method for creating a message with stylus and finger input according to some embodiments. [Figure 18B] 4 is a flow diagram illustrating a method for creating a message with stylus and finger input according to some embodiments. [Figure 19] 4 is a flow diagram illustrating a method for creating a message with stylus and finger input according to some embodiments.

[0072] [Figure 20A] FIG. 2 is a functional block diagram of an electronic device according to some embodiments.

[0073] [Figure 20B] FIG. 2 is a functional block diagram of an electronic device according to some embodiments.

[0074] [Figure 21A] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21B] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21C] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21D] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21E] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21F] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21G] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Fig. 21H] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21I] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21J] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21K]1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21L] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21M] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21N] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21O] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21P] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments. [Figure 21Q] 1 illustrates an example user interface for accessing a drawing application on a locked device according to some embodiments.

[0075] [Figure 22A] 4 is a flow diagram illustrating a method for accessing a drawing application on a locked device according to some embodiments. [Figure 22B] 4 is a flow diagram illustrating a method for accessing a drawing application on a locked device according to some embodiments.

[0076] [Figure 23A] 4 is a flow diagram illustrating a method for accessing a drawing application on a locked device according to some embodiments. [Figure 23B] 4 is a flow diagram illustrating a method for accessing a drawing application on a locked device according to some embodiments.

[0077] [Figure 24A] 4 is a flow diagram illustrating a method for accessing a drawing application on a locked device according to some embodiments. [Figure 24B] 4 is a flow diagram illustrating a method for accessing a drawing application on a locked device according to some embodiments.

[0078] [Diagram 25] FIG. 2 is a functional block diagram of an electronic device according to some embodiments.

[0079] [Figure 26A] 1 illustrates an exemplary user interface for selecting and using a virtual drawing instrument using a stylus according to some embodiments. [Figure 26B] 1 illustrates an exemplary user interface for selecting and using a virtual drawing instrument using a stylus according to some embodiments. [Figure 26C] 1 illustrates an exemplary user interface for selecting and using a virtual drawing instrument using a stylus according to some embodiments. [Figure 26D] 1 illustrates an exemplary user interface for selecting and using a virtual drawing instrument using a stylus according to some embodiments. [Figure 26E] 1 illustrates an exemplary user interface for selecting and using a virtual drawing instrument using a stylus according to some embodiments. [Figure 26F] 1 illustrates an exemplary user interface for selecting and using a virtual drawing instrument using a stylus according to some embodiments. [Figure 26G] 1 illustrates an exemplary user interface for selecting and using a virtual drawing instrument using a stylus according to some embodiments. [Fig. 26H] 1 illustrates an exemplary user interface for selecting and using a virtual drawing instrument using a stylus according to some embodiments.

[0080] [Figure 27A] 4 is a flow diagram illustrating a method for selecting and using a virtual drawing instrument using a stylus according to some embodiments. [Figure 27B] 4 is a flow diagram illustrating a method for selecting and using a virtual drawing instrument using a stylus according to some embodiments. [Figure 27C] 4 is a flow diagram illustrating a method for selecting and using a virtual drawing instrument using a stylus according to some embodiments.

[0081] [Figure 28] FIG. 2 is a functional block diagram of an electronic device according to some embodiments.

[0082] [Figure 29A] 1 illustrates an exemplary user interface for creating a calendar event according to some embodiments. [Figure 29B] 1 illustrates an exemplary user interface for creating a calendar event according to some embodiments. [Figure 29C] 1 illustrates an exemplary user interface for creating a calendar event according to some embodiments. [Figure 29D] 1 illustrates an exemplary user interface for creating a calendar event according to some embodiments. [Figure 29E] 1 illustrates an exemplary user interface for creating a calendar event according to some embodiments. [Figure 29F] 1 illustrates an exemplary user interface for creating a calendar event according to some embodiments. [Figure 29G] 1 illustrates an exemplary user interface for creating a calendar event according to some embodiments. [Fig. 29H] 1 illustrates an exemplary user interface for creating a calendar event according to some embodiments.

[0083] [Figure 30A] 1 is a flow diagram illustrating a method of calendar event creation according to some embodiments. [Figure 30B] 1 is a flow diagram illustrating a method of calendar event creation according to some embodiments. [Figure 30C] 1 is a flow diagram illustrating a method of calendar event creation according to some embodiments. [Figure 30D] 1 is a flow diagram illustrating a method of calendar event creation according to some embodiments.

[0084] [Diagram 31] FIG. 2 is a functional block diagram of an electronic device according to some embodiments.

[0085] [Figure 32A] 1 illustrates an exemplary user interface for selecting a portion of a video according to some embodiments. [Figure 32B] 1 illustrates an exemplary user interface for selecting a portion of a video according to some embodiments. [Figure 32C] 1 illustrates an exemplary user interface for selecting a portion of a video according to some embodiments. [Fig. 32D] 1 illustrates an exemplary user interface for selecting a portion of a video according to some embodiments. [Figure 32E] 1 illustrates an exemplary user interface for selecting a portion of a video according to some embodiments. [Fig. 32F] 1 illustrates an exemplary user interface for selecting a portion of a video according to some embodiments.

[0086] [Figure 33A] 4 is a flow diagram illustrating a method for selecting a portion of a video according to some embodiments. [Figure 33B] 4 is a flow diagram illustrating a method for selecting a portion of a video according to some embodiments.

[0087] [Diagram 34] FIG. 2 is a functional block diagram of an electronic device according to some embodiments.

[0088] [Figure 35A] 1 illustrates an exemplary user interface for displaying and using menus with a stylus according to some embodiments. [Figure 35B] 1 illustrates an exemplary user interface for displaying and using menus with a stylus according to some embodiments. [Figure 35C] 1 illustrates an exemplary user interface for displaying and using menus with a stylus according to some embodiments. [Figure 35D] 1 illustrates an exemplary user interface for displaying and using menus with a stylus according to some embodiments. [Figure 35E] 1 illustrates an exemplary user interface for displaying and using menus with a stylus according to some embodiments. [Fig. 35F] 1 illustrates an exemplary user interface for displaying and using menus with a stylus according to some embodiments. [Figure 35G] 1 illustrates an exemplary user interface for displaying and using menus with a stylus according to some embodiments. [Fig. 35H] 1 illustrates an exemplary user interface for displaying and using menus with a stylus according to some embodiments. [Fig. 35I] 1 illustrates an exemplary user interface for displaying and using menus with a stylus according to some embodiments. [Fig. 35J] 1 illustrates an exemplary user interface for displaying and using menus with a stylus according to some embodiments.

[0089] [Figure 36A]4 is a flow diagram illustrating a method for displaying and using a menu with a stylus according to some embodiments. [Figure 36B] 4 is a flow diagram illustrating a method for displaying and using a menu with a stylus according to some embodiments. [Figure 36C] 4 is a flow diagram illustrating a method for displaying and using a menu with a stylus according to some embodiments.

[0090] [Figure 37] FIG. 2 is a functional block diagram of an electronic device according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0091] A number of different techniques for manipulating a user interface with a stylus are described herein. Using one or more of these techniques (optionally in conjunction with one another) reduces the number, range, and / or type of inputs from a user to provide a more efficient human-machine interface. This allows a user to use a stylus with a device having a touch-sensitive surface faster and more efficiently. For battery-operated devices, these improvements conserve power and increase the time between charging the battery. For ease of explanation, devices and methods including illustrative examples of some of these techniques are described below. ● Figures 7A-7J show example user interfaces for displaying and updating indicia corresponding to a pose state of the stylus before the stylus touches the touch-sensitive surface. Figures 8A-8D are flow diagrams illustrating a method for displaying and updating indicia corresponding to a pose state of the stylus before the stylus touches the touch-sensitive surface. The user interfaces of Figures 7A-7J are used to illustrate the processes of Figures 8A-8D. ● Figures 10A-10K show exemplary user interfaces for displaying and updating indicia corresponding to a pose state of a stylus while the stylus is touching a touch-sensitive surface. Figures 11A-11D are flow diagrams illustrating a method for displaying and updating indicia corresponding to a pose state of a stylus while the stylus is touching a touch-sensitive surface. The user interfaces of Figures 10A-10K are used to illustrate the processes of Figures 11A-11D. Figures 13A-13O show exemplary user interfaces for adjusting one or more characteristics of a mark according to characteristics of input from a stylus. Figures 14A-14E are flow diagrams illustrating a method for adjusting one or more characteristics of a mark according to characteristics of input from a stylus. The user interfaces of Figures 13A-13O are used to illustrate the processes of Figures 14A-14E. Figures 16A-16N show exemplary user interfaces for creating a message with stylus and finger input. Figures 17A-17C, 18A-18B, and 19 are flow diagrams illustrating a method for creating a message with stylus and finger input. The user interfaces of Figures 16A-16N are used to illustrate the processes of Figures 17A-17C, 18A-18B, and 19. 21A-21Q show example user interfaces for accessing a drawing application on a locked device. Figures 22A-22B, 23A-23B, and 24A-24B are flow diagrams illustrating a method for accessing a drawing application on a locked device. The user interfaces of Figures 21A-21Q are used to illustrate the processes of Figures 22A-22B, 23A-23B, and 24A-24B. Figures 26A-26H show exemplary user interfaces for selecting and using a virtual drawing instrument using a stylus. Figures 27A-27C are flow diagrams illustrating a method for selecting and using a virtual drawing instrument using a stylus. The user interfaces of Figures 26A-26H are used to illustrate the processes of Figures 27A-27C. - Figures 29A-29H show an exemplary user interface for creating a calendar event. Figures 30A-30D are a flow diagram illustrating a method of creating a calendar event. The user interfaces of Figures 29A-29H are used to illustrate the process of Figures 30A-30D. ● Figures 32A-32F show exemplary user interfaces for selecting a portion of a video. Figures 33A-33B are a flow diagram illustrating a method for selecting a portion of a video. The user interfaces of Figures 32A-32F are used to illustrate the process of Figures 33A-33B. Figures 35A-35J show exemplary user interfaces for displaying and using menus with a stylus. Figures 36A-36C are flow diagrams illustrating methods for displaying and using menus with a stylus. The user interfaces of Figures 35A-35J are used to illustrate the processes of Figures 36A-36C. Exemplary Devices

[0092] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments being described. However, it will be apparent to those skilled in the art that the various embodiments being described may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0093] In this specification, terms such as first, second, etc. are used in some examples to describe various elements, but it will be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first contact can be referred to as a second contact, and similarly, a second contact can be referred to as a first contact, without departing from the scope of the various embodiments described. Although a first contact and a second contact are both contacts, they are not the same contact unless the context clearly indicates otherwise.

[0094] The terminology used in the description of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be limiting. When used in the description of the various embodiments described and in the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It is also to be understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "includes", "including", "comprises" and / or "comprising" as used herein specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0095] As used herein, the term "if" is, optionally, interpreted to mean "when," "upon," "in response to determining," or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (a stated condition or event) is detected" are, optionally, interpreted to mean "upon determining," or "in response to determining," or "upon detecting (the stated condition or event)," or "in response to detecting (the stated condition or event)," depending on the context.

[0096] Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communication device, such as a mobile telephone, that also includes other functions, such as PDA and / or music player functions. Exemplary embodiments of portable multifunction devices include, without limitation, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. (Cupertino, California). Other portable electronic devices, such as laptops or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads), are also optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but rather a desktop computer with a touch-sensitive surface (e.g., touchscreen displays and / or touchpads).

[0097] In the following description, an electronic device is described that includes a display and a touch-sensitive surface, however, it should be understood that the electronic device optionally includes one or more other physical user-interface devices, such as a physical keyboard, a mouse, and / or a joystick.

[0098] The device typically supports a variety of applications, such as one or more of drawing applications, presentation applications, word processing applications, website creation applications, disc authoring applications, spreadsheet applications, gaming applications, telephony applications, video conferencing applications, email applications, instant messaging applications, training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications, and / or digital video player applications.

[0099] Various applications running on the device optionally use at least one common physical user-interface device, such as a touch-sensitive surface. One or more features of the touch-sensitive surface and corresponding information displayed on the device are optionally adjusted and / or changed for each application and / or within the corresponding application. In this way, the common physical architecture (such as the touch-sensitive surface) of the device optionally supports a variety of applications with user interfaces that are intuitive and transparent to the user.

[0100] Attention is now directed to embodiments of portable devices with touch-sensitive displays. FIG. 1A is a block diagram illustrating a portable multifunction device 100 with a touch-sensitive display system 112, according to some embodiments. Touch-sensitive display system 112 may be conveniently referred to as a "touch screen" or simply a touch-sensitive display. Device 100 includes memory 102 (optionally including one or more computer-readable storage media), memory controller 120, one or more processing units (CPUs) 122, peripherals interface 118, RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, input / output (I / O) subsystem 106, other input or control devices 116, and external port 124. Device 100 optionally includes one or more light sensors 164. Device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of a contact on device 100 (e.g., a touch-sensitive surface such as touch-sensitive display system 112 of device 100). Device 100 optionally includes one or more tactile output generators 163 for generating tactile output on device 100 (e.g., generating tactile output on a touch-sensitive surface, such as touch-sensitive display system 112 of device 100 or touchpad 355 of device 300). These components optionally communicate via one or more communication buses or signal lines 103.

[0101] As used herein and in the claims, the term "tactile output" refers to a physical displacement of a device relative to a previous position of the device, a physical displacement of a component of the device (e.g., a touch-sensitive surface) relative to another component of the device (e.g., a housing), or a displacement of a component relative to the center of mass of the device, that is to be detected by a user with the user's sense of touch. For example, in a situation where a device or a component of the device is in contact with a touch-sensitive surface of a user (e.g., the fingers, palm, or other part of the user's hand), the tactile output generated by the physical displacement is interpreted by the user as a tactile sensation that corresponds to a perceived change in a physical property of the device or a component of the device. For example, movement of a touch-sensitive surface (e.g., a touch-sensitive display or trackpad) is optionally interpreted by the user as a "down-click" or "up-click" of a physical actuator button. In some cases, a user will feel a tactile sensation such as a "down-click" or "up-click" even when there is no movement of a physical actuator button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's action. As another example, movement of the touch-sensitive surface is optionally interpreted or sensed by a user as "roughness" of the touch-sensitive surface, even when there is no change in the smoothness of the touch-sensitive surface. Although such user interpretation of touch will depend on the user's personal sensory perception, there are many sensory perceptions of touch that are common to a majority of users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., an "up click," a "down click," "roughness"), unless otherwise noted, the generated tactile output corresponds to a physical displacement of the device, or a component of a device, that produces the described sensory perception for a typical (or average) user.

[0102] It should be understood that device 100 is only one example of a portable multifunction device, and that device 100 optionally has more or fewer components than those shown, optionally combines two or more components, or optionally has a different configuration or arrangement of those components. The various components shown in FIG. 1A are implemented in hardware, software, firmware, or a combination thereof, including one or more signal processing circuits and / or application specific integrated circuits.

[0103] Memory 102 optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory 102 by other components of device 100, such as CPU(s) 122 and peripherals interface 118, is optionally controlled by memory controller 120.

[0104] A peripherals interface 118 can be used to couple input and output peripherals of the device to the CPU(s) 122 and memory 102. The one or more processors 122 operate or execute various software programs and / or instruction sets stored in the memory 102 to perform various functions for the device 100 and to process data.

[0105] In some embodiments, peripherals interface 118, CPU(s) 122, and memory controller 120 are optionally implemented on a single chip, such as chip 104. In some other embodiments, they are optionally implemented on separate chips.

[0106] The radio frequency (RF) circuitry 108 transmits and receives RF signals, also referred to as electromagnetic signals. The RF circuitry 108 converts electrical signals to electromagnetic signals or electromagnetic signals to electrical signals and communicates with communication networks and other communication devices via electromagnetic signals. The RF circuitry 108 optionally includes well-known circuits for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and the like. The RF circuitry 108 optionally communicates by wireless communication with networks, such as the Internet (also referred to as the World Wide Web (WWW)), intranets, and / or wireless networks (such as cellular telephone networks, wireless local area networks (LANs) and / or metropolitan area networks (MANs)), and other devices. The wireless communication optionally uses any of a number of communication standards, protocols, and technologies, including Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPADA), long term evolution (LTE), near field communication (NFC), wideband code division multiple access (WDMA), and the like.Wireless technology, such as wireless LAN, wireless LANs, wireless LAN networks, wireless LAN devices, wireless LAN systems, wireless LAN networks, wireless LAN systems ... Examples of suitable communication protocols include, but are not limited to, Simple Intermediate System Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS), and / or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.

[0107] The audio circuit 110, speaker 111, and microphone 113 provide an audio interface between a user and the device 100. The audio circuit 110 receives audio data from the peripherals interface 118, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 111. The speaker 111 converts the electrical signal into sound waves audible to humans. The audio circuit 110 also receives an electrical signal converted from the sound waves by the microphone 113. The audio circuit 110 converts the electrical signal into audio data, and transmits the audio data to the peripherals interface 118 for processing. The audio data is optionally retrieved from and / or transmitted to the memory 102 and / or the RF circuit 108 by the peripherals interface 118. In some embodiments, the audio circuit 110 further comprises a headset jack (e.g., 212 in FIG. 2). The headset jack provides an interface between audio circuitry 110 and a removable audio input / output peripheral device, such as an output-only headphone or a headset that has both an output (e.g., one- or binaural headphones) and an input (e.g., a microphone).

[0108] I / O subsystem 106 couples input / output peripherals on device 100, such as touch-sensitive display system 112 and other input or control devices 116, to peripheral interface 118. I / O subsystem 106 optionally includes one or more input controllers 160 for display controller 156, light sensor controller 158, intensity sensor controller 159, haptic feedback controller 161, and other input or control devices. One or more input controllers 160 receive electrical signals from and send electrical signals to other input or control devices 116. Other input or control devices 116 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, input controller(s) 160 are optionally coupled to any (or none) of a keyboard, an infrared port, a USB port, a stylus, and / or a pointer device such as a mouse. The one or more buttons (e.g., 208 in FIG. 2) optionally include up / down buttons for adjusting the volume of the speaker 111 and / or microphone 113. The one or more buttons optionally include a push button (e.g., 206 in FIG. 2).

[0109] Touch-sensitive display system 112 provides an input and output interface between the device and a user. Display controller 156 receives electrical signals from and / or sends electrical signals to touch-sensitive display system 112. Touch-sensitive display system 112 displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to user interface objects.

[0110] Touch-sensitive display system 112 has a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on haptic / tactile contact. Touch-sensitive display system 112 and display controller 156 (along with any associated modules and / or instruction sets in memory 102) detect contacts (and any movement or breaks in contact) on touch-sensitive display system 112 and translate the detected contacts into interactions with user interface objects (e.g., one or more soft keys, icons, web pages, or images) displayed on touch-sensitive display system 112. In an example embodiment, the point of contact between touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

[0111] Touch-sensitive display system 112 optionally uses liquid crystal display (LCD), light emitting polymer display (LPD), or light emitting diode (LED) technology, although other display technologies are used in other embodiments. Touch-sensitive display system 112 and display controller 156 optionally detect contact and any movement or interruption thereof using any of a number of now known or later developed touch sensing technologies, including, but not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch-sensitive display system 112. In one exemplary embodiment, projected mutual capacitance sensing technology is used, such as that found in the iPhone®, iPod Touch®, and iPad® from Apple Inc. of Cupertino, California.

[0112] Touch-sensitive display system 112 optionally has a video resolution greater than 100 dpi. In some embodiments, the touchscreen video resolution is greater than 400 dpi (e.g., 500 dpi, 800 dpi, or higher). A user optionally contacts touch-sensitive display system 112 using any suitable object or accessory, such as a stylus, finger, or the like. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, which may be less precise than stylus-based input due to the large contact area of ​​a finger on a touchscreen. In some embodiments, the device translates coarse finger-based input into precise pointer / cursor position or commands to perform the user's desired action.

[0113] In some embodiments, in addition to a touchscreen, device 100 optionally includes a touchpad (not shown) for activating or deactivating certain functions. In some embodiments, a touchpad is a touch-sensitive area of ​​a device that, unlike a touchscreen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from touch-sensitive display system 112 or an extension of the touch-sensitive surface formed by the touchscreen.

[0114] Device 100 also includes a power system 162 for providing power to the various components. Power system 162 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharging system, power failure detection circuitry, power converters or inverters, power status indicators (e.g., light emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power within a portable device.

[0115] Device 100 also optionally includes one or more light sensors 164. FIG. 1A shows a light sensor coupled to light sensor controller 158 in I / O subsystem 106. Light sensor(s) 164 optionally include charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) phototransistors. Light sensor(s) 164 receive light from the environment, projected through one or more lenses, and convert the light into data representing an image. In conjunction with imaging module 143 (also referred to as a camera module), light sensor(s) 164 optionally capture still images and / or video. In some embodiments, the light sensor is located on the back of device 100, opposite touch-sensitive display system 112 on the front of the device, so that the touch screen is enabled for use as a viewfinder for still image and / or video capture. In some embodiments, another light sensor is placed on the front of the device to capture an image of the user (e.g., for a selfie, for a video conference while the user views other video conference participants on the touch screen, etc.).

[0116] Device 100 also optionally includes one or more contact intensity sensors 165. FIG. 1A shows a contact intensity sensor coupled to intensity sensor controller 159 in I / O subsystem 106. Contact intensity sensor(s) 165 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch-sensitive surface). Contact intensity sensor(s) 165 receive contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is located on or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112). In some embodiments, at least one contact intensity sensor is located on the back of device 100, opposite touchscreen display system 112 located on the front of device 100.

[0117] Device 100 also optionally includes one or more proximity sensors 166. Figure 1A shows proximity sensor 166 coupled to peripherals interface 118. Alternatively, proximity sensor 166 is coupled to input controller 160 in I / O subsystem 106. In some embodiments, the proximity sensor turns off and disables touch-sensitive display system 112 when the multifunction device is placed near a user's ear (e.g., when the user is making a phone call).

[0118] Device 100 also optionally includes one or more tactile output generators 163. FIG. 1A shows a tactile output generator coupled to haptic feedback controller 161 in I / O subsystem 106. Tactile output generator(s) 163 optionally include one or more electroacoustic devices, such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile output on the device). Tactile output generator(s) 163 receive tactile feedback generation instructions from haptic feedback module 133 and generate tactile outputs on device 100 that can be sensed by a user of device 100. In some embodiments, at least one tactile output generator is disposed on or proximate to a touch-sensitive surface (e.g., touch-sensitive display system 112) and generates a tactile output, optionally by moving the touch-sensitive surface vertically (e.g., in / out of the surface of device 100) or laterally (e.g., back and forth in the same plane as the surface of device 100). In some embodiments, the at least one tactile output generator sensor is disposed on the back of device 100, opposite touch-sensitive display system 112, which is disposed on the front of device 100.

[0119] Device 100 also optionally includes one or more accelerometers 167, gyroscopes 168, and / or magnetometers 169 (e.g., as part of an inertial measurement unit (IMU)) for obtaining information regarding the location (e.g., attitude) of the device. FIG. 1A shows sensors 167, 168, and 169 coupled to peripherals interface 118. Alternatively, sensors 167, 168, and 169 are optionally coupled to input controller 160 in I / O subsystem 106. In some embodiments, information is displayed on the touchscreen display in portrait or landscape view based on analysis of data received from the one or more accelerometers. Device 100 optionally includes a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information regarding the location of device 100.

[0120] In some embodiments, the software components stored in memory 102 include an operating system 126, a communication module (or instruction set) 128, a contact / motion module (or instruction set) 130, a position module (or instruction set) 131, a graphics module (or instruction set) 132, a haptic feedback module (or instruction set) 133, a text input module (or instruction set) 134, a Global Positioning System (GPS) module (or instruction set) 135, and applications (or instruction sets) 136. Additionally, in some embodiments, as shown in Figures 1A and 3, memory 102 stores device / global internal state 157. Device / global internal state 157 includes one or more of the following: active application state, indicating which application, if any, is currently active; display state, indicating what applications, views, or other information are occupying various regions of touch-sensitive display system 112; sensor state, including information obtained from the device's various sensors and other input or control devices 116; and position and / or orientation information regarding the device's position and / or orientation.

[0121] Operating system 126 (e.g., an embedded operating system such as iOS, Darwin®, RTXC®, LINUX®, UNIX®, OS X®, WINDOWS®, or VxWorks®) includes various software components and / or drivers for controlling and managing common system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware and software components.

[0122] The communications module 128 facilitates communication with other devices via one or more external ports 124 and also includes various software components for processing data received by the RF circuitry 108 and / or the external ports 124. The external ports 124 (e.g., Universal Serial Bus (USB), FIREWIRE®, etc.) are adapted to connect to other devices directly or indirectly through a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external ports are multi-pin (e.g., 30-pin) connectors identical to, similar to, and / or compatible with the 30-pin connectors used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California. In some embodiments, the external ports are Lightning® connectors identical to, similar to, and / or compatible with the Lightning® connectors used in some iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, California.

[0123] Contact / motion module 130 optionally detects contact with touch-sensitive display system 112 (in conjunction with display controller 156) and with other touch-sensitive devices (e.g., a touchpad or physical click wheel). Contact / motion module 130 includes software components for performing various operations related to detecting contact (e.g., by a finger or stylus), such as determining whether contact has occurred (e.g., detecting a finger down event), determining the strength of the contact (e.g., the force or pressure of the contact, or a proxy for the force or pressure of the contact), determining whether there is movement of the contact and tracking of the movement across the touch-sensitive surface (e.g., detecting one or more finger drag events), and determining whether the contact has ceased (e.g., detecting a finger up event or an interruption of contact). Contact / motion module 130 receives contact data from the touch-sensitive surface. Determining the movement of the contact point represented by the series of contact data optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. These actions are, optionally, applied to a single contact (e.g., a single finger contact or a stylus contact) or multiple simultaneous contacts (e.g., "multi-touch" / multiple finger contacts and / or stylus contacts). In some embodiments, contact / motion module 130 and display controller 156 detect contacts on a touchpad.

[0124] Contact / motion module 130 optionally detects gesture input by a user. Different gestures on a touch-sensitive surface have different contact patterns (e.g., different movements, timing, and / or intensity of detected contact). Thus, gestures are optionally detected by detecting a particular contact pattern. For example, detecting a finger tap gesture includes detecting a finger down event, followed by detecting a finger up (lift off) event at the same location (or substantially the same location) as the finger down event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on a touch-sensitive surface includes detecting a finger down event, followed by detecting one or more finger drag events, followed by detecting a finger up (lift off) event. Similarly, taps, swipes, drags, and other gestures are optionally detected by detecting a particular contact pattern with respect to the stylus with respect to the stylus.

[0125] In conjunction with the accelerometer 167, gyroscope 168, and / or magnetometer 169, the position module 131 optionally detects attitude information about the device, such as the device's attitude (roll, pitch, and / or yaw) within a particular coordinate system. The position module 130 includes software components for performing various operations related to detecting the position of the device and detecting changes in the position of the device. In some embodiments, the position module 131 uses information received from a stylus being used with the device to detect attitude information about the stylus, such as detecting the attitude state of the stylus relative to the device and detecting changes in the attitude state of the stylus.

[0126] Graphics module 132 includes various known software components for rendering and displaying graphics on touch-sensitive display system 112 or other display, including components for modifying the visual effects (e.g., brightness, transparency, saturation, contrast, or other visual characteristics) of the displayed graphics. As used herein, the term "graphics" includes any object that can be displayed to a user, including, but not limited to, text, web pages, icons (such as user interface objects including soft keys), digital images, videos, animations, and the like.

[0127] In some embodiments, graphics module 132 stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. Graphics module 132 receives one or more codes specifying the graphics to be displayed, including, as appropriate, coordinate data and other graphic characteristic data, from an application or the like, and then generates screen image data for output to display controller 156.

[0128] The tactile feedback module 133 includes various software components for generating instructions used by the tactile output generator(s) 163 to generate tactile outputs at one or more locations on the device 100 in response to user interaction with the device 100.

[0129] Text input module 134 is optionally a component of graphics module 132 and provides a soft keyboard for entering text in various applications (e.g., contacts 137, email 140, IM 141, browser 147, and any other application requiring text input).

[0130] The GPS module 135 determines the location of the device and provides this information for use in various applications (e.g., to the phone 138 for use in location-based dialing calls, to the camera 143 as picture / video metadata, and to applications that provide location-based services such as a weather widget, a local yellow pages widget, and a map / navigation widget).

[0131] Application 136 optionally includes the following modules (or sets of instructions), or a subset or superset thereof: ● a contacts module 137 (sometimes called an address book or contact list); ● Telephone module 138, ●Video conferencing module 139; ● an email client module 140; ● instant messaging (IM) module 141; ● Training support module 142, A camera module 143 for still and / or video images, ● Image management module 144, ● Browser module 147, ● Calendar module 148, A widget module 149, optionally including one or more of a weather widget 149-1, a stock widget 149-2, a calculator widget 149-3, an alarm clock widget 149-4, a dictionary widget 149-5, and other widgets obtained by the user, as well as user-created widgets 149-6; ● a widget creation module 150 for creating user-created widgets 149-6; ● Search module 151, ● a video and music player module 152, optionally consisting of a video player module and a music player module; ● Memo module 153, Map module 154, and / or ●Video editing module 155.

[0132] Examples of other applications 136 optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA® enabled applications, encryption, digital rights management, voice recognition, and voice duplication.

[0133] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, contact module 137 includes executable instructions (e.g., stored in memory 102 or in the application internal state 192 of contact module 137 in memory 370) for managing an address book or contact list, including adding a name(s) to the address book, deleting a name(s) from the address book, associating phone number(s), email address(es), address(es), or other information with a name, associating an image with a name, categorizing and sorting names, providing phone numbers or email addresses to initiate and / or facilitate communication by telephone 138, video conference 139, email 140, or IM 141, and the like.

[0134] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, telephone module 138 includes executable instructions to enter a series of characters corresponding to a telephone number, access one or more telephone numbers in address book 137, modify an entered telephone number, dial each telephone number, conduct a conversation, and disconnect or hang up when the conversation is completed. As mentioned above, wireless communication optionally uses any of a number of communication standards, protocols, and technologies.

[0135] In conjunction with RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, touch-sensitive display system 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact module 130, graphics module 132, text input module 134, contact list 137, and telephone module 138, video conferencing module 139 includes executable instructions for initiating, conducting, and terminating a video conference between a user and one or more other participants in accordance with the user's instructions.

[0136] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, email client module 140 contains executable instructions for creating, sending, receiving, and managing emails in response to user instructions. In conjunction with image management module 144, email client module 140 makes it very easy to create and send emails with still or video images captured by camera module 143.

[0137] In conjunction with RF circuitry 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, instant message module 141 includes executable instructions for inputting character strings corresponding to instant messages, modifying previously input characters, sending respective instant messages (e.g., using Short Message Service (SMS) or Multimedia Message Service (MMS) protocols for telephone-based instant messaging, or using XMPP, SIMPLE, Apple Push Notification Service (APNs), or IMPS for Internet-based instant messaging), and receiving and viewing received instant messages. In some embodiments, the sent and / or received instant messages optionally include graphics, photos, audio files, video files, and / or other attachments, as supported by MMS and / or Enhanced Messaging Service (EMS). As used herein, "instant messaging" refers to both telephone-based messaging (e.g., messages sent using SMS or MMS) and Internet-based messaging (e.g., messages sent using XMPP, SIMPLE, APNs, or IMPS).

[0138] In conjunction with the RF circuitry 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphics module 132, the text input module 134, the GPS module 135, the map module 154, and the music player module 152, the training support module 142 contains executable instructions for creating workouts (e.g., having time, distance, and / or calorie burn goals), communicating with training sensors (in the sports device and the smart watch), receiving training sensor data, calibrating sensors used to monitor the workouts, selecting and playing music for the workouts, and displaying, storing, and transmitting the workout data.

[0139] Camera module 143, in conjunction with touch-sensitive display system 112, display controller 156, light sensor(s) 164, light sensor controller 158, contact module 130, graphics module 132, and image management module 144, contains executable instructions for capturing still images or videos (including video streams) and storing them in memory 102, modifying characteristics of the still images or videos, and / or deleting still images or videos from memory 102.

[0140] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, and camera module 143, image management module 144 contains executable instructions to arrange, modify (e.g., edit) or otherwise manipulate, label, delete, present (e.g., in a digital slide show or album), and store still images and / or videos.

[0141] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, browser module 147 contains executable instructions for browsing the Internet according to a user's instructions, including retrieving, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

[0142] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, email client module 140, and browser module 147, calendar module 148 contains executable instructions to create, display, modify, and store calendars and data associated with the calendars (e.g., calendar entries, to-do lists, etc.) in accordance with user instructions.

[0143] In conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, widget module 149 is optionally a mini-application downloaded and used by a user (e.g., weather widget 149-1, stocks widget 149-2, calculator widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5), or a mini-application created by a user (e.g., user-created widget 149-6). In some embodiments, a widget includes a Hypertext Markup Language (HTML) file, a Cascading Style Sheets (CSS) file, and a JavaScript file. In some embodiments, a widget includes an Extensible Markup Language (XML) file and a JavaScript file (e.g., Yahoo! Widget).

[0144] In conjunction with RF circuitry 108, touch-sensing display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, and browser module 147, widget creation module 150 contains executable instructions for creating widgets (e.g., turning user-specified portions of a web page into a widget).

[0145] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, and text input module 134, search module 151 contains executable instructions to search memory 102 for text, music, sound, images, video, and / or other files that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with a user's instructions.

[0146] In conjunction with touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, RF circuitry 108, and browser module 147, video and music player module 152 contains executable instructions that enable a user to download and play pre-recorded music or other sound files stored in one or more file formats, such as MP3 or AAC files, as well as executable instructions to display, present, or otherwise play videos (e.g., on touch-sensitive display system 112 or on an external display connected wirelessly or via external port 124). In some embodiments, device 100 optionally includes the functionality of an MP3 player, such as an iPod (a trademark of Apple Inc.).

[0147] In conjunction with touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, and text input module 134, notes module 153 contains executable instructions for creating and managing notes, to-do lists, and the like according to user instructions.

[0148] Map module 154, in conjunction with RF circuitry 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, and browser module 147, contains executable instructions to receive, display, modify, and store maps and data associated with the maps (e.g., driving directions, data about businesses and other points of interest at or near a particular location, and other location-based data) in accordance with a user's instructions.

[0149] Video editing module 155, in conjunction with touch-sensing display system 112, display system controller 156, contact module 130, graphics module 132, audio circuitry 110, speaker 111, and text input module 134, contains executable instructions that enable a user to edit movies and other videos (e.g., iMovie® from Apple Inc., Cupertino, California).

[0150] Each of the above-identified modules and applications corresponds to an executable set of instructions that performs one or more of the functions described above and methods described in the present application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 102 optionally stores a subset of the above-identified modules and data structures. Additionally, memory 102 optionally stores additional modules and data structures not described above.

[0151] In some embodiments, device 100 is a device in which operation of a predefined set of functions on the device is performed exclusively via a touchscreen and / or touchpad. Using the touchscreen and / or touchpad as the primary input control device for operation of device 100 optionally reduces the number of physical input control devices (push buttons, dials, etc.) on device 100.

[0152] The set of predefined functions, performed exclusively via the touchscreen and / or touchpad, optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates device 100 to a main menu, a home menu, or a root menu from any user interface displayed on device 100. In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device instead of a touchpad.

[0153] 1B is a block diagram illustrating example components for event processing, according to some embodiments. In some embodiments, memory 102 (in FIG. 1A) or 370 (in FIG. 3) includes an event sorter 170 (e.g., in operating system 126) and a corresponding application 136-1 (e.g., any of applications 136, 137-155, and 380-390 described above).

[0154] Event sorter 170 receives the event information and determines application 136-1 and application view 191 of application 136-1 to which the event information is to be delivered. Event sorter 170 includes event monitor 171 and event dispatcher module 174. In some embodiments, application 136-1 includes application internal state 192 that indicates the current application view(s) that are displayed on touch-sensitive display system 112 when the application is active or running. In some embodiments, device / global internal state 157 is used by event sorter 170 to determine which application(s) are currently active, and application internal state 192 is used by event sorter 170 to determine application view 191 to which the event information is to be delivered.

[0155] In some embodiments, application internal state 192 includes additional information such as one or more of resume information used when application 136-1 resumes execution, user interface state information indicating information is being displayed or ready to be displayed by application 136-1, a state queue to allow the user to return to a previous state or view of application 136-1, and a redo / undo queue of actions previously performed by the user.

[0156] Event monitor 171 receives event information from peripherals interface 118. The event information includes information about sub-events (e.g., a user's touch on touch-sensitive display system 112 as part of a multi-touch gesture). Peripherals interface 118 transmits information it receives from I / O subsystem 106 or sensors such as proximity sensor 166, accelerometer(s) 167, gyroscope(s) 168, magnetometer(s) 169, and / or microphone 113 (via audio circuitry 110). Information that peripherals interface 118 receives from I / O subsystem 106 includes information from touch-sensitive display system 112 or a touch-sensitive surface.

[0157] In some embodiments, event monitor 171 sends requests to peripherals interface 118 at predetermined intervals. In response, peripherals interface 118 transmits event information. In other embodiments, peripherals interface 118 transmits event information only if there is a significant event (e.g., receiving an input above a predetermined noise threshold and / or for a predetermined duration).

[0158] In some embodiments, the event sorter 170 also includes a hit view determination module 172 and / or an active event recognizer determination module 173 .

[0159] Hit view determination module 172 provides software procedures for determining where in one or more views a sub-event occurred when touch-sensitive display system 112 displays more than one view. A view consists of controls and other elements that a user can see on the display.

[0160] Another aspect of a user interface associated with an application is the set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application view (of the respective application) in which the touch is detected optionally corresponds to a programmatic level within the programmatic or view hierarchy of the application. For example, the lowest-level view in which the touch is detected is optionally referred to as a hit view, and the set of events that are recognized as suitable inputs is optionally determined based, at least in part, on the hit view of the initial touch that initiates the touch-based gesture.

[0161] Hit view determination module 172 receives information associated with sub-events of a touch-based gesture. If an application has multiple views organized as a hierarchy, hit view determination module 172 identifies the hit view as the lowest view in the hierarchy that should process the sub-event. In most situations, the hit view is the lowest level view in which the contributing sub-event occurs (i.e., the first sub-event in the sequence of sub-events that form the event or potential event). Once a hit view is identified by the hit view determination module, the hit view typically receives all sub-events related to the same touch or input source for which it was identified as the hit view.

[0162] Active event recognizer determination module 173 determines which view(s) in the view hierarchy should receive the particular sequence of sub-events. In some embodiments, active event recognizer determination module 173 determines that only the hit view should receive the particular sequence of sub-events. In other embodiments, active event recognizer determination module 173 determines that all views that contain the physical location of the sub-event are actively participating views, and therefore all actively participating views should receive the particular sequence of sub-events. In other embodiments, even if the touch sub-event is completely confined to the area associated with a particular view, views higher in the hierarchy remain actively participating views.

[0163] Event dispatcher module 174 sends the event information to an event recognizer (e.g., event recognizer 180). In embodiments that include an active event recognizer determination module 173, event dispatcher module 174 delivers the event information to the event recognizers determined by active event recognizer determination module 173. In some embodiments, event dispatcher module 174 stores event information obtained by each event receiver module 182 in an event queue.

[0164] In some embodiments, operating system 126 includes event sorter 170. Alternatively, application 136-1 includes event sorter 170. In yet other embodiments, event sorter 170 is a stand-alone module or part of another module stored in memory 102, such as contact / motion module 130.

[0165] In some embodiments, application 136-1 includes multiple event handlers 190 and one or more application views 191, each including instructions for handling touch events occurring within a respective view of the application's user interface. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, each application view 191 includes multiple event recognizers 180. In other embodiments, one or more of the event recognizers 180 are part of a separate module, such as a user interface kit (not shown) or a higher level object from which application 136-1 inherits methods and other properties. In some embodiments, each event handler 190 includes one or more of a data updater 176, an object updater 177, a GUI updater 178, and / or event data 179 received from event sorter 170. Event handler 190 optionally utilizes or calls data updater 176, object updater 177, or GUI updater 178 to update application internal state 192. Alternatively, one or more of the application views 191 include one or more corresponding event handlers 190. In some embodiments, one or more of the data updater 176, the object updater 177, and the GUI updater 178 are included in the corresponding application view 191.

[0166] A corresponding event recognizer 180 receives event information (e.g., event data 179) from event sorter 170 and identifies an event from the event information. Event recognizer 180 includes an event receiver 182 and an event comparator 184. In some embodiments, event recognizer 180 also includes at least a subset of metadata 183 and event delivery instructions 188 (optionally including sub-event delivery instructions).

[0167] The event receiver 182 receives event information from the event sorter 170. The event information includes information about a sub-event, e.g., information about a touch or a movement of a touch. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event involves a movement of a touch, the event information optionally further includes the speed and direction of the sub-event. In some embodiments, the event includes a rotation of the device from one orientation to another (e.g., from portrait to landscape or vice versa), and the event information includes corresponding information about the current orientation of the device (also called the device's attitude).

[0168] The event comparator 184 compares the event information to predefined event or sub-event definitions and determines the event or sub-event or determines or updates the state of the event or sub-event based on the comparison. In some embodiments, the event comparator 184 includes an event definition 186. The event definition 186 includes definitions of events (e.g., predefined sequences of sub-events), such as, for example, event 1 (187-1), event 2 (187-2), etc. In some embodiments, sub-events in the event 187 include, for example, touch start, touch end, touch movement, touch stop, and multiple touches. In one example, the definition of event 1 (187-1) is a double tap on a displayed object. The double tap includes, for example, a first touch (touch start) for a given stage on the displayed object, a first lift off (touch end) for a given stage, a second touch (touch start) for a given stage on the displayed object, and a second lift off (touch end) for a given stage. In another example, the definition of event 2 (187-2) is a drag operation on the displayed object. A drag includes, for example, a touch (or contact) for a given step on a displayed object, a movement of the touch across the touch-sensitive display system 112, and a lift-off of the touch (touch end). In some embodiments, the event also includes information regarding one or more associated event handlers 190.

[0169] In some embodiments, event definition 187 includes a definition of the event for each user interface object. In some embodiments, event comparator 184 performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view in which three user interface objects are displayed on touch-sensitive display system 112, when a touch is detected on touch-sensitive display system 112, event comparator 184 performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a corresponding event handler 190, event comparator 184 uses the results of the hit test to determine which event handler 190 to activate. For example, event comparator 184 selects the event handler associated with the sub-event and the object that triggers the hit test.

[0170] In some embodiments, the definition for each event 187 also includes a delay action that delays delivery of the event information until it is determined whether the sequence of sub-events corresponds to the event type of the event recognizer.

[0171] If the respective event recognizer 180 determines that the sequence of sub-events does not match any of the events in the event definition 186, the respective event recognizer 180 enters an event disabled, event failed, or event finished state and thereafter ignores the next sub-event of the touch-based gesture. In this situation, any other event recognizers that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

[0172] In some embodiments, each event recognizer 180 includes metadata 183 with configurable properties, flags, and / or lists that indicate how the event delivery system performs sub-event delivery to actively participating event recognizers. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers interact or may interact with each other. In some embodiments, metadata 183 includes configurable properties, flags, and / or lists that indicate whether sub-events are delivered to various levels in a view or programmatic hierarchy.

[0173] In some embodiments, each event recognizer 180 activates an event handler 190 associated with an event when one or more particular sub-events of the event are recognized. In some embodiments, each event recognizer 180 delivers event information associated with the event to the event handler 190. Activating the event handler 190 is distinct from sending (and postponing sending) the sub-events to the corresponding hit view. In some embodiments, the event recognizer 180 throws a flag associated with the recognized event, and the event handler 190 associated with the flag catches the flag and performs predefined processing.

[0174] In some embodiments, the event delivery instructions 188 include sub-event delivery instructions that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery instructions deliver the event information to an event handler associated with a set of sub-events or an actively participating view. The event handler associated with the set of sub-events or an actively participating view receives the event information and performs a predetermined process.

[0175] In some embodiments, data updater 176 creates and updates data used by application 136-1. For example, data updater 176 updates phone numbers used by contacts module 137 or stores video files used by video player module 145. In some embodiments, object updater 177 creates and updates objects used by application 136-1. For example, object updater 177 creates new user interface objects or updates the positions of user interface objects. GUI updater 178 updates the GUI. For example, GUI updater 178 prepares display information and sends display information to graphics module 132 for display on the touch-sensitive display.

[0176] In some embodiments, event handler(s) 190 include or have access to data updater 176, object updater 177, and GUI updater 178. In some embodiments, data updater 176, object updater 177, and GUI updater 178 are included in a single module of a respective application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

[0177] It should be understood that the foregoing description of event processing of a user's touch on a touch-sensitive display also applies to other forms of user input for operating multifunction device 100 with an input device, not all of which are initiated on a touch screen. For example, mouse movements and mouse button presses, optionally in conjunction with single or multiple keyboard presses or holds, touch movements such as tapping, dragging, scrolling on a touchpad, pen stylus input, device movements, verbal cues, detected eye movements, biometric input, and / or any combination thereof, are optionally utilized as inputs corresponding to sub-events that define the event to be recognized.

[0178] FIG. 2 illustrates portable multifunction device 100 with a touch screen (e.g., touch-sensitive display system 112 of FIG. 1A) according to some embodiments. The touch screen optionally displays one or more graphics in a user interface (UI) 200. In this embodiment, as well as other embodiments described below, a user can select one or more of the graphics by making a gesture on the graphics, for example, with one or more fingers 202 (not drawn to scale) or one or more styluses 203 (not drawn to scale). In some embodiments, selection of one or more graphics occurs when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (left to right, right to left, upwards and / or downwards), and / or rolling of a finger in contact with device 100 (right to left, left to right, upwards and / or downwards). In some implementations or situations, accidental contact with a graphic does not select that graphic: for example, a swipe gesture sweeping over an application icon optionally does not select the corresponding application if the gesture corresponding to selection is a tap.

[0179] Device 100 also optionally includes one or more physical buttons, such as a "home" or menu button 204. As mentioned above, menu button 204 is optionally used for navigation to any application 136 in a set of applications optionally executing on device 100. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on a touchscreen display.

[0180] In some embodiments, device 100 includes a touchscreen display, a menu button 204, a push button 206 for turning power to the device on / off and locking the device, a volume control button(s) 208, a subscriber identity module (SIM) card slot 210, a headset jack 212, and an external port 124 for docking / charging. Push button 206 is optionally used to turn power on / off on the device by pressing and holding the button down for a predefined time, to lock the device by pressing and releasing the button before the predefined time has elapsed, and / or to unlock the device or initiate an unlocking process. In some embodiments, device 100 also accepts verbal input through microphone 113 to activate or deactivate some functions. Device 100 also optionally includes one or more contact intensity sensors 165 for detecting the intensity of a contact on touch-sensitive display system 112 and / or one or more tactile output generators 163 for generating a tactile output to a user of device 100.

[0181] FIG. 3 is a block diagram of an exemplary multifunction device with a display and a touch-sensitive surface, according to some embodiments. Device 300 need not be portable. In some embodiments, device 300 is a laptop computer, a desktop computer, a tablet computer, a multimedia playback device, a navigation device, an educational device (such as a child's learning toy), a gaming system, or a control device (e.g., a home or commercial controller). Device 300 typically includes one or more processing units (CPUs) 310, one or more network or other communication interfaces 360, memory 370, and one or more communication buses 320 for interconnecting these components. Communication bus 320 optionally includes circuitry (sometimes called a chipset) that interconnects and controls communication between system components. Device 300 includes an input / output (I / O) interface 330 that includes a display 340, which is typically a touchscreen display. I / O interface 330 also optionally includes a keyboard and / or mouse (or other pointing device) 350 and a touchpad 355, a tactile output generator 357 (e.g., similar to tactile output generator(s) 163 described above with reference to FIG. 1A) for generating tactile output on device 300, sensors 359 (e.g., touch-sensitive sensors, light sensors, contact intensity sensors, proximity sensors, acceleration sensors, attitude sensors, and / or magnetic sensors similar to sensors 112, 164, 165, 166, 167, 168, and 169 described above with reference to FIG. 1A). Memory 370 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices, and optionally includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 370 optionally includes one or more storage devices located remotely from the CPU(s) 310 .In some embodiments, memory 370 stores programs, modules, and data structures similar to, or a subset of, the programs, modules, and data structures stored in memory 102 of portable multifunction device 100 (FIG. 1A). Additionally, memory 370 optionally stores additional programs, modules, and data structures not present in memory 102 of portable multifunction device 100. For example, memory 370 of device 300 optionally stores drawing module 380, presentation module 382, ​​word processing module 384, website creation module 386, disk authoring module 388, and / or spreadsheet module 390, while memory 102 of portable multifunction device 100 (FIG. 1A) optionally does not store these modules.

[0182] Each of the above-identified elements of FIG. 3 is optionally stored in one or more of the memory devices described above. Each of the above-identified modules corresponds to an instruction set for performing a function described above. The above-identified modules or programs (i.e., instruction sets) need not be implemented as separate software programs, procedures, or modules, and thus, in various embodiments, various subsets of these modules are optionally combined or otherwise rearranged. In some embodiments, memory 370 optionally stores a subset of the above-identified modules and data structures. Additionally, memory 370 optionally stores additional modules and data structures not described above.

[0183] 4 is a block diagram of an exemplary electronic stylus 203, according to some embodiments. The electronic stylus 203 may also be referred to simply as a stylus. The stylus 203 includes a memory 402 (optionally including one or more computer-readable storage media), a memory controller 422, one or more processing units (CPUs) 420, a peripherals interface 418, an RF circuit 408, an input / output (I / O) subsystem 406, and other input or control devices 416. The stylus 203 optionally includes an external port 424, and one or more light sensors 464. The stylus 203 optionally includes one or more intensity sensors 465 for detecting the intensity of a contact of the stylus 203 on the device 100 (e.g., if the stylus 203 is used with a touch-sensitive surface such as the touch-sensitive display system 112 of the device 100) or on another surface (e.g., a desk surface). The stylus 203 optionally includes one or more tactile output generators 463 for generating tactile output on the stylus 203. These components optionally communicate via one or more communication buses or signal lines 403.

[0184] In some embodiments, the term "tactile output" as described above refers to a physical displacement of an accessory (e.g., stylus 203) of a device (e.g., device 100) relative to a previous position of the accessory, a physical displacement of a component of the accessory relative to another component of the accessory, or a displacement of a component relative to the center of mass of the accessory, detected by a user with the user's sense of touch. For example, in a situation where an accessory or a component of the accessory is in contact with a touch-sensitive surface of a user (e.g., a finger, palm, or other part of a user's hand), the tactile output generated by the physical displacement is interpreted by the user as a tactile sensation corresponding to a perceived change in a physical property of the accessory or a component of the accessory. For example, the movement of a component (e.g., the housing of stylus 203) is optionally interpreted by the user as a "click" of a physical actuator button. In some cases, the user will feel a tactile sensation such as a "click" even when there is no movement of a physical actuator button associated with the stylus being physically pressed (e.g., displaced) by the user's action. Although such a user's interpretation of touch will depend on the user's personal sensory perception, there are many sensory perceptions of touch that are common to most users. Thus, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., a "click"), unless otherwise noted, the generated tactile output will correspond to a physical displacement of the device, or a component of the device, that produces the described sensory perception for a typical (or average) user.

[0185] It should be understood that stylus 203 is just one example of an electronic stylus, and that stylus 203 optionally has more or fewer components than those shown, optionally combines two or more components, or optionally has a different configuration or arrangement of the components. The various components shown in Figure 4 are implemented in hardware, software, firmware, or a combination thereof, including one or more signal processing circuits and / or application specific integrated circuits.

[0186] Memory 402 optionally includes high speed random access memory, and optionally also includes non-volatile memory, such as one or more flash memory devices, or other non-volatile solid state memory devices. Access to memory 402 by other components of stylus 203, such as CPU(s) 420 and peripherals interface 418, is optionally controlled by a memory controller 422.

[0187] A peripherals interface 418 can be used to couple the stylus input and output peripherals to the CPU(s) 420 and memory 402. The one or more processors 420 operate or execute various software programs and / or instruction sets stored in the memory 402 to perform various functions and process data for the stylus 203.

[0188] In some embodiments, peripherals interface 418, CPU(s) 420, and memory controller 422 are optionally implemented on a single chip, such as chip 404. In some other embodiments, they are optionally implemented on separate chips.

[0189] The RF (radio frequency) circuitry 408 transmits and receives RF signals, also referred to as electromagnetic signals. The RF circuitry 408 converts electrical signals to electromagnetic signals or electromagnetic signals to electrical signals and communicates with the device 100 or 300, a communication network, and / or other communication devices via electromagnetic signals. The RF circuitry 408 optionally includes well-known circuits for performing these functions, including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and the like. The RF circuitry 408 optionally communicates by wireless communication with networks, such as the Internet, also referred to as the World Wide Web (WWW), an intranet, and / or wireless networks, such as cellular telephone networks, wireless local area networks (LANs), and / or metropolitan area networks (MANs), and other devices. The wireless communication optionally uses any of a number of communication standards, protocols, and technologies, including Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPADA), long term evolution (LTE), near field communication (NFC), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), and the like.Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, protocols for email (e.g., Internet message access protocol (IMAP) and / or post office protocol (POP)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Services (IMP), and other standards and techniques. Messaging and Presence Service (IMPS), and / or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.

[0190] The I / O subsystem 406 couples input / output peripherals on the stylus 203, such as other input or control devices 416, to a peripheral interface 418. The I / O subsystem 406 optionally includes one or more input controllers 460 for a light sensor controller 458, an intensity sensor controller 459, a haptic feedback controller 461, and other input or control devices. The one or more input controllers 460 receive electrical signals from and send electrical signals to the other input or control devices 416. The other input or control devices 416 optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, click wheels, etc. In some alternative embodiments, the input controller(s) 460 are optionally coupled to either (or neither) an infrared port and / or a USB port.

[0191] The stylus 203 also includes a power system 462 for providing power to the various components. The power system 462 optionally includes a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharging system, power failure detection circuitry, power converters or inverters, power status indicators (e.g., light emitting diodes (LEDs)), and any other components associated with the generation, management, and distribution of power within a portable device and / or portable accessory.

[0192] Stylus 203 also optionally includes one or more light sensors 464. Figure 4 shows a light sensor coupled to a light sensor controller 458 in I / O subsystem 406. The light sensor(s) 464 optionally include a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) phototransistor. The light sensor(s) 464 receive light from the environment, projected through one or more lenses, and convert the light into data representing an image.

[0193] Stylus 203 also optionally includes one or more contact intensity sensors 465. FIG. 4 shows a contact intensity sensor coupled to an intensity sensor controller 459 in I / O subsystem 406. Contact intensity sensor(s) 465 optionally include one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a surface). Contact intensity sensor(s) 465 receive contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is located at or proximate to the tip of stylus 203.

[0194] Stylus 203 also optionally includes one or more proximity sensors 466. Figure 4 shows proximity sensor 466 coupled to peripherals interface 418. Alternatively, proximity sensor 466 is coupled to input controller 460 in I / O subsystem 406. In some embodiments, the proximity sensor determines the proximity of stylus 203 to an electronic device (e.g., device 100).

[0195] The stylus 203 also optionally includes one or more tactile output generators 463. FIG. 4 shows a tactile output generator coupled to a tactile feedback controller 461 in the I / O subsystem 406. The tactile output generator(s) 463 optionally include one or more electroacoustic devices, such as speakers or other audio components, and / or electromechanical devices that convert energy into linear motion, such as motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other tactile output generating components (e.g., components that convert electrical signals into tactile output on the device). The tactile output generator(s) 463 receive tactile feedback generation instructions from the tactile feedback module 433 and generate a tactile output on the stylus 203 that can be sensed by a user of the stylus 203. In some embodiments, at least one tactile output generator is disposed on or proximate to a single stylus 203 (e.g., a body or housing) and optionally generates a tactile output by moving the stylus 203 vertically (e.g., in a direction parallel to the length of the stylus 203) or laterally (e.g., in a direction perpendicular to the length of the stylus 203).

[0196] Stylus 203 also optionally includes one or more accelerometers 467, gyroscopes 468, and / or magnetometers 469 (e.g., as part of an inertial measurement unit (IMU) for obtaining information regarding the position and attitude state of stylus 203). FIG. 4 shows sensors 467, 468, and 469 coupled to peripherals interface 418. Alternatively, sensors 467, 468, and 469 are optionally coupled to an input controller 460 in I / O subsystem 406. Stylus 203 optionally includes a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information regarding the position of stylus 203.

[0197] In some embodiments, the software components stored in memory 402 include an operating system 426, a communication module (or instruction set) 428, a contact / motion module (or instruction set) 430, a location module (or instruction set) 431, and a global positioning system (GPS) module (or instruction set) 435. Additionally, in some embodiments, as shown in FIG. 4, memory 402 stores device / global internal state 457. Device / global internal state 457 includes one or more of: a sensor state including information obtained from various sensors and other input or control devices 416 of the stylus, an attitude state including information regarding the attitude of the stylus relative to the device (e.g., device 100) (e.g., position, orientation, tilt, roll, and / or distance as shown in FIGS. 5A and 5B), and location information regarding the position of the stylus (e.g., as determined by GPS module 435).

[0198] Operating system 426 (e.g., an embedded operating system such as iOS, Darwin®, RTXC®, LINUX®, UNIX®, OS X®, WINDOWS®, or VxWorks®) includes various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, power management, etc.) and facilitating communication between various hardware and software components.

[0199] Communications module 428 optionally facilitates communication with other devices via one or more external ports 424 and also includes various software components for processing data received by RF circuitry 408 and / or external port 424. External port 424 (e.g., Universal Serial Bus (USB), FIREWIRE®, etc.) is adapted to connect to other devices directly or indirectly through a network (e.g., Internet, wireless LAN, etc.). In some embodiments, the external port is a Lightning® connector identical to or similar and / or compatible with the Lightning® connector used in some of the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, Calif.

[0200] Contact / motion module 430 optionally detects contact with stylus 203 and other touch-sensitive devices of stylus 203 (e.g., buttons or other touch-sensitive components of stylus 203). Contact / motion module 430 includes software components for performing various operations related to detecting contact (e.g., detecting the tip of a stylus on a touch-sensitive display such as touchscreen 112 of device 100 or on another surface such as a desk surface), such as determining if contact has occurred (e.g., detecting a touch-down event), determining the strength of the contact (e.g., the force or pressure of the contact, or a proxy for the force or pressure of the contact), determining if there is movement of the contact and tracking the movement (e.g., across touchscreen 112 of device 100), and determining if the contact has ceased (e.g., detecting a lift-off event or interruption of contact). In some embodiments, contact / motion module 430 receives contact data from I / O subsystem 406. Determining the movement of the contact point represented by the series of contact data optionally includes determining the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. As noted above, in some embodiments, one or more of these operations related to detecting a contact are performed by the device using contact / motion module 130 (in addition to or instead of a stylus using contact / motion module 430).

[0201] The contact / motion module 430 optionally detects gesture input by the stylus 203. Different gestures with the stylus 203 have different contact patterns (e.g., different movements, timing, and / or intensity of detected contact). Thus, a gesture is optionally detected by detecting a particular contact pattern. For example, detecting a single tap gesture includes detecting a touch down event (e.g., at the location of an icon) followed by detecting a lift off event at the same location (or substantially the same location) as the touch down event. As another example, detecting a swipe gesture includes detecting a touch down event followed by detecting one or more stylus dragging events followed by detecting a lift off event. As mentioned above, in some embodiments, gesture detection is performed by the device using the contact / motion module 130 (in addition to or instead of a stylus using the contact / motion module 430).

[0202] In conjunction with the accelerometer 467, gyroscope 468, and / or magnetometer 469, the position module 431 optionally detects orientation information regarding the stylus, such as the orientation (roll, pitch, and / or yaw) of the stylus within a particular coordinate system. In conjunction with the accelerometer 467, gyroscope 468, and / or magnetometer 469, the position module 431 optionally detects stylus movement gestures, such as stylus flicks, taps, and rolls. The position module 431 includes software components for performing various operations related to detecting the position of the stylus within a particular coordinate system and detecting changes in the position of the stylus. In some embodiments, the position module 431 detects the orientation state of the stylus relative to the device and detects changes in the orientation state of the stylus relative to the device. As described above, in some embodiments, device 100 or 300 uses position module 131 (in addition to or instead of the stylus using position module 431) to determine the orientation of the stylus relative to the device and changes in the orientation of the stylus.

[0203] The tactile feedback module 433 includes various software components for generating instructions used by the tactile output generator(s) 463 to generate tactile output at one or more locations on the stylus 203 in response to user interaction with the stylus 203.

[0204] The GPS module 435 determines the location of the stylus and provides this information for use in various applications (e.g., to applications that provide location-based services, such as applications for finding lost devices and / or accessories).

[0205] Each of the above-identified modules and applications corresponds to an executable set of instructions that performs one or more of the functions described above and methods described in the present application (e.g., computer-implemented methods and other information processing methods described herein). These modules (i.e., sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules are optionally combined or otherwise rearranged in various embodiments. In some embodiments, memory 402 optionally stores a subset of the identified modules and data structures described above. Additionally, memory 402 optionally stores additional modules and data structures not described above.

[0206] 5A-5B illustrate attitude states of stylus 203 relative to a touch-sensitive surface (e.g., touch screen 112 of device 100), according to some embodiments. In some embodiments, the attitude states of stylus 203 correspond to (or are indicative of) the position of a projection of a tip (or other representative portion) of the stylus on the touch-sensitive surface (e.g., (x,y) position 504 of FIG. 5A), the orientation of the stylus relative to the touch-sensitive surface (e.g., orientation 506 of FIG. 5A), the tilt of the stylus relative to the touch-sensitive surface (e.g., tilt 512 of FIG. 5B), and / or the distance of the stylus relative to the touch-sensitive surface (e.g., distance 514 of FIG. 5B). In some embodiments, the attitude states of stylus 203 correspond to (or are indicative of) the pitch, yaw, and / or roll of the stylus (e.g., the orientation of the stylus relative to a particular coordinate system, such as the touch-sensitive surface (e.g., touch screen 112) or the Earth's surface). In some embodiments, the attitude state includes a set of attitude parameters (e.g., one or more attitude parameters). In some embodiments, the attitude state is detected according to one or more measurements from stylus 203 transmitted to an electronic device (e.g., device 100). For example, the stylus measures the tilt (e.g., tilt 512 in FIG. 5B ) and / or orientation (e.g., orientation 506 in FIG. 5A ) of the stylus and transmits the measurements to device 100. In some embodiments, the attitude state is detected according to raw output from one or more electrodes in the stylus, as sensed by a touch-sensitive surface (e.g., touch screen 112 of device 100), instead of or in combination with an attitude state detected according to one or more measurements from stylus 203. For example, the touch-sensitive surface receives raw output from one or more electrodes in the stylus and calculates the tilt and / or orientation of the stylus based on the raw output (optionally in conjunction with attitude state information provided by the stylus based on sensor measurements generated by the stylus).

[0207] 5A shows stylus 203 relative to a touch-sensitive surface (e.g., touchscreen 112 of device 100) from a perspective directly above the touch-sensitive surface, in accordance with some embodiments. In FIG. 5A, z-axis 594 points out of the page (i.e., perpendicular to the plane of touchscreen 112), x-axis 590 is parallel to a first edge (e.g., length) of touchscreen 112, and y-axis 592 is parallel to a second edge (e.g., width) of touchscreen 112, and y-axis 592 is perpendicular to x-axis 590.

[0208] 5A shows the tip of stylus 203 at (x, y) position 504. In some embodiments, the tip of stylus 203 is the end of a stylus configured to determine the proximity of the stylus to a touch-sensitive surface (e.g., touch screen 112). In some embodiments, the projection of the tip of the stylus on the touch-sensitive surface is an orthogonal projection. In other words, the projection of the tip of the stylus on the touch-sensitive surface is the end point of a line from the tip of the stylus to the touch-sensitive surface that is perpendicular to the surface of the touch-sensitive surface (e.g., (x, y) position 504 where the tip of the stylus would touch the touch-sensitive surface if the stylus were moved directly along a path perpendicular to the touch-sensitive surface). In some embodiments, the (x, y) position of the bottom left corner of touch screen 112 is position (0, 0) (e.g., (0, 0) position 502), and other (x, y) positions on touch screen 112 are relative to the bottom left corner of touch screen 112. Alternatively, in some embodiments, the (0,0) position is located at another position on the touch screen 112 (e.g., at the center of the touch screen 112) and the other (x,y) positions are relative to the (0,0) position on the touch screen 112.

[0209] FIG. 5A further illustrates stylus 203 having an orientation 506. In some embodiments, orientation 506 is the orientation of a projection of stylus 203 onto touchscreen 112 (e.g., an orthogonal projection of a line corresponding to the length of stylus 203 or a line between the projections of two different points of stylus 203 onto touchscreen 112). In some embodiments, orientation 506 is relative to at least one axis in a plane parallel to touchscreen 112. In some embodiments, orientation 506 is relative to a single axis in a plane parallel to touchscreen 112 (e.g., axis 508 having a clockwise rotation angle from axis 508 ranging from 0 degrees to 360 degrees as shown in FIG. 5A). Alternatively, in some embodiments, orientation 506 is relative to a pair of axes in a plane parallel to touchscreen 112 (e.g., x-axis 590 and y-axis 592 as shown in FIG. 5A, or a pair of axes associated with an application displayed on touchscreen 112).

[0210] In some embodiments, indicia (e.g., indicia 516) are displayed on a touch-sensitive display (e.g., touch screen 112 of device 100). In some embodiments, indicia 516 indicates where the stylus will touch (or make a mark on) the touch-sensitive display before the stylus touches the touch-sensitive display. In some embodiments, indicia 516 is part of a mark drawn on the touch-sensitive display. In some embodiments, indicia 516 is separate from the mark drawn on the touch-sensitive display and corresponds to a virtual "pen tip" or other element that indicates where the mark will be drawn on the touch-sensitive display.

[0211] In some embodiments, the markings 516 are displayed according to the pose state of the stylus 203. For example, in some situations, the markings 516 are displaced from the (x,y) location 504 (as shown in FIGS. 5A and 5B ), and in other situations, the markings 516 are not displaced from the (x,y) location 504 (e.g., when the tilt 512 is zero degrees, the markings 516 are displayed at or near the (x,y) location 504). In some embodiments, the markings 516 are displayed with different colors, sizes (or radii or areas), opacities, and / or other characteristics according to the pose state of the stylus. In some embodiments, the displayed markings correspond to the thickness of a glass layer on the touch-sensitive display, so as to be transmitted through the markings "on the pixels" of the touch-sensitive display, rather than displaying the markings "on the glass" covering the pixels.

[0212] 5B illustrates stylus 203 relative to a touch-sensitive surface (e.g., touchscreen 112 of device 100) from a side perspective of the touch-sensitive surface, in accordance with some embodiments. In FIG. 5B, z-axis 594 points in a direction perpendicular to the plane of touchscreen 112, x-axis 590 is parallel to a first edge (e.g., length) of touchscreen 112, and y-axis 592 is parallel to a second edge (e.g., width) of touchscreen 112, and y-axis 592 is perpendicular to x-axis 590.

[0213] Figure 5B shows stylus 203 with tilt 512. In some embodiments, tilt 512 is an angle with respect to a normal to the surface of the touch-sensitive surface (also referred to simply as the touch-sensitive surface normal) (e.g., normal 510). As shown in Figure 5B, tilt 512 is zero when the stylus is perpendicular / orthogonal to the touch-sensitive surface (e.g., when stylus 203 is parallel to normal 510) and the tilt increases as the stylus is tilted closer to being parallel to the touch-sensitive surface.

[0214] Figure 5B also shows distance 514 of stylus 203 relative to the touch-sensitive surface. In some embodiments, distance 514 is the distance from the tip of stylus 203 to the touch-sensitive surface in a direction perpendicular to the touch-sensitive surface. For example, in Figure 5B, distance 514 is the distance from the tip of stylus 203 to (x,y) location 504.

[0215] The terms "x-axis", "y-axis", and "z-axis", as used herein, indicate particular directions in particular figures, but it will be understood that these terms do not refer to absolute directions. In other words, the "x-axis" may be any corresponding axis, and the "y-axis" may be a particular axis separate from the x-axis. Generally, the x-axis is perpendicular to the y-axis. Similarly, the "z-axis" is separate from the "x-axis" and "y-axis", and generally perpendicular to both the "x-axis" and "y-axis".

[0216] FIG. 5B further illustrates the rotation of roll 518 about the length (long axis) of stylus 203.

[0217] Attention is now directed to embodiments of a user interface (“UI”), optionally implemented on portable multifunction device 100.

[0218] 6A shows an exemplary user interface for a menu of applications on portable multifunction device 100, according to some embodiments. A similar user interface is, optionally, implemented on device 300. In some embodiments, user interface 600 includes the following elements, or a subset or superset thereof: ● signal strength indicator(s) 602 for wireless communication(s), such as cellular and Wi-Fi signals; ●Time 604, ●Bluetooth indicator 605, ● Battery status indicator 606, A tray 608 containing icons for frequently used applications, such as: ○ An icon 616 for the phone module 138, labeled "Phone", optionally including an indicator 614 of the number of missed calls or voicemail messages; ○ An icon 618 for the email client module 140, labeled “Mail”, optionally including an indicator 610 of the number of unread emails; ○ An icon 620 for the browser module 147, labeled "Browser"; and ○ An icon 622 for the video and music player module 152, also referred to as the iPod (trademark of Apple Inc.) module 152, labeled "iPod"; and ●Icons for other applications, such as: ○ Icon 624 for IM module 141, labeled "Messages"; ○ An icon 626 for the calendar module 148, labeled "Calendar"; ○ Icon 628 for the image management module 144, labeled "Photos"; ○ An icon 630 for the camera module 143, labeled “camera”; ○ Icon 632 for the video editing module 155, labeled "Video Editing"; ○ Icon 634 for stock widget 149-2, labeled "Stock Prices" ○ Icon 636 for map module 154, labeled "Map"; ○ Icon 638 for weather widget 149-1, labeled "Weather" ○ Icon 640 for alarm clock widget 149-4, labeled "Clock" ○ Icon 642 for Training Support Module 142, labeled "Training Support"; ○ An icon 644 for the notes module 153, labeled "Notes"; and ○ An icon 646 for a settings application or module that provides access to settings related to the device 100 and its various applications 136.

[0219] Note that the icon labels shown in FIG. 6A are merely exemplary. For example, in some embodiments, icon 622 for video and music player module 152 is labeled "Music" or "Music Player." Other labels are optionally used for the various application icons. In some embodiments, the label for each application icon includes the name of the application that corresponds to the respective application icon. In some embodiments, the label of a particular application icon is different from the name of the application that corresponds to the particular application icon.

[0220] 6B shows an example user interface on a device (e.g., device 300 of FIG. 3 ) with a touch-sensitive surface 651 (e.g., tablet or touchpad 355 of FIG. 3 ) that is separate from display 650. Device 300 also optionally includes one or more contact intensity sensors (e.g., one or more of sensors 359) for detecting the intensity of a contact on touch-sensitive surface 651 and / or one or more tactile output generators 359 for generating a tactile output to a user of device 300.

[0221] FIG. 6B illustrates an example user interface on a device (e.g., device 300 of FIG. 3) with a touch-sensitive surface 651 (e.g., tablet or touchpad 355 of FIG. 3) that is separate from a display 650. Although many of the following examples are described with reference to input on touchscreen display 112 (where the touch-sensitive surface and display are combined), in some embodiments, the device detects input on a touch-sensitive surface that is separate from the display, as shown in FIG. 6B. In some embodiments, the touch-sensitive surface (e.g., 651 of FIG. 6B) has a major axis (e.g., 652 of FIG. 6B) that corresponds to a major axis (e.g., 653 of FIG. 6B) on the display (e.g., 650). According to these embodiments, the device detects contact with touch-sensitive surface 651 (e.g., 660 and 662 of FIG. 6B) at locations that correspond to respective locations on the display (e.g., in FIG. 6B, 660 corresponds to 668 and 662 corresponds to 670). In this manner, when the touch-sensitive surface is separate from the display, user input (e.g., contacts 660 and 662, and their movement) detected by the device on the touch-sensitive surface (e.g., 651 in FIG. 6B ) is used by the device to operate a user interface on the display (e.g., 650 in FIG. 6B ) of the multifunction device. It is to be understood that similar methods are optionally used for the other user interfaces described herein.

[0222] Additionally, although the following examples are described primarily with reference to finger inputs (e.g., finger contacts, finger tap gestures, finger swipe gestures, etc.), it should be understood that in some embodiments, one or more of the finger inputs are replaced with inputs from another input device (e.g., mouse-based inputs or stylus inputs). For example, a swipe gesture is optionally replaced with a mouse click (e.g., instead of a contact) followed by cursor movement along the path of the swipe (e.g., instead of a contact movement). As another example, a tap gesture is optionally replaced with a mouse click while the cursor is positioned over the tap gesture location (e.g., instead of ceasing contact detection following contact detection). Similarly, when multiple user inputs are detected simultaneously, it will be understood that multiple computer mice are optionally being used simultaneously, or that a mouse and finger contacts are optionally being used simultaneously.

[0223] As used herein, the term "focus selector" refers to an input element that indicates the current portion of a user interface with which a user is interacting. In some implementations involving a cursor or other location marker, the cursor acts as a "focus selector" while the cursor is over a particular user interface element (e.g., a button, window, slider, or other user interface element) when an input (e.g., a press input) is detected on a touch-sensitive surface (e.g., touchpad 355 of FIG. 3 or touch-sensitive surface 651 of FIG. 6B), and the particular user interface element is adjusted in response to the detected input. In some implementations involving a touchscreen display (e.g., touch-sensitive display system 112 of FIG. 1A or touchscreen of FIG. 6A) that allows direct interaction with user interface elements on the touchscreen display, a contact detected on the touchscreen acts as a "focus selector" such that when an input (e.g., a press input by contact) is detected at the location of a particular user interface element (e.g., a button, window, slider, or other user interface element) on the touchscreen display, the particular user interface element is adjusted in accordance with the detected input. In some implementations, focus is moved from one region of the user interface to another region of the user interface (e.g., by using the tab or arrow keys to move focus from one button to another) without a corresponding cursor movement or contact movement on the touch-screen display. In these implementations, the focus selector moves to follow the movement of focus between different regions of the user interface. Regardless of the specific form that the focus selector takes, the focus selector is generally a user interface element (or a contact on a touch-screen display) that is controlled by the user to communicate the user's intended interaction with the user interface (e.g., by indicating to the device which element of the user interface the user intends to interact with).For example, positioning of a focus selector (e.g., a cursor, contact, or selection box) over a corresponding button while a press input is detected on a touch-sensitive surface (e.g., a touchpad or touchscreen) indicates that the user intends to activate that corresponding button (and not other user interface elements shown on the device's display).

[0224] As used herein and in the claims, the term “intensity” of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact or a stylus contact) on the touch-sensitive surface, or a proxy for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values ​​including at least four distinct values, and more typically including hundreds of different values ​​(e.g., at least 256). The intensity of a contact is optionally determined (or measured) using a variety of methods and a variety of sensors, or combinations of sensors. For example, one or more force sensors disposed under or adjacent to the touch-sensitive surface are optionally used to measure the force at various points on the touch-sensitive surface. In some embodiments, the force measurements of multiple force sensors are combined (e.g., weighted average or sum) to determine an estimated force of the contact. Similarly, a pressure-sensitive tip of a stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size and / or change in a contact area detected on the touch-sensitive surface, the capacitance and / or change in the capacitance of the touch-sensitive surface proximate the contact, and / or the resistance and / or change in the capacitance of the touch-sensitive surface proximate the contact are optionally used as a surrogate for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the surrogate measure of the force or pressure of the contact is used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the surrogate measure). In some implementations, the surrogate measure of the force or pressure of the contact is converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using the intensity of contact as an attribute of user input enables a user to access additional device functionality that may not otherwise be readily accessible to a user in a device of reduced size with limited area for displaying affordances (e.g., on a touch-sensitive display) and / or receiving user input (e.g., via a touch-sensitive display, touch-sensitive surface, or physical / mechanical controls such as knobs or buttons).

[0225] In some embodiments, the contact / motion module 130 and / or 430 uses a set of one or more intensity thresholds for determining whether an action has been performed by a user (e.g., for determining whether a user has “clicked” on an icon). In some embodiments, at least a subset of the intensity thresholds are determined according to software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of a particular physical actuator and may be adjusted without modifying the physical hardware of the device 100). For example, the mouse “click” threshold of a trackpad or touchscreen display may be set to any of a wide range of predefined thresholds without modifying the trackpad or touchscreen display hardware. Additionally, in some embodiments, a user of the device is provided with a software setting to adjust one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and / or by adjusting multiple intensity thresholds upon a system-level click of an “intensity” parameter).

[0226] As used herein and in the claims, the term "characteristic intensity" of a contact refers to a characteristic of that contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on a plurality of intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples or a set of intensity samples collected within a predefined period (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10 seconds) relative to a predefined event (e.g., after detecting a contact, before detecting lift-off of the contact, before or after detecting the start of movement of the contact, before detecting the end of the contact, before or after detecting an increase in the intensity of the contact, and / or before or after detecting a decrease in the intensity of the contact). The characteristic intensity of a contact is optionally based on one or more of: a maximum intensity of the contact, a mean value of the intensity of the contact, an average value of the intensity of the contact, the top 10% of the intensity of the contact, half the maximum intensity of the contact, 90% of the maximum intensity of the contact, etc. In some embodiments, the duration of the contact is used to determine the characteristic intensity (e.g., when the characteristic intensity is an average of the intensity of the contact over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether an action is performed by the user. For example, the set of one or more intensity thresholds may include a first intensity threshold and a second intensity threshold. In this example, a contact having a characteristic intensity that does not exceed the first threshold results in a first action being performed, a contact having a characteristic intensity that exceeds the first intensity threshold and does not exceed the second intensity threshold results in a second action being performed, and a contact having a characteristic intensity above the second intensity threshold results in a third action being performed. In some embodiments, the comparison between the characteristic intensity and one or more intensity thresholds is not used to determine whether to perform the first or second action, but rather to determine whether to perform one or more actions (e.g., to perform a respective option or to omit performing a respective action).

[0227] In some embodiments, a portion of the gesture is identified for purposes of determining the characteristic intensity. For example, a touch-sensitive surface may receive a continuous swipe contact (e.g., a drag gesture) that transitions from a start position to an end position where the intensity of the contact increases. In this example, the characteristic intensity of the contact at the end position may be based on only a portion of the swipe contact (e.g., only the portion of the swipe contact at the end position) rather than the entire continuous swipe contact. In some embodiments, a smoothing algorithm may be applied to the intensity of the swipe contact before determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of an unweighted moving average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some circumstances, these smoothing algorithms filter out small increases or decreases in the intensity of the swipe contact for purposes of determining the characteristic intensity.

[0228] The user interface diagrams described herein may include one or more intensity thresholds (e.g., a contact detection intensity threshold IT 0 , Light pressure intensity threshold IT L , deep pressure intensity threshold IT D (For example, at least initially L higher), and / or one or more other intensity thresholds (e.g., I L Lower Intensity Threshold I H ), which indicates the current intensity of contact on the touch-sensitive surface relative to a touch intensity threshold (e.g., 0.1 mm). This intensity chart is typically not part of the displayed user interface, but is provided to aid in interpretation of those charts. In some embodiments, a light pressure intensity threshold corresponds to an intensity that will cause the device to perform an action typically associated with a physical mouse button or a trackpad click. In some embodiments, a deep pressure intensity threshold corresponds to an intensity that will cause the device to perform an action different from the action typically associated with a physical mouse button or a trackpad click. In some embodiments, a slight contact detection intensity threshold IT below the light pressure intensity threshold (e.g., and below which contact is no longer detected) is set.0 If a contact is detected with a characteristic intensity (greater than the light pressure intensity threshold), the device moves the focus selector according to the movement of the contact on the touch-sensitive surface without performing any action associated with the light pressure intensity threshold or the deep pressure intensity threshold. Generally, unless otherwise noted, these intensity thresholds are consistent across different sets of user interface diagrams.

[0229] In some embodiments, the device responds to an input detected by the device according to criteria based on the intensity of contact during the input. For example, for some "light press" inputs, the intensity of contact during the input that exceeds a first intensity threshold triggers a first response. In some embodiments, the device responds to an input detected by the device according to criteria that include both the intensity of contact during the input and time-based criteria. For example, for some "deep press" inputs, the intensity of contact during the input that exceeds a second intensity threshold that is greater than the first intensity threshold for a light press triggers a second response only if a delay time has elapsed between matching the first intensity threshold and matching the second intensity threshold. This delay time is typically less than 200 ms in duration (e.g., 40 ms, 100 ms, or 120 ms, with the delay time increasing as the second intensity threshold increases, depending on the magnitude of the second intensity threshold). This delay time helps prevent accidental deep press inputs. As another example, for some "deep press" inputs, there is a period of reduced sensitivity that occurs after the first intensity threshold is met. During the period of time when the sensitivity is reduced, the second intensity threshold is increased. This temporary increase in the second intensity threshold also serves to prevent accidental deep pressure inputs. As opposed to other deep pressure inputs, the response to the detection of a deep pressure input is not based on a time-based criterion.

[0230] In some embodiments, one or more of the input intensity thresholds and / or corresponding outputs vary based on one or more factors, such as user settings, contact movement, input timing, running application, rate at which intensity is applied, number of current inputs, user history, environmental factors (e.g., ambient noise), position of focus selector, etc. Exemplary factors are described in U.S. Patent Application Nos. 14 / 399,606 and 14 / 624,296, which are incorporated by reference herein in their entireties.

[0231] Light pressure intensity threshold IT L From a strength below the light pressure strength threshold IT L and deep pressure intensity threshold IT D An increase in the characteristic intensity of a contact to an intensity between the deep pressure intensity threshold I D From an intensity below the threshold IT D An increase in the characteristic strength of a contact to an intensity above the contact-detection strength threshold IT 0 From the intensity below the contact detection intensity threshold IT 0 and light pressure intensity threshold IT L An increase in the characteristic intensity of the contact to an intensity between 0 and 1 may be referred to as detecting a contact on the touch surface. 0 From the intensity exceeding the contact detection intensity threshold IT 0 A decrease in the characteristic intensity of the contact to an intensity below may be referred to as detecting lift-off of the contact from the touch surface. 0 is zero. In some embodiments, IT 0 is greater than zero. In some figures, shaded circles or ellipses are used to represent the intensity of a contact on the touch-sensitive surface. In some figures, unshaded circles or ellipses are used to represent each contact on the touch-sensitive surface without specifying the intensity of each contact.

[0232] In some embodiments described herein, one or more actions are performed in response to detection of a gesture including a respective pressure input, or in response to detection of a respective pressure input performed on a respective contact (or contacts), which are detected at least in part based on detection of an increase in intensity of the contact (or contacts) above a pressure input intensity threshold. In some embodiments, the respective actions are performed in response to detection of an increase in intensity of the respective contact above a pressure input intensity threshold (e.g., the respective actions are performed by a "downstroke" of the respective pressure input). In some embodiments, the pressure input includes an increase in intensity of the respective contact above a pressure input intensity threshold and a subsequent decrease in intensity of the contact below the pressure input intensity threshold, and the respective actions are performed in response to detection of a subsequent decrease in intensity of the respective contact below the pressure input threshold (e.g., the respective actions are performed by an "upstroke" of the respective pressure input).

[0233] In some embodiments, the device employs intensity hysteresis to avoid accidental inputs, sometimes referred to as "jitter," and the device defines or selects a hysteresis intensity threshold that has a predefined relationship to the pressure input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units below the pressure input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable percentage of the pressure input intensity threshold). Thus, in some embodiments, the pressure input includes an increase in the intensity of each contact above the pressure input intensity threshold and a subsequent decrease in the intensity of the contact below a hysteresis intensity threshold corresponding to the pressure input intensity threshold, and a respective action is performed in response to detection of a subsequent decrease in the intensity of each contact below that hysteresis intensity threshold (e.g., a respective action is performed by an "upstroke" of a respective pressure input). Similarly, in some embodiments, a pressure input is detected only when the device detects an increase in the intensity of the contact from an intensity below the hysteresis intensity threshold to an intensity above the pressure input intensity threshold, and optionally a subsequent decrease in the intensity of the contact to an intensity below the hysteresis intensity, and a corresponding action is performed in response to detection of that pressure input (e.g., an increase in the intensity of the contact or a decrease in the intensity of the contact, as the case may be).

[0234] For ease of explanation, the description of actions performed in response to a press input associated with a press input intensity threshold or in response to a gesture including the press input is optionally triggered in response to detecting an increase in the intensity of the contact above the press input intensity threshold, an increase in the intensity of the contact from an intensity below a hysteresis intensity threshold to an intensity above the press input intensity threshold, a decrease in the intensity of the contact below the press input intensity threshold, or a decrease in the intensity of the contact below a hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in examples where an action is described as being performed in response to detecting a decrease in the intensity of the contact below a press input intensity threshold, the action is optionally performed in response to detecting a decrease in the intensity of the contact below a hysteresis intensity threshold corresponding to and lower than the press input intensity threshold. As mentioned above, in some embodiments, the triggering of these responses is also by matching time-based criteria (e.g., a delay time has elapsed between a matching first intensity threshold and a matching second intensity threshold). User Interface and Related Processes Displaying and updating indicia corresponding to the pose state of the stylus before the stylus touches the touch-sensitive surface

[0235] Some electronic devices use a stylus associated with the device to provide an additional method of input to the touch-sensitive surface of the device. For example, some cellular phones, laptops, and tablets have the capability to receive input from a stylus. However, for various reasons (e.g., thickness of the stylus tip, thickness of the glass on the device, etc.), a user may find it difficult to know exactly where a stylus interaction (e.g., making a mark) will occur before the stylus touches the touch-sensitive surface. The following embodiments address this issue by displaying and updating indicia corresponding to the pose state of the stylus (e.g., the distance, tilt, and / or orientation of the stylus relative to the touch-sensitive display of the device) before the stylus touches the touch-sensitive display.

[0236] 7A-7J show exemplary user interfaces for displaying and updating indicia corresponding to pose states of a stylus, according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes of FIGS. 8A-8D. Although some of the examples below are described with reference to input on a touch screen display (e.g., on touch screen 112, if the touch-sensitive surface and display are combined), in some embodiments the device detects input on a touch-sensitive surface 651 that is separate from display 650, as shown in FIG. 6B.

[0237] In some embodiments, the device is an electronic device with a separate display (e.g., display 650) and a separate touch-sensitive surface (e.g., touch-sensitive surface 651). In some embodiments, the device is portable multifunction device 100, the display is touch-sensitive display system 112, and the touch-sensitive surface includes tactile output generator 163 on the display ( FIG. 1A ). For ease of explanation, the embodiments described with reference to FIGS. 7A-7J and 8A-8D are described with reference to operations performed on a device with touch-sensitive display system 112. In such embodiments, the focus selector is, optionally, a respective finger or stylus contact, a representative point corresponding to a finger or stylus contact (e.g., a centroid of a respective contact or a point associated with a respective contact), or a centroid of two or more contacts detected on touch-sensitive display system 112. However, similar operations are optionally performed on a device with a display 650 and a separate touch-sensitive surface 651 in response to detecting a contact as described in FIGS. 7A-7J on (or near) touch-sensitive surface 651 while displaying the user interface shown in FIGS. 7A-7J on display 650.

[0238] 7A-7J illustrate example user interfaces for displaying and updating indicia corresponding to pose states of a stylus, according to some embodiments. While FIG. 7A illustrates touchscreen 112 with additional details of device 100 (e.g., speaker 111, light sensor 164, proximity sensor 166, etc.), for clarity, FIGs. 7B-7J simply illustrate touchscreen 112 of device 100 without illustrating other details of device 100. Additionally, while FIG. 7A illustrates an example from a perspective directly above the touch-sensitive surface (e.g., touchscreen 112 of device 100), FIGs. 7B-7J illustrate an example from two perspectives, one directly above the touch-sensitive surface (e.g., touchscreen 112 of device 100) and one from a side of the touch-sensitive surface.

[0239] 7A illustrates an example embodiment that displays indicia (e.g., indicia 716-a) corresponding to a pose state of a stylus (e.g., stylus 203). In this example, stylus 203 has a pose state that has an orientation of stylus 203 relative to touch screen 112 (e.g., orientation 706-a) and a position of a projection of the tip of stylus 203 on touch screen 112 (e.g., (x,y) position 704-a). As shown in FIG. 7A, before stylus 203 touches touch screen 112, when stylus 203 is positioned with orientation 706-a (e.g., approximately 315 degrees of clockwise rotation with respect to axis 508) and tip projection at (x,y) position 704-a, indicia 716-a is displayed on touch screen 112 to indicate where stylus 203 will touch (or make a mark on) touch screen 112. In some embodiments, when the stylus is positioned in a first orientation, the marking is displaced in a first direction on the touch screen 112 relative to the (x,y) location 704-a, and when the stylus is positioned in a second orientation distinct from the first orientation, the marking is displaced in a second direction on the touch screen 112 relative to the (x,y) location 704-a. For example, in FIG. 5A, when the stylus 203 is positioned in orientation 506 (e.g., about 45 degrees clockwise rotation angle relative to axis 508), the marking 516 is displaced in a southwest direction relative to the (x,y) location 504, and in FIG. 7A, when the stylus 203 is positioned in orientation 706-a (e.g., about 315 degrees clockwise rotation angle relative to axis 508), the marking 716-a is displaced in a southeast direction relative to the (x,y) location 704-a.

[0240] 7B-7E show an example of varying the distance (e.g., distance 714) of a stylus (e.g., stylus 203) relative to a touch-sensitive display (e.g., touch screen 112) and updating a displayed indicia (e.g., indicia 716) in response to the change in distance. In Figures 7B-7E, the orientation (e.g., orientation 706-b), the (x,y) position of the stylus tip (e.g., (x,y) position 704-b), and the tilt (e.g., tilt 712-a) of the stylus remain constant, and the distance of the stylus relative to the touch-sensitive display changes (e.g., from distance 714-a to distance 714-b to distance 714-c to distance 714-d) as the stylus moves closer to the touch-sensitive display. Additionally, as the stylus 203 moves closer to the touch screen 112, the indicia is updated on the touch screen 112 (e.g., from indicia 716-b to indicia 716-c to indicia 716-d to indicia 716-e). As shown in this example, in some embodiments, as the stylus 203 moves closer to the touch screen 112, the indicia change opacity (e.g., increase opacity), change size (or radius or area) (e.g., decrease size), and / or change color (e.g., become darker).

[0241] 7E-7G illustrate an example where the tilt (e.g., tilt 712) of a stylus (e.g., stylus 203) relative to a touch-sensitive display (e.g., touch screen 112) is changed and a displayed indicia (e.g., indicia 716) is updated in response to the change in tilt. In FIGS. 7E-7G, the orientation (e.g., orientation 706-b), the (x,y) position of the stylus tip (e.g., (x,y) position 704-b), and the distance (e.g., distance 714-d) of the stylus relative to the touch-sensitive display remain constant, and the tilt of the stylus changes (e.g., from tilt 712-a to tilt 712-b to tilt 712-c) as the stylus is tilted closer to being parallel to device 100. As shown in this example, as the tilt increases, the magnitude of the distance between the (x,y) position of the stylus tip and the indicia increases until a predefined maximum distance is reached. In this example, because the mark 716-f (in FIG. 7F) is at the predetermined maximum distance, the tilt of the stylus 203 changes from tilt 712-b (in FIG. 7F) to tilt 712-c (in FIG. 7G) but the mark 716-f remains in the same position.

[0242] Figure 7H illustrates an example of ceasing to display an indicia on a touch-sensitive display (e.g., touch screen 112) when a stylus (e.g., stylus 203) is more than a predetermined distance away from the surface of the touch-sensitive display. In comparison to Figure 7B, where an indicia (e.g., indicia 716-b) is displayed when the tip of stylus 203 is below threshold 710, the indicia ceases to display on touch screen 112 when the tip of stylus 203 exceeds threshold 710 (e.g., at distance 714-e), as shown in Figure 7H.

[0243] FIG. 7I illustrates an example of displaying indicia (e.g., indicia 716-g) when a stylus (e.g., stylus 203) is in physical contact with the surface of a touch-sensitive display (e.g., touch screen 112). In some embodiments, the distance (e.g., distance 714-f) of stylus 203 away from touch screen 112 when stylus 203 is in physical contact with touch screen 112 is calculated to be greater than zero (e.g., to correspond to the thickness of a glass layer on touch screen 112). In some embodiments, the indicia are updated in response to changes in orientation (as described above with respect to FIG. 7A) and in response to changes in tilt (as described above with respect to FIGS. 7E-7G) even when the stylus is in physical contact with the surface of the touch-sensitive display, but for brevity, the details will not be repeated here.

[0244] In some embodiments, when the stylus 203 is in physical contact with the touch screen 112, both the indicia and the mark are displayed. In some embodiments, the indicia correspond to a virtual "pen tip" or other element that indicates where the mark will be drawn on the touch-sensitive display. In some embodiments, the mark is a dot, a line, a stroke, etc. on the touch-sensitive display. In some embodiments, the indicia corresponds to one end (terminus) of the mark. For example, if the mark being drawn is a line, the indicia corresponds to (or is adjacent to) the end of the line being drawn. In some embodiments, when the indicia is displayed when the stylus is in physical contact with the surface of the touch-sensitive display, the shape (and optionally, color) of the indicia corresponds to the shape (and optionally, color) of the tip of the virtual drawing instrument being emulated by the stylus. For example, if the stylus is being used as a virtual yellow highlighter, the indicia is a yellow rectangle (corresponding to the chisel tip of the highlighter), and as the stylus is moved across the surface of the touch-sensitive display, a mark is generated (e.g., by propagating the size / shape / color of the indicia along the path of the stylus). If the stylus is being used as a virtual yellow highlighter and the stylus touches down on a point on the touch-sensitive display and then lifts off without moving across the touch-sensitive display, a yellow rectangular mark (corresponding to the landmark that is the chisel tip of the highlighter) may be displayed at the point on the touch-sensitive display where the stylus touched down.

[0245] FIG. 7J illustrates an example embodiment in which a mark (e.g., mark 718) is displayed and the indicia is discontinued when a stylus (e.g., stylus 203) is in physical contact with the surface of a touch-sensitive display (e.g., touch screen 112).

[0246] 8A-8D are flow diagrams illustrating a method 800 of displaying and updating indicia corresponding to a pose state of a stylus, according to some embodiments. Method 800 is performed on an electronic device (e.g., device 300 of FIG. 3 or portable multifunction device 100 of FIG. 1A) that includes a display and a touch-sensitive surface. In some embodiments, the display is a touchscreen display and the touch-sensitive surface is on or integrated with the display (also referred to simply as a touch-sensitive display). In some embodiments, the display is separate from the touch-sensitive surface. In some embodiments, the device includes one or more sensors that detect signals from a stylus associated with the device, the stylus including a representative portion (e.g., the tip of the stylus). In some embodiments, portions of the stylus other than the tip are used to estimate the position of the tip. Some operations of method 800 are optionally combined and / or the order of some operations is optionally changed.

[0247] As described below, method 800 provides an intuitive way to display and update indicia corresponding to pose states of a stylus. This method reduces the number, range, and / or type of inputs from a user when using a stylus to interact with an electronic device, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, allowing a user to input with a stylus faster and more efficiently conserves power and increases the time between battery charges.

[0248] The device detects (802) a postural state of the stylus, which corresponds to (or indicates) a distance of the stylus relative to the touch-sensitive display, a tilt of the stylus relative to the touch-sensitive display, and / or an orientation of the stylus relative to the touch-sensitive display. For example, FIG. 7B shows stylus 203 in a postural state corresponding to distance 714-a, tilt 712-a, and orientation 706-b. In some embodiments, the postural state includes a set of postural parameters (e.g., one or more postural parameters). In some embodiments, the postural state is detected according to one or more measurements from the stylus transmitted to the electronic device. For example, the stylus measures the tilt and / or orientation of the stylus and transmits the measurements to the electronic device. Alternatively, in some embodiments, the postural state is detected according to raw output from one or more electrodes in the stylus, which is sensed by the touch-sensitive display. For example, the touch-sensitive display receives raw output from one or more electrodes in the stylus and calculates the tilt and / or orientation of the stylus based on the raw output.

[0249] In some embodiments, the tilt of the stylus is an angle relative to a normal to the surface of the touch-sensitive display (also referred to simply as the touch-sensitive display normal), and the orientation of the stylus is the orientation of the projection of the stylus onto the touch-sensitive display (e.g., an orthogonal projection of a line corresponding to the length of the stylus onto the touch-sensitive display or a line between the projections of two different points of the stylus) relative to at least one axis in a plane parallel to the touch-sensitive display (804). For example, in FIG. 5B, the tilt 512 of the stylus 203 is an angle relative to a normal 510 of the touch screen 112, and in FIG. 5A, the orientation 506 of the stylus 203 is the orientation of the projection of the stylus 203 onto the touch screen 112 relative to an axis 508. In some embodiments, the orientation of the projection of the stylus onto the touch-sensitive display is relative to a single axis, such as axis 508 in FIG. 5A (e.g., a single axis having a clockwise rotation angle from this axis ranging from 0 degrees to 360 degrees). Alternatively, in some embodiments, the projection of the stylus onto the touch-sensitive display is oriented with respect to a pair of axes (e.g., an x-axis and a y-axis, such as x-axis 590 and y-axis 592 in FIG. 5A, or a pair of axes associated with an application displayed on the touch-sensitive display).

[0250] In some embodiments, when the stylus is in physical contact with the surface of the touch-sensitive display, the distance of the stylus away from the touch-sensitive display is calculated to be greater than zero (e.g., to correspond to the thickness of a glass layer on the touch-sensitive display (e.g., touch screen 112) so that the indicia travels through the indicia "onto the pixels" of the display rather than displaying the indicia "on the glass" covering the pixels) (806). In some embodiments, when the stylus is in physical contact with the surface of the touch-sensitive display, the distance of the stylus away from the touch-sensitive display is defined to be greater than zero. For example, in FIG. 7I, when stylus 203 is in physical contact with the surface of touch screen 112, distance 714-f is calculated (or defined) to be greater than zero.

[0251] In some embodiments, the pose state includes a location of a projection of the tip (or other representative portion) of the stylus on the touch-sensitive display (808). In some embodiments, the tip of the stylus is the terminus of the stylus configured to determine the proximity of the stylus to the touch-sensitive display. In some embodiments, the projection of the tip of the stylus on the touch-sensitive display is an orthogonal projection. In other words, the projection of the tip of the stylus on the touch-sensitive display is a point at the end of a line from the tip of the stylus to the touch-sensitive display that is perpendicular to the surface of the touch-sensitive display. For example, in FIG. 5B , (x,y) location 504 is the projection of the tip of stylus 203 on touchscreen 112 because this is the point where the tip of stylus 203 would contact touchscreen 112 if stylus 203 was moved directly along a path perpendicular to touchscreen 112 (e.g., along the path of normal 510).

[0252] In some embodiments, a first electrode of the stylus detectable by the electronic device (e.g., by a mechanism in the touch-sensitive display of the device) is proximate to the tip of the stylus, and a second electrode of the stylus detectable by the electronic device (e.g., by a mechanism in the touch-sensitive display of the device) is displaced from the first electrode by a predefined distance (810). In some embodiments, the first electrode of the stylus is at (or near) the tip of the stylus, and the second electrode of the stylus is a ring displaced from the first electrode by a predefined distance. In some embodiments, the stylus 203 has a first electrode at (or near) the tip of the stylus 203, and a second electrode that is a ring displaced from the first electrode by a predefined distance, and both electrodes are detectable by the device 100 (e.g., by a mechanism in the touch screen 112 of the device 100), as shown in FIGS.

[0253] The device determines (812) a location on the touch-sensitive display (e.g., touch screen 112) that corresponds to the detected pose state of the stylus (e.g., stylus 203). Using Figures 5A-5B as an example, the location on the touch screen 112 that corresponds to the detected pose state of the stylus 203 is determined by the orientation 506, the (x,y) position 504, the tilt 512, and / or the distance 514.

[0254] The device displays (814) a visual indicator on the touch-sensitive display (e.g., touchscreen 112) at a location determined before the stylus touches the touch-sensitive display according to the pose state of the stylus (e.g., stylus 203). In some embodiments, the displayed visual indicator indicates where the stylus will touch / mark the touch-sensitive display before the stylus touches the touch-sensitive display. For example, in FIG. 7B, visual indicator 716-b is displayed on touchscreen 112 before stylus 203 touches touchscreen 112 (because stylus 203 is still a distance 714-a away from touchscreen 112). In some embodiments, the displayed visual indicator corresponds to the thickness of the glass of the touch-sensitive display.

[0255] In some embodiments, displaying the indicia on the touch-sensitive display includes determining a color for the indicia according to a background color of the touch-sensitive display at the determined location (816). In some embodiments, determining a color for the indicia according to a background color of the touch-sensitive display at the determined location includes determining a color for the indicia that contrasts with the background color of the touch-sensitive display at the determined location. For example, in some embodiments, if the background color of the touch-sensitive display is white, the color determined for the indicia is gray or black, or another color that contrasts with white. For example, in FIG. 7B, where the background color of touch screen 112 is white, the color for indicia 716-b is gray. As another example, in some embodiments, if the background color of the touch-sensitive display is black, the color determined for the indicia is white or yellow, or another color that contrasts with black.

[0256] The device detects (818) a change in distance, tilt, and / or orientation of the stylus (e.g., stylus 203) before the stylus touches the touch-sensitive display (e.g., touchscreen 112). For example, FIGS. 7B-7E show a change in distance of stylus 203 from touchscreen 112 (e.g., from distance 714-a to distance 714-b to distance 714-c to distance 714-d) before stylus 203 touches touchscreen 112. As another example, FIGS. 7E-7G show a change in tilt of stylus 203 (e.g., from tilt 712-a to tilt 712-b to tilt 712-c) before stylus 203 touches touchscreen 112. As yet another example, FIGS. 5A and 7A show a change in orientation of stylus 203 (eg, from orientation 506 to orientation 706-a) before stylus 203 touches touch screen 112. FIG.

[0257] In response to detecting the change, the device updates (820) the displayed indicia on the touch-sensitive display (e.g., touch screen 112). For example, FIGS. 7B-7E illustrate updating the displayed indicia in response to detecting a change in the distance of stylus 203 from touch screen 112. As the distance changes from distance 714-a to distance 714-b to distance 714-c to distance 714-d, respectively, the indicia is updated on touch screen 112 from indicia 716-b to indicia 716-c to indicia 716-d to indicia 716-e. As another example, FIGS. 7E-7F illustrate updating the displayed indicia in response to detecting a change in the tilt of stylus 203. As the tilt changes from tilt 712-a to tilt 712-b, the indicia is updated on touch screen 112 from indicia 716-e to indicia 716-f. 5A and 7A illustrate updating the displayed indicia in response to detecting a change in orientation of the stylus 203. As the orientation changes from orientation 506 to orientation 706-a, the indicia is updated on the touch screen 112 from indicia 516 to indicia 716-a.

[0258] In some embodiments, when (the tip or other representative portion of) the stylus is a first distance (perpendicular to the display) from the touch-sensitive display, the indicia is displayed in a first color, and when the stylus is a second distance (perpendicular to the display) from the touch-sensitive display that is distinct from the first distance, the indicia is displayed in a second color (822) that is distinct from the first color. In some embodiments, as the distance decreases, the color of the indicia increases in darkness. For example, in FIG. 7B, when the tip of the stylus 203 is a distance 714-a from the touch screen 112, the indicia (e.g., indicia 716-b) is displayed in a light gray color, and in FIG. 7D, when the tip of the stylus 203 is a distance 714-c from the touch screen 112 (e.g., closer to the touch screen 112), the indicia (indicia 716-d) is displayed in a darker gray color.

[0259] In some embodiments, when (the tip or other representative portion of) the stylus is a first distance (in a direction perpendicular to the display) from the touch-sensitive display, the indicia is a first size (e.g., radius or area), and when the stylus is a second distance (in a direction perpendicular to the display) from the touch-sensitive display that is distinct from the first distance, the indicia is a second size (824) that is distinct from the first size. In some embodiments, as the distance decreases, the size of the indicia decreases. For example, in FIG. 7B, when the tip of the stylus 203 is a distance 714-a from the touch screen 112, the indicia (e.g., indicia 716-b) is a large circle, and in FIG. 7D, when the tip of the stylus 203 is a distance 714-c from the touch screen 112 (e.g., closer to the touch screen 112), the indicia (indicia 716-d) is a smaller circle.

[0260] In some embodiments, when (the tip or other representative portion of) the stylus is a first distance (in a direction perpendicular to the display) from the touch-sensitive display, the indicia is a first opacity, and when the stylus is a second distance (in a direction perpendicular to the display) from the touch-sensitive display that is distinct from the first distance, the indicia is a second opacity (826) that is distinct from the first opacity. In some embodiments, as the distance decreases, the opacity of the indicia increases. For example, in FIG. 7B, when the tip of the stylus 203 is a distance 714-a from the touchscreen 112, the indicia (e.g., indicia 716-b) is less opaque, and in FIG. 7D, when the tip of the stylus 203 is a distance 714-c from the touchscreen 112 (e.g., closer to the touchscreen 112), the indicia (indicia 716-d) is more opaque.

[0261] In some embodiments, when the stylus is placed in a first orientation on the touch-sensitive display, the marking is displaced in a first direction on the touch-sensitive display relative to a projection of the tip (or other representative portion) of the stylus, and when the stylus is placed in a second orientation on the touch-sensitive display distinct from the first orientation, the marking is displaced in a second direction on the touch-sensitive display distinct from the first orientation relative to the projection of the tip (or other representative portion) of the stylus (828). In some embodiments, when the orientation of the stylus (e.g., stylus 203) changes, the orientation of the marking (e.g., marking 516 of FIG. 5A) relative to the projection of the tip of the stylus (e.g., (x,y) position 504 of FIG. 5A) changes. For example, in FIG. 5A, when the stylus 203 is positioned in orientation 506, the mark 516 is displaced in a first direction relative to the (x, y) location 504 (e.g., the mark 516 is southwest of the (x, y) location 504, or the mark 516 is at a 225 degree clockwise rotation angle from the (x, y) location 504 relative to the axis 508), and in FIG. 7A, when the stylus 203 is positioned in orientation 706-a, the mark 716-a is displaced in a second direction relative to the (x, y) location 704-a (e.g., the mark 716 is southeast of the (x, y) location 704-a, or the mark 716-a is at a 135 degree clockwise rotation angle from the (x, y) location 704-a relative to the axis 508).

[0262] In some embodiments, when the stylus has a first tilt on the touch-sensitive display, the indicium is displaced a first distance on the touch-sensitive display relative to an orthogonal projection of the tip of the stylus (or other representative portion) in a direction parallel to the plane of the touch-sensitive display, and when the stylus has a second tilt on the touch-sensitive display that is distinct from the first tilt, the indicium is displaced a second distance on the touch-sensitive display that is distinct from the first distance in a direction parallel to the plane of the touch-sensitive display relative to the orthogonal projection of the tip of the stylus (830). In some embodiments, the method includes displacing the indicium from an orthogonal projection of the tip of the stylus on the touch-sensitive display (e.g., the (x,y) location 504 of FIG. 5B where the tip of the stylus would touch the touch-sensitive display if the stylus was moved directly along a path perpendicular to the touch-sensitive display (e.g., along normal 510)) by a distance corresponding to the tilt of the stylus. For example, in Figure 7E, when the stylus 203 has a first tilt (e.g., tilt 712-a), the mark 716-e is a first distance away from the (x,y) location 704-b, and in Figure 7F, when the stylus 203 has a second tilt (e.g., tilt 712-b), the mark 716-f is a second distance away from the (x,y) location 704-b. In some embodiments, the mark is displaced a greater distance from the projection of the tip as the tilt of the stylus increases. For example, when the stylus 203 is positioned with a tilt of 712-a, the marking 716-e is displaced a smaller distance relative to the (x,y) location 704-b (as shown in FIG. 7E), and when the stylus 203 is positioned with an increased tilt of 712-b, the marking 716-f is displaced a larger distance relative to the (x,y) location 704-b (as shown in FIG. 7F). In some embodiments, as shown in FIG. 5B, the respective distances from the orthogonal projection of the tip of the stylus are greater than zero but less than the distance between the point where a line projected along the entire length of the stylus intersects the touch screen and the orthogonal projection location 504.

[0263] In some embodiments, each distance does not exceed a predetermined maximum distance in a direction parallel to the plane of the touch-sensitive display (e.g., between the landmark and the projection in a direction parallel to the plane of the touch-sensitive display) (832). For example, as shown in FIGS. 7E-7G, the first distance and the second distance (as described above with respect to operation 830) do not exceed the predetermined maximum distance. In some embodiments, if the landmark is already at the predetermined maximum distance, further increases in the tilt of the stylus do not increase the magnitude of the distance of the stylus tip relative to the projection. For example, as shown in FIGS. 7F-7G, the tilt of stylus 203 increases from tilt 712-b to tilt 712-c, but landmark 716-f remains the same because it is already at the predetermined maximum distance relative to (x,y) location 704-b.

[0264] In some embodiments, the details of operations 828, 830, and 832 apply (e.g., as described below with respect to operation 838 and with respect to method 1100) even when a stylus (e.g., stylus 203) is in contact with the surface of a touch-sensitive display (e.g., touch screen 112). For example, in some embodiments, when stylus 203 is in contact with the surface of touch screen 112, as described above with respect to operation 828, the orientation of the landmark (e.g., landmark 516 in FIG. 5A ) relative to the projection of the stylus tip (e.g., (x,y) location 504 in FIG. 5A ) changes when the orientation of the stylus (e.g., stylus 203) changes. As another example, in some embodiments, when the stylus 203 is in contact with the surface of the touch screen 112, the magnitude of the distance between the mark (e.g., mark 516 in FIG. 5A ) and the projection of the tip of the stylus (e.g., (x,y) location 504 in FIG. 5A ) changes as the tilt of the stylus (e.g., stylus 203) changes, as described above with respect to operation 830. As yet another example, in some embodiments, when the stylus 203 is in contact with the surface of the touch screen 112, the distance between the mark (e.g., mark 516 in FIG. 5A ) and the projection of the tip of the stylus (e.g., (x,y) location 504 in FIG. 5A ) does not exceed a predetermined maximum distance, as described above with respect to operation 832.

[0265] In some embodiments, the device detects movement of the stylus (e.g., the tip of the stylus) away from the surface of the touch-sensitive display, and in response to detecting movement of the stylus away from the surface of the touch-sensitive display, in accordance with a determination that the stylus is away from the surface of the touch-sensitive display less than a first distance, the device updates the appearance of the indicia on the touch-sensitive display based on the movement of the stylus while maintaining display of the indicia, and in accordance with a determination that the stylus is away from the surface of the touch-sensitive display more than a second distance, the device ceases displaying the indicia on the touch-sensitive display (834). For example, as shown in Figures 7B-7G, when the tip of the stylus 203 is away from the surface of the touchscreen 112 less than a first distance (e.g., threshold 710), the device updates the appearance of the indicia on the touchscreen 112 (e.g., the indicia becomes darker in color, more opaque, and / or smaller in size as the stylus 203 moves closer to the touchscreen 112). Additionally, as shown in FIGURE 7H, when the tip of the stylus 203 is away from the surface of the touch screen 112 by more than a second distance (e.g., threshold 710), the device ceases to display indicia on the touch screen 112. In some embodiments, the first distance is the same as the second distance (e.g., the first distance and the second distance are both threshold 710, as shown in FIGS. 7B and 7H). In some embodiments, the first distance is less than the second distance.

[0266] In some embodiments, in accordance with a determination that the stylus is in physical contact with the surface of the touch-sensitive display, the device displays a mark on the touch-sensitive display in accordance with the pose state, and the device ceases displaying indicia on the touch-sensitive display (836). For example, as shown in FIG. 7J, when stylus 203 is in physical contact with the surface of touch screen 112, the device displays mark 718 and ceases displaying indicia on touch screen 112.

[0267] In some embodiments, pursuant to a determination that a representative portion of the stylus (e.g., the tip of the stylus) is in physical contact with the surface of the touch-sensitive display, the device displays a landmark on the touch-sensitive display according to the pose state, and the position of the landmark on the touch-sensitive display is constrained (838) such that the distance between the landmark and the representative portion of the stylus in a direction parallel to the plane of the touch-sensitive display does not exceed a predetermined maximum distance. For example, when the tip of the stylus is in physical contact with the surface of the touch-sensitive display, the landmark is displayed on the touch-sensitive display with a truncated distance between the landmark and the tip in a direction parallel to the plane of the touch-sensitive display. As shown in FIG. 7I, for example, when the stylus 203 is in physical contact with the surface of the touch screen 112, the device displays landmark 716-g, and the distance between landmark 716-g and (x,y) position 704-b does not exceed a predetermined maximum distance (e.g., despite an increase in the tilt of the stylus 203).

[0268] In some embodiments, before the stylus touches the surface of the touch-sensitive display (or comes within a predetermined distance of the touch-sensitive display), the device responds to the detected movement of the stylus by updating indicia and ceasing to modify the content of the application displayed on the touch-sensitive display, and while the stylus is touching the surface of the touch-sensitive display (or within a predetermined distance of the touch-sensitive display), the device responds to the detected movement of the stylus by modifying the content of the application displayed on the touch-sensitive display (840). In some embodiments, while the stylus is touching the surface of the touch-sensitive display, the device responds to the detected movement of the stylus by updating indicia. In some embodiments, while the stylus is touching the surface of the touch-sensitive display, the device ceases displaying indicia (e.g., as shown in FIG. 7J). In some embodiments, modifying the displayed content of the application includes modifying any of the tools displayed by the application and / or modifying other content of the application.

[0269] In some embodiments, in response to detecting that the stylus is in physical contact with the surface of the touch-sensitive display, the device smoothly changes the (displayed) indicia from a first state to a second state (842). In some embodiments, in response to detecting that the stylus is in physical contact with the surface of the touch-sensitive display, the indicia changes from a first state to a second state to indicate whether the stylus is interacting with the displayed content. In some embodiments, the first state is a preview mode (e.g., a non-interactive mode in which the content is not modified by stylus movement) and the second state is an interactive mode (e.g., a mode in which the content is modified by stylus movement). Optionally, in some embodiments, the indicia in the second state has a greater opacity and / or a smaller radius than the indicia in the first state.

[0270] It should be understood that the particular order described for the operations in Figures 8A-8D is merely exemplary, and the order described is not intended to indicate that the operations are the only order in which they may be performed. Those skilled in the art will recognize various ways of reordering the operations described herein. Furthermore, it should be recognized that other processing details described herein with respect to other methods described herein (e.g., those described in the first paragraph of the Detailed Description above) may also be applied in a manner similar to method 800 described above with respect to Figures 8A-8D. For example, the stylus pose state characteristics, movement, contact, gesture, user interface object, and output described above with reference to method 800, optionally have one or more of the stylus pose state characteristics, movement, contact, gesture, user interface object, and output characteristics described herein with reference to other methods described herein (e.g., those described in the first paragraph of the Detailed Description above). For the sake of brevity, these details will not be repeated here.

[0271] According to some embodiments, FIG. 9 illustrates a functional block diagram of an electronic device 900 configured according to the principles of the various described embodiments. The functional blocks of this device are optionally implemented by hardware, software, firmware, or a combination thereof to carry out the principles of the various described embodiments. Those skilled in the art will understand that the functional blocks described in FIG. 9 are optionally combined or separated into sub-blocks to implement the principles of the various described embodiments. Thus, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

[0272] 9 , electronic device 900 includes a touch-sensitive display unit 902 configured to display a user interface and receive user contacts (including stylus contacts), one or more sensor units 906 configured to detect signals from a stylus associated with the device, and a processing unit 908 coupled to the touch-sensitive display unit 902 and the one or more sensor units 906. In some embodiments, processing unit 908 includes a detection unit 910, a position determination unit 912, a landmark display unit 914, a change detection unit 916, an update unit 918, a mark display unit 920, and a content modification unit 922.

[0273] The processing unit 908 is configured to detect a pose state of the stylus corresponding to a distance of the stylus relative to the touch-sensitive display unit 902, a tilt of the stylus relative to the touch-sensitive display unit 902, and / or an orientation of the stylus relative to the touch-sensitive display unit 902 (e.g., by detection unit 910), determine a position on the touch-sensitive display unit 902 corresponding to the detected pose state of the stylus (e.g., by position determination unit 912), enable display of a landmark on the touch-sensitive display unit 902 at the determined position before the stylus touched the touch-sensitive display unit 902 according to the pose state of the stylus (e.g., by landmark display unit 914), detect a change in the distance, tilt, and / or orientation of the stylus before the stylus touched the touch-sensitive display unit 902 (e.g., by change detection unit 916), and update the landmark displayed on the touch-sensitive display unit 902 in response to detecting the change (e.g., by update unit 918).

[0274] Operations in the above-described information processing methods are optionally performed by operating one or more functional modules within an information processing device, such as a general-purpose processor (e.g., as described above in connection with Figures 1A and 3) or an application-specific chip.

[0275] 8A-8D are optionally performed by components shown in FIGS. 1A-1B or 9. For example, detection operation 802, position determination operation 812, display operation 814, change detection operation 818, and update operation 820 are optionally performed by event sorter 170, event recognizer 180, and event handler 190. Event monitor 171 of event sorter 170 detects a contact (or near-contact) on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. A corresponding event recognizer 180 of application 136-1 compares the event information to a corresponding event definition 186 to determine whether a first contact (or near-contact) at a first location on the touch-sensitive surface (or a rotation of the device) corresponds to a predefined event or sub-event, such as a selection of an object on a user interface or a rotation of the device from one orientation to another. Upon detection of each predefined event or sub-event, the event recognizer 180 activates an event handler 190 associated with the detection of that event or sub-event. The event handler 190 optionally uses or calls a data updater 176 or an object updater 177 to update an application internal state 192. In some embodiments, the event handler 190 accesses a corresponding GUI updater 178 to update what is displayed by the application. Similarly, it will be clear to one skilled in the art how other processes can be implemented based on the components shown in FIGS. 1A-1B. Displaying and updating indicia corresponding to the pose state of the stylus while the stylus is in contact with the touch-sensitive surface

[0276] Some electronic devices use a stylus associated with the device to provide an additional method of input to the touch-sensitive display of the device. For example, some cellular phones, laptops, and tablets have the capability to receive input from a stylus. However, for various reasons (e.g., thickness of the stylus tip, thickness of the glass on the device, etc.), a user may find it difficult to know exactly where the stylus interaction (e.g., making a mark) will occur. The following embodiments address this issue by displaying and updating indicia that correspond to the pose state of the stylus (e.g., the distance, tilt, and / or orientation of the stylus relative to the touch-sensitive display of the device). The indicia indicate where the mark will be made on the touch-sensitive display. The indicia are offset from the tip of the stylus so that the stylus does not obstruct the view of the indicia. The indicia help the user to more precisely position and use the stylus.

[0277] In the following, Figures 10A-10K show exemplary user interfaces for displaying and updating indicia corresponding to stylus pose states. Figures 11A-11D are flow diagrams illustrating a method for displaying and updating indicia corresponding to stylus pose states. The user interfaces of Figures 10A-10K are used to illustrate the processes of Figures 11A-11D.

[0278] 10A-10K show exemplary user interfaces for displaying and updating indicia corresponding to pose states of a stylus, according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes of FIGS. 11A-11D. Although some of the examples below are described with reference to input on a touch screen display (e.g., on touch screen 112, if the touch-sensitive surface and display are combined), in some embodiments the device detects input on a touch-sensitive surface 651 that is separate from display 650, as shown in FIG. 6B.

[0279] In some embodiments, the device is an electronic device with a separate display (e.g., display 650) and a separate touch-sensitive surface (e.g., touch-sensitive surface 651). In some embodiments, the device is portable multifunction device 100, the display is touch-sensitive display system 112, and the touch-sensitive surface includes tactile output generator 163 on the display ( FIG. 1A ). For ease of explanation, the embodiments described with reference to FIGS. 10A-10K and 11A-11D are described with reference to operations performed on a device with touch-sensitive display system 112. In such embodiments, the focus selector is, optionally, a respective finger or stylus contact, a representative point corresponding to a finger or stylus contact (e.g., a face centroid of a respective contact or a point associated with a respective contact), or a face centroid of two or more contacts detected on touch-sensitive display system 112. However, similar operations are optionally performed on a device with a display 650 and a separate touch-sensitive surface 651 in response to detecting a contact as described in FIGS. 10A-10K on (or near) touch-sensitive surface 651 while displaying the user interface shown in FIGS. 10A-10K on display 650.

[0280] 10A-10K illustrate example user interfaces for displaying and updating indicia corresponding to pose states of a stylus, according to some embodiments. While FIG. 10A illustrates touchscreen 112 along with additional details of device 100 (e.g., speaker 111, light sensor 164, proximity sensor 166, etc.), for clarity, FIGS. 10B-10K simply illustrate touchscreen 112 of device 100 without illustrating other details of device 100. Additionally, while FIG. 10A illustrates an example from a perspective directly above the touch-sensitive surface (e.g., touchscreen 112 of device 100), FIGS. 10B-10K illustrate an example from two perspectives, one directly above the touch-sensitive surface (e.g., touchscreen 112 of device 100) and one from a side of the touch-sensitive surface.

[0281] 10A illustrates an example embodiment that displays indicia (e.g., indicia 1016-a) corresponding to a pose state of a stylus (e.g., stylus 203). In this example, stylus 203 has a pose state that has an orientation of stylus 203 relative to touch screen 112 (e.g., orientation 1006-a) and a position of a projection of the tip of stylus 203 on touch screen 112 (e.g., (x,y) position 1004-a). When stylus 203 is positioned with orientation 1006-a (e.g., approximately 315 degrees of clockwise rotation with respect to axis 508) and tip projection at (x,y) position 1004-a as shown in FIG. 10A, indicia 1016-a is displayed on touch screen 112 to indicate where stylus 203 will touch (or make a mark on) touch screen 112. In some embodiments, when the stylus is positioned in a first orientation, the marking is displaced in a first direction on the touch screen 112 relative to the (x, y) location 1004-a, and when the stylus is positioned in a second orientation distinct from the first orientation, the marking is displaced in a second direction on the touch screen 112 relative to the (x, y) location 1004-a.

[0282] 10B-10C show an example of changing the orientation (e.g., orientation 1006) of a stylus (e.g., stylus 203) relative to a touch-sensitive display (e.g., touch screen 112) and updating the displayed indicia in response to the change in orientation (e.g., from displaying indicia 1016-b at a first location to displaying indicia 1016-c at a second location). In FIGs. 10B-10C, the (x,y) position of the stylus tip (e.g., (x,y) position 1004-b), the tilt of the stylus (e.g., tilt 1012-a), and the distance of the stylus relative to the touch-sensitive display (e.g., distance 1014-a when the tip of the stylus is in contact with the touch-sensitive display) remain constant, and the orientation of the stylus relative to the touch-sensitive display changes (e.g., from orientation 1006-a to orientation 1006-b). As the orientation of the stylus 203 changes from orientation 1006-a (e.g., approximately 315 degrees clockwise rotation relative to axis 508) to orientation 1006-b (e.g., approximately 45 degrees clockwise rotation relative to axis 508), the displayed marking is updated on the touch screen 112 (e.g., from marking 1016-b to marking 1016-c).

[0283] 10C-10E show an example of changing the tilt (e.g., tilt 1012) of a stylus (e.g., stylus 203) relative to a touch-sensitive display (e.g., touch screen 112) and updating a displayed indicia (e.g., indicia 1016) in response to the change in tilt. In FIGs. 10C-10E, the orientation (e.g., orientation 1006-b), the (x,y) position (e.g., (x,y) position 1004-b) of the stylus tip, and the distance (e.g., distance 1014-d when the stylus tip is in contact with the touch-sensitive display) of the stylus remain constant, and the tilt of the stylus changes (e.g., from tilt 1012-a to tilt 1012-b to tilt 1012-c) as the stylus is tilted closer to being parallel to device 100. As shown in this example, as the tilt increases, the magnitude of the distance between the (x,y) position of the stylus tip and the landmark increases until a predefined maximum distance is reached. In this example, landmark 1016-d (in FIG. 10D) is at the predefined maximum distance, so the tilt of stylus 203 changes from tilt 1012-b (in FIG. 10D) to tilt 1012-c (in FIG. 10E) while landmark 1016-d remains in the same position.

[0284] 10F-10I show an example of varying the distance (e.g., distance 1014) of a stylus (e.g., stylus 203) relative to a touch-sensitive display (e.g., touch screen 112) and updating a displayed indicia (e.g., indicia 1016) in response to the change in distance. In some embodiments, the pose state includes a position of a projection of a tip (or other representative portion) of the stylus on the touch-sensitive display. In some embodiments, the tip of the stylus is a terminus of the stylus configured to determine the proximity of the stylus to the touch-sensitive display. In some embodiments, the projection of the tip of the stylus on the touch-sensitive display is an orthogonal projection. In other words, the projection of the tip of the stylus on the touch-sensitive display is a point at the end of a line from the tip of the stylus to the touch-sensitive display that is perpendicular to the surface of the touch-sensitive display. For example, in Figures 10F-10I, (x,y) location 1004-b is the projection of the tip of stylus 203 on touch screen 112 because this is the point where the tip of stylus 203 would contact touch screen 112 if stylus 203 were moved directly along a path perpendicular to touch screen 112 (e.g., along the path of normal 510).

[0285] 10F-10I , the orientation of the stylus (e.g., orientation 1006-b), the (x,y) position of the projection of the tip of the stylus (e.g., (x,y) position 1004-b), and the tilt of the stylus (e.g., tilt 1012-a) remain constant, and as the stylus is moved away from the touch-sensitive display, the distance of the stylus relative to the touch-sensitive display changes (e.g., from distance 1014-b to distance 1014-c to distance 1014-d to distance 1014-e). Additionally, as the stylus 203 is moved away from the touchscreen 112, the displayed indicia is updated on the touchscreen 112 (e.g., from indicia 1016-e to indicia 1016-f to indicia 1016-g to indicia 1016-h). As shown in this example, in some embodiments, as the stylus 203 moves closer to the touch screen 112, the markings change opacity (e.g., decrease opacity), change size (or radius or area) (e.g., increase size), and / or change color (e.g., become lighter in color).

[0286] 10J illustrates an embodiment in which indicia cease to be displayed on a touch-sensitive display (e.g., touch screen 112) when a stylus (e.g., stylus 203) is more than a predetermined distance away from the surface of the touch-sensitive display. In comparison to FIGS. 10F-10I, if indicia (e.g., indicia 1016-e, indicia 1016-f, indicia 1016-g, indicia 1016-h, respectively) are displayed when the tip of stylus 203 is below threshold 1010, the indicia cease to be displayed on touch screen 112 when the tip of stylus 203 exceeds threshold 1010, as shown in FIG.

[0287] 10B-10E show examples of displaying indicia (e.g., indicia 1016-b, indicia 1016-c, and indicia 1016-d) when a stylus (e.g., stylus 203) is in physical contact with the surface of a touch-sensitive display (e.g., touch screen 112). In some embodiments, the distance (e.g., distance 1014-a) of stylus 203 away from touch screen 112 when stylus 203 is in physical contact with touch screen 112 is calculated to be greater than zero (e.g., to correspond to the thickness of a glass layer on touch screen 112). In some embodiments, when the stylus is in physical contact with the surface of the touch-sensitive display, the indicia are updated in response to changes in orientation (as described above with respect to FIG. 10A) and in response to changes in tilt (as described above with respect to FIGS. 10C-10E), but for brevity, the details will not be repeated here.

[0288] In some embodiments, when the stylus 203 is in physical contact with the touch screen 112, both the indicia and the mark are displayed. In some embodiments, the indicia correspond to a virtual "pen tip" or other element that indicates where the mark will be drawn on the touch-sensitive display. In some embodiments, the mark is a dot, a line, a stroke, etc. on the touch-sensitive display. In some embodiments, the indicia corresponds to one end (terminus) of the mark. For example, if the mark being drawn is a line, the indicia corresponds to (or is adjacent to) the end of the line being drawn. In some embodiments, when the indicia is displayed when the stylus is in physical contact with the surface of the touch-sensitive display, the shape (and optionally, color) of the indicia corresponds to the shape (and optionally, color) of the tip of the virtual drawing instrument being emulated by the stylus. For example, if the stylus is being used as a virtual yellow highlighter, the indicia is a yellow rectangle (corresponding to the chisel tip of the highlighter), and as the stylus is moved across the surface of the touch-sensitive display, a mark is generated (e.g., by propagating the size / shape / color of the indicia along the path of the stylus). If the stylus is being used as a virtual yellow highlighter and the stylus touches down on a point on the touch-sensitive display and then lifts off without moving across the touch-sensitive display, a yellow rectangular mark (corresponding to the landmark that is the chisel tip of the highlighter) may be displayed at the point on the touch-sensitive display where the stylus touched down.

[0289] FIG. 10K illustrates an embodiment in which a mark (e.g., mark 1018) is displayed and the indicia is discontinued when a stylus (e.g., stylus 203) is in physical contact with the surface of a touch-sensitive display (e.g., touch screen 112).

[0290] 11A-11D are flow diagrams illustrating a method 1100 of displaying and updating indicia corresponding to a pose state of a stylus, according to some embodiments. Method 1100 is performed on an electronic device (e.g., device 300 of FIG. 3 or portable multifunction device 100 of FIG. 1A) that includes a display and a touch-sensitive surface. In some embodiments, the display is a touchscreen display and the touch-sensitive surface is on or integrated with the display (also referred to simply as a touch-sensitive display). In some embodiments, the display is separate from the touch-sensitive surface. In some embodiments, the device includes one or more sensors that detect signals from a stylus associated with the device, the stylus including a tip (or other representative portion). In some embodiments, portions of the stylus other than the tip are used to estimate the position of the tip. Some operations of method 1100 are optionally combined and / or the order of some operations is optionally changed.

[0291] As described below, method 1100 provides an intuitive way to display and update indicia corresponding to the pose state of a stylus. The indicia help a user to more accurately position and use a stylus. This method reduces the number, range, and / or type of inputs from a user when using a stylus to interact with an electronic device, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, allowing a user to input with a stylus faster and more efficiently conserves power and increases the time between battery charges.

[0292] The device detects (1102) a pose state of the stylus, which corresponds to (or is indicative of) the position of the tip of the stylus on the touch-sensitive display, the tilt of the stylus relative to the touch-sensitive display, and the orientation of the stylus relative to the touch-sensitive display while the tip (or other representative portion) of the stylus is in contact with the touch-sensitive display. FIG. 10C, for example, illustrates stylus 203 in pose states corresponding to (x,y) position 1004-b, tilt 1012-a, and orientation 1006-b. In some embodiments, the pose state includes a set of pose parameters (e.g., one or more pose parameters). In some embodiments, the pose state is relative to the touch-sensitive display (e.g., touch screen 112). In some embodiments, the pose state is detected according to one or more measurements from the stylus transmitted to the electronic device. For example, the stylus measures the tilt and / or orientation of the stylus and transmits the measurements to the electronic device. Alternatively, in some embodiments, the attitude state is detected according to raw output from one or more electrodes in the stylus, which is sensed by a touch-sensitive display, for example, the touch-sensitive display receives raw output from one or more electrodes in the stylus and calculates the tilt and / or orientation of the stylus based on the raw output.

[0293] In some embodiments, the tilt of the stylus is an angle relative to a normal to the surface of the touch-sensitive display (also referred to simply as the touch-sensitive display normal), and the orientation of the stylus is the orientation of the projection of the stylus onto the touch-sensitive display (e.g., an orthogonal projection of a line corresponding to the length of the stylus onto the touch-sensitive display or a line between the projections of two different points of the stylus) relative to at least one axis in a plane parallel to the touch-sensitive display (1104). For example, in FIG. 5B, the tilt 512 of the stylus 203 is an angle relative to a normal 510 of the touch screen 112, and in FIG. 5A, the orientation 506 of the stylus 203 is the orientation of the projection of the stylus 203 onto the touch screen 112 relative to an axis 508. In some embodiments, the orientation of the projection of the stylus onto the touch-sensitive display is relative to a single axis, such as axis 508 in FIG. 5A (e.g., a single axis having a clockwise rotation angle from this axis ranging from 0 degrees to 360 degrees). Alternatively, in some embodiments, the projection of the stylus onto the touch-sensitive display is oriented with respect to a pair of axes (e.g., an x-axis and a y-axis, such as x-axis 590 and y-axis 592 in FIG. 5A, or a pair of axes associated with an application displayed on the touch-sensitive display).

[0294] In some embodiments, the pose state includes a distance of the stylus relative to the touch-sensitive display (1106). For example, in Figures 10B-10E, the distance of the stylus 203 relative to the touchscreen 112 is distance 1014-a (e.g., when the tip of the stylus is in contact with the touchscreen 112). As another example, in Figures 10F-10J, as the stylus moves away from the touchscreen 112, the distance of the stylus 203 relative to the touchscreen 112 changes (e.g., from distance 1014-b to distance 1014-c to distance 1014-d to distance 1014-e to distance 1014-f).

[0295] In some embodiments, when the stylus is in physical contact with the surface of the touch-sensitive display, the distance of the stylus away from the touch-sensitive display is calculated to be greater than zero (e.g., to correspond to the thickness of a layer of glass on the touch-sensitive display (e.g., touchscreen 112) so that the indicia travels "onto the pixels" of the display through the indicia rather than displaying the indicia "on the glass" covering the pixels) (1108). In some embodiments, when the stylus is in physical contact with the surface of the touch-sensitive display, the distance of the stylus away from the touch-sensitive display is defined to be greater than zero. For example, in FIGS. 10B-10E, when stylus 203 is in physical contact with the surface of touchscreen 112, distance 1014-a is calculated (or defined) to be greater than zero.

[0296] In some embodiments, a first electrode of the stylus detectable by the electronic device (e.g., by a mechanism in the touch-sensitive display of the device) is proximate to the tip of the stylus, and a second electrode of the stylus detectable by the electronic device (e.g., by a mechanism in the touch-sensitive display of the device) is displaced from the first electrode by a predefined distance (1110). In some embodiments, the first electrode of the stylus is at (or near) the tip of the stylus, and the second electrode of the stylus is a ring displaced from the first electrode by a predefined distance. In some embodiments, the stylus 203 has a first electrode at (or near) the tip of the stylus 203, and a second electrode that is a ring displaced from the first electrode by a predefined distance, both electrodes detectable by the device 100 (e.g., by a mechanism in the touch screen 112 of the device 100), as shown in FIGS.

[0297] The device determines (1112) a first location on the touch-sensitive display (e.g., touch screen 112) that corresponds to the detected pose state of the stylus (e.g., stylus 203). Using FIG. 10B as an example, the location on the touch screen 112 that corresponds to the detected pose state of the stylus 203 is determined by (x,y) position 1004-b, tilt 1012-a, and / or orientation 1006-a.

[0298] The device displays (1114) a visual indicator on the touch-sensitive display (e.g., touch screen 112) at a first location (determined in act 1112) that corresponds to the detected pose state of the stylus, at a first offset from the location of the stylus tip on the touch-sensitive display. For example, in FIG. 10B , visual indicator 1016-b is displayed at a first location that corresponds to the detected pose state of stylus 203, visual indicator 1016-b being at a first offset from (x,y) location 1004-b. In some embodiments, a mark drawn on the touch-sensitive display functions as the visual indicator. In some embodiments, the visual indicator is separate from a mark drawn on the touch-sensitive display. In some embodiments, as described above, the visual indicator corresponds to a virtual "pen tip" or other element that indicates where a mark will be drawn on the touch-sensitive display. In some embodiments, the mark is a dot, a line, a stroke, or the like on the touch-sensitive display. In some embodiments, each offset includes a distance from the tip of the stylus to the respective mark and an orientation of a line (not shown) between the respective mark and the tip of the stylus in a direction parallel to the plane of the touch-sensitive display, as shown in Figure 10B. In some embodiments, the displayed mark corresponds to the thickness of the glass of the touch-sensitive display.

[0299] In some embodiments, displaying the indicia on the touch-sensitive display includes determining a color for the indicia according to a background color of the touch-sensitive display at the determined location (1116). In some embodiments, determining a color for the indicia according to a background color of the touch-sensitive display at the determined location includes determining a color for the indicia that contrasts with the background color of the touch-sensitive display at the determined location. For example, in some embodiments, if the background color of the touch-sensitive display is white, the color determined for the indicia is gray or black, or another color that contrasts with white. For example, in FIG. 10B, where the background color of touch screen 112 is white, the color for indicia 1016-b is black. As another example, in some embodiments, if the background color of the touch-sensitive display is black, the color determined for the indicia is white or yellow, or another color that contrasts with black.

[0300] The device detects 1118 a change in orientation of the stylus, such as stylus 203. For example, Figures 10B-10C show a change in orientation of stylus 203, such as from orientation 1006-a to orientation 1006-b.

[0301] In response to detecting a change in orientation of the stylus (e.g., stylus 203), the device determines a second location on the touch-sensitive display (e.g., touch screen 112) corresponding to the change in orientation of the stylus, the second location being at a second offset from the location of the tip of the stylus on the touch-sensitive display (1120). For example, in FIG. 10C , eyemark 1016-c is displayed at a second location corresponding to the detected pose state of stylus 203, where eyemark 1016-c is at a second offset from (x,y) location 1004-b.

[0302] In some embodiments, when the stylus is placed in a first orientation on the touch-sensitive display, the mark is displaced in a first direction on the touch-sensitive display relative to a projection of the tip (or other representative portion) of the stylus, and when the stylus is placed in a second orientation on the touch-sensitive display distinct from the first orientation, the mark is displaced in a second direction on the touch-sensitive display distinct from the first orientation (1122). In some embodiments, when the orientation of the stylus (e.g., stylus 203) changes, the orientation of the mark (e.g., mark 1016-b in FIG. 7B) relative to the projection of the stylus tip (e.g., (x,y) position 1004-b in FIG. 7B) changes. For example, in FIG. 10B , when the stylus 203 is positioned in orientation 1006-a, the mark 1016-b is displaced in a first direction relative to the (x,y) location 1004-b (e.g., the mark 1016-b is southeast of the (x,y) location 1004-b, or the mark 1016-b is at a clockwise rotation angle of 135 degrees from the (x,y) location 1004-b relative to the axis 508). ), and in FIG. 10C, when the stylus 203 is positioned in orientation 1006-b, the mark 1016-c is displaced in a second direction relative to the (x,y) location 1004-b (e.g., the mark 1016-c is southwest of the (x,y) location 1004-b, or the mark 1016-c is at a clockwise rotation angle of 225 degrees from the (x,y) location 1004-b relative to the axis 508).

[0303] In some embodiments, the device displays (1124) the indicia at two or more different locations on the touch-sensitive display while the position of the tip of the stylus on the touch-sensitive display is at a fixed location (e.g., in FIGS. 10B-10D , (x,y) location 1004-b is at a fixed location). For example, the method includes changing the position of the indicia (e.g., from a first location to a second location) while the position of the tip of the stylus on the touch-sensitive display remains the same. In FIGS. 10B and 10C , for example, while the position of the tip of the stylus 203 on the touch screen 112 is at a fixed location (e.g., fixed at (x,y) location 1004-b), the device displays the indicia at two different locations on the touch screen 112 (e.g., at the location of indicia 1016-b and at the location of indicia 1016-c). Further, in FIG. 10D , as described below with respect to operation 1126, while the position of the tip of the stylus 203 on the touch screen 112 remains in the same fixed position (e.g., fixed at (x,y) position 1004-b), the device displays a mark on the touch screen 112 at a third position (e.g., at the position of mark 1016-d).

[0304] In some embodiments, the device detects a change in tilt of the stylus, and in response to detecting the change in tilt of the stylus, the device determines a third location on the touch-sensitive display corresponding to the change in tilt of the stylus that is a third offset from a location of the tip of the stylus on the touch-sensitive display, and the device displays (1126) an indicia at the third location on the touch-sensitive display. For example, Figures 10C-10D show a change in tilt of stylus 203 (e.g., from tilt 1012-a to tilt 1012-b), with indicia 1016-d displayed at a third location corresponding to an pose state of stylus 203, with indicia 1016-d at a third offset from (x,y) location 1004-b. In some embodiments, the method includes (1) detecting a change in both stylus orientation and stylus tilt, and (2) in response to detecting the change in both stylus orientation and stylus tilt, (a) determining a fourth location on the touch-sensitive display that is a fourth offset from a location of the stylus tip on the touch-sensitive display corresponding to the change in orientation and tilt, and (b) displaying indicia at the fourth location on the touch-sensitive display. In some embodiments, the method includes detecting a change in a location of the stylus tip on the touch-sensitive display, and in response to detecting a change in a location of the stylus tip on the touch-sensitive display, displaying indicia at a different location.

[0305] In some embodiments, when the stylus has a first tilt on the touch-sensitive display, the indicium is displaced on the touch-sensitive display relative to an orthogonal projection of the tip (or other representative portion) of the stylus by a first distance in a direction parallel to the plane of the touch-sensitive display, and when the stylus has a second tilt on the touch-sensitive display that is distinct from the first tilt, the indicium is displaced on the touch-sensitive display relative to an orthogonal projection of the tip of the stylus by a second distance that is distinct from the first distance (1128). In some embodiments, the method includes displacing the indicium from a position of the tip of the stylus on the touch-sensitive display (e.g., (x,y) position 1004-b of FIG. 10C ) a distance based on the tilt of the stylus. For example, in Figure 10C, when the stylus 203 has a first tilt (e.g., tilt 1012-a), the mark 1016-c is a first distance away from the (x,y) location 1004-b, and in Figure 10D, when the stylus 203 has a second tilt (e.g., tilt 1012-b), the mark 1016-d is a second distance away from the (x,y) location 1004-b. In some embodiments, the mark is displaced a greater distance from the projection of the tip (or the position of the tip on the touch-sensitive display) as the tilt of the stylus increases. For example, when the stylus 203 is positioned with a tilt of 1012-a, the mark 1016-c is displaced a smaller distance relative to the (x, y) location 1004-b (as shown in FIG. 10C), and when the stylus 203 is positioned with an increased tilt of 1012-b, the mark 1016-d is displaced a larger distance relative to the (x, y) location 1004-b (as shown in FIG. 10D).

[0306] In some embodiments, each distance does not exceed a predefined maximum distance in a direction parallel to the plane of the touch-sensitive display (e.g., between the landmark and the (x, y) location of the stylus tip in a direction parallel to the (plane of) the touch-sensitive display) (1130). For example, as shown in FIGS. 10D-10E, the first distance and the second distance (as described above with respect to operation 1128) do not exceed the predefined maximum distance. In some embodiments, if the landmark is already at the predefined maximum distance, a further increase in the tilt of the stylus does not increase the magnitude of the distance relative to the (x, y) location of the stylus tip. For example, as shown in FIGS. 10D-10E, the tilt of the stylus 203 increases from tilt 1012-b to tilt 1012-c, but landmark 1016-d remains the same because it is already at the predefined maximum distance relative to (x, y) location 1004-b.

[0307] In some embodiments, the device detects movement of the stylus (e.g., the tip of the stylus) away from the surface of the touch-sensitive display, and in response to detecting movement of the stylus away from the surface of the touch-sensitive display, in accordance with a determination that the stylus is away from the surface of the touch-sensitive display less than a first distance, the device updates the appearance of the indicia on the touch-sensitive display based on the movement of the stylus while maintaining the display of the indicia, and in accordance with a determination that the stylus is away from the surface of the touch-sensitive display more than a second distance, the device ceases displaying the indicia on the touch-sensitive display (1132). For example, as shown in Figures 10F-10I, when the tip of stylus 203 is away from the surface of touchscreen 112 less than a first distance (e.g., threshold 1010), the device updates the appearance of the indicia on touchscreen 112 (e.g., the indicia becomes lighter in color, less opaque, and / or larger in size as stylus 203 moves away from touchscreen 112). Further, as shown in FIGURE 10J, when the tip of the stylus 203 is away from the surface of the touch screen 112 by more than a second distance (e.g., threshold 1010), the device ceases to display indicia on the touch screen 112. In some embodiments, the first distance is the same as the second distance (e.g., both the first distance and the second distance are threshold 1010, as shown in FIGS. 10F-10J). In some embodiments, the first distance is less than the second distance.

[0308] In some embodiments, pursuant to a determination that a representative portion of the stylus (e.g., the tip of the stylus) is in physical contact with the surface of the touch-sensitive display, the device displays a landmark on the touch-sensitive display according to the pose state, and the position of the landmark on the touch-sensitive display is constrained (1134) such that the distance between the landmark and the representative portion of the stylus in a direction parallel to the plane of the touch-sensitive display does not exceed a predetermined maximum distance. For example, when the tip of the stylus is in physical contact with the surface of the touch-sensitive display, the landmark is displayed on the touch-sensitive display with a truncated distance between the landmark and the tip in a direction parallel to the plane of the touch-sensitive display. As shown in FIGS. 10D-10E, for example, when the stylus 203 is in physical contact with the surface of the touch screen 112, the device displays landmark 1016-d, and the distance between landmark 1016-g and (x,y) location 1004-b does not exceed a predetermined maximum distance (e.g., despite an increase in the tilt of the stylus 203).

[0309] It should be understood that the particular order described for the operations in FIGS. 11A-11D is merely exemplary, and the order described is not intended to indicate that the operations are the only order in which they may be performed. Those skilled in the art will recognize various ways of reordering the operations described herein. Furthermore, it should be recognized that other processing details described herein with respect to other methods described herein (e.g., those described in the first paragraph of the Detailed Description above) may also be applied in a manner similar to method 1100 described above with respect to FIGS. 11A-11D. For example, the stylus pose state characteristics, movement, contact, gesture, user interface object, and output described above with reference to method 1100 may optionally have one or more of the stylus pose state characteristics, movement, contact, gesture, user interface object, and output characteristics described herein with reference to other methods described herein (e.g., those described in the first paragraph of the Detailed Description above). For the sake of brevity, these details will not be repeated here.

[0310] According to some embodiments, FIG. 12 illustrates a functional block diagram of an electronic device 1200 configured according to the principles of the various described embodiments. The functional blocks of this device are optionally implemented by hardware, software, firmware, or a combination thereof to carry out the principles of the various described embodiments. Those skilled in the art will understand that the functional blocks described in FIG. 12 are optionally combined or separated into sub-blocks to implement the principles of the various described embodiments. Thus, the description herein optionally supports any possible combination or separation or further definition of the functional blocks described herein.

[0311] 12, electronic device 1200 includes a touch-sensitive display unit 1202 configured to display a user interface and receive user contacts (including stylus contacts), one or more sensor units 1206 associated with the device and configured to detect signals from a stylus including a tip, and a processing unit 1208 coupled to the touch-sensitive display unit 1202 and the one or more sensor units 1206. In some embodiments, processing unit 1208 includes a detection unit 1210, a position determination unit 1212, a landmark display unit 1214, a change detection unit 1216, and an update unit 1218.

[0312] The processing unit 1208 detects (e.g., with a detection unit 1210) a pose state of the stylus corresponding to a position of the tip of the stylus on the touch-sensitive display unit 1202, a tilt of the stylus relative to the touch-sensitive display unit 1202, and an orientation of the stylus relative to the touch-sensitive display unit 1202 while the tip of the stylus is in contact with the touch-sensitive display unit 1202, determines (e.g., with a position determination unit 1212) a first position on the touch-sensitive display unit 1202 corresponding to the detected pose state of the stylus, and determines (e.g., with a position determination unit 1213) a second position on the touch-sensitive display unit 1202 corresponding to the detected pose state of the stylus. 12, the touch-sensitive display unit 1202 is configured to: enable display of a first position indicator (e.g., by the indicator display unit 1214) corresponding to the detected pose state of the stylus, at a first offset from a position of the tip of the stylus on the touch-sensitive display unit 1202; detect a change in an orientation of the stylus (e.g., by the change detection unit 1216); and, in response to detecting the change in orientation of the stylus, determine (e.g., by the position determination unit 1212) a second position on the touch-sensitive display unit 1202 corresponding to the change in orientation of the stylus, at a second offset from a position of the tip of the stylus on the touch-sensitive display unit 1202; and enable display of the second position indicator (e.g., by the indicator display unit 1214) on the touch-sensitive display unit 1202.

[0313] Operations in the above-described information processing methods are optionally performed by operating one or more functional modules within an information processing device, such as a general-purpose processor (e.g., as described above in connection with Figures 1A and 3) or an application-specific chip.

[0314] 1A-11D are optionally performed by components shown in FIGS. 1A-1B or 12. For example, detection operation 1102, position determination operation 1112, display operation 1114, change detection operation 1118, and determination / display operation 1120 are optionally performed by event sorter 170, event recognizer 180, and event handler 190. Event monitor 171 of event sorter 170 detects a contact (or near-contact) on touch-sensitive display 112, and event dispatcher module 174 delivers the event information to application 136-1. A corresponding event recognizer 180 of application 136-1 compares the event information to a corresponding event definition 186 to determine whether a first contact (or near-contact) at a first location on the touch-sensitive surface (or a rotation of the device) corresponds to a predefined event or sub-event, such as a selection of an object on a user interface or a rotation of the device from one orientation to another. Upon detection of each predefined event or sub-event, the event recognizer 180 activates an event handler 190 associated with the detection of that event or sub-event. The event handler 190 optionally uses or calls a data updater 176 or an object updater 177 to update an application internal state 192. In some embodiments, the event handler 190 accesses a corresponding GUI updater 178 to update what is displayed by the application. Similarly, it will be clear to one skilled in the art how other processes can be implemented based on the components shown in FIGS. 1A-1B. Adjusting one or more characteristics of the mark according to characteristics of the input from the stylus

[0315] Some electronic devices use a stylus associated with the device to provide an additional method of input to the touch-sensitive surface of the device. For example, some cellular phones, laptops, and tablets have the capability to receive input from a stylus. However, a user may desire a stylus to emulate one or more virtual drawing instruments (e.g., a pencil, a brush, a calligraphy pen, etc.), where using the stylus with the device provides a user experience similar to using a drawing instrument on paper. In addition, a user may desire a stylus and device to provide drawing capabilities beyond those possible with a drawing instrument on paper (e.g., changing from making a red pen mark when the stylus is moving to the right to making a blue pen mark when the stylus is moving to the left, or changing from making a red pen mark when the stylus contact intensity is low to making a blue pen mark when the stylus contact intensity is high). The following embodiments address these issues by adjusting one or more characteristics of the marks made from the stylus according to the virtual drawing instrument being emulated by the stylus and the characteristics of the input from the stylus.

[0316] Below, Figures 13A-13O show exemplary user interfaces for adjusting one or more characteristics of a mark according to characteristics of input from a stylus, according to some embodiments. Figures 14A-14E are flow diagrams illustrating a method for adjusting one or more characteristics of a mark according to characteristics of input from a stylus, according to some embodiments. The user interfaces of Figures 13A-13O are used to illustrate the process of Figures 14A-14E.

[0317] 13A-13O show exemplary user interfaces for adjusting one or more characteristics of a mark according to characteristics of input from a stylus, according to some embodiments. The user interfaces in these figures are used to illustrate the processes described below, including the processes of FIGS. 14A-14E. Although some of the examples below are described with reference to input on a touch screen display (e.g., on touch screen 112, where the touch-sensitive surface and display are combined), in some embodiments the device detects input on a touch-sensitive surface 651 that is separate from display 650, as shown in FIG. 6B.

[0318] In some embodiments, the device is an electronic device with a separate display (e.g., display 650) and a separate touch-sensitive surface (e.g., touch-sensitive surface 651). In some embodiments, the device is portable multifunction device 100, the display is touch-sensitive display system 112, and the touch-sensitive surface includes tactile output generator 163 on the display ( FIG. 1A ). For ease of explanation, the embodiments described with reference to FIGS. 13A-13O are described with reference to operations performed on a device with touch-sensitive display system 112. In such embodiments, the focus selector is, optionally, a respective finger or stylus contact, a representative point corresponding to a finger or stylus contact (e.g., a face centroid of a respective contact or a point associated with a respective contact), or a face centroid of two or more contacts detected on touch-sensitive display system 112. However, similar operations are optionally performed on a device with a display 650 and a separate touch-sensitive surface 651 in response to detecting a contact described in FIGS. 13A-13O on (or near) touch-sensitive surface 651 while displaying the user interface shown in FIGS. 13A-13O on display 650.

[0319] 13A-13O show example user interfaces for adjusting one or more characteristics of a mark according to characteristics of input from a stylus, according to some embodiments. While FIG. 13A shows touchscreen 112 with additional details of device 100 (e.g., speaker 111, light sensor 164, proximity sensor 166, etc.), for clarity, FIGs. 13B-13O simply show touchscreen 112 of device 100 without showing other details of device 100. Additionally, while FIG. 13A shows the example from a perspective directly above the touch-sensitive surface (e.g., touchscreen 112 of device 100), FIGs. 13B-13O show the example from two perspectives, one directly above the touch-sensitive surface (e.g., touchscreen 112 of device 100) and one from a side of the touch-sensitive surface.

[0320] 13A illustrates an example embodiment that displays indicia (e.g., indicia 1316-a) corresponding to a pose state of a stylus (e.g., stylus 203). In this example, stylus 203 has a pose state that has an orientation of stylus 203 relative to touch screen 112 (e.g., orientation 1306-a) and a position of a projection of the tip of stylus 203 on touch screen 112 (e.g., (x,y) position 1304-a). When stylus 203 is positioned with orientation 1306-a (e.g., approximately 315 degrees of clockwise rotation with respect to axis 508) and tip projection at (x,y) position 1304-a as shown in FIG. 13A, indicia 1316-a is displayed on touch screen 112 to indicate where stylus 203 will touch (or make a mark on) touch screen 112. In some embodiments, when the stylus is positioned in a first orientation, the marking is displaced in a first direction on the touch screen 112 relative to the (x, y) location 1304-a, and when the stylus is positioned in a second orientation distinct from the first orientation, the marking is displaced in a second direction on the touch screen 112 relative to the (x, y) location 1304-a.

[0321] In some embodiments, the attitude of the stylus during stylus input is not a single, static attitude state; rather, the attitude of the stylus generally changes during stylus input (and may be referred to as characteristics of the stylus input). For example, characteristics of the input from the stylus 203 include an orientation of the stylus 203 relative to the touchscreen 112 (e.g., orientation 1306-a), a direction of movement of the stylus 203 across the touchscreen 112 during input ("direction"), a tilt of the stylus 203 relative to the touchscreen 112, a contact strength of the stylus 203 on the touchscreen 112, and / or a speed of movement of the stylus 203 on or near the touchscreen 112.

[0322] 13B-13C and 13H-13L show examples of displaying indicia (e.g., indicia 1316-b, indicia 1316-c, indicia 1316-h, indicia 1316-i, indicia 1316-j, indicia 1316-k, and indicia 1316-l) when a stylus (e.g., stylus 203) is in physical contact with the surface of a touch-sensitive surface (e.g., touchscreen 112). In some embodiments, the distance (e.g., distance 1314-a) of stylus 203 away from touchscreen 112 when stylus 203 is in physical contact with touchscreen 112 is calculated to be greater than zero (e.g., to correspond to the thickness of a glass layer on touchscreen 112). In some embodiments, when the stylus is in physical contact with the surface of the touch-sensitive surface, the markings are updated in response to changes in orientation (as described below with respect to FIGS. 13B-13C) and in response to changes in tilt (as described above), but for the sake of brevity, the details will not be repeated here.

[0323] In some embodiments, when the stylus 203 is in physical contact with the touch screen 112, both the indicia and the mark are displayed (e.g., in Figures 13H-13L). In some embodiments, the indicia correspond to a virtual "pen tip" or other element that indicates where the mark will be drawn on the touch-sensitive display (e.g., touch screen 112). In some embodiments, the mark is a point, a line, a stroke, etc. on the touch-sensitive display. In some embodiments, the indicia corresponds to one end (terminus) of the mark (e.g., in Figures 13D-13G and Figures 13M-13O). For example, if the mark being drawn is a line, the indicia corresponds to (or is adjacent to) the end of the line being drawn. In some embodiments, when the indicia are displayed when the stylus is in physical contact with the surface of the touch-sensitive display, the shape (and optionally, color) of the indicia corresponds to the shape (and optionally, color) of the tip of the virtual drawing implement being emulated by the stylus. For example, if the stylus is being used as a virtual yellow highlighter, the indicium is a yellow rectangle (corresponding to the chisel tip of the highlighter) and as the stylus is moved across the surface of the touch-sensitive display, a mark is generated (e.g., by propagating the size / shape / color of the indicium along the path of the stylus). If a stylus is being used as a virtual yellow highlighter and the stylus touches down on a point on the touch-sensitive display and then lifts off without moving across the touch-sensitive display, a yellow rectangular mark (corresponding to the indicium that is the chisel tip of the highlighter) may be displayed at the point on the touch-sensitive display where the stylus touched down.

[0324] 13B-13C show an example of changing the orientation (e.g., orientation 1306) of a stylus (e.g., stylus 203) relative to a touch-sensitive display (e.g., touch screen 112) and updating the displayed indicia in response to the change in orientation (e.g., from displaying indicia 1316-b at a first location to displaying indicia 1316-c at a second location). In Figures 13B-13C, the (x,y) position of the tip of the stylus (e.g., (x,y) position 1304-b), the tilt of the stylus (e.g., tilt 1312-a), and the distance of the stylus relative to the touch-sensitive display (e.g., distance 1314-a when the tip of the stylus is in contact with the touch-sensitive display) remain constant, and the orientation of the stylus relative to the touch-sensitive display changes (e.g., from orientation 1306-a to orientation 1306-b). As the orientation of the stylus 203 changes from orientation 1306-a (e.g., approximately 315 degrees clockwise rotation relative to axis 508) to orientation 1306-b (e.g., approximately 45 degrees clockwise rotation relative to axis 508), the displayed marking is updated on the touch screen 112 (e.g., from marking 1316-b to marking 1316-c).

[0325] 13D-13E show an example of changing directionality during input from a stylus (e.g., stylus 203) and adjusting a mark (e.g., mark 1330) in response to the change in directionality by changing the width of the mark as the directionality changes. In FIGS. 13D-13E, the orientation of the stylus (e.g., orientation 1306-b), the tilt of the stylus (e.g., tilt 1312-a), and the distance of the stylus relative to the touch-sensitive display (e.g., distance 1314-a when the tip of the stylus is in contact with the touch-sensitive display) remain constant, and the directionality of the stylus changes (e.g., from moving downward at (x,y) location 1304-c to moving upward at (x,y) location 1304-d). In some embodiments, the stylus 203 of FIGS. 13D-13E emulates a calligraphic pen. In Figure 13D, as the stylus 203 moves in one direction (e.g., downward), the width of the mark (e.g., mark 1330-a at landmark 1316-d) increases, and in Figure 13E, as the stylus 203 moves in another direction (e.g., upward), the width of the mark (e.g., mark 1330-b at landmark 1316-e) decreases. In some embodiments, as shown in Figures 13D-13E, there is a gradual transition from a first (e.g., narrow) width while the stylus is moving in a first direction (e.g., upward) to a second (e.g., wide) width while the stylus is moving in a second direction opposite the first direction (e.g., downward).

[0326] In some embodiments, the color of the mark changes as the directionality changes in response to a change in the direction of the stylus 203. In some embodiments, the mark has a subtle color change with the change in directionality. For example, when the stylus 203 moves in a first direction, the color of the mark (e.g., mark 1330) is a light color, and when the stylus 203 moves in a second direction, the color of the mark (e.g., mark 1330) is a darker of the same color. Using FIGS. 13D-13E as an example, in some embodiments, when the stylus 203 moves downward (and the width of the mark increases), the color of the mark is a light blue, and when the stylus 203 moves upward (and the width of the mark decreases), the color of the mark is a darker blue.

[0327] 13F-13G show an example of varying the tilt (e.g., tilt 1312) of a stylus (e.g., stylus 203) relative to a touch-sensitive display (e.g., touch screen 112) to adjust a mark (e.g., mark 1332) in response to the change in tilt by increasing the width of the mark as the tilt increases. In FIGS. 13F-13G, the orientation (e.g., orientation 1306-b) of the stylus and the distance (e.g., distance 1314-a when the tip of the stylus is in contact with the touch-sensitive display) of the stylus remain constant, and the tilt of the stylus changes (e.g., from tilt 1312-b to tilt 1312-c) as the stylus is tilted closer to being parallel to the touch screen 112 during input from the stylus 203. In some embodiments, the stylus 203 of FIGS. 13F-13G emulates a brush. As shown in this example, as the slope increases, the width of the mark (e.g., mark 1332) increases. In Fig. 13F, when the slope is relatively small (e.g., slope 1312-b), the width of the mark is thin (e.g., at mark 1316-f), and in Fig. 13G, when the slope is relatively large (e.g., slope 1312-c), the width of the mark is thick (e.g., at mark 1316-g).

[0328] 13H-13J show an example of varying the contact strength of a stylus (e.g., a parameter corresponding to the intensity of contact of a stylus (e.g., stylus 203) on a touch-sensitive surface (e.g., touch screen 112)) during input to adjust a mark (e.g., mark 1334) in response to the change in contact strength by increasing the width of the mark as the contact strength increases. In FIGS. 13H-13J, the orientation of the stylus (e.g., orientation 1306-b), the tilt of the stylus (e.g., tilt 1312-a), and the distance of the stylus relative to the touch-sensitive display (e.g., distance 1314-a when the tip of the stylus is in contact with the touch-sensitive display) remain constant, and as the contact strength increases during input from stylus 203, the contact strength of the stylus changes (e.g., from contact strength 1322-a in FIG. 13H to contact strength 1322-b in FIG. 13I to contact strength 1322-c in FIG. 13J). In some embodiments, the stylus 203 in Figures 13H-13J emulates a pencil, pen, or brush. As shown in Figures 13H-13J, as the contact intensity increases (e.g., as indicated by the darkening of contact intensity 1322 corresponding to contact intensity 1320), the width of the mark (e.g., mark 1334) increases. In Figure 13H, when the contact intensity is relatively low (e.g., at contact intensity 1322-a), the width of the mark is thin (e.g., at mark 1316-h). In Figure 13I, when the contact intensity is medium (e.g., at contact intensity 1322-b), the width of the mark is medium (e.g., at mark 1316-i). In Figure 13J, when the contact intensity is relatively high (e.g., at contact intensity 1322-c), the width of the mark is thick (e.g., at mark 1316-j).

[0329] In addition, although not shown in FIGS. 13H-13J, in some embodiments, the color of the mark changes as the contact intensity changes in response to a change in the contact intensity of the stylus 203. In some embodiments, as the contact intensity increases, the color of the mark gradually transitions from a first color to a second color. In some embodiments, as the contact intensity of the stylus 203 increases, the color of the mark (e.g., mark 1334) changes from a first color (e.g., red) to a second color (e.g., blue), and the color of the mark during the gradual transition includes a mixture of the first color and the second color (e.g., purple). Using FIGS. 13H-13J as an illustrative example, as the contact intensity of the stylus 203 increases, the color of the mark (e.g., mark 1334) gradually transitions from red to blue (e.g., from red to purple to blue). In Figure 13H, when the contact intensity is relatively low (e.g., at contact intensity 1322-a), the color of the mark is red (e.g., at landmark 1316-h). In Figure 13I, when the contact intensity is medium (e.g., at contact intensity 1322-b), the color of the mark is purple (e.g., a mix of red and blue at landmark 1316-i). In Figure 13J, when the contact intensity is relatively high (e.g., at contact intensity 1322-c), the color of the mark is blue (e.g., at landmark 1316-j). Similarly, in some embodiments, as the contact intensity decreases, the color of the mark gradually transitions from the second color to the first color.

[0330] 13K-13L show an example of varying the contact intensity of a stylus (e.g., stylus 203) during input to adjust a mark (e.g., mark 1336) in response to the change in contact intensity by increasing the opacity of the mark as the contact intensity increases. In FIGS. 13K-13L, the orientation of the stylus (e.g., orientation 1306-b), the tilt of the stylus (e.g., tilt 1312-a), and the distance of the stylus relative to the touch-sensitive display (e.g., distance 1314-a when the tip of the stylus is in contact with the touch-sensitive display) remain constant, and as the contact intensity increases during input from stylus 203, the contact intensity of the stylus changes (e.g., from contact intensity 1322-d in FIG. 13K to contact intensity 1322-e in FIG. 13L). In some embodiments, stylus 203 in FIGS. 13K-13L emulates a pencil, or a ballpoint pen, or a brush. As shown in Figures 13K-13L, as the contact intensity increases (e.g., as indicated by a darkening of contact intensity 1322 corresponding to contact intensity 1320), the opacity of the mark (e.g., mark 1336) increases. In Figure 13K, when the contact intensity is relatively low (e.g., at contact intensity 1322-d), the opacity of the mark is light (e.g., at landmark 1316-k). In Figure 13L, when the contact intensity is higher (e.g., at contact intensity 1322-e), the opacity of the mark is darker (e.g., at landmark 1316-l).

[0331] FIG. 13M illustrates an example of varying the speed of stylus (e.g., stylus 203) movement during input to adjust a mark (e.g., mark 1338) in response to the change in speed by decreasing the width of the mark as the speed increases. For example, as the speed of stylus 203 movement increases to complete a stroke of mark 1338-a, the width of mark 1338-a decreases. In some embodiments, the mark is adjusted in response to the change in speed by increasing the width of the mark as the speed decreases. For example, as the speed of stylus 203 movement decreases at the end of mark 1338-b, the width of mark 1338-b increases. In some embodiments, the stylus 203 in FIG. 13M emulates a writing brush for Chinese calligraphy (or other script). Alternatively, in some embodiments, the mark is adjusted in response to the change in speed by increasing the width of the mark as the speed increases or decreasing the width of the mark as the speed decreases.

[0332] 13N-13O show an example of varying the difference (in angle) between the directionality and orientation of a stylus (e.g., stylus 203) during input to adjust the width of the mark as the difference between the directionality and orientation changes. In FIGS. 13N-13O, the stylus orientation (e.g., orientation 1306-b), stylus tilt (e.g., tilt 1312-a), and distance of the stylus relative to the touch-sensitive display (e.g., distance 1314-a when the tip of the stylus is in contact with the touch-sensitive display) remain constant, and the stylus directionality changes (e.g., from moving downward at (x,y) location 1304-m to moving right at (x,y) location 1304-n). In some embodiments, stylus 203 in FIGS. 13N-13O emulates a chisel-tip drawing implement (e.g., highlighter, dry erase marker, etc.). In FIG. 13N, when the stylus 203 remains in a first orientation (e.g., about 45 degrees clockwise rotation about the axis 508) and moves in one direction (e.g., about 180 degrees clockwise rotation about the axis 508), the difference (in angles) between the directionality and the orientation is a first difference (e.g., about 135 degrees) and the mark (e.g., mark 1340) has a first width (e.g., a narrow width). In FIG. 13O, when the stylus 203 remains in a first orientation (e.g., about 45 degrees clockwise rotation about the axis 508) and moves in a second direction (e.g., about 90 degrees clockwise rotation about the axis 508), the difference (in angles) between the directionality and the orientation is a second difference (e.g., about 45 degrees) and the mark (e.g., mark 1340) has a second width (e.g., a wide width).

[0333] 14A-14E are flow diagrams illustrating a method 1400 of adjusting one or more characteristics of a mark according to characteristics of input from a stylus, according to some embodiments. Method 1400 is performed on an electronic device (e.g., device 300 of FIG. 3 or portable multifunction device 100 of FIG. 1A) that includes a display and a touch-sensitive surface. In some embodiments, the display is a touchscreen display and the touch-sensitive surface is on or integrated with the display (also referred to simply as a touch-sensitive display). In some embodiments, the display is separate from the touch-sensitive surface. In some embodiments, the device includes one or more sensors that detect signals from a stylus associated with the device. In some embodiments, the stylus includes a tip (or other representative portion). In some embodiments, portions of the stylus other than the tip are used to estimate the position of the tip. Some operations of method 1400 are optionally combined and / or the order of some operations is optionally changed.

[0334] As described below, the method 1400 provides an efficient way to adjust one or more characteristics of a mark (e.g., opacity, width, and / or color) according to characteristics of an input from a stylus (e.g., orientation, direction of movement, tilt, contact strength, and / or speed of movement). The method provides a user experience with a stylus and electronic device that is similar to the user experience of using a physical drawing means on paper. In addition, the method provides drawing capabilities with a stylus and electronic device that exceed those possible using a physical drawing means on paper. The method reduces the number, range, and / or type of inputs from a user when using a stylus to interact with an electronic device, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, the user's ability to input with a stylus faster and more efficiently conserves power and increases the time between battery charges.

[0335] The device detects (1402) input from a stylus. In some embodiments, the input from the stylus includes input from stylus touchdown to stylus liftoff. For example, in Figures 13D-13E, the input from a stylus (e.g., stylus 203) includes input from touchdown of stylus 203 on touch screen 112 to begin making a mark (e.g., mark 1330) to liftoff of stylus 203 after completing the mark.

[0336] The device determines (1404) a number of characteristics of the input from the stylus, including an orientation of the stylus relative to the touch-sensitive surface during input and a direction of movement of the stylus across the touch-sensitive surface during input (the "directionality"). For example, FIG. 13D shows stylus 203 with orientation 1306-b and a downward directionality, and FIG. 13E shows stylus 203 with orientation 1306-b and an upward directionality. Although the orientation does not change in FIGS. 13D-13E, the orientation of the stylus during input is generally not a single orientation. Rather, the orientation typically changes during input. Furthermore, the orientation of the stylus during input is generally not a single direction. Rather, the directionality typically changes during input, as shown in FIGS. 13D-13E. In some embodiments, the characteristics of the input are relative to the touch-sensitive surface (e.g., touch screen 112).

[0337] In some embodiments, characteristics of the input from the stylus (e.g., corresponding to an orientation of the stylus relative to the touch-sensitive surface, a tilt of the stylus relative to the touch-sensitive surface, etc.) are detected according to one or more measurements from the stylus transmitted to the electronic device. For example, the stylus measures the tilt and / or orientation of the stylus and transmits the measurements to the electronic device. Alternatively, in some embodiments, characteristics of the input are detected according to raw outputs from one or more electrodes in the stylus that are sensed by the touch-sensitive surface. For example, the touch-sensitive surface receives raw outputs from one or more electrodes in the stylus and calculates the tilt and / or orientation of the stylus based on the raw outputs.

[0338] In some embodiments, the tilt of the stylus is the orientation of the projection of the stylus onto the touch-sensitive surface (e.g., an orthogonal projection of a line corresponding to the length of the stylus onto the touch-sensitive surface or a line between the projections of two different points of the stylus) with respect to at least one axis in a plane parallel to the touch-sensitive surface (1406). For example, in FIG. 5A , orientation 506 of stylus 203 is the orientation of the projection of stylus 203 onto touch screen 112 with respect to axis 508. In some embodiments, the orientation of the projection of the stylus onto the touch-sensitive surface is with respect to a single axis, such as axis 508 in FIG. 5A (e.g., a single axis having a clockwise rotation angle from this axis ranging from 0 degrees to 360 degrees). Alternatively, in some embodiments, the orientation of the projection ...

Claims

1. 1. An electronic device with a touch-sensitive display, the electronic device including one or more sensors associated with said device for detecting signals from a stylus including a representative portion; the distance of the stylus relative to the touch-sensitive display; the tilt of the stylus relative to the touch-sensitive display, and / or the orientation of the stylus relative to the touch-sensitive display; Detecting a posture of the stylus corresponding to the determining a position on the touch-sensitive display corresponding to the detected orientation state of the stylus; displaying an indicia on the touch-sensitive display at the determined position before the stylus touched the touch-sensitive display according to the pose state of the stylus; detecting a change in the distance, the tilt, and / or the orientation of the stylus before the stylus touches the touch-sensitive display; updating the displayed indicia on the touch-sensitive display in response to detecting the change; A method comprising:

2. 2. The method of claim 1 , wherein the tilt of the stylus is an angle with respect to a normal to a surface of the touch-sensitive display, and the orientation of the stylus is an orientation of a projection of the stylus onto the touch-sensitive display with respect to at least one axis in a plane parallel to the touch-sensitive display.

3. The method of claim 1 or 2, wherein the pose state comprises a position of a projection of the tip of the stylus on the touch-sensitive display.

4. 4. The method of claim 1, wherein the indicia is displayed in a first color when the stylus is a first distance from the touch-sensitive display, and the indicia is displayed in a second color when the stylus is a second distance from the touch-sensitive display that is distinct from the first distance.

5. 5. The method of claim 1, wherein the indicia is a first size when the stylus is a first distance from the touch-sensitive display, and the indicia is a second size when the stylus is a second distance from the touch-sensitive display that is distinct from the first distance.

6. 6. The method of claim 1, wherein the indicia is a first opacity when the stylus is a first distance from the touch-sensitive display, and wherein the indicia is a second opacity when the stylus is a second distance from the touch-sensitive display that is distinct from the first distance.

7. 7. The method of claim 1, wherein when the stylus is positioned on the touch-sensitive display in a first orientation, the marking is displaced on the touch-sensitive display in a first direction relative to a projection of the tip of the stylus, and when the stylus is positioned on the touch-sensitive display in a second orientation distinct from the first orientation, the marking is displaced on the touch-sensitive display in a second direction distinct from the first direction relative to the projection of the tip of the stylus.

8. 8. The method of claim 1, wherein when the stylus has a first tilt on the touch-sensitive display, the marking is displaced on the touch-sensitive display by a first distance relative to an orthogonal projection of the tip of the stylus in a direction parallel to the plane of the touch-sensitive display, and when the stylus has a second tilt on the touch-sensitive display that is distinct from the first tilt, the marking is displaced on the touch-sensitive display by a second distance that is distinct from the first distance relative to the orthogonal projection of the tip of the stylus in a direction parallel to the plane of the touch-sensitive display.

9. The method of claim 8 , wherein each distance does not exceed a predetermined maximum distance in a direction parallel to the plane of the touch-sensitive display.

10. The method of claim 1 , wherein the distance of the stylus away from the touch-sensitive display is calculated to be greater than zero if the stylus is in physical contact with the surface of the touch-sensitive display.

11. Detecting movement of the stylus away from the surface of the touch-sensitive display; in response to detecting the movement of the stylus away from the surface of the touch-sensitive display; updating an appearance of the indicia on the touch-sensitive display based on the movement of the stylus while maintaining display of the indicia according to a determination that the stylus is less than a first distance from a surface of the touch-sensitive display; ceasing to display the indicia on the touch-sensitive display in accordance with determining that the stylus is greater than a second distance from a surface of the touch-sensitive display; and 11. The method of claim 1, comprising:

12. upon determining that the stylus is in physical contact with a surface of the touch-sensitive display, displaying a mark on the touch-sensitive display according to the attitude state; ceasing to display the indicia on the touch-sensitive display; and 12. The method of any one of claims 1 to 11, comprising:

13. upon determining that a representative portion of the stylus is in physical contact with a surface of the touch-sensitive display; 12. The method of claim 1, comprising displaying the indicia on the touch-sensitive display according to the posture state, wherein the position of the indicia on the touch-sensitive display is constrained such that a distance between the indicia and the representative portion of the stylus in a direction parallel to a plane of the touch-sensitive display does not exceed a predetermined maximum distance.

14. 14. The method of claim 1, wherein displaying the indicia on the touch-sensitive display comprises determining a color for the indicia according to a background color of the touch-sensitive display at the determined location.

15. responding to detected movement of the stylus by updating the indicia and ceasing to modify content of an application displayed on the touch-sensitive display before the stylus touches a surface of the touch-sensitive display; responding to detected movement of the stylus by modifying content of the application displayed on the touch-sensitive display while the stylus is touching the surface of the touch-sensitive display; 15. The method of any one of claims 1 to 14, comprising:

16. 16. The method of claim 1, comprising smoothly changing the indicia from a first state to a second state in response to detecting that the stylus is in physical contact with a surface of the touch-sensitive display.

17. 17. The method of claim 1, wherein a first electrode of the stylus detectable by the electronic device is proximate to the tip of the stylus, and a second electrode of the stylus detectable by the electronic device is displaced from the first electrode by a predetermined distance.

18. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs the distance of the stylus relative to the touch-sensitive display; the tilt of the stylus relative to the touch-sensitive display, and / or the orientation of the stylus relative to the touch-sensitive display; Detecting the posture of the stylus corresponding to determining a position on the touch-sensitive display corresponding to the detected orientation state of the stylus; displaying an indicia on the touch-sensitive display at the determined location before the stylus touched the touch-sensitive display according to the pose state of the stylus; detecting a change in the distance, the tilt, and / or the orientation of the stylus before the stylus touches the touch-sensitive display; updating the displayed indicia on the touch-sensitive display in response to detecting the change. An electronic device containing instructions.

19. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the device, cause the device to: the distance of the stylus relative to the touch-sensitive display; the tilt of the stylus relative to the touch-sensitive display, and / or the orientation of the stylus relative to the touch-sensitive display; detecting a posture of the stylus corresponding to the determining a position on the touch-sensitive display corresponding to the detected orientation of the stylus; displaying an indicia on the touch-sensitive display at the determined position before the stylus touched the touch-sensitive display according to the pose state of the stylus; detecting a change in the distance, the tilt, and / or the orientation of the stylus before the stylus touches the touch-sensitive display; updating the displayed indicia on the touch-sensitive display in response to detecting the change. A computer-readable storage medium containing instructions.

20. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; the distance of the stylus relative to the touch-sensitive display; the tilt of the stylus relative to the touch-sensitive display, and / or the orientation of the stylus relative to the touch-sensitive display; means for detecting the attitude of the stylus corresponding to the means for determining a position on the touch-sensitive display corresponding to the detected orientation state of the stylus; means for displaying an indicia on the touch-sensitive display at the determined position before the stylus touched the touch-sensitive display according to the pose of the stylus; means for detecting changes in the distance, the tilt, and / or the orientation of the stylus before the stylus touches the touch-sensitive display; means for updating the displayed indicia on the touch-sensitive display in response to detecting the change; An electronic device comprising:

21. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: an electronic device, the electronic device comprising: a touch-sensitive display; and one or more sensors for detecting signals from a stylus associated with the device; the distance of the stylus relative to the touch-sensitive display; the tilt of the stylus relative to the touch-sensitive display, and / or the orientation of the stylus relative to the touch-sensitive display; means for detecting the attitude of the stylus corresponding to the means for determining a position on the touch-sensitive display corresponding to the detected orientation state of the stylus; means for displaying an indicia on the touch-sensitive display at the determined position before the stylus touched the touch-sensitive display according to the pose of the stylus; means for detecting changes in the distance, the tilt, and / or the orientation of the stylus before the stylus touches the touch-sensitive display; means for updating the displayed indicia on the touch-sensitive display in response to detecting the change; An information processing device comprising:

22. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; 18. An electronic device comprising: one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 1 to 17.

23. 18. A computer-readable storage medium storing one or more programs comprising instructions that, when executed by an electronic device comprising a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the device, cause the device to perform the method of any one of claims 1 to 17.

24. 18. A graphical user interface on an electronic device having a touch-sensitive display, one or more sensors for detecting signals from a stylus associated with the device, a memory, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method of any one of claims 1 to 17.

25. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; means for carrying out the method according to any one of claims 1 to 17; An electronic device comprising:

26. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: an electronic device, the electronic device comprising: a touch-sensitive display; and one or more sensors for detecting signals from a stylus associated with the device; 18. Information processing device comprising means for carrying out the method according to any one of claims 1 to 17.

27. 1. An electronic device comprising: a touch-sensitive display unit configured to display a user interface and receive user contacts; one or more sensor units configured to detect signals from a stylus associated with the device; a processing unit coupled to the touch-sensitive display unit and the one or more sensor units; wherein the processing unit the distance of the stylus relative to the touch-sensitive display unit; the tilt of the stylus relative to the touch-sensitive display unit, and / or the orientation of the stylus relative to the touch-sensitive display unit; Detecting the posture of the stylus corresponding to determining a position on the touch-sensitive display unit corresponding to the detected orientation state of the stylus; enable display of an indicia on the touch-sensitive display unit at the determined position before the stylus touched the touch-sensitive display unit according to the pose state of the stylus; detecting a change in the distance, the tilt, and / or the orientation of the stylus before the stylus touches the touch-sensitive display unit; updating the displayed indicia on the touch-sensitive display unit in response to detecting the change. An electronic device configured to:

28. 28. The electronic device of claim 27, wherein the tilt of the stylus is an angle relative to a normal to a surface of the touch-sensitive display unit, and the orientation of the stylus is an orientation of a projection of the stylus onto the touch-sensitive display unit relative to at least one axis in a plane parallel to the touch-sensitive display unit.

29. 29. The electronic device of claim 27 or 28, wherein the attitude state comprises a position of a projection of the tip of the stylus on the touch-sensitive display unit.

30. 30. The electronic device of claim 27, wherein the indicia is displayed in a first color when the stylus is a first distance from the touch-sensitive display unit, and wherein the indicia is displayed in a second color when the stylus is a second distance from the touch-sensitive display unit that is distinct from the first distance.

31. 31. The electronic device of claim 27, wherein the indicia is a first size when the stylus is a first distance from the touch-sensitive display unit, and wherein the indicia is a second size when the stylus is a second distance from the touch-sensitive display unit that is distinct from the first distance.

32. 32. The electronic device of claim 27, wherein the indicia is a first opacity when the stylus is a first distance from the touch-sensitive display unit, and wherein the indicia is a second opacity when the stylus is a second distance from the touch-sensitive display unit that is distinct from the first distance.

33. 33. The electronic device of claim 27, wherein when the stylus is positioned on the touch-sensitive display unit in a first orientation, the marking is displaced on the touch-sensitive display unit in a first direction relative to a projection of the tip of the stylus, and when the stylus is positioned on the touch-sensitive display unit in a second orientation distinct from the first orientation, the marking is displaced on the touch-sensitive display unit in a second direction distinct from the first direction relative to the projection of the tip of the stylus.

34. 34. The electronic device of claim 27, wherein when the stylus has a first tilt on the touch-sensitive display unit, the marking is displaced on the touch-sensitive display unit by a first distance relative to an orthogonal projection of the tip of the stylus in a direction parallel to the plane of the touch-sensitive display unit, and when the stylus has a second tilt on the touch-sensitive display unit that is distinct from the first tilt, the marking is displaced on the touch-sensitive display unit by a second distance that is distinct from the first distance relative to the orthogonal projection of the tip of the stylus in a direction parallel to the plane of the touch-sensitive display unit.

35. 35. The electronic device of claim 34, wherein each distance does not exceed a predetermined maximum distance in a direction parallel to the plane of the touch-sensitive display unit.

36. 36. The electronic device of claim 27, wherein the distance of the stylus away from the touch-sensitive display unit is calculated to be greater than zero when the stylus is in physical contact with a surface of the touch-sensitive display unit.

37. The processing unit Detecting movement of the stylus away from the surface of the touch-sensitive display unit; in response to detecting the movement of the stylus away from the surface of the touch-sensitive display unit; pursuant to a determination that the stylus is less than a first distance from a surface of the touch-sensitive display unit, updating an appearance of the indicia on the touch-sensitive display unit based on the movement of the stylus while maintaining a display of the indicia; ceasing to enable display of the indicia on the touch-sensitive display unit in accordance with determining that the stylus is greater than a second distance from a surface of the touch-sensitive display unit; 37. An electronic device according to any one of claims 27 to 36, configured so as to

38. The processing unit upon determining that the stylus is in physical contact with a surface of the touch-sensitive display unit; enabling the display of marks on the touch-sensitive display unit according to the attitude state; ceasing to enable display of the indicia on the touch-sensitive display unit; 38. An electronic device according to any one of claims 27 to 37, configured so as to

39. The processing unit upon determining that a representative portion of the stylus is in physical contact with a surface of the touch-sensitive display unit; 38. The electronic device of claim 27, configured to enable display of the indicia on the touch-sensitive display unit according to the attitude state, and wherein the position of the indicia on the touch-sensitive display unit is constrained such that the distance between the indicia and the representative portion of the stylus in a direction parallel to the plane of the touch-sensitive display unit does not exceed a predetermined maximum distance.

40. 40. The electronic device of claim 27, wherein enabling display of the indicia on the touch-sensitive display unit comprises determining a color for the indicia according to a background color of the touch-sensitive display unit at the determined location.

41. The processing unit responding to the detected movement of the stylus by updating the indicia and ceasing to modify content of an application displayed on the touch-sensitive display unit before the stylus touches a surface of the touch-sensitive display unit; responding to detected movement of the stylus by modifying content of the application displayed on the touch-sensitive display unit while the stylus is touching the surface of the touch-sensitive display unit; 41. An electronic device according to any one of claims 27 to 40, configured so as to

42. The processing unit 42. The electronic device of claim 27, configured to smoothly change the indicia from a first state to a second state in response to detecting the stylus in physical contact with a surface of the touch-sensitive display unit.

43. 43. An electronic device according to any one of claims 27 to 42, wherein a first electrode of the stylus detectable by the electronic device is proximate to the tip of the stylus, and a second electrode of the stylus detectable by the electronic device is displaced from the first electrode by a predetermined distance.

44. 1. An electronic device with a touch-sensitive display, the electronic device including one or more sensors associated with the device that detect signals from a stylus including a tip, While the stylus tip is in contact with the touch-sensitive display, the position of the stylus tip on the touch-sensitive display; the tilt of the stylus relative to the touch-sensitive display, and the orientation of the stylus relative to the touch-sensitive display; Detecting a posture of the stylus corresponding to the determining a first position on the touch-sensitive display corresponding to the detected orientation state of the stylus; displaying indicia on the touch-sensitive display at a first position that is a first offset from the position of the stylus tip on the touch-sensitive display and that corresponds to the detected orientation state of the stylus; detecting the change in orientation of the stylus; In response to detecting the change in orientation of the stylus, determining a second position on the touch-sensitive display that is a second offset from the position of the stylus tip on the touch-sensitive display corresponding to the change in orientation of the stylus; displaying the indicia at the second location on the touch-sensitive display; A method comprising:

45. detecting a change in the tilt of the stylus; In response to detecting the change in the tilt of the stylus, determining a third position on the touch-sensitive display that is a third offset from the position of the stylus tip on the touch-sensitive display and corresponds to the change in tilt of the stylus; displaying the indicia at the third location on the touch-sensitive display; 45. The method of claim 44, comprising:

46. 46. ​​A method according to claim 44 or 45, comprising displaying the indicia at two or more different positions on the touch-sensitive display while the position of the stylus tip on the touch-sensitive display remains fixed.

47. 47. The method of any one of claims 44 to 46, wherein the tilt of the stylus is an angle with respect to a normal to a surface of the touch-sensitive display, and the orientation of the stylus is an orientation of a projection of the stylus onto the touch-sensitive display with respect to at least one axis in a plane parallel to the touch-sensitive display.

48. 48. The method of any one of claims 44 to 47, wherein when the stylus is positioned on the touch-sensitive display in a first orientation, the indicium is displaced on the touch-sensitive display in a first direction relative to a projection of the tip of the stylus, and when the stylus is positioned on the touch-sensitive display in a second orientation distinct from the first orientation, the indicium is displaced on the touch-sensitive display in a second direction distinct from the first direction relative to the projection of the tip of the stylus.

49. 49. The method of any one of claims 44 to 48, wherein when the stylus has a first tilt on the touch-sensitive display, the marking is displaced on the touch-sensitive display by a first distance relative to an orthogonal projection of the tip of the stylus in a direction parallel to the plane of the touch-sensitive display, and when the stylus has a second tilt on the touch-sensitive display that is distinct from the first tilt, the marking is displaced on the touch-sensitive display by a second distance that is distinct from the first distance relative to the orthogonal projection of the tip of the stylus in a direction parallel to the plane of the touch-sensitive display.

50. 50. The method of claim 49, wherein each distance does not exceed a predetermined maximum distance in a direction parallel to the plane of the touch-sensitive display.

51. 51. The method of any one of claims 44 to 50, wherein the attitude state comprises a distance of the stylus relative to the touch-sensitive display.

52. 52. The method of claim 51, wherein the distance of the stylus away from the touch-sensitive display is calculated to be greater than zero if the stylus is in physical contact with a surface of the touch-sensitive display.

53. Detecting movement of the stylus away from the surface of the touch-sensitive display; in response to detecting the movement of the stylus away from the surface of the touch-sensitive display; updating an appearance of the indicia on the touch-sensitive display based on the movement of the stylus while maintaining display of the indicia according to a determination that the stylus is less than a first distance from a surface of the touch-sensitive display; ceasing to display the indicia on the touch-sensitive display in accordance with determining that the stylus is greater than a second distance from a surface of the touch-sensitive display; and 53. The method of any one of claims 44 to 52, comprising:

54. upon determining that a representative portion of the stylus is in physical contact with a surface of the touch-sensitive display; displaying the indicia on the touch-sensitive display according to the attitude state; 54. The method of any one of claims 44 to 53, wherein the position of the indicia on the touch-sensitive display is constrained such that the distance between the indicia and the representative portion of the stylus in a direction parallel to the plane of the touch-sensitive display does not exceed a predetermined maximum distance.

55. 55. The method of any one of claims 44 to 54, wherein displaying the indicia on the touch-sensitive display comprises determining a color for the indicia according to a background color of the touch-sensitive display at the determined location.

56. 56. A method according to any one of claims 44 to 55, wherein a first electrode of the stylus detectable by the electronic device is proximate to the tip of the stylus, and a second electrode of the stylus detectable by the electronic device is displaced from the first electrode by a predetermined distance.

57. 1. An electronic device comprising: a touch-sensitive display; one or more sensors associated with the device that detect signals from a stylus including a tip; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs While the stylus tip is in contact with the touch-sensitive display, the position of the stylus tip on the touch-sensitive display; the tilt of the stylus relative to the touch-sensitive display, and the orientation of the stylus relative to the touch-sensitive display; Detecting the posture of the stylus corresponding to determining a first position on the touch-sensitive display corresponding to the detected orientation state of the stylus; displaying indicia on the touch-sensitive display at a first position that is a first offset from the position of the stylus tip on the touch-sensitive display and that corresponds to the detected orientation state of the stylus; Detecting the change in orientation of the stylus; In response to detecting the change in orientation of the stylus, determining a second position on the touch-sensitive display that is a second offset from the position of the stylus tip on the touch-sensitive display corresponding to the change in orientation of the stylus; displaying the indicia at the second location on the touch-sensitive display; An electronic device containing instructions.

58. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a touch-sensitive display and one or more sensors associated with the device that detect signals from a stylus including a tip, cause the device to: While the stylus tip is in contact with the touch-sensitive display, the position of the stylus tip on the touch-sensitive display; the tilt of the stylus relative to the touch-sensitive display, and the orientation of the stylus relative to the touch-sensitive display; detecting a posture of the stylus corresponding to the determining a first position on the touch-sensitive display corresponding to the detected orientation state of the stylus; displaying indicia on the touch-sensitive display at a first position that is a first offset from the position of the stylus tip on the touch-sensitive display and that corresponds to the detected orientation state of the stylus; detecting the change in orientation of the stylus; In response to detecting the change in orientation of the stylus, determining a second position on the touch-sensitive display corresponding to the change in orientation of the stylus, the second position being a second offset from the position of the stylus tip on the touch-sensitive display; displaying the indicia at the second location on the touch-sensitive display; A computer-readable storage medium containing instructions.

59. 1. An electronic device comprising: a touch-sensitive display; one or more sensors associated with the device that detect signals from a stylus including a tip; While the stylus tip is in contact with the touch-sensitive display, the position of the stylus tip on the touch-sensitive display; the tilt of the stylus relative to the touch-sensitive display, and the orientation of the stylus relative to the touch-sensitive display; means for detecting the attitude of the stylus corresponding to the means for determining a first position on the touch-sensitive display corresponding to the detected orientation state of the stylus; means for displaying indicia on the touch-sensitive display at a first position that is a first offset from the position of the stylus tip on the touch-sensitive display and that corresponds to the detected orientation state of the stylus; means for detecting said change in orientation of said stylus; means enabled in response to detecting the change in orientation of the stylus, means for determining a second position on the touch-sensitive display that is a second offset from the position of the stylus tip on the touch-sensitive display and corresponds to the change in orientation of the stylus; means for displaying the indicia at the second location on the touch-sensitive display; and a means comprising: An electronic device comprising:

60. 1. An information processing device for use in an electronic device comprising a touch-sensitive display and one or more sensors associated with said device for detecting signals from a stylus including a tip, said information processing device comprising: While the stylus tip is in contact with the touch-sensitive display, the position of the stylus tip on the touch-sensitive display; the tilt of the stylus relative to the touch-sensitive display, and the orientation of the stylus relative to the touch-sensitive display; means for detecting the attitude of the stylus corresponding to the means for determining a first position on the touch-sensitive display corresponding to the detected orientation state of the stylus; means for displaying indicia on the touch-sensitive display at a first position that is a first offset from the position of the stylus tip on the touch-sensitive display and that corresponds to the detected orientation state of the stylus; means for detecting said change in orientation of said stylus; means enabled in response to detecting the change in orientation of the stylus, means for determining a second position on the touch-sensitive display that is a second offset from the position of the stylus tip on the touch-sensitive display and corresponds to the change in orientation of the stylus; means for displaying the indicia at the second location on the touch-sensitive display; and a means comprising: An information processing device comprising:

61. 1. An electronic device comprising: a touch-sensitive display; one or more sensors associated with the device that detect signals from a stylus including a tip; one or more processors; Memory and One or more programs; 57. An electronic device comprising: one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 44 to 56.

62. 57. A computer-readable storage medium storing one or more programs comprising instructions that, when executed by an electronic device having a touch-sensitive display and one or more sensors associated with the device that detect signals from a stylus including a tip, cause the device to perform the method of any one of claims 44 to 56.

63. 57. A graphical user interface on an electronic device having a touch-sensitive display, one or more sensors for detecting signals from a stylus associated with the device, memory, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method of any one of claims 44 to 56.

64. 1. An electronic device comprising: a touch-sensitive display; one or more sensors associated with the device that detect signals from a stylus including a tip; means for carrying out the method of any one of claims 44 to 56; An electronic device comprising:

65. 1. An information processing device for use in an electronic device comprising a touch-sensitive display and one or more sensors associated with said device for detecting signals from a stylus including a tip, said information processing device comprising:

57. Information processing device comprising means for carrying out the method of any one of claims 44 to 56.

66. 1. An electronic device comprising: a touch-sensitive display unit configured to display a user interface and receive user contacts; one or more sensor units associated with the device and configured to detect signals from a stylus including a tip; a processing unit coupled to the touch-sensitive display unit and the one or more sensor units; wherein the processing unit While the stylus tip is in contact with the touch-sensitive display unit, the position of the stylus tip on the touch-sensitive display unit; the tilt of the stylus relative to the touch-sensitive display unit; and the orientation of the stylus relative to the touch-sensitive display unit; Detecting the posture of the stylus corresponding to determining a first position on the touch-sensitive display unit corresponding to the detected attitude state of the stylus; enabling display, on the touch-sensitive display unit, of the first position indicia corresponding to the detected attitude state of the stylus, the first position indicia being at a first offset from the position of the stylus tip on the touch-sensitive display unit; Detecting the change in orientation of the stylus; In response to detecting the change in orientation of the stylus, determining a second position on the touch-sensitive display unit that is a second offset from the position of the stylus tip on the touch-sensitive display unit corresponding to the change in orientation of the stylus; enabling display of the indicia at the second location on the touch-sensitive display unit; An electronic device configured to:

67. The processing unit detecting a change in the tilt of the stylus; In response to detecting the change in the tilt of the stylus, determining a third position on the touch-sensitive display unit that is a third offset from the position of the stylus tip on the touch-sensitive display unit and corresponds to the change in tilt of the stylus; enabling display of the indicia at the third location on the touch-sensitive display unit; 67. The electronic device of claim 66, configured to:

68. The processing unit 68. An electronic device according to claim 66 or 67, configured to enable display of the indicia at two or more different positions on the touch-sensitive display unit while the position of the stylus tip on the touch-sensitive display unit is in a fixed position.

69. 69. The electronic device of claim 66, wherein the tilt of the stylus is an angle relative to a normal to a surface of the touch-sensitive display unit, and the orientation of the stylus is an orientation of a projection of the stylus onto the touch-sensitive display unit relative to at least one axis in a plane parallel to the touch-sensitive display unit.

70. 70. The electronic device of claim 66, wherein when the stylus is positioned on the touch-sensitive display unit in a first orientation, the marking is displaced in a first direction on the touch-sensitive display unit relative to a projection of the tip of the stylus, and when the stylus is positioned on the touch-sensitive display unit in a second orientation distinct from the first orientation, the marking is displaced in a second direction distinct from the first direction on the touch-sensitive display unit relative to the projection of the tip of the stylus.

71. 71. The electronic device of claim 66, wherein when the stylus has a first tilt on the touch-sensitive display unit, the marking is displaced by a first distance on the touch-sensitive display unit relative to an orthogonal projection of the tip of the stylus in a direction parallel to the plane of the touch-sensitive display unit, and when the stylus has a second tilt on the touch-sensitive display unit that is distinct from the first tilt, the marking is displaced by a second distance on the touch-sensitive display unit in a direction parallel to the plane of the touch-sensitive display unit that is distinct from the first distance relative to the orthogonal projection of the tip of the stylus.

72. 72. The electronic device of claim 71, wherein each distance does not exceed a predetermined maximum distance in a direction parallel to the plane of the touch-sensitive display unit.

73. 73. The electronic device of any one of claims 66 to 72, wherein the attitude state comprises a distance of the stylus relative to the touch-sensitive display.

74. 74. The electronic device of claim 73, wherein the distance of the stylus away from the touch-sensitive display unit is calculated to be greater than zero when the stylus is in physical contact with a surface of the touch-sensitive display unit.

75. The processing unit Detecting movement of the stylus away from the surface of the touch-sensitive display unit; in response to detecting the movement of the stylus away from the surface of the touch-sensitive display unit; pursuant to a determination that the stylus is less than a first distance from a surface of the touch-sensitive display unit, updating an appearance of the indicia on the touch-sensitive display unit based on the movement of the stylus while maintaining a display of the indicia; ceasing to enable display of the indicia on the touch-sensitive display unit in accordance with determining that the stylus is greater than a second distance from a surface of the touch-sensitive display unit; 75. An electronic device according to any one of claims 66 to 74, configured so as to

76. The processing unit upon determining that a representative portion of the stylus is in physical contact with a surface of the touch-sensitive display unit; enabling display of the indicia on the touch-sensitive display unit according to the attitude state; 76. The electronic device of claim 66, wherein the position of the indicia on the touch-sensitive display unit is constrained such that the distance between the indicia and the representative portion of the stylus in a direction parallel to the plane of the touch-sensitive display unit does not exceed a predetermined maximum distance.

77. 77. The electronic device of claim 66, wherein enabling display of the indicia on the touch-sensitive display unit comprises determining a color for the indicia according to a background color of the touch-sensitive display unit at the determined location.

78. 78. An electronic device as described in any one of claims 66 to 77, wherein a first electrode of the stylus detectable by the electronic device is close to the tip of the stylus and a second electrode of the stylus detectable by the electronic device is displaced from the first electrode by a predetermined distance.

79. 1. An electronic device having a touch-sensitive surface and a display, the electronic device including one or more sensors that detect signals from a stylus associated with the device; Detecting input from the stylus; the orientation of the stylus relative to the touch-sensitive surface during the input; and the direction of movement of the stylus across the touch-sensitive surface during the input ("direction"); determining a plurality of characteristics of the input from the stylus, including: In response to detecting the input from the stylus, generating a mark having properties including opacity, width, and / or color, to be displayed on the display; a virtual drawing means emulated by said stylus; and A change in the plurality of characteristics of the input from the stylus during the input. adjusting one or more characteristics of the mark according to A method comprising:

80. 80. The method of claim 79, wherein the orientation of the stylus is an orientation of a projection of the stylus onto the touch-sensitive surface relative to at least one axis in a plane parallel to the touch-sensitive surface.

81. detecting a change in the directionality of the input; responsive to detecting the change in the directionality, adjusting the mark by changing the width of the mark when the directionality changes; 81. The method of claim 79 or 80, comprising:

82. detecting a change in the directionality of the input; responsive to detecting the change in the directionality, adjusting the mark by changing the color of the mark when the directionality changes; 82. The method of any one of claims 79 to 81, comprising:

83. 83. The method of any one of claims 79 to 82, wherein the characteristics of the input from the stylus include tilt of the stylus, which is an angle relative to a normal to a surface of the touch-sensitive surface.

84. detecting a change in the tilt of the stylus; adjusting the mark in response to detecting the change in tilt of the stylus by increasing the width of the mark as the tilt increases; 84. The method of claim 83, comprising:

85. 85. The method of any one of claims 79 to 84, wherein the plurality of characteristics of the input from the stylus includes contact intensity, a parameter corresponding to the intensity of contact of the stylus on the touch-sensitive surface.

86. detecting a change in the contact intensity of the stylus during the input; adjusting the mark in response to detecting the change in the contact intensity by increasing the opacity of the mark as the contact intensity increases; 86. The method of claim 85, comprising:

87. detecting a change in the contact intensity of the stylus; In response to detecting the change in the contact intensity, adjusting the mark by increasing the width of the mark as the contact intensity increases; 87. The method of claim 85 or 86, comprising:

88. detecting a change in the contact intensity of the stylus; In response to detecting the change in the contact intensity, adjusting the mark by changing the color of the mark when the contact intensity changes; 88. The method of any one of claims 85 to 87, comprising:

89. detecting a change in one or more of the characteristics of the input from the stylus; changing the virtual drawing implement emulated by the stylus from a first virtual drawing implement to a second virtual drawing implement in response to detecting the change in the one or more characteristics; 89. The method of any one of claims 79 to 88, comprising:

90. detecting a first change in the plurality of characteristics of the input from the stylus while the virtual drawing implement emulated by the stylus is a first virtual drawing implement and the stylus is not actively engaged in making marks on the display; switching from the first virtual drawing means to a second virtual drawing means different from the first virtual drawing means in response to detecting the first change in the plurality of characteristics of the input from the stylus; detecting a second change in the plurality of characteristics of the input from the stylus while the virtual drawing implement emulated by the stylus is the second virtual drawing implement; in response to detecting the second change in the plurality of characteristics of the input from the stylus; switching from the second virtual drawing means to a third virtual drawing means in accordance with determining that the stylus is not actively participating in making marks on the display; and maintaining selection of the second virtual drawing implement pursuant to a determination that the stylus is actively participating in making marks on the display; and 90. The method of any one of claims 79 to 89, comprising:

91. detecting changes in the speed of movement of the stylus on or near the touch-sensitive surface; adjusting the mark in response to detecting the change in velocity by decreasing the width of the mark when the velocity increases; 91. The method of any one of claims 79 to 90, comprising:

92. 92. The method of any one of claims 79 to 91, wherein adjusting one or more properties of the mark comprises adjusting the width of the mark when the difference between the directionality and the orientation changes.

93. 93. A method according to any one of claims 79 to 92, wherein the virtual drawing tool emulated by the stylus is one of a group of virtual drawing tools, and comprising adjusting the width of the mark for each of four virtual drawing tools in the group of drawing tools in accordance with different characteristics of the input from the stylus.

94. 94. A method according to any one of claims 79 to 93, wherein the virtual drawing tool emulated by the stylus is one of a group of virtual drawing tools, and comprising adjusting the opacity of the mark for each of two virtual drawing tools in the group of drawing tools in accordance with different characteristics of the input from the stylus.

95. 95. A method according to any one of claims 79 to 94, wherein the virtual drawing implement emulated by the stylus is one of a group of virtual drawing implements, and comprising adjusting, for each of two virtual drawing implements in the group of drawing implements, a different characteristic of the mark in response to a change in the same characteristic of the input.

96. 1. An electronic device comprising: The display and a touch-sensitive surface; and one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs Detecting input from the stylus; the orientation of the stylus relative to the touch-sensitive surface during the input; and the direction of movement of the stylus across the touch-sensitive surface during the input ("direction"); determining a plurality of characteristics of the input from the stylus, including: In response to detecting the input from the stylus, generating a mark to be displayed on said display, said mark having properties including opacity, width, and / or color; a virtual drawing means emulated by said stylus; and A change in the plurality of characteristics of the input from the stylus during the input. adjusting one or more characteristics of the mark according to An electronic device containing instructions.

97. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a display, a touch-sensitive surface, and one or more sensors that detect signals from a stylus associated with the device, cause the device to: Detecting input from the stylus; the orientation of the stylus relative to the touch-sensitive surface during the input; and the direction of movement of the stylus across the touch-sensitive surface during the input ("direction"); determining a plurality of characteristics of the input from the stylus, including: In response to detecting the input from the stylus, generating a mark to be displayed on the display, the mark having properties including opacity, width, and / or color; a virtual drawing means emulated by said stylus; and A change in the plurality of characteristics of the input from the stylus during the input. adjusting one or more properties of the mark according to A computer-readable storage medium containing instructions.

98. 1. An electronic device comprising: The display and a touch-sensitive surface; and one or more sensors that detect signals from a stylus associated with the device; means for detecting input from the stylus; the orientation of the stylus relative to the touch-sensitive surface during the input; and the direction of movement of the stylus across the touch-sensitive surface during the input ("direction"); means for determining a plurality of characteristics of the input from the stylus, the plurality of characteristics comprising: means enabled in response to detecting the input from the stylus, means for generating marks to be displayed on said display, said marks having properties including opacity, width, and / or color; a virtual drawing means emulated by said stylus; and A change in the plurality of characteristics of the input from the stylus during the input. means for adjusting one or more characteristics of said mark according to and a means comprising: An electronic device comprising:

99. 1. An information processing apparatus for use in an electronic device, the electronic device comprising a display, a touch-sensitive surface, and one or more sensors that detect signals from a stylus associated with the device, the information processing apparatus comprising: means for detecting input from the stylus; the orientation of the stylus relative to the touch-sensitive surface during the input; and the direction of movement of the stylus across the touch-sensitive surface during the input ("direction"); Including, means for determining a plurality of characteristics of the input from the stylus; means enabled in response to detecting the input from the stylus, means for generating marks to be displayed on said display, said marks having properties including opacity, width, and / or color; a virtual drawing means emulated by said stylus; and A change in the plurality of characteristics of the input from the stylus during the input. means for adjusting one or more characteristics of said mark according to and a means comprising: An information processing device comprising:

100. 1. An electronic device comprising: The display and a touch-sensitive surface; and one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; 96. An electronic device comprising: one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 79 to 95.

101. 96. A computer-readable storage medium storing one or more programs comprising instructions that, when executed by an electronic device comprising a display, a touch-sensitive surface, and one or more sensors that detect signals from a stylus associated with the device, cause the device to perform the method of any one of claims 79 to 95.

102. 96. A graphical user interface on an electronic device having a display, a touch-sensitive surface, one or more sensors that detect signals from a stylus associated with the device, a memory, and one or more processors that execute one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method of any one of claims 79 to 95.

103. 1. An electronic device comprising: The display and a touch-sensitive surface; and one or more sensors that detect signals from a stylus associated with the device; means for carrying out the method of any one of claims 79 to 95; An electronic device comprising:

104. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: a display; a touch-sensitive surface; and one or more sensors for detecting signals from a stylus associated with the device, the information processing apparatus comprising:

96. An information processing device comprising means for carrying out the method of any one of claims 79 to 95.

105. 1. An electronic device comprising: a display unit configured to display a user interface; a touch-sensitive surface unit configured to receive a user's touch; one or more sensor units configured to detect signals from a stylus associated with the device; a processing unit coupled to the display unit, the touch-sensitive surface unit, and the one or more sensor units; wherein the processing unit Detecting input from the stylus; the orientation of the stylus relative to the touch-sensitive surface unit during the input; and the direction of movement of the stylus across the touch-sensitive surface unit during the input ("direction"); determining a plurality of characteristics of the input from the stylus, including: In response to detecting the input from the stylus, generating a mark having characteristics including opacity, width, and / or color, to be displayed on the display unit; a virtual drawing means emulated by said stylus; and a change in the plurality of characteristics of the input from the stylus during the input; adjusting one or more characteristics of the mark according to An electronic device configured to:

106. 106. The electronic device of claim 105, wherein the orientation of the stylus is an orientation of a projection of the stylus onto the touch-sensitive surface unit relative to at least one axis in a plane parallel to the touch-sensitive surface unit.

107. The processing unit detecting a change in the directionality of the input; and adjusting the mark in response to detecting the change in the directionality by changing the width of the mark when the directionality changes.

107. An electronic device according to claim 105 or 106, configured so as to

108. The processing unit detecting a change in the directionality of the input; and adjusting the mark in response to detecting the change in the directionality by changing the color of the mark when the directionality changes.

108. An electronic device according to any one of claims 105 to 107, configured so as to

109. 109. The electronic device of any one of claims 105 to 108, wherein the characteristics of the input from the stylus include tilt of the stylus, which is an angle relative to a normal to a surface of the touch-sensitive surface unit.

110. The processing unit detecting a change in the tilt of the stylus; and adjusting the mark in response to detecting the change in tilt of the stylus by increasing the width of the mark as the tilt increases.

110. The electronic device of claim 109, configured as follows:

111. 111. The electronic device of any one of claims 105 to 110, wherein the plurality of characteristics of the input from the stylus includes contact intensity, a parameter corresponding to the intensity of contact of the stylus on the touch-sensitive surface unit.

112. The processing unit detecting a change in the contact intensity of the stylus during the input; and adjusting the mark in response to detecting the change in the intensity of the contact by increasing the opacity of the mark as the intensity of the contact increases.

112. The electronic device of claim 111 configured as follows:

113. The processing unit detecting a change in the contact intensity of the stylus; and adjusting the mark in response to detecting the change in the contact intensity by increasing the width of the mark as the contact intensity increases.

113. An electronic device according to claim 111 or 112, configured so as to

114. The processing unit detecting a change in the contact intensity of the stylus; and adjusting the mark in response to detecting the change in the contact intensity by changing the color of the mark when the contact intensity changes.

113. An electronic device according to claim 111 or 112, configured so as to

115. The processing unit detecting a change in one or more of the characteristics of the input from the stylus; changing the virtual drawing implement emulated by the stylus from a first virtual drawing implement to a second virtual drawing implement in response to detecting the change in the one or more characteristics; 115. An electronic device according to any one of claims 105 to 114, configured so as to

116. The processing unit detecting a first change in the plurality of characteristics of the input from the stylus while the virtual drawing means emulated by the stylus is a first virtual drawing means and the stylus is not actively involved in making marks on the display unit; switching from the first virtual drawing means to a second virtual drawing means different from the first virtual drawing means in response to detecting the first change in the plurality of characteristics of the input from the stylus; detecting a second change in the plurality of characteristics of the input from the stylus while the virtual drawing implement emulated by the stylus is the second virtual drawing implement; in response to detecting the second change in the plurality of characteristics of the input from the stylus; switching from the second virtual drawing means to a third virtual drawing means in response to determining that the stylus is not actively participating in making marks on the display unit; maintaining selection of the second virtual drawing implement in accordance with a determination that the stylus is actively participating in making marks on the display unit; 116. An electronic device according to any one of claims 105 to 115, configured so as to

117. The processing unit detecting changes in the speed of movement of the stylus on or near the touch-sensitive surface unit; and adjusting the mark in response to detecting a change in the velocity by decreasing the width of the mark when the velocity increases.

117. An electronic device according to any one of claims 105 to 116, configured so as to

118. 118. The electronic device of any one of claims 105 to 117, wherein adjusting one or more characteristics of the mark comprises adjusting the width of the mark when the difference between the directionality and the orientation changes.

119. The virtual drawing means emulated by the stylus is one of a group of virtual drawing means, and the processing unit 119. An electronic device according to any one of claims 105 to 118, configured to adjust the width of the mark for each of four virtual drawing means in the group of drawing means in accordance with different characteristics of the input from the stylus.

120. The virtual drawing means emulated by the stylus is one of a group of virtual drawing means, and the processing unit 120. An electronic device according to any one of claims 105 to 119, configured to adjust the opacity of the mark for each of two virtual drawing means in the group of drawing means in accordance with different characteristics of the input from the stylus.

121. The virtual drawing means emulated by the stylus is one of a group of virtual drawing means, and the processing unit 121. An electronic device according to any one of claims 105 to 120, configured to adjust, for each of two virtual drawing means in the group of drawing means, a distinct characteristic of the mark in response to a change in the same characteristic of the input.

122. 1. An electronic device with a touch-sensitive display, the electronic device including one or more sensors that detect signals from a stylus associated with the device, displaying a user interface of a communications application on the touch-sensitive display; Detecting an input corresponding to a request to create a communication; in response to detecting the input corresponding to the request to create the communication; displaying a keyboard for keying in a text message in accordance with a determination that the input is a finger input; and displaying a drawing canvas for generating a drawing message in accordance with a determination that the input is a stylus input; A method comprising:

123. the input corresponding to the request to create the communication is made within a displayed key entry area having a first size within the user interface of the communication application; the drawing canvas has a second size larger than the key input area having the first size; 123. The method of claim 122, wherein, in accordance with the determination that the input is a stylus input, the drawing canvas having the second size replaces the key input area having the first size.

124. 124. The method of claim 123, wherein display of the key input area having the first size is maintained in accordance with the determination that the input is a finger input.

125. 125. The method of any one of claims 122 to 124, wherein displaying the user interface of the communication application includes displaying a communication session window on the touch-sensitive display, the communication session window displaying an instant message from a first participant in a content balloon and an instant message from the second participant in a content balloon within an instant messaging conversation between the first participant and a second participant.

126. receiving a signal indicating that the second participant is composing an instant message; displaying a keyed message composition indicator within the communication session window in accordance with determining that the second participant is keying the instant message; displaying a drawing message composition indicator in the communication session window distinct from the keyed message composition indicator in accordance with determining that the second participant is drawing the instant message; 126. The method of claim 125, comprising:

127. in response to receiving the instant message created by the second participant corresponding to the keystroke message creation indicator; ceasing to display the keystroke message composition indicator; and displaying a content balloon containing the instant message created by the second participant within the communication session window; 127. The method of claim 126, comprising:

128. In response to receiving the instant message created by the second participant corresponding to the drawing message creation indicator, ceasing to display the drawing message composition indicator; and displaying a content balloon containing an instant message created by the second participant in the communication session window, the instant message including a drawing instant message; 127. The method of claim 126, comprising:

129. 129. The method of claim 128, wherein the displayed content balloon is resized according to the size of the drawn instant message.

130. 129. The method of claim 128, wherein displaying the content balloon containing the drawn instant message includes resizing the drawn instant message to fit within the communication session window on the touch-sensitive display.

131. 131. The method of any one of claims 122 to 130, wherein the drawing canvas is larger than an initial size of a content entry area for key input.

132. detecting a request to enter a key while displaying the drawing canvas for generating the drawing message; In response to detecting the request for key input, replacing the display of the drawing canvas with a display of the content input area for key input; 132. The method of claim 131, comprising:

133. displaying a drawing having one or more marks made in accordance with one or more stylus inputs while displaying the drawing canvas; In response to detecting an input corresponding to a request to transmit the drawing, transmitting the drawing and ceasing to display the drawing canvas; 133. The method of any one of claims 122 to 132, comprising:

134. Inserting a digital image into the drawing canvas; displaying a custom palette of colors based on the colors of the digital image for painting on the digital image; 134. The method of any one of claims 122 to 133, comprising:

135. an electronic device associated with a first individual and having a touch-sensitive display; on the touch-sensitive display; an instant messaging conversation between the first individual and a second individual, displaying a content balloon having instant messages exchanged between the first individual and the second individual; and Input area displaying a user interface of a communication application, including: Detecting a selection of a digital image including a plurality of colors; In response to detecting a selection of the digital image, displaying the digital image in the input area; displaying a color palette for drawing on the digital image in the input area, the color palette including colors based on the plurality of colors contained in the digital image; detecting an input selecting a first color within the color palette; While the first color in the color palette is selected, detecting one or more drawing inputs on the digital image within the input area; drawing on the digital image in the first color according to the one or more drawing inputs; detecting a request to send the digital image in the state depicted above as an instant message to the second person in the instant messaging conversation; In response to detecting the request to transmit the digital image in the state depicted above as an instant message, sending the digital image in the state depicted above as an instant message to the second person in the instant messaging conversation; displaying the digital image in the above-described state on the touch-sensitive display as an instant message within the instant messaging conversation; A method comprising:

136. 136. The method of claim 135, comprising ceasing to display the color palette, including colors based on the plurality of colors contained in the digital image, for drawing on the digital image in the input area in response to detecting the request to send the digital image in the state depicted above as an instant message.

137. an electronic device associated with a first individual and having a touch-sensitive display; on the touch-sensitive display; an instant messaging conversation between the first individual and a second individual, displaying a content balloon having instant messages exchanged between the first individual and the second individual; and Input area displaying a user interface of a communication application, including: receiving a signal indicating that the second individual is composing an instant message; In response to receiving the signal indicating that the second individual is composing an instant message, displaying a keyed message composition indicator within the user interface of the communication application in accordance with determining that the second individual is keying the instant message; displaying a drawing message composition indicator distinct from the keystroke message composition indicator within the user interface of the communication application in accordance with determining that the second individual is drawing the instant message; receiving the instant message created by the second individual; and in response to receiving the instant message created by the second individual corresponding to the displayed message creation indicator; ceasing to display the keystroke message creation indicator or the drawing message creation indicator; displaying a content balloon within the instant messaging conversation that includes the instant message created by the second individual; A method comprising:

138. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs displaying a user interface of a communications application on the touch-sensitive display; Detecting input corresponding to a request to create a communication; in response to detecting the input corresponding to the request to create the communication; displaying a keyboard for keying in a text message in response to a determination that the input is a finger input; displaying a drawing canvas for generating a drawing message in accordance with a determination that the input is a stylus input; An electronic device containing instructions.

139. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the device, cause the device to: displaying a user interface of a communications application on the touch-sensitive display; Detecting input corresponding to a request to create a communication; in response to detecting the input corresponding to the request to create the communication; displaying a keyboard for keying in a text message in response to a determination that the input is a finger input; displaying a drawing canvas for generating a drawing message in accordance with a determination that the input is a stylus input; A computer-readable storage medium containing instructions.

140. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; means for displaying a user interface of a communications application on the touch-sensitive display; means for detecting input corresponding to a request to create a communication; means enabled in response to detecting said input corresponding to said request to create said communication, means for displaying a keyboard for keying in a text message in response to a determination that the input is a finger input; means for displaying a drawing canvas for generating a drawing message in accordance with a determination that the input is a stylus input; and a means comprising: An electronic device comprising:

141. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: an electronic device, the electronic device comprising: a touch-sensitive display; and one or more sensors for detecting signals from a stylus associated with the electronic device; means for displaying a user interface of a communications application on the touch-sensitive display; means for detecting input corresponding to a request to create a communication; means enabled in response to detecting said input corresponding to said request to create said communication, means for displaying a keyboard for keying in a text message in response to a determination that the input is a finger input; means for displaying a drawing canvas for generating a drawing message in accordance with a determination that the input is a stylus input; and a means comprising: An information processing device comprising:

142. an electronic device associated with a first individual, a touch-sensitive display; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs on the touch-sensitive display; an instant messaging conversation between the first individual and a second individual, displaying a content balloon with instant messages exchanged between the first individual and the second individual; and Input area displaying a user interface of a communication application, Detecting a selection of digital images containing multiple colors; In response to detecting a selection of the digital image, Displaying the digital image in the input area; displaying a color palette containing colors based on the plurality of colors contained in the digital image for drawing on the digital image in the input area; detecting an input selecting a first color within the color palette; While the first color in the color palette is selected, detecting one or more drawing inputs on the digital image within the input area; drawing on the digital image in the first color according to the one or more drawing inputs; detecting a request to send the digital image in the state depicted above as an instant message to the second person in the instant messaging conversation; In response to detecting the request to transmit the digital image in the state depicted above as an instant message, sending said digital image in the state depicted above as an instant message to said second person in said instant messaging conversation; displaying the digital image in the depicted state on the touch-sensitive display as an instant message within the instant messaging conversation. An electronic device containing instructions.

143. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device associated with a first individual, the electronic device having a touch-sensitive display, causes the device to: on the touch-sensitive display; an instant messaging conversation between the first individual and a second individual, displaying a content balloon with instant messages exchanged between the first individual and the second individual; and Input area displaying a user interface of a communication application, including detecting a selection of a digital image containing a plurality of colors; In response to detecting a selection of the digital image, Displaying the digital image in the input area; displaying a color palette for drawing on the digital image in the input area, the color palette including colors based on the plurality of colors contained in the digital image; detecting an input selecting a first color within the color palette; While the first color in the color palette is selected, detecting one or more drawing inputs on the digital image within the input area; drawing on the digital image in the first color according to the one or more drawing inputs; detecting a request to send the digital image in the state depicted above as an instant message to the second person in the instant messaging conversation; In response to detecting the request to transmit the digital image in the state depicted above as an instant message, sending said digital image in the state depicted above as an instant message to said second person in said instant messaging conversation; displaying, on the touch-sensitive display, the digital image in the state depicted above as an instant message within the instant messaging conversation; A computer-readable storage medium containing instructions.

144. an electronic device associated with a first individual, a touch-sensitive display; on the touch-sensitive display; an instant messaging conversation between the first individual and a second individual, displaying a content balloon having instant messages exchanged between the first individual and the second individual; and Input area means for displaying a user interface of a communication application, the means including: means for detecting a selection of a digital image including a plurality of colors; means, enabled in response to detecting a selection of said digital image, comprising: means for displaying the digital image within the input area; means for displaying a color palette containing colors based on the plurality of colors contained in the digital image for drawing on the digital image in the input area; and a means comprising: means for detecting an input selecting a first color within the color palette; a means, enabled while the first color in the color palette is selected, comprising: means for detecting one or more drawing inputs on the digital image within the input area; means for drawing in the first color on the digital image according to the one or more drawing inputs; and a means comprising: means for detecting a request to send the digital image in the state depicted above as an instant message to the second person in the instant messaging conversation; means, enabled in response to detecting said request to transmit said digital image in the state depicted above as an instant message, comprising: means for transmitting said digital image in the above-illustrated state as an instant message to said second person in said instant messaging conversation; means for displaying the digital image in the depicted state on the touch-sensitive display as an instant message within the instant messaging conversation; and a means comprising: An electronic device comprising:

145. 1. An information processing apparatus for use in an electronic device associated with a first individual, the electronic device having a touch-sensitive display, the information processing apparatus comprising: on the touch-sensitive display; an instant messaging conversation between the first individual and a second individual, displaying a content balloon with instant messages exchanged between the first individual and the second individual; and Input area means for displaying a user interface of a communication application, the means including: means for detecting a selection of a digital image including a plurality of colors; means, enabled in response to detecting a selection of said digital image, comprising: means for displaying the digital image within the input area; means for displaying a color palette containing colors based on the plurality of colors contained in the digital image for drawing on the digital image in the input area; and a means comprising: means for detecting an input selecting a first color within the color palette; a means, enabled while the first color in the color palette is selected, comprising: means for detecting one or more drawing inputs on the digital image within the input area; means for drawing in the first color on the digital image according to the one or more drawing inputs; and a means comprising: means for detecting a request to send the digital image in the state depicted above as an instant message to the second person in the instant messaging conversation; means, enabled in response to detecting said request to transmit said digital image in the state depicted above as an instant message, comprising: means for transmitting said digital image in the above-illustrated state as an instant message to said second person in said instant messaging conversation; means for displaying the digital image in the depicted state on the touch-sensitive display as an instant message within the instant messaging conversation; and a means comprising: An information processing device comprising:

146. an electronic device associated with a first individual, a touch-sensitive display; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs on the touch-sensitive display; an instant messaging conversation between the first individual and a second individual, displaying a content balloon having instant messages exchanged between the first individual and the second individual; and Input area displaying a user interface of a communication application, receiving a signal indicating that the second individual is composing an instant message; In response to receiving the signal indicating that the second individual is composing an instant message, displaying a keyed message composition indicator within the user interface of the communication application in response to determining that the second individual is keying the instant message; displaying a drawing message composition indicator distinct from the keystroke message composition indicator within the user interface of the communication application in response to determining that the second individual is drawing the instant message; receiving the instant message created by the second individual; in response to receiving the instant message created by the second individual corresponding to the displayed message creation indicator; ceasing to display the key input message creation indicator or the drawing message creation indicator; displaying a content balloon within the instant messaging conversation that includes the instant message created by the second individual; An electronic device containing instructions.

147. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device associated with a first individual, the electronic device having a touch-sensitive display, causes the device to: on the touch-sensitive display; an instant messaging conversation between the first individual and a second individual, displaying a content balloon having instant messages exchanged between the first individual and the second individual; and Input area displaying a user interface of a communication application, including receiving a signal indicating that the second individual is composing an instant message; In response to receiving the signal indicating that the second individual is composing an instant message, displaying a keyed message composition indicator within the user interface of the communication application in response to determining that the second individual is keying the instant message; displaying a drawing message composition indicator distinct from the keystroke message composition indicator within the user interface of the communication application in response to determining that the second individual is drawing the instant message; receiving the instant message created by the second individual; in response to receiving the instant message created by the second individual corresponding to the displayed message creation indicator; ceasing the display of the key input message creation indicator or the drawing message creation indicator; displaying a content balloon containing the instant message created by the second individual within the instant messaging conversation; A computer-readable storage medium containing instructions.

148. an electronic device associated with a first individual, a touch-sensitive display; on the touch-sensitive display; an instant messaging conversation between the first individual and a second individual, displaying a content balloon having instant messages exchanged between the first individual and the second individual; and Input area means for displaying a user interface of a communication application, the means including: means for receiving a signal indicating that the second individual is composing an instant message; a means, activated in response to receiving the signal indicating that the second individual is composing an instant message, comprising: means for displaying a keyed message composition indicator within the user interface of the communication application in accordance with a determination that the second individual is keying the instant message; means for displaying a drawing message composition indicator distinct from the keystroke message composition indicator within the user interface of the communication application in accordance with a determination that the second individual is drawing the instant message; and a means comprising: means for receiving the instant message created by the second individual; a means, activated in response to receiving the instant message created by the second individual corresponding to the displayed message creation indicator, means for ceasing display of the key input message creation indicator or the drawing message creation indicator; means for displaying a content balloon within the instant messaging conversation that includes the instant message created by the second individual; and a means comprising: An electronic device comprising:

149. 1. An information processing apparatus for use in an electronic device associated with a first individual, the electronic device having a touch-sensitive display, the information processing apparatus comprising: on the touch-sensitive display; an instant messaging conversation between the first individual and a second individual, displaying a content balloon having instant messages exchanged between the first individual and the second individual; and Input area means for displaying a user interface of a communication application, the means including: means for receiving a signal indicating that the second individual is composing an instant message; a means, activated in response to receiving the signal indicating that the second individual is composing an instant message, comprising: means for displaying a keyed message composition indicator within the user interface of the communication application in accordance with a determination that the second individual is keying the instant message; means for displaying a drawing message composition indicator distinct from the keystroke message composition indicator within the user interface of the communication application in accordance with a determination that the second individual is drawing the instant message; and a means comprising: means for receiving the instant message created by the second individual; a means, activated in response to receiving the instant message created by the second individual corresponding to the displayed message creation indicator, means for ceasing display of the key input message creation indicator or the drawing message creation indicator; means for displaying a content balloon within the instant messaging conversation that includes the instant message created by the second individual; and a means comprising: An information processing device comprising:

150. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; 135. An electronic device comprising: one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing or controlling the performance of a method according to any one of claims 122 to 134.

151. 135. A computer-readable storage medium storing one or more programs comprising instructions that, when executed by an electronic device having a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the electronic device, cause the device to perform the method of any one of claims 122 to 134.

152. 135. A graphical user interface on an electronic device having a touch-sensitive display, one or more sensors that detect signals from a stylus associated with the electronic device, a memory, and one or more processors that execute one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method of any one of claims 122 to 134.

153. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; means for performing or controlling the performance of a method according to any one of claims 122 to 134; An electronic device comprising:

154. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: an electronic device, the electronic device comprising: a touch-sensitive display; and one or more sensors for detecting signals from a stylus associated with the electronic device; 135. Information processing device comprising means for performing or controlling the performance of a method according to any one of claims 122 to 134.

155. an electronic device associated with a first individual, a touch-sensitive display; one or more processors; Memory and One or more programs; 138. An electronic device comprising: one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing or controlling the performance of a method according to any one of claims 135 to 137.

156. 138. A computer-readable storage medium storing one or more programs comprising instructions that, when executed by an electronic device associated with a first individual and having a touch-sensitive display, cause the device to perform the method of any one of claims 135 to 137.

157. 138. A graphical user interface on an electronic device associated with a first individual, the graphical user interface comprising a touch-sensitive display, a memory, and one or more processors executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method of any one of claims 135 to 137.

158. an electronic device associated with a first individual, a touch-sensitive display; means for performing or controlling the performance of a method according to any one of claims 135 to 137; An electronic device comprising:

159. 1. An information processing apparatus for use in an electronic device associated with a first individual, the electronic device having a touch-sensitive display, the information processing apparatus comprising:

138. Information processing device comprising means for performing or controlling the performance of a method according to any one of claims 135 to 137.

160. 1. An electronic device comprising: a touch-sensitive display unit configured to display a user interface and receive user contacts; one or more sensor units configured to detect signals from a stylus associated with the device; a processing unit coupled to the touch-sensitive display unit and the one or more sensor units; wherein the processing unit enabling display of a user interface of a communications application on the touch-sensitive display unit; Detecting input corresponding to a request to create a communication; in response to detecting the input corresponding to the request to create the communication; In response to determining that the input is a finger input, enabling display of a keyboard for keying in a text message; enabling a display of a drawing canvas for generating drawing messages in accordance with a determination that the input is a stylus input; An electronic device configured to:

161. the input corresponding to the request to create the communication is made within a displayed key entry area having a first size within the user interface of the communication application; the drawing canvas has a second size larger than the key input area having the first size; 161. The electronic device of claim 160, wherein, in accordance with the determination that the input is a stylus input, the drawing canvas having the second size replaces the key input area having the first size.

162. 162. The electronic device of claim 161, wherein display of the key input area having the first size is maintained in accordance with the determination that the input is a finger input.

163. 163. The electronic device of any one of claims 160 to 162, wherein enabling the display of the user interface of the communication application includes enabling the display of a communication session window on the touch-sensitive display unit, the communication session window displaying an instant message from a first participant in a content balloon and an instant message from the second participant in a content balloon within an instant messaging conversation between the first participant and a second participant.

164. The processing unit receiving a signal indicating that the second participant is composing an instant message; enable display of a keyed message composition indicator within the communication session window in response to determining that the second participant is keying the instant message; enabling display of a drawing message composition indicator separate from the keystroke message composition indicator within the communication session window in response to determining that the second participant is drawing the instant message; 164. The electronic device of claim 163, configured as follows:

165. The processing unit in response to receiving the instant message created by the second participant corresponding to the keystroke message creation indicator; ceasing to enable display of said keystroke message composition indicator; enabling the display of a content balloon containing the instant message created by the second participant within the communication session window; 165. The electronic device of claim 164 configured as follows:

166. The processing unit In response to receiving the instant message created by the second participant corresponding to the drawing message creation indicator, ceasing to enable display of said drawing message composition indicator; enabling display of a content balloon containing the instant message, including a drawing instant message, created by the second participant within the communication session window; 165. The electronic device of claim 164 configured as follows:

167. 167. The electronic device of claim 166, wherein the displayed content balloon is resized according to the size of the drawn instant message.

168. 167. The electronic device of claim 166, wherein enabling display of the content balloon containing the drawn instant message comprises resizing the drawn instant message to fit within the communication session window on the touch-sensitive display unit.

169. 169. The electronic device of any one of claims 160 to 168, wherein the drawing canvas is larger than an initial size of a content entry area for key input.

170. The processing unit Detecting a request to enter a key while enabling the display of the drawing canvas for generating the drawing message; In response to detecting the request for key input, replacing the display of the drawing canvas with a display of the content input area for key input.

170. The electronic device of claim 169, configured as follows:

171. The processing unit While enabling display of the drawing canvas, enabling display of a drawing having one or more marks made in accordance with one or more stylus inputs; In response to detecting an input corresponding to a request to transmit the drawing, transmitting the drawing and ceasing to enable display of the drawing canvas.

171. An electronic device according to any one of claims 160 to 170, configured so as to

172. The processing unit Inserting a digital image into the drawing canvas; enabling the display of a custom palette of colors based on the colors of the digital image for painting on the digital image; 172. An electronic device according to any one of claims 160 to 171, configured so as to

173. an electronic device associated with a first individual, a touch-sensitive display unit configured to display a user interface and receive user contacts; a processing unit coupled to the touch-sensitive display unit; and wherein the processing unit on the touch-sensitive display unit; an instant messaging conversation between the first individual and a second individual, displaying a content balloon having instant messages exchanged between the first individual and the second individual; and Input area enabling the display of a user interface of a communication application, including Detecting a selection of digital images containing multiple colors; In response to detecting a selection of the digital image, enabling display of said digital image within said input area; enabling the display of a color palette for drawing on the digital image in the input area, the color palette including colors based on the plurality of colors contained in the digital image; detecting an input selecting a first color within the color palette; While the first color in the color palette is selected, detecting one or more drawing inputs on the digital image within the input area; drawing on the digital image in the first color according to the one or more drawing inputs; detecting a request to send the digital image in the state depicted above as an instant message to the second person in the instant messaging conversation; In response to detecting the request to transmit the digital image in the state depicted above as an instant message, sending said digital image in the state depicted above as an instant message to said second person in said instant messaging conversation; enabling a display of the digital image in the state depicted above on the touch-sensitive display unit as an instant message within the instant messaging conversation; An electronic device configured to:

174. 174. The electronic device of claim 173, wherein the processing unit is configured to cease enabling the display of the color palette, including colors based on the plurality of colors contained in the digital image, for drawing on the digital image in the input area in response to detecting the request to send the digital image in the state depicted thereon as an instant message.

175. an electronic device associated with a first individual, a touch-sensitive display unit configured to display a user interface and receive user contacts; a processing unit coupled to the touch-sensitive display unit; and wherein the processing unit on the touch-sensitive display unit; an instant messaging conversation between the first individual and a second individual, displaying a content balloon with instant messages exchanged between the first individual and the second individual; and Input area enabling the display of a user interface of a communication application, including receiving a signal indicating that the second individual is composing an instant message; In response to receiving the signal indicating that the second individual is composing an instant message, In response to determining that the second individual is keying the instant message, enable display of a keyed message composition indicator within the user interface of the communication application; In response to determining that the second individual is drawing the instant message, enable display of a drawing message composition indicator within the user interface of the communication application, the drawing message composition indicator being distinct from the keystroke message composition indicator; receiving the instant message created by the second individual; in response to receiving the instant message created by the second individual corresponding to the displayed message creation indicator; ceasing to enable display of the key input message creation indicator or the drawing message creation indicator; enabling display of a content balloon within the instant messaging conversation that includes the instant message created by the second individual; An electronic device configured to:

176. 1. An electronic device with a touch-sensitive display, the electronic device including one or more sensors that detect signals from a stylus associated with the device, While the electronic device is in a locked state, displaying a lock screen user interface on the touch-sensitive display; detecting a first input from the stylus drawing on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display; In response to detecting the first input from the drawing stylus, displaying on the lock screen user interface one or more marks of drawing corresponding to the first input from the stylus; while displaying, on the lock screen user interface, the one or more marks of the drawing corresponding to the first input from the stylus; detecting a second input from the stylus displaying a drawing application in a limited mode; responsive to detecting the second input from the stylus displaying the drawing application, executing the drawing application in the restricted mode and displaying the one or more marks of the drawing within the drawing application; A method comprising:

177. 177. The method of claim 176, wherein the second input from the stylus displaying the drawing application comprises swiping from an edge of the touch-sensitive display with the stylus.

178. 177. The method of claim 176, wherein the second input from the stylus displaying the drawing application comprises tapping the touch-sensitive display with the stylus.

179. 177. The method of claim 176, wherein the second input from the stylus displaying the drawing application comprises a liftoff of the stylus from the touch-sensitive display.

180. 180. The method of any one of claims 176 to 179, comprising, while running the drawing application in the restricted mode, enabling a user to access content generated during a current limited session of the drawing application without providing access to content generated during an unlimited session of the drawing application and to content generated during a previous limited session of the drawing application.

181. 1. An electronic device with a touch-sensitive display, the electronic device including one or more sensors that detect signals from a stylus associated with the device, While the electronic device is in a locked state, displaying a lock screen user interface on the touch-sensitive display; Detecting a user input while displaying the lock screen user interface on the touch-sensitive display; replacing a display of the lock screen user interface with a display of a drawing application in a restricted session in accordance with a determination that the user input is a default input with the stylus; maintaining display of the lock screen user interface and ceasing display of the drawing application in accordance with determining that the user input is a finger touch input; A method comprising:

182. the user input is a tap gesture; displaying the drawing application in accordance with a determination that the tap gesture was performed with a stylus; ceasing to display the drawing application in accordance with a determination that the tap gesture was performed with a finger; and 182. The method of claim 181, comprising:

183. the user input is a swipe gesture; displaying the drawing application according to a determination that the swipe gesture was performed with a stylus and satisfies a drawing application display criterion; displaying an unlock user interface in accordance with a determination that the swipe gesture is performed with a finger and satisfies an unlock initiation criterion; 182. The method of claim 181, comprising:

184. While the electronic device is in the locked state, detecting a first input from the stylus drawing on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display; In response to detecting the first input from the drawing stylus, displaying on the lock screen user interface one or more marks of drawing corresponding to the first input from the stylus; 184. The method of any one of claims 181 to 183, comprising:

185. while displaying, on the lock screen user interface, the one or more marks of the drawing corresponding to the first input from the stylus; detecting a second input from the stylus displaying a drawing application in a limited mode; responsive to detecting the second input from the stylus displaying the drawing application, executing the drawing application in the restricted mode and displaying the one or more marks of the drawing within the drawing application; 185. The method of claim 184, comprising:

186. 1. An electronic device with a touch-sensitive display, the electronic device including one or more sensors that detect signals from a stylus associated with the device, While the electronic device is in a locked state, displaying a lock screen user interface on the touch-sensitive display, the lock screen user interface including a lock screen image; detecting a first input by the stylus on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display; Detecting an initial contact by the stylus on the touch-sensitive display at the initiation of the first input; Detecting movement of the stylus across the touch-sensitive display during the first input; detecting lift-off of the stylus from the touch-sensitive display at the end of the first input; Detecting a first input, comprising: In response to detecting the movement of the stylus across the touch-sensitive display, drawing a first mark according to the movement of the stylus across the touch-sensitive display, the first mark being displayed on the lock screen image; in response to detecting the lift-off of the stylus from the touch-sensitive display; ceasing to display the lock screen user interface, including ceasing to display the lock screen image; and Displaying a user interface for a drawing application in a restricted session; displaying the first mark drawn in accordance with the movement of the stylus across the touch-sensitive display during the first input within the user interface for the drawing application in the limited session; detecting a second input by the stylus on the touch-sensitive display while displaying the user interface for the drawing application in the limited session; In response to detecting the second input by the stylus on the touch-sensitive display, drawing a second mark in the user interface for the drawing application in the limited session, the second mark being displayed together with the first mark; A method comprising:

187. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs While the electronic device is in a locked state, displaying a lock screen user interface on the touch-sensitive display; detecting a first input from the stylus drawing on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display; In response to detecting the first input from the drawing stylus, displaying on the lock screen user interface one or more marks of drawing corresponding to the first input from the stylus; while displaying, on the lock screen user interface, the one or more marks of the drawing corresponding to the first input from the stylus; Detecting a second input from the stylus displaying a drawing application in a restricted mode; executing the drawing application in the restricted mode in response to detecting the second input from the stylus displaying the drawing application and displaying the one or more marks of the drawing within the drawing application; An electronic device containing instructions.

188. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the device, cause the device to: While the electronic device is in a locked state, displaying a lock screen user interface on the touch-sensitive display; detecting a first input from the stylus drawing on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display; In response to detecting the first input from the drawing stylus, displaying one or more marks on the lock screen user interface that correspond to the first input from the stylus; while displaying, on the lock screen user interface, the one or more marks of the drawing corresponding to the first input from the stylus; detecting a second input from the stylus that displays a drawing application in a limited mode; In response to detecting the second input from the stylus displaying the drawing application, causing the drawing application to run in the restricted mode and displaying the one or more marks of the drawing within the drawing application. A computer-readable storage medium containing instructions.

189. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; a means enabled while the electronic device is in a locked state, the means comprising: means for displaying a lock screen user interface on the touch-sensitive display; means for detecting a first input from the stylus drawing on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display; means for displaying, on the lock screen user interface, one or more marks of a drawing corresponding to the first input from the stylus in response to detecting the first input from the drawing stylus; a means, activated while displaying on the lock screen user interface the one or more marks of the drawing corresponding to the first input from the stylus, means for detecting a second input from the stylus that displays a drawing application in a restricted mode; means, enabled in response to detecting the second input from the stylus displaying the drawing application, means for executing the drawing application in the restricted mode; means for displaying the one or more marks of the drawing within the drawing application; and a means comprising: and a means comprising: and a means comprising: An electronic device comprising:

190. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: an electronic device, the electronic device comprising: a touch-sensitive display; and one or more sensors for detecting signals from a stylus associated with the electronic device; a means enabled while the electronic device is in a locked state, the means comprising: means for displaying a lock screen user interface on the touch-sensitive display; means for detecting a first input from the stylus drawing on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display; means for displaying, on the lock screen user interface, one or more marks of a drawing corresponding to the first input from the stylus in response to detecting the first input from the drawing stylus; a means, activated while displaying on the lock screen user interface the one or more marks of the drawing corresponding to the first input from the stylus, means for detecting a second input from the stylus that displays a drawing application in a restricted mode; means, enabled in response to detecting the second input from the stylus displaying the drawing application, means for executing the drawing application in the restricted mode; means for displaying the one or more marks of the drawing within the drawing application; and a means comprising: and a means comprising: An information processing device comprising:

191. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs While the electronic device is in a locked state, displaying a lock screen user interface on the touch-sensitive display; Detecting a user input while displaying the lock screen user interface on the touch-sensitive display; replacing a display of the lock screen user interface with a display of a drawing application in a restricted session in response to a determination that the user input is a default input with the stylus; maintaining display of the lock screen user interface and ceasing display of the drawing application in accordance with a determination that the user input is a finger touch input; An electronic device containing instructions.

192. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the device, cause the device to: While the electronic device is in a locked state, displaying a lock screen user interface on the touch-sensitive display; detecting a user input while displaying the lock screen user interface on the touch-sensitive display; In response to determining that the user input is a default input with the stylus, replacing the display of the lock screen user interface with a display of a drawing application in a restricted session; maintaining display of the lock screen user interface and ceasing display of the drawing application in accordance with a determination that the user input is a finger touch input; A computer-readable storage medium containing instructions.

193. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; a means enabled while the electronic device is in a locked state, the means comprising: means for displaying a lock screen user interface on the touch-sensitive display; means for detecting user input while displaying the lock screen user interface on the touch-sensitive display; means for replacing a display of the lock screen user interface with a display of a drawing application in a restricted session in response to a determination that the user input is a default input with the stylus; a means, enabled in accordance with a determination that the user input is a finger touch input, comprising: means for maintaining a display of the lock screen user interface; means for canceling the display of the drawing application; and a means comprising: and a means comprising: An electronic device comprising:

194. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: an electronic device, the electronic device comprising: a touch-sensitive display; and one or more sensors for detecting signals from a stylus associated with the electronic device; a means enabled while the electronic device is in a locked state, the means comprising: means for displaying a lock screen user interface on the touch-sensitive display; means for detecting user input while displaying the lock screen user interface on the touch-sensitive display; means for replacing a display of the lock screen user interface with a display of a drawing application in a restricted session in response to a determination that the user input is a default input with the stylus; a means, enabled in accordance with a determination that the user input is a finger touch input, comprising: means for maintaining a display of the lock screen user interface; means for canceling the display of the drawing application; and a means comprising: An information processing device comprising:

195. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs While the electronic device is in a locked state, displaying a lock screen user interface on the touch-sensitive display, the lock screen user interface including a lock screen image; a first input by the stylus on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display; Detecting an initial contact by the stylus on the touch-sensitive display at the initiation of the first input; Detecting movement of the stylus across the touch-sensitive display during the first input; detecting lift-off of the stylus from the touch-sensitive display at the end of the first input; Detect, including in response to detecting the movement of the stylus across the touch-sensitive display, drawing a first mark in accordance with the movement of the stylus across the touch-sensitive display, the first mark being displayed on the lock screen image; in response to detecting the lift-off of the stylus from the touch-sensitive display; ceasing to display the lock screen user interface, including ceasing to display the lock screen image; Displaying a user interface for a drawing application in a restricted session; displaying the first mark drawn in accordance with the movement of the stylus across the touch-sensitive display during the first input within the user interface for the drawing application in the limited session; detecting a second input by the stylus on the touch-sensitive display while displaying the user interface for the drawing application in the limited session; In response to detecting the second input by the stylus on the touch-sensitive display, drawing a second mark in the user interface for the drawing application in the limited session, the second mark being displayed together with the first mark. An electronic device containing instructions.

196. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the device, cause the device to: While the electronic device is in a locked state, displaying a lock screen user interface on the touch-sensitive display, the lock screen user interface including a lock screen image; a first input by the stylus on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display; Detecting an initial contact by the stylus on the touch-sensitive display at the initiation of the first input; Detecting movement of the stylus across the touch-sensitive display during the first input; detecting lift-off of the stylus from the touch-sensitive display at the end of the first input; , and in response to detecting the movement of the stylus across the touch-sensitive display, causing a first mark to be drawn in accordance with the movement of the stylus across the touch-sensitive display, the first mark being displayed on the lock screen image; in response to detecting the lift-off of the stylus from the touch-sensitive display; ceasing to display the lock screen user interface, including ceasing to display the lock screen image; Displaying a user interface for a drawing application in a restricted session; displaying the first mark drawn in accordance with the movement of the stylus across the touch-sensitive display during the first input within the user interface for the drawing application in the limited session; detecting a second input by the stylus on the touch-sensitive display while displaying the user interface for the drawing application in the limited session; in response to detecting the second input by the stylus on the touch-sensitive display, drawing a second mark in the user interface for the drawing application in the limited session, the second mark being displayed together with the first mark. A computer-readable storage medium containing instructions.

197. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; a means enabled while the electronic device is in a locked state, the means comprising: means for displaying, on the touch-sensitive display, a lock screen user interface including a lock screen image; means for detecting a first input by the stylus on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display, the means comprising: means for detecting an initial contact by the stylus on the touch-sensitive display at the initiation of the first input; means for detecting movement of the stylus across the touch-sensitive display during the first input; means for detecting lift-off of the stylus from the touch-sensitive display at the end of the first input; means for detecting a first input, the means comprising: means for drawing a first mark in response to detecting the movement of the stylus across the touch-sensitive display, the first mark being displayed on the lock screen image in accordance with the movement of the stylus across the touch-sensitive display; means enabled in response to detecting the lift-off of the stylus from the touch-sensitive display, means for ceasing displaying the lock screen user interface, including ceasing displaying the lock screen image; means for displaying a user interface for a drawing application in a restricted session; means for displaying, within the user interface for the drawing application in the limited session, the first mark drawn in accordance with the movement of the stylus across the touch-sensitive display during the first input; and a means comprising: means for detecting a second input by the stylus on the touch-sensitive display while displaying the user interface for the drawing application in the limited session; means for drawing a second mark in the user interface for the drawing application in the limited session in response to detecting the second input by the stylus on the touch-sensitive display, the second mark being displayed together with the first mark; and a means comprising: An electronic device comprising:

198. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: an electronic device, the electronic device comprising: a touch-sensitive display; and one or more sensors for detecting signals from a stylus associated with the electronic device; a means enabled while the electronic device is in a locked state, the means comprising: means for displaying, on the touch-sensitive display, a lock screen user interface including a lock screen image; means for detecting a first input by the stylus on the touch-sensitive display while displaying the lock screen user interface on the touch-sensitive display, the means comprising: means for detecting an initial contact by the stylus on the touch-sensitive display at the initiation of the first input; means for detecting movement of the stylus across the touch-sensitive display during the first input; means for detecting lift-off of the stylus from the touch-sensitive display at the end of the first input; means for detecting a first input, the means comprising: means for drawing a first mark in response to detecting the movement of the stylus across the touch-sensitive display, the first mark being displayed on the lock screen image in accordance with the movement of the stylus across the touch-sensitive display; means enabled in response to detecting the lift-off of the stylus from the touch-sensitive display, means for ceasing displaying the lock screen user interface, including ceasing displaying the lock screen image; means for displaying a user interface for a drawing application in a restricted session; means for displaying, within the user interface for the drawing application in the limited session, the first mark drawn in accordance with the movement of the stylus across the touch-sensitive display during the first input; and a means comprising: means for detecting a second input by the stylus on the touch-sensitive display while displaying the user interface for the drawing application in the limited session; means for drawing a second mark in the user interface for the drawing application in the limited session in response to detecting the second input by the stylus on the touch-sensitive display, the second mark being displayed together with the first mark; An information processing device comprising:

199. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; 187. An electronic device comprising: one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for performing or controlling the performance of a method according to any one of claims 176 to 186.

200. 187. A computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by an electronic device having a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the electronic device, cause the device to perform the method of any one of claims 176 to 186.

201. 187. A graphical user interface on an electronic device having a touch-sensitive display, one or more sensors that detect signals from a stylus associated with the electronic device, a memory, and one or more processors that execute one or more programs stored in the memory, the graphical user interface comprising a user interface displayed in accordance with the method of any one of claims 176 to 186.

202. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; means for performing or controlling the performance of a method according to any one of claims 176 to 186; An electronic device comprising:

203. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: an electronic device, the electronic device comprising: a touch-sensitive display; and one or more sensors for detecting signals from a stylus associated with the electronic device; 187. An information processing device comprising means for performing or controlling the performance of a method according to any one of claims 176 to 186.

204. 1. An electronic device comprising: a touch-sensitive display unit configured to display a user interface and receive user contacts; one or more sensor units configured to detect signals from a stylus associated with the device; a processing unit coupled to the touch-sensitive display unit and the one or more sensor units; wherein the processing unit While the electronic device is in a locked state, enabling display of a lock screen user interface on the touch-sensitive display unit; detecting a first input from the stylus drawing on the touch-sensitive display unit while enabling display of the lock screen user interface on the touch-sensitive display unit; In response to detecting the first input from the drawing stylus, enable displaying, on the lock screen user interface, one or more marks of drawing corresponding to the first input from the stylus; while enabling display, on the lock screen user interface, of the one or more marks of the drawing corresponding to the first input from the stylus; Detecting a second input from the stylus displaying a drawing application in a restricted mode; responsive to detecting the second input from the stylus displaying the drawing application, executing the drawing application in the restricted mode and enabling display of the one or more marks of the drawing within the drawing application. An electronic device configured to:

205. 205. The electronic device of claim 204, wherein the second input from the stylus displaying the drawing application comprises swiping the stylus from an edge of the touch-sensitive display unit.

206. 205. The electronic device of claim 204, wherein the second input from the stylus displaying the drawing application comprises tapping the touch-sensitive display unit with the stylus.

207. 205. The electronic device of claim 204, wherein the second input from the stylus displaying the drawing application comprises a liftoff of the stylus from the touch-sensitive display unit.

208. The processing unit 208. The electronic device of claim 204, configured to enable a user to access content generated during a current limited session of the drawing application while running the drawing application in the limited mode without providing access to content generated during an unlimited session of the drawing application and to content generated during a previous limited session of the drawing application.

209. 1. An electronic device comprising: a touch-sensitive display unit configured to display a user interface and receive user contacts; one or more sensor units configured to detect signals from a stylus associated with the device; a processing unit coupled to the touch-sensitive display unit and the one or more sensor units; wherein the processing unit While the electronic device is in a locked state, enabling display of a lock screen user interface on the touch-sensitive display unit; Detecting a user input while enabling display of the lock screen user interface on the touch-sensitive display unit; replacing a display of the lock screen user interface with a display of a drawing application in a restricted session in response to a determination that the user input is a default input with the stylus; maintaining display of the lock screen user interface and ceasing display of the drawing application in accordance with a determination that the user input is a finger touch input; An electronic device configured to:

210. The user input is a tap gesture, and the processing unit: enabling a display of the drawing application in accordance with a determination that the tap gesture was performed with a stylus; ceasing to enable display of the drawing application in accordance with a determination that the tap gesture was performed with a finger; 210. The electronic device of claim 209, configured as follows:

211. the user input is a swipe gesture, and the processing unit: enable display of the drawing application in accordance with a determination that the swipe gesture is performed with a stylus and satisfies a drawing application display criterion; 210. The electronic device of claim 209, configured to enable display of an unlock user interface pursuant to a determination that the swipe gesture is performed with a finger and satisfies an unlock initiation criterion.

212. The processing unit While the electronic device is in the locked state, detecting a first input from the stylus drawing on the touch-sensitive display unit while enabling display of the lock screen user interface on the touch-sensitive display unit; In response to detecting the first input from the drawing stylus, enabling display, on the lock screen user interface, one or more drawing marks corresponding to the first input from the stylus.

212. An electronic device as claimed in any one of claims 209 to 211, configured so as to

213. while the processing unit enables display, on the lock screen user interface, of the one or more marks of the drawing corresponding to the first input from the stylus; Detecting a second input from the stylus displaying a drawing application in a restricted mode; responsive to detecting the second input from the stylus displaying the drawing application, executing the drawing application in the restricted mode and enabling display of the one or more marks of the drawing within the drawing application.

213. The electronic device of claim 212 configured as follows:

214. 1. An electronic device comprising: a touch-sensitive display unit configured to display a user interface and receive user contacts; one or more sensor units configured to detect signals from a stylus associated with the device; a processing unit coupled to the touch-sensitive display unit and the one or more sensor units; wherein the processing unit While the electronic device is in a locked state, enabling display of a lock screen user interface on the touch-sensitive display unit, the lock screen user interface including a lock screen image; a first input by the stylus on the touch-sensitive display unit while enabling display of the lock screen user interface on the touch-sensitive display unit; Detecting an initial contact by the stylus on the touch-sensitive display unit at the initiation of the first input; Detecting movement of the stylus across the touch-sensitive display unit during the first input; detecting lift-off of the stylus from the touch-sensitive display unit at the end of the first input; Detect, including in response to detecting the movement of the stylus across the touch-sensitive display unit, drawing a first mark in accordance with the movement of the stylus across the touch-sensitive display unit, the first mark being displayed on the lock screen image; in response to detecting the lift-off of the stylus from the touch-sensitive display unit; ceasing to enable display of the lock screen user interface, including ceasing to display the lock screen image; Enables display of a user interface for drawing applications in a restricted session, enabling a display within the user interface for the drawing application in the limited session of the first mark drawn in accordance with the movement of the stylus across the touch-sensitive display unit during the first input; detecting a second input by the stylus on the touch-sensitive display unit while enabling display of the user interface for the drawing application in the limited session; In response to detecting the second input by the stylus on the touch-sensitive display unit, drawing a second mark in the user interface for the drawing application in the limited session, the second mark being displayed together with the first mark. An electronic device configured to:

215. 1. An electronic device having a touch-sensitive surface and a display, the electronic device including one or more sensors that detect signals from a stylus associated with the device; displaying an electronic document on the display; Detecting a first input from the stylus while displaying the electronic document on the display, including detecting an initial contact by the stylus on the touch-sensitive surface; determining a plurality of characteristics of the first input from the stylus, including a tilt of the stylus at an angle relative to a normal to a surface of the touch-sensitive surface; selecting a first virtual drawing implement for the stylus to emulate in accordance with a determination that the tilt satisfies one or more selection criteria for the first virtual drawing implement; selecting a second virtual drawing implement, distinct from the first virtual drawing implement, for the stylus to emulate in accordance with determining that the tilt satisfies one or more selection criteria for a second virtual drawing implement; generating a mark in the electronic document with the selected virtual drawing implement in response to detecting the first input from the stylus after the stylus has selected one of the first virtual drawing implement and the second virtual drawing implement to emulate; A method comprising:

216. 216. The method of claim 215, wherein the one or more selection criteria for the first virtual drawing implement include the tilt of the stylus being less than a predetermined angle when the initial contact by the stylus on the touch-sensitive surface is detected, and the one or more selection criteria for the second virtual drawing implement include the tilt of the stylus being greater than the predetermined angle when the initial contact by the stylus on the touch-sensitive surface is detected.

217. 217. The method of claim 215 or 216, wherein the one or more selection criteria for the first virtual drawing implement include an average tilt of the stylus being less than a predetermined angle within a predetermined time after the initial contact by the stylus on the touch-sensitive surface is detected, and the one or more selection criteria for the second virtual drawing implement include the average tilt of the stylus being greater than the predetermined angle within the predetermined time after the initial contact by the stylus on the touch-sensitive surface is detected.

218. 218. The method of any one of claims 215 to 217, wherein the first virtual drawing means is for writing and the second virtual drawing means is for highlighting.

219. after the stylus has selected one of the first virtual drawing implement and the second virtual drawing implement to emulate; Detecting a change in the tilt of the stylus during the first input; In response to detecting the change in the tilt during the first input, adjusting a width of the mark when the tilt changes during the first input without changing the selected virtual drawing means; 219. The method of any one of claims 215 to 218, comprising:

220. 220. The method of claim 219, wherein adjusting the width of the mark as the slope changes comprises increasing the width of the mark as the slope increases.

221. Detecting an end of the first input; ceasing to generate the marks in the electronic document in response to detecting the end of the first input; 221. The method of any one of claims 215 to 220, comprising:

222. 222. The method of claim 221, wherein detecting the termination of the first input comprises detecting lift-off of the stylus from the touch-sensitive surface.

223. 223. A method according to any one of claims 215 to 222, wherein after the stylus selects one of the first virtual drawing means and the second virtual drawing means to emulate, the selected virtual drawing means remains unchanged for the remainder of the first input from the stylus, regardless of intervening changes in the tilt of the stylus.

224. the plurality of characteristics of the first input satisfy the one or more selection criteria for the first virtual rendering means; generating the mark in the electronic document with the selected virtual drawing implement in response to detecting the first input from the stylus includes generating the mark based on the first virtual drawing implement; The method includes, while continuously detecting the first input from the stylus on the touch-sensitive surface: detecting a change in the tilt of the stylus after generating the mark based on the first virtual drawing means, such that the tilt satisfies one or more tilt angle criteria in the one or more selection criteria for the second virtual drawing means; detecting movement of the stylus along the touch-sensitive surface after detecting the change in tilt of the stylus; continuing to generate the mark based on the first virtual drawing means in response to detecting the movement of the stylus; 224. The method of any one of claims 215 to 223, comprising:

225. After continuing to generate the mark based on the first virtual drawing means, detecting lift-off of the stylus from the touch-sensitive surface; after detecting lift-off of the stylus from the touch-sensitive surface; detecting a second input from the stylus that satisfies the one or more selection criteria for the second virtual drawing implement; Detecting movement of the stylus along the touch-sensitive surface while continuing to detect the second input from the stylus on the touch-sensitive surface; generating a mark based on the second virtual drawing means in response to detecting the movement of the stylus along the touch-sensitive surface; 225. The method of claim 224, comprising:

226. 1. An electronic device comprising: The display and a touch-sensitive surface; and one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs displaying an electronic document on the display; Detecting a first input from the stylus while displaying the electronic document on the display, including detecting an initial contact by the stylus on the touch-sensitive surface; determining a plurality of characteristics of the first input from the stylus, including a tilt of the stylus at an angle relative to a normal to a surface of the touch-sensitive surface; selecting the first virtual drawing implement for the stylus to emulate in accordance with determining that the tilt satisfies one or more selection criteria for the first virtual drawing implement; selecting a second virtual drawing implement, distinct from the first virtual drawing implement, for the stylus to emulate in accordance with determining that the tilt satisfies one or more selection criteria for a second virtual drawing implement; generating a mark in the electronic document with the selected virtual drawing implement in response to detecting the first input from the stylus after the stylus has selected one of the first virtual drawing implement and the second virtual drawing implement to emulate; An electronic device containing instructions.

227. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a display, a touch-sensitive surface, and one or more sensors that detect signals from a stylus associated with the device, cause the device to: displaying an electronic document on the display; detecting a first input from the stylus while displaying the electronic document on the display, including detecting an initial contact by the stylus on the touch-sensitive surface; determining a plurality of characteristics of the first input from the stylus, including a tilt of the stylus at an angle relative to a normal to a surface of the touch-sensitive surface; selecting the first virtual drawing implement for the stylus to emulate in accordance with determining that the tilt satisfies one or more selection criteria for a first virtual drawing implement; selecting a second virtual drawing implement, distinct from the first virtual drawing implement, for the stylus to emulate in accordance with determining that the tilt satisfies one or more selection criteria for a second virtual drawing implement; generating a mark in the electronic document with the selected virtual drawing implement in response to detecting the first input from the stylus after the stylus has selected one of the first virtual drawing implement and the second virtual drawing implement to emulate; A computer-readable storage medium containing instructions.

228. 1. An electronic device comprising: The display and a touch-sensitive surface; and one or more sensors that detect signals from a stylus associated with the device; means for displaying an electronic document on said display; means for detecting a first input from the stylus while displaying the electronic document on the display, including detecting an initial contact by the stylus on the touch-sensitive surface; means for determining a plurality of characteristics of the first input from the stylus, including a tilt of the stylus relative to a surface normal of the touch-sensitive surface; means for selecting a first virtual drawing implement for the stylus to emulate in accordance with a determination that the tilt satisfies one or more selection criteria for the first virtual drawing implement; means for selecting a second virtual drawing implement, distinct from the first virtual drawing implement, for the stylus to emulate in accordance with a determination that the tilt satisfies one or more selection criteria for a second virtual drawing implement; means for generating a mark in the electronic document with the selected virtual drawing implement in response to detecting the first input from the stylus after the stylus has selected one of the first virtual drawing implement and the second virtual drawing implement to emulate; An electronic device comprising:

229. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: a display; a touch-sensitive surface; and one or more sensors for detecting signals from a stylus associated with the device, the information processing apparatus comprising: means for displaying an electronic document on said display; means for detecting a first input from the stylus while displaying the electronic document on the display, including detecting an initial contact by the stylus on the touch-sensitive surface; means for determining a plurality of characteristics of the first input from the stylus, including a tilt of the stylus relative to a surface normal of the touch-sensitive surface; means for selecting a first virtual drawing implement for the stylus to emulate in accordance with a determination that the tilt satisfies one or more selection criteria for the first virtual drawing implement; means for selecting a second virtual drawing implement, distinct from the first virtual drawing implement, for the stylus to emulate in accordance with a determination that the tilt satisfies one or more selection criteria for a second virtual drawing implement; means for generating a mark in the electronic document with the selected virtual drawing implement in response to detecting the first input from the stylus after the stylus has selected one of the first virtual drawing implement and the second virtual drawing implement to emulate; An information processing device comprising:

230. 1. An electronic device comprising: The display and a touch-sensitive surface; and one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; 226. An electronic device comprising: one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 215 to 225.

231. 226. A computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by an electronic device having a display, a touch-sensitive surface, and one or more sensors that detect signals from a stylus associated with the device, cause the device to perform the method of any one of claims 215 to 225.

232. 226. A graphical user interface on an electronic device having a display, a touch-sensitive surface, one or more sensors that detect signals from a stylus associated with the device, a memory, and one or more processors that execute one or more programs stored in the memory, the graphical user interface comprising a user interface displayed according to the method of any one of claims 215 to 225.

233. 1. An electronic device comprising: The display and a touch-sensitive surface; and one or more sensors that detect signals from a stylus associated with the device; means for carrying out the method of any one of claims 215 to 225; An electronic device comprising:

234. 1. An information processing apparatus for use in an electronic device, the electronic device comprising a display, a touch-sensitive surface, and one or more sensors that detect signals from a stylus associated with the device, the information processing apparatus comprising:

226. An information processing device comprising means for carrying out the method of any one of claims 215 to 225.

235. 1. An electronic device comprising: a display unit configured to display a user interface; a touch-sensitive surface unit configured to receive a user's touch; one or more sensor units configured to detect signals from a stylus associated with the device; a processing unit coupled to the display unit, the touch-sensitive surface unit, and the one or more sensor units; wherein the processing unit enabling display of electronic documents on said display unit; Detecting a first input from the stylus, including detecting an initial contact by the stylus on the touch-sensitive surface unit while enabling display of the electronic document on the display unit; determining a plurality of characteristics of the first input from the stylus, including a tilt of the stylus at an angle relative to a normal to a surface of the touch-sensitive surface unit; selecting the first virtual drawing implement for the stylus to emulate in accordance with determining that the tilt satisfies one or more selection criteria for the first virtual drawing implement; selecting a second virtual drawing implement, distinct from the first virtual drawing implement, for the stylus to emulate in accordance with determining that the tilt satisfies one or more selection criteria for a second virtual drawing implement; generating a mark in the electronic document with the selected virtual drawing implement in response to detecting the first input from the stylus after the stylus has selected one of the first virtual drawing implement and the second virtual drawing implement to emulate; An electronic device configured to:

236. 236. The electronic device of claim 235, wherein the one or more selection criteria for the first virtual drawing means include the tilt of the stylus being less than a predetermined angle when the initial contact by the stylus on the touch-sensitive surface unit is detected, and the one or more selection criteria for the second virtual drawing means include the tilt of the stylus being greater than the predetermined angle when the initial contact by the stylus on the touch-sensitive surface unit is detected.

237. 237. The electronic device of claim 235 or 236, wherein the one or more selection criteria for the first virtual drawing implement include an average tilt of the stylus being less than a predetermined angle within a predetermined time after the initial contact by the stylus on the touch-sensitive surface is detected, and the one or more selection criteria for the second virtual drawing implement include the average tilt of the stylus being greater than the predetermined angle within the predetermined time after the initial contact by the stylus on the touch-sensitive surface is detected.

238. 238. The electronic device of any one of claims 235 to 237, wherein the first virtual drawing means is for writing and the second virtual drawing means is for highlighting.

239. The processing unit after the stylus has selected one of the first virtual drawing implement and the second virtual drawing implement to emulate; Detecting a change in the tilt of the stylus during the first input; adjusting a width of the mark when the tilt changes during the first input without changing the selected virtual drawing means in response to detecting the change in the tilt during the first input; 239. An electronic device as claimed in any one of claims 235 to 238, configured so as to

240. 240. The electronic device of claim 239, wherein adjusting the width of the mark as the tilt changes comprises increasing the width of the mark as the tilt increases.

241. The processing unit Detecting the end of the first input; ceasing to generate the marks in the electronic document in response to detecting the end of the first input.

241. An electronic device as claimed in any one of claims 235 to 240, configured so as to

242. 242. The electronic device of claim 241, wherein detecting the termination of the first input comprises detecting lift-off of the stylus from the touch-sensitive surface unit.

243. 243. An electronic device as described in any one of claims 235 to 242, wherein after the stylus selects one of the first virtual drawing means and the second virtual drawing means to emulate, the selected virtual drawing means remains unchanged for the remainder of the first input from the stylus, regardless of intervening changes in the tilt of the stylus.

244. the plurality of characteristics of the first input satisfy the one or more selection criteria for the first virtual rendering means; generating the mark in the electronic document with the selected virtual drawing implement in response to detecting the first input from the stylus includes generating the mark based on the first virtual drawing implement; While the processing unit continuously detects the first input from the stylus on the touch-sensitive surface unit, detecting a change in the tilt of the stylus after generating the mark based on the first virtual drawing means such that the tilt satisfies one or more tilt angle criteria in the one or more selection criteria for the second virtual drawing means; detecting movement of the stylus along the touch-sensitive surface unit after detecting the change in tilt of the stylus; continuing to generate the mark based on the first virtual drawing means in response to detecting the movement of the stylus; 244. An electronic device as claimed in any one of claims 235 to 243, configured to:

245. After the processing unit continues to generate the mark based on the first virtual drawing means, Detecting lift-off of the stylus from the touch-sensitive surface unit; after detecting lift-off of the stylus from the touch-sensitive surface unit; detecting a second input from the stylus that satisfies the one or more selection criteria for the second virtual drawing implement; detecting movement of the stylus along the touch-sensitive surface unit while continuing to detect the second input from the stylus on the touch-sensitive surface unit; generating a mark based on the second virtual drawing means in response to detecting the movement of the stylus along the touch-sensitive surface unit; 245. The electronic device of claim 244, configured as follows:

246. 1. An electronic device having a touch-sensitive display, displaying an electronic calendar on the touch-sensitive display, the electronic calendar including an area corresponding to a first day; detecting handwriting on the touch-sensitive display that extends over less than all of a vertical portion of the first day in the displayed calendar; In response to detecting the handwriting input, displaying one or more graphics corresponding to the handwriting input above the vertical portion of the first day; detecting an input corresponding to a request to create an event in the electronic calendar while displaying the one or more graphics corresponding to the handwritten input over the vertical portion of the first day; In response to detecting the input corresponding to a request to create an event in the electronic calendar, creating an event in the first day in the electronic calendar having a start time and an end time according to the vertical portion of the first day over which the handwriting input extends; A method comprising:

247. In response to detecting the handwriting input, determining whether the handwriting input includes handwritten characters; In accordance with a determination that the handwriting input includes handwritten characters, recognizing the handwritten characters in the handwriting input; The method of claim 246, comprising:

248. 248. The method of claim 247, comprising maintaining a display of the graphic corresponding to the handwritten input after recognizing the handwritten characters.

249. 248. The method of claim 247, comprising, after recognizing the handwritten characters, replacing the one or more graphical representations corresponding to the handwritten characters with the corresponding recognized characters.

250. 250. The method of any one of claims 246 to 249, wherein the input corresponding to a request to create an event in the electronic calendar comprises ceasing to detect the handwriting input.

251. 251. The method of any one of claims 246 to 250, wherein the input corresponding to a request to create an event in the electronic calendar is detected on the one or more graphics corresponding to the handwritten input.

252. in response to detecting the input corresponding to a request to create an event in the electronic calendar; maintaining a display of the one or more graphics corresponding to the handwriting input over the vertical portion of the first day; 252. The method of any one of claims 246 to 251, comprising:

253. in response to detecting the input corresponding to a request to create an event in the electronic calendar; maintaining a display of the one or more graphics corresponding to the handwriting input over the vertical portion of the first day; adding one or more graphics indicating the start and end times of the event; 253. The method of any one of claims 246 to 252, comprising:

254. in response to detecting the input corresponding to a request to create an event in the electronic calendar; replacing the display of the one or more graphics corresponding to the handwriting input with a display of a predetermined graphic for the event; 252. The method of any one of claims 246 to 251, comprising:

255. Detecting that a current time is the start time of the event or a predetermined time before the start time of the event; displaying an event advisory on the touch-sensitive display in response to detecting that the current time is the start time of the event or the predetermined time before the start time of the event; 255. The method of any one of claims 246 to 254, wherein the displayed event prompt includes the one or more graphics corresponding to the handwriting input.

256. 256. The method of any one of claims 246 to 255, comprising transmitting the event to a second electronic device remote from the electronic device with the touch-sensitive display, wherein a representation of the event is configured to be displayed on the second electronic device along with the one or more graphics corresponding to the handwriting input.

257. detecting an input corresponding to a request to switch between displaying the event using the one or more graphics corresponding to the handwritten input and displaying the event using a replacement graphic for the handwritten input; In response to detecting the input, determining whether the graphic corresponding to the handwriting input for the event is being displayed or whether the replacement graphic for the handwriting input for the event is being displayed; in response to determining that the one or more graphics corresponding to the handwriting input are being displayed, ceasing to display the one or more graphics corresponding to the handwriting input and displaying the event using a replacement graphic for the handwriting input for the event; in response to determining that the replacement graphic for the handwriting input is being displayed, ceasing to display the replacement graphic for the handwriting input and displaying the event using the one or more graphics corresponding to the handwriting input; 257. The method of any one of claims 246 to 256, comprising:

258. detecting an input corresponding to a request to switch between displaying the plurality of events using graphics corresponding to handwriting input for the plurality of events and displaying the plurality of events using replacement graphics for the handwriting input for the plurality of events; determining, in response to detecting the input, whether the graphic corresponding to the handwriting input for the plurality of events is being displayed or whether a replacement graphic for the handwriting input for the plurality of events is being displayed; in response to determining that the graphic corresponding to the hand-drawing input for the plurality of events is being displayed, ceasing to display the graphic corresponding to the hand-drawing input for the plurality of events and displaying the plurality of events using a replacement graphic for the hand-drawing input for the plurality of events; in response to determining that the replacement graphic for the handwriting input for the plurality of events is being displayed, canceling the display of the replacement graphic for the handwriting input for the plurality of events and displaying the plurality of events using the graphic corresponding to the handwriting input for the plurality of events; 258. The method of any one of claims 246 to 257, comprising:

259. 1. An electronic device comprising: a touch-sensitive display; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs displaying an electronic calendar on the touch-sensitive display, the electronic calendar including an area corresponding to a first day; detecting handwriting on the touch-sensitive display that extends over less than all of a vertical portion of the first day in the displayed calendar; In response to detecting the handwriting input, display one or more graphics corresponding to the handwriting input above the vertical portion of the first day; detecting an input corresponding to a request to create an event in the electronic calendar while displaying the one or more graphics corresponding to the handwritten input over the vertical portion of the first day; In response to detecting the input corresponding to a request to create an event in the electronic calendar, creating an event in the first day in the electronic calendar having a start time and an end time according to the vertical portion of the first day over which the handwriting input extends. An electronic device containing instructions.

260. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a touch-sensitive display, cause the device to: displaying an electronic calendar on the touch-sensitive display, the electronic calendar including an area corresponding to a first day; detecting handwriting on the touch-sensitive display that extends over less than all of a vertical portion of the first day in the displayed calendar; In response to detecting the handwriting input, displaying one or more graphics corresponding to the handwriting input above the vertical portion of the first day; detecting an input corresponding to a request to create an event in the electronic calendar while displaying the one or more graphics corresponding to the handwritten input over the vertical portion of the first day; in response to detecting the input corresponding to a request to create an event in the electronic calendar, creating an event in the first day in the electronic calendar having a start time and an end time according to the vertical portion of the first day over which the handwriting input extends. A computer-readable storage medium containing instructions.

261. 1. An electronic device comprising: a touch-sensitive display; means for displaying an electronic calendar on the touch-sensitive display, the electronic calendar including an area corresponding to a first day; means for detecting handwriting input on the touch-sensitive display that extends over less than all of a vertical portion of the first day in the displayed calendar; means for displaying one or more graphics corresponding to the handwriting input above the vertical portion of the first day in response to detecting the handwriting input; means for detecting an input corresponding to a request to create an event in the electronic calendar while displaying the one or more graphics corresponding to the handwritten input over the vertical portion of the first day; means for creating an event in the first day in the electronic calendar in response to detecting the input corresponding to a request to create an event in the electronic calendar, the event having a start time and an end time according to the vertical portion of the first day over which the handwritten input extends; An electronic device comprising:

262. 1. An information processing apparatus for use in an electronic device with a touch-sensitive display, comprising: means for displaying an electronic calendar on the touch-sensitive display, the electronic calendar including an area corresponding to a first day; means for detecting handwriting input on the touch-sensitive display that extends over less than all of a vertical portion of the first day in the displayed calendar; means for displaying one or more graphics corresponding to the handwriting input above the vertical portion of the first day in response to detecting the handwriting input; means for detecting an input corresponding to a request to create an event in the electronic calendar while displaying the one or more graphics corresponding to the handwritten input over the vertical portion of the first day; means for creating an event in the first day in the electronic calendar in response to detecting the input corresponding to a request to create an event in the electronic calendar, the event having a start time and an end time according to the vertical portion of the first day over which the handwritten input extends; An information processing device comprising:

263. 1. An electronic device comprising: a touch-sensitive display; one or more processors; Memory and One or more programs; 259. An electronic device comprising: one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 246 to 258.

264. 259. A computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by an electronic device with a touch-sensitive display, cause the device to perform the method of any one of claims 246 to 258.

265. 259. A graphical user interface on an electronic device comprising a touch-sensitive display, memory, and one or more processors executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed in accordance with the method of any one of claims 246 to 258.

266. 1. An electronic device comprising: a touch-sensitive display; means for carrying out the method of any one of claims 246 to 258; An electronic device comprising:

267. 1. An information processing apparatus for use in an electronic device with a touch-sensitive display, comprising:

259. An information processing device comprising means for carrying out the method of any one of claims 246 to 258.

268. 1. An electronic device comprising:

1. A touch-sensitive display unit, comprising: displaying an electronic calendar on the touch-sensitive display, the electronic calendar including an area corresponding to a first day; detecting handwriting on the touch-sensitive display that extends over less than all of a vertical portion of the first day in the displayed calendar; In response to detecting the handwriting input, display one or more graphics corresponding to the handwriting input above the vertical portion of the first day; detecting an input corresponding to a request to create an event in the electronic calendar while displaying the one or more graphics corresponding to the handwritten input over the vertical portion of the first day; a touch-sensitive display unit configured to a processing unit coupled to the touch-sensitive display unit, the processing unit configured, in response to detecting the input corresponding to a request to create an event in the electronic calendar, to create an event in the first day in the electronic calendar having a start time and an end time according to the vertical portion of the first day over which the handwriting input extends; An electronic device comprising:

269. The processing unit In response to detecting the handwriting input, determining whether the handwriting input includes handwritten characters; in accordance with a determination that the handwritten input includes handwritten characters, recognizing the handwritten characters in the handwritten input.

269. The electronic device of claim 268, configured as follows:

270. The processing unit 270. The electronic device of claim 269, configured to maintain a display of the one or more graphics corresponding to the handwritten input after recognizing the handwritten characters.

271. The processing unit 270. The electronic device of claim 269, configured to, after recognizing the handwritten character, replace the one or more graphical representations corresponding to the handwritten character with the corresponding recognized character.

272. 272. The electronic device of any one of claims 268 to 271, wherein the input corresponding to a request to create an event in the electronic calendar comprises ceasing to detect the handwriting input.

273. 273. The electronic device of any one of claims 268 to 272, wherein the input corresponding to a request to create an event in the electronic calendar is detected on the one or more graphics corresponding to the handwritten input.

274. The processing unit in response to detecting the input corresponding to a request to create an event in the electronic calendar; configured to maintain a display of the one or more graphics corresponding to the handwriting input over the vertical portion of the first day.

274. An electronic device according to any one of claims 268 to 273.

275. The processing unit in response to detecting the input corresponding to a request to create an event in the electronic calendar; maintaining a display of the one or more graphics corresponding to the handwriting input over the vertical portion of the first day; adding one or more graphics indicating the start and end times of the event; 275. An electronic device as described in any one of claims 268 to 274, configured so as to

276. The processing unit in response to detecting the input corresponding to a request to create an event in the electronic calendar; replacing the display of the one or more graphics corresponding to the handwriting input with a display of a predetermined graphic for the event; 274. An electronic device as claimed in any one of claims 268 to 273, configured so as to

277. The processing unit Detecting that the current time is the start time of the event; enabling display of an event advisory on the touch-sensitive display unit in response to detecting that the current time is the start time of the event; 277. The electronic device of any one of claims 268 to 276, wherein the displayed event prompt includes the one or more graphics corresponding to the handwriting input.

278. The processing unit detecting an input corresponding to a request to switch between displaying the event using the one or more graphics corresponding to the handwriting input and displaying the event using a replacement graphic for the handwriting input for the event; In response to detecting the input, determining whether the graphic corresponding to the handwriting input for the event is being displayed or whether the replacement graphic for the handwriting input for the event is being displayed; responsive to determining that the one or more graphics corresponding to the handwriting input are being displayed, ceasing to display the graphics corresponding to the handwriting input and enabling display of the event using a replacement graphic for the handwriting input for the event; in response to determining that the replacement graphic for the handwriting input is being displayed, ceasing to display the replacement graphic for the handwriting input and enabling display of the event using the one or more graphics corresponding to the handwriting input.

278. An electronic device as claimed in any one of claims 268 to 277, configured so as to

279. The processing unit detecting an input corresponding to a request to switch between displaying the plurality of events using graphics corresponding to handwriting input for the plurality of events and displaying the plurality of events using replacement graphics for the handwriting input for the plurality of events; determining, in response to detecting the input, whether the graphic corresponding to the hand-drawn input for the plurality of events is being displayed or whether a replacement graphic for the hand-drawn input for the plurality of events is being displayed; in response to determining that the graphic corresponding to the handwriting input for the plurality of events is being displayed, ceasing to display the graphic corresponding to the handwriting input for the plurality of events and enabling display of the plurality of events using a replacement graphic for the handwriting input for the plurality of events; in response to determining that the replacement graphic for the handwriting input for the plurality of events is being displayed, ceasing the display of the replacement graphic for the handwriting input for the plurality of events and enabling the display of the plurality of events using the graphic corresponding to the handwriting input for the plurality of events; 279. The electronic device of any one of claims 268 to 278, configured so as to

280. 1. An electronic device having a touch-sensitive display, displaying a video editing application on the touch-sensitive display, the video editing application including a timeline area for a video being edited; a gesture of contact on the touch-sensitive display, an initial contact by the touch with the timeline area occurring at a first time position within the timeline area; movement of the contact across the touch-sensitive display after the initial contact with the timeline area; and a lift-off of the contact at a position on the touch-sensitive display corresponding to a second time position within the timeline area; Detecting a gesture, including: selecting a portion of the video being edited in response to detecting the gesture by the contact on the touch-sensitive display; wherein the selected portion of the video begins at the first time location and ends at the second time location.

281. 281. The method of claim 280, wherein the contact in the gesture contacts the displayed video editing application outside the timeline area before the initial contact with the timeline area.

282. 282. The method of claim 280 or 281, comprising displaying indicia indicating the movement of the contact across the touch-sensitive display after the initial contact with the timeline area.

283. 283. The method of any one of claims 280 to 282, comprising visually distinguishing one or more regions of the timeline area that are not selected from regions of the timeline area that are selected as the contact moves across the touch-sensitive display after the initial contact with the timeline area by the contact.

284. displaying a mark indicating the movement of the contact across the touch-sensitive display from touchdown of the contact on the touch-sensitive display to liftoff of the contact from the touch-sensitive display; the mark includes a first portion of the mark before the initial contact with the timeline area; the mark includes a second portion of the mark after the initial contact with the timeline area; the second portion of the mark is visually distinct from the first portion of the mark; 284. The method of any one of claims 280 to 283.

285. and displaying the video editing program includes displaying a viewer area for the video being edited, the method comprising:

285. The method of any one of claims 280 to 284, comprising retrieving the video into the viewer area according to the movement of the contact across the touch-sensitive display after the contact makes initial contact with the timeline area.

286. 286. The method of any one of claims 280 to 285, wherein the position of the contact at lift-off is outside the timeline area, the position of the contact at lift-off has a horizontal position on the touch-sensitive display, and the second time position within the timeline area has the same horizontal position on the display as the position of the contact at lift-off.

287. 287. The method of any one of claims 280 to 286, wherein the movement of the contact across the touch-sensitive display after the initial contact with the timeline area includes continuous movement of the contact to a position on the touch-sensitive display corresponding to a third time position in the timeline area after the second time position in the timeline area, and thereafter continuous movement of the contact to the position on the touch-sensitive display corresponding to the second time position in the timeline area where the contact lifts off.

288. displaying indicia indicating the movement of the contact across the touch-sensitive display after the initial contact with the timeline area; detecting a lift-off of the contact at the location on the touch-sensitive display corresponding to the second time position within the timeline area; in response to detecting lift-off of the contact at the location on the touch-sensitive display corresponding to the second time position within the timeline area; Stop displaying the mark, displaying a selection box around the selected portion of the video; 288. The method of any one of claims 280 to 287, comprising:

289. displaying indicia indicating the movement of the contact across the touch-sensitive display after the initial contact with the timeline area; detecting a lift-off of the contact at the location on the touch-sensitive display corresponding to the second time position within the timeline area; displaying an animation of the mark to one side of a selection box around the selected portion of the video in response to detecting lift-off of the contact at the location on the touch-sensitive display corresponding to the second time position within the timeline area; and 289. The method of any one of claims 280 to 288, comprising:

290. 1. An electronic device comprising: a touch-sensitive display; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs displaying a video editing application on the touch-sensitive display including a timeline area for a video being edited; a gesture of contact on the touch-sensitive display, an initial contact by the touch with the timeline area occurring at a first time position within the timeline area; movement of the contact across the touch-sensitive display after the initial contact with the timeline area; and a lift-off of the contact at a position on the touch-sensitive display corresponding to a second time position within the timeline area; Detect gestures, including responsive to detecting the gesture by the contact on the touch-sensitive display, selecting a portion of the video being edited, the selected portion of the video beginning at the first time location and ending at the second time location.

291. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a touch-sensitive display, cause the device to: displaying a video editing application on the touch-sensitive display including a timeline area for a video being edited; a gesture of contact on the touch-sensitive display, an initial contact by the touch with the timeline area occurring at a first time position within the timeline area; movement of the contact across the touch-sensitive display after the initial contact with the timeline area; and a lift-off of the contact at a position on the touch-sensitive display corresponding to a second time position within the timeline area; Detecting a gesture, including responsive to detecting the gesture by the contact on the touch-sensitive display, selecting a portion of the video being edited, the selected portion of the video beginning at the first time location and ending at the second time location.

292. 1. An electronic device comprising: a touch-sensitive display; means for displaying a video editing application on the touch-sensitive display, the video editing application including a timeline area for a video being edited; a gesture of contact on the touch-sensitive display, an initial contact by the touch with the timeline area occurring at a first time position within the timeline area; movement of the contact across the touch-sensitive display after the initial contact with the timeline area; and a lift-off of the contact at a position on the touch-sensitive display corresponding to a second time position within the timeline area; a means for detecting a gesture, the means comprising: means for selecting a portion of the video being edited in response to detecting the gesture by the contact on the touch-sensitive display; wherein the selected portion of the video begins at the first time location and ends at the second time location.

293. 1. An information processing apparatus for use in an electronic device with a touch-sensitive display, comprising: means for displaying a video editing application on the touch-sensitive display, the video editing application including a timeline area for a video being edited; a gesture of contact on the touch-sensitive display, an initial contact by the touch with the timeline area occurring at a first time position within the timeline area; movement of the contact across the touch-sensitive display after the initial contact with the timeline area; and a lift-off of the contact at a position on the touch-sensitive display corresponding to a second time position within the timeline area; a means for detecting a gesture, the means comprising: means for selecting a portion of the video being edited in response to detecting the gesture by the contact on the touch-sensitive display; wherein the selected portion of the video begins at the first time location and ends at the second time location.

294. 1. An electronic device comprising: a touch-sensitive display; one or more processors; Memory and One or more programs; 290. An electronic device comprising: one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 280 to 289.

295. 290. A computer-readable storage medium storing one or more programs containing instructions that, when executed by an electronic device with a touch-sensitive display, cause the device to perform the method of any one of claims 280 to 289.

296. 290. A graphical user interface on an electronic device comprising a touch-sensitive display, memory, and one or more processors executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed in accordance with the method of any one of claims 280 to 289.

297. 1. An electronic device comprising: a touch-sensitive display; means for carrying out the method of any one of claims 280 to 289; An electronic device comprising:

298. 1. An information processing apparatus for use in an electronic device with a touch-sensitive display, comprising:

290. An information processing device comprising means for carrying out the method of any one of claims 280 to 289.

299. 1. An electronic device comprising: a touch-sensitive display unit configured to display a video editing application including a timeline area for the video being edited; a processing unit coupled to the touch-sensitive display unit; and wherein the processing unit a gesture of contact on the touch-sensitive display unit, an initial contact by the touch with the timeline area occurring at a first time position within the timeline area; movement of the contact across the touch-sensitive display unit after the initial contact with the timeline area; and a lift-off of the contact at a location on the touch-sensitive display unit corresponding to a second time position within the timeline area; Detect gestures, including selecting a portion of the video being edited in response to detecting the gesture by the contact on the touch-sensitive display unit; wherein the selected portion of the video begins at the first time location and ends at the second time location.

300. 300. The electronic device of claim 299, wherein the contact in the gesture contacts the displayed video editing application outside the timeline area before the initial contact with the timeline area.

301. The processing unit 301. The electronic device of claim 299 or 300, configured to enable display of indicia indicating the movement of the contact across the touch-sensitive display unit after the initial contact with the timeline area.

302. The processing unit 302. The electronic device of claim 299, wherein the electronic device is configured to visually distinguish one or more regions of the timeline area that are not selected from regions of the timeline area that are selected as the contact moves across the touch-sensitive display unit after the initial contact with the timeline area by the contact.

303. The processing unit configured to enable display of indicia indicative of the movement of the contact across the touch-sensitive display unit from touchdown of the contact on the touch-sensitive display unit to liftoff of the contact from the touch-sensitive display unit; the mark includes a first portion of the mark before the initial contact with the timeline area; the mark includes a second portion of the mark after the initial contact with the timeline area; the second portion of the mark is visually distinct from the first portion of the mark; 303. An electronic device according to any one of claims 299 to 302.

304. The processing unit enabling the display of a viewer area for said video being edited; Retrieving the video into the viewer area according to the movement of the contact across the touch-sensitive display unit after the contact makes initial contact with the timeline area.

304. An electronic device as claimed in any one of claims 299 to 303, configured as follows:

305. 305. The electronic device of claim 299, wherein the position of the contact at liftoff is outside the timeline area, the position of the contact at liftoff has a horizontal position on the touch-sensitive display unit, and the second time position within the timeline area has the same horizontal position on the display as the position of the contact at liftoff.

306. 306. The electronic device of claim 299, wherein the movement of the contact across the touch-sensitive display unit after the initial contact with the timeline area includes continuous movement of the contact to a position on the touch-sensitive display unit corresponding to a third time position in the timeline area after the second time position in the timeline area, and thereafter continuous movement of the contact to the position on the touch-sensitive display unit corresponding to the second time position in the timeline area where the contact lifts off.

307. The processing unit enabling display of indicia indicating the movement of the contact across the touch-sensitive display unit after the initial contact with the timeline area; enable detection of lift-off of the contact at the location on the touch-sensitive display unit corresponding to the second time location within the timeline area; in response to detecting lift-off of the contact at the location on the touch-sensitive display unit corresponding to the second time location within the timeline area; Stop displaying the mark, enabling the display of a selection box around the selected portion of the video; 307. An electronic device as claimed in any one of claims 299 to 306, configured so as to

308. The processing unit enabling display of indicia indicating the movement of the contact across the touch-sensitive display unit after the initial contact with the timeline area; Detecting a lift-off of the contact at the location on the touch-sensitive display unit corresponding to the second time location within the timeline area; in response to detecting lift-off of the contact at the location on the touch-sensitive display unit corresponding to the second time position within the timeline area, enabling display of an animation of the mark to one side of a selection box around the selected portion of the video.

308. An electronic device as claimed in any one of claims 299 to 307, configured so as to

309. 1. An electronic device with a touch-sensitive display, the electronic device including one or more sensors that detect signals from a stylus associated with the device, displaying a user interface on the touch-sensitive display; detecting movement of the stylus towards the touch-sensitive display without the stylus contacting the touch-sensitive display while displaying the user interface on the touch-sensitive display; determining whether the detected movement of the stylus towards the touch-sensitive display without contacting the touch-sensitive display satisfies one or more stylus movement criteria; displaying a menu overlaid on the user interface, the menu including a plurality of selectable menu options, in accordance with a determination that the detected stylus movement satisfies the one or more stylus movement criteria; detecting a selection of a first menu option within the plurality of selectable menu options; in response to detecting a selection of the first menu option within the plurality of selectable menu options; Performing an action corresponding to the first menu option; ceasing to display said menu; and A method comprising:

310. 310. The method of claim 309, wherein no contact is detected on the touch-sensitive display while detecting the stylus moving towards the touch-sensitive display.

311. Detecting the stylus moving towards the touch-sensitive display includes detecting the stylus moving towards a first position within the user interface on the touch-sensitive display; 311. The method of claim 309 or 310, wherein displaying the menu superimposed on the user interface comprises displaying the menu superimposed on the user interface at the first location.

312. Detecting the stylus moving towards the touch-sensitive display includes detecting the stylus moving while a first object within the user interface has focus; 312. The method of any one of claims 309 to 311, wherein displaying the menu overlaid on the user interface comprises displaying the menu overlaid on the user interface on or adjacent to the first object.

313. 313. The method of any one of claims 309 to 312, wherein the one or more stylus movement criteria includes detecting that a velocity of the stylus exceeds a predetermined value during at least a portion of the detected movement of the stylus toward the touch-sensitive display.

314. 314. The method of any one of claims 309 to 313, wherein the one or more stylus movement criteria include detecting that deceleration of the stylus as it moves towards the touch-sensitive display exceeds a predetermined deceleration value during at least a portion of the detected stylus movement.

315. 315. The method of any one of claims 309 to 314, wherein the one or more stylus movement criteria include detecting, during the detected stylus movement, that the stylus accelerates as the stylus moves towards the touch-sensitive display and then decelerates as the stylus moves towards the touch-sensitive display.

316. 316. The method of any one of claims 309 to 315, wherein the one or more stylus movement criteria are independent of tilt of the stylus during the detected movement of the stylus towards the touch-sensitive display.

317. 317. The method of any one of claims 309 to 316, wherein the one or more stylus movement criteria are independent of an orientation of the stylus during the detected movement of the stylus towards the touch-sensitive display.

318. 316. The method of any one of claims 309 to 315, wherein the one or more stylus movement criteria include a criterion that is satisfied when the stylus is perpendicular to the touch-sensitive display.

319. 319. A method according to any one of claims 309 to 318, comprising ceasing to display the menu superimposed on the user interface in accordance with a determination that the one or more stylus movement criteria are not met.

320. 320. The method of any one of claims 309 to 319, wherein the menu is a color palette and the plurality of selectable menu options are colors within the color palette.

321. 321. The method of any one of claims 309 to 320, wherein ceasing to display the menu comprises displaying an animation of the first menu option appearing to be absorbed by the stylus.

322. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors; The one or more programs displaying a user interface on the touch-sensitive display; detecting movement of the stylus towards the touch-sensitive display without the stylus contacting the touch-sensitive display while displaying the user interface on the touch-sensitive display; determining whether the detected movement of the stylus towards the touch-sensitive display without contacting the touch-sensitive display satisfies one or more stylus movement criteria; displaying a menu overlaid on the user interface, the menu including a plurality of selectable menu options, in accordance with a determination that the detected stylus movement satisfies the one or more stylus movement criteria; Detecting a selection of a first menu option within the plurality of selectable menu options; in response to detecting a selection of the first menu option within the plurality of selectable menu options; Performing an action corresponding to the first menu option; ceasing to display said menu; An electronic device containing instructions.

323. 1. A computer-readable storage medium storing one or more programs that, when executed by an electronic device having a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the device, cause the device to: displaying a user interface on the touch-sensitive display; detecting movement of the stylus towards the touch-sensitive display without the stylus contacting the touch-sensitive display while displaying the user interface on the touch-sensitive display; determining whether the detected movement of the stylus towards the touch-sensitive display without contacting the touch-sensitive display satisfies one or more stylus movement criteria; displaying a menu including a plurality of selectable menu options overlaid on the user interface in response to a determination that the detected stylus movement satisfies the one or more stylus movement criteria; detecting a selection of a first menu option within the plurality of selectable menu options; in response to detecting a selection of the first menu option within the plurality of selectable menu options; performing an action corresponding to the first menu option; Stop displaying the menu; A computer-readable storage medium containing instructions.

324. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; means for displaying a user interface on the touch-sensitive display; means for detecting movement of the stylus towards the touch-sensitive display without the stylus contacting the touch-sensitive display while displaying the user interface on the touch-sensitive display; means for determining whether the detected movement of the stylus towards the touch-sensitive display without contacting the touch-sensitive display satisfies one or more stylus movement criteria; means for displaying a menu including a plurality of selectable menu options overlaid on the user interface in accordance with a determination that the detected stylus movement satisfies the one or more stylus movement criteria; means for detecting a selection of a first menu option within the plurality of selectable menu options; means, enabled in response to detecting selection of the first menu option within the plurality of selectable menu options, means for performing an action corresponding to the first menu option; means for stopping the display of the menu; and a means comprising: An electronic device comprising:

325. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: an electronic device, the electronic device comprising: a touch-sensitive display; and one or more sensors for detecting signals from a stylus associated with the device; means for displaying a user interface on the touch-sensitive display; means for detecting movement of the stylus towards the touch-sensitive display without the stylus contacting the touch-sensitive display while displaying the user interface on the touch-sensitive display; means for determining whether the detected movement of the stylus towards the touch-sensitive display without contacting the touch-sensitive display satisfies one or more stylus movement criteria; means for displaying a menu including a plurality of selectable menu options overlaid on the user interface in accordance with a determination that the detected stylus movement satisfies the one or more stylus movement criteria; means for detecting a selection of a first menu option within the plurality of selectable menu options; means, enabled in response to detecting selection of the first menu option within the plurality of selectable menu options, means for performing an action corresponding to the first menu option; means for stopping the display of the menu; and a means comprising: An information processing device comprising:

326. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; one or more processors; Memory and One or more programs; 322. An electronic device comprising: one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any one of claims 309 to 321.

327. 322. A computer-readable storage medium storing one or more programs comprising instructions that, when executed by an electronic device having a touch-sensitive display and one or more sensors that detect signals from a stylus associated with the device, cause the device to perform the method of any one of claims 309 to 321.

328. 322. A graphical user interface on an electronic device having a touch-sensitive display, one or more sensors for detecting signals from a stylus associated with the device, memory, and one or more processors for executing one or more programs stored in the memory, the graphical user interface comprising a user interface displayed in accordance with the method of any one of claims 309 to 321.

329. 1. An electronic device comprising: a touch-sensitive display; one or more sensors that detect signals from a stylus associated with the device; means for carrying out the method of any one of claims 309 to 321; An electronic device comprising:

330. 1. An information processing apparatus for use in an electronic device, the information processing apparatus comprising: an electronic device, the electronic device comprising: a touch-sensitive display; and one or more sensors for detecting signals from a stylus associated with the device; 322. An information processing device comprising means for carrying out the method of any one of claims 309 to 321.

331. 1. An electronic device comprising: a touch-sensitive display unit configured to display a user interface and receive user contacts; one or more sensor units configured to detect signals from a stylus associated with the device; a processing unit coupled to the touch-sensitive display unit and the one or more sensor units; wherein the processing unit enabling display of a user interface on the touch-sensitive display unit; detecting movement of the stylus towards the touch-sensitive display unit without the stylus contacting the touch-sensitive display unit while enabling display of the user interface on the touch-sensitive display unit; determining whether the detected movement of the stylus towards the touch-sensitive display unit without contacting the touch-sensitive display unit satisfies one or more stylus movement criteria; enable displaying, in accordance with a determination that the detected stylus movement satisfies the one or more stylus movement criteria, a menu including a plurality of selectable menu options superimposed over the user interface; Detecting a selection of a first menu option within the plurality of selectable menu options; in response to detecting a selection of the first menu option within the plurality of selectable menu options; Performing an action corresponding to the first menu option; ceasing to display said menu; An electronic device configured to:

332. 332. The electronic device of claim 331, wherein no contact is detected on the touch-sensitive display unit while detecting movement of the stylus towards the touch-sensitive display unit.

333. Detecting the stylus moving towards the touch-sensitive display unit includes detecting the stylus moving towards a first position within the user interface on the touch-sensitive display unit; 333. The electronic device of claim 331 or 332, wherein enabling display of the menu overlaid on the user interface comprises enabling display of the menu overlaid on the user interface at the first position.

334. Detecting the stylus moving towards the touch-sensitive display unit includes detecting the stylus moving while a first object within the user interface has focus; 334. The electronic device of claim 331, wherein enabling display of the menu overlaid on the user interface comprises enabling display of the menu overlaid on the user interface on or adjacent to the first object.

335. 335. The electronic device of any one of claims 331 to 334, wherein the one or more stylus movement criteria include detecting that a velocity of the stylus exceeds a predetermined value during at least a portion of the detected movement of the stylus toward the touch-sensitive display unit.

336. 336. The electronic device of claim 331, wherein the one or more stylus movement criteria include detecting that deceleration of the stylus as it moves toward the touch-sensitive display unit exceeds a predetermined deceleration value during at least a portion of the detected stylus movement.

337. 337. The electronic device of any one of claims 331 to 336, wherein the one or more stylus movement criteria include detecting, during the detected stylus movement, that the stylus accelerates as the stylus moves toward the touch-sensitive display unit and then decelerates as the stylus moves toward the touch-sensitive display unit.

338. 338. The electronic device of any one of claims 331 to 337, wherein the one or more stylus movement criteria are independent of tilt of the stylus during the detected movement of the stylus towards the touch-sensitive display unit.

339. 339. The electronic device of any one of claims 331 to 338, wherein the one or more stylus movement criteria are independent of the orientation of the stylus during the detected movement of the stylus towards the touch-sensitive display unit.

340. 338. The electronic device of any one of claims 331 to 337, wherein the one or more stylus movement criteria include a criterion that is met when the stylus is perpendicular to the touch-sensitive display unit.

341. The processing unit 341. The electronic device of any one of claims 331 to 340, configured to cease enabling display of the menu overlaid on the user interface in accordance with a determination that the one or more stylus movement criteria are not met.

342. 342. The electronic device of any one of claims 331 to 341, wherein the menu is a color palette and the plurality of selectable menu options are colors within the color palette.

343. 343. The electronic device of any one of claims 331 to 342, wherein ceasing to display the menu comprises enabling the display of an animation of the first menu option appearing to be absorbed by the stylus.

Images