Delimiters for selection-action pen gesture phrases

Abstract

A pen-based user interface (PBUI) that facilitates input of a delimiter to a scope in a substantially uninterrupted stroke for generating a selection-action gesture phrase. Four delimiter techniques are provided, which include a Multi-stroke delimiter, a button delimiter, timeout delimiter and pigtail delimiter. The Pigtail delimiter uses a small loop to delimit the gesture. The delimiter techniques support integrated scope selection, command activation, and direct manipulation all in a single fluid pen gesture. The delimiter techniques can also be employed to terminate a complex scope consisting of a sequence of multiple pen strokes.

Description

TECHNICAL FIELD [0001] This invention is related to pen-based input devices, and more specifically, to software that facilitates recognition and interaction with pen-based inputs. BACKGROUND OF THE INVENTION [0002] The current technology in commercial products such as pen-based graphical user interfaces (GUIs) is often derived from mouse-based interfaces. One goal of such pen-based systems is to mimic or at least take into consideration the mechanics of handwriting: fast, brief, and repetitive phasic muscle contractions that guide a process of rhythmically making small looping motions with a stylus or pen. However, pen-based GUI techniques in the prior art suffer from limitations and drawbacks that pose serious hurdles for users (e.g., the difficulty of learning a gesture set). [0003] In order to facilitate the proliferation of such device interfaces, several factors need to be considered. Pen interfaces should seem fast and efficient to the user. Many conventional pen interfaces an...

AI Technical Summary

Benefits of technology

[0011] In another aspect of the subject invention, the PBUI is an interface for a complex application that accommodates a large number of variations, options, and commands. Several different types of scopes are provided which support different sets of commands. Moreover, the scopes are sensitive to the types of objects on which they operate. Terminating the scopes with pigtail menus, which may be hierarchical (to at least two levels deep) and which contain extra commands per leaf node by repeated looping, allows many commands to be provided to the user via prompted selection, with a corresponding rapid gesture for expert use.

[0013] The PBUI of the subject invention uses a pigtail gesture as a technique to integrate selection of the scope of an operation with command selection based on marking menus (called pigtail menus). The pigtail completes in any of the eight primary compass directions. This provides eight different “marking” directions just like a marking menu. The self-intersection of the pigtail forms a well-defined point that is used to segment the gesture, such that the initial part of the stroke defines the scope of the command, and the terminating portion of the stroke (from the intersection point onward) defines the mark. Pigtail menus can use prompted selection (a menu pops up if the user pauses with the pen in contact with the screen after drawing the pigtail) or “blind” selection by drawing the pigtail and rapidly completing the stroke without pausing.

[0015] In another aspect of the subject invention, the PBUI provides corner detection such that a right-angle scribe can be processed into a command. A backup operation can be accomplished by backwards looping. Multiple leaf nodes facilitate correspondingly multiple commands by making additional loops at the end of a gesture. Self revelation of the leaf nodes is provided in the form of pop-up menus and stroke extensions. Crossing strokes facilitate parameter specification.

Problems solved by technology

However, pen-based GUI techniques in the prior art suffer from limitations and drawbacks that pose serious hurdles for users (e.g., the difficulty of learning a gesture set).

Many conventional pen interfaces and techniques currently still build on familiar toolbars and pull-down menus, but these may require a round trip between the content area and the interaction area.

Furthermore, because pens lack the state transitions offered by mouse interfaces, techniques often become burdened with timeouts or delays (e.g., tap-and-hold to bring up a menu) that the user must wait for every single time the corresponding action is invoked.

Such delays are not only irritating to the user as proficiency increases with the device, but also prone to false activation, such as when the user unintentionally pauses with the pen during mental preparation (e.g., while thinking about what to do next).

While this technique may be helpful for selection in some situations, the decision tree can get complex and only benefits the interface design with a single inferred mode (selection vs. ink).

The approach is difficult to scale up to a system where the user may need a multiplicity of commands and gestural modes to use an application.

With the latter approach it also may not be clear to the user what the interface will provide for each given stroke or input.

There also remains the possibility of false positive recognition (e.g., shading part of a drawing can be recognized as a scrubbing gesture, thus deleting it), since no gesture can be considered completely immune to potentially occurring in the course of freeform ink input.

A fundamental problem with gestural interfaces in the prior art is that there is a delicate tradeoff in designing gesture sets that are learnable, memorable, and with gestures that are highly distinct from one another for the purposes of recognition.

A few gestures can be recognized reliably, and recalled reliably by the user, but it becomes increasingly difficult to add more gestures and operations to an existing gesture set.

However, this approach is less satisfactory for pens, as oftentimes no keyboard is available for shortcuts, and a button for right-click may be absent or awkward to use, making round trips unavoidable.

Moreover, there is a further unmet need to provide rapid, unambiguous, and effective techniques for users of pen-operated devices to fluidly articulate selection-action command phrases.

Images