Touch Event Model Programming Interface

Abstract

To provide a touch event model programming interface.SOLUTION: One or more touch input signals can be obtained from a touch sensitive device. A touch event model can be used to determine touch and / or gesture events on the basis of the touch input signals. The touch and gesture events can be associated with touch input signals generated from different regions of a web page displayed on the touch sensitive device. Access can be provided to at least one touch or gesture event through a programming interface.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates generally to web browsing services. [Background technology]

[0002] Web pages are created using markup languages, which describe the structure of text-based information in a document and provide a means for supplementing that text with interactive forms, embedded images, and other objects. One popular markup language is the Hypertext Markup Language (HTML), which is written in the form of tags enclosed in angle brackets. HTML can describe the appearance and meaning of a Web page and can include embedded scripting language code (e.g., JavaScript®) that affects the behavior of Web browsers and other HTML processors. JavaScript® gives developers the ability to add mouse event handlers or event listeners to Web pages. These mouse event handlers can be assigned to specific areas of a Web page and configured to receive mouse events, such as mouse-up or mouse-down events, in those areas. Summary of the Invention

[0003] In contrast, web pages navigated with touch-sensitive devices often need to respond to touch events generated by a user touching the web page with one or more fingers and making gestures. Traditional mouse event handlers cannot correctly interpret these touch events. Therefore, touch events require a different touch event model to correctly interpret touch events and enable developers to fully utilize the capabilities of a touch-sensitive display or device.

[0004] One or more touch input signals can be obtained from the touch-sensitive device. A touch event model can be used to determine touch and / or gesture events based on the touch input signals. The touch and gesture events can be associated with touch input signals originating from different areas of a web page displayed on the touch-sensitive device. Access to at least one touch or gesture event can be provided through a programming interface.

[0005] In one embodiment, in a web browser, a method includes receiving a rotation value associated with a gesture event and dynamically rotating an element of a web page associated with the gesture event in the web browser based on the rotation value, the rotation value being a relative delta in degrees. The gesture event can include two or more touch events.

[0006] In one embodiment, in a web browser, a method includes receiving a scaling value associated with a gesture event and dynamically resizing an element of a web page associated with the gesture event in the web browser based on the scaling value, the scaling value being a relative delta in document pixels. A gesture event can be associated with two or more touch events.

[0007] In one embodiment, in a web browser, a method includes receiving a touch list, the touch list including touch data for identifying one or more touches on a web page, the touch data including a touch identifier and at least one set of touch location coordinates, the touch list further including data for a touch event target associated with each touch, the at least one set of touch location coordinates including a set of client coordinates, a set of page coordinates, and a set of screen coordinates. The touch data can identify one or more modified touches.

[0008] Other embodiments directed to systems, methods and computer-readable media are also disclosed. [Brief explanation of the drawings]

[0009] [Figure 1A] , [Figure 1B] 1 illustrates an exemplary web page document.

[0010] [Figure 2] 1 illustrates a processing stack for an exemplary multi-touch capable device.

[0011] [Figure 3] 1 is a flowchart of an example process for handling touch events.

[0012] [Figure 4] 1 illustrates an exemplary multi-touch capable device.

[0013] [Figure 5] FIG. 5 is a block diagram of an exemplary network operating environment for the multi-touch capable device of FIG. 4.

[0014] [Figure 6] FIG. 5 is a block diagram of an embodiment of the multi-touch capable device of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0015] Exemplary Web Page Structure and DOM FIG. 1A illustrates an exemplary web page 100 that can be displayed in a browser. The browser may be hosted on a portable device, such as multi-touch capable device 400 of FIG. 4. Web page 100 may display one or more elements: element 102 (“Element 1”), element 104 (“Element 2”), and element 106 (“Element 3”). Elements 102, 104, and 106 may correspond to areas of web page 100 that a user can select and may provide additional functionality as a result of the selection. For example, these elements correspond to buttons on web page 100. Elements may also be nested, such that one element contains another. For example, element 104 contains element 108. In the illustrated example, element 108 is a scrubber control nested within element 104, which may be, for example, a media player user interface.

[0016] In some embodiments, a user can use a finger, rather than a mouse, to perform various functions on elements of web page 100. A user can touch elements of web page 100, for example, using touch-sensitive display 402 shown in FIG. 4. In one example, a user can select an element by touching the element with one or more fingers and / or performing a gesture such as a swipe, pinch, or rotate motion. To recognize touch input signals, certain areas of web page 100 can be associated with touch event handlers. This can be done with the DOM and embedded scripting languages, as described with reference to FIG. 1B.

[0017] 1B illustrates an exemplary DOM 150 associated with web page 100. DOM 150 provides a structural representation of web page 100, describing the web page content as a set of objects that can be interpreted in a scripting language (e.g., JavaScript®). In one embodiment, DOM 150 provides access to the structure of the web page by mapping elements 102, 104, 106, and 108 of web page 100 to individual nodes in a tree. For example, element 102 corresponds to node 154. Element 104 corresponds to node 156. Element 106 corresponds to node 158. Element 108 corresponds to node 160. Root node 152 corresponds to the entire web page 100.

[0018] In one embodiment, one or more elements 102, 104, 106, 108 of web page 100 can be associated with one or more corresponding touch event handlers by associating a corresponding node in DOM 150 with the touch event handler. The touch event handlers can be inserted into HTML tags of web page 100 and can execute a scripting language to perform actions, for example, when a user touches or gestures within an element of web page 100. For example, JavaScript can work with DOM 150 to attach actions to different touch events.

[0019] In some embodiments, one or more of the elements 102, 104, 106, 108 can receive touch input detected by an event handler or listener. The touch input can be detected and processed into touch events by a touch event model, which can be implemented in one or more layers of a software stack, as described with reference to FIG. 2. The touch events can be further processed by the web page 100. The touch events are in a format (e.g., attributes) that is easier for applications to use than the raw touch input signals generated by a touch-sensing device. For example, each touch event can include a set of coordinates where the touch is currently occurring.

[0020] Each element in web page 100 and its associated event handler can receive, process, and handle touch events. For example, if driver 202 (FIG. 2) senses touch point 110 associated with element 102 or senses touch point 112 associated with element 104, the event handler associated with element 102 or 104 can each receive a separate touch event indicating that the element was touched and, optionally, can send the touch event to web page 100 for further processing. In some embodiments, if an area of ​​web page 100 that does not correspond to an event handler is touched, the browser in application layer 214 can process the input rather than web page 100.

[0021] In some embodiments, touch events can be detected per finger per node in DOM 150. For example, a user can touch touch-sensitive display 402 at touch point 110 and touch point 112 substantially simultaneously, and two separate touch events can be detected by the touch event model. Because each node 102 and node 104 in DOM 150 is associated with a separate touch event handler, separate touch events can be detected for touch point 110 and touch point 112.

[0022] In one embodiment, touch events can be delivered to the web page 100 as an EventTarget. Some examples of touch events include touchstart, touchmove, touchend, and touchcancel. Other touch events are also possible. touchstart is a touch event that is detected when a user first places a finger on the touch-sensitive display 402 in an area of ​​the web page 100 associated with the event handler. As the user moves their finger around on the web page 100, one or more touchmove events can be detected. When the user lifts their finger from the web page 100, a touchend event is detected. touchcancel can be detected when the system suspends normal event handling. For example, a touchcancel event occurs when the touch-sensitive display 402 is locked to prevent accidental touches.

[0023] In some embodiments, a gesture event can also be detected by combining two or more touch events. Similar to touch events, gesture events (GestureEvents) can also be delivered to the web page 100 as EventTargets. Some examples of gesture events are gesturestart, gesturechange, and gestureend. Gesture events can include scale and / or rotation information. The rotation information can include a rotation value, which is a relative delta in degrees. Elements of the web page 100 can be dynamically rotated based on this rotation value. The scale information can include a scale value, which is a relative delta in document pixels. Elements of the web page 100 associated with the gesture event can be dynamically resized based on this scale value. Other gesture events are also possible.

[0024] In some embodiments, a touch list can be received that includes touch event data for identifying one or more touches on web page 100. The touch event data can include a touch identifier and at least one set of touch location coordinates. The touch list can also include touch event data for a touch event target associated with each touch. In some embodiments, the set of touch location coordinates can include client coordinates, page coordinates, and screen coordinates. In some embodiments, the touch event data can identify one or more modified touches.

[0025] In one embodiment, a GestureEvent can be sent to web page 100 before a TouchEvent. For example, if a user places a finger on touch point 110 and touch point 112 and moves the finger clockwise or counterclockwise on the touch-sensitive display to perform a rotation gesture, the touch event model detects these multiple touch events and combines the touch events into a gesture event. This gesture event can then be sent to web page 100, which can then send the combined touch events to form the gesture event. In this way, developers have access to the gesture event and its individual touch events, which gives developers greater flexibility when developing web applications.

[0026] In one embodiment, touch events are received in the following order: a touchstart event, one or more touchmove events, and a touchend or touchcancel event. Using the example of FIG. 1A , when a user touches touch point 110, a first touchstart event is detected by a first touch event handler associated with element 102. When a user touches touch point 112, a second touchstart event is detected by a second touch event handler associated with element 104. As the user rotates their finger without lifting their finger, the first and second touch event handlers detect touchmove events, which can be interpreted as a rotation gesture event by the touch event model. When the user finishes rotating and lifts their finger from web page 100, the first and second touch event handlers detect touchend events. All or some of these touch events can be made available to developers through a touch event “application programming interface (API).” The touch API may be made available to developers as a "software development kit (SDK)" or as part of an application (e.g., as part of a browser toolkit). The touch event API may rely on other services, frameworks, and operating systems to perform its various functions. These services, frameworks, and operating systems may be part of a software or processing stack, as described with reference to FIG. 2, where touch events are associated with attributes that can be inserted into documents to define event actions in an application.

[0027] Example IDL An exemplary touch event model is described below in "Interface Description Language (IDL)." The functions and data structures of the IDL can be accessed through an API by a web designer or application developer. Access to touch and / or gesture events can be associated with attributes that can be inserted into a markup language document (e.g., HTML, XML) to define event actions in an application. For example, these attributes can be inserted into one or more HTML tags of an HTML document to generate a web page that is displayed on touch-sensitive display 402. Event actions can include executing embedded scripts (e.g., JavaScript). TIFF0007766660000001.tif254144 TIFF0007766660000002.tif21581

[0028] The following is an example HTML code snippet for handling touch events using the exemplary IDL described above: The HTML below shows, for example, that touch event listeners TouchStart and GestureStart have been added to an element with HTML code. TIFF0007766660000003.tif24153

[0029] The HTML code corresponding to the IDL may be: TIFF0007766660000004.tif162124 TIFF0007766660000005.tif229146 TIFF0007766660000006.tif236150 TIFF0007766660000007.tif237149 TIFF0007766660000008.tif227150

[0030] Exemplary Processing Stack for a Multi-Touch Device 2 illustrates the processing stack of an exemplary multi-touch capable device. The touch event model described above can be implemented in one or more regions of the processing stack and in various resources of the user in the stack. The hardware 200 layer can include various hardware interface components, such as a touch-sensitive or enabled device or a touch-sensitive display. A touch-sensitive device can include a display and a panel that senses multiple touches simultaneously. The hardware layer 200 can also include an accelerometer for detecting the orientation (e.g., portrait, landscape) of the touch-sensitive display or device. Thus, a signal representing the orientation can be used by the touch event model to scale a web page for optimal display.

[0031] One or more drivers in driver layer 202 can communicate with hardware 200. For example, a driver can receive and process touch input signals generated by a touch-sensitive display or device in hardware layer 200. A core operating system (OS) 204 can communicate with the driver(s). Core OS 204 can process raw input data received from the driver(s). In some embodiments, the drivers can also be considered part of core OS 204.

[0032] A set of OS application programming interfaces (APIs) 206 can communicate with core OS 204. These APIs can be a set of APIs typically included with an operating system (e.g., Linux or UNIX APIs). A set of core foundation APIs 208 can use OS APIs 206, and a set of foundation APIs 210 can use core foundation APIs 208.

[0033] The web page software development kit (SDK) 210 may include a set of APIs designed for use by applications running on the device. A touch event API may be included in the web page SDK 210, for example. The APIs in the web page SDK 210 may use the foundation API 208. The web page SDK 210 may include, for example, the Web Kit provided by Apple Inc. The web page SDK 210 may be offered as an API or may be accessed through an application, for example, a browser such as SAFARI® provided by Apple Inc.

[0034] Applications 214 running on the device can generate web pages using the APIs of the web page SDK 210. The APIs of the web page SDK 210 can then communicate with lower-level elements and ultimately with a touch-sensitive display or device and various other user interface hardware. Each layer can use the layer below it, although this is not always required. For example, in one embodiment, the device 214 can sometimes communicate with the OS API 206.

[0035] Example Touch Event Process FIG. 3 is a flowchart of a process 300 for providing access to touch and / or gesture events via an API. The process 300 can begin by obtaining one or more touch input signals (302). The touch input signals can be obtained from a touch-sensitive display or device. A determination of touch events and / or gestures can be made based on the touch input signals using a touch event model (304). The touch events can be associated with regions of a web page displayed on the touch-sensitive display or device. For example, the touch-sensitive display can be the display of a mobile phone, and the touch-sensitive device can be the touch-sensitive pad of a notebook computer. Access to the touch and / or gesture events can be provided via a programming interface (306). For example, for the HTML fragment described above with reference to FIG. 2, the fragment can be inserted into an HTML document by a web developer, providing the developer with access to touch and / or gesture events. The touch and / or gesture events can be further processed by code in the HTML document to initiate event actions (306).

[0036] Overview of the mobile device 4 is a block diagram of an exemplary multi-touch capable device 400. In one embodiment, the multi-touch capable device 400 includes a touch-sensitive display 402. The touch-sensitive display 402 may implement liquid crystal display (LCD) technology, light-emitting polymer display (LPD) technology, or other display technology. The touch-sensitive display 402 is capable of sensing tactile and / or haptic contact with a user.

[0037] In some embodiments, touch-sensitive display 402 is a multi-touch-sensitive display 402. Touch-sensitive display 402 can, for example, process multiple simultaneous touch points, including processing data related to the pressure, degree, and / or location of each touch point. Such processing facilitates multi-finger gestures and interactions, chording, and other interactions. Other touch-sensitive display technologies can also be used, such as displays in which contact is made using a stylus or other pointing device. Some examples of multi-touch-sensitive display technologies are described in U.S. Patent Nos. 6,323,846, 6,570,557, 6,677,932, and U.S. Patent Publication No. 2002 / 0015024 A1, each of which is incorporated by reference in its entirety. In some embodiments, multi-touch capable device 400 can display one or more graphical user interfaces on touch-sensitive display 402 to provide the user access to various system objects and to convey information to the user.

[0038] Exemplary Multi-Touch Capable Device Functions In some embodiments, multi-touch capable device 400 can perform multiple device functions, such as a telephone device, an email device, a network data communication device, a Wi-Fi base station device, and a media processing device. In some embodiments, multi-touch capable device 400 can include a web browser 404 for displaying web pages (e.g., web page 100). Touch-sensitive display 402 can receive touch input signals made on web page 100 and can determine touch and / or gesture events based on the touch input signals using the touch model described above. In some embodiments, multi-touch capable device 400 can perform network distribution functions. In some embodiments, touch-sensitive display 402 can lock down when multi-touch capable device 400 is near a user's ear. This lockdown generates a touchcancel event, as described with reference to FIG. 1B.

[0039] In some embodiments, accelerometer 472 can be used to detect movement of multi-touch capable device 400, as indicated by directional arrow 474. Display objects and / or media can therefore be presented based on the detected orientation, such as portrait or landscape. In some embodiments, multi-touch capable device 400 can include circuitry and sensors to support location determination capabilities, such as those provided by a global positioning system (GPS) or other positioning systems (e.g., systems using Wi-Fi access points, television signals, cellular grids, uniform resource locators (URLs)). In some embodiments, the positioning system (e.g., a GPS receiver) can be integrated into multi-touch capable device 400 or can be provided as a separate device that can be coupled to multi-touch capable device 400 via an interface to provide access to location-based services. Multi-touch capable device 400 can also include one or more wireless communication subsystems.

[0040] In some embodiments, a port device, such as a Universal Serial Bus (USB) port or docking port or other wired port connection, can be included that can be used, for example, to establish a wired connection to another computing device, such as another multi-touch capable device 400, a network access device, a personal computer, a printer, or other processing device that can receive and / or transmit data. In some embodiments, the port device allows the multi-touch capable device 400 to synchronize with a host device using one or more protocols, such as TCP / IP, HTTP, UDP, and other known protocols.

[0041] Network Operating Environment Figure 5 is a block diagram of an exemplary network operating environment 600 for the multi-touch capable device 400 of Figure 4. The multi-touch capable device 400 of Figure 4 can communicate over one or more wired and / or wireless networks 510, for example, in data communication. For example, the wireless network 512, e.g., a cellular network, can communicate with a wide area network (WAN) 514, such as the Internet, through the use of a gateway 516. Similarly, an access point 518, such as an 802.11g wireless access point, can provide communication access to the wide area network 514. In some embodiments, both voice and data communication can be established over the wireless network 512 and the access point 518. For example, multi-touch capable device 400a can make and receive telephone calls (e.g., using a VoIP protocol), send and receive email messages (e.g., using a POP3 protocol), and retrieve electronic documents and / or streams such as web pages, photos, and videos via wireless network 512, gateway 516, and wide area network 514 (e.g., using a TCP / IP or UDP protocol). Similarly, multi-touch capable device 400b can make and receive telephone calls, send and receive email messages, and retrieve electronic documents via access point 518 and wide area network 514. In some embodiments, multi-touch capable device 400 can be physically connected to access point 518 using one or more cables, and access point 518 can be a personal computer. In this configuration, multi-touch capable device 400 can be referred to as a "tethered" device.

[0042] Multi-touch capable devices 400a and 400b may also establish communications through other means. For example, multi-touch capable device 400a may communicate with other wireless devices, such as other multi-touch capable devices 400, cellular telephones, etc., via wireless network 512. Similarly, multi-touch capable devices 400a and 400b may establish peer-to-peer communications 520, such as a personal area network, through the use of one or more communications subsystems, such as Bluetooth® communications device 488 shown in FIG. 4. Other communications protocols and topologies may also be implemented.

[0043] The multi-touch capable device 400 can communicate with network resources 530, for example, via one or more wired and / or wireless networks 510. For example, the network resources can be web servers for delivering touchable web pages via the touch model, as described with reference to Figures 1 and 2.

[0044] Other services may also be provided, including a software update service that automatically determines whether software updates exist for software on the multi-touch capable device 400 and then downloads the software updates to the multi-touch capable device 400, where they can be manually or automatically unpacked and / or installed.

[0045] Exemplary Mobile Device Architecture Figure 6 is a block diagram 600 of an embodiment of the multi-touch capable device 400 of Figure 4. The multi-touch capable device 400 may include a memory interface 602, one or more data processors, a video processor and / or central processing unit 604, and a peripherals interface 606. The memory interface 602, the one or more processors 604, and / or the peripherals interface 606 may be discrete components or may be integrated into one or more integrated circuits. The various components in the multi-touch capable device 400 may be coupled by one or more communication buses or signal lines.

[0046] Sensors, devices, and subsystems can be coupled to the peripherals interface 606 to facilitate multiple functions. For example, a motion sensor 610, a light sensor 612, and a proximity sensor 614 can be coupled to the peripherals interface 606 to facilitate the orientation, lighting, and proximity functions described with reference to Figure 4. Other sensors 616, such as a positioning system (e.g., a GPS receiver), temperature sensors, biometric sensors, or other sensor devices, can also be connected to the peripherals interface 606 to facilitate related functions.

[0047] Additionally, a camera subsystem 620 and optical sensor 622, such as a charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) optical sensor, can be used to facilitate camera functions such as recording photographs and video clips.

[0048] Additionally, communication functions may be facilitated by one or more wireless communication subsystems 624, including radio frequency receivers and transmitters and / or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the communication subsystem 624 will depend on the communication network(s) over which the multi-touch capable device 400 is intended to operate. For example, the multi-touch capable device 400 may include a communication subsystem 624 designed to operate over a GSM network, a GPRS network, an EDGE network, a Wi-Fi or WiMax network, and a Bluetooth network. In particular, the wireless communication subsystem 624 may include a hosting protocol such that the device 500 may be configured as a base station for other wireless devices.

[0049] An audio subsystem 626 may be coupled to a speaker 628 and a microphone 630 to facilitate voice-enabled functions, such as voice recognition, voice transcription, digital recording, and telephony functions.

[0050] The I / O subsystem 640 can include a touchscreen controller 642 and / or other input controller(s) 644. The touchscreen controller 642 can be coupled to a touchscreen 646. The touchscreen 646 and touchscreen controller 642 can detect contact and movement or detachment using any of a number of touch-sensing technologies, including, for example, but not limited to, capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements to determine one or more points of contact with the touchscreen 646.

[0051] Other input controller(s) 644 may be coupled to other input / control devices 648, such as one or more buttons, rocker switches, thumbwheels, infrared ports, USB ports, and / or pointer devices such as a stylus. One or more buttons (not shown) may include up / down buttons for volume control of speaker 628 and / or microphone 630.

[0052] In one embodiment, pressing the button for a first period of time unlocks the touchscreen 646, and pressing the button for a second period of time longer than the first period of time can turn power on or off to the multi-touch capable device 400. A user can customize the function of one or more of the buttons. The touchscreen 646 can also be used to implement, for example, virtual or soft buttons and / or a keypad or keyboard.

[0053] In one embodiment, the multi-touch capable device 400 can present recorded audio and / or video files, such as MP3, AAC, and MPEG files. TM It can contain functions such as MP3 player. The touch-capable device 400 may include a 32-pin connector compatible with an iPod, but other input / output and control devices may also be used.

[0054] The memory interface 602 can be coupled to memory 650. The memory 650 can include high-speed random-access memory and / or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices, and / or flash memory (e.g., NAND, NOR). The memory 650 can store an operating system 652, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or an embedded operating system, such as VxWorks. The operating system 652 can include instructions for handling basic system services and performing hardware-dependent tasks.

[0055] Memory 650 may also store communication instructions 654 to facilitate communication with one or more additional devices, one or more computers, and / or one or more servers. Memory 650 may include graphic user interface instructions 656 for facilitating graphic user interface processes, sensor processing instructions 658 for facilitating sensor-related processes and functions, telephony instructions 660 for facilitating telephony-related processes and functions, electronic messaging instructions 662 for facilitating electronic messaging-related processes and functions, web browsing instructions 664 for facilitating web browsing-related processes and functions, media processing instructions 666 for facilitating media processing-related processes and functions, GPS / navigation instructions 668 for facilitating GPS and navigation-related processes and functions, camera instructions 670 for facilitating camera-related processes and functions, and / or other messaging instructions 672 for facilitating the processes and functions described with reference to FIGS. 1-5 .

[0056] Each of the instructions and applications described above may correspond to a set of instructions for performing one or more of the functions described above. These instructions need not be implemented as separate software programs, procedures, or modules. Memory 650 may include additional instructions or fewer instructions. Various functions of multi-touch capable device 400 may be implemented in hardware and / or software, including one or more signal processing and / or application specific integrated circuits.

[0057] The features described herein can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. The features can also be embodied in a computer program product tangibly embodied in an information carrier, such as a machine-readable memory device or a propagated signal, for execution by a programmable processor, such that the programmable processor executes a program of instructions to perform the functions of the embodiments by operating on input data and generating output.

[0058] The features described above may be conveniently embodied in one or more computer programs executable on a programmable system including at least one programmable processor coupled to receive data and instructions from and provide data and instructions to a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used directly or indirectly in a computer to perform an activity or obtain a result. Computer programs may be written in any form of programming language (e.g., Objective-C, Java), including compiled or interpreted languages, and may be deployed in any form including components, subroutines, or other units suitable for use in a computing environment, either as stand-alone programs or modules.

[0059] Processors suitable for executing a program of instructions include, for example, both general-purpose and special-purpose microprocessors, sole processors or one of multiple processors or cores of any kind of computer. Typically, a processor receives instructions and data from a read-only memory or a random-access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Typically, a computer also includes, or is operatively coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, e.g., internal hard disks and removable disks, magneto-optical disks, and optical disks. Storage devices suitable for tangibly carrying computer program instructions and data include all forms of non-volatile memory, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).

[0060] To provide for user interaction, these features may be embodied in a computer having a display device, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user, and a keyboard and pointing device, such as a mouse or trackball, by which the user can provide input to the computer.

[0061] These features may also be embodied in a computer system that includes back-end components such as data servers, middleware components such as application servers or Internet servers, or front-end components such as client computers having a graphical user interface or Internet browser, or a combination thereof. These components of the system may be connected by any form or medium of digital data communication, such as a communications network. Communications networks include, for example, LANs, WANs, and the computers and networks forming the Internet.

[0062] A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0063] A number of embodiments have been described. Nevertheless, it should be understood that various modifications may be made. For example, elements of one or more embodiments may be combined, deleted, modified, or supplemented to form further embodiments. As a further example, the illustrated logic flows do not require the particular order shown, or sequential order, to achieve desired results. Moreover, other steps may be added or deleted from the flows described, and further, other components may be added to or removed from the systems described. Accordingly, these other embodiments are encompassed by the following claims. [Explanation of symbols]

[0064] 100:Webpage 102: Element 1 (Node 1) 104: Element 2 (Node 2) 106: Element 3 (Node 3) 108: Element 4 (Node 4) 110: Touch 112: Touch 152: Node 0 154: Node 1 156: Node 2 158: Node 3 160: Node 4 200: Hardware 202: Driver 204: Core OS 206: Core Foundation 206:OS API 208: Foundation 210:Webpage SDK 212: Application 514: Wide Area Network 516: Gateway 518: Access point 530: Network resources (web servers)

Claims

1. 1. A method executed by an electronic device having one or more processors and a memory storing one or more programs to be executed by the one or more processors, the method comprising: receiving an input event via an input device that accepts one or more finger or gesture inputs by a user of the electronic device, the input event including a plurality of differentiated input lists, the plurality of differentiated input lists comprising: a first input list including an input list for multiple simultaneous inputs associated with a target of the input event; a second input list distinct from the first input list, the second input list including an input list for one or more modified inputs of the plurality of simultaneous inputs; Acceptance, including processing the input events according to the plurality of distinct input lists; A method comprising:

2. 2. The method of claim 1, comprising processing the input event according to the plurality of concurrent inputs in the first input list and according to the one or more modified ones of the plurality of concurrent inputs in the second input list.

3. the plurality of distinct input lists includes a third input list including an input list including all inputs including the plurality of simultaneous inputs detected by one or more input devices, the third input list being distinct from the first input list and the second input list; 3. The method of claim 1, wherein the method comprises processing the input event according to the one or more modified inputs of the plurality of concurrent inputs in the second input list and the inputs in the third input list.

4. The method of claim 1 , wherein the multiple simultaneous inputs include multiple simultaneous touch inputs.

5. Processing the input events according to the plurality of distinct input lists includes: receiving, from one or more instructions embedded in a web page according to the plurality of distinct input lists, a plurality of values ​​corresponding to the plurality of concurrent inputs in at least one of the plurality of distinct input lists, the plurality of concurrent inputs corresponding to different locations on the web page; generating a representation of the web page according to the plurality of values; 5. The method of claim 1, comprising:

6. 1. An electronic device comprising: one or more processors; a memory storing one or more programs to be executed by the one or more processors; wherein the one or more programs are receiving an input event via an input device that accepts one or more finger or gesture inputs by a user of the electronic device, the input event including a plurality of differentiated input lists, the plurality of differentiated input lists comprising: a first input list including an input list for multiple simultaneous inputs associated with a target of the input event; a second input list distinct from the first input list, the second input list including an input list for one or more modified inputs of the plurality of simultaneous inputs; Acceptance, including processing the input events according to the plurality of distinct input lists; An electronic device that includes instructions for executing a program.

7. The one or more programs are 7. The electronic device of claim 6, comprising instructions for processing the input event in accordance with the plurality of concurrent inputs in the first input list and in accordance with the one or more modified ones of the plurality of concurrent inputs in the second input list.

8. the plurality of distinct input lists includes a third input list including an input list including all inputs including the plurality of simultaneous inputs detected by one or more input devices, the third input list being distinct from the first input list and the second input list; 8. The electronic device of claim 6, wherein the one or more programs include instructions for processing the input event according to the one or more modified inputs of the plurality of concurrent inputs in the second input list and the inputs in the third input list.

9. The electronic device of claim 6 , wherein the multiple simultaneous inputs include multiple simultaneous touch inputs.

10. Processing the input events according to the plurality of distinct input lists includes: receiving, from one or more instructions embedded in a web page according to the plurality of distinct input lists, a plurality of values ​​corresponding to the plurality of concurrent inputs in at least one of the plurality of distinct input lists, the plurality of concurrent inputs corresponding to different locations on the web page; generating a representation of the web page according to the plurality of values; 10. The electronic device of claim 6, comprising:

11. A computer program executed by one or more processors of an electronic device, comprising: receiving an input event via an input device that accepts one or more finger or gesture inputs by a user of the electronic device, the input event including a plurality of differentiated input lists, the plurality of differentiated input lists comprising: a first input list including an input list for multiple simultaneous inputs associated with a target of the input event; a second input list distinct from the first input list, the second input list including an input list for one or more modified inputs of the plurality of simultaneous inputs; Acceptance, including processing the input events according to the plurality of distinct input lists; A computer program containing instructions for executing a program.

12. 12. The computer program product of claim 11, comprising instructions for processing the input event in accordance with the plurality of concurrent inputs in the first input list and in accordance with the one or more modified ones of the plurality of concurrent inputs in the second input list.

13. the plurality of distinct input lists includes a third input list including an input list including all inputs including the plurality of simultaneous inputs detected by one or more input devices, the third input list being distinct from the first input list and the second input list; 13. A computer program product according to claim 11 or 12, comprising instructions for processing the input event according to the one or more modified inputs of the plurality of concurrent inputs in the second input list and the inputs in the third input list.

14. The computer program product of claim 11 , wherein the multiple simultaneous inputs include multiple simultaneous touch inputs.

15. Processing the input events according to the plurality of distinct input lists includes: receiving, from one or more instructions embedded in a web page according to the plurality of distinct input lists, a plurality of values ​​corresponding to the plurality of concurrent inputs in at least one of the plurality of distinct input lists, the plurality of concurrent inputs corresponding to different locations on the web page; generating a representation of the web page according to the plurality of values; 15. A computer program according to any one of claims 11 to 14, comprising:

Images