Physical Cursor Control in Microfluidic Display Devices

Abstract

Physically assisting in identifying an input location on the touchscreen may include detecting that a user has touched the touchscreen with a pointing device at an initial touch point, predicting an input location on the touchscreen for accepting input from one or more input locations, determining a path from the initial touch point to the predicted input location, and generating a physical cursor on the touchscreen that is a raised portion of the touchscreen in proximity to the pointing device.

Description

[Technical field]

[0001] The present invention relates to providing physical assistance for user input, and more particularly to providing a physical cursor through microfluidics on a touch screen to assist manual input for controlling the cursor. Summary of the Invention

[0002] One embodiment relates to a method for physically assisting in locating an input location on a touch screen, which may include detecting when a user touches a touch screen with a pointing device at an initial touch point, predicting an input location on the touch screen for accepting input from one or more input locations, determining a path from the initial touch point to the predicted input location, and generating a physical cursor on the touch screen that is a raised portion of the touch screen in proximity to the pointing device.

[0003] Further embodiments relate to systems and computer program products for physically assisting in locating input locations on a touch screen.

[0004] This Summary is intended to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. [Brief description of the drawings]

[0005] [Figure 1A] FIG. 2 is a cross-sectional cutaway view of a touch screen. [Figure 1B] FIG. 1B is a cutaway cross-sectional view of the touch screen of FIG. 1A when a portion of the touch screen is raised through microfluidics. [Diagram 2] 1 is a diagram of a touch screen display showing example input locations and a physical cursor according to one embodiment. [Diagram 3] 3 is a diagram of the touch screen display and physical cursor of FIG. 2 illustrating movement of the physical cursor along a path between an initial touch point and an input location, according to one embodiment. [Figure 4] 3 is a diagram of the touch screen display of FIG. 2 showing an alternative embodiment of a physical cursor. [Figure 5A] FIG. 5 is a side view of the physical cursor shown in FIGS. 2-4, according to various embodiments. [Figure 5B] FIG. 5 is a side view of the physical cursor shown in FIGS. 2-4, according to various embodiments. [Figure 6] 1 is a flowchart of a physical cursor control process for a microfluidic display device according to one or more embodiments. [Figure 7] FIG. 13 is a block diagram of a physical cursor module's input and machine learning models for determining user characteristics and level of assistance required. [Figure 8] FIG. 1 is a block diagram of an example computer system in which various embodiments may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] For some individuals who have limited hand mobility and / or are visually impaired, controlling the position of a traditional cursor on a touch screen device can be difficult. Other users may want to provide input to a touch screen device when not looking at the display. Some examples are when the cursor is between text fields, when the user wants to directly move a progress bar in video content, or when the user simply wants to guide the cursor to a specific location. For these users, moving the cursor accurately can be problematic, especially for users with visual impairments, or disorders affecting the central nervous system such as multiple sclerosis (MS), Parkinson's disease, or tardive dyskinesia (TD), or for the elderly.

[0007] FIG. 1A illustrates a touch screen of a computer system (hereinafter, "system"). At least a portion of the touch screen has multiple layers. A first layer 151 is a touch interface with which a user directly interacts. A second layer 158 is between the first layer 151 and a third layer 159. The second layer 158 has multiple holes 154. The multiple holes 154 may be distributed throughout the screen. A reservoir 160 is formed between the second layer 158 and the third layer 159. The reservoir 160 is in fluid communication with the multiple holes 154 and one or more passages 153 formed between the second layer 158 and the third layer 159 and connecting to a microfluidic source (not shown). The multiple holes may be distributed in various patterns (e.g., a matrix pattern).

[0008] In one embodiment, one or more physical cursors or other objects may be provided on the touch screen for use with a particular web page or screen of an application, where the physical cursor or other object is provided to be overlaid on the touch screen over which fluid 156 can form a ridge, button, or any other desired shape. The processor or computer of the system preferably provides the physical cursor or other object having at least one or more intersections with the microfluidic passages 153 and corresponding holes 154, such that a raised portion corresponding to the cursor or other object can be formed on the touch screen of the device.

[0009] 1A is a cross-sectional view of a portion of a touchscreen in a position where no fluid 156 is provided to reservoir 160. When reservoir 160 is not provided with fluid 156, the entire first layer 151 of touchscreen 150 remains in contact with second layer 158.

[0010] 1B, the touchscreen is shown in a position where fluid 156 is provided from a source (not shown) through passageway 153 into reservoir 160. Fluid flows from passageway 153 and reservoir 160 through holes 154 in second layer 158, forming a pocket 157 of fluid between first layer 151 and second layer 158. The pressure of fluid 156 causes first layer 151 to separate from second layer 158, inflating pocket 157. Pocket 157 forms a bubble 152 or ridge relative to the remainder of first layer 151. Raised bubble 152 does not typically rise to a uniform height above layer 151. In various embodiments, raised bubble 152 may have a maximum height in the range of 0.25 to 5 mm. By way of example, ridge 152 may have a maximum height of 1 mm. It should be noted that fluid may be selectively applied to different portions of the touch screen 150. Additionally, the bubble 152 or ridge may be moved to a series of locations on the touch screen, for example, by sequentially applying fluid to a number of first locations adjacent a first side of the bubble while simultaneously removing fluid from a number of second locations adjacent an opposite side of the bubble, as shown in Figures 2 and 3.

[0011] The microfluidic cursor or other object may be referred to herein as a "raised bubble" or a "ridge." The microfluidic cursor or other object may also be referred to herein as a "physical cursor." It is understood that these alternative terms have the same meaning and may be considered synonyms.

[0012] 2 is a top plan view of a touch screen 220 on which a device 200 can display a web page or screen of an application, according to one embodiment. The system can identify an input location 202 or 212 where input from a user may be requested. Note that although multiple input locations 202 or 212 are shown to illustrate that the system may identify the input location 202 or 212 anywhere on the touch screen 220, typically only one input location is selected at a time. In one embodiment, the input location 202 or 212 can be a form field that requests data from the user. In another embodiment, the input location 202 or 212 can be a control associated with an audio or video clip (such as volume or video progression) or a scroll bar of a software window. When a user touches the screen 220, the system can detect an initial touchpoint 201 of the user's finger 206 on the touch screen 220. It is noted that it is not necessary for the user to use a finger to interact with the touch screen 220; any pointing device suitable for use with a touch screen, such as a stylus, may be used in place of the user's finger. A path 204 and 214 between the touch point 206 and the respective input location 202 or 212 may be calculated. Assume now that the input location 202 is determined to be the input location where input from the user is required, and a path 204 is calculated. Using the microfluidic technology of the device 200, the system may form a physical cursor 208 on the touch screen by raising the display material to form a small bubble just in front of the touch point 206 on the calculated path 204. The physical cursor 208 may be of any suitable shape, but preferably fits the front side of the finger or other pointing device. The physical cursor 208 may extend along one or more side portions of the finger or other pointing device.The physical cursor 208 may extend partially (as shown) or completely along the side of the finger or other pointing device (not shown in FIG. 2). It is not necessary that the physical cursor 208 extend along the side of the finger or other pointing device, and the physical cursor 208 may be located only on the "front" side of the finger, i.e., the side of the finger that is on the path 204 between the input location 202 and the touch point 206. An example of a physical cursor 504 located on the "front" side of the finger is shown in a side view in FIG. 5A. In FIG. 5A, the physical cursor 504 moves in the direction of the arrow 510. When the system detects the movement of the finger 206, the system can track this movement and move the physical cursor 208 continuously to match the user's movement of the finger or other pointing device. The physical cursor 208 moves essentially along the path 204 to the input location 202, as indicated by the arrow 230. When an input location 212 is selected, the physical cursor 208 will move essentially along the path 214 to the input location 212, as indicated by arrow 240. The system may also determine whether this movement is along the calculated path 204. If the physical cursor 208 remains on the path 204, the system takes no action other than to proceed along the path 204. However, if the system determines that the user's finger 206 or other pointing device has deviated from the path 204, or that the cursor movement has stopped or is moving in an unpredictable direction, the system may change the direction or position of the physical cursor movement such that the physical cursor 208 guides or leads the user's finger or other pointing device 206 toward the input location 202.

[0013] FIG. 3 is a block diagram of the system when the physical cursor 208 is in a new position, and is used to explain the potential movement of the physical cursor 208 along the path 204. The system monitors the movement of the physical cursor 208 and the finger or other pointing device 206 from the initial touch point 201 until the input location 202 is reached. In the example of FIG. 2 and FIG. 3, FIG. 2 shows the position of the physical cursor 208 at a first time, and FIG. 3 shows the position of the physical cursor 208 at a second time subsequent to the first time. In this example, the physical cursor 208 may have proceeded along the path without deviation. Alternatively, the physical cursor 208 may have guided or nudged the user's finger or other pointing device 206 back to the path in response to moving right or left from the path. This guidance may occur at a time between the first time and the second time. In FIG. 3, it is understood that the physical cursor 208 moves essentially along the path 204 to the input location 202, as indicated by the arrow 310.

[0014] FIG. 4 is a plan view of the touch screen 220 from above. In the first example in FIG. 4, the input location 202 shown in FIG. 2, a user's finger or other suitable pointing device 206 at an initial touch point 201, a path 204 between the touch point 201 and the input location 202, and a physical cursor 208 are shown. In addition to the physical cursor 208, an additional ridge 209 is shown in this first example. Thus, in one embodiment, the finger or other pointing device can be guided or nudged along the path 204 by the additional ridge 209 while being guided by the physical cursor 208. It is noted that it is not necessary to use the "trailing" additional ridge 209 together with the "leading" physical cursor 208. In some embodiments, only the "trailing" additional ridge 209 may be used to guide or nudge the finger or other pointing device to the input location. In this first example, it can be seen that the physical cursor 208 moves essentially along the path 204 to the input location 202, as indicated by arrow 410.

[0015] 4 shows an input location 412, a user's finger or other pointing device 206 at an initial touch point 404, a path 414 between the initial touch point 404 and the input location 412, and an alternative physical cursor 210 according to various embodiments. The alternative physical cursor 210 may be an oval or generally circular shape that surrounds and conforms to the finger or other pointing device. In this second example, it is understood that the alternative physical cursor 210 moves essentially along the path 414 to the input location 412, as indicated by arrow 420. FIG. 5B is a side view of the alternative physical cursor 210 according to one embodiment, where the alternative physical cursor 210 moves in the direction of arrow 520.

[0016] 6 is a flow chart of a process 600 for forming and controlling a physical cursor on a microfluidic display device. The process 600 can be performed by any computer system with a touch screen display. It is noted that while the process 600 is described with reference to a computer system with a touch screen display, one or more steps or operations may be performed by a server or second computer located remotely from and in communication with the computer system with the touch screen display. At step 602, a user interface screen (e.g., a web page or a media player) is displayed on the touch screen 220. The displayed screen is then analyzed to identify or predict input locations where manual user input may be required. In one embodiment, the touch screen 220 may display a form requesting input of text information into fields, i.e., input locations, and the system may identify that some fields have been populated with data and some fields are blank, and may select the first of the blank fields, excluding the fields with data. In another embodiment, the system may determine that all data fields have been populated and select a submit button on the screen as the input location. In other embodiments, the input location may be an audio or video progression control for a multimedia clip displayed in a media player on a touch screen, or a scroll bar on an active window that requires the user to scroll the window by dragging the bar vertically. In still other embodiments, for example, if there is more than one input location, the system may select as the input location the input location closest to the finger touch point on the screen.

[0017] At step 604, the system may use a machine learning model, such as a recurrent neural network, to learn the user's characteristic behaviors and predict a series of locations for placing the physical cursor on the touch screen 220 for that particular user. In one embodiment, the continuous recurrent neural network model may learn how the user has interacted with the cursor in the past and track the user's patterns, including mistakes and tremors, while controlling the cursor movement. Based on the learning data including the user's interaction patterns with the physical or traditional cursor, the model may be configured to determine the user's level of attention while moving the cursor or identify whether the user needs extra assistance in moving his or her hand or the physical cursor. The model may also determine the user's dexterity level in determining the level of assistance needed. In another embodiment, the system may use traditional camera images or ultrasound or infrared images to detect whether the user is looking at the display while attempting to move the physical cursor and combine this information with the model's prediction of whether the user is not concentrating on the display and therefore needs assistance. In yet another embodiment, the model may take into account the medical profile of a particular user and may identify that the user is visually impaired or has difficulty controlling cursor movement for medical reasons.

[0018] The model may also retrieve the user's medications using the medical profile, and may include the medication data as a factor in predicting whether assistance is needed and, if so, in determining to adjust the position of the physical cursor accordingly. The model may receive as input the time of day, the usual or actual time to take the medication, and a pattern of hand tremor or physical cursor movement tremor throughout the day based on historical data. In one embodiment, the model may predict that assistance is needed based on the time of day, the time to take the medication, and a historical pattern of hand tremor. For example, consider a user who takes medication in the morning and experiences little or no tremor in the morning, but experiences increasing tremor as time passes. The model may predict that such a user would benefit from cursor control assistance in the afternoon or evening. In another embodiment, the model may predict that a user has forgotten to take medication at a given time based on the time of day, a given time to take the medication, and a historical pattern of hand tremor. For example, consider the same user who takes medication in the morning and experiences no tremor pattern in the morning, but experiences tremor pattern later in the day. If the system detects tremors or shaking in the morning for this user, essentially in the finger movements along the path from the touch point to the input location, it can determine that the user has not taken their medication at the required time and issue an alert to the user or a healthcare provider. In another embodiment, the model may predict that a medication will cause tremors. For example, consider a particular user who has not experienced a pattern of hand tremors in the past. If the system detects tremors or shaking in the finger movements along the path from the touch point to the input location for this user, it can determine that a medication recently prescribed for this user is causing tremors (as an undesirable side effect) and issue an alert to the user or a healthcare provider. The machine learning model and its inputs are further detailed in FIG. 7 below.

[0019] In step 606, the system can detect the touch point of a user's finger or other pointing device on the touch screen and calculate a path between the touch point and the input location by known methods.

[0020] At step 608, the system may form a physical cursor at or around the touch point on the touch screen by the microfluidic technology described in Figures 1A and 1B. The microfluidic hole array may span the entire display surface so that the cursor may be formed anywhere on the touch screen display. The physical cursor may take any of a variety of different shapes at various locations relative to the finger or other pointing device. In one embodiment, the system may form a bubble just in front of the user's finger so that the user can push the physical cursor towards the input location. In another embodiment, the system may form a physical cursor around or underneath the user's finger or other pointing device so that the finger or other pointing device is brought to the input location. The type and size of the physical cursor formed may depend on the characteristics of the user detected at step 604 and the level of assistance the system determines is needed. For example, the size of the physical cursor may be adjusted to fit the size or shape of a particular finger or other pointing device. As another example, for a user who has great difficulty moving from the initial touch point to the input location, a physical cursor may be provided that is placed in contact with or close to the finger on two, three, or four sides of the finger, rather than just one side of the finger. As yet another example, the system may determine that the user stays on the path better when the physical cursor is in front of, behind, or around the finger or other pointing device, and select a location and shape that allows the user to stay closer to or better on the path. The system may make this determination based on past dates of using physical cursors with different shapes. In another embodiment, the size of the physical cursor may be increased based on a selection of a zoom level of the displayed text, such that the dimensions of the physical cursor are consistent with the text or other features displayed on the touch screen.

[0021] In one embodiment, the system can also generate haptic effects such as vibrating the physical cursor or making other adjustments to the physical cursor in response to the user's movements if the user is a low-attention user or visually impaired and needs assistance in locating the location of the physical cursor relative to the underlying page displayed on the touch screen. The type of haptic effect and the location of use may be set or selected by the user. In another example, if the cursor is moving vertically, the cursor may move across the text lines and the user can also understand the location of the physical cursor. In another example, if the physical cursor is used with a progress bar or scroll bar for media or if a program window is displayed, haptic effects can be enabled to allow a visually impaired user or a user not looking at the display to locate the location of the physical cursor in the progress bar on the scroll bar. After the physical cursor is found, the user can move a finger or other pointing device to change the location of the physical cursor.

[0022] At step 610, the system may continually update the position of the physical cursor based on the user's movements. The system may take over control from the user as necessary to stay on the calculated path until the input location is reached. Once the physical cursor is formed on the touch screen, the system may limit the movement of the physical cursor from deviating from the path, so that touches off the calculated path or mis-touches on the touch screen should not affect the position of the physical cursor.

[0023] In one embodiment, upon reaching the input location, the system may be configured to recognize that the user wishes to select a particular content within the input location area by simultaneously touching the physical cursor and applying pressure equal to or greater than a threshold level. For example, the input area may include a "yes" and a "no" checkbox. The user may also move a finger or other pointing device over words of text while applying pressure, and may select the desired text area accordingly. The system may enable the user to physically feel the selected area, and may even raise the surface of the selected text. In another embodiment, the system may alert the user that the input location has been reached using the haptic effects described above (e.g., vibration or other adjustment of the physical cursor). In some embodiments, the system may gradually remove the physical cursor from the screen as it approaches the input location. In this case, the physical cursor disappears when the user reaches the input location.

[0024] FIG. 7 next illustrates an example of components or modules of a custom process and hardware components according to at least one embodiment. According to one embodiment, the process may include a physical cursor module 808. The physical cursor module 808 utilizes supervised machine learning 730 to determine the level of user assistance required based on potential medical reasons or analysis of images. The image module 702 may include a camera interface 704 that processes images of the user captured by a camera for use in analyzing the driver's condition. For example, processing of images captured by the camera may determine that the user is not looking at the display and needs assistance to move the physical cursor 206 to an input location. The physical cursor module 808 may also analyze user details 710. For example, the user has had difficulty moving their hands in the past and needed assistance to move the physical cursor 206 to an input location. The physical cursor module 808 may also analyze the user's medical history 712. For example, the user may have a medical condition that makes it difficult to move their hands or may be taking medication that tends to hinder dexterity, as described above in step 604. The physical cursor module 808 uses the above information to determine the level of assistance 726 the user needs to move the physical cursor 208 from the touch point 206 to the input location.

[0025] Next, FIG. 8 is a block diagram of a computer system 800 according to an embodiment. As shown, the computer system 800 includes a processor unit 802, a memory unit 804, a persistent storage 806, a communication unit 812, an input / output unit 814, a display 816, and a system bus 810. Data such as user details 710 may be stored in persistent storage 806. In addition, computer programs such as a physical cursor module 808 and an image module 702 may also be stored in persistent storage 806 until required for execution, at which time the programs are called into memory unit 804 for direct access by the processor unit 802. The processor unit 802 selects a portion of the memory unit 804 for reading and / or writing using an address that the processor 802 provides to the memory 804 along with a read and / or write request. Typically, reading and interpreting an encoded instruction at one address causes the processor 802 to fetch subsequent instructions at either a subsequent address or some other address. The processor unit 802, memory unit 804, persistent storage 806, communication unit 812, input / output unit 814, and display 816 interface with each other via a system bus 810. The input / output unit 814 may interface with a camera or other device (not shown) suitable for taking ultrasound or infrared images.

[0026] The present invention may be a system, method or computer program product, or combination thereof, integrated at any possible level of technical detail. The computer program product may include a computer readable storage medium having stored thereon computer readable program instructions for causing a processor to carry out aspects of the present invention.

[0027] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. The computer-readable storage medium may be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or a suitable combination thereof. More specific examples of computer-readable storage media include portable computer diskettes, hard disks, RAM, ROM, EPROM (or flash memory), SRAM, CD-ROM, DVD, memory sticks, floppy disks, punch cards or ridges in grooves or other mechanically encoded devices with instructions recorded thereon, and suitable combinations thereof. As used herein, a computer-readable storage medium should not be construed as a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a wave guide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electric signal transmitted through a wire.

[0028] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computer device / processing device. Alternatively, they can be downloaded to an external computer or external storage device via a network (e.g., the Internet, a LAN, a WAN, or a wireless network, or a combination thereof). The network can include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computer device / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions to a computer-readable storage medium in the respective computer device / processing device for storage.

[0029] The computer readable program instructions for carrying out the operations of the present invention may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, including object oriented programming languages ​​such as Smalltalk and C++, and procedural programming languages ​​such as the "C" programming language and similar programming languages. The computer readable program instructions may be executed entirely on the user's computer as a stand-alone software package, or partially on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a LAN or WAN, or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry, including, for example, programmable logic circuitry, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), can execute computer readable program instructions by utilizing state information of the computer readable program instructions to customize the electronic circuitry for carrying out aspects of the present invention.

[0030] Aspects of the present invention are described herein with reference to flowchart and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. Each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer readable program instructions.

[0031] These computer readable program instructions can be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, whereby the instructions executed via the processor of such computer or other programmable data processing apparatus create means for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer readable program instructions can further be stored on a computer readable storage medium capable of instructing a computer, programmable data processing apparatus or other device, or combination thereof, to function in a particular manner. The computer readable storage medium having instructions stored thereon thereby constitutes an article of manufacture including instructions for performing the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0032] Also, computer-readable program instructions may be loaded into a computer, other programmable device, or other apparatus and a sequence of operational steps executed on the computer, other programmable device, or other apparatus to generate a computer-implemented process, whereby the instructions executing on the computer, other programmable device, or other apparatus perform the functions / operations identified in one or more blocks in the flowcharts and / or block diagrams.

[0033] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for performing a specific logical function. In some other implementations, the functions shown in the blocks may be performed in a different order than the order shown in the figures. For example, depending on the functions involved, two blocks shown in succession may actually be accomplished as one step, may be performed simultaneously or nearly simultaneously, may be performed in a partially or fully overlapping manner, or the blocks may be performed in reverse order in some cases. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of multiple blocks in the block diagrams and / or flowcharts, may be executed by a dedicated hardware-based system that performs a specific function or operation or executes a combination of dedicated hardware and computer instructions.

[0034] Various embodiments of the present invention have been described by way of example, but are not intended to be exhaustive or limited to these embodiments. As will be apparent to those skilled in the art, many modifications and variations are possible without departing from the scope of the described embodiments. The terms used in this specification are selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology recognized in the market, or to allow those skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method for physically assisting in locating an input location on a touch screen, comprising: identifying a plurality of input locations on the touch screen; Detecting that a user has touched the touch screen with a finger or a pointing device at an initial touch point; predicting the input location from the plurality of input locations on the touch screen; determining a path from the initial touch point to a predicted input location; generating a ridge on the touchscreen proximate to the initial touch point; determining that a current finger or pointing device position is not on said path; moving the raised portion of the touch screen to guide the finger or the pointing device along the path; A method comprising:

2. and further comprising predicting a dexterity level of the user. The method of claim 1.

3. detecting a dexterity level of the user; determining a level of assistance for the user based on the detected dexterity level; The method of claim 1 further comprising:

4. The method described in claim 3, further comprising detecting the user's dexterity level using a machine learning model that predicts the user's attention based on the user's patterns while controlling a cursor.

5. Generating the raised portion of the touch screen comprises: The method of claim 1 , further comprising generating the ridge on a front side of the finger or pointing device between a touch point and the input location.

6. Generating the raised portion of the touch screen comprises: The method of claim 1 , further comprising generating the ridge on a back side of the finger or pointing device between a touch point and the input location.

7. Generating the raised portion of the touch screen comprises: The method of claim 1 , further comprising generating the ridges on two or more sides of the finger or pointing device.

8. 1. A computer system that provides physical support for identifying an input location on a touch screen, comprising: A processor and memory; a touchscreen display configured with microfluidic capabilities, wherein in response to instructions from the processor, fluid is selectively applied to portions of the touchscreen to raise the portions of the touchscreen; and and one or more computer-readable storage media having program instructions executable by said processor, said program instructions causing said processor to: identifying a plurality of input locations on the touch screen; Detecting that a user has touched the touch screen with a finger or a pointing device at an initial touch point; predicting the input location from the plurality of input locations on the touch screen; determining a path from the initial touch point to a predicted input location; generating a ridge on the touchscreen proximate to the initial touch point; determining that a current finger or pointing device position is not on said path; moving the raised portion of the touch screen to guide the finger or the pointing device along the path; A computer system that performs operations including:

9. and further comprising predicting a dexterity level of the user.

9. The computer system of claim 8.

10. detecting a dexterity level of the user; determining a level of assistance for the user based on the detected dexterity level; The computer system of claim 8 further comprising:

11. The computer system of claim 10, further comprising: detecting a user's dexterity level using a machine learning model that predicts a user's attention based on the user's patterns while controlling a cursor.

12. The method of claim 11, further comprising: The computer system of claim 8 , further comprising generating the ridge on a front side of the finger or pointing device between a touch point and the input location.

13. The method of claim 12, further comprising: The computer system of claim 8 , further comprising generating the ridge on a back side of the finger or pointing device between a touch point and the input location.

14. The method of claim 13, further comprising: The computer system of claim 8 , further comprising generating the ridges on two or more sides of the finger or pointing device.

15. 1. A computer program for physically assisting in locating an input location on a touch screen, comprising: instructions for identifying a plurality of input locations on the touch screen; instructions for detecting when a user touches the touch screen with a finger or a pointing device at an initial touch point; instructions for predicting the input location from the plurality of input locations on the touch screen; instructions for determining a path from the initial touch point to a predicted input location; instructions for generating a ridge on the touch screen proximate to the initial touch point; instructions for determining that a current finger or pointing device position is not on the path; instructions to move the ridges of the touch screen to guide the finger or the pointing device along the path; A computer program comprising:

Images