Changing the tactile sensitivity of interactions with the air interface

Abstract

Reflections of a subsonic signal reflected by a contact surface are captured. The contact surface contacts a simulated surface of an object projected from an air interface (MAI) device. The difference between the subsonic signal and the reflection is converted into a measurement of flow in the contact surface. When the measurement is within a measurement range, a change in temperature of a volume of a medium in which the simulated surface is projected is induced, wherein the change in temperature induces a second change in flow in the contact surface.

Description

Technical Field

[0001] The present invention relates to methods, systems, and computer program products for varying the tactile sensitivity of interactions with an air interface. Background Art

[0002] An air interface (MAI) is a simulation of a solid three-dimensional physical object in a medium (e.g., air) by projecting a shape in the medium. When touched, the simulated object feels like a three-dimensional physical object in some aspects. The projection in the medium is a holographic projection (hologram), in which the tactile properties are achieved by forming a projection in the medium using ultrasound. The hologram formed by projecting sound into the medium may or may not present a visual representation of the simulated object, but is able to provide a sense of touch, i.e., tactile feedback, when interacting with the simulated object.

[0003] Holograms can also be created using light or lasers projected into a medium. Light-based holograms can be better at visually representing simulated objects, but often lack the ability to provide tactile feedback when interacting with the simulated object.

[0004] For the purposes of the illustrative embodiments, a hologram or simulation capable of providing tactile feedback is the MAI used and described herein. In other words, the MAI contemplated within the scope of the exemplary embodiments is similar to a holographic projection formed using sound waves to control pressure in a medium or using a pressure-controlled medium column.

[0005] For example, a MAI simulated keyboard using an air medium presents the physical keys of a physical keyboard in the air, so that tactile interaction with the simulated keys induces the human user's brain (or a humanoid cognitive system) to cognitively perceive the physical keys in the air. The MAI simulated keys can be pressed and released in a manner similar to the physical keys of a physical keyboard by applying and releasing finger pressure to one or more protrusions that shape the air medium similar to the physical keys.

[0006] The simulated keys of this example differ from the physical keys of the example physical keyboard in that a finger cannot normally pierce and pass through a physical key, whereas a finger can continue to press the simulated key to eventually pass through the simulated key. Other simulated objects generally represent the physical shape, size, and feel of a corresponding physical object, but are unlike the corresponding physical object and are similarly pierceable due to the non-solid nature of the medium used to form the simulated object.

[0007] For clarity of description and without implying any limitation thereto, air is assumed as the medium when describing various operations and embodiments. Other mediums, such as water or different gases or fluids, may be used similarly in the manner of the described operations or embodiments without departing from the scope of the illustrative embodiments. Additionally, for clarity of description and without implying any limitation thereto, it is assumed that a human user interacts with the MAI. In light of this disclosure, one of ordinary skill in the art will be able to adapt the embodiments to operate in a similar manner with a humanoid machine or machine operated by a user as the MAI, and such adaptation is considered to be within the scope of the illustrative embodiments.

[0008] Currently, devices are available that project holograms in air to form simulated objects of MAI. For example, devices consisting of a grid of projection nozzles are currently available to form simulated objects above a device by adjusting an air jet projected from the device toward a volume of air above the device.

[0009] A volume of air that is shaped, formed, or pressure modulated using sound or other methods to form the entirety or a portion of a simulated object of a MAI is referred to herein as an "air form." An object or a portion thereof simulated using one or more air forms is referred to herein as a simulated object, an air simulated shape, or variations of these phrases unless clearly distinguished when used.

[0010] Aerial interaction with MAI is the manipulation of simulated objects in MAI using tactile manipulation. For example, simulated keys can be touched, pressed, and released; simulated balls can be held, turned, or squeezed; and simulated graphics can be touched, rubbed, or pushed by applying physical forces to the simulated objects. Aerial interaction induces cognitive perception of touch. When a person interacts with MAI, the interaction with MAI causes the tactile perception to be cognitively induced in the human brain. The human brain is a cognitive receptor. When a humanoid or machine interacts with MAI, the interaction with MAI causes the tactile perception to be cognitively induced in appropriate cognitive receptor units associated with the humanoid or machine. In the following, references to cognitive induction are references to tactile perceptions that result from physical interactions with simulated objects produced by MAI and that can be received and processed by appropriate receivers. Summary of the invention

[0011] A first aspect of the invention includes a method of capturing reflections of a subsonic signal that is reflected by a contact surface, wherein the contact surface contacts a simulated surface of an object projected from an air interface (MAI) device. The embodiment converts the difference between the subsonic signal and the reflection into a measurement of flow in the contact surface. The embodiment causes a change in temperature of a volume of a medium in response to the measurement being within a range of the measurement, wherein the simulated surface is projected into the volume of the medium, wherein the change in temperature causes a second change in flow in the contact surface.

[0012] Another aspect includes a computer usable program product. The computer usable program product includes a computer readable storage device and program instructions stored on the storage device and executable to perform the method of the first aspect.

[0013] Another aspect includes a computer system. The computer system includes a processor, a computer readable memory and a computer readable storage device, and program instructions stored on the storage device, which are used to be executed by the processor via the memory to perform the method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Certain novel features which are believed to be characteristic of the invention are set forth in the appended claims. However, the invention itself, as well as its preferred mode of use, further objects and advantages, will be best understood by reference to the following detailed description of illustrative embodiments taken in conjunction with the accompanying drawings, in which:

[0015] Figure 1 depicts a block diagram of a network of data processing systems in which the illustrative embodiments may be implemented;

[0016] Figure 2 depicts a block diagram of a data processing system in which the illustrative embodiments may be implemented;

[0017] Figure 3 depicts block diagrams of several example MAIs configured and operated in accordance with illustrative embodiments;

[0018] Figure 4 depicts an example configuration for varying tactile sensitivity of interactions with an air interface in accordance with an illustrative embodiment;

[0019] Figure 5 depicts a resolution change according to an illustrative embodiment;

[0020] Figure 6 depicts a thermal profile according to an illustrative embodiment;

[0021] Figure 7 A flowchart depicting an example process for changing the tactile sensitivity of an interaction with an air interface in accordance with an illustrative embodiment; and

[0022] Figure 8 Depicted is a flowchart of another example process for changing the tactile sensitivity of interactions with an air interface in accordance with an illustrative embodiment. DETAILED DESCRIPTION

[0023] Human-computer interfaces are a recognized and endeavored field of technology. Computers and other machines require input from a user and provide output to the user through such a human-computer interface. Computer keyboards, computer mice or other pointing devices, digital styluses or similar writing implements, touch screens, etc. are common examples of human-computer interfaces for providing input. Some interfaces, such as touch screens or pointing devices, can provide tactile feedback through vibration, changes in texture, changes in required force, changes in pressure, etc.

[0024] Projecting an air interface as a replacement for a human-machine interface and detecting interactions with the air interface is also a recognized area of ​​endeavor. The current state of the art in this area of ​​endeavor has certain shortcomings and limitations. The operation of the illustrative embodiments imparts additional or new capabilities to improve upon the prior art in the area of ​​endeavor of human-machine interfaces, particularly in the area of ​​air interfaces.

[0025] The illustrative embodiments recognize that there are certain disadvantages to air interfaces. As an example, human users of MAI have found that fingers, arms, and other extremities of the human body that are engaged when interacting with the MAI tend to become numb or their perception abilities gradually decrease over the period of interaction. For example, it has been observed that an arm suspended in the air above an MAI device will begin to become numb and decrease during the period of suspension. As a result, the reduced sensitivity to the sense of touch at the fingertips is also reduced.

[0026] It has also been found that repeated touching, pressing, rubbing or other tactile manipulation of a surface also tends to cause a gradual reduction in the tactile sensitivity of the limb being used. For example, if a user repeatedly touches or rubs a textured surface, the user's perception of the texture is known to degrade after a period of such tactile activity. This observation is true whether the surface is a real physical surface or a simulated surface of the MAI.

[0027] The surface of the MAI that the user physically contacts to perform haptic operations is referred to herein as a “contact surface” unless clearly distinguished. The surface of the simulated object touched by the contact surface is referred to herein as an “object surface” unless clearly distinguished.

[0028] These and other degradations of tactile perception at the contact surface are disadvantages of using currently available MAIs. Currently, the state of the art in the art of efforts through MAI human-machine interfaces does not address or mitigate degradation of tactile perception. There is a need to detect degradation of tactile perception when using MAIs. There is a need to prevent or mitigate such degradation. If tactile perception sensitivity degrades, there is a need to restore tactile perception sensitivity.

[0029] The illustrative embodiments recognize that currently available tools or solutions do not address these needs / problems or provide adequate solutions for these needs / problems.The illustrative embodiments used to describe the present invention generally solve the above-mentioned problems and other related problems by changing the tactile sensitivity of interaction with an air interface.

[0030] A device capable of projecting a MAI consisting of simulated objects formed using air forms is contemplated. Such a device is referred to herein as a MAI device. Embodiments may be used to modify prior art MAI devices to form improved MAI devices. To form the improved MAI device, embodiments may be implemented as a combination of certain hardware components (e.g., prior art MAI devices) and software applications. Implementations of embodiments or one or more components thereof may be configured as modifications of existing MAI devices, with accompanying software applications executing in some combination of (i) the MAI device itself, (ii) a data processing system that communicates with the MAI device via a short-range radio or a local area network (LAN), and (iii) a data processing system that communicates with the MAI device via a wide area network (WAN).

[0031] In prior art MAI devices, projection nozzles are arranged in a two-dimensional array. The projection nozzles project air forms into a volume of air, and the projection forms a simulated object. The air form can be considered as a single pixel or a group of pixels (or its equivalent) that renders a portion of the object surface of the simulated object. One or more simulated objects together form a MAI.

[0032] To form an embodiment of an improved MAI device, one embodiment adds additional components to the array of projection nozzles. In one embodiment, for each projection nozzle, the embodiment configures a subsonic transducer (SST) in or near the projection nozzle.

[0033] The SST is configured to transmit an acoustic output (a subsonic signal) where the acoustic frequency is below the human audible frequency range (hence "subsonic"). The SST is also configured to receive as input the reflection of the transmitted subsonic signal. The SST may be an additional subsonic component or a modification of an existing acoustic component in order to operate in the subsonic frequency range in the manner described herein.

[0034] In addition, the SST is configured such that the SST projects a subsonic signal into the air form or substantially at the same pixel or pixel group represented by the air form. As a result of this configuration, if a contact surface, such as a fingertip of a human user, manipulates a portion of the object surface formed using the air form, the emitted subsonic signal bounces off the contact surface and is received by the SST.

[0035] In one embodiment, the transmitter of the subsonic signal and the receiver of the reflection can be different devices located at different physical locations relative to the corresponding projection nozzle on the improved MAI device. In another embodiment, the transmitter of the subsonic signal and the receiver of the reflection are co-located at substantially the same physical location relative to the corresponding projection nozzle on the improved MAI device.

[0036] Subsonic sound is currently used to measure liquid flow. Measurement techniques and devices currently exist to calculate the difference between a transmitted subsonic signal and the reflection of the signal from a surface and convert the difference into a flow measurement. In other words, the difference between the transmitted and reflected subsonic signals corresponds to the liquid flow at or near the surface from which the signal was reflected. In one example case, the difference may be the difference in the transmitted and received frequencies. In another example case, the difference may be the difference in the signal strength of the transmitted and received signals. In another example case, the difference may be the difference in the phase of the transmitted signal and the phase of the received signal.

[0037] Using this technique, one embodiment measures the amount of blood flowing in capillaries located at or near the contact surface.An embodiment determines from the flow measurement whether the flow volume is normal, low, or high relative to an acceptable flow range for a given user and a particular contact surface.

[0038] When the flow measurement is within a range defined by and including the two thresholds, the embodiment considers the flow to be normal. When the flow measurement is below the lower of the two thresholds, the embodiment considers the flow to be low. When the flow measurement is above the higher of the two thresholds, the embodiment considers the flow to be high.

[0039] The embodiment also considers low flow as an indication of degraded tactile perception or reduced sensitivity to tactile input at the contact surface, such as caused by numbness of a limb. The embodiment also considers high flow as an indication of degraded tactile perception, such as caused by inflammation of a limb from repeated tactile manipulation of the contact surface.

[0040] One embodiment adds a thermal element in conjunction with the projection nozzle as part of forming an improved MAI device. In one embodiment, the thermal element is associated with a single projection nozzle and is juxtaposed with the nozzle at substantially the same location as the nozzle. In another embodiment, the thermal element is associated with multiple projection nozzles and is located in close proximity (substantially the same area on the MAI device) to the projection nozzle group. In another embodiment, the thermal element is located at a different location than the projection nozzle, such as at the periphery of the improved MAI device.

[0041] In one embodiment, the thermal element is configured to increase or decrease the temperature of the air into which the nozzle ejects the air form. In another embodiment, the thermal element is configured to only increase the temperature of the air into which the nozzle ejects the air form.

[0042] When the embodiment detects low blood flow through the subsonic measurement described herein, the embodiment activates thermal elements to increase the temperature of the air. In particular, the embodiment activates those thermal elements that are configured to heat a volume of air where one or more selected projection nozzles are projecting air forms so that those air forms are forming an object surface that comes into contact with the contact surface. This activation is selective and may not activate thermal elements of other projection nozzles that are projecting other parts of the simulated object, where those parts are not manipulated by the contact surface.

[0043] The increase in air temperature at or near the surface of the object causes the contact surface temperature to also increase during tactile manipulation of the surface of the object. The increase in temperature of the contact surface causes increased blood flow to the contact surface and results in reduced numbness at the contact surface, at a limb associated with the contact surface, or both. The reduced numbness improves tactile sensitivity of the contact surface.

[0044] When an embodiment detects high blood flow by the subsonic measurement described herein, and when the thermal element is configured to also cool the air, the embodiment activates the thermal element to reduce the temperature of the air. In particular, the embodiment activates those thermal elements that are configured to cool a volume of air where one or more selected projection nozzles are projecting air forms such that those air forms are forming the surface of the object that is in contact with the contact surface. This activation is selective and may not activate thermal elements of other projection nozzles that are projecting other portions of the simulated object, where those portions are not manipulated by the contact surface.

[0045] The reduction in air temperature at the surface of the object causes the contact surface temperature to also decrease during tactile manipulation. The reduction in temperature of the contact surface causes a reduction in blood flow at the contact surface. The reduced blood flow results in a reduction in inflammation at the contact surface, at a limb associated with the contact surface, or at both. The reduced inflammation improves and / or restores the tactile sensitivity level of the contact surface.

[0046] Another embodiment configures the projection nozzle to increase or decrease the resolution of the air form. The illustrative embodiments recognize that when a large area of ​​the contact surface is used for tactile interaction with a simulated object, the nerve endings in the large area are all activated to sense the touch sensation. Therefore, when the contact surface is numb or inflamed, all nerve endings in the entire area are adversely affected, i.e., they all experience reduced tactile sensitivity.

[0047] When an embodiment detects an indicator of reduced tactile sensitivity in the contact surface, the embodiment causes the projection nozzle to reshape the air form so that the air form projects a smaller object surface area than the size of the object surface area projected when the reduced tactile sensitivity was detected. The smaller area results in a smaller set of nerve endings participating in the sense of touch at a given time, allowing other unparticipated nerve endings time to recover.

[0048] Changing the resolution by reshaping the air form can be accomplished in a variety of ways. For example, one embodiment contracts or enlarges the nozzle opening to change the size of the air form - and therefore changes the resolution of features projected by the air form. Another embodiment obstructs or unobstructs the air form without changing the nozzle characteristics. By blocking or removing the obstruction from the path of the air form, the size of the airflow changes, resulting in a change in resolution. Another embodiment raises or lowers the projection nozzle relative to the contact surface to change the distance between the nozzle opening and the contact surface. Increasing the path of the airflow results in a decrease in feature resolution (becoming coarser in resolution or granularity). Conversely, decreasing the path of the airflow results in an increase in feature resolution (becoming finer in resolution or granularity). Therefore, a change in the path distance of the airflow results in a change in resolution.

[0049] When a feature, such as the texture or contour of an object's surface, is presented on a relatively large surface, it may be difficult for a user to perceive the feature. For example, if the feature is projected in an area of ​​1 square inch, a height difference of 0.1 inches in the feature may not be perceived with degraded tactile sensitivity. However, if the feature is projected in an area of ​​0.1 square inches, the same height difference of 0.1 inches in the feature may be perceived even with degraded tactile sensitivity.

[0050] Thus, a change in resolution, i.e., an embodiment that causes a projected feature to appear in a relatively smaller area compared to a previous projection of the same feature, increases the tactile perception of the feature. Additionally, a higher resolution, i.e., an embodiment that causes a projected feature to appear in a relatively smaller area compared to a lower resolution projection, also allows for more detailed features to be accommodated in the same area. The size of the area and the resolution of the feature are described relative to each other, and not relative to any absolute size or standard.

[0051] The illustrative embodiments recognize that a numb arm or leg may lose sensation when left in the same position for a period of time and may regain its tactile sensitivity when moved to a different position.

[0052] Another embodiment causes the user to move a limb associated with the contact surface to alter blood flow (and resulting numbness) at the contact surface. This embodiment induces movement by creating a temperature profile on a simulated object.

[0053] For example, assume that the surface of the simulated object includes regions A, B, and C, where region A is adjacent to region B, and region B is adjacent to region C, and further assume that the user has rested the contact surface at region A, or repeatedly positioned the contact surface, which forms the current object surface. Further assume that moving the contact surface from region A to region B or C has no adverse effect on the underlying command or operation performed by the tactile manipulation.

[0054] The embodiment determines that the user should be prompted to reposition the contact surface on the simulated object from region A to region C. The embodiment also determines, for example by machine learning from the user's previous MAI interactions or from other methods, that the user exhibits a dislike for hot surfaces (i.e., surfaces having a temperature greater than and including an upper limit) and a like for cold surfaces (i.e., surfaces having a temperature less than a different lower limit).

[0055] Thus, this embodiment heats a volume of air in the air form from nozzle protrusion region A to a temperature greater than an upper threshold. This embodiment causes the air flow from nozzle protrusion region B to be at a temperature between an upper threshold and a lower threshold. This embodiment causes the air flow from nozzle protrusion region C to be at a temperature below the lower threshold. By adjusting the temperature of air form forming regions A, B, and C, this embodiment constructs a thermal profile or temperature profile with a gradually decreasing temperature from region A to region C.

[0056] The aerial profile of the projected surface of the simulated object remains unchanged, but the thermal profile of the surface changes in this way. Thus, the user who prefers cooler temperatures is prompted to reposition the contact surface from area A to area B and from area B to area C. This repositioning causes movement of the limb. The movement of the limb helps to restore tactile sensitivity of the limb and / or the contact surface.

[0057] The illustrative embodiments also recognize that tonic contractions or tonic spasms can cause a limb, such as a finger, to perform rapid, repetitive movements. The illustrative embodiments recognize that tonic movements can cause a user to repeatedly touch a simulated surface.

[0058] One embodiment detects repeated contact of a user's limb with a simulated surface. The embodiment classifies the rapidity of the user's contact with the simulated surface according to a predetermined set of tonic classifications. Some tonic classifications - that is, some speeds of rapid repeated contact - are acceptable, and some are not. If the embodiment determines that the rapid repeated contact falls into an unacceptable tonic classification, the embodiment causes movement in the limb. The embodiment uses a temperature profile method or a resolution change method to cause movement, as described herein.

[0059] In currently available methods in the field of endeavor related to human-machine interfaces, and in particular in the field of MAI, the manner described herein of changing the tactile sensitivity of interaction with an air interface is not available. The methods of the embodiments described herein, when implemented as being executed on a device or data processing system, comprise a substantial advancement in the functionality of the device or data processing system in terms of improving tactile sensitivity and feature projection in MAI.

[0060] Merely by way of example, the illustrative embodiments are described with respect to specific types of MAI devices, media, volumes, media forms, users, simulated objects, contact surfaces, fluid flows, temperatures, feature projections, repeatability, tactile sensations, thermal profiles, object surfaces, algorithms, equations, configurations, locations of embodiments, additional data, devices, data processing systems, environments, components, and applications. Any particular representation of these and other similar artifacts is not intended to limit the invention. Any suitable representation of these and other similar artifacts may be selected within the scope of the exemplary embodiments.

[0061] Furthermore, the illustrative embodiments may be implemented for any type of data, data source, or access to a data source over a data network. Any type of data storage device may provide data to embodiments of the present invention locally at a data processing system or over a data network within the scope of the present invention. Where embodiments are described using a mobile device, any type of data storage device suitable for use with the mobile device may provide data to such embodiments locally at the mobile device or over a data network within the scope of the illustrative embodiments.

[0062] Illustrative embodiments are described using specific codes, designs, architectures, protocols, layouts, schematic diagrams, and tools, which are only examples, rather than limitations on the illustrative embodiments. In addition, for clarity of description, specific software, tools, and data processing environments are used in some instances to describe the illustrative embodiments only as examples. The illustrative embodiments can be used in conjunction with other comparable or similar purpose structures, systems, applications, or architectures. For example, within the scope of the present invention, other comparable mobile devices, structures, systems, applications, or their architectures can be used in conjunction with such embodiments of the present invention. The illustrative embodiments can be implemented with hardware, software, or a combination thereof.

[0063] The examples in this disclosure are only for clear description and are not limited to the illustrative embodiments. Additional data, operations, actions, tasks, activities and manipulations can be imagined from this disclosure and can be envisioned within the scope of the illustrative embodiments.

[0064] Any advantages listed herein are merely examples and are not intended to limit the illustrative embodiments. Additional or different advantages may be achieved through specific illustrative embodiments. In addition, specific illustrative embodiments may have some, all, or none of the advantages listed above.

[0065] With reference to the accompanying drawings, and with particular reference to Figure 1 and 2 , these figures are exemplary diagrams of data processing environments in which illustrative embodiments may be implemented. Figure 1 and 2 This is merely an example and is not intended to assert or imply any limitation with regard to the environments in which different embodiments may be implemented. Specific implementations may make many modifications to the depicted environment based on the following description.

[0066] Figure 1 A block diagram of a network of data processing systems in which the illustrative embodiments may be implemented is depicted. Data processing environment 100 is a network of computers in which the illustrative embodiments may be implemented. Data processing environment 100 includes network 102. Network 102 is a medium for providing communication links between various devices and computers connected together within data processing environment 100. Network 102 may include connections such as wired, wireless communication links, or fiber optic cables.

[0067] Client or server are merely example roles for certain data processing systems connected to network 102 and are not intended to exclude other configurations or roles for these data processing systems. Server 104 and server 106 are coupled to network 102 along with storage unit 108. Software applications may be executed on any computer in data processing environment 100. Clients 110, 112, and 114 are also coupled to network 102. Data processing systems such as servers 104 or 106 or clients 110, 112, or 114 may contain data and may have software applications or software tools executed thereon.

[0068] This is for example purposes only and does not imply any limitation on such architecture. Figure 1Certain components that can be used in example implementations of the embodiments are depicted. For example, servers 104 and 106 and clients 110, 112, 114 are depicted as servers, and clients are only used as examples and do not imply limitations on client-server architectures. As another example, one embodiment can be distributed across several data processing systems and data networks as shown, while another embodiment can be implemented on a single data processing system within the scope of the illustrative embodiments. Data processing systems 104, 106, 110, 112, and 114 also represent example nodes in clusters, partitions, and other configurations suitable for implementing the embodiments.

[0069] Device 132 is an example of a device described herein. For example, device 132 may take the form of a smartphone, a tablet computer, a laptop computer, a fixed or portable form of client 110, a wearable computing device, or any other suitable device. Figure 1 Any software application executing in another data processing system in may be configured to execute in device 132 in a similar manner. Figure 1 Any data or information stored or generated in another data processing system in may be configured to be stored or generated in device 132 in a similar manner.

[0070] Application 115 implements the embodiments described herein. MAI device 116 is a modified MAI device in which hardware modifications are implemented according to an embodiment, and which uses application 115 to operate software-implemented aspects of the embodiments as described herein.

[0071] Servers 104 and 106, storage unit 108, clients 110, 112, and 114, and device 132 may be coupled to network 102 using wired connections, wireless communication protocols, or other suitable data connections. Clients 110, 112, and 114 may be, for example, personal computers or network computers.

[0072] In the depicted example, server 104 may provide data, such as boot files, operating system images, and applications to clients 110, 112, and 114. In this example, clients 110, 112, and 114 may be clients to server 104. Clients 110, 112, 114, or some combination thereof, may include their own data, boot files, operating system images, and applications. Data processing environment 100 may include additional servers, clients, and other devices not shown.

[0073] In the described example, data processing environment 100 can be the Internet. Network 102 can represent a collection of networks and gateways that use transmission control protocol / Internet protocol (TCP / IP) and other protocols to communicate with each other. At the core of the Internet is the backbone of data communication links between master nodes or host computers, including thousands of commercial, government, educational and other computer systems that route data and messages. Of course, data processing environment 100 can also be implemented as many different types of networks, such as intranets, local area networks (LANs) or wide area networks (WANs). Figure 1 It is intended as an example and not as an architectural limitation to the different illustrative embodiments.

[0074] Among other uses, data processing environment 100 can be used to implement a client-server environment in which exemplary embodiments can be implemented. A client-server environment enables software applications and data to be distributed on a network so that the application works by using the interactivity between the client data processing system and the server data processing system. Data processing environment 100 can also adopt a service-oriented architecture in which interoperable software components distributed on a network can be packaged together as a consistent business application. Data processing environment 100 can also take the form of a cloud, and adopt a cloud computing model of service delivery to implement convenient on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services), which can be quickly supplied and released with minimal management effort or interaction with the provider of the service.

[0075] refer to Figure 2 , which depicts a block diagram of a data processing system in which the illustrative embodiments may be implemented. Data processing system 200 is an example of a computer, such as Figure 1 Servers 104 and 106, or clients 110, 112, and 114, or another type of device, in which computer usable program code or instructions implementing the processes may be provided for exemplary embodiments.

[0076] Data processing system 200 also represents a data processing system or configuration thereof, such as Figure 1 Data processing system 132 in which computer usable program code or instructions implementing the processes of the exemplary embodiments may be placed. Data processing system 200 is described as a computer only as an example and is not limited thereto. Without departing from the general description of the operation and functionality of data processing system 200 described herein, the computer may be represented by a computer program such as a computer program. Figure 1 Implementation in the form of other devices of device 132 may modify data processing system 200 , such as by adding a touch interface, and even removing certain depicted components from data processing system 200 .

[0077] In the depicted example, data processing system 200 employs a hub architecture including north bridge and memory controller hub (NB / MCH) 202 and south bridge and input / output (I / O) controller hub (SB / ICH) 204. Processing unit 206, main memory 208, and graphics processor 210 are coupled to north bridge and memory controller hub (NB / MCH) 202. Processing unit 206 may include one or more processors and may be implemented using one or more heterogeneous processor systems. Processing unit 206 may be a multi-core processor. In some implementations, graphics processor 210 may be coupled to NB / MCH 202 via an accelerated graphics port (AGP).

[0078] In the depicted example, a local area network (LAN) adapter 212 is coupled to the south bridge and I / O controller hub (SB / ICH) 204. An audio adapter 216, a keyboard and mouse adapter 220, a modem 222, a read-only memory (ROM) 224, a universal serial bus (USB) and other ports 232, and a PCI / PCIe device 234 are coupled to the south bridge and I / O controller hub 204 via bus 238. A hard disk drive (HDD) or solid state drive (SSD) 226 and a CD-ROM 230 are coupled to the south bridge and I / O controller hub 204 via bus 240. The PCI / PCIe devices 234 may include, for example, Ethernet adapters, add-in cards, and PC cards for notebook computers. PCI uses a card bus controller, while PCIe does not. ROM 224 may be, for example, a flash binary input / output system (BIOS). Hard disk drive 226 and CD-ROM 230 may use, for example, an integrated drive electronics (IDE), a serial advanced technology attachment (SATA) interface, or variations such as external SATA (eSATA) and micro SATA (mSATA). Super I / O (SIO) device 236 may be coupled to south bridge and I / O controller hub (SB / ICH) 204 via bus 238.

[0079] Memories, such as main memory 208, ROM 224, or flash memory (not shown), are some examples of computer usable storage devices. Hard disk drives or solid state drives 226, CD-ROM 230, and other similarly available devices are some examples of computer usable storage devices that include computer usable storage media.

[0080] An operating system runs on processing unit 206. The operating system coordinates and provides Figure 2The operating system may be a commercially available operating system for any type of computing platform, including but not limited to server systems, personal computers, and mobile devices. An object-oriented or other type of programming system may operate with the operating system and provide calls to the operating system from programs or applications executing on data processing system 200.

[0081] for operating systems, object-oriented programming systems, and such Figure 1 The instructions for the applications or programs in application 115 are located on a storage device, such as in the form of code 226A on hard drive 226, and may be loaded into at least one of one or more memories, such as main memory 208, for execution by processing unit 206. The processes of the illustrative embodiments may be performed by processing unit 206 using computer-implemented instructions, which may be located in a memory, such as main memory 208, read-only memory 224, or in one or more peripheral devices.

[0082] In addition, in one case, code 226A can be downloaded from remote system 201B via network 201A, where similar code 201C is stored on storage device 201D. In another case, code 226A can be downloaded to remote system 201B via network 201A, where downloaded code 201C is stored on storage device 201D.

[0083] Figure 1-2 The hardware in can vary depending on the implementation. Figure 1-2 In addition to or in place of the hardware described in , other internal hardware or peripheral devices such as flash memory, equivalent non-volatile memory, or optical drives may be used.

[0084] In some illustrative examples, data processing system 200 may be a personal digital assistant (PDA), which is typically configured with flash memory to provide non-volatile memory for storing operating system files and / or user-generated data. The bus system may include one or more buses, such as a system bus, an I / O bus, and a PCI bus. Of course, the bus system may be implemented using any type of communication structure or architecture that provides data transfer between different components or devices attached to the structure or architecture.

[0085] The communication unit may include one or more devices for sending and receiving data, such as a modem or a network adapter. A memory may be, for example, main memory 208 or a cache, such as that found in north bridge and memory controller hub 202. A processing unit may include one or more processors or CPUs.

[0086] Figure 1-2 The examples described in and above are not meant to imply architectural limitations.For example, data processing system 200 may be a tablet computer, laptop computer, or telephone device in addition to taking the form of a mobile or wearable device.

[0087] Where a computer or data processing system is depicted as a virtual machine, virtual appliance, or virtual component, the virtual machine, virtual appliance, or virtual component uses virtualized representations of some or all of the components depicted in data processing system 200 to operate in the manner of data processing system 200. For example, in the virtual machine, virtual appliance, or virtual component, processing unit 206 is represented as a virtualized instance of all or some number of hardware processing units 206 available in the host data processing system, main memory 208 is represented as a virtualized instance of all or some portion of main memory 208 available in the host data processing system, and disk 226 is represented as a virtualized instance of all or some portion of disk 226 available in the host data processing system. In this case, the host data processing system is represented by data processing system 200.

[0088] refer to Figure 3 , which depicts a block diagram of several example MAIs configured and operated in accordance with an illustrative embodiment. The MAI device 302 is an example of a MAI device 116 and includes an array of MAI elements 303. Each MAI member 303 includes a projection nozzle 303A as shown. The main device 302 also includes one or more SSTs (not visible), one or more thermal elements (not visible), or a combination thereof, in combination with a projection nozzle 303A. In one embodiment, the SST, thermal element, or a combination thereof is associated with a single MAI element 303. In another embodiment, the SST, thermal element, or a combination thereof is associated with multiple MAI elements 303.

[0089] The application 115 provides computational features to operate the MAI device 302. For example, the application 115 computes the flow rate of a fluid based on a subsonic measurement, instructs an SST to transmit or receive a subsonic signal, instructs a thermal element to activate, causes operations at the MAI device 302 to change the resolution of an air pattern, computes the location of areas participating in a thermal profile, determines a user's thermal likes and dislikes, computes and determines the need to cause movement along a thermal profile, and other computational operations as described herein.

[0090] These examples of computing operations implemented by software in whole or in part of the embodiments are not intended to be limiting. Based on this disclosure, one of ordinary skill in the art will be able to determine many other computing operations that can be implemented in software to provide the features of the embodiments described herein, and these computing operations are contemplated within the scope of the illustrative embodiments.

[0091] As an example, a subset of the projection nozzles in the MAI device 302 can be used to project simulated keyboard keys 304. The keys in the simulated keys 304 can be touched, tapped, or used for typing in the same manner as physical keys on a physical keyboard.

[0092] As another example, a subset of the projection nozzles in the MAI device 302 may be used to project a simulated ball 306. The ball 306 may be touched, caught, or thrown, or forces may be applied to the ball 306 in the same manner as a physical ball.

[0093] As an example, a subset of the projection nozzles in the MAI device 302 can be used to project simulated piano keys 308. The keys in the simulated piano keys 308 can be touched, pressed, and released in the same manner as physical keys on a physical piano.

[0094] refer to Figure 4 , which depicts an example configuration for varying the tactile sensitivity of an interaction with an air interface, according to an illustrative embodiment. The primary element 402 is Figure 3 An example of an MAI element 303 in FIG. 4, wherein the projection nozzle 404 is Figure 3 An example of a projection nozzle 303A in FIG.

[0095] Projection nozzle 404 projects air from 418 to render object surface 405. SST 406 transmits subsonic signal 408 and receives reflected subsonic signal 410. Signal 408 is directed toward object surface 405. Reflected signal 410 is a reflection of signal 408 from a contact surface that manipulates object surface 405. For example, contact surface may be a portion of finger 412 that is in tactile contact with object surface 405.

[0096] According to one embodiment, the SST 406 is juxtaposed with the projection nozzle 404 in the MAI element 402. According to another embodiment, the SST 406 is located outside the MAI element 402. The SST 406 is shown as being adjacent to the projection nozzle 404 within the MAI element 402, which is only a non-limiting example. In another embodiment, the SST 406 can be configured within the projection nozzle 404. In yet another embodiment, the SST 406 and the projection nozzle 404 can be one and the same, wherein a prior art projection nozzle, such as an ultrasonic projection device, has been reconfigured to also transmit and receive subsonic signals.

[0097] Thermal element 414 is associated with MAI element 402. According to one embodiment, thermal element 416 is juxtaposed with projection nozzle 404 in MAI element 402. Thermal element 414 is depicted within projection nozzle 404 as a non-limiting example only. In another embodiment, thermal element 414 may be configured adjacent to projection nozzle 404. In yet another embodiment, thermal element 414 may be located external to MAI element 402 but associated with air form 418 projected from projection nozzle 404.

[0098] refer to Figure 5 , which depicts a change in resolution according to an illustrative embodiment. Surface 502 is a portion of a surface of a simulated object. As a non-limiting example, finger 504 contacts surface 502. Area 506 is the area where finger 504 contacts surface 502. Area 506 of finger 504 forms a contact surface, while area 506 on surface 502 forms an object surface, such as Figure 4 The object surface 405 in.

[0099] The number of nerve endings or other tactile sensors in contact surface area 506 on finger 504 is proportional to the size of area 506. As described herein, embodiments cause the resolution of area 504 in surface 502 to be increased. In other words, where finger 504 previously contacted an area the size of area 506, embodiments cause surface features of area 506 to appear in relatively smaller area 508. In one embodiment, all features occupying area 506 are presented in area 508. In another embodiment, all features occupying area 506 are divided into feature portions, and the different portions are caused to appear in one or more smaller areas with increased or exaggerated detail, such as using area 508 and one or more other areas 510.

[0100] refer to Figure 6 , which depicts a thermal profile according to an illustrative embodiment. Surface 602 is Figure 5 , assume that finger 604 is located at position 1 (as indicated by circle 1). At position 1, finger 604 contacts area 606 on surface 602.

[0101] The embodiment identifies regions 608 and 610 to form thermal profile 612. Assuming that the user does not like hot surfaces and prefers cool surfaces, the embodiment, such as in application 115, changes the temperature of region 606 to a first temperature, where the first temperature is equal to or greater than an upper threshold temperature. Similarly, the embodiment changes the temperature of region 608 to a second temperature, where the second temperature is between the upper threshold temperature and the lower threshold temperature. The embodiment changes the temperature of region 610 to a third temperature, where the third temperature is equal to or lower than the lower threshold temperature.

[0102] Thus, this embodiment constructs a temperature profile 612 on the surface 602. The temperature profile 612 causes the movement of the finger 604 from position 1 to position 2 (as indicated by circle 2) and from position 2 to position 3 (as indicated by circle 3). The movement of the finger 604 is guided by the temperature profile from a relatively hot region 606 through an intermediate temperature region 608 and ultimately to a preferred cool temperature region 610.

[0103] The example temperature profile including three gradually changing temperature regions is not limiting. Based on the present disclosure, one of ordinary skill in the art will be able to modify the embodiments to form not only temperature profiles, but also other types of motion guiding profiles with greater or lesser numbers of regions of different temperatures, different shapes, different textures, different tactile characteristics, different sound characteristics, etc., and such modifications are contemplated within the scope of the illustrative embodiments.

[0104] refer to Figure 7 , which depicts a flow chart of an example process for changing the tactile sensitivity of an interaction with an air interface in accordance with an illustrative embodiment. Process 700 may be performed at Figure 1 The method is implemented in the application 115 .

[0105] The present application uses a modified MAI device to present the MAI (block 702). The application detects tactile interaction with the MAI (block 704). The application sends a subsonic signal to a contact surface (block 706). The application receives a reflected subsonic signal from the contact surface (block 708).

[0106] Using the difference between the transmitted and received subsonic signals, the application calculates a flow metric (block 710). The flow metric is indicative of blood flow at the contact surface.

[0107] If the flow metric indicates high flow ("high" path of block 712), the application reduces the temperature of the simulated surface at the contact point or contact area (block 714). Thereafter, the application ends process 700.

[0108] If the flow metric indicates low flow ("low" path of block 712), the application increases the temperature of the simulated surface at the contact point or contact area (block 716). Thereafter, the application ends process 700.

[0109] If the flow metric indicates normal flow ("Normal" path of block 712), the application leaves the temperature of the simulated surface at the contact point or area unchanged. Thereafter, the application ends process 700.

[0110] refer to Figure 8 , which depicts a flow chart of another example process for changing the tactile sensitivity of an interaction with an air interface in accordance with an illustrative embodiment. Process 800 may be performed at Figure 1The application 115 is implemented in the above embodiment.

[0111] The application presents the MAI (block 802). The application detects interactions occurring with the MAI (block 804). The application performs a tonic classification of the interactions, such as by classifying the interactions according to rapidity of touch, rapidity of change in pressure of the touch, repetitive speed of the touch, repetitiveness of motion in the touch, or some combination thereof (block 806).

[0112] If the ankylosis classification is acceptable ("no" path of block 808), the application does not induce movement in the user's limb, and thereafter ends process 800. If the ankylosis classification is unacceptable ("yes" path of block 808), the application induces movement of the user's limb by forming a temperature profile on the simulated surface (block 810), changing the resolution of the object surface (block 812), or both. Thereafter, the application ends process 800.

[0113] Thus, in illustrative embodiments, computer-implemented methods, systems or apparatus, and computer program products for varying tactile sensitivity and other related features, functions or operations of interactions with an air interface are provided. Where embodiments or portions thereof are described with respect to one type of device, the computer-implemented methods, systems or apparatus, computer program products, or portions thereof, are adapted or configured for use with appropriate and comparable representations of that type of device.

[0114] Where embodiments are described as being implemented in an application, delivery of the application in a software as a service (SaaS) model may be envisioned within the scope of the illustrative embodiments. In the SaaS model, the ability to implement the application of the embodiments is provided to the user by executing the application in a cloud infrastructure. The user may access the application using a variety of client devices through a thin client interface such as a web browser (e.g., web-based email) or other lightweight client applications. The user does not manage or control the underlying cloud infrastructure, including the network, servers, operating system, or storage of the cloud infrastructure. In some cases, the user may not even manage or control the capabilities of the SaaS application. In some other cases, a SaaS implementation of the application may allow limited possible exceptions to user-specific application configuration settings.

[0115] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, and the computer-readable program instructions are used to cause a processor to perform various aspects of the present invention.

[0116] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, computer-readable storage media, including but not limited to readable storage devices, are not to be interpreted as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber-optic cables), or electrical signals transmitted through wires.

[0117] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0118] The computer program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0119] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0120] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0121] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0122] Flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each frame in the flow chart or block diagram can represent the module, segment or part of instruction, which includes one or more executable instructions for realizing the specified logical function.In some alternative embodiments, the function noted in the frame may not occur in the order noted in the figure.For example, two frames shown continuously can actually be performed substantially simultaneously, or these frames can sometimes be performed in reverse order, depending on the function involved.It will also be noted that the combination of the frames in each frame of the block diagram and / or flow chart illustration and the block diagram and / or flow chart illustration can be realized by a dedicated hardware-based system that performs a specified function or action or performs a combination of special hardware and computer instructions.

Claims

1. A computer-implemented method comprising: capturing reflections of a subsonic signal reflected by a contact surface, wherein the contact surface contacts a simulated surface of an object projected from an air interface (MAI) device; converting a difference between the subsonic signal and the reflection into a measure of liquid flow in the contact surface; as well as In response to the measurement value being within a predetermined range of measurement values, causing a change in temperature of a volume of a medium in which the simulated surface is projected, wherein the change in temperature causes a second change in the liquid flow in the contact surface, The liquid flow is blood flow in the human body.

2. The method of claim 1, further comprising: detecting a reduction in tactile sensitivity of the contact surface from the measurement; determining a first region and a location of the first region on the simulated surface at which the contact surface contacts the simulated surface and wherein a simulated feature is affected by the contact; By simulating the simulated feature in a second region, wherein the second region occupies the location, the resolution of the simulated feature is increased to a first resolution that compensates for the reduction in tactile sensitivity.

3. The method of claim 1, further comprising: detecting repetition of said contact; changing a temperature of a first area on the simulation surface where the contact surface contacts the simulation surface to a first temperature, the first temperature not satisfying a threshold temperature; changing the temperature of a second region on the simulation surface to a second temperature, the second temperature satisfying the threshold temperature, the first region at the first temperature and the second region at the second temperature forming a temperature profile on the simulation surface; Using the temperature profile causes movement of the contact surface from the first region to the second region.

4. The method of claim 1, further comprising: A thermal element is coupled to the MAI device to cause a change in the temperature of the volume of the medium, the thermal element being configured to increase the temperature of a projected volume of air projecting a portion of the simulated surface where the contact surface contacts the simulated surface.

5. The method of claim 4, further comprising: The thermal element is arranged in an element of the MAI device, wherein the element projects the portion of the simulated surface.

6. The method of claim 4, further comprising: A projection nozzle of an element of the MAI device is modified to induce said temperature change, wherein said projection nozzle projects said portion of said simulated surface.

7. The method of claim 1, wherein the measure of the liquid flow in the contact surface is the blood flow rate in the capillaries at the contact surface.

8. The method of claim 7, wherein a first value of the measure of the fluid flow corresponds to a rate of the blood flow below a first threshold, the first value indicating a reduction in tactile sensitivity due to numbness of the contact surface.

9. The method of claim 8, further comprising: As part of changing the temperature, the temperature is increased to a first temperature that restores the tactile sensitivity in the contact surface by reducing the numbness.

10. The method of claim 7, wherein: A second value of the measure of the fluid flow corresponds to a rate of the blood flow above a second threshold, the second value indicating a reduction in tactile sensitivity due to inflammation in the contact surface.

11. The method of claim 10, further comprising: As part of changing the temperature, the temperature is reduced to a second temperature that restores the tactile sensitivity in the contact surface by reducing the inflammation.

12. The method of claim 1, further comprising: coupling a subsonic transmitter to the MAI device, the subsonic transmitter directing the subsonic signal to the contact surface; as well as A subsonic receiver is coupled to the MAI device, the subsonic receiver receiving reflections of the subsonic signal from the contact surface.

13. The method of claim 12, further comprising: The subsonic transmitter and the subsonic receiver are arranged in an element of the MAI device, wherein the element protrudes from a portion of the simulation surface and the contact surface contacts the portion.

14. The method of claim 12, further comprising: A projection nozzle of an element of the MAI device is modified to emit the subsonic signal and receive the reflection, wherein the projection nozzle projects a portion of the simulated surface and the contact surface contacts the portion.

15. The method of claim 1, wherein ultrasound is used to project the simulated surface of the object.

16. The method of claim 1, wherein the simulated surface of the object is projected using a column of air, a first column having a first controllable pressure, and a second column having a second controllable pressure.

17. A computer usable program product comprising a computer readable storage device and program instructions stored on the storage device, the program instructions being executable to perform the method of any preceding claim.

18. A computer system comprising a processor, a computer readable memory and a computer readable storage device, and program instructions stored on the storage device for execution by the processor via the memory, the program instructions being executable to perform the method according to any one of claims 1 to 16.

Images