Systems and methods for haptic intelligence

Abstract

The present invention relates generally to systems and methods for detecting, characterizing, and / or quantifying contact or touch interface aspects between a dedicated surface and other objects, and more particularly, to integration that may have one or more deformable transport layers configured to assist various aspects of haptic intelligence.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 316,915, filed on March 4, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention generally relates to systems and methods for detecting, characterizing, and / or quantifying aspects of contact or touch interfaces between specialized surfaces and other objects, and more specifically, to systems and methods that may have the integration of one or more deformable transmissive layers configured to assist with various aspects of tactile intelligence. Background Art

[0004] With the ubiquity of laptops, smartphones, and systems like video teleconferencing, computing, video communications, and various forms of telepresence have become key components of modern life. Figure 1 , a user (4) is interacting with a laptop (2) and a smartphone (6) simultaneously in a typical work or home environment. Figure 2A , a so-called "smart watch" (8) is shown detachably coupled to the arm of a user (4). Figure 2B A user (4) is shown holding a smartphone (6) while one hand (12) of the user (4) attempts to provide commands to the smartphone (6) computing system using gesture information. While these illustrative systems (2, 6, 8) can be configured to process voice-based or gesture-based commands, for example, most operations of these devices still occur through physical interfaces such as keyboards or touch screens, and most information exchanged during a video-based voice call is in the form of audio and / or video. Figures 3A to 3E ,Many efforts have been made to utilize modern systems to ,increase the richness of interpersonal communication and / or so-called "telepresence". Figure 3A A laptop (2) based video conferencing configuration is shown in which a user (4) is able to observe certain aspects of and communicate with a group of other participants through a matrixed video user interface (14) viewed through a laptop display (16). Figure 3B A conference room-based video conferencing system is shown in which a group of local participants gathered around a local conference table (20) are able to interact with remote participants via a relatively large display configured to display video of the remote participants via a teleconferencing user interface (18). Figure 3C, another system allows a group of local participants (34) seated around a local conference table in a local conference room (22) to interact via video teleconference with a group of remote participants, the remote participants being displayed via multiple integrated display / camera systems organized relative to the local conference table to help create or simulate the feeling that all participants are in the same location, or are able to communicate at least as if they were all local. Reference Figure 3D and 3E , video systems can be used to help bring remote users into local discussions about scenarios such as healthcare. Figure 3D A configuration is shown in which one user (4) from a first location is able to operate a multi-display (36, 38, 40) configuration, such as via one or more user input devices (44), to view video from a second operating location and information and / or data related to the scene, while a camera (42) captures video data of the participant (4) at the first location and provides a video feed to the second operating location to enhance communication (i.e., more than just voice). Figure 3E A configuration is shown in which a group of local healthcare providers (46, 48) with a patient (50) are using a cart (52)-based configuration having a display (54) to generate a video portrait (58) of the remote participant while simultaneously capturing a video of the local environment for the remote participant using a video camera (56) coupled to the cart (52). Figure 4 A similar video communication system for healthcare is shown, wherein a remote user (58) (e.g., a physician) is able to navigate to a local healthcare facility room (68) containing a patient (50) and a bed (60) using an electromechanically movable system (62), wherein a camera (64) and a display (66) are coupled to the electromechanically movable system to allow the remote user (58) to have a form of "telepresence" or "local presence" within the patient room (68).

[0005] While each of the above configurations has utility beyond traditional voice calls, some argue that they still lack some key aspects of true local presence. As connectivity, computing, video, audio, and telecommunications technologies continue to improve, such systems will undoubtedly continue to evolve to achieve closer to real-time local video presence. However, one key aspect of local presence that such systems do not address is the sense of local “touch” for the remote participant—which may be relevant to the continued high demand for air travel in certain business, social, and other scenarios. Touch and tactile intelligence are ubiquitous and crucial in modern people’s daily lives, and it’s no coincidence that some people, such as those with impaired vision, may be remarkably adept at relying on touch and tactile intelligence to navigate the world. As we evolved to use our eyes’ two perspectives to perform basic interpretations of the shapes of objects, we are also able to use touch and tactile intelligence to understand key aspects of the objects we physically encounter.

[0006] To study a relatively simple example, we can consider the scenario of remote inspection. If, in a given user scenario, it is crucial to inspect a specific object or surface in detail for surface distortions, potential stress concentrations and / or deformations, e.g. Figure 5A In the scenario shown where a plurality of rivets (72) are securing aircraft wing surfaces (70) in place, one solution is to travel to the location of each such aircraft wing surface and personally (74) inspect such surface (70), such as using an inspection light (76) configured to direct light onto the surface (70) at a selected angle to reveal surface anomalies. Similarly, with reference to Figure 6A If a smartphone (6) housing (80) design has a certain texture on its exterior paint, or if a "tight but not too tight" fit between the smartphone (6) camera assembly (78) and the housing (80) is critical before approval for mass manufacturing, workers will typically fly across the world to physically inspect the parts. Figure 6B Another example is shown where tactile sensations can be very valuable in determining whether the materials, fit, and finish of the crown (86), bezel (88), and / or buttons (84) of a watch (82) design are suitable for manufacture. Finally, with reference to Figure 6C , wherein the design of the detachable watch band (90) of the smart watch (8) is configured to slideably couple and separate with the watch (8) through a firm but not too firm engagement of these components with the user's hand (94, 95), touch sensation can be of great value when conducting an inspection. Techniques are needed to help users obtain touch sensation to extend their normal physical range, such as to remote locations. Systems, methods, and configurations for enhancing and broadening touch representations in various scenarios and using such representations for various purposes are described herein, including but not limited to high-precision touch sensor implementations and configurations that can be used and configured to help provide a local user with touch perception of objects outside of their normal range, such as objects in a remote environment. Summary of the Invention

[0007] One embodiment is directed to a system for characterizing interactions between surfaces, comprising: a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interfaced object; a first illumination source operatively coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer, such that at least a portion of the first illumination light interacts with the deformable transmission layer; a detector configured to detect light from within at least a portion of the deformable transmission layer; a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a location along the interface membrane based at least in part on the interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation; and an auxiliary sensor operatively coupled to the computing system and configured to provide input that the computing system can utilize to further geometrically characterize the interface object. The auxiliary sensor can be coupled to the deformable transmission layer. The system may further include an auxiliary sensor mounting structure coupled to the deformable transmission layer, wherein the auxiliary sensor is coupled to the auxiliary sensor mounting structure. The auxiliary sensor and the deformable transmission layer may be located in an operating environment including one or more wall structures, wherein the auxiliary sensor is coupled to one of the one or more wall structures. The auxiliary sensor may be selected from the group consisting of an inertial measurement unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LIDAR device, a strain sensor, a load sensor, a temperature sensor, and an image capture device. The first illumination source may include a light-emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The system may further include a lens operably coupled between the detector and the deformable transmission layer. A computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0008] Another embodiment relates to a system for characterizing interactions between surfaces, comprising: a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer, such that at least a portion of the first illumination light interacts with the deformable transmission layer; a detector configured to detect light from within at least a portion of the deformable transmission layer; a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a position along the interface membrane based at least in part on the interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation; and a robotic manipulator operably coupled to the computing system and the deformable transmission layer, the robotic arm configured to controllably position and orient the deformable transmission layer relative to the interface object such that the computing system can characterize the geometric profile of at least one aspect of the interface object interfaced with the interface membrane relative to the relative position and orientation of each of the deformable transmission layer and the interface object. The robotic manipulator may include a robotic arm. The robotic arm may include a plurality of joints coupled by substantially rigid linkage members. The robotic manipulator may include a flexible robotic instrument. The system may further include an end effector coupled to the robotic manipulator. The end effector may include a gripper. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The system may further include a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastic matrix. The pigment material may include a metal oxide.The interface film may include an elastic material.

[0009] Another embodiment relates to a system for characterizing the geometry of a portion of an appendage of a user, comprising: a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with at least one aspect of an interface object; a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; a detector configured to detect light from within at least a portion of the deformable transmission layer; and a computing system configured to operate the detector to detect light emitted from the deformable transmission layer. The system further includes a computing system configured to provide a computing system for guiding at least a portion of the light directed by the first illumination light through the deformable transmission layer to determine a surface orientation associated with a position along the interface membrane based at least in part on an interaction of the first illumination light with the deformable transmission layer, and utilizing the determined surface orientation to characterize a geometric profile of at least one aspect of an interface object interfaced with the interface membrane; a secondary sensor operably coupled to the computing system and configured to provide input that the computing system can utilize to further geometrically characterize the interface object; and a measurement housing configured to facilitate convenient engagement with a portion of a user's appendage, the measurement housing being coupled to the deformable transmission layer and the secondary sensor. The measurement housing can be configured to facilitate engagement with a plantar aspect of a user's foot. The measurement housing can be configured to be placed on a floor so that a user can step into the measurement housing. The secondary sensor can be coupled to the deformable transmission layer. The system can also include a secondary sensor mounting structure coupled to the deformable transmission layer, wherein the secondary sensor is coupled to the secondary sensor mounting structure. The secondary sensor and the deformable transmission layer can be coupled to the measurement housing. The auxiliary sensor may be selected from the group consisting of an inertial measurement unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LIDAR device, a strain sensor, a load sensor, a temperature sensor, and an image capture device. The first illumination source may include a light-emitting diode (LED). The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The system may further include a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material configured to provide an illumination reflectivity greater than that of the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0010] Another embodiment relates to a system for characterizing an interfacial interaction with at least one aspect of an interface structure, comprising: a plurality of touch sensing assemblies, each of the plurality of touch sensing assemblies being positioned circumferentially about a distal portion of a substrate member and comprising a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with the at least one aspect of the interface structure; a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; and a detector configured to detect an illumination light from the deformable transmission layer. The invention also provides a method for detecting light within at least a portion of the deformable transmission layer of the substrate member; and a computing system operably coupled to each touch sensing component and configured to operate each detector to detect at least a portion of the light directed from each deformable transmission layer to determine a surface orientation associated with a position along each interface film based at least in part on an interaction of the associated first illumination light with the associated deformable transmission layer; wherein each of the plurality of touch sensing components is configured to interface with one or more aspects of the interface structure to generally process a surface of the interface structure; and wherein the computing system is configured to characterize a geometric profile of the interface structure interfaced with the plurality of interface films using the determined surface orientation. Each of the plurality of touch sensing components may be positioned in a separated perimeter around the distal portion of the substrate member. Each of the plurality of touch sensing components may be positioned in a substantially equally separated perimeter around the distal portion of the substrate member. The plurality of touch sensing components may include two touch sensing components positioned diametrically opposite one another along the length of the substrate member. The plurality of touch sensing components may include three or more touch sensing components positioned in a circumferentially equally distributed manner along the length of the substrate member. The substrate member may include an elongated member having a proximal end and a distal end with a member length therebetween, and wherein each of the plurality of touch sensing components is peripherally positioned at substantially equal lengths along the member length. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The system may further include a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU).The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material being configured to provide a greater reflectivity of illumination than the elastic matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0011] Another embodiment relates to a system for characterizing interactions between surfaces, comprising: a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with at least one aspect of an interface object; a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; a detector configured to detect light from within at least a portion of the deformable transmission layer;

[0012] and a computing system configured to operate a detector to detect at least a portion of light directed from a deformable transmission layer to determine a surface orientation associated with a location along an interface membrane based at least in part on an interaction of first illumination light with the deformable transmission layer, and to utilize the determined surface orientation to characterize a geometric profile of at least one aspect of an interface object interfaced with the interface membrane, wherein the deformable transmission layer and the computing system are coupled to a portion of a road vehicle and configured to facilitate detection of one or more operational aspects of the road vehicle. The deformable transmission layer may be coupled to a portion of an interior of the road vehicle. The deformable transmission layer may be configured to be utilized by the computing system to detect one or more aspects of physical engagement between a user and a portion of the interior of the road vehicle. The deformable transmission layer may be configured to be utilized by the computing system to detect one or more aspects of contact between one or more parts of a user's body and a portion of the interior of the road vehicle. The deformable transmission layer may be coupled to a portion of an exterior of the road vehicle. The deformable transmission layer may be configured to be utilized by the computing system to detect one or more aspects of physical engagement between an element of the vehicle's external environment and the exterior of the road vehicle. The deformable transmission layer can be configured to be utilized by a computing system to detect physical intrusion from the vehicle's external environment, the physical intrusion causing deflection of one or more structures comprising the road vehicle's exterior. The first illumination source can include a light-emitting diode. The detector can include a photodetector. The detector can include an image capture device. The image capture device can be a CCD or CMOS device. The system can also include a lens operably coupled between the detector and the deformable transmission layer. The computing system can be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system can be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer can include an elastic material. The elastic material can be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer can include a composite material having a pigment material distributed within an elastic matrix, the pigment material configured to provide an illumination reflectivity greater than that of the elastic matrix. The pigment material can include a metal oxide. The interface film can include an elastic material.

[0013] Another embodiment relates to a system for characterizing an interaction between surfaces, comprising: a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; a detector configured to detect light from within at least a portion of the deformable transmission layer;

[0014] A computing system is also provided, configured to operate a detector to detect at least a portion of light directed from a deformable transmission layer, determine a surface orientation associated with a location along an interface membrane based at least in part on interaction of first illumination light with the deformable transmission layer, and utilize the determined surface orientation to characterize a geometric profile of at least one aspect of an interface object interfaced with the interface membrane. The deformable transmission layer and the computing system are coupled to a portion of a door lock system and configured to facilitate user authentication prior to electromechanically translating the door lock system from a locked state to an unlocked state. The door lock system may be configured to biometrically authenticate the user using the deformable transmission layer. The door lock system may be configured to capture a geometric profile associated with an outer surface of a portion of the user's appendage and compare the geometric profile to a previously stored geometric profile for authentication purposes. The outer surface of the portion of the user's appendage may be the user's fingerprint. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The system may also include a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to a first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material configured to provide a greater reflectivity of illumination than the elastic matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0015] Another embodiment relates to a handheld system for characterizing interactions between surfaces, comprising: a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; a detector configured to detect light from within at least a portion of the deformable transmission layer;

[0016] And, a computing system configured to operate a detector to detect at least a portion of light directed from a deformable transmission layer, determine a surface orientation associated with a position along an interface membrane based at least in part on an interaction of first illumination light with the deformable transmission layer, and utilize the determined surface orientation to characterize a geometric profile of at least one aspect of an interface object interfaced with the interface membrane. The deformable transmission layer and the computing system are coupled within a handheld system housing that includes a power source, and the handheld system housing is configured to facilitate manual operation by a user, such that the user can manually position and orient the deformable transmission layer to engage the interface membrane with the interface object. The system may also include a positioning sensor operably coupled to the handheld system housing and the computing system. The positioning sensor may be configured to be utilized by the computing system to determine a position of at least a portion of the handheld system housing within a global coordinate system. The computing system and the positioning sensor may be further configured to determine an orientation of at least a portion of the handheld system housing within the global coordinate system. The computing system and the positioning sensor may be further configured to determine the position and orientation of the deformable transmission layer within the global coordinate system. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The system may further include a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to a first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastic matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0017] Another embodiment relates to a system for characterizing interactions between surfaces, comprising: a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer, such that at least a portion of the first illumination light interacts with the deformable transmission layer; a detector configured to detect light from within at least a portion of the deformable transmission layer; and a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer to determine a surface orientation associated with a location along the interface membrane based at least in part on the interaction of the first illumination light with the deformable transmission layer, and to characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation, wherein the deformable transmission layer comprises a material of a known structural modulus such that the computing system can estimate a load at an interface between the deformable transmission layer and the interface object. The deformable transmission layer may comprise a known elastic modulus. The deformable transmission layer may include an unloaded shape configuration that is deformable into a loaded shape when loaded, and wherein the computing system is configured to determine one or more aspects of loading on the deformable transmission layer when in the loaded shape based on the loaded shape configuration and a material comprising a known structural modulus of the deformable transmission layer. The computing system may be configured to determine one or more aspects of loading on the deformable transmission layer, the one or more aspects of loading being selected from the group consisting of: tensile loading, compressive loading, bending loading, shear loading, and torsion loading. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The system may further include a lens operably coupled between the detector and the deformable transmission layer. The computing system is operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system is operably coupled to the first illumination source and configured to control emission of the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastic matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0018] Another embodiment relates to a system for characterizing interactions between surfaces, comprising: a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with at least one aspect of an interface object; a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer, such that at least a portion of the first illumination light interacts with the deformable transmission layer; a detector configured to detect light from within at least a portion of the deformable transmission layer; a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a location on the interface film based at least in part on the interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface object interfaced with the interface film using the determined surface orientation; and a touch-translating interface coupled to an aspect of a user's appendage and operably coupled to the computing system, the touch-translating interface configured to translate one or more aspects of the characterized geometric profile into aspects of the user's appendage such that the user can intuitively experience one or more aspects of the physical coupling between the deformable transmission layer and the interface object. The touch conversion interface can be coupled to a user's appendage at a location selected from the group consisting of: a user's finger; a user's wrist; and a user's forearm. The touch conversion interface can be coupled to an aspect of the user's appendage at a selected location such that there is a kinematic similarity between the location and the location of the deformable transmission layer. The system may also include a second touch conversion interface coupled to a second aspect of the user's appendage. The first and second touch conversion interfaces are longitudinally displaceable relative to each other when coupled to the user's appendage. The first illumination source may include a light-emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The system may also include a lens operably coupled between the detector and the deformable transmission layer. A computing system is operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0019] Another embodiment relates to a system for robotic medical intervention, comprising: a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface tissue structure; a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer, such that at least a portion of the first illumination light interacts with the deformable transmission layer; a detector configured to detect light from within at least a portion of the deformable transmission layer; a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a position along the interface membrane based at least in part on the interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface tissue structure interfaced with the interface membrane using the determined surface orientation; and a robotic medical instrument operably coupled to the computing system and the deformable transmission layer, the robotic instrument configured to controllably position and orient the deformable transmission layer relative to the interface tissue structure such that the computing system can characterize the geometric profile of at least one aspect of the interface tissue structure interfaced with the interface membrane relative to the relative position and orientation of the deformable transmission layer and the interface object. The robotic medical instrument may include a robotic arm. The robotic arm may include a plurality of joints coupled by substantially rigid linkage members. The robotic medical instrument may include a flexible robotic instrument. The robotic medical instrument may include a medical end effector coupled to the robotic medical instrument. The medical end effector may include a gripper. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The system may further include a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material may include a metal oxide.The interface film may include an elastic material.

[0020] Another embodiment relates to a method for characterizing interactions between surfaces, comprising: providing a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with at least one aspect of an interface object; providing a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer, such that at least a portion of the first illumination light interacts with the deformable transmission layer; providing a detector configured to detect light from within at least a portion of the deformable transmission layer; providing a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer to determine a surface orientation associated with a position along the interface film based at least in part on the interaction of the first illumination light with the deformable transmission layer, and characterizing a geometric profile of at least one aspect of the interface object interfaced with the interface film using the determined surface orientation; providing an auxiliary sensor operably coupled to the computing system and configured to provide input that the computing system can utilize to further geometrically characterize the interface object; and determining the surface orientation using the computing system and using the input from the auxiliary sensor to further geometrically characterize the interface object. The auxiliary sensor can be coupled to the deformable transmission layer. The method may further include providing an auxiliary sensor mounting structure coupled to the deformable transmission layer, wherein the auxiliary sensor is coupled to the auxiliary sensor mounting structure. The auxiliary sensor and the deformable transmission layer may be located in an operating environment including one or more wall structures, and wherein the auxiliary sensor is coupled to one of the one or more wall structures. The auxiliary sensor may be selected from the group consisting of an inertial measurement unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LIDAR device, a strain sensor, a load sensor, a temperature sensor, and an image capture device. The first illumination source may include a light-emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The method may further include providing a lens operably coupled between the detector and the deformable transmission layer. A computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0021] Another embodiment relates to a method for characterizing an interaction between surfaces, comprising: providing a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with at least one aspect of an interface object; providing a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; providing a detector configured to detect light from within at least a portion of the deformable transmission layer; providing a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer to determine an interaction between the deformable transmission layer and the interface film based at least in part on the interaction of the first illumination light with the deformable transmission layer. The method further includes providing a robotic manipulator operatively coupled to the computing system and the deformable transmission layer, the robotic arm being configured to controllably position and orient the deformable transmission layer relative to the interface object, such that the computing system can characterize at least one aspect of the geometric profile of the interface object interfaced with the interface membrane relative to the relative position and orientation of the deformable transmission layer and the interface object; and operating the robotic manipulator, using the computing system, to controllably position and orient the deformable transmission layer relative to the interface object and characterize at least one aspect of the geometric profile of the interface object interfaced with the interface membrane. The robotic manipulator may include a robotic arm. The robotic arm may include a plurality of joints coupled by substantially rigid linkage members. The robotic manipulator may include a flexible robotic instrument. The method may further include providing an end effector coupled to the robotic manipulator. The end effector may include a gripper. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The method may further include providing a lens operatively coupled between the detector and the deformable transmission layer. The computing system is operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system is operably coupled to a first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0022] Another embodiment relates to a method for characterizing the geometry of a portion of a user's appendage, comprising: providing a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with at least one aspect of an interface object; providing a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; providing a detector configured to detect light from within at least a portion of the deformable transmission layer; and providing a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer to at least Determining a surface orientation associated with a position along the interface membrane based in part on interaction of first illumination light with the deformable transmission layer, and utilizing the determined surface orientation to characterize a geometric profile of at least one aspect of an interface object interfaced with the interface membrane; providing an auxiliary sensor operably coupled to a computing system and configured to provide input that the computing system can utilize to further geometrically characterize the interface object; providing a measurement housing configured to facilitate convenient engagement with a user's appendage, the measurement housing coupled to the deformable transmission layer and the auxiliary sensor; and operating the detector and the auxiliary sensor using the computing system and the measurement housing to characterize the geometric profile of at least one aspect of the interface object interfaced with the interface membrane. The measurement housing can be configured to facilitate engagement with a plantar aspect of a user's foot. The measurement housing can be configured to rest on a floor so that a user can step into the measurement housing. The auxiliary sensor can be coupled to the deformable transmission layer. The method can also include providing an auxiliary sensor mounting structure coupled to the deformable transmission layer, wherein the auxiliary sensor is coupled to the auxiliary sensor mounting structure. The auxiliary sensor and the deformable transmission layer can be coupled to the measurement housing. The auxiliary sensor may be selected from the group consisting of an inertial measurement unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LIDAR device, a strain sensor, a load sensor, a temperature sensor, and an image capture device. The first illumination source may include a light-emitting diode (LED). The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The method may further include providing a lens operably coupled between the detector and the deformable transmission layer. The computing system is operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.The pigment material may include a metal oxide.The interface film may include an elastic material.

[0023] Another embodiment relates to a method for characterizing an interface interaction with at least one aspect of an interface structure, comprising: providing a plurality of touch sensing assemblies, each of the plurality of touch sensing assemblies being positioned peripherally about a distal portion of a substrate member and comprising a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with the at least one aspect of the interface structure; a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; and a detector configured to detect the first illumination light. The invention also provides a method for detecting light directed from at least a portion of the deformable transmission layer to a plurality of touch sensing components, wherein the plurality of touch sensing components are configured to detect light directed from at least a portion of the deformable transmission layer to determine a surface orientation associated with a position along each interface film based at least in part on an interaction of the associated first illumination light with the associated deformable transmission layer; configuring each of the plurality of touch sensing components to interface with one or more aspects of the interface structure to generally process the surface of the interface structure; and operating the computing system to characterize a geometric profile of the interface structure interfaced with the plurality of interface films using the determined surface orientation. Each of the plurality of touch sensing components can be positioned in a separated perimeter around the distal portion of the substrate member. Each of the plurality of touch sensing components can be positioned in a substantially equally separated perimeter around the distal portion of the substrate member. The plurality of touch sensing components can include two touch sensing components positioned diametrically opposite one another along the length of the substrate member. The plurality of touch sensing components can include three or more touch sensing components positioned in a circumferentially equally distributed manner along the length of the substrate member. The substrate member may include an elongated member having a proximal end and a distal end with a member length therebetween, wherein each of the plurality of touch sensing components is peripherally positioned at substantially equal lengths along the member length. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The method may further include providing a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU).The deformable transmission layer may include a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastomeric material.

[0024] Another embodiment relates to a method for characterizing interactions between surfaces, comprising: providing a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; providing a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer, such that at least a portion of the first illumination light interacts with the deformable transmission layer; providing a detector configured to detect light from within at least a portion of the deformable transmission layer; providing a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer to determine a surface orientation associated with a location along the interface membrane based at least in part on the interaction of the first illumination light with the deformable transmission layer, and characterizing a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation; and configuring the deformable transmission layer and the computing system to be coupled to a portion of a road vehicle and configured to facilitate detection of one or more operational aspects of the road vehicle. The deformable transmission layer may be coupled to a portion of an interior of the road vehicle. The deformable transmission layer may be configured to be utilized by a computing system to detect one or more aspects of physical engagement between a user and a portion of a road vehicle interior. The deformable transmission layer may be configured to be utilized by the computing system to detect one or more aspects of contact between one or more parts of a user's body and a portion of the road vehicle interior. The deformable transmission layer may be coupled to a portion of the road vehicle exterior. The deformable transmission layer may be configured to be utilized by the computing system to detect one or more aspects of physical engagement between an element of the vehicle exterior environment and the road vehicle exterior. The deformable transmission layer may be configured to be utilized by the computing system to detect physical intrusion from the vehicle exterior environment that causes deflection of one or more structures comprising the road vehicle exterior. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The method may further include providing a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastomeric material. The elastomeric material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastomeric matrix, the pigment material configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastomeric material.

[0025] Another embodiment relates to a method for characterizing interactions between surfaces, comprising: providing a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with at least one aspect of an interface object; providing a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer, such that at least a portion of the first illumination light interacts with the deformable transmission layer; providing a detector configured to detect light from within at least a portion of the deformable transmission layer; providing a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a location along the interface film based at least in part on the interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface object interfaced with the interface film using the determined surface orientation; and configuring the deformable transmission layer and the computing system to couple to portions of a door lock system and to facilitate authentication of a user prior to electromechanically switching the door lock system from a locked state to an unlocked state. The door lock system can be configured to biometrically authenticate the user using the deformable transmission layer. The door lock system can be configured to capture a geometric profile associated with an outer surface of a portion of a user's appendage and compare the geometric profile to a previously stored geometric profile for authentication purposes. The outer surface of the portion of the user's appendage can be a fingerprint of the user. The first illumination source can include a light-emitting diode. The detector can include a photodetector. The detector can include an image capture device. The image capture device can be a CCD or CMOS device. The method can also include providing a lens operably coupled between the detector and the deformable transmission layer. The computing system can be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system can be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer can include an elastic material. The elastic material can be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer can include a composite material having a pigment material distributed within an elastic matrix, the pigment material configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material can include a metal oxide. The interface membrane may comprise an elastic material.

[0026] Another embodiment relates to a method for characterizing interactions between surfaces using a handheld component, comprising: providing a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; providing a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer, such that at least a portion of the first illumination light interacts with the deformable transmission layer; providing a detector configured to detect light from within at least a portion of the deformable transmission layer; providing a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a location along the interface membrane based at least in part on the interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation; and coupling the deformable transmission layer and the computing system within a handheld system housing including a power source, the handheld system housing configured to facilitate manual operation by a user, such that the user can manually position and orient the deformable transmission layer to engage the interface membrane with the interface object. The method may also include providing a positioning sensor operably coupled to the handheld system housing and the computing system. The positioning sensor may be configured to be utilized by the computing system to determine the position of at least a portion of the handheld system housing within a global coordinate system. The computing system and the positioning sensor may further be configured to determine the orientation of at least a portion of the handheld system housing within the global coordinate system. The computing system and the positioning sensor may further be configured to determine the position and orientation of the deformable transmission layer within the global coordinate system. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The method may also include providing a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0027] Another embodiment relates to a method for characterizing interactions between surfaces, comprising: providing a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; providing a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer, such that at least a portion of the first illumination light interacts with the deformable transmission layer; providing a detector configured to detect light from within at least a portion of the deformable transmission layer; providing a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a location along the interface membrane based at least in part on the interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation; and configuring the deformable transmission layer to include a material having a known structural modulus such that the computing system can estimate a load at an interface between the deformable transmission layer and the interface object. The deformable transmission layer may include a known elastic modulus. The deformable transmission layer may include an unloaded shape configuration that is deformable into a loaded shape when loaded, wherein the computing system is configured to determine one or more aspects of a load on the deformable transmission layer when in the loaded shape based on the loaded shape configuration and a material comprising a known structural modulus of the deformable transmission layer. The computing system may be configured to determine one or more aspects of the loading on the deformable transmission layer, the one or more aspects of the loading being selected from the group consisting of: tensile loading, compressive loading, bending loading, shear loading, and torsion loading. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The method may further include providing a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material being configured to provide a greater illumination reflectivity than the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0028] Another embodiment relates to a method for characterizing an interaction between surfaces, comprising: providing a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with at least one aspect of an interface object; providing a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; providing a detector configured to detect light from within at least a portion of the deformable transmission layer; and providing a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer to determine a first illumination orientation based at least in part on the first illumination light. Interaction of bright light with the deformable transmission layer determines a surface orientation associated with a position along the interface film, and utilizing the determined surface orientation to characterize a geometric profile of at least one aspect of an interface object interfaced with the interface film; providing a touch conversion interface coupled to an aspect of a user's appendage and operably coupled to a computing system, the touch conversion interface configured to convert one or more aspects of the characterized geometric profile to the aspect of the user's appendage so that the user can intuitively experience one or more aspects of the physical engagement between the deformable transmission layer and the interface object; and operating the touch conversion interface using the computing system so that the user can intuitively experience one or more aspects of the physical engagement between the deformable transmission layer and the interface object. The touch conversion interface may be coupled to the user's appendage at a location selected from the group consisting of: a user's finger; a user's wrist; and a user's forearm. The touch conversion interface may be coupled to an aspect of the user's appendage at a selected location such that there is a kinematic similarity between the location and the location of the deformable transmission layer. The method may further include providing a second touch conversion interface coupled to a second aspect of the user's appendage. The first and second touch conversion interfaces are longitudinally displaceable relative to each other when coupled to the user's appendage. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The method may further include providing a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to the first illumination source and configured to control emission of the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastic material.

[0029] Another embodiment relates to a method for robotic medical intervention, comprising: providing a deformable transmission layer coupled to an interface membrane, wherein the interface membrane engages at least one aspect of an interface tissue structure; providing a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; providing a detector configured to detect light from within at least a portion of the deformable transmission layer; providing a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer to determine a position based at least in part on an interaction of the first illumination light with the deformable transmission layer; The method further includes: providing a robotic medical instrument operably coupled to a computing system and the deformable transmission layer, the robotic instrument configured to controllably position and orient the deformable transmission layer relative to the interface tissue structure so that the computing system can characterize the geometric profile of at least one aspect of the interface tissue structure interfaced with the interface membrane relative to the relative position and orientation of each of the deformable transmission layer and the interface object; and operating the robotic medical instrument using the computing system to characterize the geometric profile of at least one aspect of the interface tissue structure interfaced with the interface membrane. The robotic medical instrument may include a robotic arm. The robotic arm may include a plurality of joints coupled by substantially rigid linkage members. The robotic medical instrument may include a flexible robotic instrument. The robotic medical instrument may include a medical end effector coupled to the robotic medical instrument. The medical end effector may include a gripper. The first illumination source may include a light emitting diode. The detector may include a photodetector. The detector may include an image capture device. The image capture device may be a CCD or CMOS device. The method further includes providing a lens operably coupled between the detector and the deformable transmission layer. The computing system may be operably coupled to the detector and configured to receive information from the detector regarding light detected by the detector from within the deformable transmission layer. The computing system may be operably coupled to a first illumination source and configured to control emission from the first illumination source. The deformable transmission layer may include an elastic material. The elastic material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), and thermoplastic polyurethane (TPU). The deformable transmission layer may include a composite material having a pigment material distributed within an elastic matrix, the pigment material configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix. The pigment material may include a metal oxide. The interface film may include an elastic material. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1-4 Various aspects of a conventional computing and communications system are shown.

[0031] Figures 5A-6C Various aspects of the scene are shown where an enhanced understanding of surface geometry or contours would be useful.

[0032] Figures 7A-7H and Figure 8 Various aspects of a touch sensing assembly configured to utilize a deformable transmission layer are shown.

[0033] Figure 9A and 9B A plurality of touch sensing components are shown, e.g. Figures 7A-7H Components shown in .

[0034] Figures 10A-10I Various aspects of touch sensing assembly embodiments are shown, which may have one or more auxiliary sensor configurations integrated therein.

[0035] Figure 11-15 Aspects of touch sensing component integration are shown, where electromechanical systems, such as robotics, can be utilized to gain further tactile intelligence about a target object or surface.

[0036] Figures 16A-16B and 17 illustrate aspects of configurations in which one or more touch sensing components may be utilized to at least partially characterize a portion of an appendage, such as a foot or a portion of an arm of a user.

[0037] Figures 18A-18L Aspects of configurations are shown for integrating one or more touch sensing components into a complex system that may involve controlled electromechanical motion, such as by robotics, and placement of deformable transmission layers at various locations along the length of the various components and around the contours of the external surface shapes of the various components, such as relative to the perimeter of an elongated instrument.

[0038] Figure 19A-35 Aspects of system and method integration are shown wherein one or more touch sensing components may be used to assist in translating physical engagement back to a user at a workstation, which may be local or remote with respect to the physical engagement.

[0039] Figure 36 、 39 , 40, 42 and 46-47 illustrate aspects of medical system and method integration in which one or more touch sensing components may be used to help translate physical engagement at a tissue intervention location back to a user at a workstation, which may be local or remote relative to the physical engagement of the tissue.

[0040] Figure 37 and41 Aspects of gaming or virtual engagement system and method integration are shown in which one or more simulated touch sensing components can be utilized to help translate physical engagement at a user interface workstation.

[0041] Figures 38A-38F and 43-45 illustrate integrated aspects where one or more sensing components may be utilized to help characterize one or more key working components of a component or machine.

[0042] Figures 48-50 An integrated aspect is shown wherein one or more sensing and / or touch-enabled interfaces may be utilized to assist in the local user perception experience and facilitate user-initiated commands. DETAILED DESCRIPTION

[0043] refer to Figure 7A , a digital touch sensing assembly (146) is shown, which features a deformable transmission layer (110) operably coupled to an optical element (108) illuminated by one or more mutually coupled light sources (116, 122) and located within the field of view of an imaging device (106). A housing (118) is configured to maintain the positioning of the components relative to each other and expose a touch sensing contact surface (120). The interface film (100) can include a fixedly attached or removably coupled substantially thin layer, the thin layer including, for example, a relatively low bulk modulus polymer material, the interface film (100) can be positioned and operably coupled to the deformable transmission layer, or include a portion of the deformable transmission layer, for direct contact between other objects and the digital touch sensing assembly (146) for touch determination and characterization; thus, in the configuration where the interface film (100) is coupled to the deformable transmission layer or includes a portion of the deformable transmission layer, the resulting external touch contact surface (120) becomes the external aspect of such interface film (100). Aspects of suitable digital touch sensing assembly (146) configurations, generally featuring elastically deformable transmission layer materials, are described, for example, in U.S. Patent Nos. 10,965,854, 9,127,938, and 8,411,140, ​​each of which is incorporated herein by reference in its entirety. Figure 7AAs shown, the digital touch sensing assembly (146) shown can have a gap or void (114) that can contain an optically transmissive material (e.g., a material having a similar refractive index as the optical element 108), air, or a specialized gas (e.g., an inert gas) that is geometrically configured to place aspects of the optical element (108) and / or the deformable transmission layer (110) within a desired proximity of an imaging device (106), which can include an imaging sensor (e.g., a digital camera chip), a single light sensing element (e.g., a photodiode), or an array of light sensing elements, and which can be configured to have a field of view and depth of field facilitated by the geometric gap or void (114) (i.e., the gap or void 114 can be positioned to accommodate the field of view and / or depth of field pertaining to a particular imaging device 106). In various embodiments, the optical element (108) can include a substantially rigid material, a material of known elastic modulus, or a material of known structural modulus (i.e., given an unloaded shape and a loaded shape, a loading profile can be determined based on structural modulus information associated with the shape). Various suitable optical elements (108) may define an external shape, including, for example, a cylindrical, cubic, and / or rectangular prism shape. As shown and described below, various illumination sources may be coupled to one or more sidewall surfaces defining the optical element (108). In another embodiment, the optical element (108) may be configured to be deformable or conformable such that the effects of the rigidity of such structure on other associated elements are minimized (i.e., pulse loads, such as force / time variation (delta-time), may be minimized by greater impact compliance; furthermore, having a lower structural modulus at the contact interface may maintain greater surface contact on a given surface, such as a surface having topographical or geometric features).

[0044] Figure 7A Also shown is a computing device or system (104), which may include a computer, microcontroller, field programmable gate array, application specific integrated circuit, etc., configured to be operably coupled (128) to the imaging device (106) and further operably coupled (124, 126) to one or more light sources (30) to facilitate controlling these devices to collect data related to contact with the deformable transmission layer (110). For example, in one embodiment, each of the light sources (116, 122) includes a light emitting diode ("LED") operably coupled (124, 126) to the computing device (104) using electronic leads (124, 126), and the imaging device (106) includes a digital camera sensor chip operably coupled to the computing device using electronic leads (128), such as Figure 7AAs shown. The power supply (102) can be operatively coupled to the computing device (104) to provide power to the computing device (104), and can also be configured to controllably provide power to interconnected devices (e.g., the imaging device (106) and the light sources (116, 122)) via its couplings (128, 124, 126, respectively). Figure 7A As shown, separation (640) is depicted to indicate that these coupling interfaces (128, 124, 126) may be short or relatively long (i.e., the digital touch sensing assembly 146 may be remotely located relative to the computing device 104), and may be direct physical connections or transmit data through a wired or wireless interface, such as via an optical / optical network protocol, or a wireless network protocol, such as a Bluetooth (RTM) or 802.11 based configuration, which may be enabled by additional computing and power resources located locally at the digital touch sensing assembly (146).

[0045] refer to Figure 7B , showing the Figure 7A The configuration is similar to the one shown in , except that Figure 7B The deformable transmission layer (110) in the embodiment of the present invention includes one or more bladders or enclosed volumes (112) that can be occupied by, for example, a fluid (e.g., a liquid or gas, which can be physically considered a form of fluid). In one embodiment, for example, the deformable transmission layer (110) can include several individually controllable expandable segments or sub-volumes and can include a cross-sectional shape selected to provide specific mechanical properties under load, such as a controllable honeycomb cross-sectional shape configuration. As described above, the deformable transmission layer (110) can include one or more materials selected to match the touch sensing paradigm in terms of bulk and / or Young's modulus. In other words, for sensing relatively low loads, such as in digital contact scenarios interacting with a soap bubble or the surface of a living photosynthetic leaf of a plant, a relatively low modulus (i.e., generally locally flexible / deformable; not stiff) material (e.g., an elastomer), such as described in the aforementioned references, can be used for the deformable transmission layer (110) and / or the external interface membrane (100), which is removable as described above. The external interface membrane (100) may comprise a set of relatively thin and sequentially removable membranes such that when they become coupled with dirt or dust, they may be sequentially removed, for example, in a "tear-off" type manner. Figure 7B In the illustrated embodiment, in which the deformable transmission layer (110) includes a volume of liquid or gas that is at least temporarily trapped, the gas or liquid and its pressure can be adjusted to achieve a desired bulk modulus and sensitivity for the entire deformable transmission layer (110) (e.g., the pressure and / or volume associated with one or more bladder segments 112 can be adjusted) to generally change the functional modulus of the deformable transmission layer 110.

[0046] refer to Figure 7C , showing the Figure 7A Similar configuration, where Figure 7C The configuration shows that the gap (130) between the imaging device (106) and the optical element (108) can be reduced or even eliminated, depending on the optical layout of the imaging device (106), which can be coupled with refractive and / or diffractive optical elements to change the characteristics of the imaging device (106), such as focal length.

[0047] refer to Figure 7D , showing the Figure 7A Similar configuration, except that Figure 7D The configuration shows that one or more light sources may be more similar to light emitters (117, 123) that are configured to emit light originating from another location, such as coupled to one or more light-emitting LED light sources that are directly coupled to the computing device (104) and configured to transmit light through light-transmitting coupling members (132, 134) via optical fibers, "light pipes," or waveguides that can be configured to transfer photons from these sources to the emitters (117, 123) as efficiently as possible, such as via total internal reflection.

[0048] Similarly, reference Figure 7E , showing something similar to Figure 7D A configuration in which the imaging device (107) includes capture optics selected to collect photons and transmit them back to an image sensor via a light-transmitting coupling member (138) (e.g., a waveguide or one or more optical fibers), which may be located within the computing device (104) or other structure separate from the digital touch sensing assembly (146) or coupled to the computing device (204) or other structure.

[0049] refer to Figures 7F-7H , showing various aspects of a digital touch sensing assembly (146) configuration featuring a deformable transmission layer (110) that can be used to characterize interactions between surfaces. For example, referring to Figure 7FIn a simplified illustrative embodiment, a computing system or device (104) operatively coupled (136) to a power source (102) can be used to control, via a control coupling (124) that can be wired or wireless, light (1002) or other emissions from an illumination source (116) that can be directed into a deformable transmission layer (110). The deformable transmission layer (110) can be urged (1006) against at least a portion of an interface object (1004), such as the edge of a coin, and based on the interaction of the illumination (1002) with the deformable transmission layer (110), a detector (e.g., an image capture device (e.g., a CCD or CMOS device)) operatively coupled (128, such as via a wired or wireless connection) to the computing system (104) can be configured to detect at least a portion of the light directed from the deformable transmission layer. In other words, when an illumination source (116) is operably coupled (e.g., optically coupled to the effective transmission interface) to deliver illumination at a known orientation relative to the deformable transmission layer such that at least a portion of the illumination light interacts with the deformable transmission layer, and a detector is configured to detect light from within at least a portion of the deformable transmission layer, the computing system can be configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer to determine a surface orientation associated with a location along an interface of the deformable transmission layer with the interface object based at least in part on the interaction of the first illumination light with the deformable transmission layer, and to characterize a geometric profile of at least one aspect of the interface object interfaced with the interface film using the determined surface orientation. Figure 7G As discussed further below, an interface membrane (100) may be interposed between the interface object (1004) and the deformable transmission layer (110); the modulus of such an interface membrane may be similar to or different from the modulus of the deformable transmission layer. Preferably, an effective coupling is established between the deformable transmission layer and the membrane such that shear and primary or normal loads are efficiently transferred between these structures. Figure 7A, an embodiment is shown that includes an optical element (108) that can be configured to help precisely distribute light or other radiation among various parts of an assembled system. The optical element can include a highly transmissive, substantially rigid material; it can include a top surface, a bottom surface, and sides defined therebetween to form a three-dimensional shape, such as a cylinder, a cuboid, and / or a rectangular prism. The optical element (108) shown can be illuminated by one or more mutually coupled light sources (116, 122) and positioned within the field of view of an imaging device (106). The housing (118) is configured to maintain the positioning of the components relative to each other, and as described above, the interface film (100) can include a fixedly attached or removably coupled substantially thin layer, the thin layer including, for example, a relatively low bulk modulus polymer material, and the interface film (100) can be positioned in direct contact between other objects and the digital touch sensing assembly (146) for touch determination and characterization. Preferably, the deformable transmission layer and / or interface film comprises an elastomeric material, such as silicone, urethane, polyurethane, thermoplastic polyurethane (TPU), or thermoplastic elastomer (TPE). Other elastomers with lower light and / or radiation transmission efficiencies may also be used, such as natural rubber, chloroprene rubber, ethylene propylene diene monomer (EPDM) rubber, butyl rubber, nitrile rubber, styrene-butadiene rubber (SBR), fluororubber (Viton), fluorosilicone, and polyacrylates. The deformable transmission layer may comprise a composite material with a pigment material, such as a metal oxide (e.g., iron oxide, zinc oxide, aluminum oxide, and / or titanium dioxide), metal nanoparticles (e.g., silver nanoparticles and / or aluminum nanoparticles), or other molecules configured to differentially interact with incoming light or radiation, such as a dye distributed within an elastomeric matrix. The pigment material may be configured to provide an illumination reflectance that is greater than the illumination reflectance of the elastomeric matrix. The deformable transmission layer is defined by a bottom surface directly coupled to the interface film, a top surface closest to the detector, and a transmission layer thickness therebetween, wherein the pigment material is distributed near the bottom surface within the transmission layer thickness to provide optimized illumination reflectivity near the bottom surface. Aspects of suitable digital touch sensing assembly (146) configurations generally having elastically deformable transmission layer materials are described, for example, in U.S. Patent Nos. 10,965,854, 9,127,938, and 8,411,140, ​​each of which is incorporated herein by reference in its entirety. Figure 3CAs shown, the depicted digital touch sensing assembly (146) can have a gap or void (114) that can contain an optically transmissive material (e.g., a material having a refractive index similar to that of the optical element 108), air, or a specialty gas (e.g., an inert gas) that is geometrically configured to place aspects of the optical element (108) and / or the deformable transmissive layer (110) within a desired proximity of an imaging device (106), which can include an imaging sensor (e.g., a digital camera chip), a single light sensing element (e.g., a photodiode), or an array of light sensing elements, and can be configured to have a field of view and depth of field facilitated by the geometric gap or void (114). In another embodiment, the optical element (108) can be configured to be deformable or conformable so that the rigidity of such a structure minimizes the effect on other associated elements.

[0050] Figure 7A Also shown is a computing device or system (104), which may include a computer, microcontroller, field programmable gate array, application specific integrated circuit, etc., configured to be operatively coupled to an imaging device (106) and further coupled to one or more light sources (116, 122) to facilitate controlling these devices to collect data related to contact with the deformable transmission layer (110). For example, in one embodiment, each of the light sources (116, 122) includes a light emitting diode ("LED") operatively coupled (124, 126) to the computing device (104) using electronic leads, and the imaging device (106) includes a digital camera sensor chip operatively coupled to the computing device using electronic leads (128), such as Figure 7A As shown. The power supply (102) can be operatively coupled to the computing device (104) to provide power to the computing device (104), and can also be configured to controllably provide power to interconnected devices (e.g., the imaging device (106) and the light sources (116, 122)) via its couplings (128, 124, 126, respectively). Figure 7A As shown in (640), these coupling interfaces (124, 126, 128) can be short or relatively long (i.e., the digital touch sensing assembly 146 can be located at a remote location relative to the computing device 104) and can be direct physical connections or transmit data through a wired or wireless interface, such as via an optical / optical network protocol, or a wireless network protocol, such as a Bluetooth (RTM) or 802.11 based configuration, which can be achieved through additional computing and power resources located locally at the digital touch sensing assembly (146).

[0051] refer to Figure 7H, a partial schematic diagram shows that a computing system (104) can be operatively coupled (124, 126, 1012) to two, three, or more different illumination sources (116, 122, 1010), for example, via wired or wireless control leads; these illumination sources can be configured to have different emission wavelengths and / or different polarizations and, as shown, can be configured to emit from different orientations relative to an optical element (108) and an associated deformable transmission layer (110) to allow further data related to the geometric profile to be obtained.

[0052] refer to Figure 8 As described in the aforementioned reference (U.S. Patent No. 10,965,854), the deformable transmission layer or member (110) can include various geometric shapes and need not be planar or shaped as a rectangular prism or variations thereof; for example, the deformable transmission layer or member (110) can be curved, convex (144), saddle-shaped, etc., and can be customized for various specific contact sensing scenarios. For example, Figure 8 Multiple assemblies (146) with convex deformable transmission layers (110) are shown coupled to a grasping interface of a robotic gripper / hand to facilitate touch sensing / determination of a grasped item in a manner similar to the paradigm of skin segments between joints of a human hand grasping an object. Figure 8 The assembly (146) is configured with a housing geometry (142) and coupling features (140) to facilitate removable connection to other components.

[0053] refer to Figure 9A , multiple digital touch sensing components (146) can be used together to sense a larger surface (150) of an object (148). Each such component (146, Figure 9A five are shown in the figure) can be operatively coupled to one or more computing devices (104), such as by electronic leads (and can be interrupted by wireless connections, such as described above), as shown (152, 154, 156, 158, 160), and thus can be configured to exchange data and facilitate the transmission of power, light, and control and sensing information.

[0054] refer to Figure 9B , relative to Figure 9A , more (162) digital touch sensing components (146) can be used to partially or completely surround an object, or to monitor digital touch with two or more surfaces of the object. Figure 9B Each of the thirty digital touch sensing assemblies (146) shown in FIG can be operatively coupled to the same or different computing devices (104), and the coupling leads can be combined or coupled to form a single combined coupling lead assembly (164), as shown in FIG. Figure 9B shown.

[0055] refer to Figure 10A While an optional geometric separation (640) between various components (e.g., the digital touch sensing assembly (146) and the computing device (104)) is shown, it is important to note that these components may also be housed together and connected to other systems, components, and devices via a wireless transceiver (166), such as those designed to operate with the IEEE 802.11 so-called "WiFi" standard, and / or wireless connectivity and communication standards known under the "Bluetooth" trademark, such as Bluetooth 4.x and Bluetooth 5. Additionally, the mutually coupled (136, e.g., via direct leads) power supply (102) components shown may include one or more batteries, or one or more connections to other power sources (wired or wireless, such as via inductive power transfer) to provide further power supply and / or charging of the integrated power supply (102) components. Various embodiments described herein relate to miniaturized or miniaturizable configurations to assist integration into other systems (e.g., automotive systems), and it is desirable to facilitate integration of such systems with connectivity alternatives that may meet or be coordinated with known standards. For example, in various embodiments, the touch sensing system (e.g., Figure 10A The configuration of the touch sensing system shown in (a) can be miniaturized and encapsulated in a housing, and the connection configuration is designed for relatively simple integration into or with other systems. This system configuration can be viewed as moving toward the integration capabilities of the "Internet of Things", in which various devices are expected to collaborate relatively easily with other connected and integrated systems.

[0056] Reference again Figure 10A , shows a digital touch sensing assembly (146), which is similar to reference Figure 7AThe assembly described, but also having a series of additional sensing capabilities or "auxiliary sensor" elements that are selected to enhance the general capabilities of the assembly, for example by providing sensing data from one or more additional sensing subsystems that are typically co-located with the touch sensing capabilities provided by the deformable transmission layer and that may have their own sensing uncertainties and error levels, so that so-called "sensor fusion" techniques can be utilized to improve the overall capabilities of the integrated arrangement, for example by utilizing uncorrelated errors between the various sensing subsystems. For example, when a digital touch sensor based on the deformable transmission layer (110) may indicate contact with another object, but data from an integrated inertial measurement unit (or "IMU", such as accelerometer or gyroscope data from one or more accelerometers or gyroscopes that may include such an IMU), a LIDAR subsystem (such as point cloud data belonging to the alleged contact area), and an imaging device (such as a camera providing image data belonging to the alleged contact area) provide additional contradictory data with uncorrelated measurement / determination errors to determine that the digital touch sensor is not in contact, there is a high probability that the digital touch sensor is not in contact (the concept of at least partially uncorrelated errors is difficult for the other measurement / determination subsystems). is important because if all other measurement / determination subsystems have the same correlated errors, they may contribute some degree of redundancy or enhanced measurement, field of view, etc., but they may have similar error-based limitations; for example, mounting three pitot tubes on an aircraft wing may provide some redundancy and further measurement relative to a single pitot tube, but if they are all flying in freezing rain and fail due to the same correlated errors, the aircraft may be better off relying on subsystems with some uncorrelated errors, such as compass, GPS, trajectory planning, etc.; therefore, the concept of utilizing multiple sensors with at least some uncorrelated errors provides value, and can be achieved through the use of two or more sensors in a form known as "sensor fusion". Furthermore, as mentioned above, multiple sensors can be aggregated to supplement and extend the geometric range of the sensing paradigm, such as by coupling similar or dissimilar sensors adjacent to each other along a given surface or aspect of a structural element. Therefore, returning to reference Figure 10A , a series of additional sensing subsystems (IMU 172, capacitive touch sensing 174, resistive touch sensing 176, LIDAR sensing 178, strain or elongation sensing 180, load sensing 182, temperature sensing 184, additional image sensing 186) are shown as part of the integrated system configuration shown operatively coupled (188, 190, 192, 194, 196, 198, 200, 202 each represent connecting leads, such as wire leads, which may be as shown) with at least some uncorrelated errors. Figure 10A Shown connected to a communication / connectivity bus 170 , which may be directly coupled to the computing device 104 168 ).

[0057] For illustration purposes, Figures 10B-10I Various embodiments are depicted in which further details of the integration of various subsystems may be explored.

[0058] refer to Figure 10B , an embodiment is shown in which a digital touch sensing assembly (146) is integrated with an IMU (172) coupled thereto. The IMU (172) may include one or more accelerometers and one or more gyroscopes and may be fixedly coupled to a housing (118) of the digital touch sensing assembly (146) and operably coupled to a computing device (104), such as via leads (188; shown coupled to a communication bus 170, which is operably coupled to the computing device 104, such as via leads 168). The computing device (104) may be configured to operate not only the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing using the deformable transmission layer (110) when physically docked with one or more objects, such as at a contact interface (120), but also to operate the IMU (172) to capture data related to angular acceleration and axial acceleration, such as may be related to contact with an external object and / or changes in the position or orientation of the housing (118). In one embodiment, for example, the integrated system can be configured to increase the frame rate of touch sensing through the deformable transmission layer (110) when an unexpected change in axial or angular acceleration is detected using IMU data and knowledge of the predicted motion and acceleration of the housing (118). In other words, if the digital touch sensing component (146) is coupled to an electromechanical motion system, such as a robotic arm or robotic manipulator (e.g., Figure 11 234), and integrating a computing system (104) to receive information about the timing, direction / orientation, and kinematics of motion commands for the electromechanical motion system, it can be configured to separate expected accelerations from unexpected accelerations from the IMU and treat unexpected accelerations as potential contacts with external objects, which can be further explored using enhanced frame rates, computation, and general digital touch sensing via the deformable transmission layer (110).

[0059] refer to Figure 10C, an embodiment is shown in which a digital touch sensing assembly (146) is integrated with a mutually coupled capacitive sensing subsystem having a capacitive sensing controller (174) operatively coupled to a capacitive sensing element (206), for example, via leads (204). The capacitive sensing element may be integrated into a deformable transmission layer and configured to facilitate enhanced contact sensing based on capacitance sensed between the sensing element (206) and another object. The sensing element may include a grid or a plurality of cells, somewhat similar to how some smartphones or other touch screen interfaces are configured to detect contact based on detected capacitance. The capacitive sensing controller (174) may include one or more amplifiers and may be fixedly coupled to a housing (118) of the digital touch sensing assembly (146) and operatively coupled to a computing device (104), for example, via leads (190; shown coupled to a communication bus 170, which is operatively coupled to the computing device 104, for example, via leads 168). The computing device (104) can be configured to not only operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transmission layer (110) when physically docked with one or more objects, such as at a contact interface (120), but also operate the capacitive sensing controller (174) to capture data related to a change in capacitance near the sensing element (206), which may be related to, for example, contact with an external object. For example, in one embodiment, when a change in capacitance is detected using sensed capacitance data related to the sensing element (206), the integrated system can be configured to increase the frame rate of touch sensing via the deformable transmission layer (110). In other words, the system can be configured to provide optimized touch sensing output by utilizing uncorrelated errors in touch sensing based on capacitance and the deformable transmission layer (110) when at least some indication of contact is determined to be present at or near the sensing element (206). In other variations, various combinations of sensors may be utilized, such as sensors with uncorrelated errors having various aspects that are spatially separated relative to each other, as in a given implementation, the resolution and / or temporal response requirements may not be the same at each location.

[0060] refer to Figure 10D, an embodiment is shown in which a digital touch sensing assembly (146) is integrated with a mutually coupled resistive sensing subsystem having a resistive sensing controller (176) operatively coupled to a resistive sensing element (208), for example, via leads (210). The resistive sensing element may be integrated into a deformable transmission layer (110) and configured to facilitate enhanced contact sensing based on resistance sensed between the sensing element (208) and another object. The sensing element may include a grid or a plurality of cells, somewhat similar to how some smartphones or other touch screen interfaces are configured to detect contact based on detected resistance. The resistive sensing controller (176) may include one or more amplifiers and may be fixedly coupled to a housing (118) of the digital touch sensing assembly (146) and operatively coupled to a computing device (104), for example, via leads (192; shown coupled to a communication bus 170, which is operatively coupled to the computing device 104, for example, via leads 168). The computing device (104) can be configured to not only operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transmission layer (110) when physically docked with one or more objects, such as at a contact interface (120), but also operate the resistance sensing controller (176) to capture data related to a change in capacitance near the sensing element (208), which may, for example, be related to contact by an external object. For example, in one embodiment, when a change in capacitance is detected using sensed capacitance data associated with the sensing element (208), the integrated system can be configured to increase the frame rate of touch sensing via the deformable transmission layer (110). In other words, the system can be configured to provide optimized touch sensing output using uncorrelated errors in the resistance and deformable transmission layer (110) based on touch sensing when at least some indication of contact is determined to be present at or near the sensing element (208).

[0061] refer to Figure 10EAn embodiment is shown in which a digital touch sensing assembly (146) is integrated with a mutually coupled LIDAR sensor (178), such as a sensor available from Hokuyo Automatic USA Corporation. The LIDAR sensor (178) can be fixedly coupled to a housing (118) of the digital touch sensing assembly (146) and operably coupled to a computing device (104), such as via leads (194; shown coupled to a communication bus 170, which is operably coupled to the computing device 104, such as via leads 168). The computing device (104) can be configured to not only operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transmission layer (110) when physically docked with one or more objects, such as at a contact interface (120), but also operate the LIDAR sensor (178) to capture data related to objects within the field of view (212) of the LIDAR sensor (178), such as a point cloud related to nearby surfaces and objects. In one embodiment, for example, when an unexpected change within the LIDAR (178) field of view (212; which is preferably oriented to align at least to some extent with the position and orientation of the associated deformable transmission layer 110) is detected using LIDAR (178) data, the integrated system can be configured to increase the frame rate of the LIDAR (178) and touch sensing through the deformable transmission layer (110). In other words, when the deformable transmission layer (110) begins to approach another object (as detected by a change in the point cloud detected by the LIDAR (178) system), the deformable transmission layer (110) and associated computing and imaging capabilities can enter an enhanced functionality mode to detect and characterize any touch / contact.

[0062] refer to Figure 10F, an embodiment is shown in which a digital touch sensing assembly (146) is integrated with a mutually coupled strain or elongation sensor (180). The strain sensor (180) may include one or more elongation sensing elements (216), such as in a strain gauge whose resistance can be related to elongation. Such elongation sensing elements (216) may be integrated or embedded in a deformable transmission layer (110), and the strain controller (180) may be fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and operably coupled to a computing device (104), such as via leads (196; shown coupled to a communication bus 170, which is operably coupled to the computing device 104, such as via leads 168). The computing device (104) can be configured to operate not only the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing using the deformable transmission layer (110) when physically docked with one or more objects, such as at a contact interface (120), but also to operate the strain controller (180) to capture data related to strain or elongation, such as that associated with contact with an external object. The elongation sensing element can include a grid or network and can be operably coupled to the strain controller (180), such as via one or more leads (214). For example, in one embodiment, the integrated system can be configured to optimize the determination of the magnitude of the touch sensed by the deformable transmission layer (110) when a change in elongation is detected using the strain sensor data. For example, if the deformable transmission layer (110) moves over a protrusion on a surface, the magnitude of the protrusion determined using the deformable transmission layer (110) can be compared to a change in deflection of the contact surface detected using the strain sensors (180, 216), thereby providing two data sources for such determination, with at least some uncorrelated measurement / determination errors.

[0063] refer to Figure 10G, an embodiment is shown in which a digital touch sensing assembly (146) is integrated with a mutually coupled load sensor (182). The load sensor (182) may include one or more load sensing elements or cells (220), which may include, for example, one or more devices configured to generate an electrical output that varies with an applied load, such as one or more piezoelectric load cells. Such load sensing elements (220) may be integrated or embedded in the deformable transmission layer (110), and the load sensor controller (182) may be fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and operably coupled to the computing device (104), such as via leads (198; shown coupled to a communication bus 170, which is operably coupled to the computing device 104, such as via leads 168). The computing device (104) can be configured to operate not only the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing using the deformable transmission layer (110) when physically docked with one or more objects, such as at a contact interface (120), but also to operate the load sensing controller (182) to capture load-related data that may be associated with contact with an external object. The load sensing element can include a grid or network and can be operably coupled to the load sensing controller (182), such as via one or more leads (218). In one embodiment, for example, the integrated system can be configured to optimize the determination of the magnitude of a touch sensed by the deformable transmission layer (110) when a load change is detected using the load sensor data. For example, if a portion of the deformable transmission layer (110) is pressed against the surface of another object, the magnitude of the contact determined using the deformable transmission layer (110) can be compared to the load change of the contact surface detected using the load sensors (182, 220), thereby providing two data sources for such determination, with at least some uncorrelated measurement / determination errors.

[0064] refer to Figure 10H, an embodiment is shown in which a digital touch sensing assembly (146) is integrated with a temperature sensor (184) coupled thereto. The temperature sensing subsystem may include a temperature sensor controller (184), which may include, for example, an amplifier and / or a microcontroller, and one or more temperature sensing elements or units (224), which may include, for example, one or more devices configured to generate an electrical output that varies with temperature, such as one or more thermocouple elements. Such temperature sensing elements (224) may be integrated or embedded in the deformable transmission layer (110), and the temperature sensor controller (184) may be fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and operably coupled to the computing device (104), such as via leads (200; shown coupled to a communication bus 170, which is operably coupled to the computing device 104, such as via leads 168). The computing device (104) can be configured to operate not only the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing using the deformable transmission layer (110) when physically docked with one or more objects, such as at a contact interface (120), but also to operate the temperature sensing controller (184) to capture data related to one or more temperatures that may be associated with contact with an external object. The temperature sensing element (224) can include a grid or network and can be operably coupled to the temperature sensing controller (184), such as via one or more leads (222). In one embodiment, for example, the integrated system can be configured to optimize touch sensing characterization via the deformable transmission layer (110) when a temperature change is detected. For example, if a portion of the deformable transmission layer (110) is pressed against the surface of another object having a temperature different from the ambient temperature (e.g., as is likely to be the case when contacting most living tissue in a surgical environment), the magnitude of the contact determined using the deformable transmission layer (110) can be compared to the temperature change of the contact surface detected using the temperature sensor (184, 224), thereby providing two data sources related to the contact profile determination, with at least some uncorrelated measurement / determination errors.

[0065] refer to Figure 10I, an embodiment is shown in which, in addition to an imaging device (106) operably integrated with a deformable transmission layer (110), a digital touch sensing assembly (146) is integrated with an imaging sensor (186) coupled thereto. The imaging sensor (186) may include a camera and may be configured to operate at various selected wavelengths, such as visible light, infrared light, etc. The imaging sensor (186) may be fixedly coupled to a housing (118) of the digital touch sensing assembly (146) and operably coupled to a computing device (104), such as via leads (202; shown coupled to a communication bus 170, which is operably coupled to the computing device 104, such as via leads 168). The computing device (104) can be configured to operate not only the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transmission layer (110) when physically interfaced with one or more objects, such as at a contact interface (120), but also to operate the imaging sensor (186) to capture data related to objects within the field of view (226) of the imaging sensor (186), such as images related to surfaces and the vicinity of the objects. In one embodiment, for example, when an unexpected change within the field of view (226) of the imaging sensor (186) is detected using data from the imaging sensor (186), the integrated system can be configured to increase the frame rate of the imaging sensor (186) and touch sensing through the deformable transmission layer (110). In other words, when the deformable transmission layer (110) begins to approach another object, the deformable transmission layer (110) and associated computing and imaging capabilities can enter an enhanced functionality mode to detect and characterize any touch / contact, as detected by changes in image data detected by the imaging sensor (186) system. In alternative embodiments, the imaging sensor (186) can be configured to operate at infrared wavelengths to help detect, for example, heat distribution; further, the imaging sensor (186) can include a so-called "depth camera" or "time of flight" image sensor, such as those provided by PrimeSense, an Apple Inc. company, which can be configured to acquire not only image data, but also data related to the depth or z-axis position of such image data relative to the imaging sensor (186).

[0066] refer to Figures 10B-10I And refer to it again Figure 10A Various combinations and permutations of these illustrated sensing configurations may be integrated together in various embodiments. For example, in one embodiment, it may be desirable to have IMU sensor capabilities as well as LIDAR to supplement digital touch sensing via the deformable transmission layer (110). Various examples and embodiments are described below.

[0067] refer to Figure 11 , a configuration is shown employing a digital touch sensing assembly (146) coupled to a distal portion (236) of a robotic arm or robotic manipulator (234) mounted to a movable base (238). The robotic manipulator may include an elongated arm structure including various movable joints between rigid or semi-rigid links, as shown (234), or may include a flexible robotic manipulator, such as those that may be referred to as robotic catheters or tubular flexible robots (e.g., available from Intuitive Surgical, Inc. or Johnson & Johnson, Inc.). The digital touch sensing assembly (146) is shown as being operably coupled, such as via a wired or wireless connection (232, 230, 166), to a computing device (144), which is coupled (136) to a power source (102). The robotic arm (234) is operable by the computing system (144) to advance to and inspect an object (228) having a surface of interest (70), which may include components such as rivets (72) that are prone to failure or require periodic inspection.

[0068] refer to Figure 12 By utilizing various aspects of the foregoing configuration, the digital touch sensing assembly (146) can be used to inspect the surface (70) and the features (72) through controlled interfacing with the interface surface (120). In other words, as described above and as Figure 12 As further shown, in addition to digital touch sensing via the deformable transmission layer (240), various other sensing configurations and related data may also be used, including but not limited to IMU data (242), capacitive sensor data (244), resistive sensor data (246), LIDAR / point cloud data (248), strain or elongation sensor data (250), load sensor data (252), temperature sensor data (254), and data from additional imaging devices (256).

[0069] refer to Figure 13A , showing the Figure 11 A similar system configuration is provided with the addition of a sensing capability coupled to a connected (258, 230, 166, such as via a wired or wireless connection to the computing system 144) room or operating environment (260), and additional sensing capability coupled to the digital touch sensing assembly (146). Figure 13AAs shown, one mounting member (359) is configured to couple an additional imaging device (270) to the digital touch sensing assembly (146) at a position and orientation capable of capturing a field of view associated with an area in front of an interface surface (120) of the digital touch sensing assembly (146); another mounting member (358) is configured to couple another additional imaging device (272) to the digital touch sensing assembly (146) at a position and orientation capable of capturing a different perspective field of view associated with an area in front of the interface surface (120) of the digital touch sensing assembly (146); and further, a LIDAR device (274) is coupled to the second mounting member (358) at a position and orientation capable of assisting in capturing a point cloud and other data associated with the operating environment surrounding the digital touch sensing assembly (146). As described above, in this embodiment, the connected room (260) also has enhanced sensing capabilities, wherein a plurality of imaging devices (264, 266) and an additional LIDAR sensor (268) are coupled to the room (260) in positions and orientations selected to assist in accurately analyzing the operation of the robot (234) relative to the inspected object (228) as the object is located on the table (262) in the room (260).

[0070] refer to Figure 13B , further enhancements may be included and coupled (318) to each other on the computing device side of the system to allow a user operating the computing system (144) to remotely understand various aspects of the surface (70) of the object (228) being inspected by the digital touch sensing assembly (146). Figure 13B As shown, the display (278) can be used to help the associated user view the output from the digital touch sensing component (146), as well as images or point clouds from other interconnected sensing subsystems (270, 272, 274, 268, 264, 266). In addition, the haptic interface (280) (such as Figures 13C-13F The interconnected 3-D printer (276) can also be used to supplement this "touch sensing workstation" so that the user can decide to directly experience several layers of the detected geometry by printing the geometry locally for direct manipulation (e.g., by the user's hand).

[0071] refer to Figure 13C The tactile interface variant (282) can be configured to be coupled to a computing system (not shown) and provide a user with the sensation of experiencing a real or virtual surface through a manipulation interface (e.g., a spherical member (290)) configured to be grasped by the user's hand. Figure 13D A tactile interface variant (284) is shown that is configured to provide a user (4) with a hand (12) grasping manipulation interface (292) to experience various aspects of a real or virtual surface through a mutually coupled computing system (not shown). Figure 13E and13F Further tactile interface variations (286, 288) are shown where the hand (12) of the user (4) may be able to experience various aspects of a real or virtual surface through a pen-like (294) manipulation interface or a fingertip (296) manipulation interface. Figure 13B In one of the "touch workstation" configurations and illustrated tactile interfaces, a user from a nearby or remote location can observe (via display 278), directly feel / manipulate (via 3-D printer 276), and tactilely experience (via tactile interface 280) various aspects of the surface (70) of the object (228) being inspected. Thus, reference Figure 14 and 15 , showing various aspects of variations of such a configuration.

[0072] refer to Figure 14 , a user desires to engage a surface using the sensing system; the system is calibrated and positioned near a target surface (302). The user navigates the sensing surface to the target surface via an electromechanical arm or robotic manipulator, and feedback is provided to the user regarding the position and orientation of the sensing surface by a positioning platform (e.g., inverse kinematics, load cells, deflection sensors, joint positions) (304). As the sensing surface is navigated closer to the target surface, the integrated sensing capabilities assist in detecting the target surface and its features (e.g., the system can be configured such that the integrated camera and LIDAR detect the target surface first, followed by other integrated sensing capabilities that can be configured to sense content associated with closer engagement) (306). The system can be configured to specifically create a contact event between the sensing surface and the target surface (e.g., repositioning and reorientation of the sensing surface can be slowed, and contact can be communicated using audio, visual, and / or tactile cues) (308). The user can use integrated sensing capabilities (e.g., acceleration detected by an IMU, capacitive touch sensing, resistive touch sensing, LIDAR, strain or deflectometers, load sensing, temperature sensing, and / or cameras and other imaging devices) to reposition and reorient the sensing surface relative to the target surface to inspect the target surface (310). The system can be configured to present various aspects of the target surface to the user so that the user can better understand the target surface, such as through a combination of visual, tactile, auditory, and haptic sensations (e.g., through a locally printed surface or portion thereof) (312).

[0073] refer to Figure 15, a user, who is remote from a target surface, desires to engage the target surface using the sensing system; the system is calibrated and positioned near the target surface (314). The user navigates the sensing surface to the target surface, such as via an electromechanical arm or robotic manipulator, and feedback is provided to the user regarding the position and orientation of the sensing surface provided by a positioning platform (e.g., inverse kinematics, load cells, deflection sensors, joint positions) (304). As the sensing surface is navigated closer to the target surface, the integrated sensing capabilities assist in detecting the target surface and its features (e.g., the system can be configured such that the integrated camera and LIDAR first detect the target surface, followed by other integrated sensing capabilities that can be configured to sense content associated with closer engagement) (306). The system can be configured to specifically create a contact event between the sensing surface and the target surface (e.g., repositioning and reorientation of the sensing surface can be slowed, and contact can be communicated using audio, visual, and / or tactile cues) (308). The user can use integrated sensing capabilities (e.g., acceleration detected by an IMU, capacitive touch sensing, resistive touch sensing, LIDAR, strain or deflectometers, load sensing, temperature sensing, and / or cameras and other imaging devices) to reposition and reorient the sensing surface relative to the target surface to inspect the target surface (310). The system can be configured to present various aspects of the target surface to the remote user so that the user will have an enhanced understanding of the target surface, such as through a combination of visual, tactile, auditory, and haptic perception (e.g., through a locally printed surface or portion thereof) (316).

[0074] refer to Figures 16A-17 , showing various aspects of another exemplary configuration utilizing the integrated touch sensing system described herein. Figure 16A , showing an interconnected room, kiosk or measurement enclosure (324; connected to a computing system 144 via a wired or wireless connection 320, 230, 166, which, as described above, is integrated with and mutually coupled to other aspects of the touch workstation, such as a power supply 102, a 3D printer 276, a display 278 and / or a tactile interface 280), having several interconnected imaging, sensing and detection resources, such as a LIDAR device (286), one or more imaging devices (264, 266) and a digital touch sensing assembly (146), which are mutually coupled to other imaging devices (270, 272) and a LIDAR detector (274), each of which can be configured to help characterize the geometry and surface of an object, such as a foot (322) of a person (4), which can be lowered (326) into a position where the foot engages the digital touch sensing assembly (146), as Figure 16BAs shown. In other words, the measurement housing or information kiosk (324) can be configured to facilitate convenient engagement with a portion of a user's appendage, such as a portion of a user's leg or arm, to collect precise information related to a target such as the plantar aspect of the user's foot, which can be used to design orthoses, ski boots, etc. The combined data available at the interconnected workstations can be used not only to examine the subject object (e.g., the user's foot), but also to accurately characterize its geometry. For example, the digital touch sensing assembly (146) can be used to accurately characterize the primary load-bearing surface (i.e., the bottom surface of the user's 4 foot 322), and the image and point cloud data can be used to further understand the geometry of the object (the user's 4 foot and lower leg), so that these findings can be used to assist in orthopedic research, pre- or post-operative surgical studies, custom shoe design, etc. Figure 17 One such configuration is shown in .

[0075] refer to Figure 17 In one embodiment, it is desired to better understand the geometry and load patterns of a foot for a particular user (330). The user may expose their foot and the system may be initialized in preparation for characterization (332). The user may position / orient their foot within the measurement structure to facilitate scanning of the external geometry of the exposed foot (334). The user may reposition / reorient their foot within the measurement structure to further scan the external geometry of the exposed foot (336). The user may place the foot on the deformable transmission layer and place a load on the foot while the system collects data related to the load patterns, anatomy, and geometry (338). The system may be configured to create an anatomical / geometric profile of the user's foot and a load profile related to the anatomical / geometric profile (340). The anatomical / geometric profile and the load profile may be used to create a docking structure (e.g., a shoe, a ski boot, an orthosis) and / or diagnose a relevant medical condition (342).

[0076] Reference again Figure 13A and 13B , some surfaces and objects may be presented in a certain easily accessible configuration. Many other fine manipulation and / or contact scenarios involve greater geometric or spatial complexity. See, for example, Figure 18A, shows a relatively simple scenario for a human (346), in which the hand (348) of the human (346) can be used to controllably approach and then touch, inspect and / or grasp a target object, such as a cookie (354), which happens to be located within a container (344) that may be fragile, and therefore relatively high load or impact contact should be avoided to maintain the integrity of the container (344) and / or the object (here, the cookie 354, which may also be fragile). The supporting structure or substrate (such as table 352) on the container (344) may also be fragile or susceptible to damage under high loads or high impact. The human upper limb happens to be very dexterous and can successfully handle this example situation, in part due to the smooth motor neurons, smooth muscle and motor activity, and sensory neuron innervation of tissues such as the skin in the upper limb. For example, the illustrated human (346) would typically have sensory neurons throughout the skin, such as in the wrist (350) and hand (348) regions, so that the associated human (346) can carefully navigate the geometry of the container and target object (354) and the mechanical failure mechanisms associated with both. In other words, the human can use touch sensing of the skin and other tissues to navigate the scene without damaging the associated structures. Using a mechanical system (such as a backhoe tractor (in the scaled-up version of the scene) or a remotely controlled robot) to process the same scene presents many challenges because the person controlling it at a remote location (e.g., across the room from the robot, or across the country from the robot connected via computational connectivity) typically does not have human-level sensing or touch or feeling associated with the interaction and may not perceive that one or more of the associated structures is about to be damaged until it is too late, such as through visual or audio confirmation.

[0077] refer to Figure 18A and 18B ,body touch sensing technology can be used to address such scenarios and allow users, either nearby or in remote locations, to better feel the physical engagement involved.

[0078] like Figure 18BAs shown, an electromechanically controllable robotic arm (234) is located in a room (260) having inter-coupled touch sensing assemblies (146) (e.g., the assemblies described above) positioned to inspect an object (e.g., a cookie 354) within a container (e.g., a jar 344) placed on a substrate or support structure (e.g., a table 352). The room (260) can be configured to have multiple sensors, such as a LIDAR (268) and one or more image capture devices (264, 266), coupled to the room and positioned to capture information about the volume surrounding the robot and / or the target object (354), preferably in a manner that provides high-quality data from multiple sources with uncorrelated errors, as described above. One or more additional sensing devices (e.g., additional image capture devices (270) and LIDAR (274)) can be coupled to the robotic arm (234) to provide further information about the volume surrounding the inter-coupled touch sensing assemblies (146), as well as further high-quality data from multiple sources with uncorrelated errors to enhance data fusion capabilities. Each sensor (146, 264, 266, 268, 270, 274) can be coupled (232, 258, 230) to one or more computing devices (104), such as via a wired or wireless connection, which can be configured to facilitate control interactions. Utilizing such a configuration, the remote and target-oriented touch sensing assembly (146) can be configured to assist a user, who may be located nearby or at a remote location, in gaining awareness of physical interactions at the deformable transmission layer (110) of the touch sensing assembly (146), as described above. Furthermore, as described above with reference to Figure 13B As described, a user may be equipped with a workstation that provides one or more ways to sense physical engagement, such as a tactile interface (280), a display (278), and / or a 3D printer (276, i.e., to facilitate printing one or more layers of a subject object). To further enhance the user's perception of the physical engagement scenario of a remotely operable manipulation or inspection configuration (e.g., a robot 234 as shown), an additional touch sensing component (360) may be coupled to the remotely controllable engagement system (234), for example, in a peripheral configuration partially or entirely surrounding a distal portion of the system, as shown. In other words, the additional touch sensing component (360) may include components similar to the above-described touch sensing component (146) and be peripherally coupled around a portion of the associated structure in a manner that provides one or more outwardly facing deformable transmission layers (110) so that the one or more outwardly facing deformable transmission layers (110) are operatively coupled (232, 230) to the computing device (104) via a wired or wireless connection, thereby providing additional touch sensing to the user of the remote workstation. As Figure 18BAs shown, the additional touch sensing component (360) is preferably located on the remotely controllable engagement system (234) at a location that will assist the remote user in understanding key aspects of remote engagement, such as at the distal or "wrist" location where contact with a target or related object may occur. For example, positioning the additional touch sensing component (360) peripherally around at least a portion of the distal touch sensing component (146) may be helpful in assisting the remote user in navigating through the mouth of the container (344) and down to the target object (354), as brushing or more direct contact with either sensing component (360, 146) may occur during such proximity.

[0079] refer to Figure 18C , showing the Figure 18B A similar configuration is provided with the addition of another touch sensing component (362) that is circumferentially coupled around at least a portion of what may be referred to as a "forearm" member of the illustrated robot (234), and again operatively coupled (232, 230) to the computing system (104), for example, via a wired or wireless connection. In practice, both touch sensing components (360, 362) may be configured for circumferential sensing around the elongated component (234), for example, via a diametrically opposed pair of touch sensing components (146), a group of three or more touch sensing components that may be separated from one another, for example, in a circumferentially equally spaced configuration (i.e., to maximize coverage relative to the surrounding environment), etc. Such additional sensing capabilities at the illustrated location may further assist a remote user in successfully navigating the illustrated physical engagement challenge to touch, inspect, and / or grasp a target object (here, a dollar bill 355).

[0080] As mentioned above Figure 9A and 9B As described above, various sensor configurations can be created by assembling and operably coupling multiple touch sensing components (146), and such mutual coupling can be used to create various sensor configurations such as Figure 18B and 18C (360, 362) shows a peripheral or partial peripheral type of touch sensing component. As described above, for example, reference Figures 7A-7E Components such as optical fibers and / or waveguides can be used to move the sensor to various positions relative to the emitted or captured radiation (e.g., captured light) (i.e., rather than positioning the optical sensor or image capture device directly at the capture location, waveguides, transmission fibers, or a combination or multiples thereof can be used to capture light at the capture location to facilitate transmission from such capture location to a more remotely located optical sensor or image capture device). Figures 18D-18K , showing various configurations that provide alternatives for radiative transmission associated with touch sensing components, such as the components (146, 360, 362) described above. Figure 18D, for example, shows that Figure 7A , comprising an optical element (108) operably coupled to a light (or other wavelength radiation; e.g., alternatively, an infrared wavelength) emitting device (116), the configuration of which is selected to cause photons to propagate (364) from the emission of the light emitting device (116) to various locations on the optical element (108), where the photons can pass through the deformable transmission layer (110), e.g., with an exit angle (366) dictated by the reflective / refractive properties of the material and the geometry of the structure, e.g., between about 20 degrees and about 40 degrees. Figure 18E Shown with Figure 7A The components in the same configuration are configured with light emitted from both sides (116, 122). Return to Reference Figure 7A The image capture device is sized within the confines of a three-dimensional cube having an edge dimension of approximately 1.5 mm, a distance to the imaged object of approximately 3 mm, and a working distance of approximately 5 mm, in combination with an optical element (108) comprising a material such as a polymer or glass selected to facilitate illumination therethrough, such as polymethyl methacrylate ("PMMA"), which is relatively inexpensive, easy to form, and relatively easy to polish to facilitate optical properties such as predictable reflectivity, in a layer (368) having a thickness of approximately 4 mm and a deformable transmission layer (110) of approximately 1-2 mm of polymer material, the thickness of the assembly can be in the range of 1-15 mm, with these dimensions being selected at least in part based on illumination requirements and field load requirements. Such assembly dimensions are feasible in various configurations, but can be minimized by alternative configurations.

[0081] refer to Figure 18F For example, certain so-called "front-lit" or "front-illuminated" films (372), such as those used in computing device displays (e.g., in mobile devices that can be used outdoors or in other brightly lit environments where traditional backlighting configurations may not be as effective; for example, devices such as those sold under the trade name Kindle (RTM) may utilize a reflective display configuration selected to use ambient light, such that the illumination layer is located between the pixels of the display and the viewer), may include light extraction features to controllably extract light or other radiation in preferred directions, such as toward or away from the deformable transmission layer (110; i.e., light may bounce 902, such as by total internal reflection, through the illumination film 372 and exit 904 from the film and into the deformable transmission layer 110, which may serve as a carrier and spacer for various optical layers to allow sufficient spacing perpendicular to the plane of the deformable transmission layer, i.e., "z-axis spacing," for mixing light), such as Figure 18FAs shown, at a desired location or distribution along the length of the optical element (108), at a desired exit angle (366), and the thickness (370) can be in the range of 100 microns. A cladding (not shown), such as a cladding comprising a silicone material, can be coupled to the outer surface of the membrane (372), and the carrier layers can also be coupled to each other to provide additional structure and local planarity. With such a configuration, the component thickness can be reduced by about half, for example to about 5-6 mm, depending on the material and light extraction characteristics of the membrane (372). For example, Figure 18F In the configuration shown, certain portions (900) of the deformable transmission layer (110) may be difficult to access due to the positioning and exit paths / angles (904, 906) of the illumination layer (372). Figure 18G Another embodiment is shown in which the membrane 372 is located between the optical element 108 and the deformable transmission layer 110, and therefore closer to the deformable transmission layer (110), such as in various so-called "front-illuminated" configurations. Figure 18F As with the configuration of FIG, features within the lighting layer can assist in controlled bounce / reflection 902 (e.g., by total internal reflection) and exit or extraction 904 to direct the illumination to other layers, such as the deformable transmission layer (110) as shown. The thickness (370) of the illumination film (372) can be determined by factors related to the lighting requirements, such as the need for highly tightly controlled illumination (e.g., more light may require a thicker illumination film; tighter angular control may require a thinner illumination film). Importantly, such a layer can be substantially planar, but can also be non-planar or have various degrees of curvature (convex, concave, cylindrical, etc.), can also be illuminated from various positions, and can be elongated, such as Figure 18H and 18I As shown, this can, for example, facilitate Figure 18B and 18C The peripheral geometry shown in the cuff-shaped peripheral sensor (360, 362). In addition, as Figure 18J As shown, such a film (372) can be coupled not only to a single side for controlled reflection, but also to multiple sides; Figure 18J A configuration is shown wherein controlled reflective front lighting films are mutually coupled to four sides (372, 374, 376, 378) around the optical element (108) as shown, or in other embodiments, in a manner similar to Figure 18I Up to six sides in a configuration where two additional lighting films are coupled to either side of the optical element (108) in a coplanar manner with the drawing as shown.

[0082] refer to Figure 18K and 18L As described above, waveguides and transmission or mutual coupling structures can be used to efficiently move light between various elements. For example, Figure 18KA wedge-shaped waveguide is shown, whose maximum thickness (380) can be in the range of 1-5 mm and whose included angle (384) can be in the range of 1-15 degrees to help propagate (388) light from the transmitting device (116) through the waveguide (392) into the optical element (108) and into the deformable transmission layer (110); an air gap (908) can be configured to facilitate transmission from the waveguide (392) into the optical element (108). Figure 18L A similar wedge-shaped waveguide is shown, whose maximum thickness (382) may be in the range of 1-2 mm and whose included angle (386) may be in the range of 2-8 degrees to help propagate (390) light from the transmitting device (116) through the waveguide (394) (another air gap 909 is shown to assist in propagation and prevent total internal reflection) and directly into the deformable transmission layer (110). Figure 18K In a configuration such as that shown in FIG. 1 , a film (not shown) may be disposed on the rightmost depicted surface of the deformable transmission layer (110), and additional capture devices or cameras and additional illumination sources may be added to the opposite side of the waveguide (394) (shown on the left), as long as that opposite side does not have a specular reflective coating. Specular coatings and so-called "turning film" elements may be included to further assist in efficiently guiding and transmitting light or other radiation between elements (e.g., light exiting the illustrated waveguide 392 may be at an exit vector that is nearly parallel to a perpendicular plane of the waveguide 392, and it may be desirable to "turn" the exiting light to produce a desired illumination angle, such as by coupling a turning film to the waveguide 392). The components, materials, geometries, and refractive / reflective properties may be tailored to a variety of specific geometric challenges, such as those presented in the various use cases described and illustrated herein.

[0083] As mentioned above, providing users with awareness of activities at a remote location through their local workstation, whether the user is across the room, in another building, or across the world, is a key challenge in many computerized systems (e.g., telecommunications, telepresence, remote inspection, or remotely operated systems). Figure 19A One way to enhance the user's (4) perception at a local workstation is with the aid of a tactile master input device (280) that can be operatively coupled (396, 230) to an interconnected computer system (104), for example, via a wired or wireless connection, to enable the user (4) to perceive various aspects of sensation, such as simulated transformations of contact, friction, texture, etc., at the local workstation via the user's hand (12) and / or wrist (13). Reference Figure 19BIn another embodiment, it may be valuable to facilitate further local perception of remote physical interactions with the aid of a so-called "touch conversion interface" (398), for example, the touch conversion interface can be removably coupled to the user's wrist (13), operably coupled to the computing system (400, 230), such as via wired or wireless communication, and configured to provide the user (4) with one or more sensations at the wrist (13) or other location that are related to and / or can be intuitively associated with activities at the remote location, such as contact between objects at the remote location. Such sensations can be provided in addition to sensations provided to the user (4) via, for example, a tactile primary input device or controller (280). In other words, in various embodiments, multimodal sensations can be provided to the user (4) to help the user perceive activities at the remote location with enhanced fidelity.

[0084] refer to Figures 20A-20C , various aspects of a road vehicle (e.g., a computerized electric vehicle) provide opportunities for touch integration and enhancement. For example, typically, a human operator will have a fairly consistent touch interface with the pedals (404, 406), the floor (414), the driver's seat (412), the steering wheel (408), various aspects of the instrument panel control and / or display interface (410), and various portions of the vehicle structure (e.g., portions of what may be referred to as "A-pillars" (402)). Each of these structures, as well as others, provides opportunities for integrating touch sensing to, for example, assist with operation, control, and safety. See, for example, Figure 20B and 20C The touch sensing assembly with a deformable transmission layer can be operably coupled to various aspects of the front (438, 440, 442) and rear (444, 446, 448) vehicle bumper or frame structure to assist in detecting impact-related deformation and can be used to trigger safety systems, such as seatbelt pretensioners or passenger airbags, in addition to or as an alternative to other more conventional sensors configured to provide such functionality, such as embedded accelerometers, which may introduce more latency into the control of such safety systems than the touch sensing assembly with a deformable transmission layer. In other words, the placement of the touch sensing assembly with a deformable transmission layer can be selected to provide intrusion detection very early in the event of an intrusion, perhaps before certain acceleration detection systems detect an operative change in acceleration, such as at certain frame components. Figure 20BVarious locations and positions within a vehicle interior are shown that are operatively coupled to a touch sensing assembly with a deformable transmission layer so that a central controller or computing system can detect a user's touch and / or contact through the touch sensors, the touch sensors being operatively coupled to each of pedals (416, 418), a driver's floorboard (420), a driver's seat base (422), a driver's seat back (424), a driver's headrest (426), a shifter interface (430), a center console interface (428), a steering wheel (432), an instrument panel portion (434), and a portion (436) of an A-pillar (402) structure. The touch sensing assembly with a deformable transmission layer in each of these exemplary structures can have different geometries and include various materials to provide structural properties tailored for each usage scenario. For example, the structural modulus of a seat base (422) touch sensor is typically relatively low, and the resolution of the information sought is relatively low (e.g., without particularly high resolution, a general weight profile of the operator to help determine that a child or dog below a certain weight is not attempting to operate the vehicle); this can be compared to a center console (428) interface, where the structural modulus can be selected to be relatively high so that the operator can repeatedly control various aspects of the vehicle through the touch interface without significant physical intrusion with typical touch loads, while also providing sufficient intrusion with such typical touch loads to obtain the desired information, such as general fingerprint geometry correlation that can be analyzed when starting the vehicle to obtain a layer of biometric security related to the authorized user / operator.

[0085] One of the challenges of integrating multiple touch sensing components with a deformable transmission layer into a system such as an automobile or a robot is interconnectivity. Figure 21A For example, as described above, various aspects of the control, signal, power, and / or drive connections (232, 230) between a system such as a robot (234) having a touch sensing component (146) and a computing system (144) can be implemented via hardwired leads or wireless connections, such as via Bluetooth (RTM), IEEE 802.11, or various other standards. In fact, reference Figure 21B It may be desirable that at least some components or aspects of the system (e.g., the robot (234) having the touch sensing component (146)) be in a relatively disconnected form, such that Figure 21C and 21DAs shown in the enlarged view in FIG, a wireless transceiver (166) can be used to communicate with most, if not all, of the other coupled systems, while power and some level of controller and / or computing power can be provided by an onboard computing device (144) and power system (102), such as an embedded chipset, microcontroller, field programmable gate array, application specific integrated circuit, etc., and a rechargeable battery, such as by wireless induction. This integration and the general preference for tetherless configurations can be referred to as a variant of the "Internet of Things" and can play a role in many system integration challenges. For example, see Figure 22 , similar to Figure 21C The wirelessly connected touch sensing assembly (146) shown can be integrated into a door lock system configuration, wherein a person's thumb (452) or other finger can be used to engage the deformable transmission layer to provide biometric authentication / lock access functionality to facilitate unlocking. The touch sensing assembly (146) can be wirelessly connected to one or more computing systems within an associated building, for example, and / or one or more computing systems that can be mobile, resident in a data center, etc.

[0086] refer to Figure 23A and 23B , a variation of the handheld surface (460) analysis tool has a wirelessly connected touch sensing component (146), such as Figure 21C As shown, the housing (458) can be configured to engage with a user's hand (462) to facilitate engagement of the deformable transmission layer and associated interface surface (120) with a surface (460) of a target object for surface analysis. The touch sensing assembly (146) can be wirelessly connected to one or more computing systems within, for example, an associated building, and / or one or more computing systems that are mobile, reside in a data center, etc.; the handheld assembly can contain its own power source, such as a battery, for operational purposes.

[0087] refer to Figure 24A , shows a touch sensor integrated vehicle configuration in which the touch sensing assembly is operably coupled to various structures, such as an elongated touch sensor (436) coupled to the vehicle's A-pillar (402), touch sensors (416, 418) coupled to pedals, a touch sensor (420) coupled to the driver's floor, a touch sensor (428) coupled to the center console, a touch sensor (422) coupled to the base of the driver's seat (412), a touch sensor (424) coupled to the driver's seat back, a touch sensor (426) coupled to the driver's headrest, a touch sensor (430) coupled to a shift member, a touch sensor (432) coupled to a steering wheel, and a touch sensor (434) coupled to a portion of the vehicle's instrument panel, which are connected to a central computing system (144) via a lead-type connection (464).

[0088] refer to Figure 24B , sensors in similar locations have wireless connections to the transceiver (166) of the central computing system (144), the sensors can help simplify such integration by removing the need for certain connecting wires, and in variations where the sensors are operably coupled to a small power source (such as a battery), the need for power wires can also be eliminated, for example by wireless inductive recharge. Thus, the A-pillar touch sensor (436) is shown as being operably coupled to the wireless transceiver (466); the pedal touch sensors (416, 418) are shown as being operably coupled to the wireless transceivers (472, 470, respectively); the floor touch sensor (420) is shown as being operably coupled to the wireless transceiver (474); the seat base touch sensor (422) is shown as being operably coupled to the wireless transceiver (476); the seat back touch sensor (422) is shown as being operably coupled to the wireless transceiver (478); the headrest touch sensor (426) is shown as being operably coupled to the wireless transceiver (479); is shown as being operably coupled to a wireless transceiver; a shifter assembly touch sensor (430) is shown as being operably coupled to a wireless transceiver (486); a center console touch sensor (428) is shown as being operably coupled to a wireless transceiver (484); a steering wheel touch sensor (432) is shown as being operably coupled to a wireless transceiver (482); and an instrument panel (410) touch sensor (436) is shown as being operably coupled to a wireless transceiver (466), each of said touch sensors being wirelessly connected (166) to a central computing system (144) of the vehicle.

[0089] Review such as Figure 19A Configurations, aspects of touch sensing can be used to improve and / or enhance the user's perception of certain operations on a local workstation, and the value of having multiple sources of sensing data (e.g., with uncorrelated error configurations to enable so-called "sensor fusion" applications) has been discussed. Figure 25A In one embodiment, a system having multiple sensing configurations (e.g., multiple sensing configurations having uncorrelated error sources) is initialized for use in a first location (488). The system can be configured to provide information related to system operation to an operator via a user interface (490). Based on one or more commands input by the operator, the system can be configured to execute and provide feedback to the operator via the user interface, the user interface being based at least in part on the multiple sensing configurations (492). The system can be configured to optimize operation and feedback via sensor fusion techniques configured to exploit differences in information provided by the multiple sensing configurations (494).

[0090] refer to Figure 25B , with reference to a system including an electromechanical arm or manipulator (e.g., with reference to Figures 21A-21DThe system is described in conjunction with a method of controlling a robot manipulator system having a plurality of sensing configurations (e.g., capacitive, resistive, radar, LIDAR, camera, load cell, strain or elongation sensor, IMU, and / or joint position sensor configurations, and touch sensing based on a deformable transmission layer with uncorrelated error sources), the robot manipulator system being initialized for use in a first position (496). The system can be configured to provide information related to the operation of the system to an operator via a user interface (498). Based on one or more commands input by the operator, the robot manipulator system is used to perform a task (e.g., picking up an object from a tank), and the system is configured to perform and provide feedback to the operator via a user interface based at least in part on the plurality of sensing configurations (500). The system can be configured to optimize operation and feedback through sensor fusion techniques that are configured to exploit differences in information provided by multiple sensing configurations (e.g., as the distal portion of the robotic manipulator system is navigated into the opening of a jar, certain sensors comprising the multiple sensing configurations may become obscured or temporarily become less reliable, while at the same time preferably at least one other sensing configuration of the multiple sensing configurations has at least slightly uncorrelated errors, such as touch sensing based on a deformable transmission layer, to provide reliable information back to the system and operator) (502).

[0091] Reference again Figure 19B , integration of one or more touch conversion interfaces (398), for example at the wrist (13) of the user (4), can provide enhanced awareness of activity and engagement at a remote location. Figure 26 A configuration is shown in which an operator interface (506) local to a user or operator may have a computing system (144) coupled (318) to each of a tactile interface (280), a display system (278), a 3-D printer (276), and a touch conversion interface (504). The operator interface (506) local to the user is typically coupled to a remote manipulation system (such as a Figure 26 As shown, the robotic arm 234 having the touch sensing assembly 146 is spaced (640) apart, such as a few inches, a few feet, a few miles, or a few thousand miles apart, depending on the user configuration, the task at hand, and the connection (230, 166) alternatives, such as wired or wireless connections. Figure 27, further illustratively, the operator interface (506) may include interconnected (400) computing (144), a primary input device / controller (such as the haptic-enabled variant shown in FIG. 280), 3D printing (276) and display (278) resources, and a touch conversion interface (398), such as the variant shown, which may be removably coupled to the wrist (13) of the user (4) and configured to provide one or more sensory components that may be perceptually linked to activity at a remote location, as described in further detail below.

[0092] In various embodiments having one or more touch conversion interfaces at the operator interface (506), it may be desirable to position the one or more touch conversion interfaces at locations relative to the anatomy of the user (4) that have a kinematic relevance to the movement of the component at the remote manipulation or actuation location. For example, see Figure 28A In embodiments where the robotic arm (234) is to be operated at a remote site, wherein the robotic arm (234) has kinematics that are at least somewhat similar to a "wrist," the touch sensing assembly (362) can be functionally coupled to a touch conversion interface that can be removably coupled to the wrist (13) of the user (4) at a mutually coupled operator interface (503). In other words, if touch / contact sensed at the robot's "wrist" is converted to the user's wrist, the level of intuitive interaction between the local user / operator from the operator interface (503) and the remote robotic manipulator can be enhanced. Thus, in various embodiments, attempts can be made to provide at least somewhat kinematically similar pairings between remote and local touch sensing and conversion resources.

[0093] Reference again Figure 28A It should also be emphasized that for a given embodiment, more than one touch sensing assembly may be integrated, for example, an additional at least partially peripheral touch sensing assembly (360) located around the distal end of the robotic arm (234) at a location around the sides of the touch sensor (146) and coupled to the computing resource (232) along with another more proximal touch sensing assembly (362). Figure 28B In a kinematically similar functional pairing configuration, a more distal touch conversion interface (508), such as a finger-sized cuff removably coupled to an index finger, can be operatively coupled (510) (e.g., via a wired or wireless connection) to a computing system and configured to convert a touch or contact sensed at a more distal touch sensing assembly (360) located at Figure 28AThe robotic arm (234) is shown at a remote location; the more proximal touch sensing assembly (398) can be removably coupled to the forearm or wrist (13) of the user (4) and operatively coupled (400) to the computing system, for example, via a wired or wireless connection, and is configured to convert a touch or contact sensed at the more proximal touch sensing assembly (362) located at Figure 28A Around the "wrist" of the robotic arm (234) at the remote location shown.

[0094] refer to Figure 29A , shows a gripper (518) type end effector having two opposing movable members (520, 522) that can be controllably advanced toward each other to perform a grip. In various embodiments, touch sensing components can be integrated into these opposing movable members (520, 522) and operably coupled thereto to assist in sensing actions associated therewith. Reference Figure 29B , the primary input device configuration (516) is configured to allow two opposing fingers of a user's hand (12) to remotely control a grasping action in a manner that is at least partially kinematically similar, e.g. Figure 29A The action of the gripper shown in FIG. 5 (ie, by moving opposing fingers toward each other, opposing movable members 520 , 522 can be moved toward each other).

[0095] refer to Figure 29C and 29D , a plurality of detachably coupled touch conversion interfaces (508, 512) may be operatively coupled (510, 514, respectively) to a computing system, such as via a wired or wireless connection, which may be operatively coupled to a remote instrument, such as Figure 29A , so as to provide enhanced intuitiveness for the user or operator (also, by moving opposing fingers toward each other, the opposing movable members 520, 522 can be moved toward each other, and the touch / contact information detected by the touch sensing components at the opposing movable members 520, 522 can be used as input to create a sensation for the user at the touch conversion interface 508, 512). Figure 29C An embodiment is shown in which the touch conversion interface (508, 512) is removably coupled to the index finger (526) and middle finger (528) of the user, while Figure 29D An embodiment is shown in which the touch conversion interface (508, 512) is removably coupled to the user's index finger (526) and thumb (524).

[0096] refer to Figure 30A, shows a touch conversion interface (398) detachably coupled to a user (4), which is operably coupled to a computing system (144), for example, via a wired or wireless connection (400, 230, 166). The touch conversion interface (398) can include a single touch conversion element or a plurality (530) of touch conversion elements, as shown, to help provide the user (4) with an enhanced perception of touch and / or contact with the interconnected touch sensing components. Reference Figures 30B-33B In various embodiments, various types, combinations, and arrangements of touch conversion elements may be used. Figure 30B , an unbalanced motor (532) can be used as a touch conversion element to provide vibration and frequency-variable touch conversion. Figure 30C , a light emitting diode ("LED") (534) may be used as a touch conversion element to provide a visual conversion to the user that contact or touch has occurred; the luminance output may vary depending on the magnitude of the touch or contact load, and various colors / wavelengths may be used. Reference Figure 30D , a piezoelectric component (536) can be used as a touch conversion element to provide a relatively high frequency vibration response according to contact or touch, and the frequency and / or intensity can be varied according to the magnitude of the touch or contact load. Figure 30E , the audio speaker assembly (538) can be used as a touch conversion element to provide an audible response based on contact or touch, and the frequency and / or intensity can vary based on the size of the touch or contact load. Figure 30F and 30G One or more so-called "shape memory alloy" ("SMA") segments (540) may be used as touch conversion elements, including alloy materials such as nickel / titanium. For example, Figure 30G As shown in the graph (544) of FIG. , commercially available SMA alloys can be configured to shrink quite significantly in size (e.g., when Figure 30F ; 542 when heated by the current passing through the circuit, contracts to 1 / 2 of its cold length), and can therefore be used to controllably apply and / or relax slight hoop stress and / or hoop strain when formed into a ring or cuff-like configuration, such as Figure 32A and 32B The variant shown in .

[0097] Therefore, reference Figure 31A A touch conversion interface (398) operably coupled (400) to a computing system, such as via a wired or wireless communication configuration, may be removably coupled to a user (4), such as at a wrist (13), and may include a controllably actuatable tactile actuator motor, such as an unbalanced motor (532). Figure 31BA touch-sensitive interface (398) operably coupled (400) to a computing system, such as via a wired or wireless communication configuration, may be removably coupled to a user (4), such as at a wrist (13), and may include one or more LEDs (534). Figure 31C A touch-sensitive interface (398) operably coupled (400) to a computing system, such as via a wired or wireless communication configuration, may be removably coupled to a user (4), such as at a wrist (13), and may include a controllably actuated piezoelectric assembly (536). Figure 31D A touch-sensitive interface (398) operably coupled (400) to a computing system, such as via a wired or wireless communication configuration, may be removably coupled to a user (4), such as at a wrist (13), and may include a controllably actuatable audio speaker assembly (538). Figure 31E A touch conversion interface (398) operably coupled (400) to a computing system, such as via a wired or wireless communication configuration, may be removably coupled to a user (4), such as at a wrist (13) location, and may include one or more controllably actuated shape memory alloy segments (540). Figure 32A and 32B shows that when viewed from an orthogonal view, such as Figure 31E The configuration shown in FIG may include a single SMA segment (540) (e.g. Figure 32A ), or multiple SMA segments (540, 546, 548, 550), each of which can be individually controlled.

[0098] Reference again Figure 30A , the touch conversion interface may include a plurality of (530) touch conversion elements, which may be similar to or different from each other. For example, referring to Figure 33A A touch conversion interface (398) operably coupled (400) to a computing system, such as via a wired or wireless communication configuration, may be removably coupled to a user (4), such as at a wrist (13) location, and may include three or more controllably actuated shape memory alloy segments (540, 552, 554) that are longitudinally positioned relative to each other when coupled to the touch conversion interface (398). Figure 33B A configuration is shown in which a touch conversion interface includes a fairly extensive plurality of touch conversion elements, such as a plurality of SMA segments (540, 552, 554), a plurality of haptic motors (532, 533), a plurality of piezoelectric assemblies (536, 537), a plurality of audio speaker assemblies (538, 539), and a plurality of LEDs (534, 535), each of which can be individually and / or independently actuated and controlled to provide enhanced perception to a user at a local touch workstation.

[0099] refer to Figure 36 , a surgical robotic integration configuration is shown wherein an operator at a touch sensing auxiliary operator workstation can utilize a surgical robotic system at a remote location, such as a location (640) across the room, country, or globe from the operator workstation, and wherein touch conversion elements can be utilized to enhance the operator's understanding of contact, touch, and other activities at the remote location during surgical navigation and manipulation of a robotic surgical end effector (e.g., a gripper (518)) relative to a target portion (576) of a target tissue structure (572). Figure 36 As shown, the operator workstation can include one or more (530) touch conversion interfaces (398) that are removably coupled to a portion of the user (4), such as a wrist (13), which can be configured to respond to contact at a touch sensing assembly (360) of a wrist portion (582) of a robotic instrument (594). The operator workstation can also include two additional touch conversion interfaces (508, 512) that can be configured to respond to contact at a touch sensing assembly (602, 604) coupled to each respective robotic gripper opposing member (522, 520). The touch conversion interfaces are operably coupled (400, 510, 514, 230, 166) to the computing system (144), for example, via a wired or wireless connection. The touch sensing assembly is similarly operably coupled (592, 606, 608, 230, 166) to the computing system (144), for example, via a wired or wireless connection. Thus, as the remotely controllable robotic instrument (594) advances and navigates to a target portion (576) of a target tissue structure (572), the user (4) at the workstation can obtain intuitive sensory cues regarding contact and touch between various aspects of the instrument and various aspects of the tissue, such as contact between the robotic instrument wrist (582) and a wall or edge (578) of the tissue structure (572), and contact between the robotic instrument gripper (518) members (520, 522) and the wall or edge (578, 576) of the tissue structure (572). Preferably, one or more image capture devices can be configured to capture one or more views of the surgical scene to be presented (598) to the user (4) at the operator workstation, such as on a display (278) that is operably coupled to the computing system (144), such as by a wired or wireless connection.

[0100] Therefore, reference Figure 34In a process flow, a user at a local workstation has a connection to a remote engagement configuration in a remote environment, such as an operably coupled robotic arm having one or more connected touch sensing surfaces, to assist the user in physically engaging one or more aspects of the remote environment (556). The local workstation and the remote engagement configuration may be powered on, activated, and ready for remote touch engagement by the user (558). The user may operate a primary input device on the local workstation that is operably coupled to the remote engagement configuration (e.g., coupled to an operably coupled robotic arm in the remote environment) to physically engage one or more aspects of the remote environment (e.g., physically engage a surface of an object in the remote environment) (560). Through the local workstation, the user can experience and understand various aspects of physical engagement between the remote engagement workstation and one or more aspects of the remote environment (e.g., locally sensing various levels of touch engagement in the remote environment through the local workstation; for example, a cuff touch sensor of a distal portion of a robotic arm operably coupled to the remote environment can be configured to provide the user with an intuitive understanding of touch engagement in the remote environment, such as via a local touch translation interface that can be coupled to the user and can be configured to locally provide one or more modalities of remote touch-derived feedback, such as via a kinematically similar and / or intuitive local configuration of the local touch translation interface) (562).

[0101] refer to Figure 37 , a touch conversion interface and similar use of a touch-based operator workstation can be utilized to help users experience contact, touch, and related activities in a remote environment that is truly remote because it is a virtual environment (612) (i.e., it is "real" only insofar as it is created on a computer). For example, Figure 37In an embodiment, a user is able to utilize a tactile primary input device (280) to navigate a mobile arm robot (622) virtual element in a virtual environment (612) that includes virtual aspects such as a virtual road (614), virtual walls (616) defining a cavity (618), and a virtual prize element (620) or target, such as a game-based "gold pot" element, which the user can obtain or win if the user is able to successfully virtually grab the virtual prize element (620) using a virtual grabber element (628, 630) mounted on a virtual robotic arm (626), wherein the virtual robotic arm (626) is mounted on a virtual mobile base (624) in the depicted virtual environment (612). The virtual touch sensing elements (632, 634, 636) can be virtually coupled to the wrist portion and virtual gripper elements (628, 630) of the virtual robotic arm (626) and configured to provide a real user at the user workstation with a sense of contact with the virtual robotic structure or with other aspects of the virtual environment (612), such as portions of a virtual wall (616). In other words, if the user drives the virtual robot (622) such that the virtual gripper elements (628, 630) hit a portion of the virtual wall (616), such contact and / or intersection can be translated back to the touch translation interface (508, 512, 398) at the user workstation to help provide the user with an intuitive sense of activity in the virtual environment (612).

[0102] Therefore, reference Figure 35, a user at a local workstation can have a connection with a virtual remote engagement configuration in a virtual remote environment, such as an operably coupled virtual robotic arm having one or more connected virtual touch sensing surfaces, to assist the user in physically engaging one or more aspects of the virtual remote environment (564). The local workstation and the virtual remote engagement configuration can be powered on, activated, and ready for virtual remote touch engagement by the user (566). The user can operate a primary input device on the local workstation that is operably coupled to the virtual remote engagement configuration (e.g., coupled to the operably coupled virtual robotic arm in the virtual remote environment) to physically engage one or more aspects of the virtual remote environment (e.g., virtually physically engage a surface of an object in the virtual remote environment) (568). Through the local workstation, the user is able to experience and understand aspects of virtual physical engagement between the virtual remote engagement workstation and one or more aspects of the virtual remote environment (such as locally perceiving various levels of virtual touch engagement at the virtual remote environment through the local workstation; for example, a cuff touch sensor virtually operably coupled to a distal portion of a virtual robotic arm in the virtual remote environment may be configured to provide the user with an intuitive understanding of virtual touch engagement at the virtual remote environment, such as via a local touch translation interface that is coupleable to the user and that may be configured to locally provide one or more modalities of remote touch-derived feedback, such as via a kinematically similar and / or intuitive local configuration of the local touch translation interface) (570).

[0103] See also Figure 38A , an orthogonal view is shown showing a bushing or at least partially cylindrical touch sensing assembly (656) that can be fixedly or removably coupled to a structural element, such as a shaft member (654) of a machine or machine component, which is best understood as a load configuration during operation. For ease of illustration, the touch sensing assembly (656) is shown with the shaft member (654) mounted on a top surface (670) of a table (652), and the interface (726) between the touch sensing assembly (656) and the shaft member (654) can be bonded to generally prevent relative movement during load. The touch sensing assembly (656) can be operably coupled (658, 230, 166) to a computing system (144), such as by a wired or wireless coupling, and can include multiple imaging devices (106) and sources (116). In operation, as the shaft member (654) is loaded, such as by bending back and forth (662, 660), portions of the touch sensing assembly (656) may be placed in compression, tension, shear, etc., and such loading may be detected and characterized at the computing system using associated imaging devices (106) and sources (116), which may be placed in sectors (e.g., four pairs are shown around the perimeter of the touch sensing assembly 656). Figure 38B A side view of a similar configuration is shown.

[0104] Figure 38C Shown with Figure 38B A slightly similar configuration, but with the addition of a structural cap member (668), which can be configured to constrain the touch sensing assembly (656) at the junction of the structural cap member (668) and the shaft member (654). With this configuration, the cylindrical touch sensing assembly (656) can be placed in a more pure compression or tension state as the shaft member (654) bends (662, 660).

[0105] refer to Figure 38D , showing the Figure 38C A somewhat similar configuration but with the addition of a solid cylindrical touch sensing assembly (672) forming a cylindrical base or pad to which the structural cap (668) and the end of the shaft (654) can be mounted (i.e., Figure 38D The shaft member shown in FIG does not pass through the cylindrical touch sensing assembly 672). Such a configuration also facilitates the cylindrical touch sensing assembly (672) to detect not only bending (662, 660) type loads, but also tensile or compressive loads (667, 664) on the shaft member (654), and generally depends on the source / imaging device (e.g. Figure 38A 116 / 106 of ), characterizes the load paradigm in the relevant structural members (654) quite extensively.

[0106] refer to Figure 38E It is worth noting that the sensor and / or transmitter portion can be placed in direct contact with the optical element material of the touch sensing assembly (656), such as Figure 38A , or may be placed at a more remote location using a configuration such as optical fibers or bundles thereof (132, 138) to be operably coupled to other locations, such as an emission detection controller (734) module as shown (and operably coupled to the computing system 144 and the power supply 102; 730, 732), which may include interfaces (764, 766) configured to effectively transmit light or other radiation to and from one or more sources and one or more image capture devices that may be housed therein.

[0107] refer to Figure 38F To help remove tethers and wired couplings, such as in cyclic torsional loading (758) about an axis (760), such as in machine applications, various aspects of the system configuration can be coupled to machine components and wirelessly connected to avoid various tether-based limitations. For example, reference Figure 38FThe module or housing (742) can contain power supply (744), battery charging (748), and computer / controller (746) components coupled to each other (752, 754), which can be coupled to each other (756) to the touch sensing assembly (656) and to a more remote computing device (144) via wireless connections (167, 166). A motion-based charger (748) has a small mass (750) configured to oscillate and provide a low level of current based on oscillatory motion about an associated axis (654). The motion-based charger (748) can be configured to continuously charge the battery (744); for example, the mass (750) can be configured to move a magnetic material through one or more coils in an oscillatory manner, or can be configured to load a piezoelectric member with axis motion (e.g., via angular acceleration and velocity squared / radius relationships) to provide a low level of charging current to the battery (744).

[0108] refer to Figure 39 , showing the Figure 36 A slightly similar configuration with the addition of small touch sensing component pads (678, 680) that are coupled to the computing system (144) via a wired or wireless connection (674, 676, 230, 166) to provide a similar configuration to that described above with respect to Figure 38D In the manner described, further characterization of the relative grasping elements of the grasper tool (582) is provided.

[0109] refer to Figure 40In one embodiment, a user plans to perform a medical procedure on a patient using an electromechanical system (e.g., a robot) configured with an interventional tool (e.g., a grasper) integrated with one or more touch sensors having one or more deformable transmission layers (690). The user may activate and calibrate the system using a computing system operably coupled between the electromechanical system and a user workstation (692). The user may navigate the interventional tool to the patient's anatomical structure from a workstation that may be located near or remote from the patient, the workstation including a display system configured to display aspects of the interventional tool's surroundings, a control interface (e.g., a tactile interface) that assists the user in providing commands to the interventional tool, and a touch conversion interface configurable to provide input to the user in response to contact or touch detected at one or more touch sensors operably coupled to the interventional tool (694). The user can utilize the control interface to bring the target tissue structure of the patient into contact with the interventional tool to perform one or more aspects of the medical procedure while simultaneously obtaining and / or sensing information related to the proximal environment of the interventional tool, such as contact between the interventional tool and the target tissue structure, which can be sensed and / or observed using various aspects of the user workstation (e.g., the display system, the control interface, and / or the touch-sensitive interface) (696). The user can use the user workstation to retract the interventional tool from the target tissue structure and the patient, thereby completing the medical procedure or a portion thereof (698).

[0110] refer to Figure 41In one embodiment, a user may plan to execute a program associated with a virtual environment (e.g., a video game) using a virtual electromechanical system (e.g., a virtual robot), the virtual electromechanical system being configured to have a virtual tool (e.g., a gripper) integrated with one or more virtual touch sensors operably coupled to one or more touch conversion interfaces (702). The user may activate and calibrate the system using a computing system operably coupled between the virtual electromechanical system and a user workstation (704). The user may be able to navigate the virtual tool to a virtual target from a workstation located near or remote from the patient, the workstation including a display system configured to display aspects of the virtual tool's surroundings, a control interface (e.g., a haptic interface) that assists the user in providing commands to the virtual tool, and a touch conversion interface that may be configured to provide input to the user in response to contact or touch detected at one or more virtual touch sensors operably coupled to the virtual tool (706). The user can utilize the control interface to contact one or more virtual objects with the virtual tool to perform one or more aspects of the desired virtual tool movement while simultaneously obtaining and / or sensing information related to the environment proximate to the virtual tool, such as contact between the virtual tool and the one or more virtual objects, which can be sensed and / or observed using various aspects of the user workstation (e.g., the display system, the control interface, and / or the touch-sensitive interface) (708). The user can virtually retract the virtual tool from the one or more virtual objects using the user workstation to complete the entire process or a portion thereof (710).

[0111] refer to Figure 42In one embodiment, a user may plan to perform a medical procedure on a patient using an electromechanical system (e.g., a robot) configured with an interventional tool (e.g., a grasper) integrated with one or more touch sensors (having one or more deformable transmission layers), and one or more control sensors (which may also have one or more deformable transmission layers) (714). The user may activate and calibrate the system using a computing system operably coupled between the electromechanical system and a user workstation (716). The user may be able to navigate the interventional tool to the patient's anatomy from a workstation that may be located near or remote from the patient, the workstation including a display system configured to display aspects of the interventional tool's surroundings, a control interface (e.g., a tactile interface) that assists the user in providing commands to the interventional tool, and a touch conversion interface that may be configured to provide input to the user in response to contact or touch detected at one or more touch sensors operably coupled to the interventional tool (718). The user can utilize the control interface to bring the target tissue structure of the patient into contact with the interventional tool to perform one or more aspects of the medical procedure while simultaneously obtaining and / or sensing information related to the proximal environment of the interventional tool, such as contact between the interventional tool and the target tissue structure, which can be sensed and / or observed using various aspects of the user workstation (e.g., the display system, the control interface, and / or the touch-sensitive interface) (720). The user can complete the medical procedure or a portion thereof by retracting the interventional tool from the target tissue structure and the patient using the user workstation (722).

[0112] refer to Figure 43 A mechanical system may include a structural member, such as a shaft, beam, or elongated member, that may be subjected to loads such as bending, tension, and / or shear during operation of the mechanical system and may be coupled to a sensing assembly including a deformable transmission layer (770). The sensing assembly may be operably coupled to a computing system and an imaging device such that at least one mode of load and / or deformation of the structural member may be monitored using the computing system (772). The sensing assembly and the computing system may be initialized, calibrated, and / or configured to sense one or more aspects of the structural member during operation of the mechanical system (774). The computing system may be configured to provide an operator with an output related to a real-time or near real-time load configuration of the mechanical system, such as load data related to the structural member that may be displayed to the operator and / or indicate to the operator that one or more predetermined load thresholds have been approached or reached within the mechanical system (776). The computing system may also be configured to facilitate a change in mechanical system operation, such as reducing a load demand or shutting down one or more aspects of the mechanical system, when the computing system determines that an overload condition has been met, such as by comparing an output from the sensing assembly to one or more predetermined load thresholds (778).

[0113] refer to Figure 44A vehicle (e.g., an automobile) may include one or more structural components, such as one or more housings and / or support structures, that may be subjected to loads, such as bending, stretching, and / or shearing, during operation of the vehicle and may be coupled to one or more sensing assemblies (780) including one or more deformable transmission layers. The one or more sensing assemblies may be operably coupled to a computing system and one or more imaging devices such that at least one mode of load and / or deformation of the one or more structural components may be monitored using the computing system (782). The one or more sensing assemblies and the computing system may be initialized, calibrated, and / or configured to sense one or more aspects of the one or more structural components during operation of the one or more structural components and the vehicle (784). The computing system may be configured to provide an operator with output related to a real-time or near real-time load configuration of the one or more structural components, such as load data that may be displayed to the operator and / or used to create an indication to the operator that one or more predetermined load thresholds associated with the one or more structural components have been approached or met (786). The computing system may also be configured to facilitate changing the operation of one or more structural components and / or other components of the vehicle, such as reducing load demands or shutting down one or more operatively coupled systems, components, or subsystems, when the computing system determines that an overload condition has been met, such as by comparing outputs from one or more sensing components to one or more predetermined load thresholds (788).

[0114] refer to Figure 45 A mechanical system may include a structural member, such as a shaft, beam, or elongated member, that may be subjected to loads, such as bending, stretching, and / or shearing, during operation of the mechanical system and that may be coupled to a sensing base assembly including a deformable transmission layer (790). The sensing base assembly may be operably coupled to a computing system and an imaging device such that at least one mode of load and / or deformation of the structural member may be monitored using the computing system (792). The sensing base assembly and the computing system may be initialized, calibrated, and / or configured to sense one or more aspects of the structural member during operation of the mechanical system (794). The computing system may be configured to provide an operator with an output related to a real-time or near real-time load configuration of the mechanical system, such as load data related to the structural member that may be displayed to the operator and / or an indication to the operator that one or more predetermined load thresholds within the mechanical system have been approached or reached (796). The computing system may also be configured to facilitate a change in mechanical system operation, such as reducing a load demand or shutting down one or more aspects of the mechanical system, when the computing system determines that an overload condition has been met, such as by comparing the output from the sensing base assembly to one or more predetermined load thresholds (798).

[0115] refer to Figure 46In one embodiment, a user of a local workstation may have a connection with a remote engagement configuration in a telemedical intervention environment, such as an operatively coupled medical robotic arm having one or more connected touch sensing surfaces, to assist the user in physically engaging one or more aspects of the telemedical intervention environment (802). The local workstation and the remote engagement configuration may be powered on, activated, and prepared for telemedical touch engagement by the user (804). The user may operate a primary input device on the local workstation that is operatively coupled to the remote engagement configuration (e.g., coupled to the operatively coupled medical robotic arm in the remote environment) to physically engage one or more aspects of the remote environment (e.g., physically engage a surface of an object in the remote environment, such as a target tissue structure) (806). Through the local workstation, the user is able to experience and understand various aspects of physical engagement between the remote engagement workstation and one or more aspects of the remote environment (such as locally sensing various levels of touch engagement in the remote environment through the local workstation; for example, a cuff touch sensor operably coupled to a distal portion of a medical robotic arm in the remote environment can be configured to provide the user with an intuitive understanding of touch engagement in the remote environment, such as via a local touch translation interface that can be coupled to the user and can be configured to locally provide one or more modalities of remote touch-derived feedback, such as via a kinematically similar and / or intuitive local configuration of the local touch translation interface) (808).

[0116] refer to Figure 47In one embodiment, a user of a local workstation may have a connection to a remote engagement configuration in a telemedical intervention environment, such as an operably coupled medical robotic arm having one or more connected touch sensing surfaces, to assist the user in controlling the remote engagement configuration and physically engaging one or more aspects of the telemedical intervention environment (810). The local workstation and the remote engagement configuration may be powered on, activated, and prepared for telemedical touch engagement by the user (812). The user may operate a primary input device on the local workstation, the primary input device operably coupled to the remote engagement configuration (e.g., coupled to the operably coupled medical robotic arm in the remote environment), to physically engage one or more aspects of the remote environment (e.g., physically engage a surface of an object in the remote environment, such as a target tissue structure) within one or more predetermined load limits, which may be monitored relative to one or more loads applied to the one or more connected touch sensing surfaces (814). Through the local workstation, the user is able to experience and understand aspects of physical engagement between the remote engagement workstation and one or more aspects of the remote environment (such as locally sensing various levels of touch engagement at the remote environment through the local workstation; for example, a cuff touch sensor of a distal portion of a medical robotic arm operably coupled to the remote environment may be configured to provide the user with an intuitive understanding of touch engagement at the remote environment, such as via a local touch translation interface that is coupleable to the user and that may be configured to locally provide one or more modalities of remote touch-derived feedback, such as via kinematic similarity and / or intuitive local configuration of the local touch translation interface), and physically engage aspects of the remote medical intervention environment within one or more predetermined load limits that may be monitored relative to one or more loads applied to one or more connected touch-sensing surfaces (816).

[0117] refer to Figure 48 , showing something similar to Figure 29C An embodiment to illustrate a hybrid configuration of touch sensing and touch conversion for each of two fingers (index finger 526, middle finger 528), wherein the touch conversion interface (508, 512) can be detachably coupled to each finger to obtain the same as above reference Figure 29C The kinematically similar feedback described above, for example, by adding a cuff-type touch sensing interface (822, 820; for example, similar to the above reference Figure 18CThe touch sensing interface (822, 820) can be used to sense an associated increase or decrease in hoop stress or hoop strain, which can be associated with an actuation, activity, movement, or intent of the finger, as well as contact between the finger and another object.

[0118] Therefore, reference Figure 49 , shows an exemplary variation, in which the above reference Figure 48 Configuration described. Figure 49 , a user at a local workstation can have a connection with a remote engagement configuration in a remote environment, such as an operatively coupled robotic arm having one or more connected touch-sensing surfaces, to assist the user in physically engaging one or more aspects of the remote environment (830). The local workstation and the remote engagement configuration can be powered on, activated, and ready for remote and local touch engagement by the user (832). The user can operate a primary input device and a local touch-sensing configuration at the local workstation, both of which can be operatively coupled to the remote engagement configuration (e.g., coupled to an operatively coupled robotic arm in the remote environment) via the computing system to physically engage one or more aspects of the remote environment (e.g., physically engage a surface of an object in the remote environment) (834). Through the local workstation, touch activity of the user may be sensed to assist in operation of the remote engagement configuration, and the user may be able to experience and understand various aspects of physical engagement between the remote engagement workstation and one or more aspects of the remote environment (such as locally sensing various levels of touch engagement at the remote environment through the local workstation; for example, a cuff touch sensor of a distal portion of a robotic arm operably coupled to the remote environment may be configured to provide the user with an intuitive understanding of touch engagement at the remote environment, such as via a local touch translation interface that is coupleable to the user and that may be configured to locally provide one or more modalities of remote touch-derived feedback, such as via a kinematically similar and / or intuitive local configuration of the local touch translation interface) (836).

[0119] refer to Figure 50 , showing the Figure 49 Similar configurations, but where the operator / user operates in a synthetic or virtual environment using a similar hybrid local interface. Figure 50, a user of a local workstation may have a connection with a virtual remote engagement configuration in a virtual remote environment, such as an operably coupled virtual robotic arm having one or more connected virtual touch sensing surfaces, to assist the user in physically engaging one or more aspects of the virtual remote environment (840). The local workstation and the virtual remote engagement configuration may be powered on, activated, and ready for virtual remote touch engagement by the user (842). The user may operate a primary input device and a local touch sensing configuration at the local workstation, both operably coupled to the virtual remote engagement configuration (e.g., coupled to an operably coupled virtual robotic arm in the virtual remote environment) to physically engage one or more aspects of the virtual remote environment (e.g., virtually physically engage a surface of an object in the virtual remote environment) (844). Through the local workstation, touch activity associated with the user may be sensed to assist in operation of the virtual remote engagement configuration, and the user may be able to experience and understand aspects of virtual physical engagement between the virtual remote engagement workstation and one or more aspects of the virtual remote environment (such as locally sensing, through the local workstation, various levels of virtual touch engagement at the virtual remote environment; for example, a cuff touch sensor virtually operably coupled to a distal portion of a virtual robotic arm in the virtual remote environment may be configured to provide the user with an intuitive understanding of virtual touch engagement at the virtual remote environment, such as via a local touch translation interface that may be coupled to the user and configured to locally provide one or more modalities of remote touch-derived feedback, such as via a kinematically similar and / or intuitive local configuration of the local touch translation interface) (846).

[0120] Various exemplary embodiments of the present invention are described herein. These examples are cited in a non-limiting sense. They serve to illustrate the more widely applicable aspects of the present invention. Various changes may be made to the described invention, and equivalents may be substituted without departing from the true spirit and scope of the present invention. In addition, many modifications may be made to adapt specific circumstances, materials, compositions of matter, processes, process actions, or steps to the purpose, spirit, or scope of the present invention. Furthermore, it will be understood by those skilled in the art that each individual variation described and illustrated herein has discrete components and features that can be easily separated or combined with the features of any of the other several embodiments without departing from the scope or spirit of the present invention. All such modifications are intended to fall within the scope of the claims associated with this disclosure.

[0121] The present invention includes methods that can be performed using the subject devices. The methods can include an act of providing such suitable devices. This provision can be performed by the end user. In other words, the act of "providing" simply requires the end user to obtain, access, access, locate, set up, activate, power on, or otherwise provide the necessary devices in the subject methods. The methods described herein can be performed in any logically feasible order of events and in any order of events.

[0122] The exemplary aspects of the present invention and details regarding material selection and manufacturing have been described above. Further details of the present invention can be understood in conjunction with the aforementioned patents and publications, as well as those generally known or understood by those skilled in the art. The same applies to the method-based aspects of the present invention, with respect to any additional actions that are typically or logically employed.

[0123] In addition, although the present invention has been described with reference to several examples optionally incorporating various features, the present invention is not limited to what is described or indicated for each variant of the present invention. Various changes may be made to the described invention and equivalents (whether listed herein or not included for brevity) may be substituted without departing from the true spirit and scope of the present invention. In addition, when a range of values ​​is provided, it is understood that each intermediate value between the upper and lower limits of the range and any other stated or intermediate values ​​within the stated range are included within the present invention.

[0124] In addition, it is contemplated that any optional feature of the described variants of the invention may be set forth and claimed independently, or in combination with any one or more of the features described herein. Reference to a singular item includes the possibility that there are multiple identical items. More specifically, as used herein and in the claims associated therewith, the singular forms "a," "an," "said," and "the" include plural referents unless expressly stated otherwise. In other words, the use of the articles allows for "at least one" in the subject items in the above description and in the claims associated with the present disclosure. It should also be noted that such claims may be drafted to exclude any optional elements. Therefore, this statement is intended to serve as an antecedent basis for the use of exclusive terms such as "only," "only," or the use of a "negative" limitation in the recitation of claim elements.

[0125] In the absence of such exclusive terms, the term "comprising" in claims relating to the present disclosure shall allow for the inclusion of any additional elements, regardless of whether a given number of elements is recited in such claims or whether the addition of features may be considered to alter the nature of the elements recited in such claims. Unless otherwise expressly defined herein, all technical and scientific terms used herein shall be given the broadest possible meaning that is commonly understood while maintaining claim validity.

[0126] The breadth of the present invention is not limited to the examples and / or subject description provided, but rather only by the scope of the claim language associated with this disclosure.

Claims

1. A system for characterizing interactions between surfaces, include: a. a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; b. a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; c. a detector configured to detect light from within at least a portion of the deformable transmission layer; d. a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a position along the interface membrane based at least in part on an interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation; and e. An auxiliary sensor operably coupled to the computing system and configured to provide input that the computing system can utilize to further geometrically characterize the interface object.

2. The system according to claim 1, in, The auxiliary sensor is coupled to the deformable transmission layer.

3. The system according to claim 1, further comprising: include: An auxiliary sensor mounting structure coupled to the deformable transmission layer, wherein the auxiliary sensor is coupled to the auxiliary sensor mounting structure.

4. The system according to claim 1, in, The auxiliary sensor and the deformable transmission layer are located in an operating environment comprising one or more wall structures, and wherein the auxiliary sensor is coupled to one of the one or more wall structures.

5. The system according to claim 1, in, The auxiliary sensor is selected from the group consisting of: an inertial measurement unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LIDAR device, a strain sensor, a load sensor, a temperature sensor, and an image capture device.

6. The system according to claim 1, in, The first illumination source comprises a light emitting diode.

7. The system according to claim 1, in, The detector is a photodetector.

8. The system according to claim 1, in, The detector is an image capture device.

9. The system according to claim 8, in, The image capture device is a CCD or CMOS device.

10. The system according to claim 1, further comprising: include: A lens is operably coupled between the detector and the deformable transmission layer.

11. The system according to claim 1, in, The computing system is operably coupled to the detector and configured to receive information from the detector related to light detected by the detector from within the deformable transmissive layer.

12. The system according to claim 1, in, The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source.

13. The system according to claim 1, in, The deformable transmission layer comprises an elastic material.

14. The system according to claim 13, in, The elastic material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU).

15. The system according to claim 13, in, The deformable transmission layer includes a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.

16. The system according to claim 15, in, The pigment material includes a metal oxide.

17. The system according to claim 1, in, The interface membrane includes an elastic material.

18. A system for characterizing interactions between surfaces, include: a. a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; b. a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; c. a detector configured to detect light from within at least a portion of the deformable transmission layer; d. a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a position along the interface membrane based at least in part on an interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation; and e. a robotic manipulator operably coupled to the computing system and the deformable transmission layer, the robotic arm being configured to controllably position and orient the deformable transmission layer relative to the interface object such that the computing system can characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane relative to the relative position and orientation of each of the deformable transmission layer and the interface object.

19. The system according to claim 18, in, The robotic manipulator includes a robotic arm.

20. The system according to claim 19, in, The robotic arm includes a plurality of joints coupled by substantially rigid linkage members.

21. The system according to claim 18, in, The robotic manipulator includes a flexible robotic instrument.

22. The system according to claim 18, further comprising: include: An end effector is coupled to the robotic manipulator.

23. The system according to claim 22, in, The end effector includes a gripper.

24. The system according to claim 18, in, The first illumination source comprises a light emitting diode.

25. The system according to claim 18, in, The detector is a photodetector.

26. The system according to claim 18, in, The detector is an image capture device.

27. The system according to claim 26, in, The image capture device is a CCD or CMOS device.

28. The system according to claim 18, further comprising: include: A lens is operably coupled between the detector and the deformable transmission layer.

29. The system according to claim 18, in, The computing system is operably coupled to the detector and is configured to receive information from the detector related to light detected by the detector from within the deformable transmissive layer.

30. The system according to claim 18, in, The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source.

31. The system according to claim 18, in, The deformable transmission layer comprises an elastic material.

32. The system according to claim 31, in, The elastic material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU).

33. The system according to claim 31, in, The deformable transmission layer includes a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.

34. The system according to claim 33, in, The pigment material includes a metal oxide.

35. The system according to claim 18, in, The interface membrane includes an elastic material.

36. A system for characterizing the geometry of a portion of a user's appendage, include: a. a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; b. a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; c. a detector configured to detect light from within at least a portion of the deformable transmission layer; d. a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a position along the interface membrane based at least in part on an interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation; e. an auxiliary sensor operably coupled to the computing system and configured to provide input that the computing system can utilize to further geometrically characterize the interface object; and f. A measurement housing configured to facilitate convenient engagement with a portion of the user's appendage, the measurement housing being coupled to the deformable transmission layer and the auxiliary sensor.

37. The system according to claim 36, in, The measurement housing is configured to facilitate engagement with a plantar aspect of a user's foot.

38. The system according to claim 37, in, The measurement housing is configured to be placed on the floor so that the user can step into the measurement housing.

39. The system according to claim 36, in, The auxiliary sensor is coupled to the deformable transmission layer.

40. The system according to claim 36, further comprising: include: An auxiliary sensor mounting structure coupled to the deformable transmission layer, wherein the auxiliary sensor is coupled to the auxiliary sensor mounting structure.

41. The system according to claim 36, in, The auxiliary sensor and the deformable transmission layer are coupled to the measurement housing.

42. The system according to claim 36, in, The auxiliary sensor is selected from the group consisting of: an inertial measurement unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LIDAR device, a strain sensor, a load sensor, a temperature sensor, and an image capture device.

43. The system according to claim 36, in, The first illumination source comprises a light emitting diode.

44. The system according to claim 36, in, The detector is a photodetector.

45. The system according to claim 36, in, The detector is an image capture device.

46. ​​The system according to claim 45, in, The image capture device is a CCD or CMOS device.

47. The system according to claim 36, further comprising: include: A lens is operably coupled between the detector and the deformable transmission layer.

48. The system according to claim 36, in, The computing system is operably coupled to the detector and is configured to receive information from the detector related to light detected by the detector from within the deformable transmissive layer.

49. The system according to claim 36, in, The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source.

50. The system according to claim 36, in, The deformable transmission layer comprises an elastic material.

51. The system according to claim 50, in, The elastic material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU).

52. The system according to claim 50, in, The deformable transmission layer includes a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.

53. The system according to claim 52, in, The pigment material includes a metal oxide.

54. The system according to claim 36, in, The interface membrane includes an elastic material.

55. A system for characterizing an interfacial interaction with at least one aspect of an interfacial structure, include: a. a plurality of touch sensing assemblies, each of the plurality of touch sensing assemblies being positioned circumferentially about a distal portion of a substrate member and comprising a deformable transmission layer coupled to an interface film, wherein the interface film interfaces with at least one aspect of the interface structure; a first illumination source operably coupled to the deformable transmission layer and configured to emit a first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; and a detector configured to detect light from within at least a portion of the deformable transmission layer; and b. a computing system operably coupled to each touch sensing assembly and configured to operate each of the detectors to detect at least a portion of the light directed from each deformable transmission layer, determine a surface orientation associated with a location along each interface film based at least in part on an interaction of the associated first illumination light with the associated deformable transmission layer; wherein each of the plurality of touch sensing components is configured to interface with one or more aspects of the interface structure to roughly process a surface of the interface structure; and wherein the computing system is configured to characterize a geometric profile of the interface structure interfaced with the plurality of interface films using the determined surface orientation.

56. The system according to claim 55, in, Each of the plurality of touch sensing components is positioned in a discrete perimeter manner around the distal portion of the substrate member.

57. The system according to claim 56, in, Each of the plurality of touch sensing assemblies is positioned about the distal portion of the substrate member in a substantially equal separation circumferential manner.

58. The system according to claim 57, in, The plurality of touch sensing components includes two touch sensing components positioned diametrically opposed along a length of the substrate member.

59. The system according to claim 57, in, The plurality of touch sensing components includes three or more touch sensing components positioned relative to one another in a circumferentially equally distributed manner along a length of the substrate member.

60. The system according to claim 55, in, The substrate member includes an elongated member having a proximal end and a distal end and a member length therebetween, and wherein each of the plurality of touch sensing assemblies are peripherally positioned at substantially equal lengths along the member length.

61. The system according to claim 55, in, The first illumination source comprises a light emitting diode.

62. The system according to claim 55, in, The detector is a photodetector.

63. The system according to claim 55, in, The detector is an image capture device.

64. The system according to claim 63, in, The image capture device is a CCD or CMOS device.

65. The system according to claim 55, further comprising: include: A lens is operably coupled between the detector and the deformable transmission layer.

66. The system according to claim 55, in, The computing system is operably coupled to the detector and is configured to receive information from the detector related to light detected by the detector from within the deformable transmissive layer.

67. The system according to claim 55, in, The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source.

68. The system according to claim 55, in, The deformable transmission layer comprises an elastic material.

69. The system according to claim 68, in, The elastic material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU).

70. The system according to claim 68, in, The deformable transmission layer includes a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.

71. The system according to claim 70, in, The pigment material includes a metal oxide.

72. The system according to claim 55, in, The interface membrane includes an elastic material.

73. A system for characterizing interactions between surfaces, include: a. a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; b. a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; c. a detector configured to detect light from within at least a portion of the deformable transmission layer; and d. a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, determine a surface orientation associated with a position along the interface membrane based at least in part on an interaction of the first illumination light with the deformable transmission layer, and characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation, Wherein the deformable transmission layer and the computing system are coupled to a portion of a road vehicle and are configured to assist in detecting one or more operational aspects of the road vehicle.

74. The system according to claim 73, in, The deformable transmission layer is coupled to a portion of an interior of the road vehicle.

75. The system according to claim 74, in, The deformable transmission layer is configured to be utilized by the computing system to detect one or more aspects of physical engagement between a user and a portion of an interior of the road vehicle.

76. The system according to claim 75, in, The deformable transmission layer is configured to be utilized by the computing system to detect one or more aspects of contact between one or more parts of the user's body and a portion of an interior of the road vehicle.

77. The system according to claim 73, in, The deformable transmission layer is coupled to a portion of an exterior of the road vehicle.

78. The system according to claim 77, in, The deformable transmission layer is configured to be utilized by the computing system to detect one or more aspects of a physical engagement between an element of an environment external to the vehicle and an exterior of the road vehicle.

79. The system according to claim 78, in, The deformable transmission layer is configured to be utilized by the computing system to detect physical intrusion from an environment external to the vehicle that causes deflection of one or more structures comprising an exterior of the road vehicle.

80. The system according to claim 73, in, The first illumination source comprises a light emitting diode.

81. The system according to claim 73, in, The detector is a photodetector.

82. The system according to claim 73, in, The detector is an image capture device.

83. The system according to claim 82, in, The image capture device is a CCD or CMOS device.

84. The system of claim 73, further comprising: include: A lens is operably coupled between the detector and the deformable transmission layer.

85. The system according to claim 73, in, The computing system is operably coupled to the detector and is configured to receive information from the detector related to light detected by the detector from within the deformable transmissive layer.

86. The system according to claim 73, in, The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source.

87. The system according to claim 73, in, The deformable transmission layer comprises an elastic material.

88. The system according to claim 87, in, The elastic material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU).

89. The system according to claim 87, in, The deformable transmission layer includes a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.

90. The system according to claim 89, in, The pigment material includes a metal oxide.

91. The system according to claim 73, in, The interface membrane includes an elastic material.

92. A system for characterizing interactions between surfaces, include: a. a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; b. a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; c. a detector configured to detect light from within at least a portion of the deformable transmission layer; and d. a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, to determine a surface orientation associated with a position along the interface membrane based at least in part on an interaction of the first illumination light with the deformable transmission layer, and to characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation, Wherein the deformable transmission layer and the computing system are coupled to a portion of a door lock system and are configured to assist in authenticating a user prior to electromechanically converting the door lock system from a locked state to an unlocked state.

93. The system according to claim 92, in, The door lock system is configured to utilize the deformable transmission layer to perform biometric authentication on the user.

94. The system according to claim 93, in, The door lock system is configured to capture a geometric profile associated with an outer surface of a portion of the user's appendage and compare the geometric profile to a previously stored geometric profile for authentication purposes.

95. The system according to claim 94, in, An outer surface of a portion of the user's appendage is the user's fingerprint.

96. The system according to claim 92, in, The first illumination source comprises a light emitting diode.

97. The system according to claim 92, in, The detector is a photodetector.

98. The system according to claim 92, in, The detector is an image capture device.

99. The system according to claim 98, in, The image capture device is a CCD or CMOS device.

100. The system according to claim 92, further comprising: include: A lens is operably coupled between the detector and the deformable transmission layer.

101. The system according to claim 92, in, The computing system is operably coupled to the detector and is configured to receive information from the detector related to light detected by the detector from within the deformable transmissive layer.

102. The system according to claim 92, in, The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source.

103. The system according to claim 92, in, The deformable transmission layer comprises an elastic material.

104. The system according to claim 103, in, The elastic material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU).

105. The system according to claim 103, in, The deformable transmission layer includes a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.

106. The system according to claim 105, in, The pigment material includes a metal oxide.

107. The system according to claim 92, in, The interface membrane includes an elastic material.

108. A handheld system for characterizing interactions between surfaces, include: a. a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; b. a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; c. a detector configured to detect light from within at least a portion of the deformable transmission layer; and d. a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, to determine a surface orientation associated with a position along the interface membrane based at least in part on an interaction of the first illumination light with the deformable transmission layer, and to characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation, Wherein, the deformable transmission layer and the computing system are coupled within a handheld system housing including a power source, and the handheld system housing is configured to facilitate manual operation by a user, so that the user can manually position and orient the deformable transmission layer to engage the interface film with the interface object.

109. The system according to claim 108, further comprising: include: A positioning sensor is operably coupled to the handheld system housing and the computing system.

110. The system according to claim 109, in, The positioning sensor is configured to be utilized by the computing system to determine a position of at least a portion of the handheld system housing in a global coordinate system.

111. The system according to claim 110, in, The computing system and the positioning sensor are also configured such that an orientation of at least a portion of the handheld system housing in the global coordinate system can be determined.

112. The system according to claim 111, in, The computing system and the positioning sensor are further configured such that the position and orientation of the deformable transmission layer in the global coordinate system can be determined.

113. The system according to claim 108, in, The first illumination source comprises a light emitting diode.

114. The system according to claim 108, in, The detector is a photodetector.

115. The system according to claim 108, in, The detector is an image capture device.

116. The system according to claim 115, in, The image capture device is a CCD or CMOS device.

117. The system according to claim 108, further comprising: include: A lens is operably coupled between the detector and the deformable transmission layer.

118. The system according to claim 108, in, The computing system is operably coupled to the detector and is configured to receive information from the detector related to light detected by the detector from within the deformable transmissive layer.

119. The system according to claim 108, in, The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source.

120. The system according to claim 108, in, The deformable transmission layer comprises an elastic material.

121. The system according to claim 120, in, The elastic material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU).

122. The system according to claim 120, in, The deformable transmission layer includes a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.

123. The system according to claim 122, in, The pigment material includes a metal oxide.

124. The system according to claim 108, in, The interface membrane includes an elastic material.

125. A system for characterizing interactions between surfaces, include: a. a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; b. a first illumination source operably coupled to the deformable transmission layer and configured to emit first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; c. a detector configured to detect light from within at least a portion of the deformable transmission layer; and d. a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, to determine a surface orientation associated with a position along the interface membrane based at least in part on an interaction of the first illumination light with the deformable transmission layer, and to characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation, Wherein, the deformable transmission layer comprises a material of known structural modulus, so that the computing system can estimate the load at the interface between the deformable transmission layer and the interface object.

126. The system according to claim 125, in, The deformable transmission layer comprises a known modulus of elasticity.

127. The system according to claim 125, in, The deformable transmission layer comprises an unloaded shape configuration that is deformable into a loaded shape when loaded, and wherein the computing system is configured to determine one or more aspects of a load on the deformable transmission layer when in the loaded shape based on the loaded shape configuration and a material comprising a known structural modulus of the deformable transmission layer.

128. The system according to claim 127, in, The computing system is configured to determine one or more aspects of loading of the deformable transmission layer, the one or more aspects of loading of the deformable transmission layer being selected from the group consisting of: tensile loading, compressive loading, bending loading, shear loading, and torsional loading.

129. The system according to claim 125, in, The first illumination source comprises a light emitting diode.

130. The system according to claim 125, in, The detector is a photodetector.

131. The system according to claim 125, in, The detector is an image capture device.

132. The system according to claim 131, in, The image capture device is a CCD or CMOS device.

133. The system according to claim 125, further comprising: include: A lens is operably coupled between the detector and the deformable transmission layer.

134. The system according to claim 125, in, The computing system is operably coupled to the detector and is configured to receive information from the detector related to light detected by the detector from within the deformable transmissive layer.

135. The system according to claim 125, in, The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source.

136. The system according to claim 125, in, The deformable transmission layer comprises an elastic material.

137. The system according to claim 136, in, The elastic material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU).

138. The system according to claim 136, in, The deformable transmission layer includes a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.

139. The system according to claim 138, in, The pigment material includes a metal oxide.

140. The system according to claim 125, in, The interface membrane includes an elastic material.

141. A system for characterizing interactions between surfaces, include: a. a deformable transmission layer coupled to an interface membrane, wherein the interface membrane interfaces with at least one aspect of an interface object; b. a first illumination source operably coupled to the deformable transmission layer and configured to emit the first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; c. a detector configured to detect light from within at least a portion of the deformable transmission layer; d. a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer, to determine a surface orientation associated with a position along the interface membrane based at least in part on an interaction of the first illumination light with the deformable transmission layer, and to characterize a geometric profile of at least one aspect of the interface object interfaced with the interface membrane using the determined surface orientation; and e. a touch conversion interface coupled to an aspect of a user's appendage and operably coupled to the computing system, the touch conversion interface being configured to convert one or more aspects of the characterized geometric contour to aspects of the user's appendage so that the user can intuitively experience one or more aspects of the physical engagement between the deformable transmission layer and the interface object.

142. The system according to claim 141, in, The touch conversion interface is coupled to an appendage of a user at a location selected from the group consisting of: a finger of the user; a wrist of the user; and a forearm of the user.

143. The system according to claim 141, in, The touch conversion interface is coupled to the aspect of the appendage of the user at a location selected such that there is a kinematic similarity between the location and a location of the deformable transmission layer.

144. The system according to claim 141, in, The system includes a second touch conversion interface coupled to a second aspect of the appendage of the user.

145. The system according to claim 144, in, The first touch conversion interface and the second touch conversion interface are longitudinally displaced relative to each other when coupled to the appendage of the user.

146. The system according to claim 141, in, The first illumination source comprises a light emitting diode.

147. The system according to claim 141, in, The detector is a photodetector.

148. The system according to claim 141, in, The detector is an image capture device.

149. The system of claim 148, wherein the image capture device is a CCD or CMOS device.

150. The system of claim 141 further comprising a lens operably coupled between the detector and the deformable transmission layer.

151. The system according to claim 141, in, The computing system is operably coupled to the detector and is configured to receive information from the detector related to light detected by the detector from within the deformable transmissive layer.

152. The system according to claim 141, in, The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source.

153. The system according to claim 141, in, The deformable transmission layer comprises an elastic material.

154. The system according to claim 153, in, The elastic material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU).

155. The system according to claim 153, in, The deformable transmission layer includes a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.

156. The system according to claim 155, in, The pigment material includes a metal oxide.

157. The system according to claim 141, in, The interface membrane includes an elastic material.

158. A system for robotic medical intervention, include: a. A deformable transmission layer coupled to an interface film, wherein the interface film is in contact with at least one aspect of the interface tissue structure; b. a first illumination source operably coupled to the deformable transmission layer and configured to emit the first illumination light into the deformable transmission layer at a known first illumination orientation relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; c. a detector configured to detect light from within at least a portion of the deformable transmission layer; d. a computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transmission layer to determine a surface orientation associated with a position along the interface film based at least in part on an interaction of the first illumination light with the deformable transmission layer, and to characterize a geometric profile of at least one aspect of the interface tissue structure interfacing with the interface film using the determined surface orientation; and e. a robotic medical instrument operably coupled to the computing system and the deformable transmission layer, the robotic instrument being configured to controllably position and orient the deformable transmission layer relative to the interface tissue structure such that the computing system can characterize a geometric profile of at least one aspect of the interface tissue structure interfaced with the interface membrane relative to the relative position and orientation of each of the deformable transmission layer and the interface object.

159. The system according to claim 158, in, The robotic medical instrument includes a robotic arm.

160. The system according to claim 159, in, The robotic arm includes a plurality of joints coupled by substantially rigid linkage members.

161. The system according to claim 158, in, The robotic medical instrument comprises a flexible robotic instrument.

162. The system according to claim 158, in, The robotic medical instrument includes a medical end effector coupled to the robotic medical instrument.

163. The system according to claim 162, in, The medical end effector includes a gripper.

164. The system according to claim 158, in, The first illumination source comprises a light emitting diode.

165. The system according to claim 158, in, The detector is a photodetector.

166. The system according to claim 158, in, The detector is an image capture device.

167. The system according to claim 166, in, The image capture device is a CCD or CMOS device.

168. The system according to claim 158, further comprising: include: A lens is operably coupled between the detector and the deformable transmission layer.

169. The system according to claim 158, in, The computing system is operably coupled to the detector and is configured to receive information from the detector related to light detected by the detector from within the deformable transmissive layer.

170. The system according to claim 158, in, The computing system is operably coupled to the first illumination source and configured to control emission from the first illumination source.

171. The system according to claim 158, in, The deformable transmission layer comprises an elastic material.

172. The system according to claim 171, in, The elastic material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU).

173. The system according to claim 171, in, The deformable transmission layer includes a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than an illumination reflectivity of the elastomeric matrix.

174. The system according to claim 173, in, The pigment material includes a metal oxide.

175. The system according to claim 158, in, The interface membrane includes an elastic material.

Images