Systems and methods for tactile intelligence

Abstract

A system for geometric surface characterization comprises a deformable permeable layer coupled to a mounting structure and an interface film, the interface film comprising: a deformable permeable layer interfaced to at least one side of an interfaced object; a first illumination source operably coupled to the deformable permeable layer using an illumination control layer, the illumination control layer configured to emit first illumination light into the deformable permeable layer in one or more known first illumination orientations relative to the deformable permeable layer such that at least a portion of the first illumination light interacts with the deformable permeable layer; a detector configured to detect light from within at least a portion of the deformable permeable layer; and a computing system configured to characterize the geometric profile of the surface of an object interfaced to the interface film using the determined surface orientation.

Description

Technical Field

[0001] The present invention generally relates to systems and methods for detecting, characterizing, and / or quantifying the aspects of a contact or touch interface between a special surface and another object, and more specifically to integration that may feature one or more deformable and permeable layers configured to assist various aspects of tactile intelligence.

Background Art

[0002] Computing, video communication, and various forms of remote presence have become essential components of modern life with the proliferation of laptop computers, smartphones, and systems such as video teleconferencing. Referring to Figure 1, a user (4) is shown in a typical work or home environment, interacting simultaneously with both a laptop computer (2) and a smartphone (6). Referring to Figure 2A, a so-called "smartwatch" (8) is shown, detachably attached to the user's (4) arm. Figure 2B illustrates a smartphone (6) held by the user (4), with one of the user's (4) hands (12) attempting to provide commands to the smartphone (6) computing system using gesture information. While these illustrative systems (2, 6, 8) may be configured to handle, for example, voice-based or gesture-based commands, much of the operation of such devices still occurs through physical interfaces such as keyboards or touchscreens, and much of the information exchanged between voices in video-based calls is in auditory and / or video form. Referring to Figures 3A-3E, many attempts have been made to improve the richness of interpersonal communication and / or so-called “remote presence” using modern systems. Figure 3A illustrates a laptop (2)-based video conferencing configuration, in which a user (4) can observe and communicate with a group of other participants through a matrix-style video user interface (14) that is viewed through a laptop display (16). Figure 3B illustrates a conference room-based video conferencing system, in which a group of local participants around a local conference table (20) can interact with remote participants through a relatively large display configured to show the remote participants' video through a video conferencing user interface (18).Referring to Figure 3C, another system allows a group of local participants (34) seated around a local conference table in a local meeting room (22) to interact via video teleconferencing with a group of remote participants displayed through a multiple integrated display / camera system arranged in relation to the local conference table, helping to create or simulate the perception that all participants are in the same location, or at least to some extent, that they are all communicating in a manner that makes it seem as if they are all locally. Referring to Figures 3D and 3E, the video system may be used to help remote users participate in local discussions about scenarios such as healthcare. Figure 3D illustrates a configuration in which one user (4) from a first location can operate a multi-display (36, 38, 40) configuration via one or more user input devices (44), etc., to view video from a second operational location along with information and / or data about the scenario, while a camera (42) captures video data of the participant (4) at the first location for improved communication (i.e., beyond mere audio) and provides a video feed to the second operational location. Figure 3E illustrates a configuration in which a group of local healthcare providers (46, 48) utilize a cart (52)-based configuration featuring a display (54) for producing video portraits (58) of remote participants along with patients (50), while video of the local environment is captured for remote participants using a video camera (56) coupled to the cart (52). Figure 4 features a somewhat similar video communication system for healthcare, in which a remote user (58), such as a physician, can navigate a local healthcare facility room (68) containing a patient (50) and a hospital bed (60) using an electromechanically movable system (62) to which a camera (64) and a display (66) are coupled, enabling the remote user (58) to have a form of “remote presence” or “local presence” within the hospital room (68).

[0003] While each of the aforementioned configurations offers a level of usefulness beyond conventional voice calls, it could also be argued that they still lack some crucial aspects of true local presence. As connectivity, computing, video, auditory, and telecommunications technologies continue to improve, there is no doubt that such systems will continue to evolve to approach live local video presence. However, one crucial aspect of local presence that such systems will not address is the perception of local "touch" regarding remote participants, which may be related to the still high demand for air travel in some businesses, societies, and other scenarios. The prevalence of touch and tactile intelligence in the everyday existence of modern humans is so significant that it is no coincidence that some people, such as those who may have visual impairments, can navigate the world with great skill, relying heavily on touch and tactile intelligence. As we have evolved to develop fundamental interpretations of object shapes by utilizing the two visual heights of our eyes, we are also capable of utilizing touch and tactile intelligence to understand crucial aspects of objects that we physically encounter.

[0004] Considering a relatively simple embodiment, a remote inspection scenario can be considered. In a given user scenario, such as a scenario with multiple rivets (72) holding an airplane wing surface (70) in place, as shown in Figure 5A, if it is important to inspect a particular object or surface in detail with respect to surface aberrations, potential stress concentrations, and / or deformation, one solution is to inspect such surface (70) directly (74) by using an inspection light (76) configured to proceed to the location of each such airplane wing surface and irradiate the surface (70) vectorially at an angle selected to reveal surface anomalies. Similarly, referring to Figure 6A, with respect to the design of a smartphone (6) housing (80), if it is important to verify a certain texture of the external paint finish, or a certain fit between the smartphone (6) camera assembly (78) and the housing (80) that is "not too tight, not too loose," before undertaking mass production, then in many cases, personnel will fly to various locations around the world, make site visits, and perform touch tests on such parts. Figure 6B illustrates another embodiment in which the feel of the touch can be very important in determining whether the materials, fit, and finish of the crown (86), bezel (88), and / or buttons (84) for a watch (82) design are suitable for manufacturing. Finally, referring to Figure 6C, if the design for a detachable band (90) for a smartwatch (8) is configured to slidably connect to and detach from the watch (8) by the engagement of these parts with the user's hand (94, 95) firmly, but not excessively, then the feel of the touch can be very useful for performing the inspection. To extend the conventional physical reach of these devices to remote locations, there is a need for technologies that assist users in having a sense of touch.Described herein are systems, methods, and configurations for utilizing such characterization for various purposes, including high-precision touch sensor implementations and configurations that can be used and configured to improve and expand touch characterization in various scenarios and, but not limited to, assist in providing local users with a sense of touch regarding objects outside their conventional reach, such as objects in a remote environment. [Overview of the Initiative] [Means for solving the problem]

[0005] One embodiment is a system for geometric surface characterization, comprising a deformable permeable layer coupled to a mounting structure and an interface film, the interface film being interfaced to at least one side of an interfaced object having a surface to be characterized, and a first illumination source operably coupled to the deformable permeable layer using an illumination control layer, the illumination control layer being configured to emit first illumination light into the deformable permeable layer in one or more known first illumination orientations relative to the deformable permeable layer such that at least a portion of the first illumination light interacts with the deformable permeable layer, and at least one The system comprises a detector configured to detect light from within, and a computing system configured to operate the detector, detect at least a portion of the light directed from a deformable permeable layer, determine a surface orientation with respect to a position along an interface film based at least partially on the interaction of a first illumination light and the deformable permeable layer, and characterize the geometric profile of the surface of an object interfaced to the interface film using the determined surface orientation, wherein the deformable permeable layer is controllably pressed against at least one side of the interfaced object having a surface to be characterized. The deformable permeable layer may be configured to fluidly expand a bladder that is controllably inflated from a collapsed state to an expanded state and operably coupled using pressure injection. The fluid may be selected from the group consisting of air, inert gas, water, and saline solution. The bladder may be an elastomer bladder that is interconnected between the deformable permeable layer and the mounting structure. The deformable permeable layer may be configured to be controllably expanded relative to the mounting structure using the insertion of a mechanical expander member. The system may further include a computing system and a positioning sensor operably coupled to the deformable transparent layer. The positioning sensor may be used by the computing system to determine the position of at least a portion of the deformable transparent layer within a global coordinate system.The computing system and positioning sensor may further be configured to determine the orientation of at least a portion of the deformable transparent layer within a global coordinate system. The first illumination source may comprise a light-emitting diode. The detector may be a photodetector. The detector may be an image acquisition device. The image acquisition device may be a CCD or CMOS device. The system may further comprise a lens operably coupled between the detector and the deformable transparent layer. The computing system may be operably coupled to the detector and configured to receive from the detector information about light from within the deformable transparent layer detected by the detector. The computing system may be operably coupled to the first illumination source and configured to control emissions from the first illumination source. The system may further comprise a second illumination source operably coupled to an illumination control layer and configured to direct a second illumination into the illumination control layer using a second illumination wavelength different from a first illumination wavelength of the first illumination source. At least one of the first or second illumination wavelengths is in the infrared spectrum. The first and second illumination wavelengths may represent different colors. The system may further include a second illumination source configured to introduce a second illumination light into the illumination control layer from a different position or orientation relative to the first illumination source. The system may further include a third illumination source configured to introduce a third illumination light into the illumination control layer from a different position or orientation relative to the first and second illumination sources. The illumination control layer may be configured to have a shape selected from the group consisting of planar, substantially planar, curved, convex, semi-convex, and saddle shapes. The system may further include a second illumination source operably coupled to the second illumination control layer and configured to direct a second illumination into the deformable transmissive layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source. The first and second illumination control layers may be stacked relative to each other. The first and second illumination control layers may be stacked directly adjacent to each other. The illumination control layer may be positioned between the detector and the deformable transmissive layer.The detector, illumination control layer, and deformable permeable layer may be mechanically coupled within a fingertip assembly, configured to form part of an elongated sensing structure. The elongated sensing structure may comprise a synthetic finger or a robotic hand component. The detector, illumination control layer, and deformable permeable layer may be operably coupled to a lens, configured to create an optical path that provides a virtual camera position outside the geometry of the fingertip assembly relative to the deformable permeable layer. The deformable permeable layer may have a convex fingertip shape. The deformable permeable layer may be positioned directly adjacent to the illumination control layer within the fingertip assembly. The deformable permeable layer may be positioned separately from the illumination control layer within the fingertip assembly. The deformable permeable layer may comprise an elastomer material. The elastomer material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluoroelastomer. The deformable permeable layer comprises a composite material having a pigment material dispersed within an elastomer matrix, wherein the pigment material may be configured to provide illumination reflectance exceeding that of the elastomer matrix. The pigment material may comprise a metal oxide. The metal oxide may be selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide. The pigment material may comprise metal nanoparticles. The metal nanoparticles may be selected from the group consisting of silver nanoparticles and aluminum nanoparticles. The interface film may comprise an elastomer material. The surface of the interfaced object may be located and oriented in a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object interfaced to the interface film using its position and orientation relative to the global coordinate system. The computer system may be configured to collect two or more geometric profiles of two or more portions of the surface of the object interfaced to the interface film, and to determine the position and orientation of two or more geometric profiles relative to each other in a global coordinate system.The computing system may be configured to provide a three-dimensional mapping of two or more geometric profiles relative to each other in a global coordinate system. The computing system may be configured to stitch together geometrically adjacent geometric profiles using interpolation of geometric profiles and their relative position and orientation. The system may further include secondary sensors operably coupled to the computing system and configured to provide inputs that can be used by the computing system to further geometrically characterize the surface of the interfaced object. The secondary sensors may be selected from the group consisting of inertial measurement units (IMUs), capacitive touch sensors, resistive touch sensors, LIDAR devices, strain sensors, load sensors, temperature sensors, and image acquisition devices. The secondary sensors may include an IMU configured to output rotational and linear acceleration data to the computing system, which is configured to use the rotational and linear acceleration data to assist in characterizing the position or orientation of the deformable permeable layer in a global coordinate system. The secondary sensors may include an image acquisition device configured to capture image information about the surface of the interfaced object, which is configured to use the image information to assist in determining the location or orientation of the object relative to the deformable permeable layer. The system may further include one or more tracking tags coupled to an interfaced object and one or more detectors operably coupled to the computing system, so that the computing system can be used, at least in part, to identify and provide location information about the interfaced object based on the predetermined locations of one or more tracking tags on the interfaced object. One or more tracking tags may include radio frequency identification (RFID) tags, and one or more detectors may include RFID detectors.

[0006] Another embodiment is a system for geometric surface characterization, comprising: a deformable translucent layer coupled to a mounting structure and an interface film, the interface film being interfaced to at least one side of an interfaced object having a surface to be characterized; a first illumination source operably coupled to the deformable translucent layer using an illumination control layer, the illumination control layer being configured to emit first illumination light into the deformable translucent layer in one or more known first illumination orientations relative to the deformable translucent layer such that at least a portion of the first illumination light interacts with the deformable translucent layer; a detector configured to detect light from at least a portion of the deformable translucent layer; and a computing system, the computing system operating the detector to detect at least a portion of the light directed from the deformable translucent layer. The present invention relates to a system comprising: a computing system configured, at least in part, to determine a surface orientation with respect to a position along an interface film based on the interaction of a first illumination light and a deformable permeable layer, and to characterize the geometric profile of the surface of an object interfaced to the interface film using the determined surface orientation; and a robotic manipulator operably coupled to the computing system and the deformable permeable layer, wherein the robotic manipulator is configured to controllly position and orient the deformable permeable layer with respect to the interfaced object such that the computing system can characterize the geometric profile of the surface of the interfaced object interfaced to the interface film with respect to the relative position and orientation of the deformable permeable layer and the interfaced object. The robotic manipulator may include a robotic arm. The robotic arm may include a plurality of joints coupled by substantially rigid linkage members. The robotic manipulator may include a flexible robotic instrument. The system may further include an end effector coupled to the robotic manipulator.The end effector may include a grappler. The deformable permeable layer may be configured to fluidly expand a bladder, which is controllably inflated from a collapsed state to an expanded state using pressure injection and operably coupled. The fluid may be selected from the group consisting of air, inert gas, water, and saline solution. The bladder may be an elastomer bladder interconnected between the deformable permeable layer and the mounting structure. The deformable permeable layer may be configured to expand controllably relative to the mounting structure using the insertion of a mechanical expander member. The system may further include a computing system and a positioning sensor operably coupled to the deformable permeable layer. The positioning sensor may be used by the computing system to determine the position of at least a portion of the deformable permeable layer in a global coordinate system. The computing system and the positioning sensor may further be configured so that the orientation of at least a portion of the deformable permeable layer in a global coordinate system can be determined. The first illumination source may include a light-emitting diode. The detector may be a photodetector. The detector may be an image acquisition device. The image acquisition device may be a CCD or CMOS device. The system may further include a lens operably coupled between the detector and the deformable transparent layer. A computing system may be operably coupled to the detector and configured to receive from the detector information about the light from within the deformable transparent layer detected by the detector. The computing system may be operably coupled to the first illumination source and configured to control the emission from the first illumination source. The system may further include a second illumination source operably coupled to the illumination control layer and configured to direct a second illumination into the illumination control layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source. At least one of the first or second illumination wavelengths is in the infrared spectrum. The first and second illumination wavelengths may represent different colors. The system may further include a second illumination source configured to introduce the second illumination light into the illumination control layer from a different position or orientation relative to that of the first illumination source.The system may further include a third illumination source configured to introduce a third illumination light into the illumination control layer from a different position or orientation relative to the first and second illumination sources. The illumination control layer may be configured to have a shape selected from the group consisting of planar, substantially planar, curved, convex, semi-convex, and saddle shapes. The system may further include a second illumination source operably coupled to the second illumination control layer and configured to direct a second illumination into the deformable translucent layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source. The first and second illumination control layers may be stacked relative to each other. The first and second illumination control layers may be stacked directly adjacent to each other. The illumination control layer may be positioned between the detector and the deformable translucent layer. The detector, illumination control layer, and deformable translucent layer may be mechanically coupled within a fingertip assembly configured to form part of an elongated sensing structure. The elongated sensing structure may comprise a synthetic finger or a robotic hand component. The detector, illumination control layer, and deformable permeable layer may be operably coupled to a lens configured to create an optical path that provides a virtual camera position outside the geometric shape of the fingertip assembly relative to the deformable permeable layer. The deformable permeable layer may have a convex fingertip shape. The deformable permeable layer may be positioned directly adjacent to the illumination control layer within the fingertip assembly. The deformable permeable layer may be positioned separately from the illumination control layer within the fingertip assembly. The deformable permeable layer may comprise an elastomer material. The elastomer material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluoroelastomer. The deformable permeable layer comprises a synthetic material having a pigment material dispersed within an elastomer matrix, the pigment material may be configured to provide illumination reflectivity exceeding that of the elastomer matrix. The pigment material may include a metal oxide. The metal oxide may be selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide.The pigment material may comprise metal nanoparticles. The metal nanoparticles may be selected from the group consisting of silver nanoparticles and aluminum nanoparticles. The interface film may comprise an elastomer material. The surface of the interfaced object may be located and oriented within a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object interfaced to the interface film using its position and orientation relative to the global coordinate system. The computing system may be configured to collect two or more geometric profiles of two or more portions of the surface of the object interfaced to the interface film and to determine the position and orientation of these two or more geometric profiles relative to each other within the global coordinate system. The computing system may be configured to provide a three-dimensional mapping of these two or more geometric profiles relative to each other within the global coordinate system. The computing system may be configured to stitch together geometrically adjacent geometric profiles using interpolation of the geometric profiles and their relative position and orientation. The system may further comprise secondary sensors operably coupled to the computing system and configured to provide inputs that can be used by the computing system to further geometrically characterize the surface of the interfaced object. The secondary sensor may be selected from the group consisting of an inertial measurement unit (IMU), capacitive touch sensor, resistive touch sensor, LIDAR device, strain sensor, load sensor, temperature sensor, and image acquisition device. The secondary sensor may include an IMU configured to output rotational and linear acceleration data to a computing system, which is configured to use the rotational and linear acceleration data to assist in characterizing the position or orientation of the deformable permeable layer in a global coordinate system.The secondary sensor may include an image capture device configured to capture image information relating to the surface of the interfaced object, and the computing system may be configured to use the image information to help determine the location or orientation of the object relative to a deformable permeable layer. The system may further include one or more tracking tags coupled to the interfaced object and one or more detectors operably coupled to the computing system, so that the computing system can be used, at least in part, to identify and provide location information relating to the interfaced object based on the predetermined locations of one or more tracking tags to the interfaced object. One or more tracking tags may be radio frequency identification (RFID) tags, and one or more detectors may be RFID detectors.

[0007] Another embodiment is a method for geometric surface characterization, comprising the steps of: providing a deformable translucent layer coupled to a mounting structure and an interface film, wherein the interface film interfaces to at least one side of an interfaced object having a surface to be characterized; providing a first illumination source operably coupled to the deformable translucent layer using an illumination control layer, wherein the illumination control layer is configured to emit first illumination light into the deformable translucent layer in one or more known first illumination orientations relative to the deformable translucent layer such that at least a portion of the first illumination light interacts with the deformable translucent layer; and detecting light from at least a portion within the deformable translucent layer. The method relates to a method comprising the steps of providing a detector configured such as; and providing a computing system configured to operate the detector, detect at least a portion of the light directed from a deformable permeable layer, determine at least partially a surface orientation with respect to a position along an interface film based on the interaction of a first illumination light and the deformable permeable layer, and characterize the geometric profile of the surface of an object interfaced to the interface film using the determined surface orientation, wherein the deformable permeable layer is configured to be controllably pressed against at least one side of the interfaced object having a surface to be characterized. The deformable permeable layer may be configured to fluidly expand a bladder which is controllably expanded from a collapsed state to an expanded state and operably coupled using pressure injection. The fluid may be selected from the group consisting of air, an inert gas, water, and saline solution. The bladder may be an elastomer bladder which is interconnected between the deformable permeable layer and the mounting structure. The deformable permeable layer may be configured to be controllably expanded relative to the mounting structure using the insertion of a mechanical expander member. The method may further include the step of providing a localization sensor that is operably coupled to a computing system and a deformable permeable layer.A positioning sensor may be configured to be used by a computing system to determine the position of at least a portion of the deformable transparent layer in a global coordinate system. The computing system and the positioning sensor may further be configured to determine the orientation of at least a portion of the deformable transparent layer in a global coordinate system. The first illumination source may comprise a light-emitting diode. The detector may be a photodetector. The detector may be an image acquisition device. The image acquisition device may be a CCD or CMOS device. The method may further include the step of providing a lens operably coupled between the detector and the deformable transparent layer. The computing system may be operably coupled to the detector and configured to receive from the detector information about light from within the deformable transparent layer detected by the detector. The computing system may be operably coupled to the first illumination source and configured to control the emission from the first illumination source. The method may further include the step of providing a second illumination source operably coupled to an illumination control layer and configured to direct a second illumination into the illumination control layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source. At least one of the first or second illumination wavelengths is within the infrared spectrum. The first and second illumination wavelengths may represent different colors. The method may further include the step of providing a second illumination source configured to introduce a second illumination light into an illumination control layer from a different position or orientation relative to that of the first illumination source. The method may further include the step of providing a third illumination source configured to introduce a third illumination light into an illumination control layer from a different position or orientation relative to that of the first and second illumination sources. The illumination control layer may be configured to have a shape selected from the group consisting of planar, substantially planar, curved, convex, semi-convex, and saddle shapes. The method may further include the step of providing a second illumination source configured to be operably coupled to the second illumination control layer and to direct a second illumination into a deformable transmissive layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source.The first and second illumination control layers may be stacked relative to each other. The first and second illumination control layers may be stacked directly adjacent to each other. The illumination control layer may be positioned between the detector and the deformable permeable layer. The detector, illumination control layer, and deformable permeable layer may be mechanically coupled within a fingertip assembly configured to form part of an elongated sensing structure. The elongated sensing structure may comprise a synthetic finger or robotic hand component. The detector, illumination control layer, and deformable permeable layer may be operably coupled to a lens configured to create an optical path that provides a virtual camera position outside the geometry of the fingertip assembly relative to the deformable permeable layer. The deformable permeable layer may have a convex fingertip shape. The deformable permeable layer may be positioned directly adjacent to the illumination control layer within the fingertip assembly. The deformable permeable layer may be positioned separately from the illumination control layer within the fingertip assembly. The deformable permeable layer may comprise an elastomer material. The elastomer material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluoroelastomer. The deformable permeable layer comprises a synthetic material having a pigment material dispersed within the elastomer matrix, the pigment material may be configured to provide illumination reflectance exceeding that of the elastomer matrix. The pigment material may comprise a metal oxide. The metal oxide may be selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide. The pigment material may comprise metal nanoparticles. The metal nanoparticles may be selected from the group consisting of silver nanoparticles and aluminum nanoparticles. The interface film may comprise an elastomer material. The interface film may comprise an elastomer material. The surface of the interfaced object may be located and oriented in a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object interfaced to the interface film using its position and orientation relative to the global coordinate system.The computer system may be configured to collect two or more geometric profiles of two or more portions of the surface of an object interfaced to an interface film, and to determine the position and orientation of the two or more geometric profiles relative to each other in a global coordinate system. The computing system may be configured to provide a three-dimensional mapping of the two or more geometric profiles relative to each other in a global coordinate system. The computing system may be configured to stitch together geometrically adjacent geometric profiles using interpolation of the geometric profiles and their relative position and orientation. The method may further include the step of providing a secondary sensor operably coupled to the computing system and configured to provide input which can be used by the computing system to further geometrically characterize the surface of the interfaced object. The secondary sensor may be selected from the group consisting of an inertial measurement unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LiDAR device, a strain sensor, a load sensor, a temperature sensor, and an image acquisition device. The secondary sensor may include an IMU configured to output rotational and linear acceleration data to the computing system, which is configured to use the rotational and linear acceleration data to assist in characterizing the position or orientation of the deformable permeable layer in a global coordinate system. The secondary sensor comprises an image capture device configured to capture image information about the surface of the interfaced object, and the computing system may be configured to use the image information to help determine the location or orientation of the object relative to a deformable permeable layer.The method may further include providing one or more tracking tags coupled to an interfaced object and one or more detectors operably coupled to a computing system, such that a computing system can be used, at least in part, to identify and provide location information about an interfaced object based on the predetermined locations of one or more tracking tags to the interfaced object. The one or more tracking tags may include radio frequency identification (RFID) tags, and the one or more detectors may include RFID detectors.

[0008] Another embodiment is a method for geometric surface characterization, comprising the steps of: providing a deformable translucent layer coupled to a mounting structure and an interface film, wherein the interface film interfaces to at least one side of an interfaced object having a surface to be characterized; providing a first illumination source operably coupled to the deformable translucent layer using an illumination control layer, wherein the illumination control layer is configured to emit first illumination light into the deformable translucent layer in one or more known first illumination orientations relative to the deformable translucent layer such that at least a portion of the first illumination light interacts with the deformable translucent layer; providing a detector configured to detect light from within at least a portion of the deformable translucent layer; and a computing system, wherein the computing system operates the detector and at least a portion of the light directed from the deformable translucent layer. The present invention relates to a method comprising the steps of: providing a computing system configured to detect and, at least in part, determine a surface orientation with respect to a position along an interface film based on the interaction of a first illumination light and a deformable permeable layer, and using the determined surface orientation to characterize the geometric profile of the surface of an object interfaced to the interface film; and providing a robotic manipulator operably coupled to the computing system and the deformable permeable layer, wherein the robotic manipulator is configured to controllly position and orient the deformable permeable layer with respect to the interfaced object such that the computing system can characterize the geometric profile of the surface of the interfaced object interfaced to the interface film with respect to the relative position and orientation of the deformable permeable layer and the interfaced object. The robotic manipulator may include a robotic arm. The robotic arm may include a plurality of joints coupled by substantially rigid linkage members. The robotic manipulator may include a flexible robotic instrument.The method may further include a step of providing an end effector coupled to a robotic manipulator. The end effector may include a grappler. The deformable permeable layer may be configured to fluidly expand a bladder, which is controllably inflated from a collapsed state to an expanded state using pressure injection and operably coupled. The fluid may be selected from the group consisting of air, an inert gas, water, and saline solution. The bladder may be an elastomer bladder, which is interconnected between the deformable permeable layer and the mounting structure. The deformable permeable layer may be configured to expand controllably to the mounting structure using the insertion of a mechanical expander member. The method may further include a step of providing a computing system and a positioning sensor operably coupled to the deformable permeable layer. The positioning sensor may be used by the computing system to determine the position of at least a portion of the deformable permeable layer in a global coordinate system. The computing system and the positioning sensor may further be configured so that the orientation of at least a portion of the deformable permeable layer in a global coordinate system can be determined. The first illumination source may include a light-emitting diode. The detector may be a photodetector. The detector may be an image acquisition device. The image acquisition device may be a CCD or CMOS device. The method may further include the step of providing a lens operably coupled between the detector and a deformable transparent layer. A computing system may be operably coupled to the detector and configured to receive from the detector information about light from within the deformable transparent layer detected by the detector. The computing system may be operably coupled to a first illumination source and configured to control emissions from the first illumination source. The method may further include the step of providing a second illumination source operably coupled to an illumination control layer and configured to direct a second illumination into the illumination control layer using a second illumination wavelength different from a first illumination wavelength of the first illumination source. At least one of the first or second illumination wavelengths is in the infrared spectrum. The first and second illumination wavelengths may represent different colors.The method may further include the step of providing a second illumination source configured to introduce a second illumination light into the illumination control layer from a different position or orientation relative to the first illumination source. The method may further include the step of providing a third illumination source configured to introduce a third illumination light into the illumination control layer from a different position or orientation relative to the first and second illumination sources. The illumination control layer may be configured to have a shape selected from the group consisting of planar, substantially planar, curved, convex, semi-convex, and saddle shapes. The method may further include the step of providing a second illumination source configured to be operably coupled to the second illumination control layer and to direct a second illumination into the deformable transmissive layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source. The first and second illumination control layers may be stacked relative to each other. The first and second illumination control layers may be stacked directly adjacent to each other. The illumination control layer may be positioned between the detector and the deformable transmissive layer. The detector, illumination control layer, and deformable permeable layer may be mechanically coupled within a fingertip assembly, configured to form part of an elongated sensing structure. The elongated sensing structure may comprise a synthetic finger or a robotic hand component. The detector, illumination control layer, and deformable permeable layer may be operably coupled to a lens, configured to create an optical path that provides a virtual camera position outside the geometry of the fingertip assembly relative to the deformable permeable layer. The deformable permeable layer may have a convex fingertip shape. The deformable permeable layer may be positioned directly adjacent to the illumination control layer within the fingertip assembly. The deformable permeable layer may be positioned separately from the illumination control layer within the fingertip assembly. The deformable permeable layer may comprise an elastomer material. The elastomer material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluoroelastomer. The deformable permeable layer comprises a synthetic material having a pigment material dispersed within an elastomer matrix, wherein the pigment material may be configured to provide illumination reflectance exceeding that of the elastomer matrix.The pigment material may comprise a metal oxide. The metal oxide may be selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide. The pigment material may comprise metal nanoparticles. The metal nanoparticles may be selected from the group consisting of silver nanoparticles and aluminum nanoparticles. The interface film may comprise an elastomer material. The interface film may comprise an elastomer material. The surface of the interfaced object may be located and oriented within a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object interfaced with the interface film, with a certain position and orientation with respect to the global coordinate system. The computing system may be configured to collect two or more geometric profiles of two or more portions of the surface of the object interfaced with the interface film, and to determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system. The computing system may be configured to provide a three-dimensional mapping of the two or more geometric profiles relative to each other in the global coordinate system. The computing system may be configured to stitch together geometrically adjacent geometric profiles using interpolation of the geometric profiles and their relative position and orientation. The method may further include the step of providing a secondary sensor configured to provide input that can be operably coupled to a computing system and used by the computing system to further geometrically characterize the surface of an interfaced object. The secondary sensor may be selected from the group consisting of inertial measurement units (IMUs), capacitive touch sensors, resistive touch sensors, LiDAR devices, strain sensors, load sensors, temperature sensors, and image acquisition devices.The secondary sensor may include an IMU configured to output rotational and linear acceleration data to a computing system, which is configured to use the rotational and linear acceleration data to assist in characterizing the position or orientation of the deformable permeable layer in a global coordinate system. The secondary sensor may also include an image capture device configured to capture image information about the surface of the interfaced object, which is configured to use the image information to assist in determining the location or orientation of the object relative to the deformable permeable layer. The system may further include one or more tracking tags coupled to the interfaced object and one or more detectors operably coupled to the computing system, so that the computing system can use, at least in part, to identify and provide location information about the interfaced object based on the predetermined locations of one or more tracking tags to the interfaced object. One or more tracking tags may include radio frequency identification (RFID) tags, and one or more detectors may include RFID detectors. [Brief explanation of the drawing]

[0009] [Figure 1] Figures 1-4, 2A-2B, 3A-3E, and 4 illustrate various aspects of conventional computing and communication systems. [Figure 2A] Figures 1-4, 2A-2B, 3A-3E, and 4 illustrate various aspects of conventional computing and communication systems. [Figure 2B] Figures 1-4, 2A-2B, 3A-3E, and 4 illustrate various aspects of conventional computing and communication systems. [Figure 3A] Figures 1-4, 2A-2B, 3A-3E, and 4 illustrate various aspects of conventional computing and communication systems. [Figure 3B]Figures 1-4, 2A-2B, 3A-3E, and 4 illustrate various aspects of conventional computing and communication systems. [Figure 3C] Figures 1-4, 2A-2B, 3A-3E, and 4 illustrate various aspects of conventional computing and communication systems. [Figure 3D] Figures 1-4, 2A-2B, 3A-3E, and 4 illustrate various aspects of conventional computing and communication systems. [Figure 3E] Figures 1-4, 2A-2B, 3A-3E, and 4 illustrate various aspects of conventional computing and communication systems. [Figure 4] Figures 1-4, 2A-2B, 3A-3E, and 4 illustrate various aspects of conventional computing and communication systems.

[0010] [Figure 5A] Figures 5A-5B and 6A-6C illustrate various aspects of the scenario where a better understanding of the surface geometry or profile would be useful. [Figure 5B] Figures 5A-5B and 6A-6C illustrate various aspects of the scenario where a better understanding of the surface geometry or profile would be useful. [Figure 6A] Figures 5A-5B and 6A-6C illustrate various aspects of the scenario where a better understanding of the surface geometry or profile would be useful. [Figure 6B] Figures 5A-5B and 6A-6C illustrate various aspects of the scenario where a better understanding of the surface geometry or profile would be useful. [Figure 6C] Figures 5A-5B and 6A-6C illustrate various aspects of the scenario where a better understanding of the surface geometry or profile would be useful.

[0011] [Figure 7A] Figures 7A–7H and 8 illustrate various aspects of a touch-sensing assembly configured to utilize a deformable permeable layer. [Figure 7B] Figures 7A-7H and FIG. 8 illustrate various aspects of a touch sensing assembly configured to utilize a deformable transmissive layer. [Figure 7C] Figures 7A-7H and FIG. 8 illustrate various aspects of a touch sensing assembly configured to utilize a deformable transmissive layer. [Figure 7D] Figures 7A-7H and FIG. 8 illustrate various aspects of a touch sensing assembly configured to utilize a deformable transmissive layer. [Figure 7E] Figures 7A-7H and FIG. 8 illustrate various aspects of a touch sensing assembly configured to utilize a deformable transmissive layer. [Figure 7F] Figures 7A-7H and FIG. 8 illustrate various aspects of a touch sensing assembly configured to utilize a deformable transmissive layer. [Figure 7G] Figures 7A-7H and FIG. 8 illustrate various aspects of a touch sensing assembly configured to utilize a deformable transmissive layer. [Figure 7H] Figures 7A-7H and FIG. 8 illustrate various aspects of a touch sensing assembly configured to utilize a deformable transmissive layer. [Figure 8] Figures 7A-7H and FIG. 8 illustrate various aspects of a touch sensing assembly configured to utilize a deformable transmissive layer.

[0012] [Figure 9A] Figures 9A and 9B illustrate assemblies of a plurality of touch sensing assemblies such as those illustrated in FIGS. 7A-7H. [Figure 9B] Figures 9A and 9B illustrate assemblies of a plurality of touch sensing assemblies such as those illustrated in FIGS. 7A-7H.

[0013] [Figure 10A] Figures 10A-10I illustrate various aspects of touch sensing assembly embodiments that may be characterized by one or more secondary sensor configurations integrated therein. [Figure 10B] Figures 10A-10I illustrate various aspects of a touch-sensing assembly embodiment, which may feature one or more secondary sensor configurations integrated therein. [Figure 10C] Figures 10A-10I illustrate various aspects of a touch-sensing assembly embodiment, which may feature one or more secondary sensor configurations integrated therein. [Figure 10D] Figures 10A-10I illustrate various aspects of a touch-sensing assembly embodiment, which may feature one or more secondary sensor configurations integrated therein. [Figure 10E] Figures 10A-10I illustrate various aspects of a touch-sensing assembly embodiment, which may feature one or more secondary sensor configurations integrated therein. [Figure 10F] Figures 10A-10I illustrate various aspects of a touch-sensing assembly embodiment, which may feature one or more secondary sensor configurations integrated therein. [Figure 10G] Figures 10A-10I illustrate various aspects of a touch-sensing assembly embodiment, which may feature one or more secondary sensor configurations integrated therein. [Figure 10H] Figures 10A-10I illustrate various aspects of a touch-sensing assembly embodiment, which may feature one or more secondary sensor configurations integrated therein. [Figure 10I] Figures 10A-10I illustrate various aspects of a touch-sensing assembly embodiment, which may feature one or more secondary sensor configurations integrated therein.

[0014] [Figure 11] Figures 11-12, 13A-13F, 14, and 15 illustrate aspects of touch-sensing assembly integration, which can be used by electromechanical systems such as robots to gain further tactile intelligence regarding targeted objects or surfaces. [Figure 12]Figures 11-12, 13A-13F, 14, and 15 illustrate aspects of touch-sensing assembly integration, which can be used by electromechanical systems such as robots to gain further tactile intelligence regarding targeted objects or surfaces. [Figure 13A] Figures 11-12, 13A-13F, 14, and 15 illustrate aspects of touch-sensing assembly integration, which can be used by electromechanical systems such as robots to gain further tactile intelligence regarding targeted objects or surfaces. [Figure 13B] Figures 11-12, 13A-13F, 14, and 15 illustrate aspects of touch-sensing assembly integration, which can be used by electromechanical systems such as robots to gain further tactile intelligence regarding targeted objects or surfaces. [Figure 13C] Figures 11-12, 13A-13F, 14, and 15 illustrate aspects of touch-sensing assembly integration, which can be used by electromechanical systems such as robots to gain further tactile intelligence regarding targeted objects or surfaces. [Figure 13D] Figures 11-12, 13A-13F, 14, and 15 illustrate aspects of touch-sensing assembly integration, which can be used by electromechanical systems such as robots to gain further tactile intelligence regarding targeted objects or surfaces. [Figure 13E] Figures 11-12, 13A-13F, 14, and 15 illustrate aspects of touch-sensing assembly integration, which can be used by electromechanical systems such as robots to gain further tactile intelligence regarding targeted objects or surfaces. [Figure 13F] Figures 11-12, 13A-13F, 14, and 15 illustrate aspects of touch-sensing assembly integration, which can be used by electromechanical systems such as robots to gain further tactile intelligence regarding targeted objects or surfaces. [Figure 14] Figures 11-12, 13A-13F, 14, and 15 illustrate aspects of touch-sensing assembly integration, which can be used by electromechanical systems such as robots to gain further tactile intelligence regarding targeted objects or surfaces. [Figure 15] Figures 11-12, 13A-13F, 14, and 15 illustrate aspects of touch-sensing assembly integration, which can be used by electromechanical systems such as robots to gain further tactile intelligence regarding targeted objects or surfaces.

[0015] [Figure 16A] Figures 16A-16B and 17 illustrate aspects of a configuration in which one or more touch-sensing assemblies may be used, at least partially, to characterize a part of an appendage, such as a part of a user's foot or arm. [Figure 16B] Figures 16A-16B and 17 illustrate aspects of a configuration in which one or more touch-sensing assemblies may be used, at least partially, to characterize a part of an appendage, such as a part of a user's foot or arm. [Figure 17] Figures 16A-16B and 17 illustrate aspects of a configuration in which one or more touch-sensing assemblies may be used, at least partially, to characterize a part of an appendage, such as a part of a user's foot or arm.

[0016] [Figure 18A] Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18B] Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18C]Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18D] Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18E] Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18F] Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18G] Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18H]Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18I] Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18J] Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18K] Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments. [Figure 18L] Figures 18A-18L illustrate aspects of a configuration for integrating one or more touch-sensing assemblies into an advanced system, which may involve electromechanical movement controlled via a robot or the like, and the placement of deformable permeable layers at various positions, such as along the length of various assemblies and around the external shape profile of various assemblies, and circumferentially around elongated instruments.

[0017] [Figure 19A]Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 19B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 20A] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 20B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 20C] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 21A] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 21B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 21C]Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 21D] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 22] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 23A] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 23B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 24A] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 24B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 25A]Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 25B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 26] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 27] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 28A] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 28B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 29A] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 29B]Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 29C] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 29D] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 30A] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 30B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 30C] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 30D] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 30E]Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 30F] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 30G] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 31A] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 31B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 31C] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 31D] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 31E]Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 32A] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 32B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 33A] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 33B] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 34] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement. [Figure 35] Figures 19A-19B, 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch-sensing assemblies may be used to assist in translating a physical engagement back to a user in a workstation, which may be local or remote to the physical engagement.

[0018] [Figure 36] Figures 36, 39, 40, 42, and 46–47 illustrate aspects of medical system and method integration in which one or more touch-sensing assemblies may be used to assist in translating physical engagement at a tissue intervention site back to a user at a workstation, which may be local or remote to the tissue's physical engagement. [Figure 39] Figures 36, 39, 40, 42, and 46–47 illustrate aspects of medical system and method integration in which one or more touch-sensing assemblies may be used to assist in translating physical engagement at a tissue intervention site back to a user at a workstation, which may be local or remote to the tissue's physical engagement. [Figure 40] Figures 36, 39, 40, 42, and 46–47 illustrate aspects of medical system and method integration in which one or more touch-sensing assemblies may be used to assist in translating physical engagement at a tissue intervention site back to a user at a workstation, which may be local or remote to the tissue's physical engagement. [Figure 42] Figures 36, 39, 40, 42, and 46–47 illustrate aspects of medical system and method integration in which one or more touch-sensing assemblies may be used to assist in translating physical engagement at a tissue intervention site back to a user at a workstation, which may be local or remote to the tissue's physical engagement. [Figure 46] Figures 36, 39, 40, 42, and 46–47 illustrate aspects of medical system and method integration in which one or more touch-sensing assemblies may be used to assist in translating physical engagement at a tissue intervention site back to a user at a workstation, which may be local or remote to the tissue's physical engagement. [Figure 47] Figures 36, 39, 40, 42, and 46–47 illustrate aspects of medical system and method integration in which one or more touch-sensing assemblies may be used to assist in translating physical engagement at a tissue intervention site back to a user at a workstation, which may be local or remote to the tissue's physical engagement.

[0019] [Figure 37]Figures 37 and 41 illustrate aspects of gaming or virtual engagement system and method integration in which one or more simulated touch-sensitive assemblies may be used to assist in translating physical engagement in a user interface workstation. [Figure 41] Figures 37 and 41 illustrate aspects of gaming or virtual engagement system and method integration in which one or more simulated touch-sensitive assemblies may be used to assist in translating physical engagement in a user interface workstation.

[0020] [Figure 38A] Figures 38A-38F and 43-45 illustrate an integration aspect in which one or more sensing assemblies may be used to assist in characterizing one or more key working members of an assembly or machine. [Figure 38B] Figures 38A-38F and 43-45 illustrate an integration aspect in which one or more sensing assemblies may be used to assist in characterizing one or more key working members of an assembly or machine. [Figure 38C] Figures 38A-38F and 43-45 illustrate an integration aspect in which one or more sensing assemblies may be used to assist in characterizing one or more key working members of an assembly or machine. [Figure 38D] Figures 38A-38F and 43-45 illustrate an integration aspect in which one or more sensing assemblies may be used to assist in characterizing one or more key working members of an assembly or machine. [Figure 38E] Figures 38A-38F and 43-45 illustrate an integration aspect in which one or more sensing assemblies may be used to assist in characterizing one or more key working members of an assembly or machine. [Figure 38F]Figures 38A-38F and 43-45 illustrate an integration aspect in which one or more sensing assemblies may be used to assist in characterizing one or more key working members of an assembly or machine. [Figure 43] Figures 38A-38F and 43-45 illustrate an integration aspect in which one or more sensing assemblies may be used to assist in characterizing one or more key working members of an assembly or machine. [Figure 44] Figures 38A-38F and 43-45 illustrate an integration aspect in which one or more sensing assemblies may be used to assist in characterizing one or more key working members of an assembly or machine. [Figure 45] Figures 38A-38F and 43-45 illustrate an integration aspect in which one or more sensing assemblies may be used to assist in characterizing one or more key working members of an assembly or machine.

[0021] [Figure 48] Figures 48-50 illustrate integration aspects in which one or more sensing and / or touch-converting interfaces may be used to assist in facilitating the local user perception experience and user-issued commands. [Figure 49] Figures 48-50 illustrate integration aspects in which one or more sensing and / or touch-converting interfaces may be used to assist in facilitating the local user perception experience and user-issued commands. [Figure 50] Figures 48-50 illustrate integration aspects in which one or more sensing and / or touch-converting interfaces may be used to assist in facilitating the local user perception experience and user-issued commands.

[0022] [Figure 51A]Figures 51A-51I illustrate various geometric configurations for tactile sensing, which can be used, for example, to address various geometric shapes of a targeted surface. [Figure 51B] Figures 51A-51I illustrate various geometric configurations for tactile sensing, which can be used, for example, to address various geometric shapes of a targeted surface. [Figure 51C] Figures 51A-51I illustrate various geometric configurations for tactile sensing, which can be used, for example, to address various geometric shapes of a targeted surface. [Figure 51D] Figures 51A-51I illustrate various geometric configurations for tactile sensing, which can be used, for example, to address various geometric shapes of a targeted surface. [Figure 51E] Figures 51A-51I illustrate various geometric configurations for tactile sensing, which can be used, for example, to address various geometric shapes of a targeted surface. [Figure 51F] Figures 51A-51I illustrate various geometric configurations for tactile sensing, which can be used, for example, to address various geometric shapes of a targeted surface. [Figure 51G] Figures 51A-51I illustrate various geometric configurations for tactile sensing, which can be used, for example, to address various geometric shapes of a targeted surface. [Figure 51H] Figures 51A-51I illustrate various geometric configurations for tactile sensing, which can be used, for example, to address various geometric shapes of a targeted surface. [Figure 51I] Figures 51A-51I illustrate various geometric configurations for tactile sensing, which can be used, for example, to address various geometric shapes of a targeted surface.

[0023] [Figure 52]Figures 52-58 and 59A-59B illustrate various aspects of a tactile sensing system configuration, characterized by one or more computing devices or computing systems operably coupled to one or more deformable permeable layers, which may be used to provide geometric information about targeted structures such as riveted surface structures, engine blocks, or other structures and / or surfaces. [Figure 53] Figures 52-58 and 59A-59B illustrate various aspects of a tactile sensing system configuration, characterized by one or more computing devices or computing systems operably coupled to one or more deformable permeable layers, which may be used to provide geometric information about targeted structures such as riveted surface structures, engine blocks, or other structures and / or surfaces. [Figure 54] Figures 52-58 and 59A-59B illustrate various aspects of a tactile sensing system configuration, characterized by one or more computing devices or computing systems operably coupled to one or more deformable permeable layers, which may be used to provide geometric information about targeted structures such as riveted surface structures, engine blocks, or other structures and / or surfaces. [Figure 55] Figures 52-58 and 59A-59B illustrate various aspects of a tactile sensing system configuration, characterized by one or more computing devices or computing systems operably coupled to one or more deformable permeable layers, which may be used to provide geometric information about targeted structures such as riveted surface structures, engine blocks, or other structures and / or surfaces. [Figure 56]Figures 52-58 and 59A-59B illustrate various aspects of a tactile sensing system configuration, characterized by one or more computing devices or computing systems operably coupled to one or more deformable permeable layers, which may be used to provide geometric information about targeted structures such as riveted surface structures, engine blocks, or other structures and / or surfaces. [Figure 57] Figures 52-58 and 59A-59B illustrate various aspects of a tactile sensing system configuration, characterized by one or more computing devices or computing systems operably coupled to one or more deformable permeable layers, which may be used to provide geometric information about targeted structures such as riveted surface structures, engine blocks, or other structures and / or surfaces. [Figure 58] Figures 52-58 and 59A-59B illustrate various aspects of a tactile sensing system configuration, characterized by one or more computing devices or computing systems operably coupled to one or more deformable permeable layers, which may be used to provide geometric information about targeted structures such as riveted surface structures, engine blocks, or other structures and / or surfaces. [Figure 59A] Figures 52-58 and 59A-59B illustrate various aspects of a tactile sensing system configuration, characterized by one or more computing devices or computing systems operably coupled to one or more deformable permeable layers, which may be used to provide geometric information about targeted structures such as riveted surface structures, engine blocks, or other structures and / or surfaces. [Figure 59B]Figures 52-58 and 59A-59B illustrate various aspects of a tactile sensing system configuration, characterized by one or more computing devices or computing systems operably coupled to one or more deformable permeable layers, which may be used to provide geometric information about targeted structures such as riveted surface structures, engine blocks, or other structures and / or surfaces.

[0024] [Figure 60A] Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure. [Figure 60B] Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure. [Figure 60C] Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure. [Figure 60D] Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure. [Figure 60E] Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure. [Figure 61A]Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure. [Figure 61B] Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure. [Figure 61C] Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure. [Figure 61D] Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure. [Figure 61E] Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure. [Figure 61F] Figures 60A-60E and 61A-61F illustrate various aspects of a tactile sensing system configuration that can be detachably coupled to a physical support structure and form a handheld configuration, which may be used to provide geometric information about a targeted structure.

[0025] [Figure 62] Figures 62-65 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects. [Figure 63]Figures 62-65 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects. [Figure 64] Figures 62-65 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects. [Figure 65] Figures 62-65 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects.

[0026] [Figure 66A] Figures 66A–66E illustrate various geometric shapes and surface configurations of objects and / or structures that can be inspected using a deformable permeable layer. [Figure 66B] Figures 66A–66E illustrate various geometric shapes and surface configurations of objects and / or structures that can be inspected using a deformable permeable layer. [Figure 66C] Figures 66A–66E illustrate various geometric shapes and surface configurations of objects and / or structures that can be inspected using a deformable permeable layer. [Figure 66D] Figures 66A–66E illustrate various geometric shapes and surface configurations of objects and / or structures that can be inspected using a deformable permeable layer. [Figure 66E] Figures 66A–66E illustrate various geometric shapes and surface configurations of objects and / or structures that can be inspected using a deformable permeable layer.

[0027] [Figure 67A] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 67B] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 68A] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 68B] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 68C] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 68D] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 69A] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 69B] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 69C] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 69D] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 69E] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component. [Figure 69F] Figures 67A–67B, 68A–68D, and 69A–69F illustrate various aspects of the configuration, which features an expandable, deformable, and permeable layer component.

[0028] [Figure 70A] Figures 70A-70E and 71A-71D illustrate aspects of a configuration for utilizing a deformable permeable layer component within an instrument, such as an elongated instrument, or its distal portion, for measurement and / or characterization. [Figure 70B] Figures 70A-70E and 71A-71D illustrate aspects of a configuration for utilizing a deformable permeable layer component within an instrument, such as an elongated instrument, or its distal portion, for measurement and / or characterization. [Figure 70C] Figures 70A-70E and 71A-71D illustrate aspects of a configuration for utilizing a deformable permeable layer component within an instrument, such as an elongated instrument, or its distal portion, for measurement and / or characterization. [Figure 70D] Figures 70A-70E and 71A-71D illustrate aspects of a configuration for utilizing a deformable permeable layer component within an instrument, such as an elongated instrument, or its distal portion, for measurement and / or characterization. [Figure 70E] Figures 70A-70E and 71A-71D illustrate aspects of a configuration for utilizing a deformable permeable layer component within an instrument, such as an elongated instrument, or its distal portion, for measurement and / or characterization. [Figure 71A] Figures 70A-70E and 71A-71D illustrate aspects of a configuration for utilizing a deformable permeable layer component within an instrument, such as an elongated instrument, or its distal portion, for measurement and / or characterization. [Figure 71B] Figures 70A-70E and 71A-71D illustrate aspects of a configuration for utilizing a deformable permeable layer component within an instrument, such as an elongated instrument, or its distal portion, for measurement and / or characterization. [Figure 71C] Figures 70A-70E and 71A-71D illustrate aspects of a configuration for utilizing a deformable permeable layer component within an instrument, such as an elongated instrument, or its distal portion, for measurement and / or characterization. [Figure 71D]Figures 70A-70E and 71A-71D illustrate aspects of a configuration for utilizing a deformable permeable layer component within an instrument, such as an elongated instrument, or its distal portion, for measurement and / or characterization.

[0029] [Figure 72A] Figures 72A-72C illustrate aspects of the system configuration in which deformable permeable layer elements may be utilized for measurement and / or characterization. [Figure 72B] Figures 72A-72C illustrate aspects of the system configuration in which deformable permeable layer elements may be utilized for measurement and / or characterization. [Figure 72C] Figures 72A-72C illustrate aspects of the system configuration in which deformable permeable layer elements may be utilized for measurement and / or characterization.

[0030] [Figure 73A] Figures 73A-73C illustrate the opened or expanded side view of the expandable, deformable, and permeable layer configuration. [Figure 73B] Figures 73A-73C illustrate the opened or expanded side view of the expandable, deformable, and permeable layer configuration. [Figure 73C] Figures 73A-73C illustrate the opened or expanded side view of the expandable, deformable, and permeable layer configuration.

[0031] [Figure 74A] Figures 74A–74C illustrate aspects of the procedure in which an expandable, deformable, and permeable layer may be used to characterize and / or measure elongated defects, holes, or conduit sides. [Figure 74B] Figures 74A–74C illustrate aspects of the procedure in which an expandable, deformable, and permeable layer may be used to characterize and / or measure elongated defects, holes, or conduit sides. [Figure 74C] Figures 74A–74C illustrate aspects of the procedure in which an expandable, deformable, and permeable layer may be used to characterize and / or measure elongated defects, holes, or conduit sides.

[0032] [Figure 75] Figures 75-82 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects. [Figure 76] Figures 75-82 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects. [Figure 77] Figures 75-82 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects. [Figure 78] Figures 75-82 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects. [Figure 79] Figures 75-82 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects. [Figure 80] Figures 75-82 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects. [Figure 81] Figures 75-82 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects. [Figure 82] Figures 75-82 illustrate various process or method configurations featuring a deformable permeable layer, which are employed for the geometric characterization of one or more objects.

[0033] [Figure 83A] Figures 83A and 83B illustrate a side view of a system featuring a deformable permeable layer that is controllably movable to engage a targeted object or surface via a robotic arm or the like. [Figure 83B]Figures 83A and 83B illustrate a side view of a system featuring a deformable permeable layer that is controllably movable to engage a targeted object or surface via a robotic arm or the like.

[0034] [Figure 84A] Figures 84A and 84B illustrate aspects of a system featuring a deformable permeable layer that is controllably movable to address a targeted object or surface via an electromechanical gantry assembly, etc. [Figure 84B] Figures 84A and 84B illustrate aspects of a system featuring a deformable permeable layer that is controllably movable to address a targeted object or surface via an electromechanical gantry assembly, etc.

[0035] [Figure 85] Figures 85-94 illustrate aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects. [Figure 86] Figures 85-94 illustrate aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects. [Figure 87] Figures 85-94 illustrate aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects. [Figure 88] Figures 85-94 illustrate aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects. [Figure 89] Figures 85-94 illustrate aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects. [Figure 90] Figures 85-94 illustrate aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects. [Figure 91] Figures 85-94 illustrate aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects. [Figure 92] Figures 85-94 illustrate aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects. [Figure 93] Figures 85-94 illustrate aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects. [Figure 94] Figures 85-94 illustrate aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects.

[0036] [Figure 95A] Figures 95A–95C illustrate aspects of an illustrative scenario in which a configuration featuring one or more deformable permeable layers may be used to characterize complex objects. [Figure 95B] Figures 95A–95C illustrate aspects of an illustrative scenario in which a configuration featuring one or more deformable permeable layers may be used to characterize complex objects. [Figure 95C] Figures 95A–95C illustrate aspects of an illustrative scenario in which a configuration featuring one or more deformable permeable layers may be used to characterize complex objects.

[0037] [Figure 96] Figure 96 illustrates aspects of a process or method configuration featuring a deformable permeable layer, employed for the geometric characterization of one or more objects.

[0038] [Figure 97]Figure 97 illustrates a side view of a system configuration featuring a deformable permeable layer that is controllably movable via a robotic arm or the like to engage a targeted object or surface, and may be coupled thereto and equipped with one or more measuring probes to assist in sensing operational characteristics such as discrete point contact.

[0039] [Figure 98A] Figures 97A-97C illustrate various aspects of system-level configurations for the use of a deformable permeable layer when characterizing a targeted object or surface. [Figure 98B] Figures 97A-97C illustrate various aspects of system-level configurations for the use of a deformable permeable layer when characterizing a targeted object or surface. [Figure 98C] Figures 97A-97C illustrate various aspects of system-level configurations for the use of a deformable permeable layer when characterizing a targeted object or surface.

[0040] [Figure 99A] Figures 99A–99C illustrate various aspects of illumination configurations for the use of a deformable transmissive layer when characterizing a targeted object or surface. [Figure 99B] Figures 99A–99C illustrate various aspects of illumination configurations for the use of a deformable transmissive layer when characterizing a targeted object or surface. [Figure 99C] Figures 99A–99C illustrate various aspects of illumination configurations for the use of a deformable transmissive layer when characterizing a targeted object or surface.

[0041] [Figure 100A] Figures 100A-100E illustrate various aspects of an integrated configuration featuring one or more lighting control layers for illumination. [Figure 100B] Figures 100A-100E illustrate various aspects of an integrated configuration featuring one or more lighting control layers for illumination. [Figure 100C]Figures 100A-100E illustrate various aspects of an integrated configuration featuring one or more lighting control layers for illumination. [Figure 100D] Figures 100A-100E illustrate various aspects of an integrated configuration featuring one or more lighting control layers for illumination. [Figure 100E] Figures 100A-100E illustrate various aspects of an integrated configuration featuring one or more lighting control layers for illumination.

[0042] [Figure 101A] Figures 101A-101B illustrate orthographic projections of an illumination control layer configuration having an operablely coupled illumination source. [Figure 101B] Figures 101A-101B illustrate orthographic projections of an illumination control layer configuration having an operablely coupled illumination source.

[0043] [Figure 102A] Figures 102A-102B illustrate orthographic projections of an illumination control layer configuration having two operablely coupled illumination sources. [Figure 102B] Figures 102A-102B illustrate orthographic projections of an illumination control layer configuration having two operablely coupled illumination sources.

[0044] [Figure 102C] Figure 102C illustrates a lighting control layer configuration having four operablely coupled lighting sources.

[0045] [Figure 102D] Figure 102D illustrates a lighting control layer configuration having three operablely coupled lighting sources.

[0046] [Figure 102E] Figure 102E illustrates a lighting control layer configuration having two lighting sources operably coupled to the lighting control layer, which in the depicted diagram have a square or rectangular shape.

[0047] [Figure 102F] Figure 102F illustrates a lighting control layer configuration having six lighting sources operably coupled to the lighting control layer, which in the depicted diagram have a square or rectangular shape.

[0048] [Figure 103A] Figures 103A-103C illustrate illumination control layer shapes, which may be, for example, planar, substantially planar, arc-shaped or curved, saddle-shaped, or have at least one convex surface, while also being configured to have at least one planar or substantially planar surface. [Figure 103B] Figures 103A-103C illustrate illumination control layer shapes, which may be, for example, planar, substantially planar, arc-shaped or curved, saddle-shaped, or have at least one convex surface, while also being configured to have at least one planar or substantially planar surface. [Figure 103C] Figures 103A-103C illustrate illumination control layer shapes, which may be, for example, planar, substantially planar, arc-shaped or curved, saddle-shaped, or have at least one convex surface, while also being configured to have at least one planar or substantially planar surface.

[0049] [Figure 104A] Figures 104A-104G illustrate various aspects of a sensing configuration integrated into an elongated assembly for use in applications such as fingertip or fingertip sensing, synthetic fingers, and / or robots, synthetic hands, or grappling. [Figure 104B] Figures 104A-104G illustrate various aspects of a sensing configuration integrated into an elongated assembly for use in applications such as fingertip or fingertip sensing, synthetic fingers, and / or robots, synthetic hands, or grappling. [Figure 104C] Figures 104A-104G illustrate various aspects of a sensing configuration integrated into an elongated assembly for use in applications such as fingertip or fingertip sensing, synthetic fingers, and / or robots, synthetic hands, or grappling. [Figure 104D]Figures 104A-104G illustrate various aspects of a sensing configuration integrated into an elongated assembly for use in applications such as fingertip or fingertip sensing, synthetic fingers, and / or robots, synthetic hands, or grappling. [Figure 104E] Figures 104A-104G illustrate various aspects of a sensing configuration integrated into an elongated assembly for use in applications such as fingertip or fingertip sensing, synthetic fingers, and / or robots, synthetic hands, or grappling. [Figure 104F] Figures 104A-104G illustrate various aspects of a sensing configuration integrated into an elongated assembly for use in applications such as fingertip or fingertip sensing, synthetic fingers, and / or robots, synthetic hands, or grappling. [Figure 104G] Figures 104A-104G illustrate various aspects of a sensing configuration integrated into an elongated assembly for use in applications such as fingertip or fingertip sensing, synthetic fingers, and / or robots, synthetic hands, or grappling.

[0050] [Figure 105A] Figures 105A and 105B illustrate sample results from using a deformable permeable layer sensing configuration. [Figure 105B] Figures 105A and 105B illustrate sample results from using a deformable permeable layer sensing configuration. [Modes for carrying out the invention]

[0051] Detailed explanation Referring to Figure 7A, the digital touch-sensing assembly (146) features a deformable transparent layer (110), shown and operably coupled to an optical element (108), which is illuminated by one or more interconnected light sources (116, 122) and positioned within the field of view of the imaging device (106). The housing (118) is configured to maintain the relative positioning of the components and expose them to the touch-sensing contact surface (120). The interface film (100), which may comprise a substantially thin layer of a relatively low bulk modulus polymer material, fixedly attached or detachably bonded, may be positioned, operably bonded to, or constitute a part thereof, a deformable permeable layer for direct contact between another object and a digital touch sensing assembly (146) for touch determination and characterization, and in the configuration where the interface film (100) is bonded to or constitutes a part thereof, the final outer touch contact surface (120) will be the outer side of such interface film (100). Generally, aspects of preferred digital touch-sensing assembly (146) configurations, characterized by an elastomer deformable permeable layer material, are described, for example, in U.S. Patents No. 10,965,854, No. 10,574,944, No. 9,127,938, and No. 8,411,140, ​​as well as U.S. Patent Application Publications No. 20140253717 and No. 20140104395.As shown in Figure 7A, the depicted digital touch-sensing assembly (146) may feature a gap or void (114) which may contain an optically transparent material (similar to that of the optical element 108, having a refractive index), air, or a special gas such as an inert gas, and which is geometrically configured to place the sides of an optical element (108) and / or a deformable transparent layer (110) within a desired proximity to the imaging device (106), and which may comprise an imaging sensor such as a digital camera chip, a single photosensitive element (such as a photodiode), or an array of photosensitive elements, which may be configured to have a field of view and depth of field facilitated by the geometric gap or void (114) (i.e., the gap or void 114 may be positioned to adapt to the field of view and / or depth of view with respect to a particular imaging device 106). In various embodiments, the optical element (108) may comprise a substantially rigid material, a material with a known modulus of elasticity, or a material with a known structural modulus of elasticity (i.e., given the unloaded and loaded shapes, the load profile may be determined by given structural modulus information relating to the shape). Various suitable optical elements (108) may define an external shape, including, for example, a cylindrical, cubic, and / or rectangular parallelepiped. Various illumination sources may be coupled to one or more sidewall surfaces, as shown in the figures and described below, thereby defining the optical element (108). In another embodiment, the optical element (108) may be configured to be deformable or conformable so as to minimize the impact of the rigidity of such structure on other associated elements (i.e., impulse loads such as force / delta time may be minimized using greater impact compliance, and furthermore, a lower structural modulus at the contact interface may maintain greater surface contact across a given surface, such as with undulations or geometric features).

[0052] Also shown in Figure 7A is a computing device or system (104), which may comprise a computer, microcontroller, field-programmable gate array, application-specific integrated circuit, or equivalent, which is operably coupled (128) to an imaging device (106) and also operably coupled (124, 126) to one or more light sources (30) and configured to facilitate control of these devices when collecting data relating to touches to the deformable transparent layer (110). For example, in one embodiment, the light sources (116, 122) each comprise light-emitting diodes ("LEDs") operably coupled (124, 126) to the computing device (104) using electronic leads (124, 126), as shown in Figure 7A, and the imaging device (106) comprises a digital camera sensor chip operably coupled to the computing device using electronic leads (128). The power supply (102) may be operably coupled to the computing device (104) and provide power to the computing device (104), or it may be configured to controllly provide power to interconnected devices such as the imaging device (106) and light sources (116, 122) through their couplings (128, 124, 126, respectively). As shown in Figure 7A, the isolation (640) is depicted and these coupling interfaces (128, 124, 126) may be short or relatively long (i.e., the digital touch sensing assembly 146 may be located remotely from the computing device 104), and may be a direct physical connection, or data transmission through a wired or wireless interface via an optical / optical networking protocol, or a wireless networking protocol such as Bluetooth® or an 802.11-based configuration, which may be facilitated by additional computing and power resources local to the digital touch sensing assembly (146).

[0053] Referring to Figure 7B, a configuration similar to that shown in Figure 7A is illustrated, but the deformable permeable layer (110) in Figure 7B comprises one or more bladder or enclosed volume (112), which may be occupied by a fluid (such as a liquid or gas that can be physically handled in the form of a fluid). In one embodiment, for example, the deformable permeable layer (110) may comprise several separately controllable inflatable segments or sub-volumes, and may have a cross-sectional shape selected to provide specific mechanical properties under load, such as a controllable honeycomb type cross-sectional shape configuration. As stated above, the deformable permeable layer (110) may comprise one or more materials selected to conform to the touch-sensing paradigm in terms of bulk and / or Young's modulus. In other words, to sense relatively low loads in digital touch scenarios such as interfacing with soap bubbles or the surface of a photosynthetic plant leaf, relatively low modulus materials (i.e., generally locally flexible / deformable and not rigid), such as elastomers, may be used for the deformable permeable layer (110) and / or outer interface membrane (100), as described in the aforementioned incorporated references, which may be detachable as described above. The outer interface membrane (100) may comprise an assembly of relatively thin and sequentially detachable membranes, such that they can be sequentially removed, for example, in a "peel-off" manner, if they become soiled with mud or dust. In embodiments such as that shown in Figure 7B, in which the deformable permeable layer (110) comprises at least a temporarily captured volume of liquid or gas, the gas or liquid, along with its pressure, may be modulated to address a desired bulk modulus and sensitivity of the overall deformable permeable layer (110) (for example, the pressure and / or volume may be modulated with respect to one or more bladder segments 112), thereby generally altering the functional modulus of the deformable permeable layer 110.

[0054] Referring to Figure 7C, a configuration similar to that in Figure 7A is shown, illustrating that the gap (130) between the imaging device (106) and the optical element (108) can be reduced and even eliminated depending on the optical layout of the imaging device (106), which can be coupled with refractive and / or diffractive optical systems and change properties such as the focal length of the imaging device (106).

[0055] Referring to Figure 7D, a configuration similar to that of Figure 7A is illustrated, but the configuration of Figure 7D illustrates that one or more light sources may more closely resemble an optical emitter (117, 123), which is configured to emit light originating elsewhere, such as being coupled to one or more optical LED light sources, which is directly coupled to a computing device (104) and configured to transmit light through optical fibers, "optical pipes," or waveguides, via optical transmission coupling members (132, 134), which may be configured to pass photons from such sources to the emitter (117, 123) as efficiently as possible, via total internal reflection, etc.

[0056] Similarly, referring to Figure 7E, a configuration similar to that of Figure 7D is illustrated, in which an imaging device (107) comprises a capture optical system selected to collect photons and transmit them back to the image sensor through a light-transmitting coupling member (138) such as a waveguide or one or more optical fibers, which may be located within or coupled to a computing device (104) or other structure, and which may reside separately from a digital touch sensing assembly (146).

[0057] Referring to Figures 7F-7H, various aspects of the configuration of a digital touch-sensing assembly (146) are illustrated, featuring a deformable transmissive layer (110) which may be used to characterize surface-to-surface interactions. For example, referring to Figure 7F in a simplified illustrative embodiment, a computing system or device (104), operably coupled (136) to a power source (102), may be used to control light (1002) or other emissions from an illumination source (116) through a control coupling (124), which may be wired or wireless, and which may be directed into the deformable transmissive layer (110). The deformable transparent layer (110) may be pressed against at least a portion of an interfaced object (1004), such as the edge of a coin (1006), and may be operably coupled to a computing system (104) (128; via wired or wireless connectivity, etc.) based on the interaction between illumination (1002) and the deformable transparent layer (110), and a detector such as an image acquisition device (CCD or CMOS device, etc.) may be configured to detect at least a portion of the light directed from the deformable transparent layer. In other words, a computing system may be configured to operate the detector to detect at least a portion of the light directed from the deformable transmissive layer, and to determine, at least partially, the surface orientation of an object interfaced with the deformable transmissive layer at a position along the interface, based on the interaction of the first illumination light with the deformable transmissive layer (e.g., optically coupled with an efficient transmission interface), and a detector configured to detect light from within at least a portion of the deformable transmissive layer, to at least partially, determine the surface orientation of an object interfaced with the deformable transmissive layer at a position along the interface, and to use the determined surface orientation to characterize the geometric profile of at least one side of the interfaced object interfaced with the interface film.Referring to Figure 7G, as will be discussed further below, an interface film (100) may be interposed between the interfaced object (1004) and the deformable permeable layer (110), and such an interface film may have an elastic modulus similar to or different from that of the deformable permeable layer. Preferably, an efficient coupling is created between the deformable permeable layer and the film so that shear and principal or normal loads are efficiently transferred between these structures. Referring back to Figure 7A, an embodiment is illustrated, which includes an optical element (108) which may be configured to assist in the precise distribution of light or other radiation through the various parts of the assembled system. The optical element may comprise a substantially rigid material, which is highly permeable and comprises an upper surface, a bottom surface, and sides defined between them, and may form a three-dimensional shape such as a cylindrical, cuboid, and / or cuboidal shape. The optical element (108) to be depicted may be illuminated by one or more interconnected light sources (116, 122) and positioned within the field of view of the imaging device (106). The housing (118) is configured to maintain the positioning of the component relative to the interconnection and interface film (100) as described above, which may comprise a substantially thin layer that is fixedly attached or detachably bonded, for example, a relatively low bulk modulus polymer material, which may be positioned for direct contact between other objects and the digital touch sensing assembly (146) for touch determination and characterization. Preferably, the deformable permeable layer and / or interface film comprises an elastomer material such as silicone, urethane, polyurethane, thermoplastic polyurethane (TPU), thermoplastic elastomer (TPE), plastisol, polyvinyl chloride, polyisoprene, or fluoroelastomer. Other elastomers with lower light and / or radiation transmission efficiencies, such as natural rubber, neoprene, ethylene propylene diene monomer (EPDM) rubber, butyl rubber, nitrile rubber, styrene-butadiene rubber (SBR), Viton, fluorosilicone, and polyacrylate, may also be used.The deformable permeable layer may comprise a synthetic material having a pigment material, such as a metal oxide (e.g., iron oxide, zinc oxide, aluminum oxide, and / or titanium dioxide), a metal pigment or metal nanoparticles (e.g., silver nanoparticles and / or aluminum nanoparticles), or other molecules configured to interact differentially with introduced light or radiation, such as dyes dispersed within the elastomer matrix. The pigment material may be configured to provide illumination reflectance exceeding that of the elastomer matrix. The deformable permeable layer is bounded directly to the interface film by a bottom surface, an upper surface closest to the detector, and the permeable layer thickness between them, with the pigment material dispersed adjacent to the bottom surface within the permeable layer thickness to provide optimized illumination reflectance adjacent to the bottom surface. In general, aspects of a preferred digital touch-sensing assembly (146) configuration featuring an elastomer deformable transparent layer material are described, for example, in U.S. Patents 10,965,854, 9,127,938, and 8,411,140. As shown in Figure 3C, the depicted digital touch-sensing assembly (146) may also feature a gap or void (114) which may contain an optically transparent material (similar to that of the optical element 108, having a refractive index, etc.), air, or a special gas such as an inert gas, which is geometrically configured to place an optical element (108) and / or a side of the deformable transparent layer (110) within a desired proximity to the imaging device (106), and which may comprise an imaging sensor such as a digital camera chip, a single photosensitive element (such as a photodiode), or an array of photosensitive elements, which may be configured to have a field of view and depth of field facilitated by the geometric gap or void (114). In another embodiment, the optical element (108) may be configured to be deformable or conformable so as to minimize the impact of the rigidity of such structure on other associated elements.

[0058] Also shown in Figure 7A is a computing device or system (104), which may comprise a computer, microcontroller, field-programmable gate array, application-specific integrated circuit, or equivalent, which is operably coupled to an imaging device (106) and to one or more light sources (116, 122) and configured to facilitate control of these devices when collecting data relating to touches to the deformable transparent layer (110). For example, in one embodiment, the light sources (116, 122) each comprise light-emitting diodes ("LEDs") operably coupled to a computing device using electronic leads (128), as shown in Figure 7A, and the imaging device (106) comprises a digital camera sensor chip operably coupled to a computing device using electronic leads. The power supply (102) may be operably coupled to the computing device (104) and supply power to the computing device (104), or it may be configured to controllly supply power to interconnected devices such as the imaging device (106) and light sources (116, 122) through their couplings (128, 124, 126, respectively). As shown in Figure 7A(640), these coupling interfaces (128, 124, 126) may be short or relatively long (i.e., the digital touch sensing assembly 146 may be located remotely from the computing device 104) and may be a direct physical connection or data transmission through a wired or wireless interface via an optical / optical networking protocol or a wireless networking protocol such as Bluetooth® or an 802.11-based configuration, which may be facilitated by additional computing and power resources local to the digital touch sensing assembly (146).Referring to Figure 7H, the partial schematic diagram illustrates that the computing system (104) may be operably coupled to two, three, or more different illuminators (116, 122, 1010) via wired or wireless control leads, etc. (124, 126, 1012), and these illuminators may be configured to emit at different wavelengths and / or with different polarizations, and as depicted, may be configured to emit from different orientations relative to the optical element (108) and associated deformable transparent layer (110), allowing for further data relating to geometric profiling.

[0059] Referring to Figure 8, as described in the aforementioned incorporated reference (U.S. Patent No. 10,965,854), the deformable permeable layer or member (110) may have various geometric shapes and does not need to be planar or molded into a rectangular parallelepiped or a variation thereof. For example, the deformable permeable layer or member (110) may be curved, convex (144), saddle-shaped, and equivalent, and may be customized for various specific touch sensing scenarios. For example, multiple assemblies (146) with a convex deformable permeable layer (110), such as those shown in Figure 8, may be coupled to the gripping interface of a robotic gripper / hand to facilitate touch sensing / determination regarding the grasped item in a manner similar to the paradigm of inter-joint skin segments of a human hand grasping an object. The assembly (146) configuration in Figure 8 features a housing geometric shape (142) and coupling features (140) that assist in detachable attachment to other components.

[0060] Referring to Figure 9A, multiple digital touch-sensing assemblies (146) may be used together to sense a larger surface area (150) of an object (148). Each such assembly (146; five are illustrated in Figure 9A) may be operably coupled to one or more computing devices (104)(152, 154, 156, 158, 160) via electronic leads (which may be interrupted, for example, by wireless connectivity as described above), as shown, and thus configured to exchange data and facilitate the transmission of power, light, and control and sensing information.

[0061] Referring to Figure 9B, a larger number of (162) digital touch sensing assemblies (146) than those in Figure 9A may be used to monitor digital touches with two or more surfaces of an object, partially or completely, surrounding an object. Each of the 30 depicted digital touch sensing assemblies (146) shown in Figure 9B may be operably coupled to the same or different computing devices (104) and coupled leads, and may be combined or coupled as shown in Figure 9B to form a single combined coupled lead assembly (164).

[0062] Referring to Figure 10A, optional geometric separation (640) is shown between various components such as a digital touch-sensing assembly (146) and a computing device (104). It is important to note that these components may also be housed together and connected to other systems, components, and devices via wireless transceivers (166) designed to function with known communication standards such as IEEE 802.11, the so-called "Wi-Fi" standard, and / or wireless connectivity and Bluetooth® 4.x and Bluetooth® 5. Furthermore, the interconnected (136; via direct wire leads, etc.) power source (102) components depicted may have one or more connections (via wired or wireless, inductive power transfer, etc.) to one or more batteries or other power sources to provide further power supply and / or charging to the integrated power source (102) components. The various embodiments described herein, relating to miniaturized or miniaturizable configurations, are desirable to facilitate such system integration using connectivity alternatives that assist in integration into other systems, such as those in automobiles, and that meet or cooperate with known standards. For example, in configurations of various embodiments in which a touch-sensing system, such as that depicted in Figure 10A, can be miniaturized and packaged within a housing, and the connectivity configuration can be designed for relatively simple integration into or with other systems, such system configurations can be considered toward "Internet of Things" integration capabilities, and various devices are expected to be brought into cooperation with other connected and integrated systems relatively easily.

[0063] Referring again to Figure 10A, a digital touch-sensing assembly (146) is shown, which is similar to the one described with reference to Figure 7A, but also features a number of additional sensing capabilities or “secondary sensor” elements selected to enhance the general capabilities of the assembly, such as by providing sensing data from one or more additional sensing subsystems, which are generally co-located with touch-sensing capabilities provided by a deformable permeable layer, and which may present their own levels of uncertainty and error sensing, so-called “sensor fusion” techniques can be used to improve the overall capabilities of the integrated configuration, such as by taking advantage of uncorrelated errors between the various sensing subsystems. For example, when a digital touch sensor based on a deformable transparent layer (110) potentially indicates contact with another object, but data from an integrated inertial measurement unit (or "IMU") (such as accelerometer or gyroscope data from one or more accelerometers or gyroscopes that may be equipped with such an IMU), a LiDAR subsystem (such as point cloud data relating to the area where contact is said to occur), and an imaging device (such as a camera providing image data relating to the area where contact is said to occur) provides additional contradictory data with uncorrelated measurement / decision errors that establish that the digital touch sensor is not in contact, there is a reasonable likelihood that the digital touch sensor is not in contact. The concept of uncorrelated error with respect to other measurement / decision subsystems is important because, if all other measurement / decision subsystems have the same correlation error, they can contribute to a certain level of redundancy or improved measurement, field of view, etc., but they can have limitations based on similar errors. For example, mounting three pitot tubes on an airplane wing may provide some redundancy and additional measurement compared to a single pitot tube, but if they are all flown through frozen rain and rendered ineffective by the same correlation error, the airplane would probably prefer to rely on subsystems with some degree of uncorrelated error, such as a compass, GPS, orbital planning.Therefore, the concept of using multiple sensors, with at least some degree of uncorrelated error, provides value and can be referred to as a form of "sensor fusion" through the usefulness of two or more sensors. Furthermore, as mentioned above, multiple sensors may be aggregated, such as by coupling similar or different sensors adjacent to each other along a given surface or side of a structural element, to complement and extend the geometric reach of the sensing paradigm. Therefore, referring back to Figure 10A, a selection of additional sensing subsystems (IMU 172, capacitive touch sensing 174, resistive touch sensing 176, LIDAR sensing 178, strain or stretch sensing 180, load sensing 182, temperature sensing 184, additional image sensing 186), with at least some degree of uncorrelated error, are shown operably coupled as part of the integrated system configuration depicted (188, 190, 192, 194, 196, 198, 200, 202, respectively, representing connectivity leads such as conductive wire leads, which may be coupled to a communication / connection bus 170 as shown in Figure 10A, which may be directly interconnected with computing device 104 168).

[0064] For illustrative purposes, Figures 10B-10I depict various embodiments, and further details of various subsystem integrations can also be explored.

[0065] Referring to Figure 10B, an embodiment is shown in which a digital touch sensing assembly (146) is integrated with an interconnected IMU (172). The IMU (172) may comprise one or more accelerometers and one or more gyroscopes, and may be fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and operably coupled to the computing device (104) via wire leads (188; shown coupled to a communication bus 170, which is operably coupled to the computing device (104) via wire leads 168, etc.). The computing device (104) operates the imaging device (106) and illumination sources (116, 122) and is physically interfaced with one or more objects such as a contact interface (120). By utilizing the deformable transparent layer (110), it not only facilitates touch sensing but may also operate the IMU (172) to capture data on angular and axial acceleration, and / or changes in the position or orientation of the housing (118), which may be associated with contact with an external object. In one embodiment, for example, the integrated system may be configured to increase the frame rate for touch sensing through the deformable transparent layer (110) when an unexpected change in axial or angular acceleration is detected using IMU data and knowledge of the predicted motion and acceleration of the housing (118). In other words, when a digital touch sensing assembly (146) is coupled to an electromechanical moving system such as a robotic arm or robotic manipulator (e.g., 234 as shown in Figure 11), and a computing system (104) is integrated to receive information regarding timing, direction / orientation, and kinematics of movement commands for the electromechanical moving system, it can be configured to separate expected acceleration from unexpected ones and treat unexpected ones as potential contact with external objects, which can further be explored using improved frame rates, computing, and general digital touch sensing through a deformable permeable layer (110).

[0066] Referring to Figure 10C, an embodiment is shown in which a digital touch sensing assembly (146) is integrated with an interconnected capacitive sensing subsystem, the digital touch sensing assembly (146), the digital touch sensing assembly (146), and the capacitive sensing subsystem (174) which is operably coupled to the capacitive sensing element (206) via wire leads (204), etc., and which may be integrated into a deformable transparent layer and may have a grid or multiple cells, similar to how some smartphones or other touchscreen interfaces are configured to detect contact based on detected capacitance, and which may be configured to facilitate enhanced contact sensing based on capacitance sensed between the sensing element (206) and other objects. The capacitive sensing controller (174) may comprise one or more amplifiers and may be fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and operably coupled to the computing device (104) via wire leads (190; shown coupled to a communication bus 170, which is operably coupled to the computing device (104) via wire leads 168, etc.). The computing device (104) operates the imaging device (106) and illumination sources (116, 122) and is physically interfaced with one or more objects such as a contact interface (120). By utilizing the deformable transparent layer (110), it not only facilitates touch sensing but may also operate a capacitive sensing controller (174) to capture data on detected changes in capacitance near the sensing element (206), which may be associated with contact with an external object, for example. In one embodiment, for example, the integrated system may be configured to increase the frame rate for touch sensing through the deformable transparent layer (110) when a change in capacitance is detected using sensed capacitance data relating to the sensing element (206). In other words, the system may be configured to utilize the uncorrelated errors of both capacitive and deformable transparent layer (110)-based touch sensing to provide an optimized touch sensing output depending on the determination that there is at least some indication of contact on or near the sensing element (206).In other variations, various combinations of sensors, including those with uncorrelated errors, may be used along with various aspects of spatial separation from one another, since the resolution and / or temporal response requirements cannot be identical at each location with a given implementation.

[0067] Referring to Figure 10D, an embodiment is shown in which a digital touch sensing assembly (146) is integrated with an interconnected resistive sensing subsystem, the digital touch sensing assembly (146), which is operably coupled to a resistive sensing element (208) via wire leads (210), etc., and may be integrated into a deformable transparent layer (110) and configured to facilitate enhanced contact sensing based on the resistance sensed between the sensing element (208) and other objects, which may comprise a grid or a plurality of cells, similar to how some smartphones or other touchscreen interfaces are configured to detect contact based on detected resistance. The resistive sensing controller (176) may comprise one or more amplifiers and may be fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and operably coupled to a computing device (104) via wire leads (192; shown coupled to a communication bus 170, which is operably coupled to a computing device (104) via wire leads 168, etc.). The computing device (104) operates the imaging device (106) and illumination sources (116, 122) and is physically interfaced with one or more objects such as a contact interface (120). By utilizing the deformable transparent layer (110), it not only facilitates touch sensing but may also operate a resistive sensing controller (176) to capture data on detected changes in capacitance near the sensing element (208), which may be associated with contact with an external object, for example. In one embodiment, for example, the integrated system may be configured to increase the frame rate for touch sensing through the deformable transparent layer (110) when a change in capacitance is detected using sensed capacitance data relating to the sensing element (208). In other words, the system may be configured to provide an optimized touch sensing output by utilizing the uncorrelated errors of both resistive and deformable transparent layer (110) based touch sensing, depending on the determination that there is at least some indication of contact in or near the sensing element (208).

[0068] Referring to Figure 10E, an embodiment is illustrated in which a digital touch sensing assembly (146) is integrated with an interconnected LIDAR sensor (178), such as one available from Hokuyo Automatic USA Corporation. The LIDAR sensor (178) is fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and may be operably coupled to a computing device (104) via wire leads (194; shown coupled to a communication bus 170, which is operably coupled to a computing device (104) via wire leads 168, etc.). The computing device (104) operates an imaging device (106) and illumination sources (116, 122) and is physically interfaced with one or more objects such as a contact interface (120). By utilizing the deformable transparent layer (110), it not only facilitates touch sensing but may also operate a LiDAR sensor (178) and capture data about objects within the LiDAR sensor (178) field of view (212), such as point clouds of nearby surfaces and objects. In one embodiment, for example, the integrated system may be configured to increase the frame rate for both LiDAR (178) and touch sensing through the deformable transparent layer (110) when an unexpected change in the LiDAR (178) field of view (212; which is preferably oriented to match, at least somewhat, the position and orientation of the associated deformable transparent layer 110) is detected using LiDAR (178) data. In other words, as the deformable transparent layer (110) begins to approach another object, such as one detected by a change in the point cloud detected by a LIDAR (178) system, the deformable transparent layer (110) and its associated computing and imaging capabilities may transition to an enhanced functional mode for detecting and characterizing any touch / contact.

[0069] Referring to Figure 10F, an embodiment is illustrated in which a digital touch sensing assembly (146) is integrated with interconnected strain or stretch sensors (180). The strain sensor (180) may include one or more stretch sensing elements (216) within a strain gauge, etc., the electrical resistance of which may correlate with stretching. Such stretch sensing elements (216) may be integrated or embedded within a deformable permeable layer (110), and the strain controller (180) may be fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and operably coupled to a computing device (104) via wire leads (196; shown coupled to a communication bus 170, which is operably coupled to a computing device (104) via wire leads 168, etc.). The computing device (104) operates the imaging device (106) and illumination sources (116, 122) and is physically interfaced with one or more objects such as a contact interface (120). By utilizing the deformable transparent layer (110), it not only facilitates touch sensing but may also be configured to operate a strain controller (180) to capture data related to strain or stretching, which may be associated with, for example, contact with an external object. The stretch detection element or a group of elements may comprise a grid or network and may be operably coupled to the strain controller (180) via one or more wire leads (214), etc. In one embodiment, for example, the integrated system may be configured to optimize touch sensing size determination through the deformable transparent layer (110) as changes in stretching are detected using strain sensor data. For example, if a deformable permeable layer (110) is moved over a ridge in the surface, the magnitude of the ridge, as determined using the deformable permeable layer (110), may be compared with a change in contact surface deflection detected using strain sensors (180, 216), thereby providing two data sources for such a determination, with at least some degree of uncorrelated measurement / determination error.

[0070] Referring to Figure 10G, an embodiment is illustrated in which a digital touch sensing assembly (146) is integrated with an interconnected load sensor (182). The load sensor (182) may comprise one or more load sensing elements or cells (220), which may comprise one or more devices configured to produce an electrical output that varies with an applied load, such as one or more piezoelectric load cells. Such load sensing elements (220) may be integrated or embedded within a deformable permeable layer (110), and the load sensor controller (182) may be fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and operably coupled to a computing device (104) via wire leads (198; shown coupled to a communication bus 170, which is operably coupled to a computing device (104) via wire leads 168, etc.). The computing device (104) operates the imaging device (106) and illumination sources (116, 122) and is physically interfaced with one or more objects such as a contact interface (120). By utilizing the deformable transparent layer (110), it not only facilitates touch sensing but may also operate a load sensing controller (182) to capture load-related data that may be associated with, for example, contact with an external object. The load sensing elements or a group of elements may comprise a grid or network and may be operably coupled to the load sensing controller (182) via one or more wire leads (218), etc. In one embodiment, for example, the integrated system may be configured to optimize the determination of touch sensing magnitude through the deformable transparent layer (110) as changes in load are detected using load sensor data. For example, if a portion of a deformable permeable layer (110) is pressed against the surface of another object, the magnitude of contact, as determined using the deformable permeable layer (110), may be compared with a change in the contact surface load detected using load sensors (182, 220), thereby providing two data sources for such determination, with at least some degree of uncorrelated measurement / determination error.

[0071] Referring to Figure 10H, an embodiment is illustrated in which a digital touch sensing assembly (146) is integrated with interconnected temperature sensors (184). The temperature sensing subsystem may comprise, for example, a temperature sensor controller (184) which may comprise an amplifier and / or a microcontroller, and one or more temperature sensing elements or cells (224) which may comprise one or more devices configured to produce an electrical output that varies with temperature, such as one or more thermocouple elements. Such temperature sensing elements (224) may be integrated or embedded in a deformable permeable layer (110), and the temperature sensor controller (184) may be fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and operably coupled to a computing device (104) via wire leads (200; shown coupled to a communication bus 170, which is operably coupled to a computing device (104) via wire leads 168, etc.). The computing device (104) operates the imaging device (106) and illumination sources (116, 122) and is physically interfaced with one or more objects such as a contact interface (120). By utilizing the deformable transparent layer (110), it not only facilitates touch sensing but may also be configured to operate a temperature sensing controller (184) to capture data on one or more temperatures that may be associated with, for example, contact with an external object. The temperature sensing element or a group of elements (224) may comprise a grid or network and may be operably coupled to the temperature sensing controller (184) via one or more wire leads (222), etc. In one embodiment, for example, the integrated system may be configured to optimize the evaluation of touch sensing characteristics through the deformable transparent layer (110) as temperature changes are detected.For example, if a portion of the deformable permeable layer (110) is pressed against the surface of another object having a temperature different from the ambient temperature (as is likely to be the case when touching most biological tissues in a surgical environment), the magnitude of contact, as determined using the deformable permeable layer (110), may be compared with the change in contact surface temperature detected using temperature sensors (184, 224), thereby providing two data sources related to contact profile determination, with at least some degree of uncorrelated measurement / determination error.

[0072] Referring to Figure 10I, an embodiment is shown in which the digital touch sensing assembly (146) is integrated with an interconnected imaging sensor (186), in addition to an imaging device (106) which is operationally integrated with a deformable transparent layer (110). The imaging sensor (186) may include a camera and may be configured to operate at various selected wavelengths such as visible light, infrared light, and equivalents. The imaging sensor (186) may be fixedly coupled to the housing (118) of the digital touch sensing assembly (146) and may be operably coupled to the computing device (104) via wire leads (202; shown coupled to a communication bus 170, which is operably coupled to the computing device 104 via wire leads 168, etc.). The computing device (104) operates the imaging device (106) and illumination sources (116, 122) and is physically interfaced with one or more objects such as a contact interface (120). By utilizing the deformable transparent layer (110), it not only facilitates touch sensing but also operates the imaging sensor (186) and captures data about objects within the imaging sensor's (186) field of view (226), such as images of nearby surfaces and objects. In one embodiment, for example, the integrated system may be configured to increase the frame rate for both the imaging sensor (186) and touch sensing through the deformable transparent layer (110) when an unexpected change in the imaging sensor's (186) field of view (226; which is preferably oriented to match, at least somewhat, the position and orientation of the associated deformable transparent layer 110) is detected using data from the imaging sensor (186). In other words, as the deformable transparent layer (110) begins to approach another object, as detected by changes in image data detected by the imaging sensor (186) system, the deformable transparent layer (110) and its associated computing and imaging capabilities may transition to an enhanced functional mode for detecting and characterizing any touch / contact.In alternative embodiments, the imaging sensor (186) may operate at infrared wavelengths and be configured to assist in detecting, for example, a thermal profile. Furthermore, the imaging sensor (186) may include a “depth camera” or “time-of-flight” image sensor, such as one available from PrimeSense, Inc., a subsidiary of Apple, Inc., which may be configured to obtain not only image data but also data relating to the depth or z-axis position of such image data to the imaging sensor (186).

[0073] Referring to Figures 10B-10I, and then returning to Figure 10A, various combinations and permutations of these illustrated sensing configurations may be integrated together in various embodiments. For example, in one embodiment, it may be desirable to have IMU sensor capability together with LiDAR to complement digital touch sensing through a deformable permeable layer (110). Various examples and embodiments are described below.

[0074] Referring to Figure 11, a configuration employing a digital touch sensing assembly (146) is shown coupled to the distal portion (236) of a robotic arm or robotic manipulator (234), mounted on a movable base (238). The robotic manipulator (234) may have an elongated arm configuration with various movable joints between rigid or semi-rigid linkages, as shown, or it may have a flexible robotic manipulator such as a robotic catheter or a tubular flexible robot (which may be available, for example, from Intuitive Surgical, Inc. or Johnson & Johnson, Inc.). The digital touch sensing assembly (146) is shown operably coupled to a computing device (144) via wired or wireless connections (232, 230, 166), etc., which is coupled to a power source (102) (136). A robotic arm (234) may be operated by a computing system (144) to move forward toward an object (228) having a surface of interest (70) and inspect it, which may include elements such as rivets (72) that are prone to failure or require regular inspection.

[0075] Referring to Figure 12, by utilizing various aspects of the aforementioned configuration, the digital touch sensing assembly (146) can be used to inspect the interface surface (120) and its features (70) through a controlled interface with the interface surface (120). In other words, as described above, and further illustrated in Figure 12, various other sensing configurations and associated data may be used in conjunction with digital touch sensing through the deformable transparent layer (240), including, but not limited to, IMU data (242), capacitive sensor data (244), resistive sensor data (246), LiDAR / point cloud data (248), strain or stretch sensor data (250), load sensor data (252), temperature sensor data (254), and data from additional imaging devices (256).

[0076] Referring to Figure 13A, a system configuration similar to that in Figure 11 is illustrated, with the addition of additional sensing capabilities coupled to the room or operating environment (260) to which it is connected (via wired or wireless connectivity to computing system 144, etc.) and to the digital touch sensing assembly (146). As shown in Figure 13A, one mounting member (359) is configured to couple an additional imaging device (270) to the digital touch sensing assembly (146) in a position and orientation that can capture a field of view relating to the front zone of the interface surface (120) of the digital touch sensing assembly (146); another mounting member (358) is configured to couple a further additional imaging device (272) to the digital touch sensing assembly (146) in a position and orientation that can capture different height fields of view relating to the front zone of the interface surface (120) of the digital touch sensing assembly (146); and a second mounting member (358) is coupled to a LIDAR device (274) in a position and orientation that assists in capturing point cloud and other data relating to the operating environment around the digital touch sensing assembly (146). As described above, in this embodiment, the connected room (260) also features enhanced sensing capabilities, and multiple imaging devices (264, 266) and an additional LiDAR sensor (268) are coupled to the room (260) in positions and orientations selected to assist in high-precision analysis of the robot's (234) movements as the object to be inspected (228) is positioned on a table (262) within the room (260).

[0077] Referring to Figure 13B, further improvements may be included on the computing device side of the system, interconnected (318) and operating the computing system (144), enabling a user to remotely understand the sides of the surface (70) of an object (228) being inspected by a digital touch sensing assembly (146). As shown in Figure 13B, a display (278) may be used to assist the associated user in viewing the output from the digital touch sensing assembly (146) as well as images or point clouds from other interconnected sensing subsystems (270, 272, 274, 268, 264, 266). Furthermore, a tactile interface (280), such as those illustrated in Figures 13C-13F, may be used to assist the user in experiencing a representation of the detected surface features. An interconnected 3D printer (276) may also be used to complement this "touch-sensitive workstation," allowing the user to decide to directly experience several layers of detected geometric shapes by printing them locally for direct manipulation (via the user's hands, etc.).

[0078] Referring to Figure 13C, a modified tactile interface (282) may be configured to provide a user with the sensation of experiencing a real or virtual surface through an operating interface such as a spherical member (290) that is coupled to a computing system (not shown) and configured to be held by the user's hand. Figure 13D illustrates a modified tactile interface (284) configured to provide a user (4) with a hand (12) grasping operating interface (292) to experience sides of a real or virtual surface through an interconnected computing system (not shown). Figures 13E and 13F illustrate further modified tactile interfaces (286, 288), in which the user's (4) hand (12) may be able to experience sides of a real or virtual surface through a pen-shaped (294) operating interface or a finger socket (296) operating interface. Therefore, using the “touch workstation” configuration of Figure 13B, which includes one of the illustrated tactile interfaces, a user may be able to observe (through the display 278), directly touch / manipulate (through the 3D printer 276), and tactilely experience (through the tactile interface 280) a side of the surface (70) of the object (228) being inspected, from a nearby or remote location. Accordingly, aspects of variations of such a configuration are illustrated with reference to Figures 14 and 15.

[0079] Referring to Figure 14, the user desires to engage with a surface using the sensing system, and the system is calibrated and positioned within proximity to the targeted surface (302). The user navigates the sensing surface toward the targeted surface via an electromechanical arm or robotic manipulator, etc., using feedback to the user regarding the position and orientation of the sensing surface provided by the positioning platform (inverse kinematics, load cell, deflection sensor, joint position, etc.) (304). As the sensing surface navigates closer to the targeted surface, the integrated sensing capabilities facilitate the detection of the targeted surface and its features (for example, the system may be configured such that the integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (306). The system may be configured to specifically create an event of contact between the sensing surface and the targeted surface (for example, the repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact) (308). The user may reposition and reorient the sensing surface relative to the targeted surface and inspect the targeted surface using integrated sensing capabilities (such as acceleration detected by an IMU, capacitive touch sensing, resistive touch sensing, LIDAR, strain or deflection gauges, load sensing, temperature sensing, and / or cameras and other imaging devices) (310). The system may be configured to present the user with aspects of the targeted surface so that the user will have an improved understanding of the targeted surface through visual, tactile, auditory, and tactile combinations, etc. (such as via a locally printed surface or a portion thereof) (312).

[0080] Referring to Figure 15, a user at a remote location from the targeted surface desires to engage with the targeted surface using the sensing system, and the system is calibrated and positioned within proximity to the targeted surface (314). The user navigates the sensing surface toward the targeted surface via an electromechanical arm or robotic manipulator, etc., using feedback to the user regarding the position and orientation of the sensing surface provided by the positioning platform (inverse kinematics, load cell, deflection sensor, joint position, etc.) (304). As the sensing surface navigates closer to the targeted surface, the integrated sensing capabilities facilitate the detection of the targeted surface and its features (for example, the system may be configured such that the integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (306). The system may be configured to specifically create an event of contact between the sensing surface and the targeted surface (for example, the repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact) (308). The user may reposition and reorient the sensing surface relative to the targeted surface and inspect the targeted surface using integrated sensing capabilities (such as acceleration detected by an IMU, capacitive touch sensing, resistive touch sensing, LIDAR, strain or deflection gauges, load sensing, temperature sensing, and / or cameras and other imaging devices) (310). The system may be configured to present sides of the targeted surface to a remote user so that the user will have an improved understanding of the targeted surface through a combination of visual, tactile, auditory, and tactile cues, etc. (such as via a locally printed surface or a portion thereof) (316).

[0081] Referring to Figures 16A-17, various aspects of another illustrative configuration utilizing the integrated touch sensing system described herein are shown. Referring to Figure 16A, interconnected rooms, kiosks, or measuring enclosures (324; connected to a computing system 144 via wired or wireless connectivity 320, 230, 166, which is integrated with and interconnected to other aspects of the touch workstation such as a power source 102, a 3D printer 276, a display 278, and / or a tactile interface 280, as described above), as well as further imaging devices (270, 272) and LiDAR detection The device (274) is characterized by several imaging, sensing, and detection interconnected resources, such as a LIDAR device (286), one or more imaging devices (264, 266), and a digital touch sensing assembly (146), each of which may be configured to assist in characterizing the geometric shape and surface of an object, such as a person's (4) foot (322), which may be lowered to a position where the foot engages with the digital touch sensing assembly (146), as shown in Figure 16B (326). In other words, the measuring housing or kiosk (324) may be configured to facilitate the convenient engagement of a part of the user's appendage, such as a part of the user's leg or arm, and to collect high-precision information about an object, such as the sole of the user's foot, which may be used to design orthopedic insoles, ski boots, and equivalents. The combined data available on the interconnected workstation may be used not only to inspect the object (such as the user's foot) but also to precisely characterize its geometric shape. For example, a digital touch-sensing assembly (146) may be used to precisely characterize the primary load surface (i.e., the bottom surface of user 4's foot 322), and image and point cloud data may be used to further understand the geometric shape of the object (user 4's foot and lower leg) so that these findings may be used to aid in orthopedic research, pre- or post-surgical studies, custom shoe design, and equivalents. One such configuration is illustrated in Figure 17.

[0082] Referring to Figure 17, in one embodiment, an improved understanding of the foot geometry and load pattern is desired for a particular user (330). The user may expose their feet, and the system may be initialized in preparation for characterization (332). The user may position / orient their feet within the measurement structure to facilitate scanning of the lateral geometry of the exposed feet (334). The user may reposition / reorient their feet within the measurement structure to facilitate further scanning of the lateral geometry of the exposed feet (336). While the user places their feet on a deformable permeable layer and withstands the load applied to the feet, the system collects data on the load pattern, anatomical structure, and geometry (338). The system may be configured to create an anatomical / geometric profile of the user's feet, along with a load profile associated with the anatomical / geometric profile (340). Anatomical / geometric profiles and load profiles may be used to create interface structures (such as shoes, ski boots, or orthopedic insoles) and / or to diagnose associated medical conditions (342).

[0083] Referring back to Figures 13A and 13B, some surfaces and objects may be presented in configurations that are somewhat easily accessible. Many other fine manipulation and / or contact scenarios involve greater geometric or spatial complexity. For example, referring to Figure 18A, a scenario is illustrated on which the hand (348) of the person (346) may be used to controllly approach a targeted object, such as a cookie (354), which happens to be inside the container (344), and then touch, inspect, and / or grasp it, in order to maintain the integrity of the container (344) and / or object (here, a cookie 354, which may also be fragile), to the extent that relatively high loads or impulse contact should be avoided, as the object may be fragile. The supporting structure or substrate (table 352, etc.) on which the container (344) rests may also be fragile or susceptible to damage under high loads or high impulses. The human upper limb is coincidentally very dexterous in that it facilitates the successful handling of this exemplary situation, partly due to the smooth motor neurons, muscles, and kinematic activity of the upper limb, as well as the distribution of sensory neurons in tissues such as the skin. For example, the depicted human (346) would typically have sensory neurons throughout the skin, such as in the wrist (350) and hand (348) areas, so that the associated human (346) could carefully navigate the geometric shape of the container and the targeted object (354) as well as the mechanical breakage mechanism associated with both. In other words, the human could use touch perception through the skin and other tissues to navigate the scenario without destroying the associated structures.Attempting the same scenario using mechanical systems, such as a backhoe loader (in the expanded version of this scenario) or a remotely controlled robot, presents numerous challenges because the human controlling the system remotely (from the robot across rooms, or from the robot across countries, connected by computing connectivity capabilities, etc.) typically lacks human-level senses or touch or tactile perception related to interaction, and cannot perceive, through visual or auditory confirmation, etc., that one or more associated structures are about to be damaged until it is too late.

[0084] Referring to Figures 18A and 18B, the target touch sensing technology may be used to address such scenarios and provide users in nearby or remote locations with a superior sense of the physical engagement in question.

[0085] As shown in Figure 18B, an electromechanically controllable robotic arm (234), along with an interconnected touch-sensing assembly (146), such as those described above, is positioned within a chamber (260) and stationary on a substrate or support structure (such as a table 352), to inspect an object (such as a cookie 354) inside a container (such as a bottle 344). The chamber (260) may preferably be configured to have multiple sensors, such as a LiDAR (268), and one or more image-capturing devices (264, 266) coupled thereto and positioned to capture information about the volume around the robot and / or the targeted object (354), in a manner that provides high-quality data from multiple sources with uncorrelated errors, as described above. One or more additional sensing devices, such as an additional image capture device (270) and a LiDAR (274), may be coupled to the robot arm (234) and provide further information about the volume surrounding the interconnected touch-sensing assembly (146), and, for improved data fusion capabilities, further high-quality data from multiple sources with uncorrelated errors. Each sensor (146, 264, 266, 268, 270, 274) may be coupled to one or more computing devices (104) (232, 258, 230) via wired or wireless connections, etc., which may be configured to facilitate interaction control. Using such a configuration, the distal and target-facing touch-sensing assembly (146) may be configured to assist a user, who may be located nearby or at a distance as described above, in gaining a sense of physical interaction in the deformable permeable layer (110) of the touch-sensing assembly (146). Furthermore, as described above with reference to Figure 13B, the user may be provided with a workstation capable of providing one or more means for perceiving physical engagement, such as a tactile interface (280), a display (278), and / or a 3D printer (276, i.e., to facilitate printing one or more layers of an object).To further enhance user perception of physical engagement scenarios using a remotely operable operation or inspection configuration (such as a robot 234, as shown), an additional touch-sensing assembly (360) may be coupled to a remotely controllable engagement system (234) in a configuration, as shown, partially or entirely, circumferentially around the distal portion of such a system. In other words, the additional touch-sensing assembly (360) may comprise components similar to the aforementioned touch-sensing assembly (146), provide one or more outward-facing deformable transparent layers (110), and be coupled around a portion of the periphery of the associated structure in a manner that it is operably coupled to a computing device (104) via wired or wireless connectivity, etc. (232, 230), and provides additional touch sensing for a user of a remote workstation. As shown in Figure 18B, the additional touch-sensing assembly (360) is preferably positioned on the remotely controllable engagement system (234) in a location that would help the remote user understand important aspects of remote engagement, such as distal or “wrist” locations where contact with the targeted or associated object is likely to occur. For example, circumferential positioning of the additional touch-sensing assembly (360) around at least a portion of the distal touch-sensing assembly (146) may be useful in assisting the remote user to navigate downward through the mouth of the container (344) to the targeted object (354), since faint or more direct contact with either of the sensing assemblies (360, 146) may occur during such an approach.

[0086] Referring to Figure 18C, a configuration similar to that in Figure 18B is illustrated, with the addition of another touch-sensing assembly (362) that is circumferentially coupled around at least a portion of what may be called the “forearm” member of the depicted robot (234) and again operably coupled to a computing system (104) via wired or wireless connectivity, etc. (232, 230). In fact, both touch-sensing assemblies (360, 362) may be configured to sense circumferentially around the elongated assembly (234) via a pair of diametrically opposed touch-sensing assemblies (146), for example, a group of three or more touch-sensing assemblies that can be separated from each other in a configuration that is evenly spaced in the circumferential direction (i.e., to maximize coverage to the surrounding environment). Such additional sensing capabilities in the depicted location may further assist a remote user in successfully navigating the illustrated physical engagement task, touching a targeted object (in this case, a $355 bill), inspecting it, and / or grasping it.

[0087] As described above with reference to Figures 9A and 9B, various sensor configurations may be created by assembling and operably coupling multiple touch-sensing assemblies (146), such couplings may be used to create circumferential or partially circumferential type touch-sensing assemblies (360, 362), as shown in Figures 18B and 18C. Also, as stated above with reference to Figures 7A-7E, etc., components such as optical fibers and / or waveguides may be used to move the sensor to various positions relative to the emission or captured radiation of captured light, etc. (i.e., rather than directly positioning the optical sensor or image acquisition device at the acquisition location, the light may be captured at the acquisition location using waveguides, transparent fibers, or a combination thereof, and the transmission to the optical sensor or image acquisition device positioned further away from such acquisition location). Referring to Figures 18D-18K, various configurations are illustrated, which provide alternatives for radiation transmission relating to touch-sensing assemblies (146, 360, 362), such as those described above. Referring to Figure 18D, for example, a configuration similar to that illustrated in Figure 7A is shown, comprising an optical element (108) operably coupled to a light (or other wavelength radiation; for example, alternatively, infrared wavelengths) emitting device (116), in which the configuration is selected to result in photon propagation (364) from emission in a light-emitting device (116) to various positions along the optical element (108), and the photons can intersect in a deformable transparent layer (110) using an exit angle (366) predetermined by the reflectivity / refractivity of the material and the geometry of the structure, such as about 20 to about 40 degrees. Figure 18E illustrates a similar configuration with light emission from two sides (116, 122), as in the assembly of Figure 7A.Referring back to Figure 7A, an image acquisition device having dimensions within the range of a three-dimensional cube, with a bezel dimension of approximately 1.5 mm, a distance to the imaging object of approximately 3 mm, and a working distance of approximately 5 mm, is combined with an optical element (108) comprising a polymer or glass material, such as polymethyl methacrylate ("PMMA"), selected to facilitate illumination through which illumination is facilitated, in a layer of deformable transparent polymer material (110) of approximately 4 mm thickness (368) and approximately 1-2 mm thickness, such the assembly may be in the range of 1-15 mm thickness, and such dimensions depend, at least in part, on the standpoint of selection, on illumination requirements and in-situ load requirements. Such assembly dimensions are practical in various configurations, but may be minimized using alternative configurations.

[0088] Referring to Figure 18F, a so-called "front-illuminated" or "front-illuminated" film (372), such as those used in computing device displays (for example, mobile devices that can be used outdoors or in other brightly lit environments where conventional back-illuminated configurations may not be effective, such as those available under the trademark name Kindle®, may utilize a reflective display configuration in which ambient light is selected to be used so that the illumination layer resides between the display pixels and the viewer), has an optical element (1 Along the length of (08), the film (372) may have light extraction features for controllably extracting light or other radiation in a preferred direction, such as toward or outward from the deformable transparent layer, at a desired location or distribution (110; i.e., light may bounce through the illumination film 372 via total internal reflection, etc., exit the film 904, and enter into the deformable transparent layer 110, which can act as a spacer to allow sufficient spacing, i.e., a "z-axis spacing", for mixing the carriers of various optical layers with light perpendicular to the plane of the deformable transparent layer), and may have a thickness (370) in the range of 100 microns. A cladding layer (not shown), such as one made of silicone material, may be bonded to the outer surface of the film (372), and the carrier layers may also be interconnected, for example, to provide additional structure and localized planarity. Using such a configuration, the assembly thickness may be reduced to about half, or about 5-6 mm, depending on the material of the film (372) and the light extraction features. Using configurations such as those shown in Figure 18F, given the positioning of the illumination layer (372) and the emission path / angle (904, 906), there may be portions (900) of the deformable transparent layer (110) that are difficult to access. Figure 18G illustrates another embodiment in which the film 372 is positioned between the optical element 108 and the deformable transparent layer 110, as in various so-called "front illumination" configurations, and is therefore closer to the deformable transparent layer (110).Similar to the configuration in Figure 18F, features within the illumination layer may, as shown, assist in controlled bounce / reflection 902 and emission or extraction 904 via total internal reflection, etc., and direct illumination toward other layers such as a deformable transmissive layer (110). The thickness (370) of the illumination film (372) may be determined by factors related to the illumination requirements, such as the requirement for very tightly controlled illumination (e.g., more light may require a thicker illumination film, tighter angle control may require a thinner illumination film). Importantly, such layers may be substantially planar, but may also be non-planar or curved with varying levels of complexity (convex, concave, cylindrical, etc.), may be illuminated from various locations, and may be elongated, as shown in Figures 18H and 18I, which may facilitate circumferential geometric shapes, such as those illustrated in the cuff-shaped circumferential sensors (360, 362) in Figures 18B and 18C. Furthermore, as illustrated in Figure 18J, such a film (372) may be bonded on multiple sides, not just one side, for controlled reflectivity. Figure 18J illustrates a configuration with a controlled reflectivity front illumination film, which is bonded to four sides (372, 374, 376, 378) around the optical element (108) being depicted, as shown. In other embodiments, a configuration similar to that of Figure 18I may be bonded to up to six sides, as shown, with two additional illumination films bonded to both sides of the optical element (108) in a manner coplanar with the drawing paper.

[0089] Referring to Figures 18K and 18L, as described above, waveguides may be used as transmitting or coupling members to efficiently move light between various elements. Figure 18K illustrates, for example, a wedge-type waveguide with a maximum thickness (380), which may be in the range of 1 to 5 mm, and which has an enclosing angle (384) in the range of 1 to 15 degrees, and which may assist in the propagation (388) of light from the emitting device (116) across the waveguide (392) into the optical element (108) and into the deformable transparent layer (110), and a gap (908) may be configured to assist in transverse transmission from the waveguide (392) into the optical element (108). Figure 18L illustrates a similar wedge-type waveguide with a maximum thickness (382), which may be in the range of 1–2 mm, and which has an enclosing angle (386) in the range of 2–8 degrees, and which may help light propagate linearly (390) from the emitting device (116) across the waveguide (394) (again, the gap 909 is shown to aid transmission and prevent total internal reflection) into the deformable transparent layer (110). In the configuration of Figure 18K, a film (not shown) may be positioned in the upper right corner of the depicted surface of the deformable transparent layer (110), and additional capture devices or cameras and additional illumination sources may be added to the opposite (left shown) side of the waveguide (394), unless the opposite side has a specular reflective coating. Mirror coatings and so-called “direction-shifting films” may be included to further assist in efficiently guiding and transmitting light or other radiation between the elements (for example, the light emanating from the waveguide 392 described may have an emission vector substantially parallel to the vertical plane of the waveguide 392, and it may be desirable to “direct” the emitted light by coupling a direction-shifting film to the waveguide 392, thereby creating a desired illumination angle). Components, materials, geometric shapes, and refractive / reflective properties may be tailored for various specific geometric challenges, such as those presented in the various use cases described and illustrated herein.

[0090] As described above, increasing the perception of activity in remote locations for a user through a local workstation, whether the user is located across rooms, in another building, or across the world, is a critical challenge for many computerized systems such as telecommunications, remote presence, teleinspection, or remote action systems. Referring to Figure 19A, one improvement in perception for a user (4) at a local workstation may be via a tactile master input device (280), which is operably coupled to an interconnected computer system (104) via a wired or wireless connection, etc. (396, 230), and may enable the user (4) to perceive tactile aspects such as touch, friction, texture, and simulated translations of equivalents locally at the workstation through the user's hand (12) and / or wrist (13). Referring to Figure 19B, in another embodiment, it may be useful to facilitate further local perception of remote physical interactions by what may be called a “touch conversion interface” (398), which may be configured to provide the user (4) with one or more sensations at the wrist (13) or at other locations related to and / or intuitively associated with activities at a remote location, such as contact between objects at a remote location. Such sensations may be provided in addition to sensations provided to the user (4) through, for example, a tactile master input device or controller (280). In other words, in various embodiments, multimode sensations may be provided to the user (4) to assist the user in perceiving activities at a remote location with improved fidelity.

[0091] Referring to Figures 20A-20C, various aspects of road vehicles, such as computerized electric vehicles, present opportunities for touch integration and enhancement. For example, typically, a human operator would have highly consistent touch interfaces with parts of the vehicle's structure, such as the pedals (404, 406), the floor (414), the driver's seat (412), the steering wheel (408), the sides of the dashboard control and / or display interface (410), and parts that may be known as the “A-pillar” (402). These structures, and others, each present opportunities for integrated touch sensing to assist, for example, operation, control, and safety. For example, referring to Figures 20B and 20C, a touch-sensing assembly featuring a deformable permeable layer may be operably coupled to various sides of a front (438, 440, 442) and rear (444, 446, 448) vehicle bumper or frame structure to assist in detecting deformation related to impact and may be used to trigger safety systems such as seat belt retractors or passenger-side airbags, in addition to or as a replacement for other more conventional sensors configured to provide such functionality, such as a built-in accelerometer, which may introduce longer latency for control of such safety systems than the touch-sensing assembly featuring a deformable permeable layer. In other words, the installation of a touch-sensing assembly featuring a deformable permeable layer may be selected to provide penetration detection very early in the penetration, perhaps before an acceleration detection system detects an actionable change in acceleration in a frame component, etc.Figure 20B shows various locations and positions within a vehicle where a central controller or computing system may be operably coupled to a touch-sensing assembly featuring a deformable permeable layer, capable of detecting user touch and / or contact through touch sensors operably coupled to each of the following: pedals (416, 418), driver's floor (420), driver's seat base (422), driver's seat backrest (424), driver's seat headrest (426), shifter interface (430), central control console interface (428), steering wheel (432), dashboard section (434), and A-pillar (402) structural section (436). For each of these illustrative structures, the touch-sensing assembly featuring a deformable permeable layer may have different geometric shapes, be made of various materials, and provide structural properties that are tuned to each usage scenario. For example, the structural modulus of the seat base (422) touch sensor may generally be relatively low, and the information required may also be of relatively low resolution (especially, a general weight profile of the operator, without high resolution, to help determine, for instance, that a child or dog below a certain weight will not attempt to operate the vehicle). This may contrast with a central console (428) interface, where the structural modulus may be selected to be relatively high to provide sufficient penetration with such typical touch loads in order to obtain desired information, such as a general fingerprint geometric shape correlation, which can be analyzed when the vehicle is started, for a layer of biometric security regarding authorized users / operators.

[0092] One of the challenges in integrating multiple touch-sensing assemblies, each featuring a deformable permeable layer, into a system such as an automobile or robot is interconnectivity. Referring to Figure 21A, for example, various aspects of control, signaling, power, and / or operational connectivity (232, 230) between a system such as a robot (234) featuring a touch-sensing assembly (146), as described above, and a computing system (144), can be achieved through wired wiring leads or wireless connectivity via Bluetooth®, IEEE 802.11, or various other standards. In fact, referring to Figure 21B, as shown in the enlarged views in Figures 21C and 21D, it may be desirable for a system such as a robot (234) to have at least some components or aspects, such as a touch-sensing assembly (146), while power and some level of controller and / or computing power may be provided by onboard computing devices (144) and power systems (102), such as built-in chipsets, microcontrollers, field-programmable gate arrays, application-specific integrated circuits, and equivalents, as well as a battery that may be rechargeable via a wireless inductance, etc., while power and some level of controller and / or computing power may be provided by an onboard computing device (144) and power system (102), such as a built-in chipset, microcontroller, field-programmable gate array, application-specific integrated circuit, and equivalent, as well as a battery that may be rechargeable via a wireless inductance, etc. Such integration and general trends toward tetherless configurations may be called a “Internet of Things” variant and may be useful in many system integration challenges. For example, referring to Figure 22, a wirelessly connected touch-sensing assembly (146), similar to that shown in Figure 21C, may be integrated into the door locking system configuration, where a person's thumb (452) or other finger may engage with a deformable permeable layer and be used to provide biometric authentication / lock access functionality and facilitate unlocking.The touch-sensing assembly (146) may be wirelessly connected to one or more computing systems, for example, one or more computing systems within an associated building and / or mobile systems, which may reside in a data center and equivalents.

[0093] Referring to Figures 23A and 23B, a variation of a handheld surface (460) analysis tool is shown, featuring a wirelessly connected touch-sensing assembly (146), such as that shown in Figure 21C, the housing (458) may be configured to engage with the user's hand (462) and facilitate engagement between a deformable permeable layer and associated interface surface (120) and the surface (460) of an object targeted for surface analysis. The touch-sensing assembly (146) may be wirelessly connected to one or more computing systems, which may be, for example, one or more computing systems in an associated building and / or mobile, residing in a data center, and equivalent, and the handheld assembly may house its own power source, such as a battery, for operational purposes.

[0094] Referring to Figure 24A, a vehicle configuration with integrated touch sensors is illustrated with touch sensing assemblies that are operably coupled to various structures such as an elongated touch sensor (436) coupled to the A-pillar (402) of the vehicle, touch sensors (416, 418) coupled to the pedals, a touch sensor (420) coupled to the driver's seat floor, a touch sensor (428) coupled to the center console, a touch sensor (422) coupled to the base of the driver's seat (412), a touch sensor (424) coupled to the backrest of the driver's seat, a touch sensor (426) coupled to the driver's seat headrest, a touch sensor (430) coupled to a shifter member, a touch sensor (432) coupled to the steering wheel, and a touch sensor (434) coupled to part of the vehicle's dashboard, and such sensors are connected to a central computing system (144) by wire-lead type connectivity (464).

[0095] Referring to Figure 24B, a sensor in a similar location having wireless connectivity to a transceiver (166) of a central computing system (144) may help simplify such integration by eliminating the need for certain connectivity wiring, and similarly, in a modified example where the sensor is operably coupled to a small power source such as a battery, which may be rechargeable, for example via a wireless inductance, the need for power source wiring may also be eliminated. Therefore, the A-pillar touch sensor (436) is shown as operably coupled to a wireless transceiver (466), the pedal touch sensors (416, 418) are shown as operably coupled to wireless transceivers (472, 470, respectively), the floor touch sensor (420) is shown as operably coupled to a wireless transceiver (474), the seat base touch sensor (422) is shown as operably coupled to a wireless transceiver (476), the seat back touch sensor (422) is shown as operably coupled to a wireless transceiver (478), and the headrest touch sensor (426) is shown as operably coupled to a wireless transceiver. The shifter assembly touch sensor (430) is shown as operably coupled to a wireless transceiver (486), the center console touch sensor (428) is shown as operably coupled to a wireless transceiver (484), the steering wheel touch sensor (432) is shown as operably coupled to a wireless transceiver (482), and the dashboard (410) touch sensor (436) is shown as operably coupled to a wireless transceiver (466), and each of the touch sensors is wirelessly connected to the vehicle's central computing system (144) (166).

[0096] Referring back to the configuration shown in Figure 19A, the touch sensing aspect can be used for the user to improve and / or enhance the perception of certain actions on a local workstation, and the value of having multiple sources of sensing data, including uncorrelated error configurations, for so-called "sensor fusion" availability is discussed. Referring to Figure 25A, in one embodiment, a system featuring multiple sensing configurations (including multiple sensing configurations with uncorrelated error sources) is initialized for use at a first location (488). The system may be configured to provide the operator with information about system operation through a user interface (490). In response to one or more command inputs by the operator, the system may be configured to execute and, at least in part, provide feedback to the operator using a user interface based on the multiple sensing configurations (492). The system may be configured to optimize operation and feedback through a sensor fusion technique configured to take advantage of the differences in information provided by the multiple sensing configurations (494).

[0097] Referring to Figure 25B, a robotic manipulator system featuring multiple sensing configurations (such as capacitive, resistive, radar, lithium-ion, camera, load sensor, strain or extension sensor, IMU, and / or joint position sensor configurations, along with deformable permeable layer-based touch sensing with uncorrelated error sources, along with a deformable permeable layer-based touch sensing) may be initialized for use in a first location (496). The system may be configured to provide the operator with information about the system's operation through a user interface (498). In order to utilize the robotic manipulator system for a task (such as picking up an object from inside a bottle) in response to one or more command inputs by the operator, the system is executed and configured to provide the operator with feedback using a user interface based at least partially on the multiple sensing configurations (500). The system may be configured to optimize operation and feedback through a sensor fusion technique configured to take advantage of differences in information provided by multiple sensing configurations (for example, as the distal portion of a robotic manipulator system is navigated into the opening of a bottle, some sensors of the multiple sensing configurations may be occluded or temporarily unreliable, while at the same time, at least one other of the multiple sensing configurations, preferably having at least somewhat uncorrelated errors such as a deformable permeable layer-based touch sensing, provides reliable information back to the system and the operator) (502).

[0098] Referring back to Figure 19B, the integration of one or more touch-converting interfaces (398) on the user's (4) wrist (13), etc., may provide enhanced perception of activity and engagement in remote locations. Figure 26 illustrates a configuration in which a local operator interface (506) to the user or operator may feature a computing system (144) interconnected (318) with a tactile interface (280), a display system (278), a 3D printer (276), and a touch-converting interface (504), respectively. The operator interface (506), located locally to the user, will generally be isolated (640) inches, feet, miles, or thousands of miles from the remote operating system (e.g., a robotic arm 234 featuring a touch-sensing assembly 146, as illustrated in Figure 26), depending on the user configuration, the task at hand, and connectivity alternatives (230, 166), such as wired or wireless connectivity. Referring to Figure 27, in further illustrative detail, the operator interface (506) may include interconnected (400) computing (144), master input device / controller (280, tactile-enabled variant), 3D printing (276), and display (278) resources, as well as a touch conversion interface (398), such as a variant shown, which can be detachably coupled to the wrist (13) of the user (4) and may be configured to provide one or more components of sensing that can be perceptually linked to activities at a remote location, as will be described in more detail below.

[0099] In various embodiments featuring one or more touch conversion interfaces in the operator interface (506), it may be desirable to position one or more touch conversion interfaces in relation to the anatomical structure of the user (4) having some kinematic relevance to the activity of the component in the remotely operated or actuated facility. For example, referring to Figure 28A, a robotic arm (234) is to be operated in a remote facility, and the robotic arm (234) has a kinematic part that is at least somewhat similar to a “wrist,” and in one embodiment, a touch sensing assembly (362) may be functionally coupled to a touch conversion interface that can be detachably coupled to the wrist (13) of the user (4) in the interconnected operator interface (503). In other words, this can improve the level of intuitive interaction between the local user / operator from the operator interface (503) and the remote robotic manipulator when touches / contacts sensed at the robot's “wrist” are translated to the user’s wrist. Therefore, in various embodiments, attempts may be made to provide at least somewhat kinematically similar pairings between remote locations and local touch sensing and conversion resources.

[0100] Referring again to Figure 28A, it should also be emphasized that one or more touch sensing assemblies may be integrated for a given implementation, such as an additional at least partially circumferential touch sensing assembly (360) located around the distal end of the robot arm (234) and interconnected with other touch sensing assemblies (362) that are closer to the computing resources. Referring to Figure 28B, in a somewhat kinematically similar functional pairing configuration, a more distal touch-converting interface (508), such as a finger-sized cuff detachably coupled to the index finger, may be configured to convert touches or contacts sensed by a more distal touch-sensing assembly (360) positioned around the distal end of the robot arm (234) at a remote location as shown in Figure 28A, operably coupled to a computing system via wired or wireless connectivity (510) or the like, and positioned around the “wrist” of the robot arm (234) at a remote location as shown in Figure 28A. A more proximal touch-sensing assembly (398) may be configured to convert touches or contacts sensed by a more proximal touch-sensing assembly (362) positioned around the “wrist” of the robot arm (234) at a remote location as shown in Figure 28A, detachably coupled to the forearm or wrist (13) of the user (4), operably coupled to a computing system via wired or wireless connectivity (400) or the like, and positioned around the “wrist” of the robot arm (234).

[0101] Referring to Figure 29A, a Graspa (518) style end effector is shown with two opposing movable members (520, 522) which may be controllably advanced toward each other for gripping. In various embodiments, a touch-sensing assembly may be integrated into these opposing movable members (520, 522) and operably coupled thereto to assist in the perception of associated actions. Referring to Figure 29B, a master input device configuration (516) is configured to allow two opposing fingers of the user's hand (12) to remotely control a gripping action such as that of a Graspa, as illustrated in Figure 29A, in a manner at least partially kinematically similar (i.e., the opposing movable members 520, 522 may be moved toward each other by moving the opposing fingers toward each other).

[0102] Referring to Figures 29C and 29D, a number of detachably connectable touch conversion interfaces (508, 512) may be operably connected to a computing system via wired or wireless connectivity, etc. (510, 514, respectively), which may be operably connected to a remote instrument such as a Graspa (518) as illustrated in Figure 29A, providing enhanced intuitiveness for the user or operator (again, the opposing movable members 520, 522 may be moved toward each other by moving opposing fingers toward each other, and touch / contact information detected by the touch sensing assembly in the opposing movable members 520, 522 may be used as input to sensing created for the user in the touch conversion interfaces 508, 512). Figure 29C illustrates an embodiment in which the touch conversion interfaces (508, 512) are detachably attached to the user's index finger (526) and middle finger (528), while Figure 29D illustrates an embodiment in which the touch conversion interfaces (508, 512) are detachably attached to the user's index finger (526) and thumb (524).

[0103] Referring to Figure 30A, a touch conversion interface (398) detachably coupled to a user (4) is illustrated along with an operable coupling to a computing system (144) via wired or wireless connectivity (400, 230, 166), etc. The touch conversion interface (398) comprises a single touch conversion element, or, as shown, a plurality of touch conversion elements (530), and helps to provide the user (4) with an enhanced perception of touch and / or contact with an interconnected touch sensing assembly. Referring to Figures 30B-33B, various types, combinations, and permutations of touch conversion elements may be used in various embodiments. Referring to Figure 30B, an unbalanced electric motor (532) may be used as a touch conversion element to provide vibration and variable frequency touch conversion. Referring to Figure 30C, a light-emitting diode ("LED") (534) may be used as a touch conversion element to provide the user with a visual conversion of the occurrence of contact or touch, the output brightness may vary according to the magnitude of the touch or contact load, and various colors / wavelengths may be available. Referring to Figure 30D, a piezoelectric assembly (536) may be used as a touch conversion element to provide a relatively high-frequency vibration response in response to contact or touch, the frequency and / or intensity may vary according to the magnitude of the touch or contact load. Referring to Figure 30E, an auditory speaker assembly (538) may be used as a touch conversion element to provide an audible response in response to contact or touch, the frequency and / or intensity may vary according to the magnitude of the touch or contact load. Referring to Figures 30F and 30G, one or more so-called "shape memory alloy" ("SMA") segments (540) comprising an alloy material such as nickel / titanium may be used as a touch conversion element.As shown in the chart (544) of Figure 30G, for example, commercially available SMA alloys can be configured to shrink in size very dramatically (e.g., as shown in 542 of Figure 30F, within the range of shrinking to half their cold length when heated through an energizing circuit), and thus, when formed into hoop or cuff type configurations, as shown in the modifications illustrated in Figures 32A and 32B, they may be used to controllly apply and / or relieve moderate hoop stress and / or hoop strain.

[0104] Therefore, referring to Figure 31A, the touch conversion interface (398), which is operably coupled to a computing system via a wired or wireless communication configuration, may be detachably coupled to a user (4) at a wrist (13) position, and may include a controllably operable tactile actuator motor such as an unbalanced motor (532). Referring to Figure 31B, the touch conversion interface (398), which is operably coupled to a computing system via a wired or wireless communication configuration, may be detachably coupled to a user (4) at a wrist (13) position, and may include one or more LEDs (534). Referring to Figure 31C, the touch conversion interface (398), which is operably coupled to a computing system via a wired or wireless communication configuration, may be detachably coupled to a user (4) at a wrist (13) position, and may include a controllably operable piezoelectric assembly (536). Referring to Figure 31D, the touch conversion interface (398), which is operably coupled to a computing system via a wired or wireless connection (400), may be detachably coupled to a user (4) at a wrist (13) position, etc., and may include a controllably operable auditory speaker assembly (538). Referring to Figure 31E, the touch conversion interface (398), which is operably coupled to a computing system via a wired or wireless connection (400), may be detachably coupled to a user (4) at a wrist (13) position, etc., and may include one or more controllably operable shape memory alloy segments (540). Figures 32A and 32B illustrate that, when viewed from an orthographic perspective, the configuration shown in Figure 31E, etc., may comprise a single SMA segment (540) or a plurality of SMA segments (540, 546, 548, 550), each of which may be individually controllable, as shown in the modified example in Figure 32A.

[0105] Referring again to Figure 30A, the touch conversion interface may comprise a plurality of (530) touch conversion elements, which may be similar to or different from one another. For example, referring to Figure 33A, the touch conversion interface (398), which is operablely coupled to a computing system via a wired or wireless communication configuration, etc., may be detachably coupled to a user (4) at a wrist (13) position, etc., and may comprise three or more controllably operable shape memory alloy segments (540, 552, 554) positioned longitudinally relative to one another so as to be coupled into the touch conversion interface (398). Figure 33B illustrates a configuration in which the touch conversion interface comprises a very wide range of multiple touch conversion elements, such as multiple SMA segments (540, 552, 554), multiple tactile motors (532, 533), multiple piezoelectric assemblies (536, 537), multiple auditory speaker assemblies (538, 539), and multiple LEDs (534, 535), each of which may be operated and controlled individually and / or independently to provide enhanced perception for the user in a local touch workstation.

[0106] Referring first to Figure 36, the surgical robot integrated configuration is shown and positioned in a touch-sensitive facilitating operator workstation. The operator may utilize the surgical robot system in remote locations such as across rooms, across countries, or across the globe, separated from the operator workstation (640). Touch conversion elements may be used to enhance the operator's understanding of contact, touch, and other activities in remote locations during surgical navigation, and the operation of robotic surgical end-effectors such as a grappler (518) on a targeted portion (576) of a targeted tissue structure (572). As shown in Figure 36, the operator workstation may include a touch conversion interface (398) of one or more (530) elements detachably coupled to a part of the user (4), such as the wrist (13), which may be configured to respond to contact in a robotic instrument (594) wrist portion (582) touch-sensing assembly (360). The operator workstation may further include two additional touch conversion interfaces (508, 512) configured to respond to contact in touch-sensing assemblies (602, 604) coupled to each of the corresponding robotic grass-facing members (522, 520). The touch conversion interfaces may be operably coupled to a computing system (144) via wired or wireless connectivity, etc. (400, 510, 514, 230, 166). The touch-sensing assemblies may similarly be operably coupled to a computing system (144) via wired or wireless connectivity, etc. (592, 606, 608, 230, 166).Therefore, as the remotely controllable robotic instrument (594) advances and navigates toward the target portion (576) of the targeted tissue structure (572), the user (4) in the workstation may be provided with intuitive perceptual cues regarding contacts and touches between the side of the instrument and the side of the tissue, such as contact between the robotic instrument wrist (582) and the edge (578) of the wall or tissue structure (572), and contact between the robotic instrument grappler (518) members (520, 522) and the wall or edge (578, 576) of the tissue structure (572). Preferably, one or more image capture devices may be configured to capture one or more views (598) of a surgical scenario to be presented for the user (4) in the operator workstation on a display (278), etc., which may be operably coupled to a computing system (144) by wired or wireless connectivity, etc.

[0107] Therefore, referring to the process flow in Figure 34, the user at the local workstation has connectivity to a remote engagement configuration in the remote environment, such as a robotic arm, which is operably coupled and has one or more connected touch-sensitive surfaces, in order to assist the user in physically engaging with one or more sides of the remote environment (556). The local workstation and the remote engagement configuration may be powered on, started up, and ready for remote touch engagement by the user (558). The user may operate a master input device at the local workstation, which is operably coupled to the remote engagement configuration (such as a robotic arm operably coupled in the remote environment) and physically engages with one or more sides of the remote environment (such as physically engaging with the surface of an object in the remote environment) (560). Through the local workstation, the user may be able to experience and understand aspects of physical engagement between the remote engagement workstation and one or more aspects of the remote environment (for example, by locally perceiving various levels of touch engagement in the remote environment through the local workstation, a cuff touch sensor operably coupled to the distal part of a robot arm in the remote environment may be configured to provide the user with an intuitive understanding of touch engagement in the remote environment via a local touch conversion interface, etc., which may be coupled to the user and may be configured to locally provide one or more modalities of remote touch derivation feedback via a kinematically similar and / or intuitive local configuration of a local touch conversion interface, etc.) (562).

[0108] Referring to Figure 37, similar uses of touch conversion interfaces and touch-based operator workstations may be used to assist users in experiencing touch, touch, and related activities in a remote environment that is truly remote in that it is a virtual environment (612) (i.e., "real" only to the extent that it is created on a computer). For example, in the embodiment shown in Figure 37, the user can use a tactile master input device (280) to define virtual roads (614), cavities (618), virtual walls (616), and a virtual mobile arm robot (622) virtual element around the virtual environment (612) which has virtual aspects such as game-based "golden pot" elements or objectives that can be obtained or acquired by the user, if the user is able to successfully grasp a virtual prize element (620) using virtual gripper elements (628, 630) mounted on a virtual robot arm (626) mounted on a virtual mobile base (624) in the depicted virtual environment (612). Virtual touch-sensing elements (632, 634, 636) may be virtually coupled to the wrist portion of the virtual robot arm (626) and virtual grabber elements (628, 630) and configured to function in providing a real user in a user workstation with a sense of touch or contact with the virtual robot structure to other aspects of the virtual environment (612), such as a portion of the virtual wall (616). In other words, if the user drives the virtual robot (622) such that the virtual grabber elements (628, 630) strike a portion of the virtual wall (616), such contact and / or intersection may be translated back to the touch translation interface (508, 512, 398) in the user workstation, which may help provide the user with an intuitive sense of activity within the virtual environment (612).

[0109] Referring to Figure 35, the user in the local workstation has connectivity to a virtual remote engagement configuration in the virtual remote environment, such as a virtual robot arm, which is operably coupled and has one or more connected virtual touch-sensing surfaces, in order to assist the user in physically engaging with one or more aspects of the virtual remote environment (564). The local workstation and the virtual remote engagement configuration may be powered on, started, and ready for virtual remote touch engagement by the user (566). The user may operate a master input device in the local workstation, which is operably coupled to the virtual remote engagement configuration (such as a virtual robot arm, which is operably coupled in the virtual remote environment), and physically engage with one or more aspects of the virtual remote environment (such as virtually physically engaging with the surface of an object in the virtual remote environment) (568). Through the local workstation, the user may be able to experience and understand aspects of virtual physical engagement between the virtual remote engagement workstation and one or more aspects of the virtual remote environment (for example, by locally perceiving various levels of virtual touch engagement in the virtual remote environment through the local workstation, a cuff touch sensor, which is virtually operably coupled to the distal part of a virtual robot arm in the virtual remote environment, may be configured to provide the user with an intuitive understanding of virtual touch engagement in the virtual remote environment via a local touch conversion interface, which can be coupled to the user and may be configured to locally provide one or more modalities of remote touch derivation feedback via a kinematically similar and / or intuitive local configuration of a local touch conversion interface, etc.) (570).

[0110] Referring to Figure 38A, an orthographic projection is shown, featuring a bushing or at least partially cylindrical type touch sensing assembly (656), which may be fixedly or detachably coupled to a structural element such as a shaft member (654) of a machine or mechanical component, which is preferably understood in terms of load configuration during operation. For illustrative purposes, the touch sensing assembly (656) is shown together with a shaft member (654) mounted on the upper surface (670) of a table (652), and the interface (726) between the touch sensing assembly (656) and the shaft member (654) may generally be bonded to prevent relative motion during loading. The touch sensing assembly (656) may be operably coupled to a computing system (144) via wired or wireless coupling, etc. (658, 230, 166), and may comprise a plurality of imaging devices (106) and sources (116). During operation, when the shaft member (654) is subjected to loads such as reciprocating bending (662, 660), a portion of the touch-sensing assembly (656) may be subjected to compression, tension, shear, and equivalent forces, and such loads may be detected and characterized in a computing system using associated imaging devices (106) and sources (116) (e.g., four pairs are shown around the periphery of the touch-sensing assembly 656) which may be installed within the sector. A side view of a similar configuration is shown in Figure 38B.

[0111] Figure 38C illustrates a configuration somewhat similar to that of Figure 38B, but with the addition of a structural cap member (668), which may be configured to restrict the touch-sensing assembly (656) to the confluence of the structural cap member (668) and the shaft member (654). Using such a configuration, the cylindrical touch-sensing assembly (656) can be subjected to purer compression or tension as the shaft member (654) is bent (662, 660).

[0112] Referring to Figure 38D, a configuration somewhat similar to that of Figure 38C is illustrated, but with a solid cylindrical touch-sensing assembly (672), which forms a cylindrical base or pad on which a structural cap (668) and shaft (654) end can be mounted (i.e., the shaft shown in Figure 38D does not intersect through the cylindrical touch-sensing assembly 672). Such a configuration also facilitates the detection of not only bending (662, 660) type loads but also tensile or compressive loads (667, 664) on the shaft member (654), and generally allows for a very wide range of characterization of the load paradigms within the associated structural member (654) depending on the source / imaging device (e.g., 116 / 106 in Figure 38A).

[0113] Referring to Figure 38E, it is important to note that the sensor and / or emitter portion may be installed in direct contact with the optical element material of the touch sensing assembly (656), as in the configuration of Figure 38A, or may be installed at a more distant location through the use of a configuration such as a fiber or bundle of fibers (132, 138) to operably couple to another location, such as the illustrated emission detection controller (734) module (and operably coupled to computing system 144 and power supply 102, 730, 732), which may contain an interface (764, 766) configured to efficiently transport light or other radiation to and from one or more sources and one or more image capture devices that can house therein.

[0114] Referring to Figure 38F, in scenarios such as periodic torsional load (758) centered on an axis (760) in mechanical applications, various aspects of the system configuration may be coupled to the mechanical part, connected wirelessly, and avoid various tether-based limitations in order to assist in the elimination of tethers and wired couplings. For example, referring to Figure 38F, a module or housing (742) may contain interconnected (752, 754) power source (744), battery charger (748), and computer / controller (746) elements, which may be a computing device (144) interconnected (756) to a touch-sensing assembly (656) and located more remotely via wireless connectivity (167, 166). A motion-based charger (748) featuring a small mass (750) configured to provide a low level of current based on the oscillating motion of an associated shaft (654) may be configured to continuously charge a battery (744), for example, the mass (750) may be configured to move a magnetic material in an oscillating manner through one or more coils, or to load a piezoelectric member (via angular acceleration and velocity square / radius relationship, etc.) using shaft motion to provide a low level of charging current for the battery (744).

[0115] Referring to Figure 39, a configuration somewhat similar to that of Figure 36 is shown, interconnected to a computing system (144) via wired or wireless connectivity, etc. (674, 676, 230, 166), with the addition of small touch-sensing assembly pads (678, 680) that provide further characterization of the opposing grass element of the grass tool (582) in a manner similar to the above description with respect to Figure 38D.

[0116] Referring to Figure 40, the user plans to perform a medical procedure on a patient using an electromechanical system such as a robot, configured to have an intervention tool such as a Graspa, which is integrated with one or more touch sensors, each featuring one or more deformable permeable layers (690). The user may start and calibrate the system using a computing system operably coupled between the electromechanical system and the user workstation (692). The user may be able to navigate the intervention tool toward the patient's anatomical structures from a workstation, which may be positioned near or remotely from the patient, and the workstation may include a display system configured to show aspects of the environment surrounding the intervention tool, a control interface such as a tactile interface to assist the user in providing commands to the intervention tool, and a touch conversion interface which may be configured to provide the user with input in response to contact or touch detected by one or more touch sensors operably coupled to the intervention tool (694). The user may use a control interface to make contact with a targeted tissue structure of a patient using an intervention tool, and perform one or more aspects of a medical procedure while obtaining and / or perceiving information about the environment adjacent to the intervention tool, such as contact between the intervention tool and the targeted tissue structure, which may be perceived and / or observed by using aspects of the user workstation such as a display system, a control interface, and / or a touch conversion interface (696). The user may complete the medical procedure or a part thereof by using the user workstation to move the intervention tool away from the targeted tissue structure and the patient (698).

[0117] Referring to Figure 41, the user may plan to perform procedures in a virtual environment such as a video game using a virtual electromechanical system such as a virtual robot, which may be configured to have a virtual tool such as a Graspa, integrated with one or more virtual touch sensors that can be operably coupled to one or more touch conversion interfaces (702). The user may start and calibrate the system using a computing system operably coupled between the virtual electromechanical system and the user workstation (704). The user may be able to navigate the virtual tool toward a virtual target from a workstation which may be positioned near or remotely from the patient, and the workstation may include a display system configured to show aspects of the environment surrounding the virtual tool, a control interface such as a tactile interface to assist the user in providing commands to the virtual tool, and a touch conversion interface which may be configured to provide input to the user in response to contact or touch detected by one or more virtual touch sensors operably coupled to the virtual tool (706). The user may use a control interface to make contact with one or more virtual objects using a virtual tool, and perform one or more aspects of a desired virtual tool movement while obtaining and / or perceiving information about the environment adjacent to the virtual tool, such as contact between the virtual tool and one or more virtual objects, which can be perceived and / or observed by using aspects of the user workstation such as a display system, a control interface, and / or a touch conversion interface (708). The user may complete the procedure or part thereof by virtually moving the virtual tool away from one or more virtual objects through the use of the user workstation (710).

[0118] Referring to Figure 42, the user may plan to perform a medical procedure on a patient using an intervention tool such as a Graspa, which is integrated with one or more touch sensors, each featuring one or more deformable permeable layers, and an electromechanical system such as a robot, which is configured to have one or more control sensors, each possibly featuring one or more deformable permeable layers (714). The user may start and calibrate the system using a computing system operably coupled between the electromechanical system and the user workstation (716). The user may be able to navigate the intervention tool toward the patient's anatomical structures from a workstation, which may be positioned near or remotely from the patient, and the workstation may include a display system configured to show aspects of the environment surrounding the intervention tool, a control interface such as a tactile interface to assist the user in providing commands to the intervention tool, and a touch conversion interface which may be configured to provide the user with input in response to contact or touch detected by one or more touch sensors operably coupled to the intervention tool (718). The user may use a control interface to make contact with a targeted tissue structure of a patient using an intervention tool, and perform one or more aspects of a medical procedure while obtaining and / or perceiving information about the environment adjacent to the intervention tool, such as contact between the intervention tool and the targeted tissue structure, which may be perceived and / or observed by using aspects of the user workstation such as a display system, a control interface, and / or a touch conversion interface (720). The user may complete the medical procedure or a part thereof by using the user workstation to move the intervention tool away from the targeted tissue structure and the patient (722).

[0119] Referring to Figure 43, the mechanical system may include structural members such as shafts, beams, or slender members, which may be loaded with bending, tension, and / or shear during the operation of the mechanical system, and which may be coupled to a sensing assembly having a deformable permeable layer (770). The sensing assembly may be operably coupled to a computing system and an imaging device so that at least one mode of loading and / or deformation of the structural member can be monitored using the computing system (772). The sensing assembly and the computing system may be initialized, calibrated, and / or configured to sense one or more sides of the structural member during the operation of the mechanical system (774). The computing system may be configured to provide an output for the operator regarding real-time or near-real-time load configuration of the mechanical system, such as load data relating to the structural member, which can be displayed for the operator, and / or an indication for the operator that one or more predetermined load thresholds have approached or been met within the mechanical system (776). The computing system may further be configured to facilitate changes in the operation of the mechanical system, such as a reduction in load demand or the shutdown of one or more aspects of the mechanical system, when the computing system determines that an overload condition is met, for example by comparing the output from the sensing assembly with one or more predetermined load thresholds (778).

[0120] Referring to Figure 44, a vehicle such as an automobile may comprise one or more structural components, such as one or more housings and / or support structures, which may be loaded with bending, tension, and / or shear during the operation of the vehicle, and which may be coupled to one or more sensing assemblies comprising one or more deformable permeable layers (780). One or more sensing assemblies may be operably coupled to a computing system and one or more imaging devices so that at least one mode of loading and / or deformation of one or more structural components can be monitored using the computing system (782). One or more sensing assemblies and a computing system may be initialized, calibrated, and / or configured to sense one or more sides of one or more structural components during the operation of one or more structural components and the vehicle (784). The computing system may be configured to provide an output for the operator regarding real-time or near-real-time load configuration of one or more structural components, such as load data, which can be displayed for the operator and / or used to create an indication for the operator that one or more predetermined load thresholds have approached or been met with respect to one or more structural components (786). The computing system may further be configured to facilitate changes in the operation of one or more structural components and / or other components of the vehicle, such as a reduction in the load demand of one or more operably coupled systems, components, or subsystems, when the computing system determines that an overload condition is met, for example by comparing the output from one or more sensing assemblies with one or more predetermined load thresholds (788).

[0121] Referring to Figure 45, the mechanical system may include structural members such as shafts, beams, or slender members, which may be loaded with bending, tension, and / or shear during the operation of the mechanical system, and which may be coupled to a sensing base assembly having a deformable permeable layer (790). The sensing base assembly may be operably coupled to a computing system and an imaging device so that at least one mode of loading and / or deformation of the structural member can be monitored using the computing system (792). The sensing base assembly and the computing system may be initialized, calibrated, and / or configured to sense one or more sides of the structural member during the operation of the mechanical system (794). The computing system may be configured to provide an output for the operator regarding real-time or near-real-time load configuration of the mechanical system, such as load data relating to the structural member, which can be displayed for the operator, and / or an indication for the operator that one or more predetermined load thresholds have approached or been met within the mechanical system (796). The computing system may further be configured to facilitate changes in the operation of the mechanical system, such as a reduction in the load demand on one or more sides of the mechanical system, or a shutdown, when the computing system determines that an overload condition is met, for example by comparing the output from the sensing base assembly with one or more predetermined load thresholds (798).

[0122] Referring to Figure 46, a user at a local workstation may have connectivity to a remote engagement configuration in the remote medical intervention environment, such as a medical robot arm that is operably coupled with one or more connected touch-sensitive surfaces, in order to assist the user in physically engaging with one or more aspects of the remote medical intervention environment (802). The local workstation and the remote engagement configuration may be powered on, started, and ready for remote medical touch engagement by the user (804). The user may operate a master input device at a local workstation that is operably coupled to the remote engagement configuration (such as a medical robot arm that is operably coupled in the remote environment) to physically engage with one or more aspects of the remote environment (such as physically engaging with the surface of an object in the remote environment, such as a targeted tissue structure) (806). Through the local workstation, the user may be able to experience and understand aspects of physical engagement between the remote engagement workstation and one or more aspects of the remote environment (for example, by locally perceiving various levels of touch engagement in the remote environment through the local workstation, a cuff touch sensor operably coupled to the distal part of a medical robot arm in the remote environment may be configured to provide the user with an intuitive understanding of touch engagement in the remote environment via a local touch conversion interface, etc., which may be coupled to the user and may be configured to locally provide one or more modalities of remote touch derivation feedback via a kinematically similar and / or intuitive local configuration of a local touch conversion interface, etc.) (808).

[0123] Referring to Figure 47, a user at a local workstation may have connectivity to a remote engagement configuration in the remote medical intervention environment, such as a medical robot arm, which is operably coupled with one or more connected touch-sensing surfaces, in order to assist the user in controlling the remote engagement configuration and physically engaging with one or more aspects of the remote medical intervention environment (810). The local workstation and the remote engagement configuration may be powered on, started, and ready for remote medical touch engagement by the user (812). The user may operate a master input device at the local workstation, which is operably coupled to the remote engagement configuration (such as a medical robot arm operably coupled in the remote environment), and physically engage with one or more aspects of the remote environment within one or more predetermined load limits that can be monitored for one or more loads applied to one or more connected touch-sensing surfaces (such as physically engaging with the surface of an object in the remote environment, such as a targeted tissue structure) (814). Through the local workstation, the user may experience and understand aspects of physical engagement between the remote engagement workstation and one or more aspects of the remote environment (for example, by locally perceiving various levels of touch engagement in the remote environment through the local workstation, a cuff touch sensor operably coupled to the distal part of a medical robot arm in the remote environment may be configured to provide the user with an intuitive understanding of touch engagement in the remote environment via a local touch conversion interface, etc., which may be coupled to the user and may be configured to locally provide one or more modalities of remote touch derivation feedback via a kinematically similar and / or intuitive local configuration of a local touch conversion interface, etc.), and may be able to physically engage with aspects of the remote medical intervention environment within one or more predetermined load limits that can be monitored for one or more loads applied to one or more connected touch-sensing surfaces (816).

[0124] Referring to Figure 48, an embodiment similar to that of Figure 29C is shown to illustrate a hybrid configuration of both touch sensing and touch conversion for each of two fingers (index finger 526, middle finger 528), with the addition of a cuff-style touch sensing interface (822, 820; similar to those 360, 362 described above with reference to Figure 18C, for example), which is detachably coupled to the finger and operablely coupled to a computing system via wired or wireless connectivity (510, 514), etc. (826, 824), and the touch conversion interfaces (508, 512) may be detachably coupled to each finger for kinematically similar feedback, as described above with reference to Figure 29C, for example. Such a configuration may not only provide the user with one or more senses that are intuitively related to activity in an interconnected system such as a remotely located robotic grappa, but may also be configured and operated to provide, for example, an interconnected computing system with further information about the local activity of the user's finger (for example, touch-sensing interfaces (822, 820) may be used to sense an increase or decrease in associated hoop stress or hoop strain that can correlate with the action, activity, movement or intent of the finger, as well as contact between the finger and other objects).

[0125] Therefore, referring to Figure 49, illustrative modifications are shown, and configurations such as those described above with reference to Figure 48 may be adopted. Referring to Figure 49, a user in the local workstation may have connectivity to a remote engagement configuration in the remote environment, such as a robot arm that is operably coupled with one or more connected touch-sensing surfaces, in order to assist the user in physically engaging with one or more aspects of the remote environment (830). The local workstation and the remote engagement configuration may be powered on, started, and ready for remote and local touch engagement by the user (832). The user may operate a master input device and a local touch-sensing configuration in the local workstation, both of which may be operably coupled through the computing system to a remote engagement configuration (such as a robot arm that is operably coupled in the remote environment) and physically engage with one or more aspects of the remote environment (such as physically engaging with the surface of an object in the remote environment) (834). Through the local workstation, user touch activity may be sensed and assist in the operation of the remote engagement configuration, and the user may be able to experience and understand aspects of physical engagement between the remote engagement workstation and one or more aspects of the remote environment (for example, by locally perceiving various levels of touch engagement in the remote environment through the local workstation, a cuff touch sensor operably coupled to the distal part of a robot arm in the remote environment may be configured to provide the user with an intuitive understanding of touch engagement in the remote environment via a local touch conversion interface, etc., which may be coupled to the user and may be configured to locally provide one or more modalities of remote touch derivation feedback via a kinematically similar and / or intuitive local configuration of a local touch conversion interface, etc.) (836).

[0126] Referring to Figure 50, a configuration similar to that in Figure 49 is illustrated, but the operator / user may operate within a synthesized or virtual environment using a similar hybrid local interface. Referring to Figure 50, the user in the local workstation may have connectivity to a virtual remote engagement configuration in the virtual remote environment, such as a operably coupled virtual robot arm with one or more connected virtual touch-sensing surfaces, to assist the user in physically engaging with one or more aspects of the virtual remote environment (840). The local workstation and the virtual remote engagement configuration may be powered on, started, and ready for virtual remote touch engagement by the user (842). The user may operate a master input device and a local touch-sensing configuration in the local workstation, both of which may be operably coupled to a virtual remote engagement configuration (such as a operably coupled virtual robot arm in the virtual remote environment) and physically engage with one or more aspects of the virtual remote environment (such as virtually physically engaging with the surface of an object in the virtual remote environment) (844). Touch activity relating to the user may be sensed through the local workstation and may assist in the operation of the virtual remote engagement configuration, and the user may be able to experience and understand aspects of virtual physical engagement between the virtual remote engagement workstation and one or more aspects of the virtual remote environment (for example, by locally perceiving various levels of virtual touch engagement in the virtual remote environment through the local workstation, a cuff touch sensor which is virtually operably coupled to the distal part of a virtual robot arm in the virtual remote environment may be configured to provide the user with an intuitive understanding of virtual touch engagement in the virtual remote environment via a local touch conversion interface which can be coupled to the user and may be configured to locally provide one or more modalities of remote touch derivation feedback via a kinematically similar and / or intuitive local configuration of a local touch conversion interface, etc.) (846).

[0127] Referring to Figure 51A, a system configuration similar to that described with reference to Figure 7A is illustrated, in which a touch-sensing assembly (146) featuring a deformable permeable layer (110) is configured to be placed in contact with the surface of an object to be characterized. In various embodiments, it may be useful to have a planar or semi-planar deformable permeable layer (110) in scenarios such as when it is desired to observe and characterize the fingerprint pattern of a finger pressed toward the surface of a banknote placed on a flat table or possibly toward the deformable permeable layer. Referring to Figure 51B, for comparative purposes, a smaller version of the touch-sensing assembly (146) configuration of Figure 51A is shown. Depending on the particular scenario, it may be desirable to have a touch-sensing assembly (146) featuring a deformable permeable layer having an unloaded shape other than the planar or semi-planar shape described above. For example, referring to Figure 51C, the touch-sensing assembly (146) is shown to have an arc-shaped deformable permeable layer (1020), which may be useful when dealing with arc-shaped or concave surfaces. Figures 51D and 51E illustrate variations featuring a deformable permeable layer shape that may be, for example, an ellipsoid (1022) or a hemisphere (1024). Figures 51F and 51G illustrate variations featuring a deformable permeable layer shape that may be a semi-ellipsoid or a nearly hemisphere (1026, 1028) with a proximal elongated portion, as shown. Configurations such as those illustrated in Figures 51F and 51G may be useful for inspecting and / or characterizing surfaces that may be, for example, concave or cylindrical.Referring to Figures 51H and 51I, the touch-sensing assembly (146) may be configured to have an expandable conduit or bladder so as to be inserted in a smaller and more elongated insertion configuration (i.e., a configuration in which the expansion conduit or bladder is not relatively expanded using gas or liquid, etc.) (1030), as shown in Figure 51H, and to engage with a surface such as a hole or cylindrical surface, and then, once positioned for measurement and / or surface characterization, the deformable permeable layer may be increased in volume (i.e., a configuration in which the expansion conduit or bladder is relatively expanded via positive pressure of gas or liquid, etc.) (1032) so as to be pressed against the surrounding targeted surface for measurement and / or surface characterization, and then deflated again, returning to the minimal configuration (1030) and removed. By using knowledge of the elastic modulus of the deformable permeable layer material together with high-precision deflection information about the surface, the interface load can be similarly characterized. In fact, by utilizing knowledge of the properties of deformable permeable layer materials, various properties of the interfaced material can be similarly determined by using specific load patterns at the interface. For example, in one embodiment, the response of a targeted surface detected through the deformable permeable layer can be used to estimate, measure, and / or determine aspects of the structural modulus of the interfaced structure, as well as the static and / or kinetic friction coefficients (i.e., by detecting the interface loads during the transition to the coefficient of motion before slippage that occurs when a load is applied and when the load continues to be applied after initial slippage). In addition to sliding, rolling-type deformable permeable layers, such as those comprising cylindrical or partially cylindrical deformable permeable layers, may also be utilized. Such configurations can be used to capture data as they roll along a targeted surface in a preferred rolling direction, as determined by the rolling degrees of freedom of the rolling deformable permeable layer (i.e., as in roll painting with a paint roller), and / or the roller may slide in a different direction (i.e., in a manner in which the paint roller would be rubbed in a direction not aligned with the preferred rolling direction of the paint roller against the wall).

[0128] The radii of curvature of the deformable permeable layers (1020, 1022, 1024, 1026, 1028, 1030, 1032) as shown in Figures 51C-51I may be configured to address the specific application. For example, in various embodiments, as described above, the radii of curvature may be selected to match, at least partially, the radii of curvature of the targeted surface. In other embodiments, relatively small radii of curvature, such as in the range of about 0.5 mm to about 5 mm, may be utilized to effectively characterize the location of a point in space. In other embodiments, the deformable permeable layer may include relatively high modulus or high hardness portions (such as relatively small spherical or cuboidal portions within the larger deformable permeable layer) located at known XY locations within the larger deformable permeable layer, providing effective point sensor functionality at those known points.

[0129] Referring to Figure 52, a touch-sensing assembly (146), such as the one illustrated in Figures 51A-51I, is illustrated with respect to an electromechanical arm (234), such as a robotic arm, which can be actively controlled, for example, via drive commands from a user or from a software-based controller. The arm (234) can be used to position and orient the touch-sensing assembly (146) using active electromechanical navigation and / or movement (via interconnected motors, etc.) so that a surface (1034), which can be supported by a mount or substrate (1036), can be characterized using the touch-sensing assembly (146).

[0130] Referring to FIG. 53, a configuration similar to that of FIG. 52 is illustrated, but rather than having active electromechanical movement provided by an associated articulating arm, the arm may be configured to be maneuvered by a user using one or more handles (1040, 1041) to position and orient it. The joints of the arm may be electromechanically braked such that a user can command the brake (1038) to hold a position and / or orientation in space (in other words, the arm may be configured with a clutch and to be clutch-released to facilitate manual movement by the user using the handle). The braked joint (1038) may have a joint position sensor such as an optical encoder and may be configured to assist in determining the joint position for determining the overall position and orientation of the touch sensing assembly (146) relative to a global coordinate system or the like.

[0131] Referring to FIG. 54, a configuration similar to that of FIG. 53 is shown, but with a passive (i.e., non-braked) joint (1042) such that a user can maneuver the touch sensing assembly (146) in space and manually engage a surface (1034), while the joint positions of the arm can be utilized to track the position and / or orientation of the touch sensing assembly (146) relative to a global coordinate system or the like.

[0132] Referring to Figure 55, the configuration is illustrated without support arms so that it can be manually held in a fixed position / orientation by an operator or user, for example, by using handles (1040, 1041) coupled to the main housing (1044), which is coupled to the touch-sensing assembly (146). Referring to the previously associated Figure 56, one or more tracking systems (1046) may be operably coupled to a computing device (104) via a wired or wireless connection (1048), etc., to assist in tracking the position and / or orientation of the touch-sensing assembly (146) in space with respect to the surface of interest (1034) and / or the global coordinate system (1050), and to assist in such position and / or orientation determination. For example, in various embodiments, an optical tracking configuration may be used, for example, using a tracking base point mounted on the housing (1044), or the touch-sensing assembly (146), and a detector (such as a stereo detector-based configuration with a 3D tracking system, for example, one available from Northern Digital, Inc. Similarly, electromagnetic tracking systems, such as those available from Ascension, Inc., may be used for tracking relative to a global coordinate system (1050), etc. In fact, referring to Figure 57, such a tracking system (1046) may be used in addition to kinematic-based tracking configurations (such as those employing arm 234). Furthermore, referring to Figure 58, for example, a configuration having several components common to Figure 13A is illustrated, and also includes tracking components such as those illustrated in Figure 57 for use in tracking and / or determining position and / or orientation relative to a global coordinate system (1050), etc. The illustrated imaging or image acquisition devices (270, 272) may comprise various detector types and may be used in a stereo configuration with a texture projector to assist in depth and other characterization, and to address occlusion that may occur at various positions and / or orientations of the assembly (146) (i.e., by being positioned at different viewpoint vectors toward the target surface).Furthermore, as described above, an image acquisition device residing within the touch-sensing assembly (146) may also be used for image acquisition through a deformable transparent layer. The acquisition of various images and / or data points may be induced in various ways, such as manually by the operator (through buttons, software, voice activation, remotely connected device triggers, and equivalents, by control interface activation, etc.) and / or automatically, such as through force limits, determined geometric or measured limits, etc., or based on the optical system or image acquisition device focus limits, etc.

[0133] Referring to FIG. 59A, a scenario is illustrated where a configuration similar to that of FIG. 58 has a touch sensing assembly (146) positioned and oriented to characterize various aspects of a mechanical portion (1126) of an engine block being manufactured. In various embodiments, an articulating arm (234) may be utilized to position and / or orient the touch sensing assembly (146) in various positions and orientations such that the surface of the engine block (1126) can be characterized. Further, a model of the engine block, such as an ideal “as designed” computer-aided design (“CAD”) model, may be stored on a memory device or system (1052), which may be operatively coupled to a computing system (144) via a wired or wireless connectivity (1054) or the like, and this model may be utilized in the analysis and observation of the mechanical portion (1126) of the engine block being inspected using the touch sensing assembly (146) via comparison to the ideal model or the like. In various embodiments, the model may be aligned to the position and orientation of the observed version by collecting a sequence of points and / or surfaces and determining alignment matches, etc., and then measurements may be taken from the actual part, for example, to determine compliance with the ideal model for quality assurance purposes. Indeed, in various embodiments, a digital representation version of the ideal model may be presented to illustrate changes, defects (e.g., more subtle issues such as geometric changes, scratches, and the like, and equivalents), and / or deviations from the ideal model (i.e., if a member is supposed to be straight in the ideal model but is bent in the measured model, it may be presented as bent in the digital representation version and visually highlighted as a deviation within the associated display interface via distinguishable coloring or the like).

[0134] Referring to Figure 59B, a configuration similar to that in Figure 59A is illustrated with the addition of a operably coupled measuring system (1120) and measuring probe (1118). The measuring probe (1118) may be configured to provide point determination in addition to (i.e., in parallel with) information gathered by other integrated system components (146, 234, 144, etc.). A suitable measuring probe (1118) may also be referred to as a “touch probe,” a “coordinate measuring machine probe,” or a “CMM probe” (where “CMM” generally refers to a coordinate measuring machine, which features a measuring probe and may be configured to provide measurements using such a probe). The measuring system may be operably coupled to a computing device (144) via wired or wireless connectivity (1122), etc. In addition to utilizing information from various discrete sensors in parallel (i.e., sensors 270, 272, 274, 1118, etc.), it may be desirable to utilize the image acquisition or detection capabilities of a touch-sensing assembly (146) for additional image acquisition or photographic detection. For example, in various embodiments, the deformable layer may be removed from the position between the detector / image acquisition element and the targeted object so that the detector / image acquisition element of the touch-sensing assembly (146) can be used to collect additional data about the targeted object. Various optical configurations and / or other refractive optical treatments, such as fixed or variable focus lenses (e.g., those that can be focused using electromechanical operation, variable fluid bladder, and equivalents), may be used to assist in such acquisition and analysis. Adaptive lenses may also be used to acquire three-dimensional surface topography about the targeted object. Furthermore, stereography (via simultaneous acquisition from two different sensors, or via time displacement by repositioning and / or reorienting a given sensor within adjacent acquisition timeframes) may also be used to assist in characterizing the targeted object.Referring still to Figure 59B, the robotic system (234) is illustrated to position and / or orient the touch-sensing assembly (146) toward the targeted object (1126). However, it is important to note that various forms of touch-sensing assemblies (i.e., planar, semi-planar, curved, finger-shaped, concave, convex, etc.) may be inserted, rubbed / triggered, rolled (i.e., using cylindrical and / or rotatable deformable permeable layers), and otherwise approached / interfaced, with varying levels of complexity, to assist in the analysis of the targeted object.

[0135] In the various embodiments described herein, the acquisition of information and / or images of a targeted object may be triggered automatically or manually, in some instances by user commands such as button presses (or other user interface commands, which may be from local or more remote interconnected systems or subsystems), voice commands, force / load-based automated threshold commands, geometry-based commands (e.g., when a given targeted feature, such as a hole, is fully visible), or focus-based commands (e.g., when an image of a targeted geometric feature is at the optimal focus position). The system may be configured to capture and retain images from specific locations, orientations, vectors, and equivalents, and some data and / or images from the system may be projected onto an image of the targeted object or multiple objects for user visualization assistance.

[0136] Referring to Figure 60A, it may be desirable to have a convenient interface for mechanically and / or electromechanically interface the touch-sensing assembly (146) and associated hardware to the arm (234). A set of detachable coupling interfaces (1056, 1058) may be configured such that they can be securely pressed and locked together during operation (e.g., as shown in Figures 60B, 60C, and 60D), and then conveniently discoupled later to return to the state shown in Figure 60A. Referring to Figure 60E, an interface configuration such as one of the meshing pairs (1056, 1058) is shown and has a plurality of protruding features (1060, 1062) and one or more cavity features (1064) as well as electronic engagement features (e.g., power leads may pass through contacts through interface 1066, and an information I / O interface may pass through contacts through interface 1068). Opposing / facing interfaces (for example, a projecting member configured to fit into a cavity 1064, the cavity configured to precisely engage with the projecting members 1060, 1062) may be conveniently coupled together in a known relative orientation. When desired, a screw (1070) may be rotated using a handle (1072) for temporary fixing during coupling to maintain engagement of the mechanical and electrical (1066, 1068) interfaces (i.e., to screw into and fix against the inserted projecting member that fits into the cavity 1064). Figure 60D illustrates an electronic and / or power coupling (232) that proceeds across the detachable engagement.

[0137] Referring to Figures 61A-61C, an intermediate adapter member (1057) may be used to adapt to coupling between two interfaces, which do not necessarily have to be designed to couple with each other (in other words, if A is not designed to couple with C, the adapter 1057 may be configured to provide a detachable coupling by making one side of the adapter detachable to A and the other side of the adapter detachable to C, i.e., A-(AB / BC)-C, where the "AB / BC" part of this simple expression is the adapter (1057)).

[0138] Referring to Figures 61D-61F, one or more variations of the structural or mounted members (358) may be used to demonstrate that a detachably connectable or detachable configuration (such as those shown in detached form in Figures 60A, 60B, 61A, 61B, and 61F), which is designed to be handheld as desired, can be instrumented in a manner similar to those shown with reference to the attached variations (e.g., Figures 58, 59A-B, etc.) to improve the ability to operate on a targeted surface and / or structure. For example, referring to Figures 61D and 61E, a sensing assembly (146) is shown and is still connected to a support structure such as a robotic arm (234). A modified version of Figure 61D has a more proximal mounting member (358) coupled to the main housing (1044), which has an image acquisition device (272), a LiDAR device (274), and an inertial measurement unit (IMU 1119; comprising one or more accelerometers and one or more gyroscopes, which may assist in sensing, for example, linear and angular acceleration) coupled thereto. An opposing operating handle (1040) may be used to mount or couple to the additional image acquisition device (270) and measurement probe (1118) so that the touch-sensing interface of the sensing assembly (146) can be manually or automatically monitored and / or positioned relative to other objects such as a targeted surface. Embodiments in Figures 61E and 61F illustrate similar instruments, but with a mounting structure (358) that transports the instruments (270, 272, 274, 1119, 1118) closer to the touch sensing interface of the sensing assembly (146) using a direct coupling of the mounting structure (358) to the sensing assembly (146). Figure 61F illustrates a handheld configuration that is detached from the proximal support robot arm (234) of Figure 61E so that it may be freely movable in space relative to other objects, while also illustrating a distal portion that can be tracked using the instruments (e.g., 270, 272, 274, 1119, 1118).For example, embodiments of Figure 61E or 61F may be used to perform tactile analysis of a targeted object within the reach of a sensing assembly (146) by electromechanically moving (61E) or manually moving (61F), for example, via individual touch / contact vectors or approaches, via repeating patterns of adjacent touch / contacts, via predetermined patterns (e.g., of adjacent touch / contacts), or via a more exploratory series of approaches for exploring and characterizing one or more geometric features that are conventionally uncharacterizable (e.g., below a hole or opening, or inside a defect, or a surface or feature that is extremely difficult to access or image), using a simultaneous localization and mapping ("SLAM") approach. In various embodiments, a operably coupled computing system may be configured and used to stitch together geometrically adjacent geometric profiles using interpolation of geometric profiles and their relative position and orientation, and / or to present to the user a two- or three-dimensional mapping of one or more geometric profiles relative to each other in a global coordinate system, etc., using a graphical user interface.

[0139] Referring to Figure 62, in one embodiment, a user desires to engage a surface, which may be convex, concave, saddle-shaped, cylindrical, or more complex or simple, with the sensing system being calibrated and positioned in close proximity to the targeted surface (1080). The user may navigate the sensing surface toward the targeted surface via an electromechanical arm or robotic manipulator, etc., using feedback to the user regarding the position and orientation of the sensing surface provided by the positioning platform (e.g., inverse kinematics, load cells, deflection sensors, joint positions) (1082). As the sensing surface is navigated closer to the targeted surface, the integrated sensing capabilities may facilitate the detection of the targeted surface and its features (for example, the system may be configured such that an integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (1084). The system may be configured to specifically create an event of contact between a sensing surface and a targeted surface (for example, repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact) (1086). The system may be configured to conform to the targeted surface and utilize a deformable permeable layer to characterize the surface and store information about the characterized targeted surface, such as its geometric profile, location, and / or orientation relative to a global or other coordinate system (1088).

[0140] Referring to Figure 63, in one embodiment, a user desires to engage a surface using the sensing system, which may be convex, concave, saddle-shaped, cylindrical, or more complex or simple, and the system may be calibrated and positioned in close proximity to the targeted surface (1080). The user may navigate the sensing surface toward the targeted surface via an electromechanical arm or robotic manipulator, etc., using feedback to the user regarding the position and orientation of the sensing surface provided by the positioning platform (e.g., inverse kinematics, load cells, deflection sensors, joint positions) (1082). As the sensing surface is navigated closer to the targeted surface, the integrated sensing capabilities may facilitate the detection of the targeted surface and its features (for example, the system may be configured such that an integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (1084). The system may be configured to specifically create an event of contact between a sensing surface and a targeted surface (for example, repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact), and the system may be configured to modify the shape or compliance of the sensing surface or associated substrate structure via controlled expansion or contraction of bladder and / or conduits using a fluid or gas (1092). The system may be configured to conform to the targeted surface and utilize a deformable permeable layer to characterize the surface and store information about the characterized targeted surface, such as its geometric profile, location, and / or orientation relative to a global or other coordinate system (1094). The system may again be configured to modify the shape or compliance of the sensing surface or associated substrate structure via controlled expansion or contraction of bladder and / or conduits using a fluid or gas (1096).

[0141] Referring to Figure 64, in one embodiment, a user desires to engage a surface using the sensing system, which may be convex, concave, saddle-shaped, cylindrical, or more complex or simple, and the system may be calibrated and positioned in close proximity to the targeted surface (1080). The user may navigate the sensing surface toward the targeted surface via an actively driven robotic arm, a manually positioned articulated arm with electromechanical brakes, a manually positioned articulated arm without electromechanical brakes, and / or a manually held and oriented electromechanical arm, which may have a tethered or tetherless configuration (1102). As the sensing surface is navigated closer to the targeted surface, the integrated sensing capabilities may facilitate the detection of the targeted surface and its features (for example, the system may be configured such that an integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (1104). The system may be configured to specifically create an event of contact between a sensing surface and a targeted surface (for example, repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact) (1106). The system may be configured to conform to the targeted surface and utilize a deformable permeable layer to characterize the surface and store information about the characterized targeted surface, such as its geometric profile, location, and / or orientation relative to a global or other coordinate system (1108).

[0142] Referring to Figure 65, in one embodiment, a user desires to engage a surface using the sensing system, which may be convex, concave, saddle-shaped, cylindrical, or more complex or simple, and the system may be calibrated and positioned in close proximity to the targeted surface (1080). The user may navigate the sensing surface toward the targeted surface via an actively driven robotic arm, a manually positioned articulated arm with electromechanical brakes, a manually positioned articulated arm without electromechanical brakes, and / or a manually held and oriented electromechanical arm, which may have a tethered or tetherless configuration (1102). As the sensing surface is navigated closer to the targeted surface, the integrated sensing capabilities may facilitate the detection of the targeted surface and its features (for example, the system may be configured such that an integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (1104). The system may be configured to specifically create contact events between a sensing surface and a targeted surface (for example, repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate contact) (1106). The system may conform to the targeted surface and utilize a deformable permeable layer to characterize the surface and store information about the characterized targeted surface, such as its geometric profile, location, and / or orientation relative to a global or other coordinate system (1108). The system may be configured to align the positions of points known to be on the surface with a known part of a known model so that the system becomes aligned (i.e., so that known position / orientation relationships are determined between the model and the measured surface), and the alignment may be automated, such as through automatic alignment, based on a sequence of captured points or surfaces during measurement, with the assistance of a neural network trained using data on the known model (1112).The system may be configured to determine measured dimensions, surface orientation, or differences between equivalents for quality assurance and / or inspection purposes (1114).

[0143] Referring to Figure 66A, the substrate (1130) structure or layer is shown with various forms of holes or defects, such as defects, which may be concave in geometric shape, at least partially. For example, one such illustrated hole (1132) may have a substantially cylindrical, cubic, or rectangular volume formed in the substrate (1130) by drilling or similar machinery, or by lithography or various other techniques, etc. As stated above, it may be desirable to characterize the hole (1132) by understanding its geometric shape, elasticity, regularity, material, and other factors. Referring again to Figure 66A, another hole (1134) may be coated in whole or partially with a layer (1152) using a layer of paint or primer, etc., which presents another opportunity for characterization. Holes or defects (1136) are also shown, which may be machined or formed to define threads (1154) through processes such as drilling and threading. Holes or defects (1138) are also shown, which may be lined at least partially with layers or corrosion or oxides (1156; for example, iron oxide or so-called "rust" in the case of iron substrate 1130, or aluminum oxide in the case of aluminum substrate 1130). Variations of holes or defects (1140) are also shown, which may combine various complexities such as threads and oxides (1158). Referring to Figure 66B, naturally, the defects of interest may or may not be generally regular in their geometric shape.Also shown are geometric shapes such as substantially regular geometric shapes (1132) such as roughly cylindrical, cubic, or rectangular geometric shapes; substantially regular geometric shapes that are narrower than roughly cylindrical or rectangular geometric shapes (1142); substantially regular geometric shapes that are deeper than roughly cylindrical or rectangular geometric shapes (1144); hole or defect (1146) geometric shapes with a substantially wider bottom portion (1160) compared to an upper portion (1162); or various composite and / or irregular hole or defect geometric shapes as illustrated in Figure 66B (elements 1148, 1150) or Figures 66D and 66E (elements 1166 and 1168; elements 1170 and 1172; each presenting a relatively elongated defect that passes through the entire substrate layer 1130). Figure 66C illustrates that relatively regular defect (1142) geometric shapes, such as those formed by drilling machines, may be relatively deep or may intersect the entire thickness (1164) or part of a particular substrate (1130) layer. All of these defects, holes, conduits, and / or partial concaves may, preferably, be inspected and characterized in detail using the technical configurations of this subject.

[0144] Referring to Figures 67A and 67B, in various embodiments, a mounting structure or elongated member (1176), such as a shaft, beam, or equivalent, may be used to support a tactile sensing assembly (element 1178 in an unextended form), such as those described above, which may feature one or more deformable transparent layers configured to engage with other objects or surfaces and provide feedback on the geometry and other aspects of the engaged surface based on electromagnetic transmission (such as variations in the wavelength of radiation from a lighting source such as an LED, as described above). In other words, in various embodiments, the above-described structure (146), etc., may be formed in a sensing surface and assembly (1174), such as those illustrated in Figures 66A-66E, which are specifically configured to assist in the characterization and analysis of holes and / or defects. Referring to Figure 67B, the expanded form (1180) of the tactile sensing assembly may be expanded by the injection of pressure (e.g., by the injection of a fluid such as water, saline solution, air, or an inert gas) as described above with reference to other geometric configurations (1182). The compressed or unexpanded form (1178) may be used for access and delivery to navigate or place the distal portion of the assembly (1174) into a hole or defect, while the expanded form (1180) may be used to assist in pressing various sides of the deformable permeable layer to engage with the surface of interest for characterization.

[0145] For example, referring to Figures 68A-68D, the assembly (1174) may be inserted into a defect or hole (1132) (1184) with its distal portion in a collapsed or unexpanded form (1178) for characterization and analysis, and then, as shown in Figures 68C and 68D, it may be controllably expanded (1132) and best conformed to the geometry of the defect or hole (1132). After such analysis, the unexpanded form may be retracted for the retraction of the assembly (1174).

[0146] Referring to Figure 69A, as described above, the interaction of radiation from various sides of one or more deformable transparent layers and within various illumination wavelength spectral regions may be utilized with various types of detectors such as image capture devices (which may be configured with an optical system for capturing radiation information, which can be used by an interconnected computing system to determine geometric information regarding the engagement of other surfaces or objects engaged with the deformable transparent layers, such as CCD or CMOS type image capture devices). Figure 69A shows one variation of an illustrative assembly (1174) that may be used to characterize holes or defects, which features five or more detectors or image acquisition devices (1186) with a capture field or field of view (1188), each operably coupled to a proximal component such as a power source, illumination source, computing system, control lead, or equivalent via a central communication assembly lead or conduit (1190), etc. (1192, via wired or wireless connectivity such as IEEE-802.11 or Bluetooth® style connectivity, etc., as described above with reference to various components). The detectors (1186) depicted are distributed to cover various overlapping areas of the assembly that may engage with other surfaces containing holes or defects, etc., using their various capture fields (1188). Also illustrated are operationally coupled (1192) secondary sensors (1194) such as ultrasonic transducers, eddy current sensors, magnetic inductance sensors, X-ray diffraction sensors, and thermal / infrared detectors, which may be used to further characterize holes or defects (for example, thermal / infrared may be used to characterize temperature, X-ray diffraction may be used to characterize material and / or stress to relaxation, ultrasonic waves may be used for time-of-flight analysis and / or surface reconstruction, and eddy currents and magnetic inductance may be used, for example, to characterize the thickness of various coatings or oxide layers on a bare metal substrate or other material).

[0147] Referring to Figures 69B-69F, one or more sensor assemblies (1186) are utilized to provide various circumferential coverages, such as 360 degrees around the deformable permeable layer (whether in extended or unextended form) of the sensing assembly (1174) from an orthogonal viewpoint (i.e., an "upward" or "axial" viewpoint of the elongated support member 1176), and the entire assembly (1174;1178) is rotated relative to the substrate of interest, so as performed using the configuration in Figure 69B or 69C. Furthermore, additional data may be captured regarding a portion of the substrate surrounding the assembly (1174;1178), or alternatively, the sensor assembly (1186) may be more widely distributed and capture the periphery outside the sensing assembly (1174;1178), as in the embodiments of Figures 69D, 69E, or 69F (note that the cross-sectional configuration does not have to be circular and may be approximately square or any other geometric shape, as in the depicted slice shown in Figure 69F).

[0148] Referring to Figure 70A, the sensing assembly (1174; 1178; 1180) may be configured to include a detector or image acquisition device, such as a miniature CMOS or CCD-style device (1196), where the sensor (1186) is deployed directly within the distal portion of the assembly (1174) as shown and coupled to other components via connectivity leads (1192) and / or wireless coupling. Referring to Figure 70B, another sensor (1198) configuration is shown, where the detector and / or image acquisition device, such as a CMOS or CCD-style device, may be optically coupled for data acquisition using one or more optical fibers (1200) which can be operably coupled to a lens (1198), such as a refractive lens, and which may be positioned more proximal and configured to have a specific acquisition field with respect to the interfaced object or substrate surface. Figures 70C and 70D illustrate a configuration in which one or more optical guide or waveguide transmission configurations (1204; 1206) and one or more reflective devices (1202) can be used to help position a detector and / or image acquisition device (1196), such as a CMOS or CCD-style device, in a more proximal location and / or preferred orientation for assembly or packaging into a sensing assembly (1174; 1178; 1180), while still being directly adjacent to the sensor (1186) engagement location to acquire information about the engaged object. Figure 70E illustrates a configuration in which an optical guide or waveguide assembly is operably coupled to a parabolic reflector structure (1212) configured to help acquire a circumferential acquisition field or field of view (1210) around the farthest end of the sensing assembly (1174; 1178; 1180).

[0149] Referring back to Figure 69A, it may be desirable to have multiple sensor packages coupled in close proximity to each other to assist in the characterization and / or analysis of neighboring structures of the engaged structures. Figure 71A illustrates a compact detector or image acquisition device (1196) with a capture field or field of view (1188) extending outward, and the compact detector or image acquisition device (1196) may be positioned directly adjacent to two other secondary sensors (1194). Figures 71B-71D illustrate modifications in which one or more portions of the capture field or field of view of the compact detector or image acquisition device (1196) may be sacrificed to adapt more directly to device alignment (e.g., by creating a portal 1214 traversing one or more portions of device 1196, such a portal may affect the alignment of the field of view or capture field of device 1196). Figure 71D illustrates a highly integrated assembly in which the primary detector or image acquisition device (1196) may be configured to utilize an associated deformable permeable layer to characterize surface interactions with an engaged structure or surface, and other devices (1194) may include, for example, an ultrasonic wave transducer, an eddy current sensor, a magnetic inductance sensor, an X-ray diffraction sensor, and a thermal / infrared detector, as described above.

[0150] Accordingly, in various embodiments, sensing assemblies (1774), such as those illustrated in various forms in Figures 67A-71D, can be used to characterize defects, holes, conduits, and other geometric shapes defined by substrate structures, such as those illustrated in Figures 66A-66E, through a process of positioning / orientation, engagement (which may involve expansion via injection / inflation and / or opening / expansion, etc.), and data acquisition / analysis. With respect to engagement of one or more surfaces or sides of the target substrate structure, in various embodiments, direct engagement via the exterior of the expanded form (1180) of the sensing assembly (1774) may produce desired results to aid in characterizing the target substrate structure. In another embodiment, it may be desirable for the expanded form (1180) to have an external geometric shape configured to have a predetermined geometric shape that can be conformed to one or more sides of the substrate surface so that changes, deltas, or unexpected geometric problems can be easily identified. For example, if it is known during manufacturing that a hole is drilled and threaded to have a nominal diameter size of 4 inches with a "rough" thread pattern of 4 threads per inch, then the sensing assembly (1774) may be easily inserted into position in a non-extended configuration, and in an extended geometric shape (1180), the outer surface may be prepared to have a surface profile that approximates such a 4-inch nominal diameter 4-thread-per-inch geometric shape, and then, when the substrate surface and the extended geometric shape (1180) surface are engaged at runtime / data acquisition, there is a high probability that a delta signal representing a change in the alignment between the sensing assembly (1774) and the engaged substrate surface may be due to anomalies in the geometric matching of these structures via oxide layers, foreign matter, plastic deformation of the substrate since manufacturing, and / or drilling / threading / manufacturing errors, and a precise mapping of unexpected deltas may be performed for further analysis.

[0151] Referring to Figure 72A, the sensing assembly described above may be manually operated in a handheld configuration via the use of a proximal housing or handle (1222) interface comprising the sensing assembly (1774), so that a user (1220) can manually operate the sensing assembly (1774) to perform, for example, yaw motion, pitch motion, roll motion, insertion, retraction, and rotation (1224) relative to a surface or object of interest (1034). Referring to Figure 72B, the sensing assembly (1774) may be coupled to another elongated instrument such as a manually maneuverable medical catheter (1226), which is coupled within an elongated catheter body (1228) and can be maneuvered in controllable manner in one or more axes and / or degrees of freedom using a pull wire or push wire (or push rod) which can be activated via manual operation at a proximal handle assembly (1230). Therefore, operation of the handle assembly (1230) can provide movement of the sensing assembly (1774) relative to the surface or object of interest (1034), e.g., yaw motion, pitch motion, roll motion, insertion, retraction, and rotation (1224). Referring to Figure 72C, an electromechanical configuration (234) such as a robot may be coupled using an interface coupling (1232), etc., which may include one or more load sensors (such as piezoelectric sensors for insertion / retraction, yaw, pitch, rotational moment, and equivalents) so that controlled electromechanical motion (from automation, user input in a master input device, and equivalents) can provide movement of the sensing assembly (1774) relative to the surface or object of interest (1034), e.g., yaw, pitch, roll, insertion, retraction, and rotation (1224).

[0152] Referring to Figures 73A and 73B, in various embodiments, a mechanical expander member (1236) may be inserted (1238) into the engagement geometry (1240) of the distal portion of a sensing assembly (1774), such as that shown in Figure 67A, to provide expansion (1182), as shown in Figure 73B. In other words, expansion may be performed not only via inflation as described above, but also mechanically via expansion, and furthermore, expansion may be performed by a hybrid of both mechanical expansion and inflation, as shown in the embodiment of Figure 73C, and an expansion conduit (1242) may be used in conjunction with the insertion of the expander member (1236) for expansion (1182).

[0153] Referring to Figures 74A-74C, various aspects of the procedure for characterizing a side of a defect, hole, conduit, or equivalent are illustrated. As shown in Figure 74A, the substrate (1130) defines an elongated defect, hole, or conduit (1172). The sensing assembly in its non-extended configuration (1178) may be inserted into the substrate (1130) to the position of interest (1244), as shown in Figures 74A and 74B. Referring to Figure 74C, the sensing assembly may be converted to an extended configuration (1180) to characterize various sides of the substrate directly surrounding it, and data may be obtained. In various embodiments, to continuously obtain data on additional portions of the elongated defect, hole, or conduit (1172), the sensing assembly (1174) may be pulled proximal backward or pushed forward to continuously capture data, or to discretely capture data. For example, in one modification, it may be desirable to maintain the extended configuration (1180) while vertical repositioning is performed, in which case it may be advantageous to continue capturing sustained data at a relatively high acquisition frequency or "frame rate," etc. In another modification, it may be desirable to return to the non-extended configuration (1178) before vertical repositioning, and then return to the extended configuration (1180) before resuming data acquisition.

[0154] By using each configuration in this specification, various aspects of data and image information may be compiled so that a user can visually recognize them within a visual user interface in an intuitive manner. For example, data regarding adjacent capture or property evaluation locations with respect to engaged objects or surfaces may be displayed adjacent to each other, and a boundary line or intersection point between adjacent images and / or data may be integrated, merged, or interpolated together via so-called "stitching" techniques or the like so that an intuitive representation of the target surface can be presented to the user. The graphical user interface may be configured to display a stitched representation of a targeted object or a plurality of objects, and in a similar manner to a style in which a computer-aided design ("CAD") interface may be configured to enable a user to navigate a three-dimensional object designed by the user, the user may be configured to enable navigation of the representation.

[0155] Referring to Figure 75, a user may wish to use the sensing system to engage with a targeted surface, which may be more complex or simpler, such as a hole, defect, at least a partial concave, tunnel, conduit, or surface roughness, edge sharpness, gap, offset, geometric tolerance, and equivalent, and the system may be calibrated and positioned in close proximity to the targeted surface (1252). The user may navigate the sensing surface toward the targeted surface via manual operation of an elongated instrument (e.g., via direct manual operation or via operation of interconnected instruments such as a manually maneuverable catheter) (1254). As the sensing surface is navigated closer to the targeted surface, the integrated sensing capabilities may facilitate the detection of the targeted surface and its features (e.g., the system may be configured such that an integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (1256). The system may be configured to specifically create an event of contact between a sensing surface and a targeted surface (for example, repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact) (1258). The system may be configured to conform to the targeted surface and utilize a deformable permeable layer to characterize the surface and store information about the characterized targeted surface, such as its geometric profile, location, and / or orientation relative to a global or other coordinate system (1260).

[0156] Referring to Figure 76, a user may wish to use the sensing system to engage with a targeted surface, which may be a hole, defect, at least a partial concave, tunnel, conduit, or more complex or simple, and the system may be calibrated and positioned in close proximity to the targeted surface (1252). The user may navigate the sensing surface toward the targeted surface via an electromechanical arm or robotic manipulator, etc., with feedback to the user regarding the position and orientation of the sensing surface provided by a positioning platform (inverse kinematics, load cell, deflection sensor, joint position, etc.) (1262). As the sensing surface is navigated closer to the targeted surface, the integrated sensing capabilities may facilitate the detection of the targeted surface and its features (for example, the system may be configured such that an integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (1264). The system may be configured to specifically create an event of contact between a sensing surface and a targeted surface (for example, repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact) (1266). The system may be configured to conform to the targeted surface and utilize a deformable permeable layer to characterize the surface and store information about the characterized targeted surface, such as its geometric profile, location, and / or orientation relative to a global or other coordinate system (1268).

[0157] Referring to Figure 77, a user may wish to use the sensing system to engage with a targeted surface, which may be a hole, defect, at least a partial concave, tunnel, conduit, or more complex or simple, and the system may be calibrated and positioned in close proximity to the targeted surface (1252). The user may navigate the sensing surface toward the targeted surface via manual operation of an elongated instrument (e.g., via direct manual operation or via operation of interconnected instruments such as a manually maneuverable catheter) (1254). As the sensing surface is navigated closer to the targeted surface, the integrated sensing capabilities may facilitate the detection of the targeted surface and its features (for example, the system may be configured such that an integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (1270). The system may be configured to specifically create an event of contact between a sensing surface and a targeted surface (for example, repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact), and the system may be configured to modify the shape or compliance of the sensing surface or associated substrate structure via controlled expansion or contraction of bladder and / or conduits using a fluid or gas (1272). The system may be configured to conform to the targeted surface and utilize a deformable permeable layer to characterize the surface and store information about the characterized targeted surface, such as its geometric profile, location, and / or orientation relative to a global or other coordinate system (1274). The system may again be configured to modify the shape or compliance of the sensing surface or associated substrate structure via controlled expansion or contraction of bladder and / or conduits using a fluid or gas (1276).

[0158] Referring to Figure 78, a user may wish to use the sensing system to engage with a targeted surface, which may be a hole, defect, at least a partial concave, tunnel, conduit, or more complex or simple, and the system may be calibrated and positioned in close proximity to the targeted surface (1252). The user may navigate the sensing surface toward the targeted surface via an electromechanical arm or robotic manipulator, etc., with feedback to the user regarding the position and orientation of the sensing surface provided by a positioning platform (inverse kinematics, load cell, deflection sensor, joint position, etc.) (1262). As the sensing surface is navigated closer to the targeted surface, the integrated sensing capabilities may facilitate the detection of the targeted surface and its features (for example, the system may be configured such that an integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (1278). The system may be configured to specifically create an event of contact between a sensing surface and a targeted surface (for example, repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact), and the system may be configured to modify the shape or compliance of the sensing surface or associated substrate structure via controlled expansion or contraction of bladder and / or conduits using a fluid or gas (1280). The system may be configured to conform to the targeted surface and utilize a deformable permeable layer to characterize the surface and store information about the characterized targeted surface, such as its geometric profile, location, and / or orientation relative to a global or other coordinate system (1282). The system may again be configured to modify the shape or compliance of the sensing surface or associated substrate structure via controlled expansion or contraction of bladder and / or conduits using a fluid or gas (1284).

[0159] Referring to Figure 79, a user may wish to use the sensing system to engage with a targeted surface, which may be a hole, defect, at least a partial concave, tunnel, conduit, or more complex or simple, and the system may be calibrated and positioned in close proximity to the targeted surface (1252). The user may navigate the sensing surface toward the targeted surface via manual operation of an elongated instrument, for example (e.g., via direct manual operation or via operation of interconnected instruments such as a manually maneuverable catheter) (1254). As the sensing surface is navigated closer to the targeted surface, the integrated sensing capabilities may facilitate the detection of the targeted surface and its features (for example, the system may be configured such that an integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (1286). The system may be configured to specifically create an event of contact between a sensing surface and a targeted surface (for example, repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact) (1288). The system may be configured to conform to the targeted surface and utilize a deformable permeable layer to characterize the surface and store information about the characterized targeted surface, such as its geometric profile, location, and / or orientation relative to a global or other coordinate system (1290).

[0160] Referring to Figure 80, a user may wish to use the sensing system to engage with a targeted surface, which may be a hole, defect, at least a partial concave, tunnel, conduit, or more complex or simple, and the system may be calibrated and positioned in close proximity to the targeted surface (1252). The user may navigate the sensing surface toward the targeted surface via an actively driven robotic arm, a manually positioned articulated arm with electromechanical brakes, a manually positioned articulated arm without electromechanical brakes, and / or an electromechanical arm that is manually held and oriented, which may have a tethered or tetherless configuration (1262). As the sensing surface is navigated closer to the targeted surface, the integrated sensing capabilities may facilitate the detection of the targeted surface and its features (for example, the system may be configured such that an integrated camera and LIDAR first detect the targeted surface, followed by other integrated sensing capabilities, which may be configured for sensing related to closer engagement) (1292). The system may be configured to specifically create an event of contact between a sensing surface and a targeted surface (for example, repositioning and reorientation of the sensing surface may be slowed, and auditory, visual, and / or tactile cues may be used to communicate the contact) (1294). The system may be configured to conform to the targeted surface and utilize a deformable permeable layer to characterize the surface and store information about the characterized targeted surface, such as its geometric profile, location, and / or orientation relative to a global or other coordinate system (1296).

[0161] Referring to Figure 81, a user may wish to use the sensing system to engage with a targeted surface, which may be a hole, defect, at least a partial concave, tunnel, conduit, or more complex o...

Claims

1. A system for evaluating geometric surface properties, a. A deformable permeable layer bonded to a mounting structure and an interface film, wherein the interface film is interfaced to at least one side of an interfaced object having a surface to be characterized, b. A first illumination source operably coupled to the deformable translucent layer using an illumination control layer, wherein the illumination control layer is configured to emit the first illumination light into the deformable translucent layer in one or more known first illumination orientations relative to the deformable translucent layer such that at least a portion of the first illumination light interacts with the deformable translucent layer, c. A detector configured to detect light from at least a portion of the deformable transparent layer, d. A computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transparent layer, to determine a surface orientation with respect to a position along the interface film based at least partially on the interaction between the first illumination light and the deformable transparent layer, and to characterize the geometric profile of the surface of the object interfaced with the interface film using the determined surface orientation. Equipped with, The system is configured such that the deformable permeable layer is controllably pressed against at least one side of an interfaced object having a surface to be characterized.

2. The system according to claim 1, wherein the deformable permeable layer is configured to expand fluidly to an operable bladder, which is controllably expanded from a collapsed state to an expanded state by injecting pressure.

3. The system according to claim 2, wherein the fluid is selected from the group consisting of air, inert gas, water, and physiological saline.

4. The system according to claim 2, wherein the bladder is an elastomer bladder that is interconnected between the deformable permeable layer and the mounting structure.

5. The system according to claim 1, wherein the deformable permeable layer is configured to be controllably expanded relative to the mounting structure by inserting a mechanical opening expander member.

6. The system according to claim 1, further comprising the computing system and a positioning sensor operably coupled to the deformable permeable layer.

7. The system according to claim 6, wherein the positioning sensor is configured to be used by the computing system to determine the position of at least a portion of the deformable transparent layer in a global coordinate system.

8. The system according to claim 7, wherein the computing system and the positioning sensor are further configured to determine the orientation of at least a portion of the deformable permeable layer within the global coordinate system.

9. The system according to claim 1, wherein the first lighting source comprises a light-emitting diode.

10. The system according to claim 1, wherein the detector is a photodetector.

11. The system according to claim 1, wherein the detector is an image acquisition device.

12. The system according to claim 11, wherein the image acquisition device is a CCD or a CMOS device.

13. The system according to claim 1, further comprising a lens operably coupled between the detector and the deformable permeable layer.

14. The system according to claim 1, wherein the computing system is operably coupled to the detector and configured to receive from the detector information regarding light from within the deformable transparent layer detected by the detector.

15. The system according to claim 1, wherein the computing system is operably coupled to the first light source and configured to control emissions from the first light source.

16. The system according to claim 1, further comprising a second illumination source operably coupled to the illumination control layer and configured to direct a second illumination into the illumination control layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source.

17. The system according to claim 16, wherein at least one of the first or second illumination wavelengths is in the infrared spectrum.

18. The system according to claim 16, wherein the first and second illumination wavelengths represent different colors.

19. The system according to claim 1, further comprising a second illuminator configured to introduce a second illumination light into the illumination control layer from a different position or orientation than that of the first illuminator.

20. The system according to claim 19, further comprising a third illuminator configured to introduce a third illumination light into the illumination control layer from a different position or orientation than that of the first and second illuminators.

21. The system according to claim 1, wherein the lighting control layer is configured to have a shape selected from the group consisting of planar, substantially planar, curved, convex, semi-convex, and saddle shape.

22. The system according to claim 1, further comprising a second illumination source operably coupled to a second illumination control layer and configured to direct a second illumination into the deformable transmissive layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source.

23. The system according to claim 22, wherein the first and second lighting control layers are stacked relative to each other.

24. The system according to claim 23, wherein the first and second lighting control layers are stacked directly adjacent to each other.

25. The system according to claim 1, wherein the illumination control layer is positioned between the detector and the deformable transparent layer.

26. The system according to claim 1, wherein the detector, illumination control layer, and deformable permeable layer are mechanically coupled within a fingertip assembly configured to form part of an elongated sensing structure.

27. The system according to claim 26, wherein the elongated sensing structure comprises a synthetic finger or a robotic hand component.

28. The system according to claim 26, wherein the detector, illumination control layer, and deformable permeable layer are operably coupled to a lens configured to create an optical path that provides a virtual camera position outside the geometric shape of the fingertip assembly to the deformable permeable layer.

29. The system according to claim 26, wherein the deformable permeable layer has a convex fingertip shape.

30. The system according to claim 26, wherein the deformable permeable layer is positioned within the fingertip assembly directly adjacent to the illumination control layer.

31. The system according to claim 26, wherein the deformable permeable layer is positioned separately from the illumination control layer within the fingertip assembly.

32. The system according to claim 1, wherein the deformable permeable layer comprises an elastomer material.

33. The system according to claim 32, wherein the elastomer material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluoroelastomer.

34. The system according to claim 32, wherein the deformable permeable layer comprises a synthetic material having a pigment material dispersed within an elastomer matrix, the pigment material being configured to provide illumination reflectance greater than that of the elastomer matrix.

35. The system according to claim 34, wherein the pigment material comprises a metal oxide.

36. The system according to claim 35, wherein the metal oxide is selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide.

37. The system according to claim 34, wherein the pigment material comprises metal nanoparticles.

38. The system according to claim 37, wherein the metal nanoparticles are selected from the group consisting of silver nanoparticles and aluminum nanoparticles.

39. The system according to claim 1, wherein the interface film comprises an elastomer material.

40. The system according to claim 1, wherein the interface film comprises an elastomer material.

41. The system according to claim 1, wherein the surface of the interfaced object is located and oriented within a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object interfaced with the interface film using the position and orientation with respect to the global coordinate system.

42. The system according to claim 41, wherein the computer system is configured to collect two or more geometric profiles of two or more portions of the surface of the object that are interfaced to the interface film, and to determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system.

43. The system according to claim 42, wherein the computing system is configured to provide a three-dimensional mapping of the two or more geometric profiles relative to each other in the global coordinate system.

44. The computing system according to claim 43, wherein the computing system is configured to stitch together geometrically adjacent geometric profiles using the interpolation of the geometric profiles and their relative position and orientation.

45. The system according to claim 41, further comprising a secondary sensor operably coupled to the computing system and configured to provide an input which can be used by the computing system to further geometrically characterize the surface of the interfaced object.

46. The system according to claim 45, wherein the secondary sensor is selected from the group consisting of an inertial measuring unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LiDAR device, a strain sensor, a load sensor, a temperature sensor, and an image acquisition device.

47. The system according to claim 46, wherein the secondary sensor comprises an IMU configured to output rotational and linear acceleration data to the computing system, and the computing system is configured to use the rotational and linear acceleration data to assist in characterizing the position or orientation of the deformable permeable layer in the global coordinate system.

48. The system according to claim 46, wherein the secondary sensor comprises an image capture device configured to capture image information relating to the surface of the interfaced object, and the computing system is configured to use the image information to help determine the location or orientation of the object relative to a deformable permeable layer.

49. The system according to claim 48, further comprising one or more tracking tags coupled to the interfaced object and one or more detectors operably coupled to the computing system, such that the computing system can be used at least partially to identify and provide location information relating to the interfaced object based on the predetermined locations of one or more tracking tags to the interfaced object.

50. The system according to claim 49, wherein the one or more tracking tags include radio frequency identification (RFID) tags, and the one or more detectors include RFID detectors.

51. A system for evaluating geometric surface properties, a. A deformable permeable layer bonded to a mounting structure and an interface film, wherein the interface film is interfaced to at least one side of an interfaced object having a surface to be characterized, b. A first illumination source operably coupled to the deformable translucent layer using an illumination control layer, wherein the illumination control layer is configured to emit the first illumination light into the deformable translucent layer in one or more known first illumination orientations relative to the deformable translucent layer such that at least a portion of the first illumination light interacts with the deformable translucent layer, c. A detector configured to detect light from at least a portion of the deformable transparent layer, d. A computing system configured to operate the detector to detect at least a portion of the light directed from the deformable transparent layer, to determine a surface orientation with respect to a position along the interface film based at least partially on the interaction between the first illumination light and the deformable transparent layer, and to characterize the geometric profile of the surface of the object interfaced with the interface film using the determined surface orientation, e. A robotic manipulator operably coupled to the computing system and the deformable permeable layer, wherein the robotic manipulator is configured to controllly position and orient the deformable permeable layer with respect to the interfaced object such that the computing system can characterize the geometric profile of the surface of the interfaced object so as to be interfaced with respect to the interface film, with respect to the relative position and orientation of the deformable permeable layer and the interfaced object. A system equipped with these features.

52. The system according to claim 51, wherein the robotic manipulator comprises a robotic arm.

53. The system according to claim 52, wherein the robot arm comprises a plurality of joints connected by substantially rigid linkage members.

54. The system according to claim 51, wherein the robotic manipulator is equipped with a flexible robotic instrument.

55. The system according to claim 51, further comprising an end effector coupled to the robot manipulator.

56. The system according to claim 55, wherein the end effector comprises a Graspa.

57. The system according to claim 51, wherein the deformable permeable layer is configured to expand fluidly to an operable bonded bladder, and is controllably expanded from a collapsed state to an expanded state by the injection of pressure.

58. The system according to claim 57, wherein the fluid is selected from the group consisting of air, an inert gas, water, and physiological saline.

59. The system according to claim 57, wherein the bladder is an elastomer bladder that is interconnected between the deformable permeable layer and the mounting structure.

60. The system according to claim 51, wherein the deformable permeable layer is configured to be controllably expanded relative to the mounting structure by inserting a mechanical opening and expanding member.

61. The system according to claim 51, further comprising the computing system and a positioning sensor operably coupled to the deformable permeable layer.

62. The system according to claim 61, wherein the positioning sensor is configured to be used by the computing system to determine the position of at least a portion of the deformable transparent layer in a global coordinate system.

63. The system according to claim 62, wherein the computing system and the positioning sensor are further configured to determine the orientation of at least a portion of the deformable permeable layer within the global coordinate system.

64. The system according to claim 51, wherein the first lighting source comprises a light-emitting diode.

65. The system according to claim 51, wherein the detector is a photodetector.

66. The system according to claim 51, wherein the detector is an image acquisition device.

67. The system according to claim 66, wherein the image acquisition device is a CCD or a CMOS device.

68. The system according to claim 51, further comprising a lens operably coupled between the detector and the deformable permeable layer.

69. The system according to claim 51, wherein the computing system is operably coupled to the detector and configured to receive from the detector information relating to light from within the deformable transparent layer detected by the detector.

70. The system according to claim 51, wherein the computing system is operably coupled to the first light source and configured to control emissions from the first light source.

71. The system according to claim 51, further comprising a second illumination source operably coupled to the illumination control layer and configured to direct a second illumination into the illumination control layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source.

72. The system according to claim 71, wherein at least one of the first or second illumination wavelengths is in the infrared spectrum.

73. The system according to claim 71, wherein the first and second illumination wavelengths represent different colors.

74. The system according to claim 51, further comprising a second illuminator configured to introduce a second illumination light into the illumination control layer from a different position or orientation than that of the first illuminator.

75. The system according to claim 74, further comprising a third illuminator configured to introduce a third illumination light into the illumination control layer from a different position or orientation than that of the first and second illuminators.

76. The system according to claim 51, wherein the lighting control layer is configured to have a shape selected from the group consisting of planar, substantially planar, curved, convex, semi-convex, and saddle shape.

77. The system according to claim 51, further comprising a second illumination source operably coupled to a second illumination control layer and configured to direct a second illumination into the deformable transmissive layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source.

78. The system according to claim 77, wherein the first and second lighting control layers are stacked relative to each other.

79. The system according to claim 78, wherein the first and second lighting control layers are stacked directly adjacent to each other.

80. The system according to claim 51, wherein the illumination control layer is positioned between the detector and the deformable transparent layer.

81. The system according to claim 51, wherein the detector, illumination control layer, and deformable permeable layer are mechanically coupled within a fingertip assembly configured to form part of an elongated sensing structure.

82. The system according to claim 81, wherein the elongated sensing structure comprises a synthetic finger or a robotic hand component.

83. The system according to claim 81, wherein the detector, illumination control layer, and deformable permeable layer are operably coupled to a lens configured to create an optical path that provides a virtual camera position outside the geometric shape of the fingertip assembly to the deformable permeable layer.

84. The system according to claim 81, wherein the deformable permeable layer has a convex fingertip shape.

85. The system according to claim 81, wherein the deformable permeable layer is positioned within the fingertip assembly directly adjacent to the illumination control layer.

86. The system according to claim 81, wherein the deformable permeable layer is positioned separately from the illumination control layer within the fingertip assembly.

87. The system according to claim 51, wherein the deformable permeable layer comprises an elastomer material.

88. The system according to claim 87, wherein the elastomer material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluoroelastomer.

89. The system according to claim 87, wherein the deformable permeable layer comprises a synthetic material having a pigment material dispersed within an elastomer matrix, the pigment material configured to provide illumination reflectance greater than that of the elastomer matrix.

90. The system according to claim 89, wherein the pigment material comprises a metal oxide.

91. The system according to claim 90, wherein the metal oxide is selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide.

92. The system according to claim 89, wherein the pigment material comprises metal nanoparticles.

93. The system according to claim 92, wherein the metal nanoparticles are selected from the group consisting of silver nanoparticles and aluminum nanoparticles.

94. The system according to claim 51, wherein the interface film comprises an elastomer material.

95. The system according to claim 51, wherein the interface film comprises an elastomer material.

96. The system according to claim 51, wherein the surface of the interfaced object is located and oriented in a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object interfaced with the interface film using the position and orientation with respect to the global coordinate system.

97. The system according to claim 96, wherein the computer system is configured to collect two or more geometric profiles of two or more portions of the surface of the object that are interfaced to the interface film, and to determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system.

98. The system according to claim 97, wherein the computing system is configured to provide a three-dimensional mapping of the two or more geometric profiles relative to each other in the global coordinate system.

99. The computing system according to claim 98, wherein the computing system is configured to stitch together geometrically adjacent geometric profiles using interpolation of the geometric profiles and their relative position and orientation.

100. The system according to claim 96, further comprising a secondary sensor operably coupled to the computing system and configured to provide an input which can be used by the computing system to further geometrically characterize the surface of the interfaced object.

101. The system according to claim 100, wherein the secondary sensor is selected from the group consisting of an inertial measuring unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LiDAR device, a strain sensor, a load sensor, a temperature sensor, and an image acquisition device.

102. The system according to claim 101, wherein the secondary sensor comprises an IMU configured to output rotational and linear acceleration data to the computing system, and the computing system is configured to use the rotational and linear acceleration data to assist in characterizing the position or orientation of the deformable permeable layer in the global coordinate system.

103. The system according to claim 101, wherein the secondary sensor comprises an image capture device configured to capture image information relating to the surface of the interfaced object, and the computing system is configured to use the image information to help determine the location or orientation of the object relative to a deformable permeable layer.

104. The system according to claim 103, further comprising one or more tracking tags coupled to the interfaced object and one or more detectors operably coupled to the computing system, such that the computing system can be used at least partially to identify and provide location information relating to the interfaced object based on the predetermined locations of one or more tracking tags to the interfaced object.

105. The system according to claim 104, wherein the one or more tracking tags include radio frequency identification (RFID) tags, and the one or more detectors include RFID detectors.

106. A method for evaluating geometric surface properties, a. To provide a deformable permeable layer coupled to a mounting structure and an interface film, wherein the interface film is interfaced to at least one side of an interfaced object having a surface to be characterized, b. Providing a first illumination source operably coupled to the deformable translucent layer using an illumination control layer, wherein the illumination control layer is configured to emit the first illumination light into the deformable translucent layer in one or more known first illumination orientations relative to the deformable translucent layer such that at least a portion of the first illumination light interacts with the deformable translucent layer. c. To provide a detector configured to detect light from within at least a portion of the deformable transparent layer, d. To provide a computing system, the computing system is configured to operate the detector to detect at least a portion of the light directed from the deformable transparent layer, to determine a surface orientation with respect to a position along the interface film based at least partially on the interaction between the first illumination light and the deformable transparent layer, and to characterize the geometric profile of the surface of the object interfaced with the interface film using the determined surface orientation. Includes, A method wherein the deformable permeable layer is configured to be controllably pressed against at least one side of an interfaced object having a surface to be characterized.

107. The method according to claim 106, wherein the deformable permeable layer is configured to expand fluidly to an operable bonded bladder, and is controllably expanded from a collapsed state to an expanded state by the injection of pressure.

108. The method according to claim 107, wherein the fluid is selected from the group consisting of air, inert gas, water, and physiological saline.

109. The method according to claim 107, wherein the bladder is an elastomer bladder that is interconnected between the deformable permeable layer and the mounting structure.

110. The method according to claim 106, wherein the deformable permeable layer is configured to be controllably expanded relative to the mounting structure by inserting a mechanical opening expander member.

111. The method according to claim 106, further comprising the computing system and a positioning sensor operably coupled to the deformable permeable layer.

112. The method according to claim 111, wherein the positioning sensor is configured to be used by the computing system to determine the position of at least a portion of the deformable transparent layer in a global coordinate system.

113. The method according to claim 112, wherein the computing system and the positioning sensor are further configured to determine the orientation of at least a portion of the deformable permeable layer in the global coordinate system.

114. The method according to claim 106, wherein the first lighting source comprises a light-emitting diode.

115. The method according to claim 106, wherein the detector is a photodetector.

116. The method according to claim 106, wherein the detector is an image acquisition device.

117. The method according to claim 116, wherein the image acquisition device is a CCD or a CMOS device.

118. The method according to claim 106, further comprising providing a lens operably coupled between the detector and the deformable permeable layer.

119. The method according to claim 106, wherein the computing system is operably coupled to the detector and configured to receive from the detector information relating to light from within the deformable transparent layer detected by the detector.

120. The method according to claim 106, wherein the computing system is operably coupled to the first illuminator and configured to control emissions from the first illuminator.

121. The method according to claim 106, further comprising providing a second illumination source operably coupled to the illumination control layer and configured to direct a second illumination into the illumination control layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source.

122. The method according to claim 121, wherein at least one of the first or second illumination wavelengths is in the infrared spectrum.

123. The method according to claim 121, wherein the first and second illumination wavelengths represent different colors.

124. The method according to claim 106, further comprising providing a second illuminating source configured to introduce a second illuminating light into the illuminating control layer from a different position or orientation than that of the first illuminating source.

125. The method according to claim 124, further comprising providing a third illuminator configured to introduce a third illumination light into the illumination control layer from a different position or orientation than that of the first and second illuminators.

126. The method according to claim 106, wherein the illumination control layer is configured to have a shape selected from the group consisting of planar, substantially planar, curved, convex, semi-convex, and saddle shape.

127. The method according to claim 106, further comprising providing a second illumination source operably coupled to a second illumination control layer and configured to direct a second illumination into the deformable transmissive layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source.

128. The method according to claim 127, wherein the first and second lighting control layers are stacked relative to each other.

129. The method according to claim 128, wherein the first and second lighting control layers are stacked directly adjacent to each other.

130. The method according to claim 106, wherein the illumination control layer is positioned between the detector and the deformable transparent layer.

131. The method according to claim 106, wherein the detector, illumination control layer, and deformable permeable layer are mechanically coupled within a fingertip assembly configured to form part of an elongated sensing structure.

132. The method according to claim 131, wherein the elongated sensing structure comprises a synthetic finger or a robotic hand component.

133. The method according to claim 131, wherein the detector, illumination control layer, and deformable permeable layer are operably coupled to a lens configured to create an optical path that provides a virtual camera position outside the geometric shape of the fingertip assembly to the deformable permeable layer.

134. The method according to claim 131, wherein the deformable permeable layer has a convex fingertip shape.

135. The method according to claim 131, wherein the deformable permeable layer is positioned within the fingertip assembly directly adjacent to the illumination control layer.

136. The method according to claim 131, wherein the deformable permeable layer is positioned separately from the illumination control layer within the fingertip assembly.

137. The method according to claim 106, wherein the deformable permeable layer comprises an elastomer material.

138. The method according to claim 137, wherein the elastomer material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluoroelastomer.

139. The method according to claim 137, wherein the deformable permeable layer comprises a synthetic material having a pigment material dispersed within an elastomer matrix, and the pigment material is configured to provide illumination reflectance greater than that of the elastomer matrix.

140. The method according to claim 139, wherein the pigment material comprises a metal oxide.

141. The method according to claim 140, wherein the metal oxide is selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide.

142. The method according to claim 139, wherein the pigment material comprises metal nanoparticles.

143. The method according to claim 142, wherein the metal nanoparticles are selected from the group consisting of silver nanoparticles and aluminum nanoparticles.

144. The method according to claim 106, wherein the interface film comprises an elastomer material.

145. The method according to claim 106, wherein the interface film comprises an elastomer material.

146. The method according to claim 106, wherein the surface of the interfaced object is located and oriented in a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object interfaced with the interface film using the position and orientation with respect to the global coordinate system.

147. The method according to claim 146, wherein the computer system is configured to collect two or more geometric profiles of two or more portions of the surface of the object that are interfaced to the interface film, and to determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system.

148. The method according to claim 147, wherein the computing system is configured to provide a three-dimensional mapping of the two or more geometric profiles relative to each other in the global coordinate system.

149. The method according to claim 148, wherein the computing system is configured to stitch together geometrically adjacent geometric profiles using interpolation of the geometric profiles and their relative position and orientation.

150. The method according to claim 146, further comprising providing a secondary sensor operably coupled to the computing system and configured to provide an input which can be used by the computing system to further geometrically characterize the surface of the interfaced object.

151. The method according to claim 150, wherein the secondary sensor is selected from the group consisting of an inertial measuring unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LiDAR device, a strain sensor, a load sensor, a temperature sensor, and an image acquisition device.

152. The method according to claim 151, wherein the secondary sensor comprises an IMU configured to output rotational and linear acceleration data to the computing system, and the computing system is configured to use the rotational and linear acceleration data to assist in characterizing the position or orientation of the deformable permeable layer in the global coordinate system.

153. The method according to claim 151, wherein the secondary sensor comprises an image capture device configured to capture image information relating to the surface of the interfaced object, and the computing system is configured to use the image information to help determine the location or orientation of the object relative to a deformable permeable layer.

154. The method according to claim 153, further comprising providing one or more tracking tags coupled to the interfaced object and one or more detectors operably coupled to the computing system, such that the computing system can be used, at least in part, to identify and provide location information relating to the interfaced object based on the predetermined locations of one or more tracking tags to the interfaced object.

155. The method according to claim 154, wherein the one or more tracking tags include radio frequency identification (RFID) tags, and the one or more detectors include RFID detectors.

156. A method for evaluating geometric surface properties, a. To provide a deformable permeable layer coupled to a mounting structure and an interface film, wherein the interface film is interfaced to at least one side of an interfaced object having a surface to be characterized, b. Providing a first illumination source operably coupled to the deformable translucent layer using an illumination control layer, wherein the illumination control layer is configured to emit the first illumination light into the deformable translucent layer in one or more known first illumination orientations relative to the deformable translucent layer such that at least a portion of the first illumination light interacts with the deformable translucent layer. c. To provide a detector configured to detect light from within at least a portion of the deformable transparent layer, d. To provide a computing system, the computing system is configured to operate the detector, detect at least a portion of the light directed from the deformable transparent layer, determine at least partially a surface orientation with respect to a position along the interface film based on the interaction between the first illumination light and the deformable transparent layer, and use the determined surface orientation to characterize the geometric profile of the surface of the object interfaced with the interface film. e. To provide a robotic manipulator operably coupled to the computing system and the deformable permeable layer, wherein the robotic manipulator is configured to controllly position and orient the deformable permeable layer with respect to the interfaced object such that the computing system can characterize the geometric profile of the surface of the interfaced object so as to be interfaced with respect to the interface film with respect to the relative position and orientation of the deformable permeable layer and the interfaced object. Methods that include...

157. The method according to claim 156, wherein the robot manipulator comprises a robot arm.

158. The method according to claim 157, wherein the robot arm comprises a plurality of joints connected by substantially rigid linkage members.

159. The method according to claim 156, wherein the robot manipulator is equipped with a flexible robot instrument.

160. The method according to claim 156, further comprising providing an end effector that is coupled to a robotic manipulator.

161. The method according to claim 160, wherein the end effector comprises a Graspa.

162. The method according to claim 156, wherein the deformable permeable layer is configured to expand fluidly to a movably coupled bladder, and is controllably expanded from a collapsed state to an expanded state by the injection of pressure.

163. The method according to claim 162, wherein the fluid is selected from the group consisting of air, inert gas, water, and physiological saline.

164. The method according to claim 162, wherein the bladder is an elastomer bladder that is interconnected between the deformable permeable layer and the mounting structure.

165. The method according to claim 156, wherein the deformable permeable layer is configured to be controllably expanded relative to the mounting structure by inserting a mechanical opening expander member.

166. The method according to claim 156, further comprising the computing system and a positioning sensor operably coupled to the deformable permeable layer.

167. The method according to claim 166, wherein the positioning sensor is configured to be used by the computing system to determine the position of at least a portion of the deformable transparent layer in a global coordinate system.

168. The method according to claim 167, wherein the computing system and the positioning sensor are further configured to determine the orientation of at least a portion of the deformable permeable layer within the global coordinate system.

169. The method according to claim 156, wherein the first lighting source comprises a light-emitting diode.

170. The method according to claim 156, wherein the detector is a photodetector.

171. The method according to claim 156, wherein the detector is an image acquisition device.

172. The method according to claim 171, wherein the image acquisition device is a CCD or a CMOS device.

173. The method according to claim 156, further comprising providing a lens operably coupled between the detector and the deformable permeable layer.

174. The method according to claim 156, wherein the computing system is operably coupled to the detector and configured to receive from the detector information regarding light from within the deformable transparent layer detected by the detector.

175. The method according to claim 156, wherein the computing system is operably coupled to the first illuminator and configured to control emissions from the first illuminator.

176. The method according to claim 156, further comprising providing a second illumination source operably coupled to the illumination control layer and configured to direct a second illumination into the illumination control layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source.

177. The method according to claim 176, wherein at least one of the first or second illumination wavelengths is in the infrared spectrum.

178. The method according to claim 176, wherein the first and second illumination wavelengths represent different colors.

179. The method according to claim 156, further comprising providing a second illuminator configured to introduce a second illumination light into the illumination control layer from a different position or orientation than that of the first illuminator.

180. The method according to claim 179, further comprising providing a third illuminator configured to introduce a third illumination light into the illumination control layer from a different position or orientation than that of the first and second illuminators.

181. The method according to claim 156, wherein the lighting control layer is configured to have a shape selected from the group consisting of planar, substantially planar, curved, convex, semi-convex, and saddle shape.

182. The method according to claim 156, further comprising providing a second illumination source operably coupled to a second illumination control layer and configured to direct a second illumination into the deformable transmissive layer using a second illumination wavelength different from the first illumination wavelength of the first illumination source.

183. The method according to claim 182, wherein the first and second lighting control layers are stacked relative to each other.

184. The method according to claim 183, wherein the first and second lighting control layers are stacked directly adjacent to each other.

185. The method according to claim 156, wherein the illumination control layer is positioned between the detector and the deformable transparent layer.

186. The method according to claim 156, wherein the detector, illumination control layer, and deformable permeable layer are mechanically coupled within a fingertip assembly configured to form part of an elongated sensing structure.

187. The method according to claim 186, wherein the elongated sensing structure comprises a synthetic finger or a robotic hand component.

188. The method according to claim 186, wherein the detector, illumination control layer, and deformable permeable layer are operably coupled to a lens configured to create an optical path that provides a virtual camera position outside the geometric shape of the fingertip assembly to the deformable permeable layer.

189. The method according to claim 186, wherein the deformable permeable layer has a convex fingertip shape.

190. The method according to claim 186, wherein the deformable permeable layer is positioned in the fingertip assembly directly adjacent to the illumination control layer.

191. The method according to claim 186, wherein the deformable permeable layer is positioned separately from the illumination control layer within the fingertip assembly.

192. The method according to claim 156, wherein the deformable permeable layer comprises an elastomer material.

193. The method according to claim 192, wherein the elastomer material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluoroelastomer.

194. The deformable permeable layer comprises a synthetic material having a pigment material dispersed within an elastomer matrix, wherein the pigment material is configured to provide illumination reflectance exceeding that of the elastomer matrix. The method according to claim 192.

195. The method according to claim 194, wherein the pigment material comprises a metal oxide.

196. The method according to claim 195, wherein the metal oxide is selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide.

197. The method according to claim 194, wherein the pigment material comprises metal nanoparticles.

198. The method according to claim 197, wherein the metal nanoparticles are selected from the group consisting of silver nanoparticles and aluminum nanoparticles.

199. The method according to claim 156, wherein the interface film comprises an elastomer material.

200. The method according to claim 156, wherein the interface film comprises an elastomer material.

201. The method according to claim 156, wherein the surface of the interfaced object is located and oriented in a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object interfaced with the interface film using the position and orientation with respect to the global coordinate system.

202. The method according to claim 201, wherein the computer system is configured to collect two or more geometric profiles of two or more portions of the surface of the object that are interfaced to the interface film, and to determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system.

203. The method according to claim 202, wherein the computing system is configured to provide a three-dimensional mapping of the two or more geometric profiles relative to each other in the global coordinate system.

204. The method according to claim 203, wherein the computing system is configured to stitch together geometrically adjacent geometric profiles using interpolation of the geometric profiles and their relative position and orientation.

205. The method according to claim 201, further comprising providing a secondary sensor operably coupled to the computing system and configured to provide an input which can be used by the computing system to further geometrically characterize the surface of the interfaced object.

206. The method according to claim 205, wherein the secondary sensor is selected from the group consisting of an inertial measuring unit (IMU), a capacitive touch sensor, a resistive touch sensor, a LiDAR device, a strain sensor, a load sensor, a temperature sensor, and an image acquisition device.

207. The method according to claim 206, wherein the secondary sensor comprises an IMU configured to output rotational and linear acceleration data to the computing system, and the computing system is configured to use the rotational and linear acceleration data to assist in characterizing the position or orientation of the deformable permeable layer in the global coordinate system.

208. The method according to claim 206, wherein the secondary sensor comprises an image capture device configured to capture image information relating to the surface of the interfaced object, and the computing system is configured to use the image information to help determine the location or orientation of the object relative to the deformable permeable layer.

209. The method according to claim 208, further comprising providing one or more tracking tags coupled to the interfaced object and one or more detectors operably coupled to the computing system, such that the computing system can be used at least partially to identify and provide location information relating to the interfaced object based on the predetermined locations of one or more tracking tags to the interfaced object.

210. The method according to claim 209, wherein the one or more tracking tags include radio frequency identification (RFID) tags, and the one or more detectors include RFID detectors.

Images