Systems and methods for tactile intelligence

Abstract

One embodiment relates to a system for geometric surface characterization, comprising: a deformable transmission layer coupled to a mounting structure and an interface membrane, wherein the interface membrane interfaces with at least one aspect of a docked object; a first illumination source operably coupled to the deformable transmission layer using an illumination control layer configured to emit first illumination light into the deformable transmission layer at one or more known first illumination orientations relative to the deformable transmission layer such that at least a portion of the first illumination light interacts with the deformable transmission layer; a detector configured to detect light from within at least a portion of the deformable transmission layer; and an ultrasound integrated computing system configured to characterize a geometric profile of a surface of an object docked with the interface membrane with a determined surface orientation.

Description

Citation of relevant applications

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 531,299, filed on August 7, 2023. Technical Field

[0002] This invention generally relates to systems and methods for detecting, characterizing, and / or quantifying aspects of contact or touch interfaces between a dedicated surface and other objects, and more specifically, to integration that may have one or more transmissive layers configured to aid various aspects of tactile intelligence. Background Technology

[0003] With the widespread adoption of systems such as laptops, smartphones, and video conferencing, computing, video communication, and various forms of telepresence have become key components of modern life. (Reference) Figure 1 This illustrates a user (4) interacting simultaneously with a laptop (2) and a smartphone (6) in a typical work or home environment. (Reference) Figure 2A This shows a so-called "smartwatch" (8) that is removably coupled to the user's (4) arm. Figure 2B The illustration shows a user (4) holding a smartphone (6), while one of the user's (4) hands (12) attempts to provide commands to the smartphone (6)'s computing system using gesture information. Although these illustrative systems (2, 6, 8) can be configured to handle voice-based or gesture-based commands, many operations of these devices are still performed via physical interfaces such as keyboards or touchscreens, and much of the information exchanged during video-based voice calls is in audio and / or video form. References Figures 3A to 3E Many efforts have been made to leverage modern systems to enhance the richness of interpersonal communication and / or so-called “telepresence.” Figure 3A A video conferencing configuration based on a laptop (2) is shown, in which a user (4) is able to observe and communicate with a group of other participants through a matrix video user interface (14) viewed via the laptop display (16). Figure 3B A conference room-based video conferencing system is illustrated, in which a group of local participants around a local conference table (20) can interact with remote participants via a relatively large display configured to display the remote participants' video via a teleconference user interface (18). Reference Figure 3CAnother system allows a group of local participants (34) seated around a local conference table in a local meeting room (22) to interact via video conference with a group of remote participants who are displayed via multiple integrated display / camera systems organized relative to the local conference table, to help create or simulate the feeling that all participants are in the same location, or at least able to communicate in a way that they are all local. Reference Figure 3D and 3E Video systems can be used to help bring remote users into local discussions on topics such as healthcare. Figure 3D A configuration is shown in which a user (4) from a first location is able to operate a multi-display (36, 38, 40) configuration, such as through one or more user input devices (44), to view video and information and / or data related to the scene at a second operating location, while a camera (42) captures video data of the participant (4) at the first location and provides video feedback to the second operating location to enhance communication (i.e., not just voice). Figure 3E A configuration is shown in which a group of local healthcare providers (46, 48) with a patient (50) are using a cart-based configuration (52) with a display (54) to generate video portraits (58) of remote participants, while using a video camera (56) coupled to the cart (52) to capture video of the local environment for the remote participants. Figure 4 A somewhat similar video communication system for healthcare is shown, in which a remote user (58) (e.g., a doctor) is able to navigate to a local healthcare facility room (68) containing a patient (50) and a bed (60) using an electromechanical mobile system (62), wherein a camera (64) and a display (66) are coupled to the electromechanical mobile system to allow the remote user (58) to have either a “remote presentation” or a “local presentation” within the room (68).

[0004] While each of these configurations offers a level of practicality that surpasses traditional voice calls, some argue that they still lack some key aspects of truly local presentation. As connectivity, computing, video, audio, and telecommunications technologies continue to improve, such systems will undoubtedly continue to evolve to get closer to real-time local video presentation. However, one key aspect of local presentation that such systems haven't addressed is the local "touch" experience for remote participants—this may be related to the continued high demand for air travel in certain business, social, and other scenarios. The ubiquity of touch and haptic intelligence in modern daily life is crucial, and it's no coincidence that some people (such as the visually impaired) may be remarkably adept at relying on touch and haptic intelligence to explore the world. As we've evolved to utilize the two perspectives of our eyes to form a fundamental interpretation of the shape of objects, we've also been able to leverage touch and haptic intelligence to understand key aspects of the objects we actually encounter.

[0005] To study relatively simple examples, we can examine scenarios involving remote inspection. If, in a given user scenario, detailed inspection of the surface distortion, potential stress concentration, and / or deformation of a specific object or surface is crucial, for example… Figure 5A In the scenario shown where multiple rivets (72) secure an aircraft wing surface (70) in place, one solution is to go to the location of each such aircraft wing surface and personally (74) inspect such surfaces (70), for example using an inspection light (76) configured to direct light onto the surface (70) at a selected angle to reveal surface anomalies. Similarly, see reference... Figure 6A If the exterior paint finish of the smartphone (6) housing (80) has a certain texture, or if a certain fit between the smartphone (6) camera assembly (78) and the housing (80) is “tight but not too tight”, then typically, staff will fly around the world to personally touch and inspect these components. Figure 6B Another example is shown where tactile sensation can be very valuable in determining whether the materials, fit, and finish of the crown (86), bezel (88), and / or pushers (84) of a watch (82) design are suitable for manufacture. Finally, see reference Figure 6C In this smartwatch (8), the removable strap (90) is designed to engage securely, but not too securely, with the user's hand (94, 95) and slide out of the watch (8), where tactile sensation can be highly valuable during inspection. Technologies are needed to help users gain tactile sensation to extend their normal physical range, such as to remote locations. This paper describes systems, methods, and configurations for enhancing and broadening tactile representation in various scenarios and for using such representation for various purposes, including but not limited to high-precision touch sensor implementations and configurations that can be used and configured to help provide local users with tactile perception of objects outside their normal range, such as objects in remote environments. Summary of the Invention

[0006] One embodiment relates to a system for geometric surface characterization, comprising: a deformable transport layer coupled to a mounting structure and an interface membrane, wherein the interface membrane abuts at at least one aspect of an interfaced object having a surface to be characterized; a first illumination source operatively coupled to the deformable transport layer using an illumination control layer configured to emit first illumination light into the deformable transport layer in one or more known first illumination orientations relative to the deformable transport layer, such that at least a portion of the first illumination light interacts with the deformable transport layer; a detector configured to detect light from at least a portion of the deformable transport layer; an ultrasonic emission source operatively coupled to the deformable transport layer; and an ultrasonic detection module operatively coupled to the deformable transport layer and controlled by... The system is configured to detect emission directed from the ultrasonic emission source toward the deformable transport layer; and a computing system configured to operate a detector to detect at least a portion of the light directed from the deformable transport layer to determine a surface orientation related to a position along an interface membrane, based at least in part on the interaction of a first irradiation light with the deformable transport layer, and to characterize the geometric profile of the surface of an object docked to the interface membrane using the determined surface orientation; wherein the computing system is operatively coupled to an ultrasonic detection module and is further configured to collect information relating to the interaction of the emission directed from the ultrasonic emission source with the deformable transport layer, the information being associated with the relative positioning of portions of the deformable transport layer; and wherein the deformable transport layer is configured to controllably adhere to at least one aspect of the docking object having the surface to be characterized.

[0007] The deformable transport layer can be configured to controllably expand from a contracted form to an expanded form by inflating an operably coupled capsule with fluid injection pressure. The fluid can be selected from the group consisting of air, inert gases, water, and saline solution. The capsule can be an elastomeric capsule connected between the deformable transport layer and the mounting structure. The deformable transport layer can be configured to controllably expand by inserting a mechanical expander member relative to the mounting structure. The system may also include a positioning sensor operably coupled to a computing system and the deformable transport layer. The positioning sensor can be configured for use by the computing system to determine the position of at least a portion of the deformable transport layer in a global coordinate system. The computing system and the positioning sensor can also be configured such that the orientation of at least a portion of the deformable transport layer in the global coordinate system can be determined. The first illumination source may include a light-emitting diode. The detector may be a photodetector. The detector may be an image acquisition device. The image acquisition device may be a CCD or CMOS device. The system may also include a lens operably coupled between the detector and the deformable transport layer. The computing system may be operatively coupled to a detector and configured to receive information from the detector relating to light detected by the detector from within a deformable transport layer. The computing system may be operatively coupled to a first illumination source and configured to control emission from the first illumination source. The system may also include a second illumination source operatively coupled to the illumination control layer and configured to direct second illumination having a second illumination wavelength different from the first illumination wavelength of the first illumination source into the illumination control layer. At least one of the first or second illumination wavelengths is within the infrared spectrum. The first and second illumination wavelengths may represent different colors. The system may also include a second illumination source configured to introduce second illumination light into the illumination control layer from a position or orientation different from the first illumination source. The system may also include a third illumination source configured to introduce third illumination light into the illumination control layer from a position or orientation different from the first and second illumination sources. The illumination control layer may be configured to have a shape selected from the group consisting of: planar, substantially planar, curved, convex, semi-convex, and saddle-shaped. The system may also include a second illumination source operatively coupled to a second illumination control layer and configured to direct second illumination with a second illumination wavelength different from the first illumination wavelength of the first illumination source into the deformable transport layer. The first and second illumination control layers may be stacked on top of each other. The first and second illumination control layers may be stacked adjacent to each other. The illumination control layer may be located between the detector and the deformable transport layer. The detector, illumination control layer, and deformable transport layer may be mechanically coupled within a fingertip assembly configured to include part of an elongated sensing structure. This elongated sensing structure may include a synthetic finger or robotic hand component.The detector, illumination control layer, and deformable transport layer can be operatively coupled to a lens configured to create an optical path providing a virtual camera position relative to the deformable transport layer, the virtual camera position being located outside the geometry of the fingertip assembly. The deformable transport layer may include a convex fingertip structure. The deformable transport layer may be positioned within the fingertip assembly adjacent to the illumination control layer. The deformable transport layer may also be positioned separately within the fingertip assembly from the illumination control layer. The deformable transport layer may include an elastomeric material. The elastomeric material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomers (TPE) and thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluororubber. The deformable transport layer may include a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than that of the elastomeric matrix. The pigment material may include a metal oxide. The metal oxide may be selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide. The pigment material may include metal nanoparticles. The metallic nanoparticles may be selected from the group consisting of silver nanoparticles and aluminum nanoparticles. The interface membrane may include an elastomeric material. The surface of the mating object may be located and oriented in a global coordinate system, and the computational system is configured to characterize the geometric profile of the surface of the object mating with the interface membrane by means of its position and orientation relative to the global coordinate system. The computer system may be configured to collect two or more geometric profiles of two or more portions of the surface of the object mating with the interface membrane and determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system. The computational system may be configured to provide a three-dimensional mapping of the two or more geometric profiles relative to each other in the global coordinate system. The computational system may be configured to stitch geometrically adjacent geometric profiles together using interpolation of the geometric profiles and their relative positions and orientations. The system may also include an auxiliary sensor operatively coupled to the computational system and configured to provide input that can be utilized by the computational system to further geometrically characterize the surface of the mating object. Auxiliary sensors can be selected from the group consisting of: inertial measurement units (IMUs), capacitive touch sensors, resistive touch sensors, lidar (LIDAR) devices, strain sensors, load sensors, temperature sensors, and image acquisition devices. Auxiliary sensors may include an inertial measurement unit configured to output rotational and linear acceleration data to a computing system, wherein the computing system is configured to use the rotational and linear acceleration data to help characterize the position or orientation of the deformable transport layer in a global coordinate system. Auxiliary sensors may include image acquisition devices configured to acquire image information related to the surface of the docking object, wherein the computing system is configured to use the image information to help determine the position or orientation of the object relative to the deformable transport layer.The system may also include one or more tracking tags coupled to the docking object, and one or more detectors operatively coupled to a computing system, such that the computing system can be used to identify and provide location information related to the docking object, at least in part, based on a predetermined position of the one or more tracking tags relative to the docking object. The one or more tracking tags may include radio frequency identification (RFID) tags, and the one or more detectors may include RFID detectors.

[0008] An ultrasonic emission source can be directly operably coupled to the deformable transport layer. An ultrasonic emission source can be indirectly operably coupled to the deformable transport layer. An ultrasonic detection module can be directly operably coupled to the deformable transport layer. An ultrasonic detection module can be indirectly operably coupled to the deformable transport layer. The ultrasonic emission source may include a piezoelectric power source. The deformable transport layer, when unloaded, may include a substantially planar shape. The deformable transport layer, when unloaded, may include a substantially cylindrical shape. A substantially cylindrical deformable transport layer may include the distal portion of an elongated medical device. This elongated medical device may be selected from the group consisting of catheters, endoscopes, and robotic devices.

[0009] Another embodiment relates to a system for geometric surface characterization, comprising: a deformable transport layer coupled to a mounting structure and an interface membrane, wherein the interface membrane abuts at least one aspect of a mating object having a surface to be characterized; a first illumination source operably coupled to the deformable transport layer using an illumination control layer configured to emit first illumination light into the deformable transport layer in one or more known first illumination orientations relative to the deformable transport layer, such that at least a portion of the first illumination light interacts with the deformable transport layer; a detector configured to detect light from at least a portion of the deformable transport layer; an ultrasonic emission source operably coupled to the deformable transport layer; an ultrasonic detection module operably coupled to the deformable transport layer and configured to detect emission directed from the ultrasonic emission source toward the deformable transport layer; and a computing system configured to operate the detector to... The system detects at least a portion of light guided from a deformable transport layer to determine a surface orientation related to its position along an interface membrane, based at least in part on the interaction between a first irradiation light and the deformable transport layer, and uses the determined surface orientation to characterize the geometric profile of the surface of an object docked with the interface membrane; wherein the computing system is operatively coupled to an ultrasonic detection module and is further configured to collect information related to the interaction between emission guided from an ultrasonic emission source and the deformable transport layer, the information being associated with the relative positioning of portions of the deformable transport layer; and a robotic manipulator operatively coupled to the computing system and the deformable transport layer, the robotic manipulator being configured to controllably position and orient the deformable transport layer relative to the docking object, such that the computing system can characterize the geometric profile of the surface of the docking object docked with the interface membrane with respect to the relative position and orientation of the deformable transport layer and the docking object.

[0010] The robot manipulator may include a robot arm. The robot arm may include multiple joints coupled by substantially rigid link members. The robot manipulator may include flexible robotic instruments. The system may also include an end effector coupled to the robot manipulator. The end effector may include a gripper. The deformable transport layer may be configured to controllably expand from a contracted form to an expanded form by inflating an operably coupled bladder with fluid injection pressure. The fluid may be selected from the group consisting of air, inert gases, water, and saline solution. The bladder may be an elastomeric bladder connected between the deformable transport layer and the mounting structure. The deformable transport layer may be configured to controllably expand by inserting a mechanical expander member relative to the mounting structure. The system may also include a positioning sensor operably coupled to a computing system and the deformable transport layer. The positioning sensor may be configured for use by the computing system to determine the position of at least a portion of the deformable transport layer in a global coordinate system. The computing system and the positioning sensor may also be configured such that the orientation of at least a portion of the deformable transport layer in the global coordinate system can be determined. A first illumination source may include a light-emitting diode. The detector may be a photodetector. The detector may be an image acquisition device. The image acquisition device may be a CCD or CMOS device. The system may also include a lens operatively coupled between a detector and a deformable transport layer. A computing system may be operatively coupled to the detector and configured to receive information from the detector relating to light detected by the detector from within the deformable transport layer. The computing system may be operatively coupled to a first illumination source and configured to control emission from the first illumination source. The system may also include a second illumination source operatively coupled to the illumination control layer and configured to direct second illumination having a second illumination wavelength different from the first illumination wavelength of the first illumination source into the illumination control layer. At least one of the first or second illumination wavelengths is within the infrared spectrum. The first and second illumination wavelengths may represent different colors. The system may also include a second illumination source configured to introduce second illumination light into the illumination control layer from a position or orientation different from the first illumination source. The system may also include a third illumination source configured to introduce third illumination light into the illumination control layer from a position or orientation different from the first and second illumination sources. The illumination control layer can be configured to have a shape selected from the group consisting of: planar, substantially planar, curved, convex, semi-convex, and saddle-shaped. The system may also include a second illumination source operatively coupled to the second illumination control layer and configured to direct second illumination having a second illumination wavelength different from the first illumination wavelength of the first illumination source into the deformable transmission layer. The first and second illumination control layers can be stacked on top of each other. The first and second illumination control layers can be stacked adjacent to each other. The illumination control layer can be located between the detector and the deformable transmission layer.A detector, an illumination control layer, and a deformable transport layer can be mechanically coupled within a fingertip assembly configured to include part of an elongated sensing structure. This elongated sensing structure may include a synthetic finger or robotic hand component. The detector, illumination control layer, and deformable transport layer can be operatively coupled to a lens configured to create an optical path providing a virtual camera position relative to the deformable transport layer, located outside the geometry of the fingertip assembly. The deformable transport layer may include a convex fingertip shape. The deformable transport layer may be positioned within the fingertip assembly adjacent to the illumination control layer. The deformable transport layer may also be positioned separately within the fingertip assembly from the illumination control layer. The deformable transport layer may include an elastomeric material. The elastomeric material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluororubber. The deformable transport layer may include a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than that of the elastomeric matrix. The pigment material may include a metal oxide. The metal oxide may be selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide. The pigment material may include metal nanoparticles. The metal nanoparticles may be selected from the group consisting of silver nanoparticles and aluminum nanoparticles. The interface membrane may include an elastomeric material. The interface membrane may include an elastomeric material. The surface of the mating object can be located and oriented in a global coordinate system, and wherein the computational system can be configured to characterize the geometric profile of the surface of the object mating with the interface membrane, the geometric profile having a position and orientation relative to the global coordinate system. The computer system can be configured to collect two or more geometric profiles of two or more portions of the surface of the object mating with the interface membrane, and determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system. The computational system can be configured to provide a three-dimensional mapping of the two or more geometric profiles relative to each other in the global coordinate system. The computational system can be configured to stitch together geometrically adjacent geometric contours using interpolation of their geometric profiles and their relative positions and orientations. The system may also include auxiliary sensors operatively coupled to the computational system and configured to provide input that can be utilized by the computational system to further geometrically characterize the surfaces of the docked objects. The auxiliary sensors may be selected from the group consisting of: inertial measurement units (IMUs), capacitive touch sensors, resistive touch sensors, LiDAR devices, strain sensors, load sensors, temperature sensors, and image acquisition devices. The auxiliary sensors may include an inertial measurement unit (IMU) configured to output rotational and linear acceleration data to the computational system, wherein the computational system is configured to utilize this rotational and linear acceleration data to help characterize the position or orientation of the deformable transport layer in a global coordinate system.The auxiliary sensor may include an image acquisition device configured to acquire image information relating to the surface of the docking object, wherein a computing system is configured to utilize the image information to help determine the position or orientation of the object relative to the deformable transport layer. The system may also include one or more tracking tags coupled to the docking object, and one or more detectors operatively coupled to the computing system, such that the computing system can be used to identify and provide positional information relating to the docking object, at least in part, based on predetermined positions of the one or more tracking tags relative to the docking object. The one or more tracking tags may include radio frequency identification (RFID) tags, and the one or more detectors may include RFID detectors.

[0011] An ultrasonic emission source can be directly operably coupled to the deformable transport layer. An ultrasonic emission source can be indirectly operably coupled to the deformable transport layer. An ultrasonic detection module can be directly operably coupled to the deformable transport layer. An ultrasonic detection module can be indirectly operably coupled to the deformable transport layer. The ultrasonic emission source may include a piezoelectric power source. The deformable transport layer, when unloaded, may include a substantially planar shape. The deformable transport layer, when unloaded, may include a substantially cylindrical shape. A substantially cylindrical deformable transport layer may include the distal portion of an elongated medical device. This elongated medical device may be selected from the group consisting of catheters, endoscopes, and robotic devices.

[0012] Another embodiment relates to a method for geometric surface characterization, comprising: providing a deformable transport layer coupled to a mounting structure and an interface membrane, wherein the interface membrane abuts at at least one aspect of a mating object having a surface to be characterized; providing a first illumination source operably coupled to the deformable transport layer using an illumination control layer configured to emit first illumination light into the deformable transport layer at one or more known first illumination orientations relative to the deformable transport layer, such that at least a portion of the first illumination light interacts with the deformable transport layer; providing a detector configured to detect light from at least a portion of the deformable transport layer; providing an ultrasonic emission source operably coupled to the deformable transport layer; and providing an ultrasonic detection module operably coupled to the deformable transport layer and configured to... The system is configured to detect emission directed from the ultrasonic emission source toward the deformable transport layer; provide a computing system configured to operate a detector to detect at least a portion of the light directed from the deformable transport layer, to determine a surface orientation related to a position along an interface membrane based at least in part on the interaction of a first irradiation light with the deformable transport layer, and to characterize the geometric profile of the surface of an object docked to the interface membrane using the determined surface orientation; wherein the computing system is operatively coupled to an ultrasonic detection module and is further configured to collect information relating to the interaction of the emission directed from the ultrasonic emission source with the deformable transport layer, the information being associated with the relative positioning of portions of the deformable transport layer; and wherein the deformable transport layer is configured to controllably adhere to at least one side of the docking object having the surface to be characterized.

[0013] The deformable transport layer can be configured to controllably expand from a contracted form to an expanded form by inflating an operably coupled capsule with fluid injection pressure. The fluid can be selected from the group consisting of air, inert gases, water, and saline solution. The capsule can be an elastomeric capsule connected between the deformable transport layer and the mounting structure. The deformable transport layer can be configured to controllably expand by inserting a mechanical expander member relative to the mounting structure. The method may further include providing a positioning sensor operably coupled to a computing system and the deformable transport layer. The positioning sensor can be configured for use by the computing system to determine the position of at least a portion of the deformable transport layer in a global coordinate system. The computing system and the positioning sensor can also be configured such that the orientation of at least a portion of the deformable transport layer in the global coordinate system can be determined. A first illumination source may include a light-emitting diode. The detector may be a photodetector. The detector may be an image acquisition device. The image acquisition device may be a CCD or CMOS device. The method may further include a lens operably coupled between the detector and the deformable transport layer. The computing system can be operatively coupled to a detector and configured to receive information from the detector relating to light detected by the detector from within a deformable transport layer. The computing system can be operatively coupled to a first illumination source and configured to control emission from the first illumination source. The method may further include providing a second illumination source operatively coupled to an illumination control layer and configured to direct second illumination having a second illumination wavelength different from a first illumination wavelength of the first illumination source into the illumination control layer. At least one of the first or second illumination wavelengths is within the infrared spectrum. The first or second illumination wavelength may represent different colors. The method may further include providing a second illumination source configured to introduce second illumination light into the illumination control layer from a position or orientation different from the first illumination source. The method may further include providing a third illumination source configured to introduce third illumination light into the illumination control layer from a position or orientation different from the first and second illumination sources. The illumination control layer may be configured to have a shape selected from the group consisting of: planar, substantially planar, curved, convex, semi-convex, and saddle-shaped. The method may further include providing a second illumination source operatively coupled to a second illumination control layer and configured to direct second illumination having a second illumination wavelength different from the first illumination wavelength of the first illumination source into the deformable transport layer. The first and second illumination control layers may be stacked on top of each other. The first and second illumination control layers may be stacked adjacent to each other. The illumination control layer may be located between the detector and the deformable transport layer. The detector, illumination control layer, and deformable transport layer may be mechanically coupled within a fingertip assembly configured to include part of an elongated sensing structure. This elongated sensing structure may include a synthetic finger or robotic hand component.The detector, illumination control layer, and deformable transport layer can be operatively coupled to a lens configured to create an optical path providing a virtual camera position relative to the deformable transport layer, the virtual camera position being located outside the geometry of the fingertip assembly. The deformable transport layer can include a convex fingertip shape. The deformable transport layer can be positioned within the fingertip assembly adjacent to the illumination control layer. The deformable transport layer can also be positioned separately within the fingertip assembly from the illumination control layer. The deformable transport layer can include an elastomeric material. The elastomeric material can be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomers (TPE) and thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluororubber. The deformable transport layer can include a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than that of the elastomeric matrix. The pigment material can include a metal oxide. The metal oxide can be selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide. The pigment material can also include metal nanoparticles. The metal nanoparticles may be selected from the group consisting of silver nanoparticles and aluminum nanoparticles. The interface membrane may include an elastomeric material. The surface of the docking object may be located and oriented in a global coordinate system, and wherein the computational system is configured to characterize the geometric profile of the surface of the object docking with the interface membrane, the geometric profile having a position and orientation relative to the global coordinate system. The computer system may be configured to collect two or more geometric profiles of two or more portions of the surface of the object docking with the interface membrane, and determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system. The computational system may be configured to provide a three-dimensional mapping of the two or more geometric profiles relative to each other in the global coordinate system. The computational system may be configured to stitch geometrically adjacent geometric profiles together using interpolation of the geometric profiles and their relative positions and orientations. The method may also include providing an auxiliary sensor operatively coupled to the computational system and configured to provide input that can be utilized by the computational system to further geometrically characterize the surface of the docking object. The auxiliary sensor can be selected from the group consisting of: inertial measurement units (IMUs), capacitive touch sensors, resistive touch sensors, lidar devices, strain sensors, load sensors, temperature sensors, and image acquisition devices. The auxiliary sensor may include an inertial measurement unit configured to output rotational and linear acceleration data to a computing system, wherein the computing system is configured to use the rotational and linear acceleration data to help characterize the position or orientation of the deformable transport layer in a global coordinate system. The auxiliary sensor may include an image acquisition device configured to acquire image information related to the surface of the docking object, wherein the computing system is configured to use the image information to help determine the position or orientation of the object relative to the deformable transport layer.The method may further include providing one or more tracking tags coupled to the docking object, and one or more detectors operatively coupled to a computing system, such that the computing system can be used to identify and provide location information related to the docking object, at least in part, based on a predetermined position of the one or more tracking tags relative to the docking object. The one or more tracking tags may include radio frequency identification (RFID) tags, and the one or more detectors include RFID detectors.

[0014] An ultrasonic emission source can be directly operably coupled to the deformable transport layer. An ultrasonic emission source can be indirectly operably coupled to the deformable transport layer. An ultrasonic detection module can be directly operably coupled to the deformable transport layer. An ultrasonic detection module can be indirectly operably coupled to the deformable transport layer. The ultrasonic emission source may include a piezoelectric power source. The deformable transport layer, when unloaded, may include a substantially planar shape. The deformable transport layer, when unloaded, may include a substantially cylindrical shape. A substantially cylindrical deformable transport layer may include the distal portion of an elongated medical device. This elongated medical device may be selected from the group consisting of catheters, endoscopes, and robotic devices.

[0015] Another embodiment relates to a method for characterizing a geometric surface, comprising: providing a deformable transport layer coupled to a mounting structure and an interface membrane, wherein the interface membrane abuts at least one aspect of a mating object having a surface to be characterized; providing a first illumination source operably coupled to the deformable transport layer using an illumination control layer configured to emit first illumination light into the deformable transport layer in one or more known first illumination orientations relative to the deformable transport layer, such that at least a portion of the first illumination light interacts with the deformable transport layer; providing a detector configured to detect light from at least a portion of the deformable transport layer; providing an ultrasonic emission source operably coupled to the deformable transport layer; providing an ultrasonic detection module operably coupled to the deformable transport layer and configured to detect emission directed from the ultrasonic emission source toward the deformable transport layer; and providing a computing system configured to operate... The system comprises a detector for detecting at least a portion of light guided from a deformable transport layer, determining a surface orientation related to the position along the interface membrane based at least in part on the interaction of a first irradiation light with the deformable transport layer, and using the determined surface orientation to characterize the geometric profile of the surface of the object docked with the interface membrane; wherein the computing system is operatively coupled to an ultrasonic detection module and is further configured to collect information related to the interaction of emission guided from an ultrasonic emission source with the deformable transport layer, the information being associated with the relative positioning of portions of the deformable transport layer; and a robotic manipulator operatively coupled to the computing system and the deformable transport layer, the robotic manipulator being configured to controllably position and orient the deformable transport layer relative to the docking object, such that the computing system can characterize the geometric profile of the surface of the docking object docked with the interface membrane with respect to the relative position and orientation of the deformable transport layer and the docking object.

[0016] The robot manipulator may include a robotic arm. The robotic arm may include multiple joints coupled by substantially rigid link members. The robot manipulator may include flexible robotic instruments. The method may also include providing an end effector coupled to the robot manipulator, the end effector including a gripper. The deformable transport layer may be configured to controllably expand from a contracted form to an expanded form by inflating an operably coupled bladder with fluid injection pressure. The fluid may be selected from the group consisting of air, inert gases, water, and saline solution. The bladder may be an elastomeric bladder connected between the deformable transport layer and a mounting structure. The deformable transport layer may be configured to controllably expand by inserting a mechanical expander member relative to the mounting structure. The method may also include providing a positioning sensor operably coupled to a computing system and the deformable transport layer. The positioning sensor may be configured for use by the computing system to determine the position of at least a portion of the deformable transport layer in a global coordinate system. The computing system and the positioning sensor may also be configured such that the orientation of at least a portion of the deformable transport layer in the global coordinate system can be determined. A first illumination source may include a light-emitting diode. The detector may be a photodetector. The detector may be an image acquisition device. The image acquisition device can be a CCD or CMOS device. The method may further include providing a lens operatively coupled between a detector and a deformable transport layer. A computing system may be operatively coupled to the detector and configured to receive information from the detector relating to light detected by the detector from within the deformable transport layer. The computing system may be operatively coupled to a first illumination source and configured to control emission from the first illumination source. The method may further include providing a second illumination source operatively coupled to an illumination control layer and configured to direct second illumination having a second illumination wavelength different from the first illumination wavelength of the first illumination source into the illumination control layer. At least one of the first or second illumination wavelengths may be within the infrared spectrum. The first and second illumination wavelengths may represent different colors. The method may further include providing a second illumination source configured to introduce second illumination light into the illumination control layer from a position or orientation different from the first illumination source. The method may further include providing a third illumination source configured to introduce third illumination light into the illumination control layer from a position or orientation different from the first and second illumination sources. The illumination control layer can be configured to have a shape selected from the group consisting of: planar, substantially planar, curved, convex, semi-convex, and saddle-shaped. The method may further include providing a second illumination source operatively coupled to the second illumination control layer and configured to direct second illumination having a second illumination wavelength different from the first illumination wavelength of the first illumination source into the deformable transmission layer. The first and second illumination control layers can be stacked on top of each other. The first and second illumination control layers can be stacked adjacent to each other. The illumination control layer can be located between the detector and the deformable transmission layer.A detector, an illumination control layer, and a deformable transport layer may be mechanically coupled within a fingertip assembly configured to include part of an elongated sensing structure. The elongated sensing structure may include a synthetic finger or robotic hand component. The detector, illumination control layer, and deformable transport layer may be operatively coupled to a lens configured to create an optical path providing a virtual camera position relative to the deformable transport layer, located outside the geometry of the fingertip assembly. The deformable transport layer may include a convex fingertip shape. The deformable transport layer may be positioned within the fingertip assembly adjacent to the illumination control layer. The deformable transport layer may also be positioned separately within the fingertip assembly from the illumination control layer. The deformable transport layer may include an elastomeric material. The elastomeric material may be selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomers (TPE) and thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluororubber. The deformable transport layer may include a composite material having a pigment material distributed within an elastomeric matrix, the pigment material being configured to provide an illumination reflectivity greater than that of the elastomeric matrix. The pigment material may include metal oxides. The metal oxides may be selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide. The pigment material may include metal nanoparticles. The metal nanoparticles may be selected from the group consisting of silver nanoparticles and aluminum nanoparticles. The interface membrane may include an elastomeric material. The surface of the mating object can be located and oriented in a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object mating with the interface membrane by its position and orientation relative to the global coordinate system. The computer system may be configured to collect two or more geometric profiles of two or more portions of the surface of the object mating with the interface membrane and determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system. The computing system may be configured to provide a three-dimensional mapping of the two or more geometric profiles relative to each other in the global coordinate system. The computational system can be configured to stitch together geometrically adjacent geometric contours using interpolation of their geometric profiles and their relative positions and orientations. The method may also include providing an auxiliary sensor operatively coupled to the computational system and configured to provide input that can be utilized by the computational system to further geometrically characterize the surfaces of the docking objects. The auxiliary sensor may be selected from the group consisting of: inertial measurement units (IMUs), capacitive touch sensors, resistive touch sensors, lidar devices, strain sensors, load sensors, temperature sensors, and image acquisition devices. The auxiliary sensor may include an inertial measurement unit configured to output rotational and linear acceleration data to the computational system, wherein the computational system is configured to utilize the rotational and linear acceleration data to help characterize the position or orientation of the deformable transport layer in a global coordinate system.The auxiliary sensor may include an image acquisition device configured to acquire image information relating to the surface of the docking object, wherein a computing system is configured to utilize the image information to help determine the position or orientation of the object relative to the deformable transport layer. The system may also include one or more tracking tags coupled to the docking object, and one or more detectors operatively coupled to the computing system, such that the computing system can be used to identify and provide positional information relating to the docking object, at least in part, based on a predetermined position of the one or more tracking tags relative to the docking object. The one or more tracking tags may include radio frequency identification (RFID) tags, and the one or more detectors may include RFID detectors.

[0017] An ultrasonic emission source can be directly operably coupled to the deformable transport layer. An ultrasonic emission source can be indirectly operably coupled to the deformable transport layer. An ultrasonic detection module can be directly operably coupled to the deformable transport layer. An ultrasonic detection module can be indirectly operably coupled to the deformable transport layer. The ultrasonic emission source may include a piezoelectric power source. The deformable transport layer, when unloaded, may include a substantially planar shape. The deformable transport layer, when unloaded, may include a substantially cylindrical shape. A substantially cylindrical deformable transport layer may include the distal portion of an elongated medical device. This elongated medical device may be selected from the group consisting of catheters, endoscopes, and robotic devices. Attached Figure Description

[0018] Figure 1-4 Figures 2A-2B, 3A-3E, and 4 illustrate various aspects of conventional computing and communication systems.

[0019] Figures 5A-5B Figures 6A-6C illustrate various aspects of the scene, in which enhancing the understanding of surface geometry or contours would be useful.

[0020] Figures 7A-7H and Figure 8 Various aspects of a touch sensing component configured to utilize a deformable transport layer are shown.

[0021] Figure 9A and 9B The diagram shows a component with multiple touch sensing components, for example... Figures 7A-7H The touch sensing component shown.

[0022] Figure 10A-10I Various aspects of an embodiment of a touch sensing component are illustrated, which may have one or more auxiliary sensor configurations integrated therein.

[0023] Figure 11-12Figures 13A-13F, 14, and 15 illustrate various aspects of touch sensing component integration, in which electromechanical systems such as robots can be used to obtain further tactile intelligence about a target object or surface.

[0024] Figures 16A-16B Figures 1 and 17 illustrate various aspects of the configuration in which one or more touch-sensing components can be used to at least partially characterize a portion of an appendage, such as a part of a user's foot or arm.

[0025] Figure 18A-18L The diagram illustrates various aspects of a configuration for integrating one or more touch-sensing components into a complex system, which may involve controlled electromechanical motion, such as by a robot, and the placement of deformable transport layers at various locations along the length of the various components, as well as around the outer surface shape profile of the various components, such as the periphery relative to an elongated instrument.

[0026] Figures 19A-19B Figures 20A-20C, 21A-21D, 22, 23A-23B, 24A-24B, 25A-25B, 26-27, 28A-28B, 29A-29D, 30A-30G, 31A-31E, 32A-32B, 33A-33B, 34, and 35 illustrate aspects of system and method integration in which one or more touch sensing components can be used to help transfer physical engagement back to the user at a workstation, which may be local or remote relative to the physical engagement.

[0027] Figure 36 , 39 Figures 40, 42, and 46-47 illustrate aspects of the integration of medical systems and methods in which one or more touch-sensing components can be used to help translate the physical engagement at the site of tissue intervention back to the user at a workstation, which may be local or remote relative to the physical engagement of the tissue.

[0028] Figure 37 and 41 This illustrates aspects of integrating gaming or virtual bonding systems and methods, where one or more analog touch sensing components can be utilized to help transform physical bonding at a user interface workstation.

[0029] Figures 38A-38F Figures 43-45 illustrate aspects of integration, in which one or more sensing components can be used to help characterize one or more key working components of a component or machine.

[0030] Figures 48-50 The integration demonstrates various aspects, including the ability to utilize one or more sensing and / or touch-conversion interfaces to assist local users in perceiving the experience and facilitating user-issued commands.

[0031] Figure 51A-51I Various geometric configurations for tactile sensing are shown, for example, which can be used to process various geometries of target surfaces.

[0032] Figures 52-58 and Figures 59A-59B Various aspects of tactile sensing system configurations are illustrated, which have one or more computing devices or computing systems operatively coupled to one or more deformable transport layers that can be used to provide geometric information about a target structure, such as a riveted surface structure, an engine block, or other structures and / or surfaces.

[0033] Figure 60A-60E Figures 61A-61F illustrate various aspects of tactile sensing system configurations that can be removably coupled from certain physical support structures to form handheld configurations that can be used to provide geometric information about a target structure.

[0034] Figures 62-65 Various process or method configurations are shown, which have deformable transport layers for geometrically representing one or more objects.

[0035] Figures 66A-66E Various geometries and surface configurations related to objects and / or structures can be studied using deformable transport layers.

[0036] Figures 67A-67B Figures 68A-68D and 69A-69F illustrate various aspects of a configuration with expandable and deformable transport layer components.

[0037] Figures 70A-70E 71A-71D illustrates various aspects of configurations for measurement and / or characterization using deformable transport layer components in instruments (e.g., elongated instruments or their distal portions).

[0038] Figures 72A-72C The system configurations that can be measured and / or characterized using deformable transport layer elements are shown.

[0039] Figures 73A-73C The expansion or extension aspects of the expandable and deformable transport layer configuration are shown.

[0040] Figures 74A-74C The procedure illustrates various aspects of the process, in which an expandable and deformable transport layer can be used to characterize and / or measure aspects of elongated defects, holes, or lumens.

[0041] Figure 75-82 Various process or method configurations are shown, which have deformable transport layers for geometric representation of one or more objects.

[0042] Figure 83A and 83B Aspects of a system with a deformable transport layer are shown, which can be controllably moved, for example, by a robotic arm, to address a target object or surface.

[0043] Figure 84A and 84B Aspects of a system with a deformable transport layer are shown, which can be controllably moved, for example, by an electromechanical gantry assembly, to address a target object or surface.

[0044] Figures 85-94 This illustrates aspects of a process or method configuration with a deformable transport layer for geometrically representing one or more objects.

[0045] Figure 95A-95C Aspects of an example scenario are shown, demonstrating how a configuration with one or more deformable transport layers can be used to characterize complex objects.

[0046] Figure 96 This illustrates aspects of a process or method configuration with a deformable transport layer for geometrically representing one or more objects.

[0047] Figure 97 The diagram illustrates various aspects of a system configuration with a deformable transport layer that can be controllably moved, for example by a robotic arm, to address a target object or surface, and may include one or more measurement probes coupled thereto to assist in sensing operational characteristics, such as discrete point contacts.

[0048] Figure 97 A-97C illustrates various aspects of system-level configurations related to characterizing target objects or surfaces using deformable transport layers.

[0049] Figures 99A-99C Various aspects of the irradiation configuration related to characterizing a target object or surface using a deformable transport layer are shown.

[0050] Figure 100A-100E Various aspects of an integrated configuration with one or more lighting control layers for illumination are shown.

[0051] Figure 101A-101B An orthographic view of an illumination control layer configuration with operably coupled illumination sources is shown.

[0052] Figure 102A-102B An orthogonal view of an illumination control layer configuration with two operatively coupled illumination sources is shown.

[0053] Figure 102C A view of an illumination control layer configuration with four operatively coupled illumination sources is shown.

[0054] Figure 102D A view of an illumination control layer configuration with three operatively coupled illumination sources is shown.

[0055] Figure 102E A view is shown of a lighting control layer configuration with two illumination sources operatively coupled to a lighting control layer having a square or rectangular shape in the depicted diagram.

[0056] Figure 102F A view is shown of a lighting control layer configuration with six illumination sources operatively coupled to a lighting control layer having a square or rectangular shape in the depicted diagram.

[0057] Figure 103A-103C The shape of the lighting control layer is shown, which may be, for example, planar, substantially planar, arc-shaped or curved, saddle-shaped, or configured to have at least one convex surface while also having at least one planar or substantially planar surface.

[0058] Figure 104A-104G Various aspects of a sensing configuration integrated into an elongated component are shown, which is intended for applications such as fingertip / finger pad sensing, synthetic fingers and / or robotic / synthetic hands or graspers.

[0059] Figure 105A and 105B Sample results from a sensing configuration using a deformable transport layer are shown.

[0060] Figure 106A and Figure 106B A system-level view of the deformable transport layer sensing configuration is shown.

[0061] Figures 106C to 106J This illustrates various aspects of a sensing configuration that utilizes a deformable transmission layer and also integrates ultrasonic emission, detection, and sensing components.

[0062] Figures 107 to 110 Various aspects of the process or method configuration are illustrated, wherein in a fusion sensing configuration, deformable transport layer sensing can be integrated with and used in conjunction with ultrasonic sensing.

[0063] Figure 111A It shows various aspects of the patient's blood vessels.

[0064] Figure 111B Aspects of a deformable transport layer sensing configuration are shown, which can be used in conjunction with integrated ultrasonic sensing in a fusion sensing configuration.

[0065] Figures 111C to 111E as well as Figure 112Various aspects of the process or method configuration are illustrated, wherein in a fusion sensing configuration, deformable transport layer sensing can be integrated with and used in conjunction with ultrasonic sensing. Detailed Implementation

[0066] refer to Figure 7A A digital touch sensing component (146) is shown, having a deformable transport layer (110) operatively coupled to an optical element (108) illuminated by one or more mutually coupled light sources (116, 122) and located within the field of view of an imaging device (106). A housing (118) is configured to maintain the positioning of components relative to each other and expose a touch-sensing contact surface (120). An interface membrane (100) may comprise a substantially thin layer that is fixedly or removably coupled, comprising, for example, a polymeric material with a relatively low bulk modulus. The interface membrane (100) may be positioned and operatively coupled to the deformable transport layer, or may comprise a portion of the deformable transport layer, for direct contact between other objects and the digital touch sensing component (146) for touch determination and characterization; thus, in the configuration where the interface membrane (100) is coupled to or comprises a portion of the deformable transport layer, the final external touch contact surface (120) becomes the external aspect of such an interface membrane (100). For example, aspects of a suitable digital touch sensing component (146) configuration, typically having an elastically deformable transport layer material, are described in U.S. Patent Nos. 10,965,854, 10,574,944, 9,127,938, and 8,411,140, ​​and U.S. Patent Application Publications Nos. 20140253717 and 20140104395. Figure 7AAs shown, the digital touch sensing component (146) may have a gap or void (114) that may contain an optical transmission material (e.g., a material having a refractive index similar to that of the optical element 108), air, or a special gas (e.g., an inert gas), which is geometrically configured to place aspects of the optical element (108) and / or the deformable transmission layer (110) within a desired proximity range of an imaging device (106), which may include an imaging sensor (e.g., a digital camera chip), a single photosensing element (e.g., a photodiode), or an array of photosensing elements, and may be configured to have a field of view and depth of field facilitated by the geometric gap or void (114) (i.e., the gap or void 114 may be positioned to accommodate the field of view and / or depth of field associated with a particular imaging device 106). In various embodiments, the optical element (108) may include a material that is substantially rigid, a material with a known elastic modulus, or a material with a known structural modulus (i.e., given an unloaded shape and a loaded shape, the loading profile can be determined based on structural modulus information associated with the shape). Various suitable optical elements (108) can be defined in external shapes, including, for example, cylindrical, cubic, and / or rectangular prism shapes. As shown and described below, various illumination sources can be coupled to one or more sidewall surfaces defining the optical element (108). In another embodiment, the optical element (108) can be configured to be deformable or compliant, such that the rigidity of this structure minimizes the effect on other associated elements (i.e., pulsed loads, such as force / time variations (delta-time), can be minimized by greater impact compliance; furthermore, greater surface contact can be maintained on a given surface, such as a surface with topographic or geometric features, by means of a lower structural modulus at the contact interface).

[0067] Figure 7A A computing device or system (104) is also shown, which may include a computer, microcontroller, field-programmable gate array, application-specific integrated circuit, etc., configured to be operatively coupled (128) to an imaging device (106) and operatively coupled (124, 126) to one or more light sources (30) to facilitate control of these devices to collect data related to contact with the deformable transport layer (110). For example, in one embodiment, such as Figure 7AAs shown, each of the light sources (116, 122) includes a light-emitting diode (“LED”) operably coupled (124, 126) to the computing device (104) using electronic leads (124, 126), and the imaging device (106) includes a digital camera sensor chip operably coupled to the computing device using electronic leads (128). A power supply (102) can be operably coupled to the computing device (104) to provide power to the computing device (104) and can also be configured to controllably provide power to interconnected devices (e.g., the imaging device (106) and the light sources (116, 122)) via its couplings (128, 124, 126, respectively). Figure 7A As shown, separations (640) are depicted to indicate that these coupling interfaces (128, 124, 126) can be short or relatively long (i.e., the digital touch sensing component 146 may be located at a remote location relative to the computing device 104) and can be directly physically connected or transmit data via wired or wireless interfaces, such as via optical / optical network protocols, or wireless network protocols, such as Bluetooth (RTM) or 802.11-based configurations, which can be implemented by additional computing and power resources located locally on the digital touch sensing component (146).

[0068] refer to Figure 7B It shows the relationship with Figure 7A The configuration shown is similar to the configuration shown, except that, Figure 7B The deformable transport layer (110) comprises one or more capsules or enclosed volumes (112) that may be occupied by, for example, a fluid (e.g., a liquid or gas, which may be physically considered a form of fluid). In one embodiment, for example, the deformable transport layer (110) may comprise several individually controllable expandable segments or sub-volumes and may include a cross-sectional shape selected to provide specific mechanical properties under load, such as a controllable honeycomb cross-sectional shape configuration. As described above, the deformable transport layer (110) may include one or more materials selected to match the touch sensing paradigm in terms of bulk and / or Young's modulus. In other words, for sensing relatively low loads, such as in digital contact scenarios involving interaction with the surface of a soap bubble or a living photosynthetic leaf of a plant, as described in the foregoing references, a relatively low modulus (i.e., typically locally flexible / deformable; not rigid) material (e.g., an elastomer) may be used for the deformable transport layer (110) and / or the external interface membrane (100), which, as described above, is removable. The external interface membrane (100) may include a set of relatively thin and sequentially removable membranes, such that when they are coupled with dirt or dust, they can be removed sequentially, for example, in a "tear-off" manner. For example, Figure 7BIn the illustrated embodiment, the deformable transport layer (110) includes at least a temporarily trapped volume of liquid or gas, which can be adjusted along with its pressure to achieve the required bulk modulus and sensitivity of the entire deformable transport layer (110) (e.g., the pressure and / or volume associated with one or more sac segments 112 can be adjusted to generally change the functional modulus of the deformable transport layer 110).

[0069] refer to Figure 7C It shows the relationship with Figure 7A The configuration is similar to the configuration, where Figure 7C The configuration shows that the gap (130) between the imaging device (106) and the optical element (108) can be reduced or even eliminated, depending on the optical layout of the imaging device (106), which can be coupled to each other with refractive and / or diffractive optical elements to change the characteristics of the imaging device (106) such as focal length.

[0070] refer to Figure 7D It shows the relationship with Figure 7A The configuration is similar, the difference being... Figure 7D The configuration shows that one or more light sources may be more like light emitters (117, 123), which are configured to emit light originating from another location, for example, coupled to another location of one or more light-emitting LED sources that are directly coupled to the computing device (104) and are configured to transmit light through optical transmission coupling members (132, 134) via optical fibers, “light guides” or waveguides, which can be configured to transmit photons from these sources to the emitter (117, 123) as efficiently as possible, for example via total internal reflection.

[0071] Similarly, refer to Figure 7E It shows the relationship with Figure 7D The configuration is similar, wherein the imaging device (107) includes a collection optics selected for collecting photons and transmitting them back to an image sensor via an optical transmission coupling member (138) (e.g., a waveguide or one or more optical fibers), which may be located within or coupled to the computing device (104) or other structure separate from the digital touch sensing component (146).

[0072] refer to Figure 7F-7H This illustrates various aspects of a configuration of a digital touch sensing component (146) having a deformable transport layer (110) that can be used to characterize interactions between surfaces. For example, see reference... Figure 7FIn a simplified illustrative embodiment, a computing system or device (104) operably coupled (136) to a power source (102) can be used to control light (1002) or other emissions from an illumination source (116) via a control coupling (124), which may be wired or wireless, and the light or other emissions may be directed into a deformable transport layer (110). The deformable transport layer (110) may be urged (1006) to at least a portion of a mating object (1004), such as the edge of a coin, and based on the interaction between the illumination (1002) and the deformable transport layer (110), a detector (e.g., an image acquisition device (e.g., a CCD or CMOS device)) operably coupled (128, e.g., via a wired or wireless connection) to the computing system (104) may be configured to detect at least a portion of the light directed from the deformable transport layer. In other words, with the illumination source (116) operatively coupled (e.g., optically coupled to an effective transmission interface) to deliver illumination at a known orientation relative to the deformable transmission layer, such that at least a portion of the illumination light interacts with the deformable transmission layer, and the detector configured to detect light from at least a portion of the deformable transmission layer, the computational system can be configured to operate the detector to detect at least a portion of the light guided from the deformable transmission layer to determine, at least in part, based on the interaction of the first illumination light with the deformable transmission layer, a surface orientation relating to the position of the interface between the deformable transmission layer and the docking object, and to utilize the determined surface orientation to characterize the geometric profile of at least one aspect of the docking object confronting the interface membrane. (Reference) Figure 7G As discussed further below, an interface membrane (100) can be inserted between the interface object (1004) and the deformable transport layer (110); the modulus of this interface membrane can be similar to or different from that of the deformable transport layer. Preferably, an effective coupling is established between the deformable transport layer and the membrane, enabling efficient transfer of shear and principal or normal loads between these structures. (Refer to back) Figure 7AAn embodiment is shown, including an optical element (108) configured to help precisely distribute light or other radiation in various parts of an assembled system. The optical element may comprise a highly transmissive, substantially rigid material; it may include a top surface, a bottom surface, and sides defined therebetween to form a three-dimensional shape, such as a cylinder, cuboid, and / or rectangular prism. The illustrated optical element (108) may be illuminated by one or more mutually coupled light sources (116, 122) and located within the field of view of an imaging device (106). The housing (118) is configured to hold the components in position relative to each other, and as described above, the interface membrane (100) may comprise a substantially thin layer of fixed attachment or removable coupling, comprising, for example, a polymeric material with a relatively low bulk modulus, and the interface membrane (100) may be positioned for direct contact between other objects and digital touch sensing components (146) for touch determination and characterization. Preferably, the deformable transport layer and / or interface membrane comprises an elastic material, such as silicone, urethane, polyurethane, thermoplastic polyurethane (TPU) or thermoplastic elastomer (TPE), plastisol, polyvinyl chloride, polyisoprene, or fluororubber. Other elastomers with lower light and / or radiation transport efficiencies may also be used, such as natural rubber, neoprene, ethylene propylenediene monomer (EPDM) rubber, butyl rubber, nitrile rubber, styrene-butadiene rubber (SBR), fluororubber (Viton), fluorosilicone, and polyacrylate. The deformable transport layer may comprise a composite material containing pigment materials, such as metal oxides (e.g., iron oxide, zinc oxide, aluminum oxide, and / or titanium dioxide), metallic pigments or metallic nanoparticles (e.g., silver nanoparticles and / or aluminum nanoparticles), or other molecules configured to interact differentially with introduced light or radiation, such as dyes distributed within the elastic matrix. The pigment material may be configured to provide an illumination reflectivity greater than that of the elastomer matrix. The deformable transport layer is defined by a bottom surface directly coupled to the interface film, a top surface closest to the detector, and a transport layer thickness between the two, wherein pigment material is distributed near the bottom surface within the transport layer thickness to provide optimized illumination reflectivity near the bottom surface. For example, aspects of a suitable digital touch sensing component (146) configuration typically having an elastically deformable transport layer material are described in U.S. Patent Nos. 10,965,854, 9,127,938, and 8,411,140. Figure 3CAs shown, the depicted digital touch sensing component (146) may have a gap or void (114) that may contain an optical transmission material (e.g., a material having a refractive index similar to that of the optical element 108), air, or a special gas (e.g., an inert gas), which is geometrically configured to place aspects of the optical element (108) and / or the deformable transmission layer (110) within a desired proximity range of an imaging device (106), which may include an imaging sensor (e.g., a digital camera chip), a single light-sensing element (e.g., a photodiode), or an array of light-sensing elements, and may be configured to have a field of view and depth of field facilitated by the geometric gap or void (114). In another embodiment, the optical element (108) may be configured to be deformable or compliant, such that the rigidity of this structure minimizes the impact on other associated components.

[0073] Figure 7A A computing device or system (104) is also shown, which may include a computer, microcontroller, field-programmable gate array, application-specific integrated circuit, etc., configured to be operatively coupled to an imaging device (106) and also coupled to one or more light sources (116, 122) to control these devices to collect data related to contact with the deformable transport layer (110). For example, in one embodiment, such as Figure 7A As shown, each of the light sources (116, 122) includes a light-emitting diode (“LED”) operably coupled (124, 126) to the computing device (104) using electronic leads, and the imaging device (106) includes a digital camera sensor chip operably coupled to the computing device using electronic leads (128). A power supply (102) can be operably coupled to the computing device (104) to provide power to the computing device (104) and can also be configured to controllably provide power to interconnected devices (e.g., the imaging device (106) and the light sources (116, 122)) via its couplings (128, 124, 126, respectively). Figure 7A As shown in (640), these coupling interfaces (124, 126, 128) can be short or relatively long (i.e., the digital touch sensing component 146 can be located at a remote position relative to the computing device 104), and can be directly physically connected or transmit data via wired or wireless interfaces, such as via optical / optical network protocols, or wireless network protocols, such as Bluetooth (RTM) or 802.11-based configurations, which can be implemented using additional computing and power resources located locally on the digital touch sensing component (146). Reference Figure 7HThe partial schematic diagram shows that the computing system (104) can be operatively coupled (124, 126, 1012) to two, three or more different illumination sources (116, 122, 1010) via wired or wireless control leads, for example; these illumination sources can be configured to have different emission wavelengths and / or different polarizations, and, as shown, can be configured to emit from different orientations relative to the optical element (108) and the associated deformable transmission layer (110) to allow for the acquisition of further data related to the geometric profile.

[0074] refer to Figure 8 As described in the aforementioned reference (US Patent No. 10,965,854), the deformable transmission layer or component (110) can include various geometries, and is not necessarily planar or shaped as a rectangular prism or a variation thereof; for example, the deformable transmission layer or component (110) can be curved, convex (144), saddle-shaped, etc., and can be customized for various specific contact sensing scenarios. For example, it can be... Figure 8 Multiple components (146) with a convex deformable transmission layer (110) are coupled to the gripping interface of the robotic gripper / hand to enable touch sensing / identification of the grasped object in a manner similar to the skin segment between the joints of a human hand grasping the object. Figure 8 The component (146) is configured with a housing geometry (142) and coupling features (140) to facilitate removable attachment to other components.

[0075] refer to Figure 9A Multiple digital touch sensing components (146) can be used together to sense the larger surface (150) of an object (148). Each such component (146, Figure 9A Five of them are shown in the figure and can be operatively coupled to one or more computing devices (104) via, for example, electronic leads (and can be interrupted via wireless connection, for example, as described above), as shown in the figures (152, 154, 156, 158, 160), and can therefore be configured to exchange data and facilitate the transmission of power, light, and control and sensing information.

[0076] refer to Figure 9B In contrast Figure 9A More (162) digital touch sensing components (146) can be used to partially or completely surround an object, or to monitor digital touches on two or more surfaces of the object. Figure 9B Each of the thirty digital touch sensing components (146) shown can be operatively coupled to the same or different computing devices (104), and the coupling leads can be combined or coupled to form, as shown in the figure. Figure 9B The single-combined coupling lead assembly (164) is shown.

[0077] refer to Figure 10A While optional geometric separation (640) between various components (e.g., digital touch sensing components (146) and computing devices (104)) is shown, it is important to note that these components can also be housed together and connected to other systems, components, and devices via wireless transceivers (166), such as those designed to operate via the so-called “WiFi” standard of IEEE 802.11, and / or wireless connectivity and communication standards known using the “Bluetooth” trademark (e.g., Bluetooth 4.x and Bluetooth 5). Furthermore, the mutually coupled (136, e.g., via direct leads) power supply (102) components shown may include one or more batteries, or one or more connections to other power sources (wired or wireless, e.g., via inductive power transmission) to provide further power supply and / or charging to the integrated power supply (102) components. The various embodiments described herein relate to miniaturized or miniaturizable configurations to facilitate integration into other systems (e.g., automotive systems) and are intended to facilitate integration of such systems with connectivity alternatives that meet or are harmonized with known standards. For example, in various embodiments, touch sensing systems (e.g., Figure 10A The touch sensing system shown can be miniaturized and packaged in a housing, and has a connectivity configuration designed for relatively simple integration into other systems or integration with other systems. This system configuration can be seen as a move toward “Internet of Things” integration capabilities, in which various devices are expected to collaborate relatively easily with other connected and integrated systems.

[0078] Refer again Figure 10A A digital touch sensing component (146) is shown, which is similar to the reference numeral. Figure 7AThe described component, but also possesses a series of additional sensing capabilities or “auxiliary sensor” elements selected to enhance the general capabilities of the component, for example by providing sensing data from one or more additional sensing subsystems, which typically coexist with the touch sensing capabilities provided by the deformable transport layer and may have their own sensing uncertainties and error levels, allowing the use of so-called “sensor fusion” techniques to improve the overall capabilities of the integrated configuration, for example by utilizing uncorrelated errors between the individual sensing subsystems. For example, when a digital touch sensor based on the deformable transport layer (110) may indicate contact with another object, data from an integrated inertial measurement unit (or “IMU,” such as accelerometer or gyroscope data that may include one or more accelerometers or gyroscopes), a LIDAR subsystem (e.g., point cloud data relating to the claimed contact area), and an imaging device (e.g., a camera providing image data belonging to the claimed contact area) provide additional contradictory data with uncorrelated measurement / determination errors to determine that the digital touch sensor is not in contact, suggesting a high probability of no contact (the concept of at least partially uncorrelated errors for other measurement / determination subsystems). This is important because while all other measurement / determination subsystems may contribute some degree of redundancy or enhanced measurement, field of view, etc., if all other measurement / determination subsystems have the same correlated error, they may also have similar error-based limitations. For example, mounting three pitot tubes on an aircraft wing can provide some redundancy and further measurement relative to a single pitot tube, but if they all fly in freezing rain and fail due to the same correlated error, the aircraft may be better off relying on subsystems with some uncorrelated errors, such as a compass, GPS, trajectory planning, etc. Therefore, the concept of utilizing multiple sensors with at least some uncorrelated errors offers value and can be achieved through the use of two or more sensors (a process known as "sensor fusion"). Furthermore, as mentioned above, multiple sensors can be aggregated to complement and extend the geometry of the sensing paradigm, for example, by coupling similar or different sensors adjacent to each other along a given surface or aspect of a structural element. Therefore, return to the reference. Figure 10A A series of additional sensing subsystems (IMU 172, capacitive touch sensor 174, resistive touch sensor 176, LiDAR sensor 178, strain or elongation sensor 180, load sensor 182, temperature sensor 184, additional image sensor 186) with at least some uncorrelated errors are shown, which are operatively coupled as part of the integrated system configuration shown (188, 190, 192, 194, 196, 198, 200, 202 represent connection leads, such as wire leads, which can be as follows: Figure 10A As shown, it is connected to a communication / connection bus 170, which can be directly coupled to computing device 104 (168).

[0079] For the purpose of explanation, Figure 10B-10I Various embodiments are described, in which further details of the integration of various subsystems can be explored.

[0080] refer to Figure 10B An embodiment is shown in which a digital touch sensing component (146) is integrated with an intercoupled IMU (172). The IMU (172) may include one or more accelerometers and one or more gyroscopes and may be fixedly coupled to the housing (118) of the digital touch sensing component (146) and operatively coupled to a computing device (104), for example via a lead (188; shown coupled to a communication bus 170, which is operatively coupled to the computing device 104, for example via a lead 168). The computing device (104) may be configured not only to operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transport layer (110) (when it is physically mated to one or more objects, for example at a contact interface (120)), but also to operate the IMU (172) to acquire data related to angular and axial accelerations, for example, these data may be related to changes in the position or orientation of contact with external objects and / or the housing (118). In one embodiment, for example, the integrated system may be configured to increase the frame rate of touch sensing via the deformable transmission layer (110) when an unwanted change in axial or angular acceleration is detected using IMU data and knowledge of predicted motion and acceleration from the housing (118). In other words, if the digital touch sensing component (146) is coupled to an electromechanical motion system, such as a robotic arm or robotic manipulator (e.g., Figure 11 (234 in the middle), and the integrated computing system (104) is integrated to receive timing, direction / orientation and kinematic information about motion commands of the electromechanical motion system, which can be configured to separate desired acceleration from undesired acceleration from the IMU and treat undesired acceleration as potential contact with external objects, which can be further explored by the deformable transport layer (110) with enhanced frame rate, computing and general digital touch sensing.

[0081] refer to Figure 10CAn embodiment is shown in which a digital touch sensing component (146) is integrated with an intercoupled capacitive sensing subsystem having a capacitive sensing controller (174) operatively coupled to a capacitive sensing element (206) via, for example, a lead (204). This capacitive sensing element may be integrated into a deformable transport layer and configured to facilitate enhanced contact sensing based on capacitance sensed between the sensing element (206), which may include a grid or multiple cells, and other objects. This is somewhat similar to how some smartphones or other touchscreen interfaces are configured to detect contact based on detected capacitance. The capacitive sensing controller (174) may include one or more amplifiers and may be fixedly coupled to a housing (118) of the digital touch sensing component (146) and operatively coupled to a computing device (104), for example, via a lead (190; shown coupled to a communication bus 170, which is operatively coupled to the computing device 104, for example, via a lead 168). The computing device (104) can be configured not only to operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transport layer (110) (when it is physically mated to one or more objects, for example, at the contact interface (120)), but also to operate the capacitance sensing controller (174) to acquire data relating to detected capacitance changes near the sensing element (206), which may be related to contact with, for example, an external object. For example, in one embodiment, the integrated system can be configured to increase the frame rate of touch sensing via the deformable transport layer (110) when a capacitance change is detected using sensing capacitance data relating to the sensing element (206). In other words, the system can be configured to provide optimized touch sensing output by utilizing the uncorrelated errors of touch sensing based on capacitance and the deformable transport layer (110) when it is determined that at least some contact indication exists at or near the sensing element (206). In other variations, combinations of various sensors can be utilized, such as sensors with uncorrelated errors, which have aspects that are spatially separated from each other, since in a given implementation, resolution and / or time response requirements may not be the same at each location.

[0082] refer to Figure 10DAn embodiment is shown in which a digital touch sensing component (146) is integrated with an intercoupled resistive sensing subsystem having a resistive sensing controller (176) operatively coupled to a resistive sensing element (208) via, for example, a lead (210). This resistive sensing element may be integrated into a deformable transport layer (110) and configured to facilitate enhanced contact sensing based on resistance sensed between the sensing element (208), which may include a grid or multiple cells, and other objects. This is somewhat similar to how some smartphones or other touchscreen interfaces are configured to detect contact based on detected resistance. The resistive sensing controller (176) may include one or more amplifiers and may be fixedly coupled to a housing (118) of the digital touch sensing component (146) and operatively coupled to a computing device (104), for example, via a lead (192; shown coupled to a communication bus 170, which is operatively coupled to the computing device 104, for example, via a lead 168). The computing device (104) can be configured not only to operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transmission layer (110) (when it is physically mated to one or more objects, for example, at the contact interface (120)), but also to operate the resistance sensing controller (176) to acquire data relating to detected capacitance changes near the sensing element (208), which may be related, for example, to contact with an external object. For example, in one embodiment, the integrated system can be configured to increase the frame rate of touch sensing via the deformable transmission layer (110) when a capacitance change is detected using sensing capacitance data relating to the sensing element (208). In other words, the system can be configured to provide optimized touch sensing output by utilizing the uncorrelated errors of touch sensing based on resistance and the deformable transmission layer (110) when it is determined that at least some contact indication exists at or near the sensing element (208).

[0083] refer to Figure 10EAn embodiment is shown in which a digital touch sensing component (146) is integrated with an intercoupled LIDAR sensor (178), such as a sensor available from Hokuyo Automatic USA Corporation. The LIDAR sensor (178) may be fixedly coupled to a housing (118) of the digital touch sensing component (146) and operatively coupled to a computing device (104), for example via a lead (194; shown coupled to a communication bus 170, which is operatively coupled to the computing device 104, for example via a lead 168). The computing device (104) may be configured to not only operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing a deformable transport layer (110) (when it is physically mated to one or more objects, for example at a contact interface (120)), but also operate the LIDAR sensor (178) to acquire data relating to objects within the field of view (212) of the LIDAR sensor (178), such as point clouds relating to nearby surfaces and objects. In one embodiment, for example, the integrated system may be configured to increase the frame rate of the LiDAR (178) and touch sensing via the deformable transport layer (110) when an unwanted change is detected within the LiDAR (178) field of view (212; which is preferably oriented to be at least partially aligned with the position and orientation of the associated deformable transport layer 110) using LiDAR (178) data. In other words, when the deformable transport layer (110) begins to approach another object, such as another object detected by point cloud changes detected by the LiDAR (178) system, the deformable transport layer (110) and its associated computational and imaging capabilities can enter an enhanced mode to detect and characterize any touch / contact.

[0084] refer to Figure 10FAn embodiment is shown in which a digital touch sensing component (146) is integrated with a strain or elongation sensor (180) coupled to each other. The strain sensor (180) may include one or more elongation detection elements (216), for example in a strain gauge in which resistance is correlated with elongation. Such elongation detection elements (216) may be integrated or embedded in a deformable transmission layer (110), and a strain controller (180) may be fixedly coupled to a housing (118) of the digital touch sensing component (146) and operatively coupled to a computing device (104), for example via a lead (196; shown coupled to a communication bus 170, which is operatively coupled to the computing device 104, for example via a lead 168). The computing device (104) can be configured to not only operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transport layer (110) (when it is physically mated to one or more objects, for example, at the contact interface (120)), but also operate the strain controller (180) to acquire data related to strain or elongation, which may be related to contact with, for example, an external object. One or more elongation detection elements may include a mesh or network and are operatively coupled to the strain controller (180), for example, via one or more leads (214). For example, in one embodiment, the integrated system can be configured to optimize the determination of the touch sensing amplitude via the deformable transport layer (110) when a change in elongation is detected using strain sensor data. For example, if the deformable transport layer (110) moves over a protrusion in the surface, the magnitude of the protrusion determined using the deformable transport layer (110) can be compared with the change in contact surface deflection detected using strain sensors (180, 216), thus providing two data sources for such determination, with at least some uncorrelated measurement / determination errors.

[0085] refer to Figure 10GAn embodiment is shown in which a digital touch sensing component (146) is integrated with a load sensor (182) coupled to each other. The load sensor (182) may include one or more load sensing elements or units (220), for example, it may include one or more devices configured to generate an electrical output that varies with the applied load, such as one or more piezoelectric load units. Such load sensing elements (220) may be integrated or embedded in a deformable transport layer (110), and a load sensor controller (182) may be fixedly coupled to a housing (118) of the digital touch sensing component (146) and operatively coupled to a computing device (104), for example via leads (198; shown coupled to a communication bus 170, which is operatively coupled to the computing device 104, for example via leads 168). The computing device (104) can be configured to not only operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transport layer (110) (when it is physically mated to one or more objects, for example, at a contact interface (120)), but also operate the load sensing controller (182) to acquire load-related data, which may be related to contact with, for example, an external object. The load sensing element may include a mesh or network and is operatively coupled to the load sensing controller (182), for example, via one or more leads (218). In one embodiment, for example, the integrated system can be configured to optimize the determination of the touch sensing amplitude via the deformable transport layer (110) when a load change is detected using load sensor data. For example, if a portion of the deformable transport layer (110) presses against the surface of another object, the amplitude of the contact determined using the deformable transport layer (110) can be compared with the load change on the contact surface detected by the load sensors (182, 220), thus providing two data sources for such determination, with at least some uncorrelated measurement / determination errors.

[0086] refer to Figure 10HAn embodiment is shown in which a digital touch sensing component (146) is integrated with a temperature sensor (184) coupled to each other. The temperature sensing subsystem may include a temperature sensor controller (184), which may include, for example, an amplifier and / or a microcontroller; and one or more temperature sensing elements or units (224), which may include one or more devices, such as one or more thermocouple elements, configured to generate an electrical output that varies with temperature. Such temperature sensing elements (224) may be integrated or embedded in a deformable transport layer (110), and the temperature sensor controller (184) may be fixedly coupled to a housing (118) of the digital touch sensing component (146) and operatively coupled to a computing device (104), for example via leads (200; shown coupled to a communication bus 170, which is operatively coupled to the computing device 104, for example via leads 168). The computing device (104) may be configured to not only operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transport layer (110) (when it is physically mated to one or more objects, for example, at a contact interface (120)), but also operate the temperature sensing controller (184) to acquire data relating to one or more temperatures, which may be related to, for example, contact with an external object. One or more temperature sensing elements (224) may comprise a mesh or network and be operatively coupled to the temperature sensing controller (184), for example, via one or more leads (222). In one embodiment, for example, the integrated system may be configured to optimize the touch sensing characterization via the deformable transport layer (110) upon detection of a temperature change. For example, if a portion of the deformable transport layer (110) is pressed against the surface of another object at a temperature different from the ambient temperature (as is likely to happen when contacting most living tissues in a surgical setting), the magnitude of the contact determined using the deformable transport layer (110) can be compared with the temperature changes of the contact surface detected using temperature sensors (184, 224), thus providing two data sources related to the determination of the contact profile, with at least some unrelated measurement / determination errors.

[0087] refer to Figure 10IAn embodiment is shown in which a digital touch sensing component (146) is integrated with an imaging sensor (186) coupled to each other, in addition to an imaging device (106) operably integrated with a deformable transport layer (110). The imaging sensor (186) may include a camera and may be configured to operate at a variety of selected wavelengths, such as visible light, infrared light, etc. The imaging sensor (186) may be fixedly coupled to a housing (118) of the digital touch sensing component (146) and operably coupled to a computing device (104), for example via a lead (202; shown coupled to a communication bus 170, which is operably coupled to the computing device 104, for example via a lead 168). The computing device (104) may be configured to not only operate the imaging device (106) and the illumination source (116, 122) to facilitate touch sensing by utilizing the deformable transmission layer (110) (when it is physically mated to one or more objects, for example at the contact interface (120)), but also operate the imaging sensor (186) to acquire data relating to objects within the field of view (226) of the imaging sensor (186), such as images relating to nearby surfaces and objects. In one embodiment, for example, the integrated system may be configured to increase the frame rate of touch sensing by the imaging sensor (186) and through the deformable transmission layer (110) when an undesired change is detected within the field of view (226; preferably, which is oriented to be at least partially aligned with the position and orientation of the associated deformable transmission layer 110) using data from the imaging sensor (186). In other words, when the deformable transport layer (110) begins to approach another object, such as another object detected by changes in image data detected by the imaging sensor (186) system, the deformable transport layer (110) and its associated computing and imaging capabilities can enter an enhanced mode to detect and characterize any touch / contact. In an alternative embodiment, the imaging sensor (186) may be configured to operate at infrared wavelengths to aid in the detection of, for example, heat distribution; furthermore, the imaging sensor (186) may include a so-called “depth camera” or “time-of-flight” image sensor, such as a sensor provided by Apple’s PrimeSense, which may be configured to acquire not only image data but also data relating to the depth or z-axis position of such image data relative to the imaging sensor (186).

[0088] refer to Figure 10B-10I And refer again Figure 10A Various combinations and arrangements of the sensing configurations illustrated in these figures can be integrated together in various embodiments. For example, in one embodiment, it may be desirable to have IMU sensor capabilities as well as LIDAR to supplement digital touch sensing via a deformable transport layer (110). Various examples and embodiments are described below.

[0089] refer to Figure 11 A configuration employing a digital touch sensing component (146) coupled to the distal portion (236) of a robotic arm or robotic manipulator (234) mounted to a movable base (238) is shown. The robotic manipulator may include an elongated arm structure comprising various movable joints between rigid or semi-rigid links, as shown in (234); or may include a flexible robotic manipulator, such as those that may be referred to as a robotic conduit or tubular flexible robot (e.g., available from Intuitive Surgical, Inc. or Johnson & Johnson, Inc.). The digital touch sensing component (146) is shown operatively coupled, for example, via a wired or wireless connection (232, 230, 166) to a computing device (144) coupled (136) to a power source (102). The robotic arm (234) can be operated by the computing system (144) to move toward and inspect an object (228) having a surface of interest (70), which may include components that are prone to failure or require periodic inspection, such as rivets (72).

[0090] refer to Figure 12 By utilizing the various aspects of the aforementioned configuration, the digital touch sensing component (146) can be used to inspect the surface (70) and these features (72) through controlled docking with the interface surface (120). In other words, as described above and as... Figure 12 As further shown, in addition to digital touch sensing via the deformable transmission layer (240), various other sensing configurations and associated data can also be used, including but not limited to IMU data (242), capacitive sensor data (244), resistive sensor data (246), LIDAR / point cloud data (248), strain or elongation sensor data (250), load sensor data (252), temperature sensor data (254), and data from an additional imaging device (256).

[0091] refer to Figure 13A It shows the relationship with Figure 11 The system configuration is similar to that of the previous one, but with added sensing capabilities coupled to the connected (258, 230, 166, for example, via a wired or wireless connection to the computing system 144) room or operating environment (260), and additional sensing capabilities coupled to the digital touch sensing component (146). Figure 13AAs shown, a mounting member (359) is configured to couple an additional imaging device (270) to the digital touch sensing component (146), the additional imaging device (270) being positioned and oriented to acquire a field of view relating to the area in front of the interface surface (120) of the digital touch sensing component (146); another mounting member (358) is configured to couple another additional imaging device (272) to the digital touch sensing component (146), the other additional imaging device (272) being positioned and oriented to acquire a different field of view relating to the area in front of the interface surface (120) of the digital touch sensing component (146); in addition, a LIDAR device (274) is coupled to the second mounting member (358), the position and orientation of which help acquire point clouds and other data relating to the operating environment around the digital touch sensing component (146). As described above, in this embodiment, the connected room (260) also has enhanced sensing capabilities, wherein multiple imaging devices (264, 266) and an additional LIDAR sensor (268) are coupled to the room (260), the positions and orientations of which are selected to assist in the precise analysis of the robot (234)’s actions with respect to the object (228) when the object to be inspected (228) is located on a table (262) in the room (260).

[0092] refer to Figure 13B Further enhancements can be included and coupled (318) to the computing device side of the system to allow users operating the computing system (144) to remotely view various aspects of the surface (70) of an object (228) being inspected by the digital touch sensing component (146). Figure 13B As shown, the display (278) can be used to help the user view the output from the digital touch sensing component (146), as well as images or point clouds from other interconnected sensing subsystems (270, 272, 274, 268, 264, 266). Additionally, a haptic interface (280) (e.g., as...) Figure 13C-13F Those shown can be used to help users experience the representation of detected surface features. An interconnected 3-D printer (276) can also be used to complement this "touch sensing workstation," allowing users to decide to directly experience several layers of the detected geometry by printing the geometry locally for direct manipulation (e.g., by the user's hand).

[0093] refer to Figure 13C The haptic interface variant (282) can be configured to be coupled to a computing system (not shown) and provide the user with the sensation of experiencing a real or virtual surface through a manipulation interface (e.g., a spherical member (290)) configured to be held by the user's hand. Figure 13DA haptic interface variant (284) is shown, which is configured to provide a user (4) with a hand (12) gripping and manipulating interface (292) to experience various aspects of a real or virtual surface through a mutually coupled computing system (not shown). Figure 13E and 13F Further variations of the haptic interface (286, 288) are shown, in which the user's (4) hand (12) may be able to manipulate the interface via a pen-like (294) or a fingertip (296) to experience various aspects of a real or virtual surface. Therefore, utilizing... Figure 13B The “touch workstation” configuration and illustration are one of the tactile interfaces, allowing the user to observe (via display 278), directly feel / manipulate (via 3-D printer 276), and tactilely experience (via tactile interface 280) various aspects of the surface (70) of the object (228) being inspected from a nearby or remote location, including all aspects of the surface (70). Therefore, refer to... Figure 14 and 15 This illustrates various aspects of variations of this configuration.

[0094] refer to Figure 14 The user wishes to use a sensing system to engage a surface; the system is calibrated and positioned near the target surface (302). The user navigates the sensing surface to the target surface, for example, via an electromechanical arm or robotic manipulator, and the positioning platform (e.g., inverse kinematics, load cell, deflection sensor, joint position) provides feedback to the user regarding the position and orientation of the sensing surface (304). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities help detect the target surface and its features (e.g., the system can be configured to first enable integrated cameras and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing related to closer engagement) (306). The system can be configured to specifically generate contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (308). Users can use integrated sensing capabilities (such as acceleration detected by an IMU, capacitive touch sensing, resistive touch sensing, LIDAR, strain or deflection meters, load sensing, temperature sensing, and / or cameras and other imaging devices) to reposition and reorient the sensing surface relative to the target surface for inspection (310). The system can be configured to present various aspects of the target surface to the user, thereby enhancing the user's understanding of the target surface, for example, through a combination of visual, tactile, auditory, and tactile perceptions (e.g., through a locally printed surface or a portion thereof) (312).

[0095] refer to Figure 15A user at a location far from the target surface wishes to use a sensing system to engage the target surface; the system is calibrated and positioned near the target surface (314). The user navigates the sensing surface to the target surface, for example, via an electromechanical arm or robotic manipulator, and receives feedback from the positioning platform (e.g., inverse kinematics, load cell, deflection sensor, joint position) regarding the position and orientation of the sensing surface (304). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities aid in detecting the target surface and its features (e.g., the system can be configured to first enable integrated cameras and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing related to closer engagement) (306). The system can be configured to specifically generate contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (308). Users can use integrated sensing capabilities (such as acceleration detected by an IMU, capacitive touch sensing, resistive touch sensing, LIDAR, strain or deflection meters, load sensing, temperature sensing, and / or cameras and other imaging devices) to reposition and reorient the sensing surface relative to the target surface for inspection (310). The system can be configured to present various aspects of the target surface to a remote user, thereby enhancing the user's understanding of the target surface, for example, through a combination of visual, tactile, auditory, and tactile perceptions (e.g., through a locally printed surface or a portion thereof) (316).

[0096] refer to Figure 16A-17 This illustrates various aspects of another exemplary configuration utilizing the integrated touch sensing system described herein. (Reference) Figure 16A The diagram illustrates interconnected rooms, kiosks, or measurement enclosures (324; connected to computing system 144 via wired or wireless connections 320, 230, 166, which, as described above, are integrated with and coupled to other aspects of a touch workstation (e.g., power supply 102, 3D printer 276, display 278, and / or haptic interface 280)). These have several mutually coupled imaging, sensing, and detection resources, such as a LiDAR device (286), one or more imaging devices (264, 266), and a digital touch sensing component (146) mutually coupled to other imaging devices (270, 272) and the LiDAR detector (274). Each mutually coupled resource can be configured to help characterize the geometry and surface of an object (e.g., the foot (322) of a person (4)), whose foot (322) can be lowered (326) to a position where it engages with the digital touch sensing component (146), as shown. Figure 16BAs shown. In other words, the measuring housing or kiosk (324) can be configured to easily engage with a part of a user's appendage, such as a part of the user's leg or arm, to collect precise information related to targets such as the plantar aspect of the user's foot, which can be used to design orthotics, ski boots, etc. The combined data available at the interconnected workstation can be used not only to examine the subject object (e.g., the user's foot) but also to precisely characterize its geometry. For example, the digital touch sensing component (146) can be used to precisely characterize the main load surface (i.e., the bottom surface of the user 4's foot 322), and image and point cloud data can be used to further understand the geometry of the object (the user 4's foot and lower leg), making these findings useful for orthopedic research, pre- or post-operative surgical studies, custom shoe design, etc. Figure 17 One such configuration is shown in the figure.

[0097] refer to Figure 17 In one embodiment, it is desirable to better understand the geometry and load patterns of a foot for a specific user (330). A user can expose their foot, and the system can be initialized in preparation for characterization (332). The user can position / orient their foot within the measurement structure to facilitate scanning the external geometry of the exposed foot (334). The user can reposition / reorient their foot within the measurement structure to further scan the external geometry of the exposed foot (336). The user can place their foot on a deformable transport layer and bear loads on the foot while the system collects data related to load patterns, anatomy, and geometry (338). The system can be configured to create an anatomical / geometric profile of the user's foot, and a load profile associated with the anatomical / geometric profile (340). The anatomical / geometric profile and load profile can be used to create docking structures (e.g., shoes, ski boots, orthotics) and / or diagnose relevant medical conditions (342).

[0098] Refer again Figure 13A and 13B Some surfaces and objects can be presented in a configuration that is easily accessible. Many other fine-grained manipulation and / or contact scenarios involve greater geometric or spatial complexity. See, for example... Figure 18AThis illustrates a fairly simple scenario for a human (346), where the human's (346) hand (348) is used to controllably approach and then touch, examine, and / or grasp a target object, such as a cookie (354), which happens to be located inside a potentially fragile container (344). Therefore, relatively high-load or impact contact should be avoided to maintain the integrity of the container (344) and / or the object (here, the potentially fragile cookie 354). The supporting structure or substrate on the container (344) (e.g., a table 352) may also be fragile or susceptible to damage under high load or impact. The human upper limb happens to be very dexterous, which facilitates successful handling of this example situation, partly due to the smooth motor neurons, muscles, and motor activity of the upper limb, as well as the sensory neurons innervating tissues such as the skin. For example, the human (346) in the illustration typically has sensory neurons throughout the skin in areas such as the wrist (350) and hand (348), allowing the human (346) to carefully navigate the geometry of containers and target objects (354) and the mechanical failure mechanisms associated with them. In other words, a human can use tactile sensing of the skin and other tissues to navigate a scene without damaging the associated structures. Using mechanical systems (such as backhoe tractors (in a magnified version of the scene) or remotely controlled robots) to handle the same scene presents many challenges because a human controlled at a remote location (e.g., a room opposite the robot, or a country opposite the robot connected by computational connectivity) typically does not have the human-level sensing or tactile or sensory perception associated with the interaction and may not perceive that one or more associated structures are about to be damaged until it is too late, for example, through visual or audio confirmation.

[0099] refer to Figure 18A and 18B Body touch sensing technology can be used to address such scenarios and allow users in nearby or remote locations to better feel the physical connection involved.

[0100] like Figure 18BAs shown, an electromechanically controlled robotic arm (234) is depicted located in a room (260), with mutually coupled touch sensing components (146) (e.g., those described above) positioned to inspect an object (e.g., a cookie 354) within a container (e.g., a jar 344) placed on a substrate or support structure (e.g., a table 352). The room (260) may be configured to have multiple sensors, such as a LiDAR (268) and one or more image acquisition devices (264, 266), coupled to the room and positioned to acquire information relating to the volume around the robot and / or the target object (354), preferably in a manner that provides high-quality data from multiple sources with uncorrelated errors, as described above. One or more additional sensing devices (e.g., additional image acquisition devices (270) and a LiDAR (274)) may be coupled to the robotic arm (234) to provide further information relating to the volume around the mutually coupled touch sensing components (146), as well as further high-quality data from multiple sources with uncorrelated errors, to enhance data fusion capabilities. Each sensor (146, 264, 266, 268, 270, 274) can be coupled (232, 258, 230) to one or more computing devices (104) via wired or wireless connections, for example, which can be configured to facilitate control of the interaction. With such a configuration, a remote and target-oriented touch sensing component (146) can be configured to assist a user, possibly located nearby or remotely, in obtaining perception of physical interaction at the deformable transport layer (110) of the touch sensing component (146), as described above. Furthermore, as referenced above… Figure 13B As described, the user can equip a workstation capable of providing one or more means for sensing physical engagement, such as a tactile interface (280), a display (278), and / or a 3D printer (276, even for printing one or more layers of the subject object). To further enhance the user's perception of the physical engagement scenario of a remotely operable manipulation or inspection configuration (e.g., robot 234 as shown), an additional touch sensing component (360) can be coupled to the remotely controllable engagement system (234), for example, as shown, with a configuration surrounding a portion or all of the remote portion of the system. In other words, the additional touch sensing component (360) may include components similar to the touch sensing component (146) described above and is coupled around a portion of the periphery of the associated structure in a manner that provides one or more outward-facing deformable transport layers (110) so that one or more outward-facing deformable transport layers (110) can be operatively coupled (232, 230) to a computing device (104) via wired or wireless connections to provide additional touch sensing for the user of the remote workstation. Figure 18BAs shown, the additional touch sensing component (360) is preferably located at a position on the remotely controlled engagement system (234) that will help the remote user understand key aspects of the remote engagement, such as at a distal or “wrist” position where contact with a target or associated object may occur. For example, positioning the additional touch sensing component (360) peripherally around at least a portion of the distal touch sensing component (146) can help the remote user navigate through the opening of the container (344) and down to the target object (354), as there may be a brush or more direct contact with either sensing component (360, 146) during such approach.

[0101] refer to Figure 18C It shows the relationship with Figure 18B The configuration is similar to that of the robot shown (234), but with the addition of another touch sensing component (362) which is peripherally coupled around at least a portion of the “forearm” member, and again operably coupled (232, 230) to the computing system (104) via, for example, a wired or wireless connection. In practice, both touch sensing components (360, 362) can be configured to perform peripheral sensing around the elongated component (234), for example, through a pair of diametrically opposed touch sensing components (146), a group of three or more touch sensing components that can be separated from each other, such that these touch sensing components are configured, for example, with circumferentially equal spacing (i.e., for maximum coverage relative to the surrounding environment), etc. This additional sensing capability at the illustrated location can further assist a remote user in successfully navigating the illustrated physical engagement challenge to touch, inspect, and / or grasp a target object (here, US dollar bill 355).

[0102] As referenced above Figure 9A and 9B As described above, various sensor configurations can be created by assembling and operably coupling multiple touch sensing components (146), and this mutual coupling can be used to create, for example, Figure 18B and 18C The touch sensing components shown are of the peripheral or partial peripheral type (360, 362). Also as described above, for example, refer to... Figures 7A-7E Components such as optical fibers and / or waveguides can be used to move the sensor to various locations relative to the emitted or acquired radiation (e.g., acquired light). (That is, instead of directly positioning the optical sensor or image acquisition device at the acquisition location, waveguides, transmission optical fibers, or combinations thereof can be used to acquire light at the acquisition location to facilitate transmission from this acquisition location to a more distant optical sensor or image acquisition device.) Reference Figure 18D-18K Various configurations are shown that provide alternatives for radiative transmission associated with touch sensing components such as the aforementioned components (146, 360, 362). Reference Figure 18D For example, it shows the relationship with Figure 7A The configuration shown is similar to that of an optical element (108) operatively coupled to a light (or other wavelength radiation; for example, alternatively, an infrared wavelength) emitting device (116), the configuration of which is selected to cause photons emitted from the light emitting device (116) to propagate (364) to various locations on the optical element (108), at which photons can penetrate the deformable transport layer (110), for example having an exit angle (366) defined by the reflective / refractive properties of the material and the geometry of the structure, for example between about 20 degrees and about 40 degrees. Figure 18E It shows the relationship with Figure 7A The components are configured similarly, with light emanating from both sides (116, 122). (Return to reference) Figure 7A The image acquisition device is sized within a three-dimensional cube with an edge dimension of approximately 1.5 mm, a distance of approximately 3 mm from the imaging object, and a working distance of approximately 5 mm. It is combined with an optical element (108) comprising a material such as a polymer or glass, chosen to facilitate illumination through which light passes, for example, polymethyl methacrylate (“PMMA”), which is relatively inexpensive, easy to form, and relatively easy to polish to facilitate optical properties such as predictable reflectivity. The component thickness can range from 1 to 15 mm in a polymer material with a layer (368) of approximately 4 mm thickness and a deformable transmission layer (110) of approximately 1–2 mm. From the perspective of selection, these dimensions depend at least in part on the illumination requirements and field load demands. Such component dimensions are feasible in various configurations but can be minimized through alternative configurations.

[0103] refer to Figure 18F For example, certain so-called “front-lit” or “front-illuminated” films (372), such as those used in computing device displays (e.g., in mobile devices that can be used outdoors or in other bright environments where conventional backlight configurations may be less effective; for example, devices such as those sold under the trade name Kindle (RTM) may utilize a reflective display configuration selected for use with ambient light, such that the illumination layer is located between the pixels of the display and the viewer), may include light extraction features to controllably extract light or other radiation in a preferred direction, such as toward or away from the deformable transport layer (110; i.e., light bounces 902, for example by total internal reflection, through the illumination film 372, and exits from the film 904 and enters the deformable transport layer 110, which can serve as a carrier and spacer for various optical layers to allow sufficient spacing perpendicular to the plane of the deformable transport layer, i.e., “z-axis spacing,” for mixing light), such as Figure 18FAs shown, at desired locations or distributions along the length of the optical element (108), at the desired exit angle (366), and with a thickness (370) in the range of 100 micrometers. Cladding layers (not shown), such as those comprising silicone materials, can be coupled to the outer surface of the film (372), and carrier layers can also be coupled to each other to provide, for example, additional structure and local planarity. With such a configuration, the component thickness can be reduced by about half, for example to about 5-6 millimeters, depending on the material of the film (372) and the light extraction characteristics. For example... Figure 18F In the configuration shown, due to the positioning of the irradiation layer (372) and the emission path / angle (904, 906), there may be hard-to-access parts (900) of the deformable transmission layer (110). Figure 18G Another embodiment is shown in which the film 372 is located between the optical element 108 and the deformable transport layer 110, and thus closer to the deformable transport layer (110), as in various so-called "front-facing illumination" configurations. Figure 18F Similar to the configuration, features within the irradiation layer can facilitate controlled bounce / reflection 902 (e.g., by total internal reflection) and emission or extraction 904 to direct irradiation toward other layers, such as the deformable transport layer (110) shown. The thickness (370) of the irradiation film (372) can be determined by factors related to irradiation requirements, such as the need for highly controlled irradiation (e.g., more light may require a thicker irradiation film; tighter angle control may require a thinner irradiation film). Importantly, such a layer can be substantially planar, but can also be non-planar or curved to various degrees of complexity (convex, concave, cylindrical, etc.), can be irradiated from various locations, and can be elongated, such as... Figure 18H and 18I As shown -- this can, for example, benefit things like Figure 18B and 18C The peripheral geometry shown in the cuff-shaped peripheral sensors (360, 362). Furthermore, as... Figure 18J As shown, such a membrane (372) can be coupled not only to one side for controlled reflection, but also to multiple sides; Figure 18J One configuration is shown in which the controlled reflective front illumination film is coupled to each other to the four sides (372, 374, 376, 378) around the depicted optical element (108) as shown in the figure, or in other embodiments, in a similar manner... Figure 18I The configuration has up to six sides, with two additional irradiation films coupled to each other on either side of the optical element (108) in a manner coplanar with the drawing, as shown in the figure.

[0104] refer to Figure 18K and 18L As mentioned above, waveguides and transmission or mutual coupling components can be used to efficiently move light between various elements. For example, Figure 18K A wedge-shaped waveguide is shown, the maximum thickness (380) of which can be in the range of 1-5 mm and the included angle (384) of which can be in the range of 1-15 degrees, to help light from the transmitting device (116) propagate (388) across the waveguide (392) to the optical element (108) and the deformable transmission layer (110); an air gap (908) can be configured to help the transmission from the waveguide (392) to the optical element (108). Figure 18L A similar wedge-shaped waveguide is shown, with a maximum thickness (382) ranging from 1 to 2 mm and an included angle (386) ranging from 2 to 8 degrees, to facilitate the propagation of light from the transmitting device (116) across the waveguide (394) (again, an air gap 909 is shown to aid transmission and prevent total internal reflection) and directly into the deformable transmission layer (110). For Figure 18K The configuration allows for the placement of a film (not shown) on the rightmost depicted surface of the deformable transmission layer (110), and additional acquisition devices or cameras, along with additional illumination sources, can be added to the opposite side of the waveguide (394) (shown as the left), provided that the opposite side does not have a specular coating. Specular coatings and so-called “steering film” elements may be included to further aid in the efficient guiding and transmission of light or other radiation between elements (e.g., light exiting the illustrated waveguide 392 may have an outgoing vector that is nearly parallel to a plane perpendicular to waveguide 392, and it may be desirable to “steering” the outgoing light to produce the desired illumination angle, for example, by coupling a steering film to waveguide 392). Components, materials, geometries, and refractive / reflective properties can be tailored to various specific geometric challenges, such as those presented in the various use cases described and illustrated herein.

[0105] As mentioned above, increasing user awareness of activity at remote locations via local workstations—whether the user is across the room, in another building, or around the world—is a key challenge for many computerized systems, such as telecommunications, telepresence, remote inspection, or remote activity systems. (Reference) Figure 19A One way to enhance the user's (4) perception at the local workstation is by means of a haptic main input device (280), which can be operatively coupled (396, 230) to an interconnected computer system (104) via, for example, a wired or wireless connection, so that the user (4) can perceive various aspects of sensation, such as analog transformations of touch, friction, texture, etc., at the local workstation through the user's hand (12) and / or wrist (13). Reference Figure 19BIn another embodiment, facilitating further local perception of remote physical interactions may be valuable by means of a so-called “touch conversion interface” (398), for example, which may be removably coupled to the user’s wrist (13), operatively coupled to a computing system (400, 230), for example via wired or wireless communication, and configured to provide the user (4) with one or more sensations at the wrist (13) or other locations that are related to and / or can be visually associated with activities at remote locations, such as contact between objects at remote locations. Such sensations may be provided in addition to those provided to the user (4) via, for example, a haptic master input device or controller (280). In other words, in various embodiments, multimodal sensations may be provided to the user (4) to help the user perceive activities at remote locations with enhanced fidelity.

[0106] refer to Figures 20A-20C Various aspects of road vehicles (such as computerized electric vehicles) offer opportunities for touch integration and enhancement. For example, human operators typically have fairly consistent touch interfaces with various aspects of the pedals (404, 406), floor (414), driver's seat (412), steering wheel (408), instrument panel control and / or display interface (410), and various parts of the vehicle structure (e.g., parts that may be referred to as "A-pillars" (402)). Each of these structures, along with others, provides opportunities for integrating touch sensing to, for example, aid in operation, control, and safety. See, for example, […]. Figure 20B and 20C Touch sensing components with deformable transmission layers can be operatively coupled to various aspects of the front (438, 440, 442) and rear (444, 446, 448) vehicle bumper or frame structure to help detect impact-related deformation and can be used to trigger safety systems such as seatbelt pretensioners or passenger airbags, as a complement or alternative to other more conventional sensors (e.g., embedded accelerometers) configured to provide such functionality. These more conventional sensors may introduce more delays in the control of such safety systems compared to touch sensing components with deformable transmission layers. In other words, the placement of touch sensing components with deformable transmission layers can be selected to provide intrusion detection very early in the event of an intrusion, possibly before some acceleration detection systems detect an operative change in acceleration, for example, at certain frame components. Figure 20BVarious locations and orientations within a vehicle are illustrated, operatively coupled to touch-sensing components with deformable transmission layers, enabling a central controller or computing system to detect user touches and / or contacts via touch sensors operatively coupled to each of the following structures: pedals (416, 418), driver floor (420), driver seat base (422), driver seat back (424), driver headrest (426), gear shift interface (430), center console interface (428), steering wheel (432), dashboard section (434), and a portion (436) of the A-pillar (402) structure. The touch-sensing components with deformable transmission layers in each of these exemplary structures can have different geometries and incorporate various materials to provide structural characteristics tailored to each use case. For example, the structural modulus of the seat base (422) touch sensor is typically relatively low, and the resolution of the information sought is relatively low (e.g., the operator’s general weight profile, without particularly high resolution to, for example, help determine that a child or dog below a certain weight is not attempting to operate the vehicle); this can be compared to the central console (428) interface, where the structural modulus can be selected to be relatively high, allowing the operator to repeatedly control various aspects of the vehicle by touching the interface without causing significant physical intrusion into the typical touch load, while also providing sufficient intrusion through this typical touch load to obtain the desired information, such as general fingerprint geometric correlation, which can be analyzed upon vehicle startup to obtain a layer of biometric security related to the authorized user / operator.

[0107] One of the challenges in integrating multiple touch sensing components with deformable transmission layers into systems such as automobiles or robots is interconnectivity. (Reference) Figure 21A For example, as described above, various aspects of the control, signal, power, and / or drive connections (232, 230) between a system such as a robot (234) with touch sensing components (146) and a computing system (144) can be achieved via hardwired leads or wireless connections, such as via Bluetooth (RTM), IEEE 802.11, or various other standards. In fact, referencing... Figure 21B It may be desirable that at least some parts or aspects of a system (e.g., a robot (234)) having touch sensing components (146) are in a relatively unconnected form, such that, Figure 21C and 21DAs shown in the enlarged diagram, the wireless transceiver (166) can be used for most (if not all) communication with other coupled systems, while power and some level of controller and / or computing power can be provided by onboard computing devices (144) and power systems (102), such as embedded chipsets, microcontrollers, field-programmable gate arrays, application-specific integrated circuits, etc., and, for example, rechargeable batteries via wireless induction. This integration and the general preference for tetherless configurations can be termed an “Internet of Things” variant and can play a role in many system integration challenges. See, for example, [link to relevant documentation]. Figure 22 Similar to Figure 21C The wirelessly connected touch sensing component (146) shown can be integrated into a door lock system configuration, where a person's thumb (452) or other fingers can be used to engage a deformable transmission layer to provide biometric authentication / lock access for easy unlocking. The touch sensing component (146) can be wirelessly connected to one or more computing systems, such as those within the relevant building, and / or to one or more computing systems that can be mobile, reside in a data center, etc.

[0108] refer to Figure 23A and 23B Variations of the handheld surface (460) analysis tool have features such as Figure 21C The wirelessly connected touch sensing component (146) shown includes a housing (458) configured to engage with a user's hand (462) to facilitate engagement of the deformable transport layer and associated interface surface (120) with the surface (460) of a target object for surface analysis. The touch sensing component (146) can be wirelessly connected to one or more computing systems, such as those within a relevant building, and / or to one or more mobile computing systems residing in a data center, etc.; the handheld component may house its own power source, such as a battery, for operational purposes.

[0109] refer to Figure 24A The diagram illustrates a vehicle configuration with integrated touch sensors, wherein touch sensing components are operatively coupled to various structures, such as an extended touch sensor (436) coupled to the A-pillar (402) of the vehicle, touch sensors (416, 418) coupled to the pedals, a touch sensor (420) coupled to the driver's floor, a touch sensor (428) coupled to the center console, a touch sensor (422) coupled to the base of the driver's seat (412), a touch sensor (424) coupled to the driver's seat back, a touch sensor (426) coupled to the driver's headrest, a touch sensor (430) coupled to the shift member, a touch sensor (432) coupled to the steering wheel, and a touch sensor (434) coupled to a portion of the vehicle's dashboard. These sensors are connected to a central computing system (144) via a lead-type connection (464).

[0110] refer to Figure 24B Sensors with wireless connections to the transceiver (166) of the central computing system (144) in similar locations can help simplify this integration by removing the need for certain connection lines, and can also eliminate the need for power lines in variants where the sensors are operatively coupled to a small power source (e.g., a battery that can be wirelessly inductively charged). Therefore, the A-pillar touch sensor (436) is shown operatively coupled to the wireless transceiver (466); the pedal touch sensors (416, 418) are shown operatively coupled to the wireless transceiver (472, 470, respectively); the floor touch sensor (420) is shown operatively coupled to the wireless transceiver (474); the seat base touch sensor (422) is shown operatively coupled to the wireless transceiver (476); the seat back touch sensor (422) is shown operatively coupled to the wireless transceiver (478); and the headrest touch sensor (426) is shown operatively coupled to the wireless transceiver (478). The following are shown operatively coupled to a wireless transceiver: a shifter assembly touch sensor (430) is shown operatively coupled to a wireless transceiver (486); a center console touch sensor (428) is shown operatively coupled to a wireless transceiver (484); a steering wheel touch sensor (432) is shown operatively coupled to a wireless transceiver (482); and an instrument panel (410) touch sensor (436) is shown operatively coupled to a wireless transceiver (466), each of which is wirelessly connected (166) to the vehicle’s central computing system (144).

[0111] Return to references such as Figure 19A The configuration of touch sensing, in various aspects, can be used to improve and / or enhance user perception of certain operations at the local workstation, and the value of having multiple sensing data sources has been discussed (e.g., with uncorrelated error configurations to enable so-called "sensor fusion" applications). References Figure 25A In one embodiment, a system having multiple sensing configurations (e.g., multiple sensing configurations with uncorrelated error sources) is initialized for use in a first position (488). The system may be configured to provide an operator with information relating to system operation via a user interface (490). Based on one or more commands input by the operator, the system may be configured to execute and provide feedback to the operator via the user interface, the feedback being at least partially based on the multiple sensing configurations (492). The system may be configured to optimize operation and feedback using a sensor fusion technique configured to utilize the differences in information provided by the multiple sensing configurations (494).

[0112] refer to Figure 25B References include systems containing electromechanical arms or manipulators (e.g., references to...) Figures 21A-21DThe described system is a robot manipulator system having multiple sensing configurations (e.g., capacitive, resistive, radar, LiDAR, camera, load sensor, strain or elongation sensor, IMU, and / or joint position sensor configurations with uncorrelated error sources, and touch sensing based on a deformable transmission layer), the robot manipulator system being initialized for use in a first position (496). The system can be configured to provide an operator with information relating to system operation via a user interface (498). To perform a task (e.g., picking up an object from a container) using the robot manipulator system based on one or more commands input by the operator, the system is configured to perform this task and provide feedback to the operator via a user interface at least in part based on the multiple sensing configurations (500). The system can be configured to optimize operation and feedback through sensor fusion technology, which is configured to utilize the differences in information provided by multiple sensing configurations (e.g., when the remote portion of the robot manipulator system is navigated into the opening of a can, some sensors of the multiple sensing configurations may become occluded or temporarily less reliable, while at the same time, preferably at least one other sensing configuration of the multiple sensing configurations with at least slightly uncorrelated errors, such as touch sensing based on a deformable transmission layer, provides reliable information back to the system and the operator) (502).

[0113] Return to reference Figure 19B The integration of one or more touch conversion interfaces (398), such as at the wrist (13) of the user (4), can provide enhanced perception of activity and engagement at a remote location. Figure 26 One configuration is shown in which a user- or operator-local operator interface (506) may have a computing system (144) that is coupled (318) to each of a haptic interface (280), a display system (278), a 3-D printer (276), and a touch conversion interface (504). The user-local operator interface (506) will typically be coupled to a remote control system (such as...) Figure 26 As shown, the robotic arm 234 with touch sensing component 146 is spaced (640) apart, for example, by inches, feet, miles, or thousands of miles, depending on user configuration, the task at hand, and connectivity (230, 166) alternatives, such as wired or wireless connections. Reference Figure 27In further detail, the operator interface (506) may include interconnect (400) computing (144), a main input device / controller (a haptic-enabled variant as shown in FIG280), 3D printing (276) and display (278) resources, and a touch conversion interface (398), such as the variant shown, which may be removably coupled to the user's (4) wrist (13) and configured to provide one or more sensory components that may be perceptibly linked to activities at a remote location, as described in further detail below.

[0114] In various embodiments having one or more touch conversion interfaces at the operator interface (506), it may be desirable to position one or more touch conversion interfaces at locations relative to the user's (4) anatomy, locations that have some kinematic correlation with the activity of components at remote manipulation or actuation locations. For example, see Figure 28A In embodiments where the robotic arm (234) is to be operated at a remote site, wherein the robotic arm (234) has a kinematic portion at least somewhat similar to a “wrist,” a touch sensing component (362) can be functionally coupled to a touch conversion interface, which can be removably coupled to the user’s (4) wrist (13) at a mutually coupled operator interface (503). In other words, if a touch / contact sensed at the robot’s “wrist” is converted to the user’s wrist, the level of intuitive interaction between the local user / operator from the operator interface (503) and the remote robotic manipulator can be enhanced. Thus, in various embodiments, an attempt can be made to provide a pairing of at least some degree of kinematic similarity between remote and local touch sensing and conversion resources.

[0115] Refer again Figure 28A It should also be emphasized that, for a given implementation, more than one touch sensing component can be integrated, for example, an additional at least partial peripheral touch sensing component (360) located around the distal end of the robotic arm (234), around the side of the touch sensor (146), and coupled (232) to computing resources together with another more proximal touch sensing component (362). Reference Figure 28B In a functional pairing configuration that is somewhat kinematically similar, a more distal touch conversion interface (508), such as a finger-sized cuff removably coupled to the index finger, can be operatively coupled (510) (e.g., via a wired or wireless connection) to a computing system and is configured to convert touches or contacts sensed at a more distal touch sensing component (360), which is located at... Figure 28AThe robotic arm (234) is positioned at a remote location around its distal end; a more proximal touch sensing component (398) may be removably coupled to the user's (4) forearm or wrist (13) and operatively coupled (400) to a computing system, for example via a wired or wireless connection, and is configured to convert touches or contacts sensed at the more proximal touch sensing component (362), which is located at... Figure 28A Around the "wrist" of the robotic arm (234) at the remote location shown.

[0116] refer to Figure 29A A gripper (518) type end effector is shown, having two opposing movable members (520, 522) that can be controllably moved toward each other for gripping. In various embodiments, touch sensing components may be integrated into and operatively coupled to these opposing movable members (520, 522) to aid in sensing associated motion. Reference Figure 29B The main input device configuration (516) is configured to allow two opposing fingers of the user's hand (12) to remotely control a grasping action in a manner that is at least partially kinematically similar, for example... Figure 29A The grasping action of the gripper shown (i.e., by moving the opposite hands toward each other, the opposite movable members 520, 522 can move toward each other).

[0117] refer to Figure 29C and 29D Multiple removably coupled touch conversion interfaces (508, 512) can be operatively coupled (510, 514, respectively) to a computing system via, for example, wired or wireless connections, which can be operatively coupled to remote instruments, such as... Figure 29A The gripper (518) shown is designed to provide enhanced intuitiveness for the user or operator (again, by moving opposite fingers toward each other, opposite movable members 520, 522 can move toward each other, and touch / contact information detected by touch sensing components at opposite movable members 520, 522 can be used as input for the senses created for the user at touch conversion interfaces 508, 512). Figure 29C An embodiment is shown in which a touch conversion interface (508, 512) is removably coupled to a user's index finger (526) and middle finger (528), while Figure 29D An embodiment is shown in which a touch conversion interface (508, 512) is removably coupled to a user’s index finger (526) and thumb (524).

[0118] refer to Figure 30AA touch conversion interface (398) removably coupled to a user (4) is shown, which is operatively coupled to a computing system (144) via, for example, a wired or wireless connection (400, 230, 166). The touch conversion interface (398) may include a single touch conversion element or multiple (530) touch conversion elements, as shown, to help provide the user (4) with enhanced perception of touch and / or contact with interconnected touch sensing components. Reference Figures 30B-33B In various embodiments, touch switching elements of various types, combinations, and arrangements can be used. (See reference...) Figure 30B An unbalanced electric motor (532) can be used as the touch switching element to provide touch switching with variable vibration and frequency. (Reference) Figure 30C A light-emitting diode (“LED”) (534) can be used as a touch switching element to provide the user with a visual transition of a touch or contact; the brightness output can vary according to the magnitude of the touch or contact load, and various colors / wavelengths can be used. Reference Figure 30D A piezoelectric component (536) can be used as a touch transducer to provide a relatively high-frequency vibration response to contact or touch, and the frequency and / or intensity can vary depending on the amplitude of the touch or contact load. (Reference) Figure 30E The audio speaker assembly (538) can be used as a touch-transfer element to provide an audible response to touch or contact, and the frequency and / or intensity can vary according to the amplitude of the touch or contact load. Reference Figure 30F and 30G One or more so-called "shape memory alloy" ("SMA") segments (540) can be used as touch switching elements, including alloy materials such as nickel / titanium. For example, such as Figure 30G As shown in Figure (544), commercially available SMA alloys can be configured to shrink considerably in size (e.g., when by means of...). Figure 30F The current shown in 542, when heated by the circuit, contracts to 1 / 2 of its cold length, and is therefore used to controllably apply and / or relax slight ring stresses and / or ring strains when formed as a ring or cuff configuration, for example, as Figure 32A and 32B The variant shown is illustrated.

[0119] Therefore, refer to Figure 31A For example, a touch switching interface (398) operably coupled (400) to a computing system via wired or wireless communication can be removably coupled to a user (4), for example at a wrist (13) position, and may include a controllably actuated haptic actuator motor, such as an unbalanced motor (532). Reference Figure 31BFor example, a touch conversion interface (398) operatively coupled (400) to a computing system via wired or wireless communication can be removably coupled to a user (4), for example at a wrist (13) location, and may include one or more LEDs (534). Reference Figure 31C For example, a touch conversion interface (398) operably coupled (400) to a computing system via wired or wireless communication can be removably coupled to a user (4), for example at a wrist (13) location, and may include a controllably actuated piezoelectric component (536). Reference Figure 31D For example, a touch conversion interface (398) operatively coupled (400) to a computing system via wired or wireless communication can be removably coupled to a user (4), for example at the wrist (13), and may include a controllably actuated audio speaker assembly (538). Reference Figure 31E For example, a touch conversion interface (398) operatively coupled (400) to a computing system via wired or wireless communication can be removably coupled to a user (4), for example at a wrist (13) location, and may include one or more controllably actuated shape memory alloy segments (540). Figure 32A and 32B This shows what, when viewed from an orthographic view, such as Figure 31E The configuration shown may include a single SMA segment (540) (e.g. Figure 32A (as shown in the variant), or multiple SMA segments (540, 546, 548, 550), each of which can be controlled individually.

[0120] Refer again Figure 30A The touch conversion interface may include multiple (530) touch conversion elements, which may be similar to or different from each other. For example, see reference. Figure 33A For example, a touch conversion interface (398) operatively coupled (400) to a computing system via wired or wireless communication can be removably coupled to a user (4), for example at the wrist (13) position, and may include three or more controllably actuated shape memory alloy segments (540, 552, 554) that are longitudinally positioned relative to each other when coupled to the touch conversion interface (398). Figure 33B The touch conversion interface is shown to include a fairly extensive configuration of multiple touch conversion elements, such as multiple SMA segments (540, 552, 554), multiple haptic motors (532, 533), multiple piezoelectric components (536, 537), multiple audio speaker components (538, 539), and multiple LEDs (534, 535), each of which can be individually and / or independently actuated and controlled to provide the user with enhanced perception at the local touch workstation.

[0121] Forward Reference Figure 36 This illustrates an integrated configuration of a surgical robot, in which an operator at a touch-sensing assisted operator workstation can utilize the surgical robot system at a remote location, such as a room, country, or global location (640) separate from the operator workstation, and in which touch-transfer elements can be used to enhance the operator's awareness of contact, touch, and other activities at the remote location during surgical navigation and manipulation of the robotic surgical end effector (e.g., a gripper (518)) relative to a target portion (576) of a target tissue structure (572). Figure 36 As shown, the operator workstation may include one or more (530) element touch conversion interfaces (398) removably coupled to a portion of the user (4), such as the wrist (13) of the user, which may be configured to respond to contact at a touch sensing component (360) of the wrist portion (582) of the robotic instrument (594). The operator workstation may also include two additional touch conversion interfaces (508, 512) which may be configured to respond to contact at a touch sensing component (602, 604) coupled to a relative component (522, 520) of each respective robotic gripper. The touch conversion interfaces are operatively coupled (400, 510, 514, 230, 166) to the computing system (144) via, for example, wired or wireless connections. The touch sensing components may similarly be operatively coupled (592, 606, 608, 230, 166) to the computing system (144) via, for example, wired or wireless connections. Therefore, as the remotely controlled robotic instrument (594) advances and navigates to the target portion (576) of the target tissue structure (572), the user (4) at the workstation can be provided with intuitive perceptual cues relating to contact and touch between various aspects of the instrument and various aspects of the tissue, such as contact between the robotic instrument wrist (582) and the wall or edge (578) of the tissue structure (572), and contact between the robotic instrument gripper (518) components (520, 522) and the wall or edge (578, 576) of the tissue structure (572). Preferably, one or more image acquisition devices may be configured to acquire one or more views of the surgical scene to be presented (598) to the user (4) at the operator's workstation, for example on a display (278), which is operatively coupled to a computing system (144) for example via a wired or wireless connection.

[0122] Therefore, refer to Figure 34In this process, a user at a local workstation has a connection to a remote engagement configuration in a remote environment, such as a connection to a robotic arm operatively coupled to one or more touch-sensing surfaces with one or more connections, to assist the user in physically engaging one or more aspects of the remote environment (556). The local workstation and the remote engagement configuration can be powered on, started, and ready for remote touch engagement by the user (558). The user can operate a main input device at the local workstation, which is operatively coupled to the remote engagement configuration (e.g., a robotic arm operatively coupled to the remote environment), to physically engage one or more aspects of the remote environment (e.g., physically engaging the surface of an object in the remote environment) (560). Through the local workstation, users can experience and understand various aspects of the physical engagement between the remote engagement workstation and one or more aspects of the remote environment (e.g., by locally perceiving various levels of touch engagement at the remote environment via the local workstation; for example, a cuff-shaped touch sensor operatively coupled to the distal portion of a robotic arm in the remote environment can be configured to provide the user with an intuitive understanding of the touch engagement at the remote environment, for example via a local touch conversion interface that can be coupled to the user and can be configured to locally provide one or more modalities of remote touch-derived feedback, for example via kinematic similarity and / or intuitive local configuration via the local touch conversion interface) (562).

[0123] refer to Figure 37 This can be achieved by leveraging touch-sensitive interfaces and similar uses of touch-based operator workstations to help users experience touch, contact, and related activities in a remote environment that is truly remote because it is a virtual environment (612) (i.e., "real" only if created on a computer). For example, in Figure 37In one embodiment, a user can navigate a mobile arm robot (622) virtual element in a virtual environment (612) using a tactile main input device (280). The virtual environment (612) includes virtual aspects such as virtual roads (614), virtual walls (616) defining cavities (618), and virtual prize elements (620) or objects, such as a game-based "gold can" element. If the user can successfully virtually grasp the virtual prize element (620) using virtual grasper elements (628, 630) mounted on the virtual robot arm (626), the user can obtain or win the gold can element. The virtual robot arm (626) is mounted on a virtual mobile base (624) in the depicted virtual environment (612). Virtual touch sensing elements (632, 634, 636) can be virtually coupled to the wrist portion of the virtual robot arm (626) and the virtual gripper elements (628, 630), and configured to provide the actual user at the user workstation with a sense of touch or contact with other aspects of the virtual robot structure and the virtual environment (612), such as portions of the virtual wall (616). In other words, if the user drives the virtual robot (622) such that the virtual gripper elements (628, 630) come into contact with a portion of the virtual wall (616), this contact and / or intersection can be translated back to the touch translation interface (508, 512, 398) at the user workstation to help provide the user with an intuitive sense of activity within the virtual environment (612).

[0124] Therefore, refer to Figure 35A user at the local workstation may have a connection to a virtual remote engagement configuration in a virtual remote environment, such as a connection to a virtual robotic arm operatively coupled to one or more virtual touch-sensing surfaces, to help the user physically engage one or more aspects of the virtual remote environment (564). The local workstation and the virtual remote engagement configuration may be powered on, started, and ready for virtual remote touch engagement by the user (566). The user may operate a main input device at the local workstation, which is operatively coupled to the virtual remote engagement configuration (e.g., to a operatively coupled virtual robotic arm in the virtual remote environment), to physically engage one or more aspects of the virtual remote environment (e.g., virtually physically engaging the surface of an object in the virtual remote environment) (568). Through the local workstation, users can experience and understand various aspects of the virtual physical engagement between the virtual remote engagement workstation and one or more aspects of the virtual remote environment (e.g., various levels of virtual touch engagement at the virtual remote environment locally perceived by the local workstation; for example, cuff-shaped touch sensors of the distal portion of a virtual robotic arm virtually operably coupled to the virtual remote environment can be configured to provide users with an intuitive understanding of the virtual touch engagement at the virtual remote environment, for example via a local touch conversion interface that can be coupled to the user and can be configured to provide one or more modalities of remote touch-derived feedback locally, for example via kinematic similarity and / or intuitive local configuration via the local touch conversion interface) (570).

[0125] See Figure 38AAn orthographic view is shown, illustrating a bushing or at least partially cylindrical touch sensing assembly (656) that can be fixedly or removably coupled to a structural element, such as a shaft member (654) of a machine or machine part, best understood as a load configuration during operation. For ease of illustration, the touch sensing assembly (656) and the shaft member (654) mounted on the top surface (670) of a table (652) are shown, and the interface (726) between the touch sensing assembly (656) and the shaft member (654) can be adhesively bonded to generally prevent relative movement during load. The touch sensing assembly (656) can be operably coupled (658, 230, 166) to a computing system (144), for example via wired or wireless coupling, and can include multiple imaging devices (106) and sources (116). During operation, as the shaft member (654) is loaded, such as by bending back and forth (662, 660), the various parts of the touch sensing component (656) may be in a state of compression, stretching, shearing, etc., and such load can be detected and characterized at the computing system using associated imaging devices (106) and sources (116), which may be placed in sectors (e.g., four pairs are shown near the periphery of the touch sensing component 656). Figure 38B A side view showing a similar configuration is shown.

[0126] Figure 38C It shows the relationship with Figure 38B A slightly similar configuration, but with the addition of a structural cap member (668) that can be configured to constrain the touch sensing component (656) at the junction of the structural cap member (668) and the shaft member (654). With this configuration, the cylindrical touch sensing component (656) can be in a more purely compressed or stretched state as the shaft member (654) bends (662, 660).

[0127] refer to Figure 38D It shows the relationship with Figure 38C A slightly similar configuration, but with a solid cylindrical touch sensing component (672) forming a cylindrical base or pad, to which a structural cap (668) and a shaft (654) end can be mounted (i.e., Figure 38D The shaft shown does not pass through the cylindrical touch sensing assembly 672. This configuration also allows the cylindrical touch sensing assembly (672) to detect not only bending (662, 660) type loads, but also tensile or compressive loads (667, 664) on the shaft member (654), and is generally dependent on the source / imaging device (e.g., Figure 38A 116 / 106 in the text) broadly characterizes the load paradigm in the relevant structural member (654).

[0128] refer to Figure 38E It is worth noting that the sensor and / or transmitter portion can be positioned to directly contact the optical element material of the touch sensing component (656), such as Figure 38A As shown in the configuration, or by using a configuration such as optical fiber or bundles thereof (132, 138) placed at a more distant location to be operatively coupled to other locations, such as the emission detection controller (734) module shown (and operatively coupled to computing system 144 and power supply 102; 730, 732), which may include interfaces (764, 766) configured to efficiently transmit light or other radiation to and from one or more sources and one or more image acquisition devices that may be housed therein.

[0129] refer to Figure 38F To help remove tethering and wired coupling, such as in scenarios involving cyclic torsional loads (758) around an axis (760), and in machine applications, various aspects of the system configuration can be coupled to machine components and wirelessly connected to avoid various tether-based limitations. For example, see reference... Figure 38F The module or housing (742) may contain mutually coupled (752, 754) power supply (744), battery charging (748), and computer / controller (746) components, which may be wirelessly coupled (756) to touch sensing components (656) and further computing devices (144). The motion-based charger (748) has a small mass (750) configured to oscillate based on an oscillating motion of a related axis (654) and provide a low level of current. The motion-based charger (748) may be configured to continuously charge the battery (744); for example, the mass (750) may be configured to move magnetic material oscillatingly through one or more coils, or may be configured to utilize axial motion to load piezoelectric components (e.g., by angular acceleration and velocity square / radius relationship) to provide a low level of charging current to the battery (744).

[0130] refer to Figure 39 It shows the relationship with Figure 36 A slightly similar configuration, with the addition of small touch-sensing component pads (678, 680), which are coupled to the computing system (144) (674, 676, 230, 166) via wired or wireless connections, for example, to achieve similar functionality to that described above. Figure 38D The manner of description provides further characterization of the relative grasping elements of the grasper tool (582).

[0131] refer to Figure 40The user plans to use an electromechanical system (e.g., a robot) to perform medical procedures on a patient. This electromechanical system is configured to have an intervention tool (e.g., a gripper) integrated with one or more touch sensors having one or more deformable transmission layers (690). The user can initiate and calibrate the system using a computing system operatively coupled between the electromechanical system and the user's workstation (692). The user can navigate the intervention tool to the patient's anatomy from a workstation that may be located near or away from the patient. The workstation includes a display system configured to display aspects of the environment surrounding the intervention tool, a control interface (e.g., a haptic interface) to assist the user in issuing commands to the intervention tool, and a touch-transfer interface configured to provide input to the user in response to contact or touch detected at one or more touch sensors operatively coupled to the intervention tool (694). Users can use the control interface to bring the patient's target tissue structure into contact with the intervention tool to perform one or more aspects of a medical procedure, while simultaneously obtaining and / or perceiving information related to the environment adjacent to the intervention tool, such as the contact between the intervention tool and the target tissue structure. This information can be perceived and / or observed using various aspects of the user workstation, such as the display system, control interface, and / or touch conversion interface (696). Users can complete a medical procedure or a part thereof by withdrawing the intervention tool from the target tissue structure and the patient using the user workstation (698).

[0132] refer to Figure 41Users can plan to use a virtual machine electrical system (e.g., a virtual robot) to execute programs (e.g., video games) related to a virtual environment. This virtual machine electrical system can be configured to have virtual tools (e.g., grippers) integrating one or more virtual touch sensors operatively coupled to one or more touch conversion interfaces (702). Users can use a computing system operatively coupled between the virtual machine electrical system and a user workstation to start and calibrate the system (704). Users can navigate the virtual tool to a virtual target from a workstation located near or away from the patient. The workstation includes a display system configured to display aspects of the environment surrounding the virtual tool, a control interface (e.g., a haptic interface) to assist the user in issuing commands to the virtual tool, and a touch conversion interface operatively configured to provide input to the user in response to contact or touch detected at one or more virtual touch sensors operatively coupled to the virtual tool (706). Users can use the control interface to contact one or more virtual objects with virtual tools to perform one or more aspects of the desired virtual tool movement, while simultaneously obtaining and / or perceiving information related to the environment adjacent to the virtual tool, such as the contact between the virtual tool and one or more virtual objects. This information can be perceived and / or observed using various aspects of the user workstation (e.g., display system, control interface, and / or touch conversion interface) (708). Users can also use the user workstation to virtually withdraw the virtual tool from one or more virtual objects, thereby completing a program or a part thereof (710).

[0133] refer to Figure 42A user can plan to perform medical procedures on a patient using an electromechanical system (e.g., a robot) configured to have an intervention tool (e.g., a gripper) integrating one or more touch sensors with one or more deformable transmission layers, and one or more control sensors, which may also have one or more deformable transmission layers (714). The user can initiate and calibrate the system using a computing system operatively coupled between the electromechanical system and a user workstation (716). The user can navigate the intervention tool to the patient's anatomy from a workstation that may be located near or away from the patient. The workstation includes a display system configured to display aspects of the environment surrounding the intervention tool, a control interface (e.g., a haptic interface) to assist the user in giving commands to the intervention tool, and a touch-transformation interface that can be configured to provide input to the user in response to contact or touch detected at one or more touch sensors operatively coupled to the intervention tool (718). Users can use the control interface to bring the patient's target tissue structure into contact with the intervention tool to perform one or more aspects of a medical procedure, while simultaneously obtaining and / or perceiving information related to the environment adjacent to the intervention tool, such as the contact between the intervention tool and the target tissue structure. This information can be perceived and / or observed using various aspects of the user workstation, such as the display system, control interface, and / or touch conversion interface (720). Users can complete the medical procedure or a portion thereof by withdrawing the intervention tool from the target tissue structure and the patient using the user workstation (722).

[0134] refer to Figure 43 The mechanical system may include structural members, such as shafts, beams, or elongated members, which can be subjected to loads such as bending, tension, and / or shear during operation of the mechanical system, and may be coupled to a sensing component including a deformable transmission layer (770). The sensing component may be operatively coupled to a computing system and an imaging device, such that at least one mode of load and / or deformation of the structural member can be monitored using the computing system (772). The sensing component and the computing system may be initialized, calibrated, and / or configured to sense one or more aspects of the structural member during operation of the mechanical system (774). The computing system may be configured to provide an operator with output relating to the real-time or near-real-time load configuration of the mechanical system, such as load data relating to the structural member that can be displayed to the operator and / or an indication to the operator that one or more predetermined load thresholds have been approached or met within the mechanical system (776). The computing system may also be configured to facilitate changes in the operation of the mechanical system, such as reducing load demand or shutting down one or more aspects of the mechanical system (778), when the computing system determines that an overload condition has been met (e.g., by comparing the output from the sensing component with one or more predetermined load thresholds).

[0135] refer to Figure 44 The vehicle (e.g., an automobile) may include one or more structural components, such as one or more housings and / or support structures, which may be subjected to loads, such as bending, tension, and / or shear, during vehicle operation and may be coupled to one or more sensing components (780) including one or more deformable transmission layers. The one or more sensing components are operatively coupled to a computing system and one or more imaging devices, such that the computing system can be used to monitor at least one mode of load and / or deformation of the one or more structural components (782). The one or more sensing components and the computing system may be initialized, calibrated, and / or configured to sense one or more aspects of the one or more structural components during vehicle operation (784). The computing system may be configured to provide an operator with output related to the real-time or near-real-time load configuration of the one or more structural components, such as load data related to the one or more structural components that can be displayed to the operator and / or used to create an indication for the operator that one or more predetermined load thresholds have been approached or met (786). The computing system can also be configured to facilitate the operation of one or more structural components and / or other components of the vehicle, such as reducing load demand or shutting down one or more operatively coupled systems, components or subsystems (788), when the computing system determines that an overload condition has been met (e.g., by comparing the output from one or more sensing components with one or more predetermined load thresholds).

[0136] refer to Figure 45 The mechanical system may include structural members, such as shafts, beams, or elongated members, which may be subjected to loads, such as bending, tension, and / or shear, during operation of the mechanical system, and may be coupled to a sensing base assembly (790) including a deformable transmission layer. The sensing base assembly may be operatively coupled to a computing system and an imaging device, such that the computing system can monitor at least one mode of load and / or deformation of the structural member (792). The sensing base assembly and the computing system may be initialized, calibrated, and / or configured to sense one or more aspects of the structural member during operation of the mechanical system (794). The computing system may be configured to provide an operator with output relating to the real-time or near-real-time load configuration of the mechanical system, such as load data relating to the structural member that may be displayed to the operator and / or an indication to the operator that one or more predetermined load thresholds have been approached or met within the mechanical system (796). The computing system may also be configured to facilitate changes in the operation of the mechanical system, such as reducing load demand or shutting down one or more aspects of the mechanical system (798), when the computing system determines that an overload condition has been met (e.g., by comparing the output from the sensing base assembly with one or more predetermined load thresholds).

[0137] refer to Figure 46A user at the local workstation may have a connection to a remote engagement configuration in a telemedicine intervention environment, such as a connection to a medical robotic arm operably coupled to one or more touch-sensing surfaces with one or more connections, to help the user physically engage one or more aspects of the telemedicine intervention environment (802). The local workstation and the remote engagement configuration may be powered on, started, and ready for the user to perform telemedicine touch engagement (804). The user may operate a main input device at the local workstation that is operably coupled to the remote engagement configuration (e.g., to a medical robotic arm operably coupled to the remote environment) to physically engage one or more aspects of the remote environment (e.g., physically engaging the surface of an object (e.g., a target tissue structure) in the remote environment) (806). Through the local workstation, the user is able to experience and understand aspects of the physical engagement between the remote engagement workstation and one or more aspects of the remote environment (e.g., by locally perceiving various levels of touch engagement at the remote environment via the local workstation; for example, a cuff-shaped touch sensor operatively coupled to the distal portion of a medical robotic arm in the remote environment can be configured to provide the user with an intuitive understanding of the touch engagement at the remote environment, for example via a local touch conversion interface that can be coupled to the user and can be configured to locally provide one or more modalities of remote touch-derived feedback, for example via kinematic similarity and / or intuitive local configuration of the local touch conversion interface) (808).

[0138] refer to Figure 47A user at the local workstation may have a connection to a remote engagement configuration in a telemedicine intervention environment, such as a connection to a medical robotic arm operatively coupled to one or more touch-sensing surfaces with one or more connections, to help the user control the remote engagement configuration and physically engage one or more aspects of the telemedicine intervention environment (810). The local workstation and the remote engagement configuration may be powered on, started, and ready for remote medical touch engagement by the user (812). The user may operate a main input device at the local workstation, which is operatively coupled to the remote engagement configuration (e.g., to a medical robotic arm operatively coupled to the remote environment), to physically engage one or more aspects of the remote environment (e.g., physically engaging the surface of an object (e.g., a target tissue structure) in the remote environment) within one or more predetermined load limits, which may be monitored relative to one or more loads applied to one or more connected touch-sensing surfaces (814). Through the local workstation, a user can experience and understand various aspects of the physical engagement between the remote engagement workstation and one or more aspects of the remote environment (e.g., by locally sensing various levels of touch engagement at the remote environment via the local workstation; for example, a cuff-shaped touch sensor operatively coupled to the distal portion of a medical robotic arm in the remote environment can be configured to provide the user with an intuitive understanding of the touch engagement at the remote environment, for example via a local touch conversion interface that can be coupled to the user and can be configured to locally provide one or more modalities of remote touch-derived feedback, for example via kinematic similarity and / or intuitive local configuration of the local touch conversion interface), and physically engage various aspects of the remote medical intervention environment within one or more predetermined load limits that can be monitored relative to one or more loads applied to one or more connected touch sensing surfaces (816).

[0139] refer to Figure 48 It shows something similar to Figure 29C The embodiments illustrate a hybrid configuration of touch sensing and touch conversion for each of two fingers (index finger 526, middle finger 528), wherein touch conversion interfaces (508, 512) can be removably coupled to each finger to obtain the same as those referenced above. Figure 29C The kinematically similar feedback described above can be achieved, for example, by adding a cuff-type touch sensing interface (822, 820; for example, similar to the reference above). Figure 18CThe aforementioned 360, 362) are removably coupled to the finger and operatively coupled (826, 824) to a computing system, for example, via a wired or wireless connection (510, 514). Such a configuration can be configured and operated to, for example, not only provide the user with one or more sensations relating to activity at an interconnected system (e.g., a remote robotic gripper), but also provide the interconnected computing system with further information relating to local activity of the user's finger (e.g., a touch-sensing interface (822, 820) can be used to sense a related increase or decrease in circumferential stress or strain, which may be related to the actuation, activity, movement or intention of the finger, and contact between the finger and other objects).

[0140] Therefore, refer to Figure 49 This shows that the above reference can be used. Figure 48 Exemplary variations of the described configuration. (See reference) Figure 49 A user at the local workstation can have a connection to a remote engagement configuration in a remote environment, such as a connection to a robotic arm operatively coupled to one or more touch-sensing surfaces with one or more connections, to help the user physically engage one or more aspects of the remote environment (830). The local workstation and the remote engagement configuration can be powered on, started, and ready for remote and local touch engagement by the user (832). The user can operate the main input device and the local touch-sensing configuration at the local workstation, both of which can be operatively coupled to the remote engagement configuration (e.g., to a operatively coupled robotic arm in the remote environment) via a computing system to physically engage one or more aspects of the remote environment (e.g., physically engaging the surface of an object in the remote environment) (834). Through the local workstation, the user's touch activity can be sensed to aid in the operation of the remote engagement configuration, and the user can experience and understand various aspects of the physical engagement between the remote engagement workstation and one or more aspects of the remote environment (e.g., various levels of touch engagement at the remote environment can be perceived locally by the local workstation; for example, a cuff-shaped touch sensor operatively coupled to the distal portion of a robotic arm in the remote environment can be configured to provide the user with an intuitive understanding of the touch engagement at the remote environment, for example via a local touch conversion interface that can be coupled to the user and can be configured to provide one or more modalities of remote touch-derived feedback locally, for example via kinematic similarity and / or intuitive local configuration via the local touch conversion interface) (836).

[0141] refer to Figure 50 It shows the relationship with Figure 49 Similar configurations, but in which operators / users can use a similar hybrid local interface to operate in synthetic or virtual environments. (See reference) Figure 50A user at the local workstation can have a connection to a virtual remote engagement configuration in a virtual remote environment, such as a connection to a virtual robotic arm operatively coupled to one or more virtual touch-sensing surfaces, to help the user physically engage one or more aspects of the virtual remote environment (840). The local workstation and the virtual remote engagement configuration can be powered on, started, and ready for virtual remote touch engagement by the user (842). The user can operate a main input device and a local touch-sensing configuration at the local workstation, both operatively coupled to the virtual remote engagement configuration (e.g., operatively coupled to a virtual robotic arm operatively coupled to the virtual remote environment) to physically engage one or more aspects of the virtual remote environment (e.g., virtually physically engaging the surface of an object in the virtual remote environment) (844). Through the local workstation, the user's touch activity can be sensed to aid in the operation of the virtual remote engagement configuration, and the user can experience and understand various aspects of the virtual physical engagement between the virtual remote engagement workstation and one or more aspects of the virtual remote environment (e.g., various levels of virtual touch engagement at the virtual remote environment can be perceived locally by the local workstation; for example, a cuff-shaped touch sensor that is virtually operably coupled to the distal portion of a virtual robotic arm in the virtual remote environment can be configured to provide the user with an intuitive understanding of the virtual touch engagement at the virtual remote environment, for example via a local touch conversion interface that can be coupled to the user and can be configured to provide one or more modalities of remote touch-derived feedback locally, for example via kinematic similarity and / or intuitive local configuration via the local touch conversion interface) (846).

[0142] refer to Figure 51A It shows the reference Figure 7A The described system configuration is similar to that described above, such that a touch sensing component (146) with a deformable transmission layer (110) is configured to be placed in contact with the surface of the object to be characterized. In various embodiments, a deformable transmission layer (110) having a planar or semi-planar shape may be useful, for example, in scenarios where it is necessary to observe and characterize the surface of a banknote placed on a flat table, or the fingerprint pattern of a finger pressed towards the deformable transmission layer. References Figure 51B For comparison, the following is shown Figure 51A A smaller version of the touch sensing component (146) configuration. Depending on the specific scenario, it may be desirable to use a touch sensing component (146) with a deformable transmission layer having a load-free shape, rather than the planar or semi-planar shape described above. For example, refer to Figure 51C The touch sensing component (146) is shown to have an arc-shaped deformable transmission layer (1020), which may be useful when processing arc-shaped or concave surfaces. Figure 51D and 51EVariations of the deformable transport layer shape are shown, for example, it can be elliptical (1022) or hemispherical (1024). Figure 51F and 51G Variations in the shape of the deformable transport layer are shown, for example, it can be a semi-elliptical or hemispherical shape with a proximal elongation as shown (1026, 1028). For example Figure 51F and 51G The configuration shown can be used to inspect and / or characterize surfaces such as concave or cylindrical shapes. Reference Figure 51H and 51I The touch sensing component (146) can be configured to have an expandable lumen or sac, such that it can be used as... Figure 51H The small, more elongated insertion configuration shown (i.e., an inflation lumen or bladder in a relatively uninflated configuration, e.g., with gas or liquid) is inserted to engage a surface, such as a hole or cylindrical surface (1030). Then, once in position for measurement and / or surface characterization, the volume of the deformable transport layer increases (i.e., the inflation lumen or bladder in a relatively inflated configuration, e.g., by positive pressure of gas or liquid) (1032), causing it to be pushed against the surrounding target surface for measurement and / or surface characterization. Afterward, it can deflate again and return to the minimum configuration (1030) and be removed. Interface loads can be characterized by understanding the modulus of the deformable transport layer material and precise surface-dependent deflection information. In fact, by understanding the properties of the deformable transport layer material, various properties of the mating materials can also be determined by using specific loading modes at the interface. For example, in one embodiment, aspects of the structural modulus of the mating structure, as well as the static and / or dynamic coefficients of friction (i.e., by detecting the interface load before slippage occurs after a load is applied, and after initial slippage into the dynamic coefficient of friction under sustained load) can be estimated, measured, and / or determined using the response of the target surface detected by the deformable transport layer. In addition to sliding, rolling-type deformable transport layers can also be used, such as layers comprising cylindrical or partially cylindrical deformable transport layers. This configuration can be used to acquire data on rolling along the target surface in a preferred rolling direction (i.e., like rolling paint with a paint roller), determined by the rolling degrees of freedom of the rollable deformable transport layer, and / or the roller can slide in another direction (i.e., in a manner that paint is applied in a direction not aligned with the preferred rolling direction of the paint roller relative to the wall).

[0143] For example Figure 51C-51IDeformable transport layers with curvature radii shown in (1020, 1022, 1024, 1026, 1028, 1030, 1032) can be configured to handle specific applications at hand. For example, as described above, in various embodiments, the curvature radius can be selected to at least partially match the curvature radius of the target surface. In other embodiments, relatively small curvature radii, such as in the range of about 0.5 mm to about 5 mm, can be used to help effectively characterize the location of a point in space. In other embodiments, the deformable transport layer may include a relatively high modulus or high stiffness portion (e.g., a relatively small spherical or cubic portion within the larger deformable transport layer) at a known XY location to provide effective point sensor functionality at that known point.

[0144] refer to Figure 52 It shows the reference Figure 11 The configuration described is similar to that described, in which the touch sensing component (146) (e.g., reference) Figure 51A-51I The components described are coupled to an electromechanical arm (234), such as a robotic arm, which can be actively controlled, for example by drive commands from a user or from a software-based controller. The arm (234) can be used to controllably and accurately position and orient the touch sensing component (146) via active electromechanical navigation and / or movement (e.g., via interconnected motors), making it possible to use the touch sensing component (146) to characterize the surface (1034) supported by the base or substrate (1036).

[0145] refer to Figure 53 It shows the relationship with Figure 52 A similar configuration, but without the active electromechanical motion provided by the associated articulated arm, can be configured for positioning and orientation by being pulled back and forth by a user using one or more handles (1040, 1041), and the joints of the arm can be electromechanically braked, allowing the user to command a brake (1038) to maintain the position and / or orientation in space (in other words, the arm can be configured to engage and disengage for manual movement by the user using the handles). The brake joint (1038) can be configured to have a joint position sensor, such as an optical encoder, to help determine the joint position, thereby determining the overall position and orientation of the touch sensing component (146), such as its position and orientation relative to a global coordinate system.

[0146] refer to Figure 54 It shows the relationship with Figure 53The configuration is similar, but has a passive (i.e., unbraked) joint (1042) that allows the user to pull the touch sensing component (146) in space and manually engage it with the surface (1034), while the joint position of the wall can be used to track the position and / or orientation of the touch sensing component (146), for example, relative to the global coordinate system.

[0147] refer to Figure 55 The figure shows a configuration without a support arm, allowing the operator or user to manually hold it in place / orientation, for example, by using handles (1040, 1041) coupled to a main housing (1044) which is coupled to a touch-sensing assembly (146). Referring back to the associated... Figure 56 To aid in tracking the position and / or orientation of the touch sensing component (146) in space and relative to a surface of interest (1034) and / or a global coordinate system (1050), one or more tracking systems (1046) can be operatively coupled to a computing device (104) via, for example, a wired or wireless connection (1048) to assist in determining such position and / or orientation. For example, in various embodiments, an optical tracking configuration can be utilized, using tracking reference points mounted, for example, on a housing (1044) or the touch sensing component (146); and a detector (e.g., a stereo detector-based configuration) including a 3D tracking system (e.g., a system available from Northern Digital, Inc.). Similarly, tracking can be performed using an electromagnetic tracking system (e.g., a system available from Ascension, Inc.), for example, relative to a global coordinate system (1050). In fact, refer to... Figure 57 In addition to kinematic-based tracking configurations (such as the tracking configuration of arm 234), such tracking systems (1046) can also be used. Furthermore, see reference... Figure 58 It shows that it has the same characteristics as Figure 13A The same configuration of some components, for example, also includes such as Figure 57The tracking component shown is used to track and / or determine, for example, position and / or orientation relative to a global coordinate system (1050). The imaging or image acquisition device (270, 272) shown may include various types of detectors and may also be used in conjunction with a texture projector in a stereo configuration to aid in depth and other feature characterization, as well as to address occlusion that may occur at different positions and / or orientations of the component (146) (i.e., by positioning with different viewpoint vectors toward the target surface). Furthermore, as described above, the image acquisition device located within the touch sensing component (146) may also be used for image acquisition via a deformable transport layer. The acquisition of various images and / or data points may be induced in various ways, such as manually by an operator (e.g., initiated via a control interface triggered by a button, software, voice activation, remote connection device, etc.) and / or automatically (e.g., by force constraints, defined geometric or measurement constraints, or focal length constraints based on optical or image acquisition devices).

[0148] refer to Figure 59A It shows the relationship with Figure 58 In a similar configuration, the touch sensing component (146) is being positioned and oriented to characterize various aspects of a manufactured engine block mechanical component (1126). In various embodiments, the touch sensing component (146) can be positioned and / or oriented to various locations and orientations using an articulated arm (234), thereby characterizing the surfaces of the engine block (1126). Furthermore, a model of the engine block, such as an ideal “by-design” computer-aided design (“CAD”) model, can be stored in a storage device or system (1052) operably coupled to a computing system (144), for example via a wired or wireless connection (1054) – and this model can be used to analyze and observe the engine block mechanical component (1126) being examined using the touch sensing component (146), for example by comparison with an ideal model. In various embodiments, the model can be registered with the observed version in position and orientation, for example by collecting a series of points and / or surfaces and determining the registration alignment. The actual part can then be measured to determine its conformity to the ideal model, for example, for quality assurance purposes. In practice, in various embodiments, a digital representation of the ideal model can be presented to show variations, defects (e.g., geometrical changes, more subtle issues such as scratches, etc.) and / or deviations from the ideal model (i.e., if a component should be straight in the ideal model but is curved in the measured model, the component can be represented as curved in the digital representation and can be visually highlighted as a deviation, for example by distinguishing it with different colors in the relevant display interface).

[0149] refer to Figure 59B It shows the relationship with Figure 59AThe configuration is similar, but with the addition of an operablely coupled measurement system (1120) and a measurement probe (1118). The measurement probe (1118) can be configured to provide point determination in addition to (i.e., in parallel) information collected by other integrated system components (146, 234, 144, etc.). A suitable measurement probe (1118) may also be referred to as a “contact probe,” a “coordinate measuring machine probe,” or a “CMM probe” (“CMM” generally refers to a coordinate measuring machine having a measurement probe and being configured to provide measurements using such a probe). The measurement system can be operablely coupled to a computing device (144), for example, via a wired or wireless connection (1122). In addition to utilizing information from the various discrete sensors (i.e., sensors 270, 272, 274, 1118) in parallel, it may be desirable to utilize image acquisition or detection capabilities, including a touch sensing component (146), for additional image acquisition or light detection. For example, in various embodiments, the deformable layer can be removed from the location between the detector / image acquisition element and the target object, so that the detector / image acquisition element of the touch sensing component (146) can be used to collect additional data related to the target object. Various optical configurations (e.g., fixed or variable-focus lenses, such as lenses that can be focused using electromechanical actuation, variable fluid capsules, etc.) and / or other refractive optical processing can be used to assist in this collection and analysis. Adaptive lenses can also be used to capture the three-dimensional surface topology associated with the target surface. Furthermore, characterization of the target object can be assisted by using stereo imaging techniques (acquiring simultaneously by two different sensors, or by achieving time shifting by repositioning and / or reorienting a given sensor within adjacent collection time periods). Still referring to… Figure 59B Although the robotic system (234) is shown to position and / or orient the touch sensing component (146) toward the target object (1126), it should be noted that touch sensing components of various shapes (i.e., planar, half-planar, curved, finger-shaped, concave, convex, etc.) can be inserted, rubbed / dragged, rolled (i.e., using cylindrical and / or rotatable deformable transmission layers), and otherwise approached / docked in various levels of complexity to assist in the analysis of the target object.

[0150] In the various embodiments described herein, the acquisition of information and / or images related to the target object can be triggered automatically or manually, and in some cases by user commands, such as button presses (or other user interface commands, which may be local or from a remotely connected system / subsystem), voice commands, force / load-based automatic thresholding commands, geometry-based commands (e.g., when a given target feature (e.g., a hole) is fully visible), and focus-based commands (e.g., when an image of the target geometry is in optimal focus). The system can be configured to acquire and retain images from specific locations, orientations, vectors, etc., and specific data and / or images from the system can be projected onto images of one or more target objects to assist the user in visualization.

[0151] refer to Figure 60A It may be desirable to have a convenient interface for mechanically and / or electromechanically coupling the touch sensing component (146) and its associated hardware to the arm (234). A set of removable coupling interfaces (1056, 1058) may be configured such that they are firmly pushed in and locked together during operation (e.g., as shown in the image). Figure 60B , 60C (as shown in 60D), and then conveniently decouple back later. Figure 60A The state shown. (Reference) Figure 60E The diagram illustrates interface configurations, such as one of a pair of mating pairs (1056, 1058), having multiple protruding features (1060, 1062) and one or more cavity features (1064), as well as electronic engagement features (e.g., power lines can pass through interface 1066 via contact; information I / O interfaces can pass through interface 1068 via contact). Relative / reverse interfaces (e.g., with protruding members configured to fit into illustrated cavity 1064, and cavities configured to precisely engage illustrated protruding members 1060, 1062) can be conveniently and removably coupled to each other in a known relative orientation. To maintain engagement of the mechanical and electrical interfaces (1066, 1068) when needed, a handle (1072) can be used to rotate a screw (1070) (i.e., screwed into and secured to the inserted protruding member mating with illustrated cavity 1064) for temporary fixation during coupling. Figure 60D Electronic and / or power coupling through a removable joint is shown (232).

[0152] refer to Figures 61A-61CThe intermediate adapter component (1057) can be used to accommodate coupling between two interfaces that are not designed to be coupled to each other (in other words, if A is not designed to be coupled to C, the adapter 1057 can be configured to provide removable coupling by coupling one side of the adapter to A and the other side of the adapter to C; i.e. A-(AB / BC)-C, where the “AB / BC” part in this simple representation is the adapter (1057).

[0153] refer to Figure 61D-61F One or more variations of structural or mounting components (358) may be used to demonstrate removably coupled or detachable configurations designed to become handheld as needed (e.g. Figure 60A , 60B Those configurations shown in detached form in 61A, 61B and 61F can be in similar variants to those attached to the reference shown (e.g. Figure 58 , 59A The method shown in -B) is mechanized to enhance the ability to manipulate target surfaces and / or structures. For example, refer to Figure 61D and 61E The image shows a sensing component (146) that is still coupled to a support structure (e.g., a robotic arm (234)). Figure 61D The variant has a closer mounting member (358) coupled to the main housing (1044), which has an image acquisition device (272), a lidar (LIDAR) device (274), and an inertial measurement unit (IMU 1119) coupled thereto; for example, it may include one or more accelerometers and one or more gyroscopes to help sense linear and angular acceleration. A reverse operating handle (1040) can be used to mount or couple the additional image acquisition device (270) and measurement probe (1118) as described above, enabling manual or automatic monitoring and / or positioning of the sensing component (146) relative to other objects (e.g., target surfaces) via a touch-sensing interface. Figure 61E and 61F The embodiments show similar devices, but the mounting structure (358) of the carrying device (270, 272, 274, 1119, 1118) is closer to the touch sensing interface of the sensing component (146), and the mounting structure (358) is directly coupled to the sensing component (146). Figure 61F The distal portion is shown with Figure 61E The proximal support robotic arm (234) is decoupled, allowing it to be held and moved freely in space relative to other objects, while also being tracked using instruments (e.g., 270, 272, 274, 1119, 1118). For example, Figure 61EThe embodiments of 61F can be used for electromechanical movement (6IE) or manual movement (6IF) to perform tactile analysis of target objects within the range reached by the sensing component (146), for example by individual touch / contact vectors or methods, by repeating patterns of adjacent touches / contacts, by predetermined patterns (e.g., adjacent touches / contacts), or by a series of more exploratory methods using simultaneous localization and mapping (SLAM) methods, to explore and characterize one or more geometric features that may, for example, be previously uncharacterizable (e.g., under a hole or aperture, or inside a defect, or very difficult to access or image a surface or feature). In various embodiments, the operatively coupled computing system can be configured and used to stitch together geometrically adjacent geometric contours by interpolation of the geometric contours and their relative positions and orientations, and / or to present to a user, using a graphical user interface, a two-dimensional or three-dimensional mapping of one or more geometric contours relative to each other, for example, in a global coordinate system.

[0154] refer to Figure 62 In one embodiment, the user wishes to engage the sensing system with a surface that may be convex, concave, saddle-shaped, cylindrical, or other more complex or simpler surfaces; the system can be calibrated and positioned near the target surface (1080). The user can navigate the sensing surface to the target surface, for example, via an electromechanical arm or robotic manipulator, and the positioning platform (e.g., inverse kinematics, load cell, deflection sensor, joint position) provides feedback to the user regarding the position and orientation of the sensing surface (1082). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can readily detect the target surface and its features (e.g., the system can be configured to first enable integrated cameras and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing related to a closer engagement) (1084). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (1086). The system can be configured to fit onto a target surface to characterize the surface using a deformable transport layer and store information related to the characterized target surface, such as geometric profile, location, and / or orientation, for example, relative to a global or other coordinate system (1088).

[0155] refer to Figure 63In one embodiment, the user wishes to engage the sensing system with a surface that may be convex, concave, saddle-shaped, cylindrical, or other more complex or simpler surfaces; the system can be calibrated and positioned near the target surface (1080). The user can navigate the sensing surface to the target surface, for example, via an electromechanical arm or robotic manipulator, and the positioning platform (e.g., inverse kinematics, load cell, deflection sensor, joint position) provides feedback to the user regarding the position and orientation of the sensing surface (1082). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can aid in detecting the target surface and its features (e.g., the system can be configured to first enable integrated cameras and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing related to a closer engagement) (1084). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., to slow down the repositioning and reorientation of the sensing surface and to convey contact using audio, visual, and / or tactile cues), and the system can be configured to alter the shape or compliance of the sensing surface or associated substrate structure, for example by controlled expansion or contraction of the capsule and / or lumen with a liquid or gas (1092). The system can be configured to adhere to the target surface to characterize the surface using a deformable transport layer and to store information associated with the characterized target surface, such as geometric profile, position, and / or orientation, for example, relative to a global or other coordinate system (1094). The system can be configured to again alter the shape or compliance of the sensing surface or associated substrate structure, for example by controlled expansion or contraction of the capsule and / or lumen with a liquid or gas (1096).

[0156] refer to Figure 64In one embodiment, the user wishes to engage with a sensing system that may be convex, concave, saddle-shaped, cylindrical, or other more complex or simpler surfaces; the system can be calibrated and positioned near the target surface (1080). The user can navigate the sensing surface to the target surface, for example, via an electromechanical arm, which may include an actively driven robotic arm, a manually positioned articulated arm with electromechanical brakes, a manually positioned articulated arm without electromechanical brakes, and / or a tethered or untethered configuration for manual holding and orientation (1102). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can aid in detecting the target surface and its features (e.g., the system can be configured to first enable integrated cameras and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing associated with closer engagement) (1104). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (1106). The system can be configured to fit onto a target surface to characterize the surface using a deformable transport layer and store information related to the characterized target surface, such as geometric profile, location, and / or orientation, for example, relative to a global or other coordinate system (1108).

[0157] refer to Figure 65In one embodiment, the user wishes to engage the sensing system with a surface that may be convex, concave, saddle-shaped, cylindrical, or other more complex or simple surfaces; the system can be calibrated and positioned near the target surface (1080). The user can navigate the sensing surface to the target surface, for example, via an electromechanical arm, which may include an actively driven robotic arm, a manually positioned articulated arm with electromechanical brakes, a manually positioned articulated arm without electromechanical brakes, and / or a tethered or untethered configuration for manual holding and orientation (1102). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can readily detect the target surface and its features (e.g., the system can be configured to first enable an integrated camera and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing related to a closer engagement) (1104). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (1106). The system can be configured to adhere to a target surface to characterize the surface using a deformable transport layer and store information related to the characterized target surface, such as geometric profiles, location, and / or orientation, for example, relative to a global or other coordinate system (1108). The system can be configured to register the locations of known points on the surface with portions of a known model, thereby registering the system (i.e., determining the known location / orientation relationship between the model and the measured surface); registration can be automated, for example, through automatic registration based on a series of points or surfaces acquired during the measurement process, for example, with the aid of a neural network trained using data associated with the known model (1112). The system can be configured to determine differences between measured dimensions, surface orientation, etc., for quality assurance and / or inspection purposes (1114).

[0158] refer to Figure 66A The substrate (1130) structure or layer is shown as having various forms of holes or defects, such as defects whose geometry may be at least partially concave. For example, an illustrated hole (1132) may comprise a generally cylindrical, cubic, or rectangular volume, which can be formed in the substrate (1130) by drilling or similar machining, or by photolithography or other various techniques. As mentioned above, it may be desirable to characterize the hole (1132), for example, to understand the geometry, flexibility, regularity, material, and other factors associated with the hole (1132). (See again...) Figure 66AAnother hole (1134) may be wholly or partially coated with a layer (1152), such as a paint or primer layer, which presents another opportunity for characterization. A hole or defect (1136) is also shown in the figure, which may be machined or formed into defined threads (1154), for example, by drilling and tapping processes. A hole or defect (1138) is also shown in the figure, which may be at least partially lined with a layer of corrosion or oxide (1156; for example, iron oxide, or so-called "rust," in the case of iron substrate 1130, and aluminum oxide in the case of aluminum substrate 1130). Variations of the hole or defect (1140) are also shown in the figure, which can combine various complexities, such as threads and oxides (1158). Reference Figure 66B Of course, the geometry of the defect of interest can be completely regular or not completely regular. The figure also shows various geometries, such as basic regular geometries (1132), for example, roughly cylindrical, cubic, or cuboid geometries; narrower versions of basic regular geometries (1142), for example, roughly cylindrical or cuboid geometries; deeper versions of basic regular geometries (1144), for example, roughly cylindrical or cuboid geometries; holes or defects (1146) with a bottom (1160) that is substantially wider than the top (1162); or various complex and / or irregular hole or defect geometries, for example... Figure 66B (Components 1148, 1150) or Figure 66D and 66E (Components 1166 and 1168; Components 1170 and 1172; each component exhibits a relatively elongated defect that extends completely through the substrate 1130 layer) as shown in the hole or defect geometry. Figure 66C The relatively regular geometry of defects (1142) (e.g., defects formed by a drilling machine) may be relatively deep, or may penetrate the entire thickness (1164) of a particular substrate (1130) layer or a portion thereof. All these defects, holes, lumens, and / or partial depressions can be desirably studied and characterized in detail using this technique.

[0159] refer to Figure 67A and 67BIn various embodiments, mounting structures or elongated members (1176), such as shafts, beams, etc., can be used to support tactile sensing components (non-expanded form, element 1178), such as those described above. These components may, for example, have one or more deformable transmission layers configured to engage with other objects or surfaces and provide feedback related to the geometry and other aspects of the engaged surfaces based on electromagnetic transmission (e.g., electromagnetic transmission of radiation of various wavelengths, as described above, such as changes in light from an illumination source (e.g., an LED)). In other words, in various embodiments, sensing surfaces and components (1174) can be formed using, for example, the configuration described above (146), specifically configured to assist in characterizing and analyzing holes and / or defects, such as... Figures 66A-66E The holes and / or defects shown. Reference. Figure 67B The expanded form (1180) of the tactile sensing component can be formed by inflating the contained elastomeric capsule (1182) by injecting pressure (e.g., by injecting a fluid, such as water, saline, air, or an inert gas), as mentioned above with reference to other geometric configurations. The compressed or unexpanded form (1178) can be used for access and delivery, such as introducing or placing a distal portion of the component (1174) into a hole or defect; while the expanded form (1180) can be used to assist in tightly bonding various aspects of the deformable transport layer to the surface of interest for characterization.

[0160] For example, refer to Figures 68A-68D The component (1174) can be inserted into the defect or hole (1184) with its distal portion in a contracted or unexpanded form (1178), and then as Figure 68C and 68D As shown, the component is controlled to expand (1132) to best fit the geometry of the defect or hole (1132) for characterization and analysis. After such analysis, the component (1174) can be withdrawn by re-presenting the unexpanded form.

[0161] As mentioned above, refer to Figure 69A The various aspects of one or more deformable transport layers and the interaction of radiation, such as radiation in various spectral regions of illumination wavelengths, can be used with various types of detectors, such as image acquisition devices (e.g., CCD or CMOS type image acquisition devices, which can be configured with optics to acquire radiation information, which can be used by a coupled computing system to determine geometric information related to the bonding of the deformable transport layer with other surfaces or objects it is bonded to). Figure 69AA variant of an exemplary component (1174) for characterizing holes or defects is shown, having five or more detectors or image acquisition devices (1186) having acquisition fields of view (1188), and each detector or image acquisition device being operatively coupled (1192, e.g., via wired or wireless connections, such as IEEE-802.11 or Bluetooth™ type connections, as mentioned above in reference to various components) to adjacent components, such as power supplies, illumination sources, computing systems, control leads, etc., for example, via a central communication component lead or conduit (1190). The illustrated detectors (1186) are distributed with their respective acquisition fields of view (1188) to cover various overlapping areas of components that may engage with other surfaces, such as holes or defects. The figure also shows operatively coupled (1192) auxiliary sensors (1194), such as ultrasonic transducers, eddy current sensors, magnetic induction sensors, X-ray diffraction sensors, and thermal / infrared detectors, which can be used to further characterize pores or defects (e.g., thermal / infrared detectors can be used to characterize temperature; X-ray diffraction can be used to characterize materials and / or stress relaxation; ultrasonic waves can be used for time-of-flight analysis and / or surface reconstruction; eddy currents and magnetic induction can be used, for example, to characterize the thickness of various coatings or oxide layers relative to the bare substrate metal or other materials).

[0162] refer to Figures 69B-69F Viewed from an orthographic view (i.e., a "top view" or a "cylinder orientation" view along the elongated support member 1176), to provide varying degrees of circumferential coverage, such as 360 degrees around the deformable transmission layer of the sensing assembly (1174) (whether in expanded 1180 or non-expanded 1178 form), one or more sensor assemblies (1186) can be utilized, and the entire assembly (1174; 1178) can be rotated relative to the substrate of interest to acquire more data relating to the substrate portion surrounding the assembly (1174; 1178), for example, through... Figure 69B Alternatively, a configuration similar to 69C can be used; alternatively, the sensor assembly (1186) can be more widely distributed to acquire data around the outside of the sensing assembly (1174; 1178), such as... Figure 69D , 69E Or as shown in the embodiment of 69F (it should be noted that the cross-sectional configuration does not have to be circular; it can be basically square, such as...) Figure 69F (The slices or any other geometry shown).

[0163] refer to Figure 70AThe sensing components (1174; 1178; 1180) can be configured such that: the sensor (1186) includes a detector or image acquisition device, such as a small CMOS or CCD type device (1196), as shown, which is directly deployed at the distal end of the component (1174) and coupled to other components via a connection lead (1192) and / or wireless coupling. Reference Figure 70B Another sensor (1198) configuration is shown, in which a detector and / or image acquisition device (e.g., a CMOS or CCD type device) can be placed at a closer position and optically coupled using one or more optical fibers (1200) to acquire data. The one or more optical fibers can be operatively coupled to a lens (1198), such as a refractive lens, which can be configured to have a specific acquisition field of view relative to the surface of the docking object or substrate. Figure 70C and 70D A configuration is shown in which one or more optical or waveguide transmission configurations (1204; 1206) and one or more reflective devices (1202) can be used to assist in positioning the detector and / or image acquisition device (1196) (e.g., CMOS or CCD type device) at a position closer to the component and / or a more preferred orientation, or encapsulated within the sensing component (1174; 1178; 1180), while still being able to acquire information related to the mating object directly adjacent to the mating position of the sensor (1186). Figure 70E A configuration of an optical guide or waveguide assembly having an optical guide or waveguide assembly operatively coupled to a parabolic reflector structure (1212) configured to assist in acquiring a perimeter field of view or field of view (1210) around the furthest end of an acquisition sensing assembly (1174; 1178; 1180).

[0164] Return to reference Figure 69A It may be desirable to package multiple sensors or couple them closely together to assist in characterizing and / or analyzing structures that are joined nearby. Figure 71A A compact detector or image acquisition device (1196) is shown, whose acquisition field of view or field of view (1188) extends outward; the compact detector or image acquisition device (1196) can be placed adjacent to two other auxiliary sensors (1194). Figure 71B-71D A variation is shown in which one or more portions of the acquisition field of view or field of view of the compact detector or image acquisition device (1196) may be sacrificed (e.g., by creating channels 1214 on one or more portions of device 1196; these channels may affect the field of view or integrity of the acquisition field of view of device 1196) to accommodate a more direct device alignment. Figure 71DA highly integrated component is shown, wherein the main detector or image acquisition device (1196) can be configured to characterize surface interactions with the joint structure or surface using an associated deformable transport layer; other devices (1194) may include, for example, an ultrasonic transducer, an eddy current sensor, a magnetic induction sensor, an X-ray diffraction sensor, and a thermal / infrared detector, as described above.

[0165] Therefore, in various embodiments, it is possible to utilize, for example, in Figure 67A-71D The sensing components (1774) shown in various forms in the diagram, through a process of positioning / orientation, bonding (which may involve expansion, such as by injection / inflation and / or expansion), and data acquisition / analysis, detect defects, holes, cavities, and other geometries (e.g., defined by the substrate structure). Figures 66A-66E The geometry shown in the figure is used for characterization. Regarding the bonding of one or more surfaces or aspects of the target substrate structure, in various embodiments, it is possible that direct bonding of the external portion of the expanded form (1180) of the sensing component (1774) can produce the desired results to assist in characterizing the target substrate structure. In another embodiment, it may be desirable to configure the external geometry to have a predetermined geometry under the expanded form (1180), which can be configured to conform to one or more aspects of the substrate surface, thereby facilitating the identification of variations, deviations, or unexpected geometric problems. For example, if it is known that drilling and tapping are performed during manufacturing to give the hole a 4-inch nominal diameter “coarse thread” of 4 threads per inch, the sensing component (1774) can be fabricated to be easily inserted into position in a non-expanded configuration, and in an expanded geometry (1180), the surface profile of its outer surface will approximate that 4-inch nominal diameter, 4-thread-per-inch geometry; then when the substrate surface engages with the expanded geometry (1180) surface during operation / data acquisition, a deviation signal representing the alignment variation between the sensing component (1774) and the engaged substrate surface is more likely to originate from deviations in the geometric matching of these structures, such as by oxide layers, foreign matter, post-fabrication plastic deformation of the substrate, and / or drilling / tapping / manufacturing errors, and these anomalous deviations can be precisely mapped for further analysis.

[0166] refer to Figure 72A The sensing component described above can be manually (1220) operated in a handheld configuration using a proximal housing or handle (1222) interface including the sensing component (1774), allowing the user (1220) to manually manipulate the sensing component (1774), for example, by yaw, pitch, roll, insert, retract, and rotate (1224) relative to a surface or object of interest (1034). Reference Figure 72BThe sensing component (1774) can be coupled (1226) to another elongated instrument (e.g., a manually manipulated medical catheter), such as a catheter that can be controllably manipulated in one or more axes and / or degrees of freedom using a drawwire or pushwire (or push rod), which can be coupled within the elongated catheter body (1228) and activated by manual manipulation at the proximal handle assembly (1230). Thus, manipulation of the handle assembly (1230) can provide movement of the sensing component (1774), for example, yaw, pitch, roll, insertion, retraction, and rotation (1224) relative to a surface or object of interest (1034). Reference Figure 72C The electromechanical configuration (234) (e.g., a robot) can be coupled, for example, via an interface coupling (1232), which may include one or more load sensors (e.g., piezoelectric sensors for insertion / retraction, yaw, pitch, rotational torque, etc.) such that controlled electromechanical motion (e.g. from automation, user input from a main input device, etc.) can provide motion of the sensing component (1774), such as yaw, pitch, roll, insertion, retraction, and rotation (1224) relative to the surface or object of interest (1034).

[0167] refer to Figure 73A and 73B In various embodiments, the mechanical expander component (1236) can be inserted (1238) into the engagement geometry (1240) of the distal portion of the sensing component (1774), for example... Figure 67A The sensing component shown is used to provide an expansion (1182), as... Figure 73B As shown. In other words, expansion can be achieved through dilation as described above, but it can also be achieved through mechanical expansion; furthermore, expansion can also be achieved through a combination of mechanical expansion and dilation, such as... Figure 73C As shown in the embodiment, expansion (1182) can be achieved by simultaneously inserting an expansion conduit (1242) and an expander member (1236).

[0168] refer to Figures 74A-74C This illustrates various procedures used to characterize aspects such as defects, pores, and lumens. For example... Figure 74A As shown, the substrate (1130) defines elongated defects, holes, or cavities (1172). As... Figure 74A and 74B As shown, a sensing component in its unexpanded form (1178) can be inserted (1244) at the desired location relative to the substrate (1130). Reference Figure 74CTo characterize the various aspects directly surrounding the substrate, the sensing component can be converted to an expanded form (1180) and data can be acquired. In various embodiments, to continue acquiring data related to other portions of elongated defects, holes, or cavities (1172), the sensing component (1174) can be pulled back or pushed forward proximally while acquiring data continuously or discretely. For example, in one variant, it may be desirable to maintain the expanded form (1180) during longitudinal repositioning, in which case it may be advantageous to continue acquiring continuous data, for example, at a relatively high acquisition frequency or "frame rate." In other variants, it may be desirable to return to a non-expanded configuration (1178) before longitudinal repositioning, and then revert to the expanded form (1180) before resuming data acquisition.

[0169] Each configuration described herein allows for the compilation of various aspects of data and image information so that users can view them intuitively in a visual user interface. For example, data related to adjacent acquisition or characterization locations relative to a joint object or surface can be displayed adjacent to each other, and boundaries or intersections between adjacent images and / or data can be connected, merged, or interpolated together, for example, through so-called “stitching” techniques, to present a visual representation of the target surface to the user. The graphical user interface can be configured to display a stitched representation of one or more target objects and can be configured to allow users to browse the representation, similar to how a computer-aided design (“CAD”) interface can be configured to allow users to browse 3D objects of a design.

[0170] refer to Figure 75Users may wish to use the sensing system to engage a target surface that can be a hole, defect, at least a partial depression, tunnel, lumen, or other more complex or simpler features (such as surface roughness, edge sharpness, gaps, offsets, geometric tolerances, etc.); the system can be calibrated and positioned near the target surface (1252). Users can navigate the sensing surface to the target surface, for example, by manually manipulating a slender instrument (e.g., by direct manual manipulation or by manipulating interconnected instruments, such as a manually operable conduit) (1254). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can aid in detecting the target surface and its features (e.g., the system can be configured to first enable the integrated camera and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing associated with a closer engagement) (1256). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (1258). The system can be configured to fit onto a target surface to characterize the surface using a deformable transport layer and store information related to the characterized target surface, such as geometric profile, location, and / or orientation, for example, relative to a global or other coordinate system (1260).

[0171] refer to Figure 76 Users may wish to engage the sensing system with a target surface that may be a hole, defect, at least a partial depression, tunnel, lumen, or other more complex or simpler object; the system can be calibrated and positioned near the target surface (1252). The user navigates the sensing surface to the target surface, for example, via an electromechanical arm or robotic manipulator, and the positioning platform (e.g., inverse kinematics, load cell, deflection sensor, joint position) provides feedback to the user regarding the position and orientation of the sensing surface (1262). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can aid in detecting the target surface and its features (e.g., the system can be configured to first enable integrated cameras and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing related to a closer engagement) (1264). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (1266). The system can be configured to fit onto a target surface to characterize the surface using a deformable transport layer and store information related to the characterized target surface, such as geometric profile, location, and / or orientation, for example, relative to a global or other coordinate system (1268).

[0172] refer to Figure 77Users may wish to use the sensing system to engage with a target surface that may be a hole, defect, at least a partial depression, tunnel, lumen, or other more complex or simpler object; the system can be calibrated and positioned near the target surface (1252). Users can navigate the sensing surface to the target surface, for example, by manually manipulating a slender instrument (e.g., by direct manual manipulation or by manipulating interconnected instruments, such as a manually operable conduit) (1254). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can aid in the detection of the target surface and its features (e.g., the system can be configured to first enable the integrated camera and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing related to a closer engagement) (1270). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., to slow down the repositioning and reorientation of the sensing surface and to convey contact using audio, visual, and / or tactile cues), and the system can be configured to alter the shape or compliance of the sensing surface or associated substrate structure, for example by controlled expansion or contraction of a capsule and / or lumen with a liquid or gas (1272). The system can be configured to adhere to the target surface to characterize the surface using a deformable transport layer and to store information associated with the characterized target surface, such as geometric profile, position, and / or orientation, for example, relative to a global or other coordinate system (1274). The system can be configured to again alter the shape or compliance of the sensing surface or associated substrate structure, for example by controlled expansion or contraction of a capsule and / or lumen with a liquid or gas (1276).

[0173] refer to Figure 78Users may wish to use the sensing system to engage with a target surface that may be a hole, defect, at least a partial depression, tunnel, lumen, or other more complex or simpler object; the system can be calibrated and positioned near the target surface (1252). Users can navigate the sensing surface to the target surface, for example, via an electromechanical arm or robotic manipulator, and the positioning platform (e.g., inverse kinematics, load cell, deflection sensor, joint position) provides feedback to the user regarding the position and orientation of the sensing surface (1262). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can aid in the detection of the target surface and its features (e.g., the system can be configured to first enable integrated cameras and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing related to a closer engagement) (1278). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., to slow down the repositioning and reorientation of the sensing surface and to convey contact using audio, visual, and / or tactile cues), and the system can be configured to alter the shape or compliance of the sensing surface or associated substrate structure, for example by controlled expansion or contraction of a capsule and / or lumen with a liquid or gas (1280). The system can be configured to adhere to the target surface to characterize the surface using a deformable transport layer and to store information associated with the characterized target surface, such as geometric profile, position, and / or orientation, for example, relative to a global or other coordinate system (1282). The system can be configured to again alter the shape or compliance of the sensing surface or associated substrate structure, for example by controlled expansion or contraction of a capsule and / or lumen with a liquid or gas (1284).

[0174] refer to Figure 79Users may wish to engage the sensing system with a target surface that may be a hole, defect, at least a partial depression, tunnel, lumen, or other more complex or simpler object; the system can be calibrated and positioned near the target surface (1252). Users can navigate the sensing surface to the target surface, for example, by manually manipulating a slender instrument (e.g., by direct manual manipulation or by manipulating interconnected instruments, such as a manually operable conduit) (1254). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can aid in detecting the target surface and its features (e.g., the system can be configured to first enable the integrated camera and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing associated with a closer engagement) (1286). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (1288). The system can be configured to fit onto a target surface to characterize the surface using a deformable transport layer and store information related to the characterized target surface, such as geometric profile, location, and / or orientation, for example, relative to a global or other coordinate system (1290).

[0175] refer to Figure 80 Users may wish to engage the sensing system with a target surface that may be a hole, defect, at least a partial depression, tunnel, lumen, or other more complex or simpler object; the system can be calibrated and positioned near the target surface (1252). Users can navigate the sensing surface to the target surface, for example, via an electromechanical arm, which may include an actively driven robotic arm, a manually positioned articulated arm with electromechanical brakes, a manually positioned articulated arm without electromechanical brakes, and / or a tethered or untethered configuration for manual holding and orientation (1262). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can aid in detecting the target surface and its features (e.g., the system can be configured to first enable integrated cameras and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing related to closer engagement) (1292). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (1294). The system can be configured to fit onto a target surface to characterize the surface using a deformable transport layer and store information related to the characterized target surface, such as geometric profile, location, and / or orientation, for example, relative to a global or other coordinate system (1296).

[0176] refer to Figure 81Users may wish to engage the sensing system with a target surface that may be a hole, defect, at least a partial depression, tunnel, lumen, or other more complex or simpler object; the system can be calibrated and positioned near the target surface (1252). Users can navigate the sensing surface to the target surface, for example, by manually manipulating a slender instrument (e.g., by direct manual manipulation or by manipulating interconnected instruments, such as a manually operable conduit) (1254). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can aid in detecting the target surface and its features (e.g., the system can be configured to first enable the integrated camera and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing associated with a closer engagement) (1302). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (1304). The system can be configured to adhere to a target surface to characterize the surface using a deformable transport layer and store information related to the characterized target surface, such as geometric profiles, location, and / or orientation, for example, relative to a global or other coordinate system (1306). The system can be configured to register the locations of known points on the surface with portions of a known model, thereby registering the system (i.e., determining the known positional / orientation relationship between the model and the measured surface); registration can be automated, for example, through automatic registration based on a series of points or surfaces acquired during the measurement process, for example, with the aid of a neural network trained using data associated with the known model (1308). The system can be configured to determine differences between measured dimensions, surface orientation, etc., for quality assurance and / or inspection purposes (1310).

[0177] refer to Figure 82Users may wish to engage the sensing system with a target surface that may be a hole, defect, at least a partial depression, tunnel, lumen, or other more complex or simpler object; the system can be calibrated and positioned near the target surface (1252). Users can navigate the sensing surface to the target surface, for example, via an electromechanical arm, which may include an actively driven robotic arm, a manually positioned articulated arm with electromechanical brakes, a manually positioned articulated arm without electromechanical brakes, and / or a tethered or untethered configuration for manual holding and orientation (1262). As the sensing surface is navigated closer to the target surface, integrated sensing capabilities can aid in detecting the target surface and its features (e.g., the system can be configured to first enable integrated cameras and lidar to detect the target surface, followed by other integrated sensing capabilities that can be configured for sensing related to closer engagement) (1312). The system can be configured to specifically trigger contact events between the sensing surface and the target surface (e.g., the repositioning and reorientation of the sensing surface can be slowed down, and contact can be communicated using audio, visual, and / or tactile cues) (1314). The system can be configured to adhere to a target surface to characterize the surface using a deformable transport layer and store information related to the characterized target surface, such as geometric profiles, location, and / or orientation, for example, relative to a global or other coordinate system (1316). The system can be configured to register the locations of known points on the surface with portions of a known model, thereby registering the system (i.e., determining the known positional / orientation relationship between the model and the measured surface); registration can be automated, for example, through automatic registration based on a series of points or surfaces acquired during the measurement process, for example, with the aid of a neural network trained using data associated with the known model (1318). The system can be configured to determine differences between measured dimensions, surface orientation, etc., for quality assurance and / or inspection purposes (1320).

[0178] refer to Figure 83AA touch sensing component (1402) having a deformable transmission layer (1400) is coupled to a robotic arm (1404) to enable it to be electromechanically positioned and oriented relative to other objects in the nearby environment, such as a target surface (1418) which can be mounted on a mounting structure (1416) which can be placed on the ground (1414), for example, on the same ground (1414) as the robotic arm system base (1412). The robotic arm (1404) and / or associated base (1412) can be operatively coupled to the computing system (1406) (1410, for example, via a wired or wireless connection, such as by using the IEEE 802.11 standard); similarly, the touch sensing component (1402) can be operatively coupled to the computing system (1406) (1408, for example, via a wired or wireless connection, such as by using the IEEE 802.11 standard) so that various aspects of contact on the deformable transport layer (1400) can be detected and analyzed, for example, as described in the aforementioned document.

[0179] refer to Figure 83B In another variant, additional integrated sensing capabilities can be incorporated into the system, such as pairing a wireless positioning sensor (1430) with a transceiver (1428), for example, based on magnetic flux acquisition capabilities (e.g., magnetic tracking systems provided by vendors such as Polhemus), or based on optical tracking capabilities (e.g., systems provided by Northern Digital), or based on GPS-style signal triangulation capabilities, to enable the position and / or orientation of the sensor (1430) and its associated deformable transport layer (1400) to be determined with high precision in real-time or near real-time. The position and / or orientation of the deformable transport layer (1400) can also be determined by the inverse kinematics of the robotic system (e.g., by determining joint positions). In addition to data from the deformable transport layer (1400) and the position / or orientation tracking subsystem (1428, 1430), the computing system can also be configured to receive (1424, 1426, e.g. via wired or wireless connections, e.g., by using the IEEE 802.11 standard) additional data from one or more additional image acquisition detectors (1450, 1451, 1452) and / or one or more additional sensors (e.g., compact lidar sensors (1440, 1441, 1442)) to provide additional information relating to the target surface (1418), as well as the deformable transport layer (1400) and associated structures, which can be coupled to various structures, such as a robotic arm (1404) and / or a wall (1420) or a mounting structure (1422) coupled thereto.

[0180] refer to Figure 84A and Figure 84B A gantry structure can be used above the sample stage coupled to the target surface (1418) to assist in acquiring data related to the target surface (1418). Figure 84A and 84B In the view shown, two end structures (1462, 1463) can be used to assist in providing Y-axis motion (1482), for example via an electric lead screw, while an electric Z-axis actuator (1470) can be configured to provide Z-axis range motion (1484), and X-axis range motion (1480) can be provided along the horizontal member (1460) via, for example, a lead screw or belt drive configuration. Such a configuration enables controllable X, Y, and Z-axis motion of the deformable transmission layer (1400) so that it can dock with the target surface (1418).

[0181] refer to Figure 85 , can utilize such Figures 83A-84B The configuration shown generates one or more surface profiles. For example... Figure 85 As shown, the deformable transport layer can be coupled to a positioning system and operatively coupled to an associated tactile sensing computing system and components (1490). A target surface can be positioned within the reach of the deformable transport layer positioning system (1492). The positioning system can be used for repositioning and / or reorientation to acquire data related to the target surface, as well as relevant data related to its position on the target surface, the position and / or orientation of the deformable transport layer, and / or the loading configuration at the time of acquisition (1494). The positioning system can be used to sequentially acquire data related to various aspects, positions, areas, and / or regions of the target surface, as well as relevant data related to each acquisition (e.g., position on the target surface, the position and / or orientation of the deformable transport layer, and / or the loading configuration at the time of acquisition) (1496). A profile related to the target surface can be created (1498). The target surface profile can be compared with various aspects of another surface profile (e.g., a known and / or predetermined surface profile), and differences can be highlighted and / or analyzed (1500). Such systems can be configured to analyze, for example, artworks such as paintings or sculptures, and compare them with known originals or authenticate known originals. They can also analyze the surfaces of coins of equivalent value, machine surfaces, textiles, various documents, and printed materials. This system can be used for fraud detection and / or quality control, and / or to match (or perform relative analysis / authentication) digital representations (such as so-called "digital twins") with physical representations.

[0182] refer to Figures 86 to 93 An embodiment is shown, wherein, reference is made to Figure 59AThe configuration described in 59B, for example, can be used to enhance measurement and / or verification system environments, such as by utilizing a system called a coordinate measuring machine or "CMM" system to address challenges such as so-called geometric dimensional and tolerance (or "GD&T") analysis, which may involve specific target geometry, flatness relative to a plane, roundness of holes, etc. (See reference) Figure 86 For example, haptic-enhanced CMM system configuration (e.g.) Figure 59B The system configuration shown can be powered on, started, and prepared to achieve the goal of characterizing the geometric aspects of a target object (1502). The target object (e.g., Figure 59B The engine block object (1126) shown is placed within the system's analysis area (i.e., within the system's range of motion or contact; and / or within the range of other operatively coupled sensors, such as lidar, image acquisition, or other devices that can be configured to provide non-contact data related to the target object), for example, on an object measurement stage, preferably in a physically stable configuration (1504). A digital model of the target object (e.g., Figure 59B (As shown in element 1052) is loaded into the associated computing system (e.g., Figure 59B (as shown in element 144) or to make the digital model of the target object available to the associated computing system for registration and / or comparison purposes (1506). One or more operatively coupled deformable transport layers (e.g., integrated into) can be used. Figure 59B Discrete contact analysis is performed on the target object using the deformable transmission layer of the sensing component 146 as described above and / or one or more protruding probes (e.g., protruding elongated probes in a conventional CMM system) (1508). Registration with the digital model can be performed based on a comparison between various aspects of the digital model and the results of the discrete contact analysis (e.g., automated / computerized comparison) (1510). Registration navigation can be used to advance the sensing component (146) using one or more deformable transmission layers (e.g., as described above, by manual or electromechanical operation) to a position of physical engagement with the target object for tactile enhancement measurement and confirmation related to the actual geometry of the target object (1514). The tactile enhancement CMM system can be configured to highlight the changes between the digital model and the actual geometry, for example, through a 3D graphical user interface (1516).

[0183] refer to Figure 87The haptic-enhanced CMM system can be powered on, started, and ready to achieve the goal of characterizing the geometric and / or structural aspects of a target object (1518). The target object can be placed within the system's analysis area (i.e., within the system's range of motion or contact; and / or within the range of other operatively coupled sensors, such as lidar, image acquisition, or other devices that can be configured to provide non-contact data related to the target object), for example, placed on an object measurement stage, preferably in a physically stable configuration (1520). A digital model of the target object can be loaded into an associated computing system for registration and / or comparison purposes; the digital model may include structural characterization aspects (1522). Discrete contact analysis of the target object can be performed using one or more operatively coupled deformable transport layers and / or protruding probes (1524). Registration with the digital model can be performed based on a comparison between various aspects of the digital model and the discrete contact analysis (1526). Registration navigation can be used to advance one or more deformable transport layers to a location where they physically engage with the target object, enabling haptic-enhanced measurements and verifications relating to the actual geometry and structural properties of the target object (e.g., local structural modulus at a specific point) (1528). Haptic-enhanced CMM systems can be configured to highlight variations between the digital model and the actual geometry and structural properties, for example, via a 3D graphical user interface (1530).

[0184] refer to Figure 88The haptic-enhanced CMM system can be powered on, started, and ready to achieve the goal of characterizing the geometric aspects of a target object (1532). The target object can be placed within the system's analysis area (i.e., within the system's range of motion or contact; and / or within the range of other operatively coupled sensors, such as lidar, image acquisition, or other devices that can be configured to provide non-contact data related to the target object), for example, placed on an object measurement stage, preferably in a physically stable configuration (1534). A digital model of the target object can be loaded into an associated computing system or made available to a computing system associated with its digital model for registration and / or comparison purposes (1536). Non-contact analysis of the target object can be performed using operatively coupled sensors (e.g., lidar and / or image acquisition) (1538). Registration with the digital model can be performed based on a comparison between various aspects of the digital model and the non-contact analysis (1540). Registration navigation can be used to advance one or more deformable transport layers to a position physically engaged with the target object for haptic-enhanced measurements and verification related to the actual geometry of the target object (1542). Haptic-enhanced CMM systems can be configured to highlight the differences between the digital model and the actual geometry, for example, through a 3D graphical user interface (1544).

[0185] refer to Figure 89The haptic-enhanced CMM system can be powered on, started, and ready to achieve the goal of characterizing the geometric and / or structural aspects of a target object (1546). The target object can be placed within the system's analysis area (i.e., within the system's range of motion or contact; and / or within the range of other operatively coupled sensors, such as lidar, image acquisition, or other devices that can be configured to provide non-contact data related to the target object), for example, placed on an object measurement stage, preferably in a physically stable configuration (1548). A digital model of the target object can be loaded into or made available to an associated computing system for registration and / or comparison purposes; the digital model may include structural characterization aspects (1550). Non-contact analysis of the target object can be performed using operatively coupled sensors (e.g., lidar and / or image acquisition) (1552). Registration with the digital model can be performed based on a comparison between various aspects of the digital model and the non-contact analysis (1554). Registration navigation can be used to advance one or more deformable transport layers to a location where they physically engage with the target object, enabling haptic-enhanced measurements and verifications relating to the actual geometry and structural properties of the target object (e.g., local structural modulus at specific points) (1556). Haptic-enhanced CMM systems can be configured to highlight differences between the digital model and the actual geometry and structural properties, for example, via a 3D graphical user interface (1558).

[0186] Reference Figure 90 The haptic-enhanced CMM system can be powered on, started, and ready to achieve the goal of characterizing the geometric and / or structural aspects of a target object (1560). The user can load a digital model of the target object into the system or otherwise make the digital model of the target object available to the system (1562). The user can load an actual sample of the target object into the analysis area (e.g., on a measuring stage) (1564). Initial contact-based sample characterization can be performed to provide points for registering the digital model with the target object sample (1566). The digital model can be “snapped” onto or registered with the target object sample, for example, automatically by using interconnected computing systems (1568). Various surfaces of the target object sample can be analyzed using one or more deformable transport layers of the haptic-enhanced CMM system (e.g., by manual or electromechanical repositioning and / or reorientation) (1570). Haptic-enhanced CMM systems can be configured to highlight the differences between the digital model and the actual geometry, for example, through a 3D graphical user interface (1572).

[0187] refer to Figure 91The haptic-enhanced CMM system can be powered on, started, and ready to achieve the goal of characterizing the geometric and / or structural aspects of a target object (1574). The user can load a digital model of the target object into an interconnected computing system or otherwise make the digital model of the target object available to an interconnected computing system (1576). The user can load an actual sample of the target object into the analysis area (e.g., a measuring stage) (1578). Initial sample characterization based on contact can be performed to provide points for registering the digital model with the target object sample (1580). The digital model can be “attached” to or registered with the target object sample, for example, automatically “attached” or registered via an interconnected computing system (1582). Various surface and / or structural performance aspects of the target object sample can be analyzed using one or more deformable transport layers of the haptic-enhanced CMM system (e.g., by manual or electromechanical repositioning and / or reorientation) (1584). Haptic-enhanced CMM systems can be configured to highlight the differences between the digital model and the actual geometry and structural properties, for example, through a 3D graphical user interface (1586).

[0188] refer to Figure 92 The haptic-enhanced CMM system can be powered on, started, and ready to achieve the goal of characterizing the geometric and / or structural aspects of a target object (1588). Users can load a digital model of the target object into an interconnected computing system or otherwise make the digital model of the target object available to an interconnected computing system (1590). Users can load actual target object samples into the analysis area (e.g., a measurement stage) (1592). Initial vision-based sample characterization can be performed, for example, automatically via an interconnected computing system, to facilitate registration of the digital model with the target object sample (1594). The digital model can be “snap-on” to or registered with the target object sample, for example, automatically “snap-on” or registered via an interconnected computing system (1596). Various surfaces of the target object sample can be analyzed using one or more deformable transport layers of the haptic-enhanced CMM system (e.g., by manual or electromechanical repositioning and / or reorientation) (1598). Haptic-enhanced CMM systems can be configured to highlight the differences between the digital model and the actual geometry, for example, through a 3D graphical user interface (1600).

[0189] refer to Figure 93The haptic-enhanced CMM system can be powered on, started, and ready to achieve the goal of characterizing the geometric and / or structural aspects of a target object (1602). Users can load a digital model of the target object into an interconnected computing system or otherwise make the digital model of the target object available to interconnected computing systems (1604). Users can load actual target object samples into the analysis area (e.g., a measuring stage) (1606). Initial vision-based sample characterization can be performed (e.g., automatically using interconnected computing resources) to facilitate registration of the digital model with the target object sample (1608). The digital model can be “attached” to or registered with the target object sample, for example, automatically “attached” or registered using interconnected computing resources (1610). Various surfaces of the target object sample can be analyzed using one or more deformable transport layers of the haptic-enhanced CMM system (e.g., by manual or electromechanical repositioning and / or reorientation) (1612). Haptic-enhanced CMM systems can be configured to highlight the differences between the digital model and the actual geometry, for example, through a 3D graphical user interface (1614).

[0190] refer to Figure 94 A haptic-enhanced CMM system can be powered on, started, and ready to achieve the goal of characterizing the geometric and / or structural aspects of a target object (1620). Various surface and / or structural performance aspects of a target object sample can be analyzed (e.g., automatically via interconnected computing resources) using one or more deformable transport layers of the haptic-enhanced CMM system (e.g., by manual or electromechanical repositioning and / or reorientation), while simultaneously tracking the position and orientation associated with the engagement of one or more deformable transport layers and the target object (e.g., in an environment-dependent global coordinate system) (1622). Acquired and / or determined information associated with one or more deformable transport layers and / or the target object can be integrated and / or displayed, for example, in a 3D representation (1624). Multiple positions and / or orientations can be displayed simultaneously or sequentially, for example, by a “stitched” representation of information acquired and / or determined within a global coordinate system (1626). The system can be configured to overlay numerical, calculated and / or determined data (e.g., deflection, load, strain, temperature, measurement and / or deformation) related to a particular geometry, location and / or orientation onto an associated display representation (1628).

[0191] refer to Figure 95A An electromechanical and / or robotic system (234) can be used to advance multiple (here, a pair of oppositely arranged 146, 147) touch sensing components (146, 147) toward a target object (1127; here, a knurled cylindrical object), which can be mounted on a detection or measurement stage (262). Reference Figure 95B Each sensing component (147, 146, refer to respectively) Figure 95A The model (1640, 1642; each shown in shaded mode) and the loads and / or contact profiles (1644, 1648) associated with each contact interface are displayed together; this display may include shading, color, and other indications of load-related quantities, and may be dynamically updated in real-time or near real-time based on the loads at these interfaces. Reference Figure 95C With the target object / sample (e.g. Figure 95A Element 1127) continuously repositions and / or reorients relative to sensing components (146, 147), while tracking the position and / or orientation of these objects in space relative to each other (i.e., relative to the global coordinate system of the room), which can form and display the target object / sample (e.g., Figure 95A The surface model associated with element 1127 in the model can depict not only geometric information but also other perceptual aspects, such as temperature, local structure or Young's modulus and associated strain characteristics (i.e., determined by sample loading), texture, impedance and other factors, depending on the combination of sensor functions applied.

[0192] refer to Figure 96 The tactile-enhanced mechanical device can be configured to have one or more deformable transport layers oriented outwards (1630). The device can be moved toward the environment or object of interest, for example, by manual or electromechanical navigation (1632). The movement of the device relative to the target environment or object of interest can be monitored by an operatively coupled 3D tracking configuration and / or a vision- and / or time-of-flight sensing configuration (1634). Contact between the device and one or more objects and / or surfaces can be monitored by one or more deformable transport layers and / or interconnected load sensing capabilities (e.g., by using inverse kinematics, load sensors, deflection sensors, or by using the fusion of two or more such devices, preferably with uncorrelated error profiles) (1636). An interconnected computing system can be configured to process and integrate acquired and determined information to create and update maps of the environment and / or object of interest in a tactile-enhanced SLAM (simultaneous localization and mapping) configuration, wherein the output can be displayed, for example, by displaying a 3D map on a monitor (1638).

[0193] refer to Figure 97 For example, refer to the above text. Figure 59B The description indicates that measurement probes (1118) can be interconnected and utilized, for example... Figure 97 As shown by element 1652. In related embodiments, a measurement probe or similar function may be integrated within the deformable transmission layer (1400) of the sensing components (146, 1402), such as Figure 97As shown in element 1654, it provides at least one input point from a discrete interface, which may be substantially rigid, for example, relative to the compliance of the surrounding and / or integrated deformable transport layer (1400).

[0194] refer to Figure 98A and Figure 99B Various embodiments have been discussed in this document, such as those described in reference to Figure 7A and Figure 7D In the embodiments described, the illumination (116, 117) can be directed into the optical element (108) and the deformable transmission layer (110) so that it can be at least partially detected by an image capture device or detector (106) upon return. As previously described, the illumination can be directed into the optical element (108) and the deformable transmission layer (110) in a variety of ways, such as directly via a local transmitter (116) (e.g., an LED), or semi-directly (117), such as via optical fiber from a more remotely located transmitter (e.g., an LED). Reference Figure 98C Through reference Figure 70D The described configuration allows illumination to be reciprocated between the transmitter and the image acquisition / detection device (1196) using an optical or waveguide transmission element (1206), which may be associated with one or more reflective elements (1202) for redirection, thereby forming a target acquisition area (1186), as described above.

[0195] refer to Figure 99A With this configuration, illumination can be input into discrete regions (116, 117; 122, 123), for example, into an optical element (108), and then guided into the deformable transport layer (110), and the interaction between the illumination and the deformable transport layer (110) can be analyzed as described above using an imaging device or detector (106). Reference Figure 105A and 105B The composite images (1741, 1745) and the leftmost sample images (1740, 1744) demonstrate that configurations with this type of illumination can provide detailed and valuable results.

[0196] refer to Figure 99B and 99C In various embodiments, it may be desirable to provide illumination emission from locations and orientations selected for improved distribution and analysis / detection. For example, refer to Figure 99B An improved illumination distribution can be provided by utilizing three discrete illumination positions and orientations (116, 117); Reference Figure 99C A relatively large irradiation distribution configuration (116, 117) can be configured to provide a relatively large area distribution irradiation directly across the associated optical element (108) and deformable transmission layer (110). Figure 100A-104GSeveral configurations are shown, in which one or more lighting control layers can be integrated to achieve similar results. Figure 99B and 99C The method shown provides improved illumination distribution and other illumination distribution paradigms, with high configurability depending on the components of one or more lighting control layers.

[0197] refer to Figure 100A A variant of the digital touch sensing component (146) is shown, wherein a light source (116, 117) (e.g., an LED) is operatively coupled to an illumination control layer (1704) using a photo-shaping optics element (1702). The illumination control layer (1704) may include one or more illumination guiding features configured to redirect light from the light source (116, 117) toward the optics element (108) and the deformable transport layer (110) and back to an image acquisition device or detector (106) for tactile analysis, as shown in the sample paths (1706, 1708). The one or more illumination guiding features may include reflection and / or redirection features that may be encoded in the illumination control layer (1704; for example, a series of lenses, material coating patterns, diffraction patterns, and / or reflective elements may be formed in the material comprising the illumination control layer) to provide a desired outgoing illumination distribution (110) from the illumination control layer to the associated optics element (108) and the deformable transport layer (110). As previously mentioned, such a distribution can be configured to be relatively simplified, for example, with two, three, or four discrete emission zones projecting downwards into the associated optical element (108) and deformable transmission layer (110); alternatively, depending on the reflection and / or redirection characteristics that can be encoded in the illumination control layer (1704), such a distribution can be configured to be more dispersed, for example, through more discrete emission zones, or a fairly uniform, fully dispersed emission paradigm across the entire illumination control layer. (See previous reference) Figure 105A and 105B The composite images (1741, 1745) and the rightmost images (1742, 1746) show the configurations related to the tactile analysis of the coin (1742) and banknote (1746), in which the illumination distribution has been improved by using an illumination control layer (1704; here employing a thermoplastic elastomer gel deformable transport layer 110 and polymethyl methacrylate illumination control layer material) compared to the more discrete illumination described above in the leftmost images (1740, 1744).

[0198] refer to Figure 100B Illumination can be introduced into the lighting control layer (1704) from multiple points; here, light can be introduced through the photo-shaping optics (1702) using two light sources (116 / 117; 122 / 123) at two different locations. Reference Figure 100C and 100DMulti-layer lighting control layers (1704, 1714, 1716) can be used to introduce and direct illumination radiation into different layers and / or forms. For example, one layer can be used to introduce radiation of one wavelength (e.g., a selected color versus a second selected color; or infrared spectrum versus visible spectrum), while another layer is used to introduce radiation of another wavelength, each wavelength traveling through slightly different paths (…). Figure 100C In the embodiments, 1706 / 1708; and 1710 / 1712) enter the deformable transport layer (110) and return to the imaging device / detector (106).

[0199] refer to Figure 100E In various embodiments, it may be desirable to position the illumination control layer (1704) between the optical element layer (108) and the deformable transport layer (110), which creates an optical path (1707 / 1709) to the imaging device / detector (106) as shown, to simplify the structure and / or allow illumination extraction to be closer to the tactile surface of the deformable transport layer (110). Such a configuration can also minimize any “dead zones” or “shadow areas” that may appear in the periphery of the deformable transport layer (110) due to the associated optical path.

[0200] refer to Figure 101A The diagram shows an illumination control layer (1704) and operatively coupled light sources (116, 117) and photo-shaping optics (1702), but other typically integrated components are not shown, such as... Figure 101B As shown in the relevant orthogonal views, the lighting control layer (1704) and its features can be used to distribute illumination across a relatively large area. The lighting control layer can have various shapes (e.g., Figure 101B The circle shown, or Figure 102E and 102F (The rectangle / square shown). Figure 102A and 102B A similar orthogonal view of the configuration of the lighting control layer (1704) with two illumination input sources (116 / 117; 122 / 123) is shown. Figure 102C A similar orthogonal view is shown of a configuration of an illumination control layer (1704) with four illumination input sources (116 / 117; 122 / 123; 1720; 1722). Figure 102D A similar orthogonal view is shown of a configuration of an illumination control layer (1704) with three illumination input sources (116 / 117; 122 / 123; 1720). Figure 102E A similar orthogonal view is shown of a lighting control layer (1704) configuration with two illumination input sources (116 / 117; 122 / 123) and a rectangular and / or square orthogonal geometry as described above for illustrative purposes. Reference Figure 102FFurthermore, one or more irradiation sources (116 / 117; coupled via photoshaping optics 1702 as shown) or groups of irradiation sources (1724, 1726) can be used to improve irradiation and / or its distribution. Additionally, as... Figure 103A-103C As shown, it should be specifically noted that although the lighting control layer (1704) can be configured as planar or substantially planar (as in...), Figure 100A-102F In the illustrated embodiment), it can also be curved (as in...). Figure 103A and 103B In the described embodiments, convex, concave, saddle-shaped, or virtually arbitrary geometric shapes (e.g., Figure 103C (The semi-convex shape of the embodiment).

[0201] refer to Figure 104A-104G , can utilize such Figure 100A-103C The configuration shown allows for the creation of various fingertip or fingertip tactile sensing components. For example... Figure 104A As shown, the deformable transport layer (110) can be geometrically selected to provide, for example, Figure 104D The elongated finger-like component shown (1736; such components may include part of a synthetic hand or robotic hand, for example, Figure 104E The external sensing of the synthetic hand or robotic hand (1738) shown is integrated within a modular housing structure (1758) as illustrated, along with optical elements (108), an illumination control layer (1704), an optical prism (1736), a lens (1750), and an image acquisition device / detector (106). Its optical path (1752) provides an arrangement functionally equivalent to the position / orientation of a "virtual camera" relative to a deformable transmission layer (110; shown with a convex shape to accommodate use in a fingertip or fingertip configuration), as illustrated in (1738). Reference Figure 104B As mentioned above Figure 100E To reduce the "dead zone" or "shadow zone" in the tactile sensing capability relative to the deformable transmission layer (110), and to simplify the structure and / or make the illumination extraction closer to the tactile surface of the deformable transmission layer (110), the illumination control layer (1704) may be positioned relative to the deformable transmission layer (110) and the optical element (108) as shown, wherein different paths (1754) pass through these elements sequentially. Reference Figure 104C Alternatively, various embodiments as described above (e.g.) can be used. Figure 7A The embodiment shown constructs a fingertip sensor by lateral injection illumination (116, 117) into the optical elements without using an illumination control layer for distribution, wherein the optical path (1756) as shown passes through each element in sequence. Figure 104F and 104G It shows something similar to Figure 104A Computer-aided design drawings related to the embodiments ( Figure 104F External orthogonal view; Figure 104G (This is a cross-sectional orthogonal view), which shows the integration of the deformable transmission layer (110) with the illumination control layer (1704), optical elements (108), optical prisms (1736), lenses (1750), image acquisition devices / detectors (106), and housing (1758).

[0202] Reference Figure 106A-112 Various embodiments may include ultrasonic emission / detection modules; these modules may be configured to form an ultrasonic imaging subsystem, such as those subsystems comprising one or more piezoelectric or other ultrasonic emission sources and one or more detection receivers. For example, such subsystems may be used in fault analysis, medical, manufacturing, and other environments, and may be supplied by manufacturers such as General Electric Corp., Siemens AG, or Royal Philips NV. See reference. Figure 106A Various embodiments have been discussed in this document (e.g., see reference 1). Figure 7A and Figure 7D Those described herein include variations of a touch sensing component (146), wherein illumination (116, 117) can be directed into an optical element (108) and a deformable transmission layer (110) for at least partial detection upon return using an image acquisition device or detector (106). As described above, illumination can be directed into the optical element (108) and the deformable transmission layer (110) in various ways, such as directly via a local transmitter (116) (e.g., an LED), or via a semi-direct method (117), such as transmission from a remotely located transmitter (e.g., an LED) via optical fiber. For illustrative purposes, Figure 106B-106J In this simplified subset, a computing system (104) coupled (136) to a power source (102) is included; this computing system can typically be coupled (1802; e.g., via wired or wireless coupling, fiber optics, etc.) to sensing components (as described in the various embodiments above) housed within a housing (118) to provide control, power, illumination, signal and / or data exchange, etc. Therefore, Figure 106B As shown above Figure 106A The configuration shown is a simplified version.

[0203] refer to Figure 106C It shows the relationship with Figure 106B The sensing component is similar to the sensing component (2000), the difference being that... Figure 106BThe components also include an integrated ultrasonic transmitting / detecting module (1804) and an effective transmission medium (1810; such as a liquid, gel, or other material configured to transmit ultrasonic energy relatively efficiently without full functional damping or signal loss), which can be positioned in the gas or air gap originally located between the ultrasonic transmitting / detecting module (1804) and the target structure (1034). Figure 106C As shown, the emitted ultrasonic energy can be guided away from the ultrasonic emission / detection module (1804), passing through the imaging device layer (106), the transmission medium layer (1810), the optical element layer (108), the deformable transmission layer (110), and (if included) the film layer (100) to be reflected from at least a portion of the target structure (1034). The ultrasonic energy / radiation / wave emitted from the ultrasonic emission source can be used not only to be reflected from the direct surface of the target structure (1034) back to the ultrasonic emission / detection module (1804), but also to be reflected from deep within the target structure (1034), as is well known in other practical applications (e.g., medical or industrial applications) where ultrasound is used to “image” various structures with varying densities, depths, and resonant variability. Figure 106CThe configuration shown provides an ultrasonic “imaging” modality that is substantially coaxially aligned with the touch sensing capabilities provided by integrated touch sensing modules (106, 108, 110, 100), which, as described above, are configured to provide information relating to the contact surface (e.g., the surface of the same target structure (1034)). These two integrated detection modalities (i.e., ultrasonic sensing and touch detection using illumination (e.g., light)) provide an opportunity for “fusion” analysis or fusion configurations (as described above), where the two modalities provide information relating to the same target in modes with at least partially uncorrelated errors. For example, in various embodiments, an ultrasonic emission source can be used in conjunction with an ultrasonic detection module, both operatively coupled to a computing system to collect, analyze, and provide information relating to the interaction of emission guided from the ultrasonic emission source with the deformable transport layer, which is associated with the relative positions of the portions of the deformable transport layer; in other words, ultrasonic transduction and signal processing can be used to collect information relating to the positioning of the portions of the mutually coupled deformable transport layer, such as time-of-flight and reflectivity data used in ultrasonic analysis, which can also be used in conjunction with the irradiation interaction configuration described herein to provide fused data relating to the dynamic behavior of the deformable transport layer when engaging with a docking object (i.e., how it moves, deforms, and its general mechanical behavior, including at and beneath specific surfaces or layers of the target material). In various embodiments, ultrasonic emission from one or more ultrasonic emission sources and the reflection of such emission can travel through the layers with greater effectiveness and / or efficiency than irradiation involved in an integrated tactile configuration, thus enabling [further benefits]. Figure 106C In the illustrated embodiment, the layers are arranged to have a longer path for ultrasound compared to contact sensing illumination / reflection (e.g., path 1820 to the target and back). This arrangement is not always preferred, and... Figure 106D , 106E In the embodiments shown in 106F, 106G, 106H, and 106I, the path configurations (1822, 1824, 1826, 1828, 1830, and 1832 / 1833, respectively) are different. In some embodiments, they include transmitters / detectors (1804) at the same level, while in others, they include separate ultrasonic transmitters or emission sources (1808) and detectors or detection modules (1806) structures. Figure 106D , 106EAs shown in 106F, 106G, 106H and 106I, the ultrasonic emission source can be directly integrated or coupled to the deformable transmission layer (110), or indirectly integrated or coupled, so that it remains operatively coupled but not directly and tightly docked to the deformable transmission layer (110); similarly, the ultrasonic detection module (1806) can be directly integrated or coupled to the deformable transmission layer (110), or indirectly integrated or coupled, so that it remains operatively coupled but not directly and tightly docked to the deformable transmission layer (110). Figure 106I The illustrated embodiment shows a fusion sensing configuration in which a slightly non-coaxial (i.e., not directly stacked with optical element 108, imaging device 106, and deformable transmission layer 110) transmitter or source structure (1808) can be configured to emit toward a target structure (1034) such that reflection paths (1832, 1833) return through the components to the detection module (1806) as shown. Furthermore, to facilitate transmission in regions along these paths—specifically, regions where gaps might otherwise exist (e.g., air or gas gaps that could impede effective ultrasound transmission)—transmission materials (1812, such as gels, fluids, or other effective transmission materials) can be maintained and / or inserted as shown. It is noteworthy that maintaining such a transmission layer can often be challenging in various environments; therefore, embodiments configured to eliminate the need for such components may be desirable. For example, in… Figure 106J In the illustrated embodiment, utilizing only ultrasound (i.e., without involving light or other radiation-based contact sensing as described above), one can benefit from the flexibility and transmissivity of the deformable transmission layer (110) shown, enabling the use of ultrasound without the need for... Figure 106I With an effective transport material layer maintained as shown, ultrasonic analysis of the target structure (1034) can be performed, while ensuring sufficient contact between the target structure (1034) and the deformable transport layer / membrane (110 / 100) at the path of interest (1826).

[0204] refer to Figure 107In a fusion sensing configuration, the deformable transport layer can be operatively coupled to an ultrasonic emission / detection system (1840). A target object can be located within the reach of the fusion sensing configuration (1842). The fusion sensing configuration can be positioned and oriented to acquire data related to the target object based on ultrasonic reflections from the target object and / or contact between the target object and the deformable transport layer (1844). Data acquired from the fusion sensing configuration can be based on ultrasonic data and contact data, which have error patterns that are at least partially uncorrelated with each other (1846). The captured data can be analyzed and / or displayed to a user to help characterize various aspects of the target object relative to the fusion sensing configuration (1848). The fusion sensing configuration can be repositioned and / or reoriented relative to the target object to acquire additional data (1850).

[0205] refer to Figure 108 In a fusion sensing configuration, a deformable transport layer can be operatively coupled to an ultrasonic emission / detection system (1840). A target object can be located within the reach of the fusion sensing configuration (1842). The fusion sensing configuration can be positioned and oriented to acquire data related to the target object based on ultrasonic reflections from the target object prior to contact between the target object and the deformable transport layer (1852). This ultrasonic data can be analyzed and / or displayed to a user to help preliminarily characterize various aspects of the target object relative to the fusion sensing configuration (1854). The target object can be located within the contact reach of the fusion sensing configuration (1856). Data acquired from the fusion sensing configuration can be based on ultraso...

Claims

1. A system for characterizing geometric surfaces, comprising: a. A deformable transport layer coupled to an mounting structure and an interface membrane, wherein the interface membrane is mated to at least one aspect of a mating object having a surface to be characterized; b. A first illumination source operatively coupled to the deformable transport layer using an illumination control layer configured to emit first illumination light into the deformable transport layer in one or more known first illumination orientations relative to the deformable transport layer, such that at least a portion of the first illumination light interacts with the deformable transport layer. c. A detector configured to detect light from at least a portion of the deformable transport layer; and d. An ultrasonic emission source operatively coupled to the deformable transmission layer; e. An ultrasonic detection module operatively coupled to the deformable transmission layer and configured to detect emissions directed from the ultrasonic emission source toward the deformable transmission layer; f. A computing system configured to operate the detector to detect at least a portion of light guided from the deformable transport layer, to determine a surface orientation related to a position along the interface membrane based at least in part on the interaction between the first irradiation light and the deformable transport layer, and to use the determined surface orientation to characterize the geometric profile of the surface of the object docked with the interface membrane. The computing system is operatively coupled to the ultrasonic detection module and is further configured to collect information relating to the interaction between the emission guided from the ultrasonic source and the deformable transmission layer, the information being associated with the relative positioning of the portions of the deformable transmission layer; and The deformable transport layer is configured to controllably adhere to at least one aspect of the docking object having the surface to be characterized.

2. The system according to claim 1, wherein, The deformable transport layer is configured to controllably expand from a contractile form to an expanded form by expanding an operably coupled sac with fluid injection pressure.

3. The system according to claim 2, wherein, The fluid is selected from the group consisting of air, inert gas, water, and saline solution.

4. The system according to claim 2, wherein, The capsule is an elastomeric capsule connecting the deformable transport layer and the mounting structure.

5. The system according to claim 1, wherein, The deformable transport layer is configured to expand controllably by inserting a mechanical expander member relative to the mounting structure.

6. The system of claim 1 further includes a positioning sensor operatively coupled to the computing system and the deformable transport layer.

7. The system according to claim 6, wherein, The positioning sensor is configured to be used by the computing system to determine the position of at least a portion of the deformable transport layer in the global coordinate system.

8. The system according to claim 7, wherein, The computing system and positioning sensors are also configured to determine the orientation of at least a portion of the deformable transport layer in the global coordinate system.

9. The system according to claim 1, wherein, The first irradiation source includes a light-emitting diode.

10. The system according to claim 1, wherein, The detector is a photodetector.

11. The system according to claim 1, wherein, The detector is an image acquisition device.

12. The system according to claim 11, wherein, The image acquisition device is a CCD or CMOS device.

13. The system of claim 1, further comprising a lens operatively coupled between the detector and the deformable transmission layer.

14. The system according to claim 1, wherein, The computing system is operatively coupled to the detector and configured to receive information from the detector relating to light detected by the detector from within the deformable transport layer.

15. The system according to claim 1, wherein, The computing system is operatively coupled to the first irradiation source and configured to control emission from the first irradiation source.

16. The system of claim 1, further comprising a second irradiation source operably coupled to the lighting control layer and configured to direct a second irradiation having a second irradiation wavelength to the lighting control layer, the second irradiation wavelength being different from the first irradiation wavelength of the first irradiation source.

17. The system according to claim 16, wherein, At least one of the first irradiation wavelength or the second irradiation wavelength is within the infrared spectrum.

18. The system according to claim 16, wherein, The first irradiation wavelength and the second irradiation wavelength represent different colors.

19. The system of claim 1 further includes a second illumination source configured to introduce second illumination light into the illumination control layer from a position or orientation different from the first illumination source.

20. The system of claim 19 further includes a third illumination source configured to introduce third illumination light into the illumination control layer from a position or orientation different from the first and second illumination sources.

21. The system according to claim 1, wherein, The lighting control layer is configured to have a shape selected from the group consisting of: planar, substantially planar, curved, convex, semi-convex and saddle-shaped.

22. The system of claim 1, further comprising a second irradiation source operably coupled to the second illumination control layer and configured to direct a second irradiation having a second irradiation wavelength into the deformable transmission layer, the second irradiation wavelength being different from the first irradiation wavelength of the first irradiation source.

23. The system according to claim 22, wherein, The first lighting control layer and the second lighting control layer are stacked on top of each other.

24. The system according to claim 23, wherein, The first lighting control layer and the second lighting control layer are stacked adjacent to each other.

25. The system according to claim 1, wherein, The lighting control layer is located between the detector and the deformable transmission layer.

26. The system according to claim 1, wherein, The detector, illumination control layer, and deformable transmission layer are mechanically coupled within the fingertip assembly, which is configured to include part of an elongated sensing structure.

27. The system according to claim 26, wherein, The elongated sensing structure includes synthetic finger or robotic hand components.

28. The system according to claim 26, wherein, The detector, illumination control layer, and deformable transport layer are operatively coupled to a lens configured to create an optical path providing a virtual camera position relative to the deformable transport layer, the virtual camera position being outside the geometry of the fingertip assembly.

29. The system according to claim 26, wherein, The deformable transport layer includes a convex fingertip shape.

30. The system according to claim 26, wherein, The deformable transmission layer is positioned within the fingertip assembly adjacent to the lighting control layer.

31. The system according to claim 26, wherein, The deformable transmission layer is positioned separately from the lighting control layer within the fingertip assembly.

32. The system according to claim 1, wherein, The deformable transport layer comprises an elastomeric material.

33. The system according to claim 32, wherein, The elastomer material is selected from the group consisting of: silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluororubber.

34. The system according to claim 32, wherein, The deformable transport layer includes a composite material having a pigment material distributed within an elastomer matrix, the pigment material being configured to provide an illumination reflectivity greater than that of the elastomer matrix.

35. The system according to claim 34, wherein, The pigment material includes metal oxides.

36. The system according to claim 35, wherein, The metal oxide is selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide.

37. The system according to claim 34, wherein, The pigment material includes metal nanoparticles.

38. The system according to claim 37, wherein, The metal nanoparticles are selected from the group consisting of silver nanoparticles and aluminum nanoparticles.

39. The system according to claim 1, wherein, The interface membrane comprises an elastomer material.

40. The system according to claim 1, wherein, The interface membrane comprises an elastomer material.

41. The system according to claim 1, wherein, The surface of the docking object is positioned and oriented in a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object docking with the interface membrane by means of its position and orientation relative to the global coordinate system.

42. The system according to claim 41, wherein, The computer system is configured to collect two or more geometric profiles of two or more portions of the surface of the object that is docked with the interface membrane, and to determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system.

43. The system according to claim 42, wherein, The computing system is configured to provide a three-dimensional mapping of the two or more geometric contours relative to each other in the global coordinate system.

44. The system according to claim 43, wherein, The computing system is configured to stitch geometrically adjacent geometric contours together using interpolation of the geometric contours and their relative positions and orientations.

45. The system of claim 41 further includes an auxiliary sensor operatively coupled to the computing system and configured to provide input that can be utilized by the computing system to further geometrically characterize the surface of the object.

46. ​​The system according to claim 45, wherein, The auxiliary sensor is selected from the group consisting of: inertial measurement unit (IMU), capacitive touch sensor, resistive touch sensor, lidar device, strain sensor, load sensor, temperature sensor and image acquisition device.

47. The system according to claim 46, wherein, The auxiliary sensor includes an inertial measurement unit configured to output rotational and linear acceleration data to the computing system, wherein the computing system is configured to use the rotational and linear acceleration data to help characterize the position or orientation of the deformable transport layer in the global coordinate system.

48. The system according to claim 46, wherein, The auxiliary sensor includes an image acquisition device configured to acquire image information related to the surface of the docking object, and wherein the computing system is configured to use the image information to help determine the position or orientation of the object relative to the deformable transport layer.

49. The system of claim 48, further comprising one or more tracking tags coupled to the docking object, and one or more detectors operatively coupled to the computing system, such that the computing system can be used to identify and provide location information related to the docking object, at least in part based on a predetermined position of the one or more tracking tags relative to the docking object.

50. The system according to claim 49, wherein, The one or more tracking tags include radio frequency identification (RFID) tags, and the one or more detectors include RFID detectors.

51. The system according to claim 1, wherein, The ultrasonic emission source is directly and operably coupled to the deformable transmission layer.

52. The system according to claim 1, wherein, The ultrasonic emission source is indirectly and operably coupled to the deformable transmission layer.

53. The system according to claim 1, wherein, The ultrasonic testing module is directly and operably coupled to the deformable transmission layer.

54. The system according to claim 1, wherein, The ultrasonic testing module is indirectly and operably coupled to the deformable transmission layer.

55. The system according to claim 1, wherein, The ultrasonic emission source includes a piezoelectric power source.

56. The system according to claim 1, wherein, The deformable transport layer has a substantially planar shape when unloaded.

57. The system according to claim 1, wherein, The deformable transport layer has a substantially cylindrical shape when unloaded.

58. The system according to claim 57, wherein, The essentially cylindrical deformable transport layer comprises the distal portion of an elongated medical device.

59. The system according to claim 58, wherein, The elongated medical device is selected from the group consisting of catheters, endoscopes, and robotic devices.

60. A system for characterizing geometric surfaces, comprising: a. A deformable transport layer coupled to an mounting structure and an interface membrane, wherein the interface membrane is mated to at least one aspect of a mating object having a surface to be characterized; b. A first illumination source operatively coupled to the deformable transport layer using an illumination control layer configured to emit first illumination light into the deformable transport layer in one or more known first illumination orientations relative to the deformable transport layer, such that at least a portion of the first illumination light interacts with the deformable transport layer. c. A detector configured to detect light from at least a portion of the deformable transport layer; d. An ultrasonic emission source operatively coupled to the deformable transmission layer; e. An ultrasonic detection module operatively coupled to the deformable transmission layer and configured to detect emissions directed from the ultrasonic emission source toward the deformable transmission layer; d. A computing system configured to operate the detector to detect at least a portion of light guided from the deformable transport layer, to determine a surface orientation related to a position along the interface membrane based at least in part on the interaction of the first irradiation light with the deformable transport layer, and to characterize the geometric profile of the surface of the object docked with the interface membrane using the determined surface orientation; wherein the computing system is operatively coupled to the ultrasonic detection module and further configured to collect information relating to the interaction of emission guided from the ultrasonic emission source with the deformable transport layer, the information being associated with the relative positioning of portions of the deformable transport layer; and e. A robotic manipulator operatively coupled to the computing system and the deformable transport layer, the robotic manipulator being configured to controllably position and orient the deformable transport layer relative to the docking object, such that the computing system is able to characterize the geometric profile of the surface of the docking object docked with the interface membrane, with respect to the relative position and orientation of the deformable transport layer and the docking object.

61. The system according to claim 60, wherein, The robot manipulator includes a robot arm.

62. The system according to claim 61, wherein, The robotic arm includes multiple joints coupled by essentially rigid link members.

63. The system according to claim 60, wherein, The robot manipulator includes flexible robotic instruments.

64. The system of claim 60 further includes an end effector coupled to the robot manipulator.

65. The system according to claim 64, wherein, The end effector includes a gripper.

66. The system according to claim 60, wherein, The deformable transport layer is configured to controllably expand from a contractile form to an expanded form by expanding an operably coupled sac with fluid injection pressure.

67. The system according to claim 66, wherein, The fluid is selected from the group consisting of air, inert gas, water, and saline solution.

68. The system according to claim 66, wherein, The capsule is an elastomeric capsule connecting the deformable transport layer and the mounting structure.

69. The system according to claim 60, wherein, The deformable transport layer is configured to expand controllably by inserting a mechanical expander member relative to the mounting structure.

70. The system of claim 60, further comprising a positioning sensor operatively coupled to the computing system and the deformable transport layer.

71. The system according to claim 70, wherein, The positioning sensor is configured to be used by the computing system to determine the position of at least a portion of the deformable transport layer in the global coordinate system.

72. The system according to claim 71, wherein, The computing system and the positioning sensor are also configured to determine the orientation of at least a portion of the deformable transport layer in the global coordinate system.

73. The system according to claim 60, wherein, The first irradiation source includes a light-emitting diode.

74. The system according to claim 60, wherein, The detector is a photodetector.

75. The system according to claim 60, wherein, The detector is an image acquisition device.

76. The system according to claim 75, wherein, The image acquisition device is a CCD or CMOS device.

77. The system of claim 60 further includes a lens operatively coupled between the detector and the deformable transmission layer.

78. The system according to claim 60, wherein, The computing system is operatively coupled to the detector and configured to receive information from the detector relating to light detected by the detector from within the deformable transport layer.

79. The system according to claim 60, wherein, The computing system is operatively coupled to the first irradiation source and configured to control emission from the first irradiation source.

80. The system of claim 60 further includes a second irradiation source operatively coupled to the lighting control layer and configured to direct a second irradiation having a second irradiation wavelength to the lighting control layer, the second irradiation wavelength being different from the first irradiation wavelength of the first irradiation source.

81. The system according to claim 80, wherein, At least one of the first irradiation wavelength or the second irradiation wavelength is within the infrared spectrum.

82. The system according to claim 80, wherein, The first irradiation wavelength and the second irradiation wavelength represent different colors.

83. The system of claim 60 further includes a second illumination source configured to introduce second illumination light into the illumination control layer from a position or orientation different from the first illumination source.

84. The system of claim 83 further includes a third illumination source configured to introduce third illumination light into the illumination control layer from a position or orientation different from the first illumination source and the second illumination source.

85. The system according to claim 60, wherein, The lighting control layer is configured to have a shape selected from the group consisting of: planar, substantially planar, curved, convex, semi-convex and saddle-shaped.

86. The system of claim 60 further includes a second irradiation source operatively coupled to the second illumination control layer and configured to direct a second irradiation having a second irradiation wavelength into the deformable transmission layer, the second irradiation wavelength being different from the first irradiation wavelength of the first irradiation source.

87. The system according to claim 86, wherein, The first lighting control layer and the second lighting control layer are stacked on top of each other.

88. The system according to claim 87, wherein, The first lighting control layer and the second lighting control layer are stacked adjacent to each other.

89. The system according to claim 60, wherein, The lighting control layer is located between the detector and the deformable transmission layer.

90. The system according to claim 60, wherein, The detector, illumination control layer, and deformable transmission layer are mechanically coupled within the fingertip assembly, which is configured to include part of an elongated sensing structure.

91. The system according to claim 90, wherein, The elongated sensing structure includes synthetic finger or robotic hand components.

92. The system according to claim 90, wherein, The detector, illumination control layer, and deformable transport layer are operatively coupled to a lens configured to create an optical path providing a virtual camera position relative to the deformable transport layer, the virtual camera position being outside the geometry of the fingertip assembly.

93. The system according to claim 90, wherein, The deformable transport layer includes a convex fingertip shape.

94. The system according to claim 90, wherein, The deformable transmission layer is positioned within the fingertip assembly adjacent to the lighting control layer.

95. The system according to claim 90, wherein, The deformable transmission layer is positioned separately from the lighting control layer within the fingertip assembly.

96. The system according to claim 60, wherein, The deformable transport layer comprises an elastomeric material.

97. The system according to claim 96, wherein, The elastomer material is selected from the group consisting of: silicone, urethane, polyurethane, thermoplastic elastomer (TPE), thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene, and fluororubber.

98. The system according to claim 96, wherein, The deformable transport layer includes a composite material having a pigment material distributed within an elastomer matrix, the pigment material being configured to provide an illumination reflectivity greater than that of the elastomer matrix.

99. The system according to claim 98, wherein, The pigment material includes metal oxides.

100. The system according to claim 99, wherein, The metal oxide is selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide.

101. The system according to claim 98, wherein, The pigment material includes metal nanoparticles.

102. The system according to claim 101, wherein, The metal nanoparticles are selected from the group consisting of silver nanoparticles and aluminum nanoparticles.

103. The system according to claim 60, wherein, The interface membrane comprises an elastomer material.

104. The system according to claim 60, wherein, The interface membrane comprises an elastomer material.

105. The system according to claim 60, wherein, The surface of the docking object is positioned and oriented in a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object docking with the interface membrane by means of its position and orientation relative to the global coordinate system.

106. The system according to claim 105, wherein, The computer system is configured to collect two or more geometric profiles of two or more portions of the surface of the object that is docked with the interface membrane, and to determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system.

107. The system according to claim 106, wherein, The computing system is configured to provide a three-dimensional mapping of the two or more geometric contours relative to each other in the global coordinate system.

108. The system according to claim 107, wherein, The computing system is configured to stitch geometrically adjacent geometric contours together using interpolation of the geometric contours and their relative positions and orientations.

109. The system of claim 105 further includes an auxiliary sensor operatively coupled to the computing system and configured to provide input that can be utilized by the computing system to further geometrically characterize the surface of the docking object.

110. The system according to claim 109, wherein, The auxiliary sensor is selected from the group consisting of: inertial measurement unit (IMU), capacitive touch sensor, resistive touch sensor, lidar device, strain sensor, load sensor, temperature sensor and image acquisition device.

111. The system according to claim 110, wherein, The auxiliary sensor includes an inertial measurement unit configured to output rotational and linear acceleration data to the computing system, wherein the computing system is configured to use the rotational and linear acceleration data to help characterize the position or orientation of the deformable transport layer in the global coordinate system.

112. The system according to claim 110, wherein, The auxiliary sensor includes an image acquisition device configured to acquire image information related to the surface of the docking object, and wherein the computing system is configured to use the image information to help determine the position or orientation of the object relative to the deformable transport layer.

113. The system of claim 112 further includes one or more tracking tags coupled to the docking object, and one or more detectors operatively coupled to the computing system, such that the computing system can be used to identify and provide location information related to the docking object, at least in part based on a predetermined position of the one or more tracking tags relative to the docking object.

114. The system according to claim 113, wherein, The one or more tracking tags include radio frequency identification (RFID) tags, and the one or more detectors include RFID detectors.

115. The system according to claim 60, wherein, The ultrasonic emission source is directly and operably coupled to the deformable transmission layer.

116. The system according to claim 60, wherein, The ultrasonic emission source is indirectly and operably coupled to the deformable transmission layer.

117. The system according to claim 60, wherein, The ultrasonic testing module is directly and operably coupled to the deformable transmission layer.

118. The system according to claim 60, wherein, The ultrasonic testing module is indirectly and operably coupled to the deformable transmission layer.

119. The system according to claim 60, wherein, The ultrasonic emission source includes a piezoelectric power source.

120. The system according to claim 60, wherein, The deformable transport layer has a substantially planar shape when unloaded.

121. The system according to claim 60, wherein, The deformable transport layer has a substantially cylindrical shape when unloaded.

122. The system according to claim 121, wherein, The essentially cylindrical deformable transport layer comprises the distal portion of an elongated medical device.

123. The system according to claim 122, wherein, The elongated medical device is selected from the group consisting of catheters, endoscopes, and robotic devices.

124. A method for characterizing geometric surfaces, comprising: a. Provide a deformable transport layer coupled to an mounting structure and an interface membrane, wherein the interface membrane is mated to at least one aspect of a mating object having a surface to be characterized; b. Provide a first illumination source operatively coupled to the deformable transport layer using an illumination control layer configured to emit first illumination light into the deformable transport layer in one or more known first illumination orientations relative to the deformable transport layer, such that at least a portion of the first illumination light interacts with the deformable transport layer. c. Provide a detector configured to detect light from at least a portion of the deformable transport layer; d. Provide an ultrasonic emission source operatively coupled to the deformable transmission layer; e. Provide an ultrasonic detection module operatively coupled to the deformable transmission layer and configured to detect emission directed from the ultrasonic emission source toward the deformable transmission layer; and f. Provide a computing system configured to operate the detector to detect at least a portion of light guided from the deformable transport layer to determine a surface orientation related to a position along the interface membrane, based at least in part on the interaction between the first irradiation light and the deformable transport layer, and to use the determined surface orientation to characterize the geometric profile of the surface of the object docked with the interface membrane. The computing system is operatively coupled to the ultrasonic detection module and is further configured to collect information relating to the interaction between the emission guided from the ultrasonic source and the deformable transmission layer, the information being associated with the relative positioning of the portions of the deformable transmission layer; and The deformable transport layer is configured to controllably adhere to at least one side of the docking object having the surface to be characterized.

125. The method according to claim 124, wherein, The deformable transport layer is configured to controllably expand from a contractile form to an expanded form by expanding an operably coupled sac with fluid injection pressure.

126. The method according to claim 125, wherein, The fluid is selected from the group consisting of air, inert gas, water, and saline solution.

127. The method according to claim 125, wherein, The capsule is an elastomeric capsule connecting the deformable transport layer and the mounting structure.

128. The method according to claim 124, wherein, The deformable transport layer is configured to expand controllably by inserting a mechanical expander member relative to the mounting structure.

129. The method of claim 124, further comprising providing a positioning sensor operatively coupled to the computing system and the deformable transport layer.

130. The method according to claim 129, wherein, The positioning sensor is configured to be used by the computing system to determine the position of at least a portion of the deformable transport layer in the global coordinate system.

131. The method according to claim 130, wherein, The computing system and the positioning sensor are also configured to determine the orientation of at least a portion of the deformable transport layer in the global coordinate system.

132. The method according to claim 124, wherein, The first irradiation source includes a light-emitting diode.

133. The method according to claim 124, wherein, The detector is a photodetector.

134. The method according to claim 124, wherein, The detector is an image acquisition device.

135. The method according to claim 134, wherein, The image acquisition device is a CCD or CMOS device.

136. The method of claim 124, further comprising a lens operatively coupled between the detector and the deformable transmission layer.

137. The method according to claim 124, wherein, The computing system is operatively coupled to the detector and configured to receive information from the detector relating to light detected by the detector from within the deformable transport layer.

138. The method according to claim 124, wherein, The computing system is operatively coupled to the first irradiation source and configured to control emission from the first irradiation source.

139. The method of claim 124, further comprising providing a second irradiation source operatively coupled to the lighting control layer and configured to direct a second irradiation having a second irradiation wavelength to the lighting control layer, the second irradiation wavelength being different from the first irradiation wavelength of the first irradiation source.

140. The method of claim 139, wherein, At least one of the first irradiation wavelength or the second irradiation wavelength is within the infrared spectrum.

141. The method according to claim 139, wherein, The first irradiation wavelength and the second irradiation wavelength represent different colors.

142. The method of claim 124, further comprising providing a second illumination source configured to introduce second illumination light into the illumination control layer from a position or orientation different from the first illumination source.

143. The method of claim 142 further includes providing a third illumination source configured to introduce third illumination light into the illumination control layer from a position or orientation different from the first illumination source and the second illumination source.

144. The method according to claim 124, wherein, The lighting control layer is configured to have a shape selected from the group consisting of: planar, substantially planar, curved, convex, semi-convex and saddle-shaped.

145. The method of claim 124, further comprising providing a second irradiation source operatively coupled to a second illumination control layer and configured to direct second irradiation having a second irradiation wavelength into the deformable transmission layer, the second irradiation wavelength being different from the first irradiation wavelength of the first irradiation source.

146. The method according to claim 145, wherein, The first lighting control layer and the second lighting control layer are stacked on top of each other.

147. The method according to claim 146, wherein, The first lighting control layer and the second lighting control layer are stacked adjacent to each other.

148. The method according to claim 124, wherein, The lighting control layer is located between the detector and the deformable transmission layer.

149. The method according to claim 124, wherein, The detector, illumination control layer, and deformable transmission layer are mechanically coupled within the fingertip assembly, which is configured to include part of an elongated sensing structure.

150. The method of claim 149, wherein, The elongated sensing structure includes synthetic finger or robotic hand components.

151. The method according to claim 149, wherein, The detector, illumination control layer, and deformable transport layer are operatively coupled to a lens configured to create an optical path providing a virtual camera position relative to the deformable transport layer, the virtual camera position being outside the geometry of the fingertip assembly.

152. The method according to claim 149, wherein, The deformable transport layer includes a convex fingertip shape.

153. The method according to claim 149, wherein, The deformable transmission layer is positioned within the fingertip assembly adjacent to the lighting control layer.

154. The method according to claim 149, wherein, The deformable transmission layer is positioned separately from the lighting control layer within the fingertip assembly.

155. The method according to claim 124, wherein, The deformable transport layer comprises an elastomeric material.

156. The method according to claim 155, wherein, The elastomer material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene and fluororubber.

157. The method according to claim 155, wherein, The deformable transport layer includes a composite material having a pigment material distributed within an elastomer matrix, the pigment material being configured to provide an illumination reflectivity greater than that of the elastomer matrix.

158. The method according to claim 157, wherein, The pigment material includes metal oxides.

159. The method according to claim 158, wherein, The metal oxide is selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide.

160. The method of claim 157, wherein, The pigment material includes metal nanoparticles.

161. The method of claim 160, wherein, The metal nanoparticles are selected from the group consisting of silver nanoparticles and aluminum nanoparticles.

162. The method according to claim 124, wherein, The interface membrane comprises an elastomer material.

163. The method according to claim 124, wherein, The interface membrane comprises an elastomer material.

164. The method according to claim 124, wherein, The surface of the docking object is positioned and oriented in a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object docking with the interface membrane, the geometric profile having position and orientation relative to the global coordinate system.

165. The method according to claim 164, wherein, The computer system is configured to collect two or more geometric profiles of two or more portions of the surface of the object that is docked with the interface membrane, and to determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system.

166. The method according to claim 165, wherein, The computing system is configured to provide a three-dimensional mapping of the two or more geometric contours relative to each other in the global coordinate system.

167. The method according to claim 166, wherein, The computing system is configured to stitch geometrically adjacent geometric contours together using interpolation of the geometric contours and their relative positions and orientations.

168. The method of claim 164, further comprising an auxiliary sensor operatively coupled to the computing system and configured to provide input that can be utilized by the computing system to further geometrically characterize the surface of the docking object.

169. The method according to claim 168, wherein, The auxiliary sensor is selected from the group consisting of: inertial measurement unit (IMU), capacitive touch sensor, resistive touch sensor, lidar device, strain sensor, load sensor, temperature sensor and image acquisition device.

170. The method of claim 169, wherein, The auxiliary sensor includes an inertial measurement unit configured to output rotational and linear acceleration data to the computing system, wherein the computing system is configured to use the rotational and linear acceleration data to help characterize the position or orientation of the deformable transport layer in the global coordinate system.

171. The method according to claim 169, wherein, The auxiliary sensor includes an image acquisition device configured to acquire image information related to the surface of the docking object, and wherein the computing system is configured to use the image information to help determine the position or orientation of the object relative to the deformable transport layer.

172. The method of claim 171, further comprising providing one or more tracking tags coupled to the docking object, and one or more detectors operatively coupled to the computing system, such that the computing system can be used to identify and provide location information associated with the docking object, at least in part based on a predetermined position of the one or more tracking tags relative to the docking object.

173. The method according to claim 172, wherein, The one or more tracking tags include radio frequency identification (RFID) tags, and the one or more detectors include RFID detectors.

174. The method according to claim 124, wherein, The ultrasonic emission source is directly and operably coupled to the deformable transmission layer.

175. The method according to claim 124, wherein, The ultrasonic emission source is indirectly and operably coupled to the deformable transmission layer.

176. The method according to claim 124, wherein, The ultrasonic testing module is directly and operably coupled to the deformable transmission layer.

177. The method according to claim 124, wherein, The ultrasonic testing module is indirectly and operably coupled to the deformable transmission layer.

178. The method according to claim 124, wherein, The ultrasonic emission source includes a piezoelectric power source.

179. The method according to claim 124, wherein, The deformable transport layer has a substantially planar shape when unloaded.

180. The method of claim 124, wherein, The deformable transport layer has a substantially cylindrical shape when unloaded.

181. The method according to claim 180, wherein, The essentially cylindrical deformable transport layer comprises the distal portion of an elongated medical device.

182. The method according to claim 181, wherein, The elongated medical device is selected from the group consisting of catheters, endoscopes, and robotic devices.

183. A method for characterizing geometric surfaces, comprising: a. Provide a deformable transport layer coupled to an mounting structure and an interface membrane, wherein the interface membrane is mated to at least one aspect of a mating object having a surface to be characterized; b. Provide a first illumination source operatively coupled to the deformable transport layer using an illumination control layer configured to emit first illumination light into the deformable transport layer in one or more known first illumination orientations relative to the deformable transport layer, such that at least a portion of the first illumination light interacts with the deformable transport layer. c. Provide a detector configured to detect light from at least a portion of the deformable transport layer; d. Provide an ultrasonic emission source operatively coupled to the deformable transmission layer; e. Provide an ultrasonic detection module operatively coupled to the deformable transmission layer and configured to detect emissions directed from the ultrasonic emission source toward the deformable transmission layer; d. A computing system is provided, configured to operate the detector to detect at least a portion of light guided from the deformable transport layer, to determine a surface orientation related to a position along the interface membrane, based at least in part on the interaction of the first irradiation light with the deformable transport layer, and to characterize the geometric profile of the surface of the object docked with the interface membrane using the determined surface orientation; wherein the computing system is operatively coupled to the ultrasonic detection module and is further configured to collect information relating to the interaction of emission guided from the ultrasonic emission source with the deformable transport layer, the information being associated with the relative positioning of portions of the deformable transport layer; and e. Provide a robot manipulator operatively coupled to the computing system and the deformable transport layer, the robot manipulator being configured to controllably position and orient the deformable transport layer relative to the docking object, such that the computing system can characterize the geometric profile of the surface of the docking object docked with the interface membrane with respect to the relative position and orientation of the deformable transport layer and the docking object.

184. The method according to claim 183, wherein, The robot manipulator includes a robot arm.

185. The method according to claim 184, wherein, The robotic arm includes multiple joints coupled by essentially rigid link members.

186. The method according to claim 183, wherein, The robot manipulator includes flexible robotic instruments.

187. The method of claim 183, further comprising providing an end effector coupled to the robot manipulator.

188. The method according to claim 187, wherein, The end effector includes a gripper.

189. The method according to claim 183, wherein, The deformable transport layer is configured to controllably expand from a contractile form to an expanded form by expanding an operably coupled sac with fluid injection pressure.

190. The method according to claim 189, wherein, The fluid is selected from the group consisting of air, inert gas, water, and saline solution.

191. The method according to claim 189, wherein, The capsule is an elastomeric capsule connecting the deformable transport layer and the mounting structure.

192. The method according to claim 183, wherein, The deformable transport layer is configured to expand controllably by inserting a mechanical expander member relative to the mounting structure.

193. The method of claim 183 further includes providing a positioning sensor operatively coupled to the computing system and the deformable transport layer.

194. The method according to claim 193, wherein, The positioning sensor is configured to be used by the computing system to determine the position of at least a portion of the deformable transport layer in the global coordinate system.

195. The method according to claim 194, wherein, The computing system and positioning sensors are also configured to determine the orientation of at least a portion of the deformable transport layer in the global coordinate system.

196. The method according to claim 183, wherein, The first irradiation source includes a light-emitting diode.

197. The method according to claim 183, wherein, The detector is a photodetector.

198. The method according to claim 183, wherein, The detector is an image acquisition device.

199. The method according to claim 198, wherein, The image acquisition device is a CCD or CMOS device.

200. The method of claim 183, further comprising providing a lens operatively coupled between the detector and the deformable transmission layer.

201. The method according to claim 183, wherein, The computing system is operatively coupled to the detector and configured to receive information from the detector relating to light detected by the detector from within the deformable transport layer.

202. The method according to claim 183, wherein, The computing system is operatively coupled to the first irradiation source and configured to control emission from the first irradiation source.

203. The method of claim 183 further includes providing a second irradiation source operatively coupled to the lighting control layer and configured to direct second irradiation having a second irradiation wavelength to the lighting control layer, the second irradiation wavelength being different from the first irradiation wavelength of the first irradiation source.

204. The method according to claim 203, wherein, At least one of the first irradiation wavelength or the second irradiation wavelength is within the infrared spectrum.

205. The method according to claim 203, wherein, The first irradiation wavelength and the second irradiation wavelength represent different colors.

206. The method of claim 183 further includes providing a second illumination source configured to introduce second illumination light into the illumination control layer from a position or orientation different from the first illumination source.

207. The method of claim 206 further includes providing a third illumination source configured to introduce third illumination light into the illumination control layer from a position or orientation different from the first and second illumination sources.

208. The method according to claim 183, wherein, The lighting control layer is configured to have a shape selected from the group consisting of: planar, substantially planar, curved, convex, semi-convex and saddle-shaped.

209. The method of claim 183, further comprising providing a second irradiation source operatively coupled to a second illumination control layer and configured to direct second irradiation having a second irradiation wavelength into the deformable transmission layer, the second irradiation wavelength being different from the first irradiation wavelength of the first irradiation source.

210. The method according to claim 209, wherein, The first lighting control layer and the second lighting control layer are stacked on top of each other.

211. The method according to claim 210, wherein, The first lighting control layer and the second lighting control layer are stacked adjacent to each other.

212. The method according to claim 183, wherein, The lighting control layer is located between the detector and the deformable transmission layer.

213. The method according to claim 183, wherein, The detector, illumination control layer, and deformable transmission layer are mechanically coupled within the fingertip assembly, which is configured to include part of an elongated sensing structure.

214. The method according to claim 213, wherein, The elongated sensing structure includes synthetic finger or robotic hand components.

215. The method according to claim 213, wherein, The detector, illumination control layer, and deformable transport layer are operatively coupled to a lens configured to create an optical path providing a virtual camera position relative to the deformable transport layer, the virtual camera position being outside the geometry of the fingertip assembly.

216. The method according to claim 213, wherein, The deformable transport layer includes a convex fingertip shape.

217. The method according to claim 213, wherein, The deformable transmission layer is positioned within the fingertip assembly adjacent to the lighting control layer.

218. The method according to claim 213, wherein, The deformable transmission layer is positioned separately from the lighting control layer within the fingertip assembly.

219. The method according to claim 183, wherein, The deformable transport layer comprises an elastomeric material.

220. The method of claim 219, wherein, The elastomer material is selected from the group consisting of silicone, urethane, polyurethane, thermoplastic elastomer (TPE) and thermoplastic polyurethane (TPU), plastisol, natural rubber, polyvinyl chloride, polyisoprene and fluororubber.

221. The method according to claim 219, wherein, The deformable transport layer includes a composite material having a pigment material distributed within an elastomer matrix, the pigment material being configured to provide an illumination reflectivity greater than that of the elastomer matrix.

222. The method according to claim 221, wherein, The pigment material includes metal oxides.

223. The method according to claim 222, wherein, The metal oxide is selected from the group consisting of iron oxide, zinc oxide, aluminum oxide, and titanium dioxide.

224. The method according to claim 221, wherein, The pigment material includes metal nanoparticles.

225. The method according to claim 224, wherein, The metal nanoparticles are selected from the group consisting of silver nanoparticles and aluminum nanoparticles.

226. The method according to claim 183, wherein, The interface membrane comprises an elastomer material.

227. The method according to claim 183, wherein, The interface membrane comprises an elastomer material.

228. The method according to claim 183, wherein, The surface of the docking object is positioned and oriented in a global coordinate system, and the computing system is configured to characterize the geometric profile of the surface of the object docking with the interface membrane by means of its position and orientation relative to the global coordinate system.

229. The method according to claim 228, wherein, The computer system is configured to collect two or more geometric profiles of two or more portions of the surface of the object that is docked with the interface membrane, and to determine the position and orientation of the two or more geometric profiles relative to each other in the global coordinate system.

230. The method according to claim 229, wherein, The computing system is configured to provide a three-dimensional mapping of the two or more geometric contours relative to each other in the global coordinate system.

231. The method according to claim 230, wherein, The computing system is configured to stitch geometrically adjacent geometric contours together using interpolation of the geometric contours and their relative positions and orientations.

232. The method of claim 228 further includes providing an auxiliary sensor operatively coupled to the computing system and configured to provide input that can be utilized by the computing system to further geometrically characterize the surface of the docking object.

233. The method according to claim 232, wherein, The auxiliary sensor is selected from the group consisting of: inertial measurement unit (IMU), capacitive touch sensor, resistive touch sensor, lidar device, strain sensor, load sensor, temperature sensor and image acquisition device.

234. The method according to claim 233, wherein, The auxiliary sensor includes an inertial measurement unit configured to output rotational and linear acceleration data to the computing system, wherein the computing system is configured to use the rotational and linear acceleration data to help characterize the position or orientation of the deformable transport layer in the global coordinate system.

235. The method according to claim 233, wherein, The auxiliary sensor includes an image acquisition device configured to acquire image information related to the surface of the docking object, and wherein the computing system is configured to use the image information to help determine the position or orientation of the object relative to the deformable transport layer.

236. The method of claim 235, further comprising providing one or more tracking tags coupled to the docking object, and one or more detectors operatively coupled to the computing system, such that the computing system can be used to identify and provide location information associated with the docking object, at least in part based on a predetermined position of the one or more tracking tags relative to the docking object.

237. The method according to claim 236, wherein, The one or more tracking tags include radio frequency identification (RFID) tags, and the one or more detectors include RFID detectors.

238. The method according to claim 183, wherein, The ultrasonic emission source is directly and operably coupled to the deformable transmission layer.

239. The method according to claim 183, wherein, The ultrasonic emission source is indirectly and operably coupled to the deformable transmission layer.

240. The method according to claim 183, wherein, The ultrasonic testing module is directly and operably coupled to the deformable transmission layer.

241. The method according to claim 183, wherein, The ultrasonic testing module is indirectly and operably coupled to the deformable transmission layer.

242. The method according to claim 183, wherein, The ultrasonic emission source includes a piezoelectric power source.

243. The method according to claim 183, wherein, The deformable transport layer has a substantially planar shape when unloaded.

244. The method according to claim 183, wherein, The deformable transport layer has a substantially cylindrical shape when unloaded.

245. The method according to claim 244, wherein, The essentially cylindrical deformable transport layer comprises the distal portion of an elongated medical device.

246. The method according to claim 245, wherein, The elongated medical device is selected from the group consisting of catheters, endoscopes, and robotic devices.

Images