Tactile sensor

Abstract

Disclosed is a tactile sensor for sensing an object, comprising: a deformable layer having an outer surface and an inner surface, wherein the outer surface is arranged for coming into contact with the object thereby deforming the deformable layer; a light arrangement arranged for emitting light, and illuminating the inner surface; a plurality of 3D shapes distributed over the inner surface; wherein the light arrangement is posed to the inner surface such that one or more shape shadows originate from an interaction from the light, and one or more of the plurality of 3D shapes, respectively; wherein the light arrangement is also posed to the inner surface such that a longitudinal deformation of the deformable layer causes a layer shadow originating from an interaction from the light, and the inner surface; wherein a lateral deformation of the deformable layer causes a change in a shape shadow of the one or more shape shadows; and wherein the tactile sensor further comprises a camera arranged for imaging the layer shadow, and the one or more shape shadows.

Description

[0001] TACTILE SENSOR

[0002] FIELD OF THE INVENTION

[0003] The invention relates to a tactile sensor. The invention further relates to use of the tactile sensor, a robot limb employing the tactile sensor, a gripper employing the tactile sensor, and a computer-readable storage product for controlling the tactile sensor.

[0004] BACKGROUND OF THE INVENTION

[0005] Vision Based Tactile Sensors -VBTS- are a developing technology in the field of robotic tactile sensing. VBTS can provide detailed information on the sensor-object contact interactions in the form of high- resolution tactile images. When compared to other sensing technologies e.g. capacitive, resistive, piezoresistive, etc., VBTS can capture fine details of the object surface by using off-the-shelf components that are easy to assemble.

[0006] US2023251149 A1 discloses a retrographic sensor including a flexible transparent structure, a transparent elastomeric pad, and an at least partially reflective layer. The flexible transparent structure may be configured to elastically deform between multiple configurations. At least one light source emits light into the transparent structure. The flexible transparent structure may include one or more markers that are illuminated by the at least one light source. The retrographic sensor may also include a photosensitive detector configured to image the flexible transparent structure and one or more markers. A processor may determine a configuration of the flexible transparent structure based at least in part on an image from the one or more markers.

[0007] In applications for these types of sensors, it is common that an object while gripped deforms the flexible transparent structure whereby a balance is to be struck between gripping the object while not damaging the object. A disadvantage of the disclosed retrographic sensor is that amount of gripping power is difficult to determine or almost not possible to determine.

[0008] SUMMARY OF THE INVENTION

[0009] An object of the invention is to overcome one or more of the disadvantages mentioned above. According to a first aspect of the invention, a tactile sensor for sensing an object, comprising: a deformable layer having an outer surface and an inner surface, wherein the outer surface is arranged for coming into contact with the object thereby deforming the deformable layer; a light arrangement arranged for emitting light, and illuminating the inner surface; a plurality of 3D shapes distributed over the inner surface; wherein the light arrangement is posed to the inner surface such that one or more shape shadows originate from an interaction from the light, and one or more of the plurality of 3D shapes, respectively; wherein the light arrangement is also posed to the inner surface such that a longitudinal deformation of the deformable layer causes a layer shadow originating from an interaction from the light, and the inner surface; wherein a lateral deformation of the deformable layer causes a change in a shape shadow of the one or more shape shadows; and wherein the tactile sensor further comprises a camera arranged for imaging the layer shadow, and the one or more shape shadows.

[0010] Tactile sensors are commonly used for controlling the gripping power when gripping an object. Tactile sensors sense the presence of the object for controlling this gripping power. The tactile sensor according to the invention comprises a deformable layer, a light arrangement, a plurality of 3D shapes, and a camera.

[0011] The deformable layer is deformable typically under the influence of contact with the object to be sensed and / or gripped. The deformable layer has an outer surface and an inner surface. The outer surface is arranged for coming into contact with the object for deforming the deformable layer. The light arrangement is arranged for emitting light. The light arrangement is arranged for illuminating the inner surface. The plurality of 3D shapes is distributed over the inner surface. The inner surface may define an orthogonal axis orthogonal to the inner surface, wherein the inner surface typically is axis symmetric around this orthogonal axis. This orthogonal axis may be taken as longitudinal axis. Any deformation along this longitudinal axis may be typed as longitudinal deformation. Furthermore, any deformation perpendicular to the orthogonal axis may be typed as a lateral deformation.

[0012] The light arrangement is posed, such as positioned and / or oriented, to the inner surface such that one or more shape shadows originate from an interaction from the light, and one or more of the plurality of 3D shapes, respectively. The shape shadows are typically visible on the inner surface. A shape shadow typically originates from where an associated 3D shape joins the inner surface and stretches out away from the associated 3D shape. A shadow may be a dark figure or image cast on the inner surface by a body intercepting light, such as one or more of the 3D shapes. A shadow may comprise a shade or comparative darkness, in a particular area. A shadow may also comprise a change in light intensity compared to the surrounding area.

[0013] The light arrangement is also posed, such as positioned and / or oriented, to the inner surface such that a longitudinal deformation of the deformable layer causes a layer shadow originating from an interaction between the light, and the inner surface. The appearance of or change in layer shadow may be in position and / or shape of the layer shadow. The lateral deformation of the deformable layer causes a change in the shape shadow. This change may be in position and / or shape / form of a shape shadow of the one or more shape shadows. Based on the change of the shape shadow, or multiple shape shadows, the lateral deformation may be determined. Alternatively, based on the change of the shape shadow, the position of the associated 3D shape is determined or better determined for determining the lateral deformation. Further, the camera is arranged for imaging the layer shadow, and the one or more shape shadows. Typically, the light intensity change coming from the layer shadow is larger compared to the one or more shape shadows.

[0014] The one or more shape shadows, preferably together with the associated plurality of 3D shapes allow accurate imaging of lateral deformation to a very local level. Furthermore, the layer shadow allows for accurate imaging of longitudinal deformation, ne or more shape shadows, preferably together with the associated plurality of 3D shapes, may supplement imaging of the longitudinal deformation, preferably to a very local level measurement. The combination of imaging very locally the lateral deformation and accurately imaging the longitudinal deformation allows to determine the areas where the contact between the outer surface and the object experience stick, partial slip, or slip. The imaging therefore provides the technical effect of accurate input or measurement for regulating the gripping power or pressure exerted on the object in an application. The imaging may further provide the technical effect of detecting the shape of the object. The imaging may further provide the technical effect of detecting friction between the object and the tactile sensor. The imaging may further provide the technical effect of detecting slip between the object and the tactile sensor.

[0015] According to another aspect of the invention, use of an inventive tactile sensor according to any of the embodiments for sensing an object, typically in an automated application, such as a robot limb. The use provides the advantages also mentioned for the other aspects of the invention, specifically those provided by the tactile sensor.

[0016] According to another aspect of the invention, a robot limb comprising: an elongated part having two ends; and an inventive tactile sensor according to any of the embodiments, and arranged to one of the ends for sensing an object contacted, touched, gripped, lifted, and / or positioned by the robot limb. The robot limb provides the advantages also mentioned for the other aspects of the invention, specifically those provided by the tactile sensor.

[0017] According to another aspect of the invention, a gripper for gripping an object, comprising: a gripping part; an inventive tactile sensor according to any of the embodiments; and actuating means for actuating the gripper such that the tactile sensor and the gripping part both contact the object for together gripping the object. The gripper provides the advantages also mentioned for the other aspects of the invention, specifically those provided by the tactile sensor. According to another aspect of the invention, a computer-readable storage product comprising a computer readable medium comprising instructions which, when executed by a computer, cause the computer to carry out the steps: receiving an image from a camera of an inventive tactile sensor according to any of the embodiments; determining a shape image based on filtering out pixels comprising shape shadows and preferably the plurality of 3D shapes from the received image; determining a lateral deformation of the deformable layer based on the shape image; providing a masked image based on masking out filtered out pixels in the received image; and determining a longitudinal deformation of the deformable layer based on the masked image. The computer-readable storage product provides the advantages also mentioned for the other aspects of the invention, specifically those provided by the tactile sensor.

[0018] DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

[0019] In an embodiment of the sensor, the change in the shape shadow is a change in the position of the shape shadow and / or in the form of the shape shadow. Any change of the shape shadow is advantageously taken into account for accurately determining the lateral deformation.

[0020] In an embodiment of the sensor, the plurality of 3D shapes are opaque and / or reflective to the emitted light. This feature or features allow the plurality of 3D shapes to cast a shadow when exposed to the emitted light under particular angles. In an embodiment of the sensor, the plurality of 3D shapes is at least not transparent to the emitted light. This feature allows the plurality of 3D shapes to cast a shadow when exposed to the emitted light under particular angles.

[0021] In an embodiment of the sensor, the plurality of 3D shapes are advantageously a plurality of domes, hemispheres, cylinders, dimples, cupolas, bubbles, pillars, columns, pyramids, square-based pyramids, cones, cylinders, cuboids, triangular prisms, tetrahedrons, and / or cubes, more preferably a cylinder with a hemisphere on top.

[0022] In an embodiment of the sensor, each of the plurality of 3D shapes has an apex for advantageously casting a long shadow typically improving recognizing changes in the position or shape of the shape shadows improving the accuracy of recognizing a deformation, particularly a lateral deformation.

[0023] In a further embodiment of the sensor, each of the plurality of 3D shapes is tapered towards the apex for allowing a far extending shape shadow for improved recognition or detection of changes or deformation while also providing enough area without the shape shadow for recognizing or detecting the layer shadow more specific changes in the layer shadow.

[0024] In an embodiment of the sensor, each of the plurality of 3D shapes has a symmetry axis; and each of the plurality of 3D shapes is axis symmetric around the symmetry axis. The symmetry axis provides the advantage that the shape of each of the shape shadows remains unchanged when the 3D shapes are rotated allowing for simplified recognition in the software when analysing the image of the camera for deformations of the deformable layer. In a further embodiment of the sensor, the symmetry axis is orthogonal to the inner surface for advantageously providing a simple symmetry axis. In a further embodiment of the sensor, the symmetry axis is arranged at the apex of the 3D shape for advantageously providing a reference point for the symmetry axis.

[0025] In an embodiment of the sensor, the plurality of 3D shapes protrudes from the inner surface having a protrusion height in the range of 100 pm to 1 ,000 pm, preferably 300 pm to 900 pm, more preferably 350 pm to 700 pm, more preferably 400 pm to 600 pm, most preferably substantially 500 pm. The specified range provides shape shadows that are advantageously easily detectable in an image of the camera.

[0026] With a combination of lenses and camera, the shape shadows are typically in the range of 5 to 20 pixels, preferably 10 to 15 pixels, in the image from the camera. These number of pixels is advantageously easily detectable while leaving enough pixels and thus field of view left to cover a large inner surface with the same camera. In an embodiment of the sensor, the plurality of 3D shapes are randomly distributed over and / or positioned on the inner surface. The random distribution causes that locally more shape shadows are present compared to other areas of the inner surface. This results in that for particular areas of the inner surface the lateral deformation can be better measured, while for other locations the longitudinal deformation can be better measured. Furthermore, as these different areas are randomly distributed, the variety of areas causes that on average the lateral as well as the longitudinal deformation is advantageously better measurable.

[0027] In an embodiment of the sensor, the plurality of 3D shapes are distributed in a pattern, preferably a repetitive pattern, over the inner surface. The pattern allows the software processing the image of the camera to use pattern knowledge for simplifying recognition of the different shadows for simplifying recognition of the deformation.

[0028] In an embodiment of the sensor, the layer has a convex curved shape, a dome, or hemisphere shape; and the inner surface is on the inside of the convex curved shape, the dome, or the hemisphere shape, respectively. The convex curved shape, preferably one of the more specific shapes, allows the tactile sensor to contact the object more gently and more gradually for advantageously being able to contact object of a large range of softness or hardness.

[0029] In an embodiment of the sensor, the inner surface defines an orthogonal axis; and the inner surface is axis symmetric around the orthogonal axis. This advantageously provides a tactile sensor which is substantially equally sensitive to deformations from all directions, preferably from all lateral deformation directions or directions perpendicular to the orthogonal axis.

[0030] In an embodiment of the sensor, the light impinges on the inner surface under an incidence angle in the incidence range of 30 to 80 degrees, preferably 50 to 75 degrees, more preferably 60 to 70 degrees, most preferably substantially 65 degrees. With reference to the previous embodiment, the incidence angle is the angle between the direction of the light and the orthogonal axis. The selected range for the incidence angle advantageously balances between the shadow length, shape and / or size of the layer and / or shape shadow.

[0031] In a further embodiment of the sensor, the inner surface and the orthogonal axis define an intersection point; and the light impinges on the inner surface at the intersection point within the incidence range. The intersection point is an advantageously easily selectable reference position.

[0032] In a further embodiment of the sensor, each shape shadow has a shape shadow length; the shape shadow length is defined as tan(90-incidence angle)* a / the protrusion height; and in use without deformation of the deformable layer, at least one shape shadow, preferably at least 10% of the shape shadows, more preferably at least 25% of the shape shadows, more preferably a majority of the shape shadows, most preferably 75% of the shape shadows advantageously have a shape shadow length within the range of more preferably 0.20 mm to 0.45 mm, more preferably 0.25 mm to 0.35 mm, more preferably 0.27 mm to 0.33 mm, most preferably substantially 0.29 mm. In a further embodiment of the sensor, the shape shadow is measured substantially at the orthogonal axis, preferably at the orthogonal axis.

[0033] In an embodiment of the sensor, the sensor comprises a medium arranged between the inner surface and the light arrangement and transparent to the light. This medium advantageously provides a base to the deformable layer whereupon the deformable layer can be arranged, placed, and / or produced.

[0034] In a further embodiment of the sensor, the tactile sensor has a shear property, a deformability property, a stiffness property, a flexibility property, and / or an elastic property of the deformable layer; the object contacting the tactile sensor encounters the property of the tactile sensor; and the medium substantially advantageously determines, preferably predominantly determines, one or more of the properties. Further, based on the images from the camera, a stiffness of the object may be determinable.

[0035] In an embodiment of the sensor, the light arrangement comprises multiple lights arranged for emitting light of different colours, and / or illuminating the inner surface from different directions. Typically, the light arrangement comprises lights emitting light from three directions onto the inner surface for advantageously allowing to detect a direction of deformation. Further typically, the light arrangement comprises lights emitting light with three colours for distinguishing shadows for advantageously allowing to detect a direction of deformation. In a preferred embodiment three lights from different direction with three different colours are used.

[0036] In an embodiment of the sensor, the deformable layer comprises a coating; and the coating forms the inner surface and / or the outer surface. In a preferred embodiment, the deformable layer form or is a membrane. In a preferred embodiment, the coating is arranged on top of the medium for advantageously simplifying manufacturing.

[0037] In an embodiment of the sensor, the deformable layer, the inner surface and / or the outer surface are opaque, preferably reflective, to light for advantageously causing shadows. In an embodiment of the sensor, posing comprises orientating and / or positioning.

[0038] In an embodiment of the sensor, the inner surface has an inner surface area; the 3D shape has a joining surface where the 3D shape joins the inner surface; a surface ratio is defined by dividing the inner surface area by a summation of the joining surfaces; and the surface ratio is advantageously in the range of 500 to 3, preferably 200 to 5, more preferably 100 to 7, more preferably 50 to 8, more preferably 40 to 9, most preferably substantially 18.

[0039] In an embodiment of the sensor, the 3D shape has a / the joining surface where the 3D shape joins the inner surface; each of the plurality of 3D shapes has a / the protrusion height defined by a distance each of the plurality of 3D shapes protrudes from the inner surface; a protrusion ratio is defined by dividing the joining surface area by the square of the protrusion height; and the protrusion ratio for the plurality of 3D shapes is advantageously in the range of 0.1 to 5, preferably 0.2 to 3, more preferably 0.3 to 2, more preferably 0.4 to 1 , most preferably 0.5 to 0.7.

[0040] In an embodiment of the sensor, the tactile sensor comprises a controller arranged for: receiving an image from the camera; determining a shape image based on filtering out pixels comprising shape shadows and preferably the plurality of 3D shapes from the received image; determining a lateral deformation of the deformable layer based on the shape image; providing a masked image based on masking out filtered out pixels in the received image; and determining a longitudinal deformation of the deformable layer based on the masked image. Separating the processing and / or detecting of the different shadows advantageously simplifies the detection of lateral and longitudinal deformation.

[0041] In an embodiment of the sensor, the light arrangement is a LED arrangement; and the light is a LED light, which is advantageously readable available as well reliable.

[0042] In an embodiment of the sensor, the deformable layer without contacting the object has a flat shape for advantageously allowing to contact objects over a surface for advantageously easily determining the most protruding part of the object. In an embodiment of the sensor, the deformable layer without contacting the object has a curved shape, preferably a dome or hemisphere shape for advantageously gradually contacting most or typical objects.

[0043] BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The invention will be apparent from and elucidated further with reference to the embodiments described by way of example in the following description and with reference to the accompanying drawings, in which:

[0045] Figure 1 shows an inventive tactile sensors mounted on a gripper. B. Retrographic image with the dimple array. C. Tracked deformation in the normal and tangential direction. D. Reconstructed shape of the indentation.

[0046] Fig. 2. shows the working principle of the inventive tactile sensor. Light is emitted around the reflective membrane or the deformable layer. The dimple or the 3D shape features embedded in or arranged on the membrane create distinctive shadows from which the lateral displacement can be tracked. Retrographic methods as described below and above can reconstruct the normal displacement field. Fig. 3. Comparison of two dimple shapes or 3D shapes. Sharp edges generate inconsistent shadows. Instead, asymmetric structures eliminate distortions and provide larger shadows.

[0047] Fig. 4. Sensor manufacturing procedure of the inventive tactile sensor. The mold is first created, then the silicone membrane is cast. Once the membrane is unmolded, we paint the reflective layer and assemble it to the sensor body.

[0048] Fig. 5. Exploded view of the inventive tactile sensor. The silicone membrane and the camera can be mounted in a plastic housing.

[0049] Fig. 6. Image processing pipeline for the normal (A) and lateral (B) movement or deformation of the soft silicone membrane.

[0050] Fig. 7. A. View of the setup used for data acquisition. B. The safety margin represents the amount of lateral force before the object starts slipping. C. Example of sliding test: the plot shows the acquired forces and the calculated safety margin, along with the sensed displacement field.

[0051] Fig. 8. Sensor images and 3D reconstructions of a spur gear pressed on the sensor. A. Tactile sensor dimples. B. 0.5 mm dark markers. C. 0.7 mm dark markers. D. Comparison of cross-section height maps.

[0052] Fig. 9. Repeatability test for normal and tangential displacements when the sensor is pressed with a spherical object.

[0053] Fig. 10. A. Slip degree estimation pipeline. The image is processed to find the 3d deformation field, which is used by a simple neural net to estimate the safety margin. B. Performance of the slip estimation on the test dataset. C. Performance of the slip estimation for dry 50mm sphere, which was unseen during training.

[0054] Figure 11 schematically shows an embodiment of a computer program product, computer readable medium and / or non-transitory computer readable storage medium according to the invention.

[0055] The figures are purely diagrammatic and not drawn to scale. In the figures, elements which correspond to elements already described may have the same reference numerals.

[0056] LIST OF REFERENCE NUMERALS DETAILED DESCRIPTION OF THE FIGURES

[0057] The following figures may detail different embodiments. Embodiments can be combined to reach an enhanced or improved technical effect. These combined embodiments may be mentioned explicitly throughout the text, may be hint upon in the text or may be implicit.

[0058] To robustly handle objects, robots should perceive mechanical interactions through touch with sufficient richness. New tactile sensors leverage miniature cameras to provide dense measurements of these interactions, allowing for the extraction of material properties and frictional information. Among the plethora of solutions, retrographic sensing is popular for its ability to finely resolve the shape of the object being touched. The current sensor uses a reflective membrane or deformable layer, illuminated at a shallow angle by three RGB lights, which cast colored shadows. From the illumination pattern of the deformed membrane, both the normal deformation and fine surface details can be recovered.

[0059] Although this part of the retrographic sensor detects longitudinal displacement, the lateral displacement of the membrane or deformable layer cannot be detect and, therefore, overlooks frictional information, which is crucial for grasping and manipulation. A tactile sensor is invented that combines retrographic illumination with non-intrusive markers created by colored shadows. The retrographic surface is patterned with a dense array of submillimeter dimples, which are small enough not to obstruct the view yet cast shadows large enough to be visible to the camera. The current tactile sensor captures a dense image of both the normal displacement field with fine details and a precise lateral displacement field by tracking the markers. Additionally, the current tactile sensor is easy to manufacture, as the dimple pattern can simply be molded. The reliability of the current tactile sensor and its effectiveness in estimating the incipient slip of arbitrary objects is measured. The dense measurement of both the normal and shear deformation that the sensor captures makes it ideal fortracking dynamic interactions between robotic fingertips and manipulated objects.

[0060] I. INTRODUCTION

[0061] To robustly manipulate unknown objects, robots must understand the mechanical interaction that occurs at the interface. This information is readily available from the sense of touch since extracting tactile cues can inform on the material properties of the object (e.g. friction, softness, shape, textures) and the dynamic state of the contact (e.g. interaction forces, linear and rotational slip). In recent years, vision-based tactile sensors, which use embedded cameras to track the movement of an elastic body, have emerged as promising technologies to capture these tactile cues, as they boast a high resolution in a relatively small package.

[0062] These sensors are equipped with a camera, an illumination system, and an elastomeric skin. The latter converts mechanical information into optical data, making surface deformations visible to the camera. The construction of the membrane is of high importance for collecting relevant data. Ideally, the membrane or deformable layer would be built to capture the entire interaction, that is the normal pressure and tangential traction fields that are distributed across the interface between the sensor and the object. If the membrane is soft, these pressure fields induce noticeable normal and tangential deformations that can be captured by a camera.

[0063] Having access to the displacement fields at the interface is of high importance for obtaining a complete overview of the contact state, from which tactile cues can be extracted. With sufficiently soft elastomers and high camera resolution, textural features, e.g. screw threads or fingerprints, can be recognized and used, for example, for object classification tasks. However, a dense estimation of both the normal and tangential displacements while maintaining good visibility for superficial features is not trivial to achieve. Most sensors can effectively provide normal or tangential deformation, offering a partial overview of the contact.

[0064] The most common technique uses markers embedded in the elastomer to create tactile sensors. The camera is pointed at the marker patterns and tracks their motion in the tangential direction. Although excellent in retrieving lateral information, marker-based methods suffer from low sensitivity when measuring normal displacement, which is important for shape or softness measurement. Stereo vision techniques circumvent this limitation by capturing two viewpoints of the membrane and reconstructing the 3d movement of the markers, at the expense of complex hardware and processing. In addition to lacking normal deformation perception, these sensors struggle to detect surface features due to the sparsity of the markers.

[0065] By using the Lambertian reflection of a soft reflective membrane, retrographic sensors achieve pixellevel resolution and can recover a map of the normal deformation. The resolution is high enough to capture detailed features of objects. The membrane is made from a clear elastomer coated with a layer of reflective material, whose inner part is illuminated by a multicolor light pattern, usually 3 RGB LEDs positioned around the membrane at 120 of each other. The deformations of the elastomer create colored patterns that can be decoded using photometric stereo methods to recoverthe normal deformation. However, because they capture light bouncing off the reflective surface, they cannot detect if the membrane is stretched laterally, and therefore, they are not sensitive to shear.

[0066] To add the shear sensitivity to retrographic sensors, it is possible to include opaque markers on the membrane. These markers are either directly painted on the membranes, or made by filling laser-cut holes with dark paint. In both cases, it is challenging to obtain visible and consistent markers below 1 mm in size. It is especially problematic if the sensing surface is curved, which is preferable for robotic grasping and improves the illumination. In addition, since the markers are opaque, they occlude portions of the membrane and hinder the visibility of superficial details. This can be software-corrected but nonetheless undermines one of the main advantages of retrographic sensors.

[0067] The inventive tactile sensor solves these limitations with our sensor (Fig.1 ). The sensor can reconstruct the three-dimensional displacement field without suffering from marker occlusions. The sensor uses a single camera and a curved retrographic surface patterned with a dense array of submillimeter dimples illuminated by 3 RGB LEDs. The triplet of shadows generated locally by each dimple is used to track the lateral movement of the skin, while the normal component is retrieved by using photometric stereo. The dimples are unobtrusive enough that the retrographic images can resolve a high level of detail, sufficient to detect small scale features and textures, while maintaining high sensitivity to lateral movements. Achieving similar size and density with traditional markers would present manufacturing challenges, especially on curved surfaces. In contrast, the current tactile sensor only requires filament-printed components for the sensor casing and an SLA-printed mold to build the membrane or deformable layer, avoiding the use of specialized equipment and making it inexpensive and easy to build. Finally, we show that the dense displacement fields measured by our sensor are repeatable and can be used to continuously estimate the degree of object slippage, which is fundamental to designing real-time delicate grasping algorithms.

[0068] II. RELATED WORKS A. Marker-based sensors

[0069] These sensors are characterized by the color and positioning of the markers. When these are uniform in color and positioned on uniform depth, only their lateral movement can be recovered. Instead, the Tactip sensor family uses as markers an array of protruding pins to magnify the effect of shear and edge contacts. A similar marker design is used for ViTacTip, also featuring a transparent skin to capture information as objects’ color.

[0070] By positioning the markers at various depths, using color changing effects as with the Chromatouch, or both, the normal component of displacement can also be recovered. However, in most cases, these sensors require a careful alignment of markers, challenging manufacturing techniques, and suffer from poor sensitivity.

[0071] Finally, stereo vision methods can reconstruct the 3d motion of markers by capturing two images with different points of view. These can be implemented by using either two cameras, one camera and mirrors as in Tac3D, or a custom stereo vision camera as for the Gelstereo-type sensors.

[0072] B. Retrographic sensors

[0073] A subset of the retrographic sensors features markers to estimate shear or slippage. These sensors are usually flat since adding markers on curved surfaces is challenging. A peculiar implementation is presented in, where the markers are only visible under UV light.

[0074] III. DESIGN RATIONALE A. Quantifying shear with curved retrographic sensors

[0075] To estimate shear distributed multicolor shadows produced by a RGB light source shining light on a collection of dimples are tracked. The movement of these shadows directly correlates with the lateral movement of the reflective membrane. Because the dimples are small, occlusion of the retrographic membrane is minimize, as shown in Fig .2. The shadows are created because when the colored light shines on a dimple, it illuminates one side of it and casts a shadow on the other side. Combining the three color channels generates a stereotypical yellow, cyan, and magenta color scheme around each dimple. The multicolored shadow occupies about 120°, and each shadow is the combination of two light sources with independent wavelengths. The triplet of shadows produced by each color is processed further to determine the lateral motion of the membrane. The dimples are designed such that they (1) give uniform local shadows, (2) minimize camera occlusion and (3) support high density on curved surfaces.

[0076] Making the shadows uniform across the membrane affects directly the robustness of the lateral tracking pipeline. To minimize discrepancies, great care is taken in choosing the shape and size of the dimples. In fact, the illumination on each dimple was chosen so it remains uniform when the membrane undergoes deformations.

[0077] The design encompasses a compromise between having dimples as small as possible to avoid occlusion of the retrographic membrane and as large as possible to cast shadows that are detectable by the camera. A satisfactory trade-off is achieved for 1 mm dimples or smaller in width. At this scale, the recognition of surface features even if the density of dimples is high is preserved. Reaching similar density and size with traditional opaque markers is challenging and close to impossible.

[0078] B. How dimple shape affects shadows

[0079] The chosen shape of each dimple is highly important for a correct shadowing effect and, by extension, for the proper behavior of the tracking pipeline in the software encompassed by the tactile sensor. In particular, when the membrane twists, it can cause illumination problems since it induces a rotation of the dimples (see Fig.3). If the dimples have a shape with sharp edges, as is the case of tetrahedrons, the shadow will change upon rotation since each edge will receive a different amount of light. Instead, axisymmetric structures such as hemispheres produce rotation-invariant shadow profiles, which are suited fortracking complex motions are advantageously used.

[0080] The design can improve over simple hemispheres since to reduce the view occlusion, hemispheric dimples must have small radii, but they generate weak shadows, creating a paradox. Worse, the small dimples have shadows that tend to disappear under small forces due to their insufficient height. As a result, we found that dimples with a cylindrical shape and hemispheric tip solve both issues. Their height is adjustable as desired, thus improving the shadow formation, and their symmetry makes rotation invariant shadows. Moreover, they can be small enough to reduce occlusion.

[0081] IV. MANUFACTURING

[0082] A. Sensing skin

[0083] The material used for the fabrication of the sensing skin or medium is typically as transparent as possible, and soft to enable delicate object handling, and durable to avoid wear after extensive use. The medium may comprise the fully clear Smooth-On Solaris with a shore hardness of substantially 15A and further softened by adding about 40% Smooth-On Silicone Thinner.

[0084] The mold was designed to cast a hemispherical skin with a diameter of 35 mm and a height of 8 mm. After multiple design iterations, the height and base of the dimples were selected to be about 0.5 mm and about 0.45 mm, respectively. The dimples may be arranged concentrically on the silicone with a spacing of about 1 .4 mm, resulting in 200 detectable markers or 3D shapes in total. The mold was first sprayed with a mold release product, and then the silicone was poured into the mold, degassed, and left to cure for 24 hours. After demolding, a thin reflective layer was brushed to the dimple-engraved silicone. The paint consists of Smooth-On PsychoPaint mixed with aluminum powder, Silc Pig white pigment to increase opacity and reduce external light interference, and a silicone solvent. Finally, the painted silicone is glued to a supporting PMMA acrylic plate using Smooth-On SilPoxy glue. The procedure to fabricate the sensor is shown in Fig. 4.

[0085] B. Illumination, camera and assembly

[0086] The illumination consists of three RGB LEDs cut from a Mini Skinny NeoPixel RGB strip. They feature a large emission angle, are low-power and individually controllable. The red, blue, and green lights are evenly spaced and positioned at an angle of 120°, close to the acrylic plate to avoid spurious reflections.

[0087] A Basler daA1920-160uc (S-Mount) camera mounted with the Evetar Lens M13B02820W is used. The wide-angle lens has a small focal length, suitable to be placed close to the acrylic plate, which is glued to the case by using cyanoacrylate glue. The casing consists of two filament-printed components: one that supports the camera, LEDs, and sensing membrane, while the other is a base to mount the sensor on a robotic gripper. An exploded view of the ShadowTac is visible in Fig. 5.

[0088] V. TACTILE IMAGE PROCESSING

[0089] The reconstruction of normal and tangential information is inventively solved with two separated pipelines, which are shown in Fig. 6. The normal component (Fig.6(A)) is obtained using a photometric stereo algorithm. This algorithm calculates a normal map, which is then used to generate a depth (or height) map that provides the normal component for each pixel. To reveal details of the objects, the height map is calculated over the entire tactile image minus the initial dome height. Instead, for dynamic events, computation speed can be improved by subsampling the raw image for height calculation and evaluating the depth at the dimple positions. An experimental scaling factor is applied to convert pixels to millimeters.

[0090] To track the lateral motion of the dimples or 3D shapes, the pipeline is used whose main steps are shown in Fig.6(B). The tactile image is first separated into red, green, and blue color channels, and then each of this channel is shifted by a few pixels in the direction of its corresponding LED. This operation overlaps the three colored shadows into a single dark spot centered at the location of the respective dimple or 3D shape. At this stage, the image can be converted into HSV format and then binarized using an adaptive threshold. Finally, the centroids of each marker are detected using a blob detection method, and another scaling factor is applied to convert pixels to millimeters.

[0091] Sudden contacts can cause random shadows to disappear and consequently cause errors in the estimation of their lateral position. To mitigate this effect, tracking was made more robust. Given a current frame with a marker or 3D shape distribution, each marker is associated with its previous position if their relative distance is smaller than a specified threshold. If there are markers from the previous frame that are not matched with any in the current frame, a local displacement is calculated by using its nearest neighbors, which ensures a consistent number of detected markers or 3D shapes and simplifies computations such as slippage estimation.

[0092] VI. SETUP AND DATA ACQUISITION

[0093] A dataset was collected to evaluate the repeatability of our sensor and another dataset to test its ability to estimate tangential slip. The used setup to capture the data is shown in Fig.7(A). It comprises two motorized linear stages (Thorlabs NRT150 / M), a FT force sensor (ATI Nano 43). The setup lowers one of three 3D- printed objects onto the sensor. Two spheres and one cylinder with diameters 20, 50 and 40 mm were used, respectively. The objects used were positioned at the center of the elastomeric skin at the beginning of every trial. The tactile images and normal and lateral forces were acquired at a frequency of 5 Hz, with careful synchronization between the images and forces.

[0094] To measure how far away an object is from sliding, the safety margin (T) is calculated, which was previously used to assess the degree of partial slip and regulate grasping forces automatically. T is the amount of lateral force that would push the object outside the frictional limit and make it slip, as shown in Fig.7(B). It is defined as r =

[0095] F, where Ft* is the critical tangential force for a given object when full slippage occurs (i.e. macroscopic t relative movement between sensor and object) and Ftis the current one. T was calculated from the recorded lateral forces, with Ft* defined as the maximum value reached in each sliding trial, T ranges from 0 (gross slippage reached and object sliding away from grasp) to 100% (no tangential forces applied). When T is between 0 and 100%, the contact is partially slipping, where some regions are stuck and others are slipping.

[0096] During the acquisition of the slippage dataset (see an example in Fig. 7(C)), an indenter was first vertically pressed on the current tactile sensor, then laterally displaced with a velocity of 0.15 mm / s until full slippage was reached. In specific trials, the frictional properties were modified by wetting the contact surface with water. 14 sliding experiments were conducted:

[0097] • cylinder: indentations of 1 , 2, 3 mm; 2, 3mm wet.

[0098] • 50mm sphere: indentations of 2, 3, 4mm; 3mm wet.

[0099] • 20mm sphere: indentations of 2, 3, 4mm; 3, 4mm wet.

[0100] In this dataset, normal forces up to 6 N were considered, and the friction coefficient varied from about 0.35 for dry surfaces to 0.25 for wet ones. The sliding direction was the same for all the trials, however, to simulate multiple directions of slippage, the original dataset was rotated six times considering a spacing of 60 (i.e. from 0 to 360 ).

[0101] For the repeatability dataset, the spherical indenter with a diameter of 20 mm was used and the following two experiments were performed, executed 8 times each:

[0102] • Normal indentation up to 3 mm at a velocity of 0.1 mm / s. No lateral displacement was considered.

[0103] • Normal indentation of 3 mm followed by a lateral displacement at a velocity of 0.1 mm / s. Only the data was considered before the full slip was detected.

[0104] VII. RESULTS

[0105] A. Occlusion reduction

[0106] The advantage of our dimple design was assessed when compared to standard opaque markers by reconstructing the shape of the same object with the two approaches.

[0107] A spur gear was pressed on the inventive sensor and generated a 3D reconstruction of the contact surface using the height map computed from the photometric stereo algorithm (Fig.8A). Next, black markers were simulated at each dimple position, with a width of 0.5mm in (B) and 0.7mm in (C), which is smaller than the state of the art and therefore does not form part of the state of the art. Finally, the full 3D reconstruction was computed. Although the dimples were still visible in the height map, they conformed to the gear geometry, leaving a high precision of the contour. In contrast, the opacity of the black markers created non-physical flat regions on the height map, completely lacking local shape information.

[0108] Black markers also affected the global intensity of the height map, for an unclear reason. Fig.8(D) shows the height map cross-section for the case with dimples or with dark markers. The height maps generated with dark markers show lower values compared to those with dimples or 3D shapes. Specifically, the 0.5 mm markers reduced the maximum computed height by 12% relative to the dimples, while the 0.7 mm markers caused a reduction of 20%. One possible explanation is that the dark markers absorb all the light that hits the membrane, resulting in an overall reduction of the reflected light available forthe photometric stereo computations.

[0109] B. Displacement repeatability

[0110] Repeatability was evaluated forthe computed displacement fields when the inventive tactile sensor is pressed on an object. The repeatability test is crucial to assess whether the dimples create consistent shadows during both the indentation and sliding phases when interacting with the same object.

[0111] Fig.9 shows a clear linear trend between the average displacement field and the force for both the normal and tangential case. Specifically, a linear fit gives a mean absolute error of 1 .4 x10“3mm and R2of 0.997 for the normal displacement, and a mean absolute error of 1.85x10“2mm and R2of 0.988 forthe lateral one. These results show the consistency of the sensor behavior when subjected to the same indentation modality.

[0112] The sensor could detect a minimal displacement of 0.05 mm normally and 0.5 mm tangentially, based on noise level analysis at rest.

[0113] C. Slip degree estimation

[0114] A schematic of the T estimation pipeline is shown in Fig. 10(A). The tactile images, processed as in section V, produce a 3D displacement field that was used as input to a Multi-Layer Perceptron (MLP). The input dimension is the number of sensor markers multiplied by the number of coordinates (3), which is in the order of 600 depending on the specific skin. The network included three hidden layers with 256, 128, and 32 units, all using the ReLU activation function. The output layer had 1 unit with Sigmoid activation to constrain the T predictions between 0 and 1 . The update frequency reached 10 Hz on the CPU (Intel Xeon(R) W-2223 CPU @ 3.60GHzx8).

[0115] The trial with the dry 50 mm sphere indented 3 mm was set aside of the entire dataset. The model was trained using data split forthe remaining 13 trials: 20% of the slip dataset was reserved fortesting, while the remaining 80% was divided into 80% fortraining and 20% for validation. The mean absolute error as a loss function was used and an Adam optimizer: after 400 epochs, train and validation loss converged to 0.011 and 0.016, respectively. Given the shallow MLP architecture, the training time was about 2 minutes on a regular CPU laptop (Intel i7-9750H @ 2.60GHz).

[0116] Fig. 10(B) shows the predicted versus measured T forthe test dataset, demonstrating high estimation accuracy with an average mean absolute error of 0.014. The gray bargraph represents the mean absolute error for 10% variation of T. The mean absolute error is slightly higher when T decreases, which is expected since conditions close to full slippage are more challenging to predict.

[0117] Finally, Fig. 10(C) shows the predicted versus the measured T (ground truth) forthe dry 50mm sphere indented 3mm, which was an unseen condition during the training. The results show that the network is able to match to a high degree the ground truth, even for data it has not seen before. T forthe six rotated versions of the dataset was predicted to evaluate the ability of the model to predict slippage in different directions. The model maintains high accuracy, with a mean absolute error of 0.014, averaged across all directions.

[0118] VIII. DISCUSSION AND CONCLUSIONS

[0119] A retrographic tactile sensor was introduced that provides dense 3D deformation measurements and detailed contact features. The sensor achieves this ability by tracking markers made of colored shadows. Such shadows are generated by the interaction of the RGB illumination with submillimetric dimples on the reflective membrane or deformable layer. These dimples or 3D shapes are obtained by simply casting the silicone in an SLA mold, bypassing the tedious process of adding black markers through laser cutting or manual painting. The current inventive tactile sensor robustly implements the retrographic reconstruction and avoids the interference that opaque markers usually create.

[0120] The advantages of dimples compared to black dots, which were artificially added in the same positions as the real dimples in the image, were assessed. Although the dimples remain visible in the resulting height map, they tend to conform to local features qualitatively better than the marker array. Since the dimples cast a clearly identifiable pattern of colored shadows on the retrographic image, an algorithm to remove their influence and completely filter them out from the height map reconstruction is envisioned. Conversely, opaque markers generate non-physical flat regions that completely remove the local information about shape, making it impossible to reconstruct the actual height map.

[0121] When subjected to repeated measurements during the same interaction, the sensor shows a high consistency in the output provided, which by extension is a good indication that the shadows generated by the dimples are stable upon deformation of the membrane. Finally, this inventive sensor is suitable for predicting dynamic events, such as the transition from stick to slip for arbitrary objects with unknown geometries and frictional properties. To do this estimation, the displacement of 200 markers was used. The field of displacement of the markers is then fed into a basic multi-layer perceptron. A previous attempt using the Chromatouch sensor with 4 times fewer markers required a Convolutional Neural Network (CNN) to compute the degree of partial slip from the entire tactile image. The complex network compensated for the poor measurement density by processing the full tactile images. Instead, the current tactile sensor uses the dense displacement field and is more fitted to be processed by parsimonious models, as the sensor extracts meaningful data that strongly correlates with the interaction at the contact and dynamic events such as partial slip.

[0122] Figure 11 schematically shows an embodiment of a computer program product 1000, computer readable medium 1010 and / or non-transitory computer readable storage medium according to the invention comprising computer readable code 1020. The compounding system typically comprises a controller arranged for executing one or more of the methods as specified throughout the description and claims as typically coded in software.

[0123] It will also be clear that the above description and drawings are included to illustrate some embodiments of the invention, and not to limit the scope of protection. Starting from this disclosure, many more embodiments will be evident to a skilled person without departing from the scope of the invention as set forth in the appended claims. These embodiments are within the scope of protection and the essence of this invention and are obvious combinations of prior art techniques and the disclosure of this patent. Devices functionally forming separate devices may be integrated in a single physical device.

[0124] The term “substantially” herein, such as in “substantially all emission” or in “substantially consists”, will be understood by the person skilled in the art. The term “substantially” may also include embodiments with “entirely”, “completely”, “all”, etc. Hence, in embodiments the adjective substantially may also be removed. Where applicable, the term “substantially” may also relate to 90% or higher, such as 95% or higher, especially 99% or higher, even more especially 99.5% or higher, including 100%. The term “comprise” also includes embodiments wherein the term “comprises” means “consists of.

[0125] The term "functionally" will be understood by, and be clear to, a person skilled in the art. The term “substantially” as well as “functionally” may also include embodiments with “entirely”, “completely”, “all”, etc. Hence, in embodiments the adjective functionally may also be removed. When used, for instance in “functionally parallel”, a skilled person will understand that the adjective “functionally” includes the term substantially as explained above. Functionally in particular is to be understood to include a configuration of features that allows these features to function as if the adjective “functionally” was not present. The term “functionally” is intended to cover variations in the feature to which it refers, and which variations are such that in the functional use of the feature, possibly in combination with other features it relates to in the invention, that combination of features is able to operate or function. For instance, if an antenna is functionally coupled or functionally connected to a communication device, received electromagnetic signals that are receives by the antenna can be used by the communication device. The word “functionally” as for instance used in “functionally parallel” is used to cover exactly parallel, but also the embodiments that are covered by the word “substantially” explained above. For instance, “functionally parallel” relates to embodiments that in operation function as if the parts are for instance parallel. This covers embodiments for which it is clear to a skilled person that it operates within its intended field of use as if it were parallel.

[0126] Furthermore, the terms first, second, third and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and that the embodiments of the invention described herein are capable of operation in other sequences than described or illustrated herein. Thus, these terms are not necessarily intended to indicate temporal or other prioritization of such elements. The devices or apparatus herein are amongst others described during operation. As will be clear to the person skilled in the art, the invention is not limited to methods of operation or devices in operation.

[0127] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "to comprise" and ‘‘to include’’, and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. Also, the use of introductory phrases such as ‘‘at least one’’ and ‘‘one or more’’ in the claims should not be construed to imply that the introduction of another claim element by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim element to inventions containing only one such element, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an." The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

[0128] The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device or apparatus claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0129] The invention further applies to an apparatus or device comprising one or more of the characterising features described in the description and / or shown in the attached drawings. The invention further pertains to a method or process comprising one or more of the characterising features described in the description and / or shown in the attached drawings.

[0130] It will be appreciated that the invention also applies to computer programs, particularly computer programs on or in a carrier, adapted to put the invention into practice. The program may be in the form of a source code, a code intermediate source and an object code such as in a partially compiled form, or in any other form suitable for use in the implementation of the method according to the invention. It will also be appreciated that such a program may have many different architectural designs. For example, a program code implementing the functionality of the method or system according to the invention may be sub-divided into one or more sub-routines. Many different ways of distributing the functionality among these sub-routines will be apparent to the skilled person. The sub-routines may be stored together in one executable file to form a self-contained program. Such an executable file may comprise computer-executable instructions, for example, processor instructions and / or interpreter instructions (e.g. Java interpreter instructions). Alternatively, one or more or all of the sub-routines may be stored in at least one external library file and linked with a main program either statically or dynamically, e.g. at run-time. The main program contains at least one call to at least one of the sub-routines. The sub-routines may also comprise function calls to each other. An embodiment relating to a computer program product comprises computerexecutable instructions corresponding to each processing stage of at least one of the methods set forth herein. These instructions may be sub-divided into sub-routines and / or stored in one or more files that may be linked statically or dynamically. Another embodiment relating to a computer program product comprises computerexecutable instructions corresponding to each means of at least one of the systems and / or products set forth herein. These instructions may be sub-divided into sub-routines and / or stored in one or more files that may be linked statically or dynamically.

[0131] The carrier of a computer program may be any entity or device capable of carrying the program. For example, the carrier may include a data storage, such as a ROM, for example, a CD ROM or a semiconductor ROM, or a magnetic recording medium, for example, a hard disk. Furthermore, the carrier may be a transmissible carrier such as an electric or optical signal, which may be conveyed via electric or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier may be constituted by such a cable or other device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted to perform, or used in the performance of, the relevant method.

[0132] The various aspects discussed in this patent can be combined in order to provide additional advantages. The mere fact that certain measures are recited in mutually different claims does not indicate that a combination of these measures cannot be used to advantage. Furthermore, some of the features can form the basis for one or more divisional applications.

Claims

CLAIMS1. Tactile sensor (9) for sensing an object, comprising:- a deformable layer (8) having an outer surface and an inner surface, wherein the outer surface is arranged for coming into contact with the object thereby deforming the deformable layer;- a light arrangement (4) arranged for emitting light, and illuminating the inner surface;- a plurality of 3D shapes (10) distributed over the inner surface; and- a camera (6); wherein the light arrangement is posed to the inner surface such that a longitudinal deformation of the deformable layer causes a layer shadow originating from an interaction from the light, and the inner surface; and characterized in that the light arrangement is also posed to the inner surface such that one or more shape shadows originate from an interaction from the light, and one or more of the plurality of 3D shapes, respectively; a lateral deformation of the deformable layer causes a change in a shape shadow of the one or more shape shadows; and the camera is arranged for imaging the layer shadow, and the one or more shape shadows.

2. Tactile sensor according to the preceding claim, wherein the change in the shape shadow is a change in the position of the shape shadow and / or in the form of the shape shadow.

3. Tactile sensor according to any of the preceding claims, wherein the plurality of 3D shapes are opaque and / or reflective to the emitted light.

4. Tactile sensor according to any of the preceding claims, wherein the plurality of 3D shapes are a plurality of domes, hemispheres, cylinders, dimples, cupolas, bubbles, pillars, columns, pyramids, square-based pyramids, cones, cuboids, triangular prisms, tetrahedrons, and / or cubes, preferably a cylinder with a hemisphere on top.

5. Tactile sensor according to any of the preceding claims, wherein each of the plurality of 3D shapes has an apex.

6. Tactile sensor according to the preceding claim, wherein each of the plurality of 3D shapes is tapered towards the apex.

7. Tactile sensor according to any of the preceding claims, wherein each of the plurality of 3D shapes has a symmetry axis; and wherein each of the plurality of 3D shapes is axis symmetric around the symmetry axis.

8. Tactile sensor according to the preceding claim, wherein the symmetry axis is orthogonal to the inner surface.

9. Tactile sensor according to any of the preceding claims, wherein the plurality of 3D shapes protrudes from the inner surface having a protrusion height in the range of 100 pm to 1 ,000 pm, preferably 300 pm to 900 pm, more preferably 350 pm to 700 pm, more preferably 400 pm to 600 pm, most preferably substantially 500 pm.

10. Tactile sensor according to any of the preceding claims, wherein the plurality of 3D shapes are randomly distributed over and / or positioned on the inner surface, or are distributed in a pattern, preferably a repetitive pattern, over the inner surface.11 . Tactile sensor according to any of the preceding claims, wherein the layer has a convex curved shape, a dome, or hemisphere shape; and wherein the inner surface is on the inside of the convex curved shape, the dome, or the hemisphere shape, respectively.

12. Tactile sensor according to any of the preceding claims, wherein the inner surface defines an orthogonal axis; and wherein the inner surface is axis symmetric around the orthogonal axis.

13. Tactile sensor according to any of the preceding claims, wherein the light impinges on the inner surface under an incidence angle in the incidence range of 30 to 80 degrees, preferably 50 to 75 degrees, more preferably 60 to 70 degrees, most preferably substantially 65 degrees.

14. Tactile sensor according to the preceding claim and claim 12, wherein the inner surface and the orthogonal axis define an intersection point; and wherein the light impinges on the inner surface at the intersection point within the incidence range.

15. Tactile sensor according to any of the preceding claims 13-14, wherein each shape shadow has a shape shadow length; wherein the shape shadow length is defined as tan(90-incidence angle)* a / the protrusion height; and wherein in use without deformation of the deformable layer, at least one shape shadow, preferably at least 10% of the shape shadows, more preferably at least 25% of the shape shadows, more preferably a majority of the shape shadows, most preferably 75% of the shape shadows have a shape shadow length within the range of preferably 0.20 mm to 0.45 mm, more preferably 0.25 mm to 0.35 mm, more preferably 0.27 mm to 0.33 mm, most preferably substantially 0.29 mm.

16. Tactile sensor according to the preceding claim, wherein the shape shadow is measured substantially at the orthogonal axis .

17. Tactile sensor according to any of the preceding claims, comprising a medium arranged between the inner surface and the light arrangement and transparent to the light.

18. Tactile sensor according to the preceding claim, wherein the tactile sensor has a shear property, a deformability property, a stiffness property, a flexibility property, and / or an elastic property of the deformable layer; wherein the object contacting the tactile sensor encounters the property of the tactile sensor; and wherein the medium substantially determines, preferably predominantly determines, one or more of the properties.

19. Tactile sensor according to any of the preceding claims, wherein the light arrangement comprises multiple lights arranged for emitting light of different colours, and / or illuminating the inner surface from different directions.

20. Tactile sensor according to any of the preceding claims, wherein the deformable layer comprises a coating; and wherein the coating forms the inner surface and / or the outer surface.21 . Tactile sensor according to any of the preceding claims, wherein the deformable layer, the inner surface and / or the outer surface are opaque, preferably reflective, to light.

22. Tactile sensor according to any of the preceding claims, wherein posing comprises orientating and / or positioning.

23. Tactile sensor according to any of the preceding claims, wherein the inner surface has an inner surface area; wherein the 3D shape has a joining surface where the 3D shape joins the inner surface; wherein a surface ratio is defined by dividing the inner surface area by a summation of the joining surfaces; and wherein the surface ratio is in the range of 500 to 3, preferably 200 to 5, more preferably 100 to 7, more preferably 50 to 8, more preferably 40 to 9, most preferably substantially 18.

24. Tactile sensor according to any of the preceding claims, wherein the 3D shape has a / the joining surface where the 3D shape joins the inner surface; wherein each of the plurality of 3D shapes has a / the protrusion height defined by a distance each of the plurality of 3D shapes protrudes from the inner surface; wherein a protrusion ratio is defined by dividing the joining surface area by the square of the protrusion height; and wherein the protrusion ratio for the plurality of 3D shapes is in the range of 0.1 to 5, preferably 0.2 to 3, more preferably 0.3 to 2, more preferably 0.4 to 1 , most preferably 0.5 to 0.7.

25. Tactile sensor according to any of the preceding claims, comprising a controller arranged for:- receiving an image from the camera;- determining a shape image based on filtering out pixels comprising shape shadows and preferably the plurality of 3D shapes from the received image;- determining a lateral deformation of the deformable layer based on the shape image;- providing a masked image based on masking out filtered out pixels in the received image; and- determining a longitudinal deformation of the deformable layer based on the masked image.

26. Tactile sensor according to any of the preceding claims, wherein the light arrangement is a LED arrangement; and wherein the light is a LED light.

27. Tactile sensor according to any of the preceding claims, wherein the deformable layer without contacting the object has a flat shape; or wherein the deformable layer without contacting the object has a curved shape, preferably a dome or hemisphere shape.

28. Use of a tactile sensor for sensing an object, typically in an automated application, such as a robot limb, according to any of the claims 1-27.

29. Robot limb comprising:- an elongated part having two ends; and- a tactile sensor according to any of the claims 1-27, and arranged to one of the ends for sensing an object contacted, touched, gripped, lifted, and / or positioned by the robot limb.

30. Gripper for gripping an object, comprising:- a gripping part;- a tactile sensor according to any of the claims 1-27; and- actuating means for actuating the gripper such that the tactile sensor and the gripping part both contact the object for together gripping the object.31 . Computer-readable storage product (1000) comprising a computer readable medium (1010) comprising instructions which, when executed by a computer, cause the computer to carry out the steps:- receiving an image from a camera of a tactile sensor according to any of the claims 1-27;- determining a shape image based on filtering out pixels comprising shape shadows and preferably the plurality of 3D shapes from the received image;- determining a lateral deformation of the deformable layer based on the shape image;- providing a masked image based on masking out filtered out pixels in the received image; and- determining a longitudinal deformation of the deformable layer based on the masked image.

Images