Visual and tactile sensor and manufacturing method therefor

Abstract

To improve the durability of an elastic portion with which a body to be contacted is brought into contact by enabling detection of information corresponding to human somatic sensation and vision with a simple configuration.SOLUTION: A visual and tactile sensor 10 includes a surface layer body 15 with which a body to be contacted is brought into contact, and an imaging part 19 for imaging the surface layer body 15. The surface layer body 15 includes a body part 21 which enables self-repair by application of thermal stimulation, is formed by a self-repair gel having light transmissivity and has light transmissivity with deformable elasticity at contact with the body to be contacted, and a plurality of markers 22 attached dispersedly to the body part 21 to be able to be imaged by the imaging part 19. The markers 22 are arranged to be able to acquire image data of the body to be contacted to be visual information in a state of transmitting the surface layer body 15, and are arranged to be able to acquire tactile information and temperature information of the body to be contacted about power and vibrations acting on the surface layer body 15 from the image data of the markers 22.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a visual-tactile sensor capable of acquiring information equivalent to human vision and information equivalent to somatic sensation including tactile sensation when contacting an object including a human, and a method for manufacturing the same. [Background technology]

[0002] Recently, robots that can perform various actions while coexisting with humans have appeared, and the present inventors are conducting research and development of robots that aim to reproduce or replace the palpation of a patient by a doctor, etc. In such a robot, sensors must be placed on the parts of the robot hand that come into contact with the body surface of the human being, and the robot must sense information corresponding to the condition of the body surface of the human being through human somatic sensation and vision.

[0003] Conventionally, in robot hands that contact and grasp various objects, a tactile sensor is provided to detect force information acting on the object when the gripping force is adjusted depending on the hardness of the object, etc. Such a tactile sensor is known, for example, as described in Patent Document 1. In this structure, when an object comes into contact with an elastic body to which a marker is attached, a camera captures the displacement of the marker accompanying the deformation of the elastic body due to the contact, and calculates force information acting upon the contact depending on the state of the displacement. Furthermore, like the elastic body in Patent Document 1, the elastic body in this structure is often made of a silicone-based or urethane-based material.

[0004] To enable palpation using the robotic hand currently under development by the present inventors, it is necessary to detect somatosensory and visual information similar to that obtained by human somatosensory and visual examinations, while the robotic hand performs movements such as "pressing," "rubbing," and "pinching." The somatosensory information in this case includes tactile information, such as force and vibration acting upon the patient when in contact, and temperature information obtained by contact with the affected area. Tactile information is used to detect and determine the softness and surface properties of the affected area, while temperature information is used to detect and determine, for example, the sensation of heat due to viral activation in skin diseases or a partial drop in body temperature due to poor blood circulation. Visual information is used to detect and determine the location, shape, color, etc. of the affected area. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-288033 Summary of the Invention [Problem to be solved by the invention]

[0006] The tactile sensor described in Patent Document 1 detects only tactile information and is unable to simultaneously detect somatosensory information, including temperature information, obtained through human somatosensory sensation and visual information, obtained through human vision, using a single sensor configuration. Furthermore, in typical tactile sensors such as those described in Patent Document 1, soft urethane resin, silicone rubber, or the like is used as the material for the elastic body that contacts an object to ensure unimpeded contact. Elastic bodies made from such conventional materials are prone to cracking and other partial damage due to contact with an object over time. In particular, elastic bodies made of urethane gel have durability issues, such as becoming opaque and brittle upon contact with water, making them susceptible to damage. This requires the entire surface of the robot hand, along with markers and other components fixed integrally to the elastic body, to be replaced at relatively short intervals, which is cumbersome and increases maintenance costs.

[0007] The present invention was devised with an eye to these problems in mind, and its purpose is to provide a visual-tactile sensor and a method for manufacturing the same that can detect information equivalent to human somatic sensations and vision with a simple configuration.

[0008] Another object of the present invention is to provide a visual-tactile sensor that can improve the durability of the elastic portion that comes into contact with the contacted object compared to conventional sensors. [Means for solving the problem]

[0009] In order to achieve the above-mentioned object, the visual-tactile sensor of the present invention is primarily equipped with a surface body that is in contact with a predetermined contacted body, and an imaging unit that images the surface body, and is capable of acquiring visual information and somatosensory information corresponding to human vision and somatosensory sensation for the contacted body based on image data from the imaging unit.The surface body comprises a main body that is elastic and translucent so that it can deform when contacted by the contacted body, and a plurality of markers attached in a scattered manner to the main body so that they can be imaged by the imaging unit.The markers are positioned so that the image data of the contacted body, which becomes the visual information, can be acquired while passing through the surface body, and so that tactile information regarding the forces and vibrations acting on the surface body and temperature information of the contacted body can be acquired as the somatosensory information from the image data of the markers.

[0010] Furthermore, the manufacturing method of the visual-tactile sensor of the present invention is a method for manufacturing a visual-tactile sensor that mainly includes a surface layer that elastically deforms when a specified contacted object comes into contact with it, and is capable of acquiring visual information and somatosensory information corresponding to human vision and somatosensory sensation for the contacted object.The surface layer is made of a self-healing gel that is self-repairable when a thermal stimulus is applied and has translucency, and the self-healing gel is produced by adding the monomer N-acryloylnipecotamide and a polymerization initiator to a specified solvent, and then applying a response stimulus in the form of ultraviolet light to polymerize the N-acryloylnipecotamide. [Effects of the Invention]

[0011] According to the present invention, the configuration of the surface body enables somatosensory information, such as tactile information and temperature information, acting upon contact with a contacted body, and visual information about the contacted body to be acquired in a compact, integrated configuration. Furthermore, the translucency of the surface body enables the imaging unit to image not only the external space passing through the surface body but also the interior of the surface body, facilitating early detection of damage to the surface body over time. Furthermore, forming the surface body from a self-healing gel enables the surface body to quickly self-repair when partial damage, such as cracks or scratches, occurs, thereby contributing to improving the durability of the entire sensor. In other words, the surface body can function as a sensor that can acquire pain information equivalent to human pain sensation by detecting partial damage, such as cracks or scratches. It can also function as a sensor for detecting the self-healing status of the self-healing gel. Furthermore, the elasticity of the surface body can be adjusted during production to appropriately acquire various information depending on the type and hardness of the contacted body to be used. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic diagram illustrating the configuration of a visual-tactile sensor according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic exploded perspective view of a sensor body. [Figure 3] FIG. 1 is a diagram showing a chemical formula for explaining the structure of poly(N-acryloylnipecotamide). [Figure 4] FIG. 10 is a schematic cross-sectional view of a sensor main body according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] 1 shows a schematic diagram of a visual-tactile sensor according to this embodiment. Visual-tactile sensor 10 is configured to simultaneously acquire sensory information corresponding to human senses, i.e., visual information corresponding to human vision for recognizing the space around the sensor, and somatosensory information corresponding to somatosensory sensations, including tactile sensations, when touching a contacted object, which is a specific object including a human. In other words, visual-tactile sensor 10 can detect somatosensory information including tactile information inside and outside the sensor relating to forces and vibrations acting when touching a contacted object, temperature information about the contacted object and the sensor itself, and pain information relating to damage to the sensor itself.

[0015] As shown in Figure 1, the visual-tactile sensor 10 comprises a sensor body 11 that acquires data that changes depending on the sensory information when the sensor body 11 comes into contact with the contacted object, and a calculation processing unit 12 that derives visual information and somatosensory information based on the data from the sensor body 11.

[0016] Although the sensor body 11 is not particularly limited, in this embodiment it is mounted on the surface portion of a robot hand used in medical and welfare settings, and the surface portion is positioned so that it can contact a human patient as the contact object and simultaneously acquire various types of sensory information equivalent to the information obtained when a doctor palpates the patient.

[0017] As shown in Figure 2, this sensor body 11 comprises a surface body 15 with which the contacted body comes into contact, a glass plate 16 that supports the surface body 15 from the back side (lower side in Figure 2) opposite the contact surface 15A of the surface body 15 with which the contacted body comes into contact, a light source irradiation unit 17 that is arranged below the glass plate 16 in the same figure, a lens unit 18 that is attached to the center of the light source irradiation unit 17, and an imaging unit 19 that is arranged below the lens unit 18 in Figure 2 and that images the surface body 15.

[0018] The surface layer 15 is made of an elastic material that can elastically deform when contacted with a contacted object, and is made of a polymeric material with self-repairing properties. The polymeric material used is a self-repairing gel made of a translucent hydrogel that includes a polymer having structural units derived from a monomer containing N-acryloylnipecotamide.

[0019] An example of such a self-healing gel is poly(N-acryloylnipecotamide), whose chemical structure is shown in Figure 3. Poly(N-acryloylnipecotamide) is a polymeric material with self-healing properties due to hydrogen bonding. It is synthesized by polymerizing N-acryloylnipecotamide, and self-healing is possible when thermal stimulation is applied due to hydrogen bonding between amide groups present in the side chains of the monomer. While conventional hydrogels (hydrogen-bonding polymeric materials) require the addition of a crosslinking agent during polymerization, poly(N-acryloylnipecotamide) gels by hydrogen bonding between the amide groups in the side chains of the polymer (physical crosslinking only), eliminating the need for a crosslinking agent. Poly(N-acryloylnipecotamide) has higher extensibility than other hydrogels and becomes transparent when formulated at a monomer concentration of 30 to 50 wt%.

[0020] As shown in Figures 1 and 2, the surface layer 15 made of the above-mentioned self-repairing gel comprises a main body 21 that is elastic and translucent and can be deformed when the contacted object comes into contact with it, and a plurality of markers 22 attached to the main body 21 in a scattered manner with gaps between them so that they can be imaged by the imaging unit 19.

[0021] The markers 22 are formed from a self-repairing gel containing thermosensitive microparticles made of a coloring material that reversibly changes color between colored and colorless depending on the temperature, and multiple types of markers 22 with different temperature characteristics when changing color are scattered on the contact surface 15A. These multiple types of markers 22 are produced by mixing multiple types of thermosensitive microparticles, each with a different color when colored. The thermosensitive microparticles used may be of multiple types with different temperature characteristics when changing color, such as those that become red at low temperatures and transparent at high temperatures, with a threshold of 32°C, or those that become blue at low temperatures and transparent at high temperatures, with a threshold of 29°C.

[0022] In one embodiment, the main body 21 is fabricated as follows.

[0023] First, the monomer N-acryloylnipecotamide is produced through the following steps 1 to 5.

[0024] 1. Under an argon gas atmosphere, nipecotamide (e.g., 5.00 g) is added to N,N-dimethylacetamide (ultra-dehydrated) (80 mL) and triethylamine (TEA) (6.5 mL) with stirring to produce solution 1. 2. Under an argon gas atmosphere, add acrylic acid chloride (3.78 mL) to N,N-dimethylacetamide (super-dehydrated) (20 mL) and stir while cooling to produce solution 2. 3. While stirring in a cold bath and under an argon gas atmosphere, the solution 1 is added dropwise at a rate of 5 mL / min to the solution 2. Then, the stopcock is closed with the cold bath removed, and the mixture is stirred overnight to produce solution 3. 4. The salt of the solution 3 is filtered under reduced pressure, and the filtrate is concentrated in an evaporator (to about 10 mL), and then made up to about 30 mL with acetone to produce solution 4. 5. While filtering off the precipitated salt, only the target substance (N-acryloylnipecotamide) is extracted from the solution 4 by column chromatography. Here, a silica column is used, containing silica (approximately 240 g) and a mobile phase consisting of a mixture of acetone and ethyl acetate in a volume ratio of 7:3. Fractions with an Rf value of 0.2 to 0.3, which is the ratio of the volume of the silica column to the volume of the mobile phase passing through the column, are collected and concentrated in an evaporator until a small amount of solid precipitates. The mixture is then ice-cooled to promote precipitation. Hexane is then added, ice-cooled, and the precipitate is collected by vacuum filtration. The collected material is then dried overnight under reduced pressure to obtain N-acryloylnipecotamide.

[0025] Next, the monomer N-acryloylnipecotamide is polymerized by the following procedure to obtain poly(N-acryloylnipecotamide).

[0026] A monomer, a polymerization initiator, and other components are added to a solvent, and a response stimulus from the polymerization initiator is applied. A photopolymerization initiator, such as 2-hydroxy-4'-(2-hydroxyethoxy)-2-methylpropiophenone (Irgacure-2959, manufactured by Merck), is used as the polymerization initiator, and ultraviolet light is irradiated as the response stimulus. The irradiation time depends on the thickness of the gel and the concentration of the monomer in the solvent. An example of this process is described below.

[0027] Ultrapure water is used as a solvent for the monomer N-acryloylnipecotamide, and aqueous solution A, obtained by mixing adamantane acrylamide (Ad-Aam) and β-cyclodextrin acrylamide (β-CD-Aam), is used as a crosslinking agent. For example, the concentration of Ad-Aam in this aqueous solution A is 32.8 mg / mL, and the concentration of β-CD-Aam is 190 mg / mL. Specifically, N-acryloylnipecotamide is added to the crosslinking agent along with ultrapure water, and a photopolymerization initiator is then added. The mixture is degassed with nitrogen gas, and then subjected to polymerization by irradiation with ultraviolet light, resulting in a transparent self-healing gel made of poly(N-acryloylnipecotamide). The photopolymerization initiator is added to aqueous solution A to a concentration of 5.0 mg / mL. The photopolymerization reaction is carried out, for example, by irradiating the solution with a mercury lamp at room temperature (approximately 23°C) under atmospheric pressure in an argon gas environment. Furthermore, the mixing ratio of aqueous solution A, N-acryloylnipecotamide, photopolymerization initiator, and ultrapure water is, for example, 144 μL of aqueous solution A, 12 μL of photopolymerization initiator, and 144 μL of ultrapure water when 200 mg of N-acryloylnipecotamide is added. Note that the transparent self-healing gel of the present invention can be produced even if the addition of the crosslinking agent is omitted from the above procedure.

[0028] The marker 22 is produced by mixing the thermosensitive particles described below with the aqueous solution A to produce aqueous solution B, adding N-acryloylnipecotamide to the aqueous solution B together with ultrapure water, and then performing a polymerization reaction by degassing and light irradiation, similar to the main body 21. The marker 22 thus produced is made of a self-healing gel containing thermosensitive particles. The marker 22 is produced by preparing multiple types of thermosensitive particles with different color change temperatures, and using one type of thermosensitive particle for each type, resulting in multiple types of markers 22 with different temperature change characteristics. For example, Sakura Color Color Corporation's TC Color is used as the thermosensitive particles, and the concentration of the thermosensitive particles in aqueous solution B is 5.0 mg / mL. Here, the mixing ratio of aqueous solution B, N-acryloylnipecotamide, photopolymerization initiator, and ultrapure water is, for example, 120 μL of aqueous solution B, 60 μL of photopolymerization initiator, and 120 μL of ultrapure water when 200 mg of N-acryloylnipecotamide is added.

[0029] Then, markers 22 each having a different temperature change characteristic are embedded or bonded at a plurality of locations on the surface side of the main body 21 at scattered positions at predetermined intervals with gaps between them, thereby obtaining the surface layer 15.

[0030] The light source irradiation section 17 is plate-shaped with a through hole 17A formed in the center, and light from an external light source (not shown) is irradiated onto the edge of the through hole 17A through an optical fiber or the like, supplying light to the surface body 15 and the imaging section 19.

[0031] The lens 18 is a pinhole lens and is housed inside the through-hole 17A.

[0032] The imaging unit 19 is composed of a known imaging device such as a camera or a known imaging element such as a CMOS sensor that can capture image data in the direction of the surface body 15 through a pinhole lens. Since the surface body 15 is translucent, the imaging unit 19 can capture images of each marker 22, as well as images of damage such as scratches occurring inside or on the surface of the main body 21, and images of the space outside the main body 21, including the contacted body, through gaps between the markers 22.

[0033] The arithmetic processing unit 12 is configured by a computer including an arithmetic processing unit such as a CPU, and storage devices such as a memory and a hard disk, and programs for making the computer function as the following units are installed.

[0034] This calculation processing unit 12 is configured to perform various processes based on image data acquired by the imaging unit 19, and as shown in Figure 1, it is equipped with a tactile information acquisition unit 24 that acquires tactile information regarding the forces and vibrations acting on the main body unit 21 based on the displacement of each marker 22 due to the elastic deformation of the main body unit 21 when the contacted object comes into contact with it, a temperature information acquisition unit 25 that acquires temperature information such as the temperature distribution on the surface of the contacted object coming into contact with the main body unit 21 from the temperature change of each marker 22, and a visual information acquisition unit 26 that acquires visual information corresponding to image data of the inside and outside of the main body unit 21.

[0035] The tactile information acquisition unit 24 uses a known algorithm to track the movement of each marker 22 based on time-series image data, and calculates the magnitude of the force acting on each part of the main body 21 where each marker 22 is installed and the vibration of each part based on the displacement of each marker 22, making it possible to detect the mechanical condition of the surface of the contacted object.

[0036] The temperature information acquisition unit 25 detects the discoloration state of each marker 22 from the initial state from the acquired image data of each marker 22 by image matching or the like, thereby identifying the temperature corresponding to the discoloration state for each part of the contacted object close to each marker 22, making it possible to grasp the temperature distribution of the contacted object. In addition, the discoloration state makes it possible to identify the temperature change of the sensor itself before and after contact with the contacted object.

[0037] The visual information acquisition unit 26 acquires image data of the inside of the translucent main body 21 and the outside thereof that has passed through the main body 21 from the gaps formed between the markers 22. This image data makes it possible to detect damage to the inside of the main body 21 and to grasp the condition of the surface of a contacted object that is in contact with or facing the main body 21. Furthermore, because it is possible to detect damage to the inside of the main body 21 from image data of the inside of the main body 21, the visual-tactile sensor 10 also functions as a pain sensor that can acquire information corresponding to pain information equivalent to human pain sensation.

[0038] In the visual-tactile sensor 10 configured as described above, each marker 22 functions as a temperature-sensing element for acquiring temperature information from image data of the discolored area that changes color due to temperature changes (fluctuations) when the contacted object makes contact, and as the discolored area follows and displaces when the contacted object makes contact, it functions as a displacement element for acquiring tactile information based on time-series image data of the discolored area that corresponds to the displacement, making it possible to realize a more compact configuration.

[0039] In the above embodiment, the marker 22 made of a self-healing gel containing thermosensitive particles functions as the temperature-sensing portion and the displacement portion. However, as a modified example, as shown in FIG. 4, a self-healing gel containing the same thermosensitive particles as the marker 22 functions as the temperature-sensing portion 29, and a plurality of solid particles (e.g., approximately 1 mm in diameter) made of spheres of metal such as iron or resin, etc., can function as the displacement portion 30, separate from the temperature-sensing portion 29. These temperature-sensing portions 29 and displacement portions 30 constitute the marker 22 and are arranged in a scattered manner on the main body 21 with gaps formed between them. The displacement portions 30 are obtained by gelling poly(N-acryloylnipecotamide) while mixing the solid particles during polymerization in the above-described self-healing gel production process, thereby obtaining the solid particles fixedly arranged inside the self-healing gel.

[0040] Furthermore, the elasticity, or softness, of the surface layer 15 made of the aforementioned self-healing gel can be adjusted by changing the concentrations of the monomer, polymerization initiator, and / or crosslinker. The higher the monomer concentration, the longer the polymer main chains become, leading to more complex entanglement of the polymers and stronger physical crosslinks, resulting in a harder surface. The higher the polymerization initiator concentration, the greater the number of polymerization initiation points, leading to shorter polymer main chains, which make the polymers less likely to entangle and weaken the physical crosslinks, resulting in a softer surface. The higher the crosslinker concentration, the stronger the mutual attraction between the polymers, resulting in a harder surface. Alternatively, physical properties can be changed by copolymerizing two types of monomers or by changing the solvent used during gelation. In this way, the hardness of the surface layer 15 can be adjusted according to the physical properties and type of contacted object to be detected for sensory information, allowing the formation of a visual-tactile sensor 10 with different detection ranges for tactile information such as force and vibration. In the above-described modified example, the elasticity of the surface layer 15 can also be changed by adjusting the density of the solid particles in the main body 21.

[0041] Furthermore, the configuration of each part of the device in the present invention is not limited to the illustrated configuration example, and various modifications are possible as long as they provide substantially the same effect. [Explanation of symbols]

[0042] 10 Visual and tactile sensors 15 Surface body 19 Imaging unit 21 Main body 22 Markers 29 Temperature sensing part 30 Displacement section

Claims

1. A visual-tactile sensor includes a surface body that a predetermined contacted body contacts, and an imaging unit that images the surface body, and is capable of acquiring visual information corresponding to human vision and somatosensory information corresponding to somatosensory sensation including tactile sensation, based on image data from the imaging unit, the surface body includes a main body portion having elasticity and translucency such that it can be deformed when contacted by the contacted body, and a plurality of markers attached at scattered points to the main body portion so as to be imaged by the imaging unit; A visual-tactile sensor characterized in that the marker is positioned so that it can acquire image data of the contacted body, which serves as the visual information, while passing through the surface body, and is positioned so that tactile information regarding forces and vibrations acting on the surface body and temperature information of the contacted body can be acquired as the somatosensory information from the image data of the marker.

2. 2. The visual-tactile sensor according to claim 1, wherein the surface layer is formed of a self-repairing gel that is self-repairable when a thermal stimulus is applied and that has light-transmitting properties.

3. The visual-tactile sensor according to claim 2, characterized in that the marker is formed from the self-healing gel containing thermosensitive microparticles made of a coloring material that changes color with temperature changes, and is positioned so that the temperature information can be obtained based on the color change state when the contacted object comes into contact with the marker.

4. 4. The visual-tactile sensor according to claim 3, wherein a plurality of types of markers having different temperature characteristics when changing color are scattered on the main body.

5. A visual-tactile sensor as described in any one of claims 1 to 4, characterized in that the imaging unit is positioned so as to be able to acquire image data of the inside of the surface body relating to pain sensation information corresponding to human pain sensation.

6. A visual-tactile sensor as described in any one of claims 1 to 4, characterized in that the surface body is configured so that each of the markers is arranged with a predetermined gap between them, and so that the imaging unit can image the contacted body through the gap.

7. The marker functions as a temperature-sensing unit for acquiring the temperature information from the image data of a discoloration area that changes color due to a temperature change when the contacted object comes into contact with the marker, and also functions as a displacement unit for acquiring the tactile information based on the time-series image data of the discoloration area by the discoloration area displacing in response to contact with the contacted object.

8. 5. The visual-tactile sensor according to claim 2, wherein the self-repairing gel comprises a polymer having structural units derived from a monomer containing N-acryloylnipecotamide.

9. A method for manufacturing a visual-tactile sensor that includes a surface layer that elastically deforms when a predetermined contacted object comes into contact with the surface layer, and that can acquire visual information corresponding to human vision and somatosensory information corresponding to human somatosensory sensation including tactile sensation, the method comprising: The surface layer is capable of self-repairing when subjected to a thermal stimulus and is made of a self-repairing gel that is translucent, and the self-repairing gel is produced by adding the monomer N-acryloylnipecotamide and a polymerization initiator to a specified solvent, and then applying a response stimulus in the form of ultraviolet light to polymerize the N-acryloylnipecotamide.

10. A method for manufacturing a visual-tactile sensor as described in claim 9, characterized in that a crosslinking agent is appropriately added during polymerization of N-acryloylnipecotamide, and the elasticity of the surface layer is adjusted by changing the concentration of the monomer, the polymerization initiator, and / or the crosslinking agent.

Images