Sensing devices for tactile sensing applications
By designing a detection device including an elastic deformable annular belt, a rigid support member and a microprocessor electronic board, the problem of difficulty in accurately detecting and reproducing the vibration of the object surface in the prior art is solved, and a more realistic haptic feeling is achieved Experience.
Patent Information
- Application Number
- CN201980092954.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-01-02
- Filing Date
- 2019-12-30
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2039-12-30
AI Technical Summary
The existing tactile interfaces are difficult to accurately detect and reproduce vibrations and other tactile sensations on the surface of an object without hindering the user's tactile exploration, especially the vibrations generated when the fingers come into contact with the object and the sliding feeling of the rough surface.
A detection device including an elastic deformable annular belt, a rigid support and a microprocessor electronic board was designed to capture tactile interactive data through vibration sensors, force sensors and temperature sensors, and transmit vibration feelings through rigid outer body to ensure the data Accurate collection and reproduction.
It realizes the accurate detection and reproduction of vibration and other tactile sensations of the object surface without hindering the user's tactile exploration, enhancing the reality and user experience of the tactile interface.
Smart Images

Figure CN113924541B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to detection devices, and in particular to wearable devices for detecting tactile sensations. Background Art
[0002] Nowadays, tactile interfaces, i.e. those devices that allow to provide the user with a sense of touch, are becoming more and more widespread in robotics and virtual reality applications. These interfaces must be properly controlled to generate the same tactile sensation on the user's skin when the user touches the surface of an object.
[0003] Ring-shaped tactile interfaces are known, such as described in Italian patent application 102018000003965 entitled “Tactile Ring”, which can be worn by a user and connected by a wireless or wired connection to a system providing signals for controlling them.
[0004] In order to make the reconstruction of the tactile sensation realistic, it is necessary to sample the tactile sensation using a device equipped with sensors organized to obtain a series of data that, properly processed, will allow controlling the tactile interface to reproduce the same tactile sensation as faithfully as possible.
[0005] It would be desirable to produce a detection device equipped with one or more such sensors that is easy to manufacture and wearable and that does not hinder tactile exploration of an object, but at the same time allows the collection of data that together provide an accurate reproduction of the perceived tactile sensation, so that the tactile sensation can be faithfully reproduced in a tactile interface. Summary of the invention
[0006] Among the various sensations generated during tactile exploration of an object, the detection of vibrations generated by touch on a user's fingers appears to be particularly critical, for example, vibrations generated by the impact of the finger against the object and / or the sliding of the fingertip over the rough surface of the object.
[0007] In order to accurately detect these vibrations without hindering the tactile exploration, a detection device as defined in claim 1 is designed. The detection device basically comprises: an elastically deformable annular belt defining a lower part and an upper part, the lower part is configured to remain exposed and placed in direct contact with the object to be touched and with the user's fingers, and the lower part is configured to remain in an inner compartment of the detection device; a rigid support, fixed to the upper part of the belt and placed outside it; a microprocessor electronic board, fixed to the rigid support and having at least one vibration sensor, the at least one vibration sensor sensing the vibration transmitted to the electronic board; a rigid body, including a front cover and a rear cover, which can be connected together to define an inner compartment and at least one rigid curved surface, the at least one rigid curved surface is configured to be placed in direct contact with the user's fingers, wherein the front cover and the rear cover, when connected together, enclose the upper part of the annular belt, the electronic board and the rigid support in the inner compartment, and the rigid support remains in contact with the inner face of the rigid body.
[0008] The claims as filed are an integral part of this specification and are expressly incorporated herein by reference. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 is an exploded view of a tactile sensing device according to one aspect of the present disclosure.
[0010] Figure 2 yes Figure 1 Front view of the device.
[0011] Figure 3 is along Figure 2 A cross-sectional view of a first plane of the device.
[0012] Figure 4 yes Figure 1 A top view of the device.
[0013] Figure 5 is along Figure 2 A cross-sectional view of a second plane of the device.
[0014] Figure 6 yes Figure 1 Side view of the device.
[0015] Figure 7 A block diagram of an application for generating enhanced multimedia content using haptic information is shown.
[0016] Figure 8 A block diagram of an application for editing enhanced multimedia content using haptic information is shown.
[0017] Fig. 9 is a block diagram for creating and storing haptic content for VR / AR applications.
[0018] Fig.10 is a block diagram for real-time transmission of tactile sensations. DETAILED DESCRIPTION
[0019] According to one aspect of the present disclosure, a tactile sensing device is Figure 1 The tactile sensing device has an annular shape to be preferably worn as a ring, or possibly as a bracelet, or even as a thimble, and is configured to record tactile interactions occurring between the hand of the user wearing it and a body part of a living being or any object that the user of the device can touch. For simplicity of presentation, reference will be made to the illustrated case in which the detection device is a ring that the user can wear on the finger of a hand, but what is about to be said may be repeated mutatis mutandis if the detection device is a bracelet or a thimble.
[0020] Figure 1 The device shown in the comprises an outer body made of a rigid material, which is made into a front cover 1 and a back cover 2 that snap into each other. It also has an annular band 4 made of a deformable material such as silicone, which defines a lower part 5 intended to be in direct contact with an object, and an upper part 6 intended to be kept in the inner compartment of the device. The front cover 1 and the back cover 2 are defined so that the lower part 5 is not covered, and surround the upper part 6 when the upper part 6 is sandwiched in the middle, wherein the lower part 5 protrudes from the body.
[0021] The detection device has an electronic board 3 arranged so as to be held enclosed between the front cover 1 and the rear cover 2 of the outer body together with the upper part 6 of the endless belt 4. The electronic board 3 has a microprocessor and is configured to read the values detected by one or more sensors of the detection device.
[0022] According to one aspect, the detection means can be used to measure the vibrations of a finger touching an object, the vibrations being determined by the surface characteristics of the touched object (roughness, texture, etc.). To this end, a vibration sensor can be present on the electronic board 3, for example of the piezoelectric type or based on MEMS technology.
[0023] According to a preferred aspect, the vibrations sensed on the electronic board 3 are detected by an inertial measurement system (not shown in the figure) based on an accelerometer and gyroscope (IMU) so as to detect the spatial orientation of the device in addition to the vibrations transmitted to the device worn by the user when the user slides his fingertips against the rough surface of an object or hits a rough surface.
[0024] As in Figure 3 and Figure 5 As shown more clearly in Figure 3 and Figure 5 Is the detection device along Figure 2 and Figure 4In the cross-sectional view of the planes AA and BB indicated in FIG, the electronic board 3 is exposed from the upper part 6 of the annular band 4 and is supported by a rigid support 7 in direct contact with the outer body 1, 2. The outer body 1, 2 also defines a rigid curved surface 8 to surround the upper part 6 from the inside of the annular band 4 and directly rest on the back of the finger. Therefore, the structure of the detection device is such that when the user wears it on the finger, the rigid outer body 1, 2 is in direct contact with the back of the finger through the rigid curved surface 8, and the lower part 5 is in contact with the finger on the inside of the palm. By sliding the fingertip of the finger wearing the detection device on the rough surface, vibration occurs in the finger, and the vibration is transmitted to the inertial sensor of the electronic board 3 through the rigid curved surface 8 of the outer body 1, 2 and through the rigid support 7. Therefore, due to the structure of the detection device, the vibration of the outer body 1, 2 is actually transmitted to the electronic board 3 on which the vibration sensor (such as an inertial measurement system) is installed without attenuation, and even if the detection device does not directly contact the object touched by the user, the tactile sensation caused by the surface roughness can be detected.
[0025] Since the annular band 4 is made of a deformable material, the detection device will adapt (within a certain range) to the size of the user's finger.
[0026] According to one aspect, one or more sensors are embedded in the lower portion 5 of the deformable annular band 4, such as force sensors 9 and / or pressure sensors and / or temperature sensors 10, which simultaneously detect data describing the tactile interaction of the user with objects of the surrounding environment.
[0027] According to one aspect, the force sensor 9 is a sensor based on resistive technology (force sensor resistor) or a capacitive sensor, which changes their electrical characteristics when subjected to pressure. Thus, when the user holds an object in the palm of his hand, the lower part 5 made of deformable material is pressed against the object and transmits the force to the force sensor 9, which detects its intensity. The force sensors 9 based on resistive technology are small in size and therefore easy to wear, and their full-scale value is usually in the order of magnitude of the force applied by a human hand (tens of Newtons).
[0028] Other types of sensors (e.g. optical sensors, deformation sensors, sensors detecting biometric parameters, etc.) can be used to estimate the force applied to an object based on, for example, monitoring the deformation caused by the tension of the finger tendons (extensor or flexor muscles) when the fingertip is pushed against the object (non-local measurement) and / or analysis of the blood flow in the finger arteries.
[0029] According to one aspect, the temperature sensor 10 is embedded in the lower part 5 to detect the temperature of an object held in the palm of the user. According to one aspect, the temperature sensor 10 is a temperature sensor of NTC (Negative Temperature Coefficient) type and / or semiconductor and / or thermocouple.
[0030] As with the force, the detection of the temperature of the touched object can also be performed locally, i.e., by contact of the sensor with the object, or in a non-localized manner. In the latter case, the temperature of the object can be estimated by optical means sensitive to infrared wavelengths. According to one aspect, the temperature of the touched object can be estimated by monitoring biometric parameters affected by heat exchange or by an image processing system (e.g., an internal or external camera interfaced thereto).
[0031] According to one aspect, the force sensor 9 and the temperature sensor 10 are mounted on a strip of flexible material 12, which is also embedded in the lower part 5, connected to a rigid support 7 leading from the lower part 5, and the force sensor 9 and the temperature sensor 10 are electrically connected to the electronic board 3 to allow a microprocessor mounted on the electronic board 3 to read the values sensed by all the sensors of the detection device.
[0032] According to one aspect, not shown in the figures, the force sensor 9 and the temperature sensor 10 are mounted on the same printed circuit connected to the electronic board 3. According to an alternative aspect, the force sensor 9 and / or the temperature sensor 10 are connected to the electronic board 3 by wires embedded in the lower portion 5 of the endless belt 4.
[0033] According to one aspect, the electronic board 3 is powered by the device's internal battery 11. In order to meet the need to make the detection device compact and aesthetically pleasing, while meeting the need to have a long-lasting battery 11, the rigid support 7 supporting the electronic board 3 may be C-shaped to define a housing for the battery 11 that remains longitudinally aligned with the user's finger.
[0034] According to one aspect, an electric control button 13 is mounted on the rigid support 7 to switch the detection device on / off and to perform predetermined control operations.
[0035] According to one aspect, the microprocessor electronic board 3 periodically reads the sensors of the device and has an internal memory to store the read values in the internal memory.
[0036] According to one aspect, the electronic board 3 has a wireless or wired communication interface to transmit the values detected by the sensors to an external processing unit (computer, mobile phone, tablet, etc.) or a simple storage unit (server, cloud, etc.).
[0037] According to one aspect, in order to charge the battery 11, the device can be equipped with a connector 14 of the "Pogo pin" type, fixed to the outer body 1, 2 and passing through the opening 15 of the annular strip 4 to contact the charging pad of the electronic board 3. According to an alternative aspect, the device can be configured to charge the battery 11 using an inductive charging system.
[0038] According to one aspect, the electronic board 3 may have at least one light emitting diode that illuminates to emit light of a predetermined color, visible through the light guide 16 , to indicate the operating state of the device and / or the charging of the battery 11 .
[0039] According to one aspect, the device is equipped with a vibration actuator 17, which is configured to generate vibrations when a significant event needs to be reported to the user, for example when switched on or off, and / or when the microprocessor electronic board 3 is connected to an application running on an external device to collect the values of the sensors of the detection device read by the microprocessor of the electronic board 3.
[0040] According to one aspect, the detection device is configured to activate the temperature sensor 10 only when the user holds the object in the palm of his hand, pressing the force sensor 9. Due to this configuration, consumption can be reduced by putting the components of the electronic board 3 for reading the temperature sensor 10 in a standby state.
[0041] The detection device of the present specification can be used to capture and collect data that can be displayed, processed, studied or presented through a tactile interface, as described in Italian patent application 102018000003965 entitled "Tactile Ring", which is a second device ("action device") equipped with an actuator suitable for reproducing at least part of the sensation that produces the acquired data.
[0042] In particular, the tactile sensations that can be obtained are:
[0043] - force exerted on an object by the hand or part of the hand affected;
[0044] - vibrations of the hand and part of the affected hand from tactile exploration of objects;
[0045] -The exchange of heat between the hand or part of the affected hand and an object.
[0046] Different embodiments of the device are foreseen and allow for measuring a range of tactile sensations different from those listed. Thus, the measurement system may be equipped with a range of sensors different from those described above. For example, embodiments of the device may be provided that measure one or a sub-series of the tactile sensations listed above, or that are capable of measuring other aspects of tactile interaction, such as the wetness of the object being touched.
[0047] By wearing multiple detection devices of the present disclosure, a user can simultaneously record tactile interactions from multiple points on his or her hand.
[0048] Possible applications and systems involving the detection device
[0049] Said devices can be used to obtain signals that in some way "describe" the tactile interactions that occur between the user and the surrounding environment. These data can simply be collected for research, analysis, statistics, various inferences, for the construction of databases or for the construction of data sets for training artificial intelligence systems. However, as mentioned above, their most interesting use is related to touch: they can be used in practice as a "source" of tactile data, providing input data for various types of tactile interfaces, that is, devices capable of reproducing recorded tactile sensations on the user who uses them (as described in Italian patent 102018000003965 "Tactile Ring").
[0050] Depending on the application under consideration, the number and type of action devices used and the communication channels to be used, the data recorded by the detection devices can be appropriately processed, combined and possibly compressed by means of special software.
[0051] Generating enhanced multimedia content using tactile information
[0052] like Figure 7 As explained, the detection device can be used to generate multimedia content (usually only containing audio and / or video information) enhanced using the tactile tracks from the device. During the recording of the video that occurs when the user (preferably framed) wears the detection device, the tactile interaction of the user with the surrounding environment is recorded (together with audio and video). This produces content enriched with the tactile tracks from the detection device (in this case it can be regarded as a kind of "tactile microphone"). The generated content can now be processed (stored, sent, shared, edited) as any video. At a second moment, any user (even the same user who generated the content) can use any action device to enjoy the multimedia content with added tactile information, which will reproduce the tactile sensation on its skin, and the tactile sensation corresponds to the scene in which these sensations have been recorded.
[0053] One possible version of the present system has been implemented to work with the device described in patent 102018000003965 "Haptic Ring" as described below.
[0054] A specific mobile application is able to communicate with the detection device and acquire data from the detection device during video recording. Video, audio and tactile data are then recorded simultaneously. At a second moment, the same application can connect to the action device and start playing the video. During this reproduction, for each recorded frame, the video information will be sent to the screen, the audio information will be sent to the audio channel, and the tactile information will be sent to the action device for a multi-sensory use of the content.
[0055] For this application, ad-hoc algorithms for processing and compressing the signals read by the detection device have been implemented, taking into account the action devices involved, the bandwidth of the communication channel and the need to occupy little memory on the mobile device. For example, since the action device that reproduces the recorded tactile sensations contains only one actuator for rendering the vibrations, the information from the inertial system IMU of the detection device has been "compressed" into a single signal using appropriate algorithms. In order to improve the fidelity of the rendering, this signal is processed using filtering and compression techniques before being sent to the actuator.
[0056] Likewise, with regard to temperature, the response of the actuators (Peltier cells) used in the actuation means is taken into account in order to properly sample the signal read by the detection means and to compress the information to be transmitted, thereby obtaining savings and greater efficiency in terms of bandwidth and occupied memory.
[0057] In some cases, the acquired signals can also be "combined" through "sensor fusion" techniques. In an embodiment of this application, for example, the data of the inertial system IMU and the pressure sensor are used to identify the moments of contact between the hand and objects in the surrounding environment during the recording. For example, the data from the temperature sensor are ignored at all "non-contact" moments: the temperature read at these moments is related to the surrounding environment of the device and has nothing to do with the actual temperature of the object touched.
[0058] Editing enhanced multimedia content using tactile information
[0059] However, Figure 8 An application slightly different from the previous application, schematically shown in the flow chart of, provides the possibility of adding these sensations at a later time for creating multimedia content with increased tactile sensations, rather than during the generation of the multimedia content. If a video (even a movie) or a photo is to be enriched with tactile content, the data collected by the detection device can be added to the multimedia content by specific editing software. In the case of a photo, a detection device that generates a tactile sequence associated with the photo can be used to start recording, such as for sending, sharing or simply storing. Such enriched photos can be shared with another user, or can be placed on a website, such as an e-commerce website, where the product can be viewed (from the photo) and can be virtually "touched" due to the associated tactile content. For example, the texture of the fabric of a garment sold can be recorded and associated with a photo of the garment itself for a multi-sensory e-commerce experience.
[0060] In the case of video, one or more haptic sensations can be "placed" in specific haptic tracks (overlaid on the audio and video tracks) along the timeline of the video, corresponding to specific frames, like a kind of "haptic dubbing". As in the first case, the content created in this way can be used by any motion device in their haptic components.
[0061] Share haptic content only
[0062] Another possible use of the detection device is the simple possibility of sharing with another user a tactile sequence recorded as in the previous case, but considered alone, not designed to enrich another multimedia content. For example, this recorded sequence can be used as a "tactile note", similar to the audio notes transmitted via Whatsapp or similar applications.
[0063] Creation and storage of haptic content for VR / AR applications
[0064] Most haptic interfaces ("action devices") on the market today are designed for virtual and augmented reality applications: users wearing such devices to interact with virtual objects can have the sensation of "touching them". To do this, creators of virtual content must also associate haptic content with the virtual objects they create from a graphical perspective. Properties such as hardness, roughness, temperature, etc. will determine the tactile sensations perceived by users when interacting with virtual objects.
[0065] The detection device shown in this specification is used to implement Fig. 9 The block diagram in the figure creates an instant-use tool for these tactile contents. For example, suppose that the creator wants to associate a certain material with a specific virtual object (such as wood). In the absence of a detection device, the content creator should "create" a certain virtual texture that gives the user the feeling of touching a wooden object. Thanks to the detection device of the present disclosure, the process can be greatly simplified: the content creator will only have to touch and record a few seconds of tactile interaction with any real wooden object, and then associate the recorded tactile content with the virtual object he is creating. Therefore, a database of these tactile sensations can also be created, which contains various types of materials. If necessary, all developers and content creators can download these materials from the website. Then, the tactile information associated with a certain virtual object will be reprocessed in this way to be presented on the action device, also taking into account (in real time) the movement and tactile exploration patterns of the user interacting with the virtual object.
[0066] The concept is applicable to all tactile sensations that can be recorded by the detection device, not only to vibrations that determine the surface properties of an object.
[0067] Real-time transmission of tactile sensations
[0068] In one application, two users ("user A" and "user B") are connected to each other through a physical connection, a local network or the Internet, and the physical connection, local network or the Internet connects two devices worn by user A and user B at the same time, one for detection and the other for action. Fig.10 As schematically shown in the figure, user A wearing a detection device can interact tactilely with objects around him, and the acquired tactile information is sent directly to the action device worn by user B in real time so that he can feel the same tactile sensation. Therefore, user B will have the feeling of interacting with the same object touched by user A. An example of this application is a video chat with an increased sense of touch. In this case, the two devices are physically connected to two computers (or mobile devices) used by two users, who also capture and send audio and video information at the same time.
Claims
1. A tactile sensing device, include: an elastically deformable annular band (4), the elastically deformable annular band (4) being wearable as a ring on a user's finger, the elastically deformable annular band (4) defining an annular band lower portion (5) and an annular band upper portion (6), the annular band lower portion (5) being configured to remain exposed and placed in direct contact with an object to be touched and with a user's finger, the annular band upper portion (6) being configured to remain in an inner compartment of a tactile sensing device, wherein the annular band lower portion and the annular band upper portion are integral with each other; a rigid support member (7), the rigid support member (7) being different from the elastically deformable annular band, wherein the rigid support member is fixed to the upper part (6) above the upper part (6) and is placed outside the elastically deformable annular band; an electronic board (3) fixed to the rigid support (7) having at least one vibration sensor and a microprocessor, the microprocessor being functionally configured to receive an electrical signal generated by the at least one vibration sensor, the at least one vibration sensor corresponding to a vibration transmitted on the electronic board (3); A rigid body comprises a rigid front cover (1) and a rigid rear cover (2) which are different from and separate from the rigid support member and the elastically deformable annular band, wherein the rigid front cover (1) and the rigid rear cover (2) can be connected together to define the inner compartment and at least one rigid curved surface (8), at least one rigid curved surface (8) being configured to be in direct contact with the user's finger, and the front cover (1) and the rear cover (2) are configured to enclose the upper portion (6) of the annular band (4), the electronic board (3) and the rigid support member (7) in the inner compartment when connected together, wherein the rigid support member (7) remains in contact with the inner surface of the rigid body. 2 . The sensing device according to claim 1 , wherein the vibration sensor is an inertial measurement system or a piezoelectric sensor.
3. A sensing device according to claim 1 or 2, comprising at least one force or pressure sensor (9) embedded in the lower part (5) of the endless belt (4), the at least one force or pressure sensor (9) being configured to detect the force with which an object is pressed against the lower part (5) of the endless belt (4).
4. The sensing device according to claim 3, wherein the at least one force sensor (9) is a resistive force sensor.
5. The sensing device according to claim 1 or 2, comprising at least one temperature sensor (10) embedded in the lower part (5) of the endless belt (4), the at least one temperature sensor (10) being configured to detect the temperature of an object in contact with the lower part (5) of the endless belt (4).
6. A sensing device according to claim 3, comprising a flexible strip (12) embedded in the lower part (5) of the endless belt (4), said at least one force or pressure sensor (9) and / or temperature sensor (10) being mounted on said flexible strip (12) and being electrically connected to said electronic board (3) to be read by a microprocessor.
7. A sensing device according to claim 1 or 2, wherein the rigid support (7) is C-shaped to define a housing for a battery (11), the housing being longitudinally aligned with the user's finger when the device is worn on the finger.
8. A sensing device according to claim 1 or 2, comprising a vibration actuator (17) functionally connected to the electronic board (3) and configured to vibrate the rigid body upon receipt of a corresponding command from a microprocessor.
9. The sensing device according to claim 1 or 2, wherein the electronic board (3) comprises a wireless or wired communication interface, the wireless or wired communication interface being configured to transmit a value detected by at least one sensor of the detection device to an external unit.
10. The sensing device according to claim 1 or 2, configured to be worn as a ring on a user's finger, wherein the lower portion (5) of the deformable band (4) contacts the skin on the palm side of the finger, and the rigid curved surface (8) of the rigid body directly contacts the back of the finger.
