Haptic feedback device and control method

Abstract

A tactile presentation device (10) equipped with a screen display circuit (14) configured so as to display a target object on a screen, a tactile presentation circuit (15) configured so as to present a tactile sense which corresponds to the hardness of the target object to a person, and a pressing force detection circuit (16) configured so as to detect a pressing force produced when a person touches the tactile presentation circuit (15), wherein the control circuit (17) changes the tactile sense presented by the tactile presentation circuit (15) on the basis of the hardness of the target object and the pressing force detected by the pressing force detection circuit. As a result, it is possible to present a tactile sense which is not an absolute tactile sense corresponding merely to the hardness of the target object, but is instead a relative tactile sense which corresponds to the pressing force produced when the target object is touched, because the repulsive force against the pressing force produced when a person touches the target object changes.

Description

Technical Field

[0001] The present invention relates to a tactile presentation technology for presenting virtual touch by driving a tactile presentation device.

Background Art

[0002] In recent years, research on virtual reality (VR) has been progressing to allow people to experience a virtual world generated by a computer as if it were real. Also, a virtual reality reproduction device that can realistically reproduce vision, hearing, and touch by video and sound is being developed. For example, as a device that can reproduce three-dimensional video and sound, there is a head-mounted display (HMD).

[0003] On the other hand, as a device that can reproduce human touch, a tactile presentation device has attracted attention. The tactile presentation device is configured to present skin sensation feedback, that is, virtual touch, to a person by touch such as force, vibration, and temperature by driving a tactile presentation device such as a vibration device or a Peltier device.

[0004] Conventionally, the use of a tactile presentation device for tactile media in remote communication has also been studied. Non-Patent Document 1 describes a remote communication system in which tactile presentation devices each including a screen, a contact sensor, and a vibrator are interconnected via the Internet. In this remote communication system, when a person touches one screen, the vibration is detected by one communication device via the contact sensor. This vibration is notified to the other communication device via the Internet and the vibrator is driven. As a result, the other screen vibrates and is transmitted to the other person as touch. Also, when a person touches an object of a different material on the screen, vibrations corresponding to the hardness of the material are output from the vibrator, presenting different tactile sensations.

Prior Art Documents

Non-Patent Documents

[0005] [Non-Patent Document 1] Hayakawa, Hirohiko et al., "A Proposal for 'Public Tactile Communication,' an Interactive Audiovisual and Haptic Media for Remote Communication with a High Sense of Presence," TVRSJ Vol.25 No.4 pp.412-421, 2020. [Overview of the project] [Problems that the invention aims to solve]

[0006] According to the tactile feedback device described in Non-Patent Document 1, it is possible to present different tactile sensations depending on the hardness of the material of the object. Here, actual tactile sensation changes depending on the strength of the force applied when touching; a person feels a strong touch when touching strongly and a weak touch when touching lightly. However, the aforementioned tactile feedback device merely presents a fixed, absolute hardness for each material, and the presented tactile sensation does not change even if the force applied when touching the object differs. Therefore, there was a problem in that it could not present a highly realistic virtual tactile sensation.

[0007] The present invention aims to solve these problems and provides a tactile feedback device that can provide tactile feedback corresponding to the force generated when touching an object. [Means for solving the problem]

[0008] To achieve this objective, the tactile presentation device according to the present invention includes a storage device for storing image data representing an object, a screen display circuit configured to display the object on a screen based on the image data, a tactile presentation circuit configured to present a person with a tactile sensation corresponding to the hardness of the object, a pressure detection circuit configured to detect the pressure generated when a person touches the tactile presentation circuit, and is configured to change the tactile sensation presented by the tactile presentation circuit based on the hardness of the object and the pressure detected by the pressure detection circuit.

[0009] Furthermore, the control method according to the present invention is a control method used in a haptic presentation device comprising a memory device, a screen display circuit, a haptic presentation circuit, a pressure detection circuit, and a control circuit, comprising: a storage step in which the memory device is configured to store image data indicating an object; a screen display step in which the screen display circuit is configured to display the object on the screen based on the image data; a haptic presentation step in which the haptic presentation circuit is configured to present a person with a tactile sensation corresponding to the hardness of the object; a pressure detection step in which the pressure detection circuit is configured to detect the pressure generated when a person touches the haptic presentation circuit; and a control step in which the control circuit is configured to change the tactile sensation presented by the haptic presentation circuit based on the hardness of the object and the pressure detected by the pressure detection circuit. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a tactile presentation device that presents tactile sensations corresponding to the force generated when touching an object. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 is a block diagram showing the configuration of the haptic presentation device according to this embodiment. [Figure 2] Figure 2 is an explanatory diagram illustrating the operation of a haptic feedback device. [Figure 3] Figure 3 is a sequence diagram of the control method for a haptic feedback device. [Figure 4] Figure 4 is an explanatory diagram showing the amount of displacement at the contact point. [Figure 5] Figure 5 is an explanatory diagram showing the adjustment displacement amount of the contact area. [Modes for carrying out the invention]

[0012] Next, embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments described below.

[0013] [This embodiment] First, with reference to Figure 1, the tactile feedback device 10 according to this embodiment will be described. Figure 1 is a block diagram showing the configuration of the tactile feedback device according to this embodiment. The tactile feedback device 10 is a device that presents skin sensory feedback, that is, virtual touch, to a person through the sensation of force.

[0014] In this embodiment, we describe as an example a case in which the haptic presentation device 10 virtually presents the tactile sensation obtained when a person presses on an arbitrary position on an object, based on the hardness of the position selected by the person from among the objects and the pressing force applied by the person. However, the embodiment is not limited to this. For example, if the hardness of the object is constant at all positions, the configuration for a person to select an arbitrary position from among the objects and the configuration for obtaining the hardness of the selected arbitrary position can be omitted.

[0015] As shown in Figure 1, the haptic feedback device 10 comprises, as its main circuit configuration, a communication interface (hereinafter referred to as communication I / F) 11, a storage device 12, an operation circuit 13, a screen display circuit 14, a haptic feedback circuit 15, a pressure detection circuit 16, and a control circuit 17.

[0016] [Communication Interface] The communication interface 11 is configured to acquire various processing data used by the tactile presentation device 10 when presenting tactile sensations, such as the hardness of an object, by communicating data with the information storage device 20 via a communication network such as the Internet.

[0017] [Information storage device] The information storage device 20 consists of, for example, a server device, and stores image data DV composed of an image V showing the appearance and shape of an object, hardness data DO showing the hardness O at each position of the object, and adjustment characteristic data DG showing the characteristics of an adjustment coefficient g(F) corresponding to a person's pressing force with respect to the displacement amount of the contact portion 15A provided in the tactile presentation circuit 15. The information storage device 20 is configured to present the object image V of the specified object, the hardness O related to the specified object and position, and the adjustment coefficient g(F) corresponding to the specified pressing force F in response to a request from the tactile presentation device 10 via the communication network NW.

[0018] [Storage device] The storage device 12 consists of a storage device such as a hard disk or a semiconductor memory, and is configured to store various processing data and programs 12P used when the tactile presentation device 10 presents a tactile sensation. The processing data stored in the storage device 12 includes image data DV, hardness data DO, and adjustment characteristic data DG.

[0019] The image data DV, the hardness data DO, and the adjustment characteristic data DG are equivalent to the DV, DO, and DG stored in the information storage device 20, and each may be stored in at least one of the information storage device 20 and the storage device 12. The information storage device 20 can be regarded as being externally connected to the tactile presentation device 10 via the communication I / F 11 and the communication network NW. If the information storage device 20 stores DV, DO, and DG, they can be shared by a plurality of tactile presentation devices 10 and can be centrally managed. If the storage device 12 stores DV, DO, and DG, the tactile sensation can be presented from the tactile presentation device 10 in a stand-alone manner.

[0020] The adjustment characteristic data DG is a characteristic showing an adjustment coefficient corresponding to a person's pressing force with respect to the displacement amount of the contact portion 15A provided in the tactile presentation circuit 15. Since DG is represented by a function g(F) having the person's pressing force F as a variable, it may be stored as a specific function formula, or may be stored as a table showing the relationship between the pressing force F and the adjustment coefficient g(F).

[0021] The program 12P stored in the memory device 12 works in cooperation with the CPU of the control circuit 17 to realize various processing units used for haptic feedback processing in the control circuit 17. This program 12P is pre-read from an external device or recording medium (neither shown) connected to the haptic feedback device 10 and stored in the memory device 12.

[0022] [Operation circuit] The operation circuit 13 consists of a pointing device such as a mouse or touch panel, and is configured to detect human operation and output it to the control circuit 17. Operations detected by the operation circuit 13 include pointer operations to move the position of the pointer displayed on the screen of the screen display circuit 14, and switching operations to switch the operation of the control circuit 17.

[0023] [Screen display circuit] The screen display circuit 14 consists of a screen display device such as an LCD, and is configured to display various screens, such as an image V showing the appearance and shape of an object, as well as operation menu screens and setting screens, in response to instructions from the control circuit 17.

[0024] [Haptic feedback circuit] The tactile feedback circuit 15 has a contact portion 15A whose indentation position is displaced in response to human pressure, and is configured to provide tactile feedback corresponding to the hardness of the object by displacing the indentation position of the contact portion 15A in response to instructions from the control circuit 17.

[0025] [Pressing force detection circuit] The pressing force detection circuit 16 consists of a sensing device such as a pressure sensor and is configured to detect the pressing force applied by a person to the contact portion 15A and output it to the control circuit 17.

[0026] [control circuit] The control circuit 17 is configured to change the tactile sensation presented by the tactile presentation circuit 15 based on the hardness of the object and the pressure applied by the person detected by the pressure detection circuit 16. Specifically, the control circuit 17 displays the image data DV from the storage device 12 on the screen of the screen display circuit 14, changes the display position of the pointer on the screen to a new position in response to the pointer operation of the person detected by the operation circuit 13, and changes the tactile sensation presented by the tactile presentation circuit 15 based on the hardness of the new position on the object and the pressing force detected by the pressing force detection circuit 16.

[0027] In this case, the control circuit 17 may switch between two operations based on a user's switching operation detected by the operation circuit 13: changing the display position of the pointer on the screen to a new position in response to the pointer operation detected by the operation circuit 13, and changing the tactile sensation presented by the tactile presentation circuit 15 based on the hardness of the new position on the object and the pressing force detected by the pressing force detection circuit 16. The control circuit 17 has a CPU and its peripheral circuits, and reads the program 12P from the storage device 12 and executes it with the CPU, thereby realizing various processing units used for haptic feedback processing in the control circuit 17 by coordinating the CPU and the program 12P.

[0028] [Operation of this embodiment] Next, the operation of the haptic feedback device 10 according to this embodiment will be described with reference to Figures 2 and 3. Figure 2 is an explanatory diagram showing the operation of the haptic feedback device. Figure 3 is a sequence diagram of the control method for the haptic feedback device.

[0029] Here, we will explain as an example how the haptic presentation device 10 presents tactile sensations to a person based on image data DV, hardness data DO, and adjustment characteristic data DG stored in the storage device 12. When using DV, DO, and DG stored in the information storage device 20, they can be obtained from the information storage device 20 via the communication interface 11 and communication network NW, as described above.

[0030] The storage device 12 stores image data DV, hardness data DO, and adjustment characteristic data DG (storage step 100). The control circuit 17 of the haptic presentation device 10 acquires the image data DV from the storage device 12 and displays an image V showing the appearance and shape of the object T on the screen of the screen display circuit 14 based on this image data DV (screen display step 104). Thus, although the haptic presentation device 10 is a device that haptically presents the tactile sensation when touching the object T, it can also visually present the object T, and can be said to be a visual and tactile multimodal device.

[0031] On screen V, the image of the captured object T is projected directly, or the object T is displayed as 3D data, allowing for a visual understanding of the object's shape. In particular, when the object is converted to 3D data, its shape can be viewed from any 360-degree viewpoint, making it possible to visually understand the condition of the affected area in medical settings.

[0032] On the other hand, when data is converted to 3D, information such as the color of the affected area is lost, so it is desirable to present it together with video (live-action). In that case, display methods include displaying the 3D data and video side by side, switching modes, or superimposing them. Furthermore, it is most desirable to create a 3D model from video footage of the actual object taken from multiple viewpoints, without losing the actual texture.

[0033] This image V includes a pointer P for a person to select a touch location. When the control circuit 13 detects a person's pointer operation C, the control circuit 17 calculates a new position DP for the pointer P displayed on the screen according to that pointer operation C, and displays image V with the pointer's display position changed to the new position DP (screen display step 112).

[0034] The control circuit 13 allows for the manipulation of the pointer P's position, enabling free movement in directions such as left, right, up, down, forward, and backward. It is designed to be synchronized with the on-screen display, similar to a trackpad or mouse.

[0035] In this case, the operation circuit 13 may be configured to switch the operating mode of the control circuit 17 between an operation mode that moves the pointer P and an operation mode that displays the hardness O of the current position of the pointer P on the screen using the screen display circuit 14. Since the operation circuit 13 has a function to sense human movement, it is possible to switch modes through specific interactions.

[0036] For example, when the pointer P is in an area of ​​the screen where tactile information of an object can be displayed, mode switching operations can be freely defined, such as when a double-click or long-press is performed. By switching modes, the user can move the pointer P to find out the hardness O of any position on the object T.

[0037] When the control circuit 13 detects a human pointer operation C (operation step 110), the control circuit 17 obtains the calculated new position DP and the corresponding hardness O from the hardness data DO in the storage device 12. The tactile presentation circuit 15 is configured to present a tactile sensation to the person corresponding to the hardness O of the object T, based on the instructions from the control circuit 17 (tactile presentation step 132).

[0038] When the pressure detection circuit 16 detects the pressure generated when a person touches the tactile presentation circuit 15 (pressure detection step 120), the control circuit 17 obtains the pressure F from the pressure detection circuit 16 and changes the tactile sensation presented by the tactile presentation circuit 15 based on the hardness O and the pressure F (control step 132). At this time, the control circuit 17 calculates the displacement L of the contact portion 15A based on the hardness O and the pressure F, and displaces the indentation position of the contact portion 15A by the displacement L.

[0039] Figure 4 is an explanatory diagram showing the displacement of the contact area. The displacement L is calculated as the product of hardness O and pressing force F, O·F. Therefore, when a person touches the contact area 15A with their hand H and pressing force F, the contact area 15A will be pushed in by a displacement L = O·F, which is the product of hardness O and pressing force F at the new position DP of the object T.

[0040] As a result, instead of simply an absolute tactile sensation corresponding to the hardness of the object, the repulsive force against the pressing force F generated when a person touches the object changes, thus presenting a relative tactile sensation to the person's hand that corresponds to the pressing force F generated when contacting the object.

[0041] Alternatively, the control circuit 17 may obtain an adjustment coefficient g(F) corresponding to the pressing force F based on the adjustment characteristic data DG of the storage device 12, adjust the displacement amount L obtained from the product of hardness O and pressing force F based on the adjustment coefficient g(F), and displace the pressing position of the contact portion 15A by the obtained adjustment displacement amount L'.

[0042] Figure 5 is an explanatory diagram showing the adjustment displacement amount of the contact area. The adjustment displacement amount L' is calculated as the product of hardness O, pressing force F, and adjustment coefficient g(F), L = O·F·g(F). Therefore, when a person touches the contact area 15A with their hand H with pressing force F, the contact area 15A will be pushed in by an adjustment displacement amount L = O·F, which is the product of the hardness O, pressing force F, and adjustment coefficient g(F) at the new position DP of the object T.

[0043] Generally, the tactile sensation a person experiences when touching an object of a certain hardness O changes depending on the strength of the force applied during the touch; a strong touch results in a strong tactile sensation, while a weak touch results in a weak tactile sensation. When this tactile sensation is altered by the displacement L of the contact point 15A, the relationship between the pressing force F and the displacement L may not be linear, but rather nonlinear, with hardness O as the coefficient. Therefore, by adjusting the displacement L using an adjustment coefficient g(F) with pressing force F as the variable, as described above, it is possible to handle cases where the relationship between pressing force F and displacement L is nonlinear. This allows for the presentation of a more realistic tactile sensation.

[0044] [Effects of this embodiment] As described above, this embodiment includes a haptic presentation device 10 comprising a screen display circuit 14 configured to display an object on the screen, a haptic presentation circuit 15 configured to present a person with a tactile sensation corresponding to the hardness of the object, and a pressure detection circuit 16 configured to detect the pressure generated when a person touches the haptic presentation circuit 15. The control circuit 17 is configured to change the tactile sensation presented by the haptic presentation circuit 15 based on the hardness of the object and the pressure detected by the pressure detection circuit 16.

[0045] This allows for the presentation of a relative tactile sensation to a person, corresponding to the pressure applied when they touch the object. This is achieved because the repulsive force against the pressure applied when a person touches the object changes adaptively, rather than simply providing an absolute tactile sensation based on the object's hardness.

[0046] Furthermore, the haptic presentation device 10 may also include an operation circuit 13 configured to detect pointer operations to move the position of a pointer displayed on the screen, and a control circuit 17 may be configured to change the display position of the pointer on the screen to a new position in response to the pointer operation detected by the operation circuit 13, and to change the tactile sensation presented by the haptic presentation circuit 15 based on the hardness O of the new position on the object and the pressing force F detected by the pressing force detection circuit 16.

[0047] Specifically, the memory device 12 may further store hardness data DO at each position of the object, and the control circuit 17 may obtain the hardness O of a new position on the object from the hardness data DO in the memory device 12, and change the tactile sensation presented by the tactile presentation circuit 15 based on the hardness O and pressing force F. This makes it possible to present the tactile sensation when any position on the object is pressed.

[0048] Furthermore, the control circuit 17 may switch between two operations based on a switching operation detected by the operation circuit 13: changing the display position of the pointer on the screen to a new position in response to the pointer operation detected by the operation circuit 13, and displaying the hardness of the object at the new position on the screen. This allows not only the pointer position to be changed but also the hardness of the object at the new position to be displayed on the screen through an operation in the operation circuit 13.

[0049] Alternatively, the tactile presentation circuit 15 may have a contact portion 15A whose indentation position is displaced in response to human pressure, and the control circuit 17 may calculate the amount of displacement L of the contact portion 15A based on the hardness O and the pressing force F, and displace the indentation position of the contact portion 15A by the amount of displacement L, thereby presenting a tactile sensation corresponding to the hardness O of the object. This allows for the presentation of a relative tactile sensation corresponding to the pressing force F generated when a person touches the object, rather than simply an absolute tactile sensation corresponding to the hardness O of the object, because the repulsive force against the pressing force F generated when a person touches the object changes.

[0050] In this case, the memory device 12 may further store adjustment characteristic data DG that indicates an adjustment coefficient g(F) corresponding to the human pressing force F for the displacement L of the contact portion 15A, and the control circuit 17 may obtain the adjustment coefficient g(F) corresponding to the pressing force F from the memory device 12 based on the adjustment characteristic data DG, and adjust the displacement L obtained from the product of hardness O and pressing force F based on the adjustment coefficient g(F), thereby displacing the pressing position of the contact portion 15A by the obtained adjustment displacement L'.

[0051] Generally, the tactile sensation a person experiences when touching an object of a certain hardness O changes depending on the strength of the force applied during the touch; a strong touch results in a strong tactile sensation, while a weak touch results in a weak tactile sensation. When this tactile sensation is altered by the displacement L of the contact point 15A, the relationship between the pressing force F and the displacement L may not be linear, but rather nonlinear, with hardness O as the coefficient. Therefore, by adjusting the displacement L using an adjustment coefficient g(F) with pressing force F as the variable, as described above, it is possible to handle cases where the relationship between pressing force F and displacement L is nonlinear. This allows for the presentation of a more realistic tactile sensation.

[0052] [Expansion of the embodiment] Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above embodiments. Various modifications to the configuration and details of the present invention can be made that will be understood by those skilled in the art within the scope of the present invention. [Explanation of symbols]

[0053] 10...Haptic feedback device, 11...Communication I / F, 12...Storage device, 12P...Program, 13...Operation circuit, 14...Screen display circuit, 15...Haptic feedback circuit, 15A...Contact part, 16...Pressure detection circuit, 17...Control circuit, 20...Information storage device, NW...Communication network, DV...Image data, DO...Hardness data, DG...Adjustment characteristic data, O...Hardness, F...Pressure, g(F)...Adjustment coefficient, L...Displacement amount.

Claims

1. A storage device that stores image data showing the object and adjustment characteristic data showing the characteristics of the adjustment coefficient corresponding to the pressure applied by a person, A screen display circuit configured to display the object on the screen based on the image data, A tactile presentation circuit configured to present a person with a tactile sensation corresponding to the hardness of the object, A pressure detection circuit configured to detect the pressure generated when a person touches the aforementioned tactile presentation circuit, A control circuit is configured to change the tactile sensation presented by the tactile presentation circuit based on the hardness of the object, the pressing force detected by the pressing force detection circuit, and an adjustment coefficient corresponding to the pressing force obtained from the adjustment characteristic data, The system includes an operation circuit configured to detect pointer operations for moving the position of the pointer displayed on the screen, The memory device further stores the hardness at each position of the object, The aforementioned control circuit is The control circuit changes the display position of the pointer on the screen to a new position in response to the pointer operation detected by the control circuit. The hardness of the new position in the object is obtained from the storage device. Based on the hardness of the new position within the object and the pressure detected by the pressure detection circuit, the tactile sensation presented by the tactile presentation circuit is changed. A haptic feedback device characterized by the following features.

2. In the haptic presentation device described in claim 1, The control circuit, upon receiving a switching operation detected by the operation circuit, switches between two operations: changing the display position of the pointer on the screen to a new position in response to the pointer operation detected by the operation circuit, and displaying the hardness of the object at the new position on the screen. A haptic feedback device characterized by the following features.

3. In the haptic presentation device described in claim 1, The aforementioned tactile feedback circuit has a contact portion whose pressed position is displaced in response to human pressure. The control circuit calculates the displacement of the contact portion based on the hardness, the pressing force, and an adjustment coefficient corresponding to the pressing force obtained from the adjustment characteristic data, and presents a tactile sensation corresponding to the hardness of the object by shifting the pressing position of the contact portion by the amount of displacement. A haptic feedback device characterized by the following features.

4. A control method used in a haptic presentation device comprising a memory device, a screen display circuit, a haptic presentation circuit, a pressure detection circuit, a control circuit, and an operation circuit, The storage device is configured to store image data representing an object and adjustment characteristic data showing the characteristics of an adjustment coefficient corresponding to a person's pressing force, The screen display circuit is configured to display the object on the screen based on the image data, and the screen display step is as follows: The tactile presentation circuit is configured to present a person with a tactile sensation corresponding to the hardness of the object, and the tactile presentation step is as follows: The pressing force detection step is configured such that the pressing force detection circuit detects the pressing force generated when a person touches the tactile presentation circuit, A control step comprising: the control circuit being configured to change the tactile sensation presented by the tactile presentation circuit based on the hardness of the object and the pressing force detected by the pressing force detection circuit; The operation circuit includes an operation step configured to detect a pointer operation for moving the position of a pointer displayed on the screen, The storage step includes a step in which the storage device further stores the hardness at each position of the object. The control step is, The control circuit includes the steps of changing the display position of the pointer on the screen to a new position in response to the pointer operation detected by the operation circuit, and changing the tactile sensation presented by the tactile presentation circuit based on the hardness of the new position on the object, the pressing force detected by the pressing force detection circuit, and an adjustment coefficient corresponding to the pressing force obtained from the adjustment characteristic data. A control method characterized by the following:

5. In the control method described in claim 4, The tactile presentation step includes a step of displacing the pressed position of the contact portion of the tactile presentation circuit in accordance with the pressure applied by a person. The control step is, The control circuit performs the step of calculating the amount of displacement of the contact portion based on the hardness, the pressing force, and an adjustment coefficient corresponding to the pressing force obtained from the adjustment characteristic data, The control circuit provides a tactile sensation corresponding to the hardness of the object by displacing the pressing position of the contact portion by the amount of displacement. including A control method characterized by the following:

Images