Information processing device, information processing method, transmission system, and program

Abstract

The present technology relates to an information processing device, an information processing method, a transmission system, and a program that enable more efficient and accurate tactile presentation. This information processing device is provided with a cancellation method selection unit that adaptively selects a cancellation method for cancelling the transmission characteristics of a sensing tool used for tactile signal sensing, the sensing tool including a sensing tool that has contacted a tactile signal sensing target. The present technology can be applied to a tactile transmission system.

Description

Information processing device, information processing method, transmission system, and program

[0001] The present technology relates to an information processing device, an information processing method, a transmission system, and a program, and in particular to an information processing device, an information processing method, a transmission system, and a program that enable more efficient and accurate tactile presentation.

[0002] In recent years, technologies have been developed that vibrate a device worn by a user to provide the user with tactile stimulation. Here, tactile stimulation refers to a physical phenomenon that causes the user to feel a sense of touch through a vibration phenomenon or the like. In the following, the generation of tactile stimulation will be referred to as tactile presentation.

[0003] Tactile presentation technology is used in devices in various fields.

[0004] For example, in a terminal device equipped with a touch panel such as a smartphone, the touch panel vibrates in response to touch operations by the user, providing tactile stimulation to the user's fingers, making it possible to express the sensation of touching buttons, etc. displayed on the touch panel.

[0005] As another example, in a music listening device such as headphones, providing tactile stimuli in sync with the music being played can enhance the bass of the music being played. Also, in a device that provides computer games or virtual reality (VR), vibrating the controller or the like to provide tactile stimuli in response to the controller operation or content scene can enhance the user's sense of immersion in the content.

[0006] Furthermore, technologies have been developed that provide tactile stimulation to a user based on tactile signals received from an external device, such as a technology that provides tactile stimulation to a user while changing the frequency and amplitude of vibrations based on the received tactile signals (see, for example, Patent Document 1).

[0007] In addition, a technology has been proposed that estimates the transmission characteristics of the tool used for measurement from the tactile signal obtained through measurement and cancels the vibration characteristics, making it feel as if the user has touched the object themselves without using a tool when it is presented (see, for example, Patent Document 2).

[0008] JP 2016-202486 A Japanese Patent No. 4769342

[0009] However, the above-mentioned techniques have not been able to efficiently and accurately present tactile sensations. Accurate tactile sensations here refer to sensations that are close to the sensations that are originally intended to be presented to the user. More specifically, accurate tactile sensations also include the timing of the tactile sensation presentation.

[0010] For example, in the technology described in Patent Document 2, the vibration characteristics are canceled by simply convolving the inverse characteristics of the transfer characteristics on the sensing side. This type of vibration characteristics cancellation process does not take into account the characteristics of the tactile receptors that perceive the tactile presentation, and in some cases, this can result in unnecessary computational costs and processing delays, reducing the efficiency of the process when presenting the tactile presentation.

[0011] The present technology has been made in view of such circumstances, and makes it possible to present tactile sensations more efficiently and accurately.

[0012] An information processing device according to a first aspect of the present technology includes a cancellation method selection unit that adaptively selects a cancellation method for canceling the transfer characteristics of a sensing tool used to sense a tactile signal, including a sensing tool that has come into contact with a sensing target of the tactile signal.

[0013] The information processing method or program of a first aspect of the present technology includes a step of adaptively selecting a cancellation technique for canceling the transfer characteristics of a sensing tool used to sense the tactile signal, including one that has come into contact with a sensing target of the tactile signal.

[0014] In a first aspect of the present technology, a cancellation method is adaptively selected to cancel the transfer characteristics of a sensing tool used to sense a tactile signal, including a sensing tool that has come into contact with a sensing target of the tactile signal.

[0015] A transmission system according to a second aspect of the present technology is a transmission system having a transmitting device that transmits a haptic signal, a receiving device that receives the haptic signal, and a determination unit, wherein the transmitting device comprises a first cancellation method selection unit that adaptively selects a cancellation method for canceling the transfer characteristics of a sensing tool used to sense the haptic signal, including one that has come into contact with a sensing target of the haptic signal, and a first cancellation unit that performs a cancellation process on the haptic signal to cancel the transfer characteristics using the selected cancellation method; the receiving device comprises a second cancellation method selection unit that adaptively selects the cancellation method, and a second cancellation unit that performs the cancellation process; and the determination unit determines whether the first cancellation method selection unit or the second cancellation method selection unit will perform the process of selecting the cancellation method, and whether the first cancellation unit or the second cancellation unit will perform the cancellation process.

[0016] In a second aspect of the present technology, in a transmission system having a transmitting device that transmits a haptic signal, a receiving device that receives the haptic signal, and a determination unit, the transmitting device is provided with a first cancellation method selection unit that adaptively selects a cancellation method for canceling the transfer characteristics of a sensing tool used to sense the haptic signal, including one that has come into contact with a sensing target of the haptic signal, and a first cancellation unit that performs a cancellation process on the haptic signal to cancel the transfer characteristics using the selected cancellation method, and the receiving device is provided with a second cancellation method selection unit that adaptively selects the cancellation method and a second cancellation unit that performs the cancellation process, and it is determined whether the first cancellation method selection unit or the second cancellation method selection unit will perform the process of selecting the cancellation method, and whether the first cancellation unit or the second cancellation unit will perform the cancellation process.

[0017] 1 is a diagram illustrating an example of a use case using haptic technology; FIG. 2 is a diagram illustrating an example of the configuration of a haptic transmission system; FIG. 3 is a diagram illustrating an example of the configuration of a transmitting device; FIG. 4 is a diagram illustrating an example of the configuration of a receiving device; FIG. 5 is a diagram illustrating an example of the configuration of a cancellation method determination unit; FIG. 6 is a diagram illustrating an example of a cancellation method corresponding to a haptic receptor; FIG. 7 is a flowchart illustrating a transmission process; FIG. 8 is a flowchart illustrating a cancellation method selection process; FIG. 9 is a flowchart illustrating a reception process; FIG. 10 is a diagram illustrating a haptic telephone system; FIG. 11 is a diagram illustrating the cancellation of transfer characteristics; FIG. 12 is a diagram illustrating the presentation of the feel of a product; FIG. 13 is a diagram illustrating haptic feedback of a remote arm; FIG. 14 is a diagram illustrating an example of the configuration of a computer.

[0018] Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

[0019] First Embodiment Tactile Receptor and Cancellation Processing The present technology relates to a tactile transmission system having a tactile signal transmission function and for controlling a tactile presentation device.

[0020] Figure 1 shows an example of a use case using haptic technology.

[0021] The use case shown in Figure 1 is called teletouch, in which tactile stimuli sensed at a certain location are transmitted to a remote location and perceived.

[0022] In this example, a user U11 touches a predetermined object OB11 with his or her finger FG11. A sensor is attached to the finger FG11, and a haptic signal, which is data sensed by the sensor, is supplied to a transmitting device 12 via a canceling unit 11. The transmitting device 12 then transmits the haptic signal to a receiving device 13 located in a remote location.

[0023] At the remote location, the receiving device 13 supplies the tactile signal received from the transmitting device 12 to the tactile presentation device 14, causing it to present a tactile sensation. As a result, when the user U12 touches the tactile presentation device 14 connected to the receiving device 13 with his or her finger FG12, the user U12 can perceive the tactile sensation perceived by the user U11 while in a remote location.

[0024] Here, the tactile signal sensed by the finger FG11 of the user U11 is a tactile signal originating from the object OB11 convolved with the transfer characteristics of the finger FG11.

[0025] Therefore, if the tactile signal obtained by sensing is transmitted as is, the presenting user U12 will perceive the tactile signal (touch) of the object OB11 that is convoluted with the characteristics (transmission characteristics) of the finger FG11 of the sensing user U11.

[0026] This is different from the tactile sensation that is originally desired to be perceived by the user U12. In other words, the originally intended tactile sensation, that is, the accurate tactile sensation, is not provided to the user U12.

[0027] To avoid this, it is proposed that in teletouch, a tactile signal is generated in the cancellation unit 11 by canceling the transfer characteristics of the sensing tool, in this case the transfer characteristics of the finger FG11 of the user U11, and a tactile sensation is presented to the user U12 based on this tactile signal.

[0028] However, the cancellation process to cancel such transfer characteristics simply involves convolving the inverse of the transfer characteristics on the sensing side. This cancellation process does not take into account the characteristics of the tactile receptors that perceive the tactile presentation, and in some cases may result in unnecessary computational costs and processing delays.

[0029] For example, Pacinian corpuscles, a type of tactile receptor, are thought to be more sensitive to differences in the energy of the tactile waveform than to differences in the waveform itself, while Meissner's corpuscles, another tactile receptor, count the number of peaks in the tactile waveform and are thought to be more sensitive to differences in waveform.

[0030] Therefore, using a linear-phase FIR (Finite Impulse Response) filter with a long filter length for cancellation processing, which can cancel out the characteristics while preserving the tactile waveform as much as possible, is appropriate for tactile stimulation of Meissner's corpuscles, but is excessive for Pacinian corpuscles, which do not need to preserve the waveform. Furthermore, when using a linear-phase FIR filter, the long filter length increases the processing delay, compromising the low latency that is important for the tactile experience.

[0031] On the other hand, considering the characteristics of Pacinian corpuscles, small-order IIR (Infinite Impulse Response) filters are superior in terms of processing volume and delay, and also satisfy the requirement of energy conservation. However, since IIR filters do not preserve the waveform of tactile sensation, IIR filters are insufficient (not suitable) for Meissner's corpuscles.

[0032] In this way, the appropriate cancellation method (cancellation process) differs depending on the characteristics of the tactile receptors.

[0033] Therefore, this technology identifies the tactile receptors that respond to tactile signals and selects an appropriate cancellation method for each tactile receptor. This allows for the selection of a cancellation method that is sufficient for the characteristics of the tactile stimulus and applies efficient cancellation processing. In other words, it allows for more efficient and accurate tactile presentation.

[0034] <Configuration Example of Haptic Transmission System> FIG. 2 is a diagram showing a configuration example of an embodiment of a haptic transmission system to which the present technology is applied.

[0035] The tactile transmission system 41 shown in FIG. 2 includes a tactile input unit 51 , a transmitting device 52 , a receiving device 53 , a tactile presentation device 54 , and a determining unit 55 .

[0036] In the tactile transmission system 41, the environment for realizing tactile transmission is divided into a transmission environment having a transmitting device 52 that obtains and transmits the target tactile information (tactile signal), and a presentation environment having a receiving device 53 that receives the tactile information and presents the tactile information to the user.

[0037] The transmission environment includes a tactile input unit 51 that acquires tactile signals (tactile information) to be transmitted, and a transmission device 52 connected to the tactile input unit 51 .

[0038] The presentation environment also includes a receiving device 53 configured to acquire the transmitted tactile signal, and a tactile presentation device 54 connected to the receiving device 53 and presenting a tactile sensation to the user based on the received tactile signal.

[0039] The tactile input unit 51 acquires a tactile signal obtained by sensing the tactile sensation of an object, i.e., the tactile stimulation caused by the object, using a predetermined sensing tool, and inputs (supplies) the acquired tactile signal to the transmitting device 52.

[0040] The term "sensing tool" as used herein refers to a device used to sense a tactile signal, including a device that comes into contact with an object to be sensed for tactile stimulation (tactile signal) and a sensor device that performs the sensing. In particular, the sensing tool is made up of one or more objects (objects) that transmit tactile sensations and are located between the object to be sensed and the sensor device.

[0041] For example, when a sensor is in direct contact with an object, the sensor becomes the sensing tool, and when the sensor senses the tactile sensation of the object through a specified object, the sensing tool consists of the thing that touches (comes into contact with) the object and the sensor that is in contact with that object.

[0042] As a specific example, consider a case where a sensor is attached to a person's finger, and when the finger touches an object, the sensor senses the tactile stimulus caused by the object, i.e., the tactile sensation perceived by the finger. In this case, the finger that touches the object and the sensor attached to that finger are the sensing tools.

[0043] The transmitting device 52 is configured to include a computer device such as a CPU (Central Processing Unit) and a DSP (Digital Signal Processor).

[0044] The transmitting device 52 encodes the haptic signal supplied from the haptic input unit 51 in accordance with a predetermined data format, and transmits the resulting encoded haptic data to the receiving device 53 .

[0045] The transmitting device 52 and the receiving device 53 may be connected by wire or wirelessly, or may be connected by any network such as the Internet.

[0046] The receiving device 53 is configured with a computer device such as a CPU, a DSP, etc. The receiving device 53 decodes the haptic encoded data received from the transmitting device 52 and drives the haptic presentation device 54 based on the resulting haptic signal.

[0047] The tactile presentation device 54 is composed of a device that generates a tactile stimulus, and performs tactile presentation under the control of the receiving device 53. That is, the tactile presentation device 54 is driven in response to the tactile signal supplied from the receiving device 53, and generates the tactile stimulus.

[0048] As an example, the tactile presentation device 54 is configured with a vibration device such as a vibrator or an actuator. As another example, the tactile presentation device 54 may be a speaker array that presents a tactile sensation using focused ultrasound or an air cannon, a pressure presentation device that presents a tactile sensation by controlling the amount of fluid inside the device, or an electrical stimulation device that directly stimulates tactile receptors.

[0049] In addition, the tactile transmission system 41 as a whole is provided with a decision unit 55 that decides the cancellation method for the characteristics (transmission characteristics) of the sensing tool described below, and decides whether the cancellation processing using the decided cancellation method will be performed by the transmitting device 52 or the receiving device 53.

[0050] For example, the determination unit 55 determines the device that will determine (select) the cancellation method and the device that will perform the cancellation processing based on at least one of the processing resources in at least one of the transmitting device 52 and the receiving device 53, the remaining battery capacity in at least one of the transmitting device 52 and the receiving device 53, and user settings.

[0051] The determination unit 55 may be provided in an independent device different from the transmitting device 52 or the receiving device 53 , or may be provided in either the transmitting device 52 or the receiving device 53 .

[0052] <Configuration Example of Transmitting Device> Fig. 3 is a diagram showing an example of the internal configuration of the transmitting device 52. Note that, in Fig. 3, the tactile input unit 51 is also shown together with the transmitting device 52 for ease of understanding.

[0053] The transmitting device 52 shown in FIG. 3 includes an encoding unit 81 , a control unit 82 , a storage unit 83 , a communication unit 84 , a bus 85 , a cancellation method determination unit 86 , and a cancellation unit 87 .

[0054] In particular, the encoding unit 81, the control unit 82, the storage unit 83, and the communication unit 84 are connected via a bus 85, and are capable of mutual data communication (transmission and reception of data).

[0055] The haptic signal output from the haptic input unit 51 is supplied to a cancellation method determination unit 86 and a cancellation unit 87 .

[0056] The cancellation method determination unit 86 determines a cancellation method for canceling the characteristics (transfer characteristics) of the sensing tool contained (convoluted) in the haptic signal based on the haptic signal supplied from the haptic input unit 51, and supplies the result of the determination to the cancellation unit 87. Note that specific examples of determining the cancellation method will be described later.

[0057] The cancellation unit 87 performs cancellation processing on the haptic signal supplied from the haptic input unit 51 using the cancellation method determined (selected) by the cancellation method determination unit 86, and supplies the resulting haptic signal to the encoding unit 81. The cancellation processing by the cancellation unit 87 cancels the characteristics of the sensing tool that are convoluted in the haptic signal. In other words, components related to the characteristics (transfer characteristics) of the sensing tool are removed from the haptic signal.

[0058] If the cancellation section 87 does not perform cancellation processing, the cancellation method determination section 86 supplies cancellation method information indicating the determined cancellation method to the encoding section 81 as appropriate.

[0059] The encoding unit 81 is configured, for example, by a DSP, and generates haptic encoded data by encoding necessary information, such as the haptic signal supplied from the cancellation unit 87, in a predetermined data format. For example, the haptic encoded data stores, as necessary, cancellation method information obtained by the cancellation method determination unit 86 and sensing tool characteristic information indicating the characteristics of the sensing tool. Note that the sensing tool characteristic information may be any information regarding the characteristics of the sensing tool, such as information indicating the inverse characteristics of the transfer characteristics of the sensing tool.

[0060] The control unit 82 is configured with a microcomputer having, for example, a CPU, ROM (Read Only Memory), RAM (Random Access Memory), etc., and controls the entire transmitting device 52 by executing processing in accordance with a program stored in the ROM.

[0061] For example, the control unit 82 controls the communication unit 84 to transmit the tactile encoded data obtained by the encoding unit 81 to the receiving device 53, and performs data communication with an external device via the communication unit 84.

[0062] The storage unit 83 is composed of a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and is used to store various types of data in the transmission device 52.

[0063] For example, the storage unit 83 stores data necessary for control by the control unit 82. Furthermore, under the control of the control unit 82, the storage unit 83 can also store haptic encoded data obtained by the encoding unit 81.

[0064] The communication unit 84 is configured to be able to perform data communication with an external device via a network such as the Internet. The control unit 82 is capable of performing data communication with an external device such as the receiving device 53 connected to the network via the communication unit 84. In particular, the haptic encoded data obtained by the encoding unit 81 can be transmitted to an external device via the communication unit 84. In this case, the external device to which the haptic encoded data is transmitted (transmitted) is the receiving device 53.

[0065] <Configuration Example of Receiving Device> Fig. 4 is a diagram showing an example of the internal configuration of the receiving device 53. Note that, in Fig. 4, for ease of understanding, the tactile presentation device 54 is also shown together with the receiving device 53.

[0066] The receiving device 53 includes an amplifier 111 , a D / A (Digital / Analog) converter 112 , a decoder 113 , a controller 114 , a memory 115 , a communication unit 116 , a bus 117 , a cancellation method determination unit 118 , and a cancellation unit 119 .

[0067] In particular, the decoding unit 113, the control unit 114, the storage unit 115, and the communication unit 116 are connected via a bus 117, and are capable of mutual data communication (transmission and reception of data).

[0068] The control unit 114 is configured with, for example, a microcomputer having a CPU, ROM, RAM, etc., and controls the entire receiving device 53. The storage unit 115 is, for example, a storage device similar to the storage unit 83, and is used to store various data used by the control unit 114, etc.

[0069] The communication unit 116 is configured to be able to perform data communication with an external device via a network such as the Internet. For example, the communication unit 116 receives haptic encoded data transmitted by the transmitting device 52, which is the device that is the sender (transmitter) of the haptic encoded data.

[0070] The decoding unit 113 decodes the haptic encoded data input via the communication unit 116, and obtains a haptic signal or the like as the decoding result.

[0071] If the characteristics (transfer characteristics) of the sensing tool on the sensing side included in the haptic signal have not been canceled, the haptic signal obtained by the decoding unit 113 is supplied to the cancellation method determination unit 118 and the cancellation unit 119. Then, similar to the case in the transmitting device 52, the cancellation method determination unit 118 determines an appropriate cancellation method, and the cancellation unit 119 performs cancellation processing using the cancellation method. The haptic signal obtained by the cancellation processing is supplied from the cancellation unit 119 to the D / A converter 112.

[0072] In this case, cancellation method information is supplied from the decoding unit 113 to the cancellation method determination unit 118 as appropriate and used to determine the cancellation method, and sensing tool characteristic information is supplied from the decoding unit 113 to the cancellation unit 119 and used in the cancellation process.

[0073] Furthermore, if the tactile signal obtained by decoding in the decoding unit 113 has the characteristics (transmission characteristics) of the sensing tool canceled, the tactile signal is supplied from the decoding unit 113 to the D / A converter 112 via the cancellation unit 119.

[0074] The D / A converter 112 performs D / A conversion (digital / analog conversion) on the haptic signal supplied from the cancellation unit 119, and supplies the resulting analog haptic signal to the amplification unit 111. The amplification unit 111 amplifies the haptic signal supplied from the D / A converter 112, appropriately adjusts the dynamic range of the haptic signal, and then outputs the haptic signal to the haptic presentation device 54, which then presents a haptic sensation.

[0075] Although the above describes an example in which only haptic signals are transmitted, audio and video signals may be recorded along with the haptic signals, and audio and video may be presented to the haptic receiver (user) simultaneously in addition to the haptic presentation. In other words, content consisting of video, audio, and haptics may be presented.

[0076] <Configuration Example of Cancellation Method Determining Unit> The cancellation method determining unit 86 and the cancellation unit 87 described with reference to FIG. 3 will now be described in more detail.

[0077] It should be noted that the cancellation method determination unit 118 and the cancellation unit 119 in the receiving device 53 also perform the same operations as the cancellation method determination unit 86 and the cancellation unit 87, and therefore detailed description of the cancellation method determination unit 118 and the cancellation unit 119 will be omitted here.

[0078] Fig. 5 is a diagram showing an example of the configuration of the cancellation method determination unit 86. Note that in Fig. 5, in order to make the drawing easier to understand, some processing blocks of the transmission device 52 are not shown.

[0079] The cancellation method determination unit 86 includes a modal determination unit 151 , a presentation part estimation unit 152 , a receptor selection unit 153 , and a cancellation method selection unit 154 .

[0080] The modal determination unit 151 determines the modality of the tactile signal acquired by the tactile input unit 51, i.e., the type of tactile presentation (type of tactile stimulation) performed by the tactile presentation device 54, and supplies the determination result to the receptor selection unit 153.

[0081] Specifically, the modal determination unit 151 determines whether the tactile stimulus represented by the tactile signal is a vibration stimulus, a temperature stimulus, or a force stimulus, for example.

[0082] The modality may be determined by the cancellation method determination unit 86 acquiring information indicating the modality of the tactile signal handled by the tactile transmission system 41 from an external device such as the tactile input unit 51 or the receiving device 53 via the encoding unit 81 and the communication unit 84.

[0083] The modality determination result may be stored (set) in advance in the modality determination unit 151. In many cases, the modality handled by the tactile presentation device 54 is determined at the manufacturing stage, so a method of setting the modality determination result in advance is practical.

[0084] The presentation part estimation unit 152 estimates which part of the user will receive the tactile presentation (tactile stimulation) from the tactile presentation device 54, i.e., which part the tactile stimulation will be presented to, and supplies the estimation result to the receptor selection unit 153. Here, the estimation result obtained by the presentation part estimation unit 152 is presentation part information indicating the presentation part of the tactile stimulation.

[0085] For example, the cancellation method determination unit 86 may obtain information indicating the actual installation position of the tactile presentation device 54 from the receiving device 53 via the encoding unit 81 and the communication unit 84, and the installation position indicated by the information may be used as the presentation part estimation result by the presentation part estimation unit 152.

[0086] Furthermore, for example, the significance of the tactile presentation system is that the part of the user that touches the object when the tactile signal acquired by the tactile input unit 51 is sensed, that is, the part of the user that functions as the sensing tool, is often the same as the part of the user that is presented during tactile presentation by the tactile presentation device 54. Therefore, the cancellation method determination unit 86 may acquire information indicating the part of the user that is used as the sensing tool from an external device such as the tactile input unit 51 via the encoding unit 81 and the communication unit 84, and the part indicated by that information may be used as the presented part estimation result by the presented part estimation unit 152.

[0087] The receptor selection unit 153 selects one or more tactile receptors based on at least one of the tactile signal supplied from the tactile input unit 51, the modal determination result supplied from the modal determination unit 151, and the presentation part information supplied from the presentation part estimation unit 152, and supplies the selection result to the cancellation method selection unit 154. The receptor selection unit 153 selects tactile receptors stimulated by the tactile signal, more specifically, tactile receptors that respond to tactile stimulation based on the tactile signal, as tactile receptors to be used in selecting the cancellation method. The method of selecting tactile receptors will be described later.

[0088] Based on the selection result supplied from the receptor selection unit 153, the cancellation method selection unit 154 adaptively selects a cancellation method suited to the characteristics of one or more tactile receptors selected by the receptor selection unit 153. In other words, for example, a cancellation method determined in advance for the tactile receptor is selected.

[0089] As an example, cancellation methods suitable for tactile receptors are set in advance as a table, and a cancellation method shown in FIG. 6 is selected in accordance with the selected tactile receptor.

[0090] FIG. 6 shows an example of a cancellation method selection table in which tactile receptors are associated with filters as cancellation methods defined for those tactile receptors.

[0091] For example, if Merkel's disk is selected as the tactile receptor, the timing of tactile presentation is important, so a low-delay (short filter length) FIR filter is selected as the cancellation method. More specifically, a method using an FIR filter is selected as the cancellation method.

[0092] When Meissner's corpuscles are selected as tactile receptors, the number of peaks on the waveform of the tactile signal becomes important, so a linear-phase FIR filter is selected as the cancellation method.

[0093] When Pacinian corpuscles are selected as tactile receptors, the energy of the tactile signal becomes important, so an IIR filter, which excels in light weight and low delay, is selected as the cancellation method.

[0094] When the thermoreceptor is selected as the tactile receptor, the rate of change of temperature during tactile presentation becomes important, so a linear-phase FIR filter that maintains phase is selected as the cancellation method.

[0095] When the cold nerve is selected as the tactile receptor, the timing of tactile presentation becomes important, so a low-delay (short filter length) FIR filter is selected as the cancellation method.

[0096] When muscle spindles or tendon spindles are selected as tactile receptors, the timing of tactile presentation is important, so a low-delay (short filter length) FIR filter is selected as the cancellation method.

[0097] The tactile receptors are not limited to those shown in FIG. 6, but may be any other tactile receptors, such as Ruffini endings, as long as a cancellation method suitable for the characteristics of any tactile receptor is prepared.

[0098] Returning to the description of FIG. 5, the cancellation method selection unit 154 supplies cancellation method information indicating the selected cancellation method to the cancellation unit 87 or the encoding unit 81 .

[0099] For example, when cancellation processing is performed on the transmitting device 52 side, cancellation method information is supplied from the cancellation method selection unit 154 to the cancellation unit 87, and the cancellation unit 87 performs cancellation processing using the cancellation method indicated by the cancellation method information.

[0100] Furthermore, for example, if only the selection (determination) of a cancellation method is performed on the transmitting device 52 side and the cancellation process is performed on the receiving device 53 side, the cancellation method information is supplied from the cancellation method selection unit 154 to the encoding unit 81, where it is encoded and stored in the haptic encoded data. In this case, the cancellation method information can be, for example, ID information indicating the cancellation method.

[0101] <Description of Transmission Processing> The operation of the haptic transmission system 41 will be described.

[0102] First, the transmission process by the transmission device 52 will be described with reference to the flowchart of Fig. 7. Note that the case where the decision on the cancellation method and the cancellation process are performed on the transmission device 52 side will be described here.

[0103] In step S11 , the modality determination unit 151 determines the modality of the tactile signal acquired by the tactile input unit 51 , and supplies the determination result to the receptor selection unit 153 .

[0104] For example, in step S11, the modality of the tactile signal is identified based on information regarding the modality of the tactile signal handled by the tactile transmission system 41 obtained from an external device or information regarding the modality that has been stored (set) in advance.

[0105] For example, in the tactile transmission system 41, it is assumed that the modality of the tactile signal being handled will not change midway, and modality determination is performed before acquiring the tactile signal from the tactile input unit 51, but the timing of modality determination may also be after acquiring the tactile signal. Furthermore, modality determination may be performed multiple times while tactile encoded data is being transmitted.

[0106] In step S12, the cancellation unit 87 and the receptor selection unit 153 acquire the haptic signal.

[0107] That is, when the tactile input unit 51 acquires a tactile signal obtained by sensing using a sensing tool from a device or the like having a sensing tool, the tactile input unit 51 outputs the tactile signal. The cancellation unit 87 and the receptor selection unit 153 acquire the tactile signal output by the tactile input unit 51.

[0108] In step S13, the presentation part estimation unit 152 estimates the presentation part of the tactile stimulation based on information indicating the installation position of the tactile presentation device 54 and information indicating the part of the user that functioned as a sensing tool, obtained from external devices such as the receiving device 53 and the tactile input unit 51.

[0109] The presentation part estimation unit 152 supplies the presentation part information obtained as the presentation part estimation result to the receptor selection unit 153 .

[0110] In step S14, the receptor selection unit 153 selects a tactile receptor based on the tactile signal acquired in step S12, the modal determination result supplied from the modal determination unit 151, and the presentation part information supplied from the presentation part estimation unit 152.

[0111] For example, in step S14, a tactile receptor that responds to the tactile presentation based on the tactile signal acquired in step S12 is selected. The receptor selection unit 153 supplies the selection result of the tactile receptor to the cancellation method selection unit 154.

[0112] In step S15, the cancellation method selection unit 154 adaptively selects a cancellation method based on the selection result supplied from the receptor selection unit 153, and supplies cancellation method information indicating the selection result to the cancellation unit 87.

[0113] In step S16, the cancellation unit 87 performs cancellation processing on the haptic signal acquired in step S12 based on the cancellation method information supplied from the cancellation method selection unit 154, and supplies the resulting haptic signal to the encoding unit 81.

[0114] In step S16, a cancellation process is performed using the cancellation method indicated by the cancellation method information. Specifically, for example, the haptic signal is filtered using a filter indicated by the cancellation method information.

[0115] This removes (cancels) from the tactile signal the components related to the transfer characteristics of the sensing tool, i.e., the transfer characteristics of the tactile signal in the sensing tool.

[0116] In step S17 , the encoding unit 81 encodes the haptic signal supplied from the cancellation unit 87 and supplies the resulting encoded haptic data to the communication unit 84 .

[0117] In step S18, the communication unit 84 transmits the haptic encoded data supplied from the encoding unit 81 to the receiving device 53, which is the destination of the haptic encoded data.

[0118] In step S19, the transmitting device 52 determines whether to end the process of transmitting the haptic signal. For example, the transmitting device 52 determines to end the process when an external device provides information indicating that haptic sensing has ended, or when a user receiving the haptic presentation instructs the end of the process.

[0119] If it is determined in step S19 that the process is not yet finished, the process then returns to step S12, and the above-described process is repeated.

[0120] In this case, the next tactile signal is acquired, and the process of outputting (transmitting) the tactile encoded data continues. This processing flow allows an appropriate cancellation method to be selected at any time, even when the tactile receptors stimulated by the tactile signal change over time, and the transfer characteristics of the sensing tool are appropriately canceled using that cancellation method.

[0121] On the other hand, if it is determined in step S19 that the processing is to be ended, each unit of the transmitting device 52 stops the processing it is currently performing, and the transmission processing ends.

[0122] As described above, the transmitting device 52 selects an appropriate cancellation method based on the results of modality determination, etc., and performs cancellation processing using the selected cancellation method.

[0123] In this way, tactile sensations can be presented more efficiently and accurately.

[0124] In other words, by selecting a cancellation method that is suitable for the characteristics of the tactile receptors that respond to tactile stimuli by tactile signals, the transmission characteristics of the sensing tool can be canceled more efficiently, resulting in more efficient tactile presentation.

[0125] Furthermore, by canceling the transfer characteristics of the sensing tool, it is possible to provide a more accurate tactile sensation. In other words, it is possible to provide a tactile stimulus that is closer to the tactile stimulus that is originally intended to be provided to the user. Moreover, by performing cancellation processing using an appropriate cancellation method, it is possible to present a tactile sensation without any perceived delay.

[0126] If a cancellation method is selected on the transmitting device 52 side and cancellation processing is performed on the receiving device 53 side, in step S15 the cancellation method selection unit 154 supplies cancellation method information to the encoding unit 81. Furthermore, the processing of step S16 is not performed, and the cancellation unit 87 supplies the haptic signal acquired in step S12, i.e., the haptic signal that has not been subjected to cancellation processing, to the encoding unit 81 as is.

[0127] Furthermore, in step S17, the encoding unit 81 encodes the haptic signal supplied from the cancellation unit 87 and the cancellation method information supplied from the cancellation method selection unit 154, and generates haptic encoded data including the encoded haptic signal and the cancellation method information.

[0128] At this time, the cancellation unit 87 may, as appropriate, acquire sensing tool characteristic information indicating the transmission characteristics of the sensing tool from an external device such as the tactile input unit 51 and supply it to the encoding unit 81, and the encoding unit 81 may then encode the sensing tool characteristic information and store it in the tactile encoding data.

[0129] Alternatively, the cancellation unit 87 may generate a filter for canceling the transfer characteristics of the sensing tool, i.e., a filter for adding the inverse characteristics of the sensing tool, based on the sensing tool characteristic information and the cancellation method information, and the filter may be encoded and stored in the tactile encoded data.

[0130] Furthermore, if both the selection of the cancellation method and the cancellation process are performed on the receiving device 53 side, the processes of step S11 and steps S13 to S16 are not performed, and in step S17, the haptic signal acquired in step S12 is encoded as is and stored in the haptic encoded data. In this case, the sensing tool characteristic information may also be stored in the haptic encoded data.

[0131] In addition, the selection of the cancellation method and whether the cancellation process is performed by the transmitting device 52 or the receiving device 53 may be dynamically switched while the haptic signal (haptic encoded data) is being transmitted at each time.

[0132] In the transmission process described with reference to FIG. 7, the processes of steps S14 and S15 are performed to select a cancellation method, but more specifically, the cancellation method selection process shown in FIG. 8 is performed as the process for selecting a cancellation method.

[0133] Hereinafter, with reference to the flowchart of FIG. 8, a cancellation method selection process performed by the transmitting device 52 and corresponding to the processes of steps S14 and S15 of FIG. 7 will be described.

[0134] In step S51, the receptor selection unit 153 determines, based on the modal determination result supplied from the modal determination unit 151, whether or not the modal indicated by the determination result is vibration (vibration stimulation).

[0135] If it is determined in step S51 that the haptic signal is a vibration, the process then proceeds to step S52. In this case, the haptic signal is a signal with a vibration waveform that indicates the vibration detection result (sensing result), i.e., a signal for providing a vibration stimulus.

[0136] In step S52, the receptor selection unit 153 estimates the frequency and energy of the vibration based on the haptic signal by known frequency analysis and energy calculation based on the haptic signal acquired in step S12 of Fig. 7. Here, for example, the frequency (vibration frequency) that is the main component of the haptic signal and the energy at the vibration frequency of the haptic signal are estimated.

[0137] In step S53, the receptor selection unit 153 estimates the response of each tactile receptor to vibration based on the estimation result obtained in step S52.

[0138] For example, tactile receptors that respond to vibration include Merkel's disks, Meissner's corpuscles, and Pacinian corpuscles, and it is known that these tactile receptors differ in their ease of response to vibration frequencies.

[0139] The receptor selection unit 153 stores in advance a threshold table in which frequencies are associated with vibration detection thresholds for the tactile receptors of Merkel's disks, Meissner's corpuscles, and Pacinian corpuscles.

[0140] The vibration detection threshold for each frequency here is the lowest tactile signal energy at which the tactile receptor responds to vibrations of that frequency (vibration stimulus), i.e., the energy at which the tactile receptor begins to respond to vibration. Therefore, if the energy of the vibration frequency of the tactile signal is equal to or greater than the vibration detection threshold, when a tactile sensation is presented based on that tactile signal, the tactile receptors of the user who receives the tactile sensation will respond (react) to the vibration, and the user will perceive the vibration stimulus. The vibration detection threshold differs for each tactile receptor and each vibration frequency.

[0141] The receptor selection unit 153 estimates the degree to which each tactile receptor, i.e., Merkel's disks, Meissner's corpuscles, and Pacinian corpuscles, responds to vibrations based on the tactile signal based on the estimated vibration frequency and energy of the tactile signal and the vibration detection threshold of the vibration frequency of each tactile receptor. In other words, the response degree of the tactile receptor is estimated.

[0142] In step S54, the receptor selection unit 153 selects one or more tactile receptors from among Merkel's disks, Meissner's corpuscles, and Pacinian corpuscles based on the response degree estimation result in step S53, and supplies the selection result to the cancellation method selection unit 154.

[0143] For example, possible methods for selecting a tactile receptor include selecting one tactile receptor that responds most (most dominantly) to a tactile signal, or selecting one or more tactile receptors that respond significantly to a tactile signal.

[0144] For example, if a method of selecting one tactile receptor that responds most is adopted, the receptor selection unit 153 selects the tactile receptor with the lowest vibration detection threshold for the vibration frequency estimated in step S52.

[0145] Furthermore, for example, when a method of selecting one or more tactile receptors that respond significantly is adopted, the receptor selection unit 153 selects all tactile receptors whose vibration frequency energy estimated in step S52 is equal to or greater than the vibration detection threshold.

[0146] In addition, as the vibration detection threshold for each frequency of each tactile receptor, i.e., threshold table, a single representative template may be used, or different ones may be used for each user receiving the tactile presentation, each part receiving the tactile presentation (presentation part), or each combination of user and presentation part.

[0147] Each user has a different perception of a tactile stimulus because of differences in age, race, sex, height, weight, etc. For example, it is conceivable that the sensitivity to a certain tactile stimulus may differ depending on the age of the user.

[0148] Therefore, the receptor selection unit 153 may acquire user profile information, which is information about the user, and information indicating the presentation area by some means, and a threshold table prepared in advance for the user profile information and presentation area may be used to select the tactile receptor.

[0149] As an example, if the receptor selection unit 153 holds a threshold table for each presentation part, the receptor selection unit 153 reads out and uses the threshold table corresponding to the presentation part indicated by the presentation part information supplied from the presentation part estimation unit 152.

[0150] Furthermore, if the threshold table differs for each user profile, which is information about the user himself / herself, such as the user's age, the cancellation method determination unit 86 can acquire user profile information from an external device such as the receiving device 53 via the encoding unit 81 and the communication unit 84.

[0151] Additionally, in step S54, the method of selecting the tactile receptor may be dynamically switched (determined) depending on at least one of the processing resources in at least one of the transmitting device 52 and the receiving device 53, the remaining battery power in at least one of the transmitting device 52 and the receiving device 53, user settings, external instructions from the determination unit 55, etc.

[0152] In step S55, the cancellation method selection unit 154 selects a cancellation method corresponding to the tactile receptor indicated by the selection result supplied from the receptor selection unit 153, and supplies cancellation method information indicating the selection result to the cancellation unit 87.

[0153] For example, if the Pacinian corpuscles are selected as the tactile receptors in step S54, the cancellation method selection unit 154 selects an IIR filter as the cancellation method. Also, if multiple tactile receptors are selected by the receptor selection unit 153, a cancellation method is selected for each of the multiple tactile receptors.

[0154] When the cancellation method selection unit 154 selects the cancellation method (filter) determined for the tactile receptor, the cancellation method selection process ends.

[0155] In this case, in step S16 of FIG. 7, cancellation processing is performed using the cancellation method (filter) selected in step S55.

[0156] For example, if an IIR filter is selected as the cancellation method, the cancellation unit 87 generates an IIR filter that adds the inverse characteristic of the transfer characteristic indicated by the sensing tool characteristic information, i.e., an IIR-type inverse filter of the sensing tool. The cancellation unit 87 then performs a cancellation process by convolving the generated IIR filter with the haptic signal.

[0157] If it is determined in step S51 that the modal is not vibration, the process proceeds to step S56.

[0158] In step S56, the receptor selection unit 153 determines, based on the modal determination result supplied from the modal determination unit 151, whether the modal indicated by the determination result is temperature (thermal stimulus).

[0159] If it is determined in step S56 that the tactile signal is temperature, then the process proceeds to step S57. In this case, the tactile signal is a signal indicating the temperature sensed by the sensing tool, i.e., a signal for providing a thermal stimulus.

[0160] In step S57, the receptor selection unit 153 appropriately uses the sensor characteristics of the temperature sensor serving as a known sensing tool to estimate the temperature indicated by the tactile signal acquired in step S12 of Fig. 7. For example, if the tactile signal acquired by the tactile input unit 51 is a voltage value corresponding to temperature, the temperature indicated by the tactile signal is estimated from that voltage value.

[0161] In step S58, the receptor selector 153 compares the temperature estimated in step S57 with a predetermined threshold value.

[0162] In step S59, the receptor selection unit 153 selects either the warm nerve or the cold nerve tactile receptor depending on the result of the threshold processing in step S58, i.e., the result of comparing the temperature indicated by the tactile signal with the threshold, and supplies the selection result to the cancellation method selection unit 154.

[0163] For example, the threshold used in step S58 is set to 35°C, which is the standard human body surface temperature, and by comparing with this threshold, it is determined whether the thermal stimulus is a warm stimulus or a cold stimulus.

[0164] In this case, for example, if the temperature indicated by the tactile signal is above a threshold value (35°C), the warm nerve is selected as the tactile receptor, and if the temperature indicated by the tactile signal is below the threshold value (35°C), the cold nerve is selected as the tactile receptor.

[0165] It is known that warm nerves are distributed relatively sparsely on the body surface. Therefore, depending on the presentation part indicated by the presentation part information from the presentation part estimation unit 152, it may be determined that the warm nerves will not be stimulated even by the warm stimulation. In other words, depending on the comparison result with the threshold and the presentation part information, it is possible to prevent any tactile receptors from being selected in step S59.

[0166] In such a case, the tactile presentation device 54 may perform processing that does not provide tactile presentation or significantly reduces the accuracy of tactile presentation. In this case, for example, by setting the tactile signal output from the cancellation unit 87 to a zero signal or by preventing the tactile signal from being included in the tactile encoded data, tactile presentation will not be performed substantially or actually in the tactile presentation device 54. Furthermore, by performing appropriate processing on the tactile signal in the cancellation unit 87, it is possible to reduce the accuracy of tactile presentation.

[0167] In step S60, the cancellation method selection unit 154 selects a cancellation method corresponding to the tactile receptor indicated by the selection result supplied from the receptor selection unit 153, and supplies cancellation method information indicating the selection result to the cancellation unit 87.

[0168] For example, if a thermoreceptor is selected as the tactile receptor, an FIR filter is selected as the cancellation method. When the cancellation method selection unit 154 selects the cancellation method (filter) determined for the tactile receptor, the cancellation method selection process ends.

[0169] In this case, in step S16 of FIG. 7, cancellation processing is performed using a filter (cancellation method) suited to the characteristics of the tactile receptor selected in step S59.

[0170] If it is determined in step S56 that the modal is not temperature, i.e., if it is determined that the modal is force (force stimulus), then the process proceeds to step S61. In this case, the tactile signal is a signal indicating the force (pressure) sensed by the sensing tool, i.e., a signal for applying a force stimulus.

[0171] In step S61, the receptor selection unit 153 selects muscle spindles and tendon spindles, which are force receptors, as tactile receptors regardless of the nature of the tactile signal from the tactile input unit 51, and supplies the selection result to the cancellation method selection unit 154.

[0172] In step S62, the cancellation method selection unit 154 selects a cancellation method corresponding to the tactile receptor indicated by the selection result supplied from the receptor selection unit 153, and supplies cancellation method information indicating the selection result to the cancellation unit 87.

[0173] For example, an FIR filter is selected as the cancellation method for muscle spindles and tendon spindles selected as tactile receptors. When the cancellation method selection unit 154 selects the cancellation method (filter) determined for the tactile receptors, the cancellation method selection process ends.

[0174] In this case, in step S16 of FIG. 7, cancellation processing is performed using an FIR filter (cancellation method) suited to the characteristics of the tactile receptor selected in step S61.

[0175] In this way, the transmitting device 52 selects a cancellation method based on at least one of the modal determination result, the presentation part estimation result, and the haptic signal, i.e., the characteristics of the haptic signal obtained by analyzing the haptic signal, etc. Note that, although vibration, temperature, and force stimulation have been given as examples of modalities, examples of modalities are not limited to these.

[0176] This allows the transfer characteristics of the sensing tool to be cancelled more efficiently in accordance with the characteristics of the tactile receptors that respond to tactile stimuli generated by tactile signals, thereby enabling more efficient tactile presentation.

[0177] <Description of Reception Processing> The reception processing by the reception device 53 will be described with reference to the flowchart of Fig. 9. Note that here, a case will be described in which the cancellation method is determined and the cancellation processing is performed on the reception device 53 side.

[0178] In step S101 , the cancellation method determination unit 118 determines the modality of the haptic signal acquired by the haptic input unit 51 .

[0179] In step S101, the same processing as in step S11 in Fig. 7 is performed. That is, the modality of the haptic signal is identified based on information regarding the modality of the haptic signal handled by the haptic transmission system 41, which is acquired from an external device, for example, or information regarding the modality that is stored (set) in advance.

[0180] In step S102, the communication unit 116 receives the haptic encoded data transmitted by the transmitting device 52 (the device from which the data was transmitted) in step S18 of FIG.

[0181] In step S103 , the decoding unit 113 decodes the haptic encoded data supplied from the communication unit 116 , and supplies the resulting haptic signal to the cancellation method determination unit 118 and the cancellation unit 119 .

[0182] In addition, if the tactile encoded data includes sensing tool characteristic information, i.e., if the sensing tool characteristic information is obtained by decoding, the decoding unit 113 supplies the sensing tool characteristic information to the cancellation unit 119 in addition to the tactile signal.

[0183] In step S104, the cancellation method determination unit 118 estimates the site to which the tactile stimulus is to be presented.

[0184] In step S104, the same process as in step S13 in Fig. 7 is performed. That is, for example, the cancellation method determination unit 118 estimates the presented part based on information indicating the attachment position of the tactile presentation device 54 and information indicating the part of the user that functioned as a sensing tool, which information is acquired from an external device such as the tactile presentation device 54, the transmission device 52, or the tactile input unit 51.

[0185] In step S105, the cancellation method determination unit 118 selects a tactile receptor based on the tactile signal obtained in step S103, the modal determination result obtained in step S101, and the presentation part information estimated in step S104.

[0186] In step S106, the cancellation method determination unit 118 adaptively selects a cancellation method determined for the tactile receptor selected in step S105, and supplies cancellation method information indicating the selection result to the cancellation unit 119.

[0187] In steps S105 and S106, the same processing as in steps S14 and S15 in Fig. 7 is carried out. That is, the same processing as the cancellation method selection processing described with reference to Fig. 8 is carried out to select a cancellation method.

[0188] In step S107 , the cancellation unit 119 performs cancellation processing on the haptic signal supplied from the decoding unit 113 based on the cancellation method information supplied from the cancellation method determination unit 118 .

[0189] For example, when sensing tool characteristic information is included in the haptic encoded data and is supplied from the decoding unit 113, the cancellation unit 119 generates a filter for canceling the transfer characteristics of the sensing tool based on the sensing tool characteristic information and the cancellation method information. The cancellation unit 119 then performs a cancellation process in which the generated filter is convolved with the haptic signal, thereby obtaining a haptic signal in which the transfer characteristics of the sensing tool have been canceled.

[0190] If a filter for canceling the transfer characteristics of the sensing tool is stored in the haptic encoded data, the cancellation process is performed using that filter. Alternatively, the cancellation unit 119 may acquire sensing tool characteristic information from an external device such as the haptic input unit 51 via the decoding unit 113 and the communication unit 116, and generate a filter to be used in the cancellation process.

[0191] In step S108, the cancellation unit 119 outputs the haptic signal obtained in step S107 to the haptic presentation device 54 via the D / A converter 112 and the amplifier 111, causing the haptic presentation device 54 to present a haptic sensation based on the haptic signal. As a result, the haptic presentation device 54 presents a haptic sensation to the presentation area of ​​the user.

[0192] In step S109, the receiving device 53 determines whether to end the process of receiving the haptic signal. For example, the receiving device 53 determines to end the process when it receives information from the transmitting device 52 indicating that the transmission of the haptic signal is to be ended, when it receives information from an external device indicating that tactile sensing has ended, or when it receives an instruction to end the process from the user or the like who is receiving the haptic presentation.

[0193] If it is determined in step S109 that the process is not yet finished, the process then returns to step S102, and the above-described process is repeated.

[0194] In this case, the next time haptic encoded data is received, and the process of outputting the haptic signal continues.

[0195] On the other hand, if it is determined in step S109 that the process is to be ended, each unit of the receiving device 53 stops the process it is currently performing, and the receiving process ends.

[0196] In this way, the receiving device 53 selects an appropriate cancellation method based on the results of modality determination, etc., and performs cancellation processing using the selected cancellation method. In this way, it is possible to present a haptic sensation more efficiently and accurately.

[0197] In addition, if a cancellation method is selected on the transmitting device 52 side and the cancellation processing is performed on the receiving device 53 side, in step S103, a haptic signal and cancellation method information are obtained by decoding, and the haptic signal and cancellation method information are supplied to the cancellation unit 119.

[0198] In this case, the processes of step S101 and steps S104 to S106 are not performed, and in step S107, cancellation processing is performed based on the haptic signal obtained by decoding and the cancellation method information. That is, if the haptic encoded data includes cancellation method information, cancellation processing is performed based on the cancellation method information.

[0199] In addition, if sensing tool characteristic information or a filter for canceling the transfer characteristics of the sensing tool is stored in the haptic encoded data, the sensing tool characteristic information or filter is used in the cancellation process as appropriate.

[0200] Also, for example, if both the selection of the cancellation method and the cancellation processing are performed on the transmitting device 52 side, the processing of step S101 and steps S104 to S107 is not performed, and in step S108, the tactile signal obtained by decoding is output as is to the tactile presentation device 54.

[0201] According to the above-described tactile transmission system 41, an appropriate method for canceling the characteristics of the sensing tool of the tactile signal can be selected based on the tactile receptor selected according to the modality of the tactile signal, the tactile signal (the characteristics of the tactile signal), the presentation area, the user profile, etc.

[0202] In conventional tactile transmission systems, the same cancellation method is used regardless of which tactile receptor is stimulated, which results in unnecessary processing costs for the cancellation process and unnecessary delays.

[0203] In contrast, the tactile transmission system 41 to which the present technology is applied adaptively selects a cancellation method according to the tactile receptor to be stimulated, thereby selecting a cancellation method that is necessary and sufficient for the nature of the tactile stimulation, and performing efficient cancellation processing. In other words, tactile sensations can be presented efficiently.

[0204] Although the above description has been given of an example in which one tactile signal is acquired by the tactile input unit 51, multiple tactile signals may be acquired simultaneously by the tactile input unit 51. Even in such a case, the above-described processing is performed for each acquired tactile signal.

[0205] For example, an example of simultaneously acquiring multiple tactile signals is one in which a tactile signal for each modal is acquired, such as simultaneously acquiring a tactile signal for the modal "force" and a tactile signal for the modal "temperature." Furthermore, even when there are multiple presentation sites, multiple tactile signals may be acquired simultaneously, such as for each presentation site.

[0206] <Application Example 1 of the Present Technology> A specific application example of the present technology and a specific example of selection of a cancellation method and cancellation processing will be described with reference to FIGS. 10 to 13 .

[0207] 10 to 13, parts corresponding to those in Figures 2 to 5 are denoted by the same reference numerals, and their explanation will be omitted where appropriate. Also, in Figures 10 to 13, in order to make the drawings easier to see, some processing blocks in the transmitting device 52 and the receiving device 53 are not shown.

[0208] FIG. 10 shows an example in which a tactile force stimulus perceived by a user U71 is transmitted to a remote location, and another user U72 is made to perceive the hardness of an object OB71 via a tactile presentation device 54.

[0209] Here, a system corresponding to the tactile transmission system 41 and consisting of the tactile input unit 51, the transmitting device 52, and the receiving device 53 will be referred to as a tactile telephone system 201.

[0210] The tactile telephone system 201 is a system for transmitting tactile sensations to a remote location, just like a telephone for transmitting voice to a remote location. The tactile presentation device 54 may also be included in the tactile telephone system 201.

[0211] First, the user U71 touches the object OB71 with his / her finger FG71, which generates a haptic signal as a force, and the user U71 perceives the tactile sensation through the finger FG71.

[0212] A tactile sensor 202, such as a strain gauge sensor, is attached to the finger FG71, and a tactile signal acquired by this tactile sensor 202 is input to a tactile telephone system 201. That is, a tactile signal is acquired from the tactile sensor 202 by the tactile input unit 51, and the acquired tactile signal is transmitted to a receiving device 53 via a transmitting device 52.

[0213] The tactile signal transmitted to the remote location by the tactile telephone system 201 is presented to the finger FG72 of the user U72 via the tactile presentation device 54.

[0214] The tactile telephone system 201 is assumed to be designed to handle tactile signals, which are force signals, using a set of the tactile sensor 202 and the tactile presentation device 54.

[0215] The operation of the tactile telephone system 201 will now be described.

[0216] First, the transmitting device 52 acquires a haptic signal via the haptic input unit 51. The haptic signal acquired by the transmitting device 52 is input to a cancellation method determination unit 86 within the transmitting device 52.

[0217] In the cancellation method determination unit 86, the modality of the tactile sensation is first determined in the modal determination unit 151. However, since the tactile telephone system 201 is based on the premise of handling force signals from the beginning, the preset modal determination result "force" is used.

[0218] Next, the presentation part estimation unit 152 estimates (specifies) the presentation part from information indicating the attachment position of the tactile sensor 202. This is because, considering the use of a "tactile telephone," it is reasonable that the tactile (force) sensing position and the tactile presentation position (presentation part) are the same.

[0219] The mounting position of the tactile sensor 202 may be set in advance at the time of design, as in the case of the modal, or may be input into the tactile telephone system 201 by the user U71.

[0220] Next, the tactile receptors that respond predominantly are selected by receptor selection unit 153. Here, since the modal is force (force stimulation), the tactile signal is not analyzed, and the tactile receptors are assumed to be muscle spindles and tendon spindles.

[0221] Finally, a cancellation method is determined (selected) in accordance with the selected tactile receptor in the cancellation method selection unit 154. Here, a low-delay FIR filter is selected as the cancellation method to stimulate muscle spindles and tendon spindles, which have excellent timing discrimination.

[0222] This completes the selection of the cancellation method.

[0223] Next, the transfer characteristics of the finger FG71 and tactile sensor 202 functioning as the sensing tool are cancelled by the cancellation unit 87 using the cancellation method that has actually been selected.

[0224] The effect of this cancellation will be described with reference to FIG.

[0225] For example, as shown in the upper part of the figure, if the tactile signal obtained from object OB71 is X, the tactile sensation perceived by user U71 through finger FG71 and tactile sensor 202 can be expressed as HX, where H is the transmission characteristic of finger FG71 and tactile sensor 202.

[0226] This tactile signal HX is input to the tactile telephone system 201, and further passes through the tactile presentation device 54 and the transfer characteristic G of the finger FG72 of the user U72, and ultimately the user U72 perceives a tactile signal GHX.

[0227] On the other hand, the tactile signal obtained when user U72 touches object OB71 via tactile presentation device 54 and user U72's finger FG72 is GX, and GHX is "the tactile sensation of user U72 touching object OB71 via his / her own finger FG72, tactile presentation device 54, user U71's finger FG71, and tactile sensor 202."

[0228] Therefore, what is necessary here is to cancel the transfer characteristic H of the combined finger FG71 and tactile sensor 202. By doing so, the user U72 can perceive the tactile sensation of touching the object OB71 with his or her own finger FG72, albeit through the tactile presentation device 54. Note that the transfer characteristic of the tactile presentation device 54 may be canceled separately using a known method.

[0229] In the tactile telephone system 201, as shown at the bottom of the figure, the transfer characteristic H of the finger FG 71 and the tactile sensor 202 is cancelled by the selected "low-delay FIR filter."

[0230] A known method can be used to cancel the convoluted transfer characteristic H from a given signal using a low-delay FIR filter.

[0231] For example, one specific method for canceling the transfer characteristic H may involve storing the force response characteristics of a standard human index finger and a tactile sensor in advance, and creating an inverse filter from the characteristics obtained by truncating the obtained force response characteristics (time response) over a short period of time. Furthermore, the transfer characteristic H may be measured by providing a calibration step before acquiring the tactile signal, or may be selected from multiple preset patterns based on a user profile.

[0232] Through the above procedure, a haptic signal X is obtained in which the transfer characteristic H of the sensing tool has been cancelled, and this haptic signal X is transmitted to the receiving device 53 and presented to the user U72 via the haptic presentation device 54. In this case, the user U72 perceives a haptic signal GX.

[0233] In this way, a tactile telephone system 201 can be constructed in which the other user U72 feels as if he or she has perceived the tactile sensation that was originally perceived by the user U71.

[0234] Furthermore, the tactile telephone system 201 adaptively selects a processing method according to the characteristics of the tactile receptors being stimulated, thereby minimizing system processing delays. This allows, for example, when the tactile telephone system 201 is combined with video, to enable user U72 to move his / her finger FG72 in accordance with the movement of user U71's finger FG71, thereby enabling user U72 to perceive tactile sensations with a greater sense of immersion. In this case, the video and tactile sensations can be synchronized with low delay, thereby preventing changes in tactile perception due to delays.

[0235] Although the example of perceiving force as a tactile sensation has been described here, a system can be realized with a similar configuration even if the tactile sensation is vibration or temperature.

[0236] The above-described tactile telephone system 201 is characterized by selecting a tactile receptor according to the modality and presentation location of the tactile signal, determining a cancellation method according to the selected tactile receptor, and canceling the transmission characteristics of the sensing tool that are convoluted in the tactile signal using the determined cancellation method.

[0237] <Application Example 2 of the Present Technology> FIG. 12 shows an example of presenting the feel of a product in online shopping.

[0238] That is, as shown in FIG. 12, consider an example in which a seller touches an item OB91 with an arm 241 in online shopping, allowing a buyer in a remote location to sense the texture of the item OB91.

[0239] In this case, the characteristic (transfer characteristic) to be cancelled is the transfer characteristic of the arm 241 including the sensor, that is, the transfer characteristic of the arm 241 and the tactile sensor 242, and the modal of the tactile signal to be handled is vibration (vibration stimulus).

[0240] In this example, the transmitting device 52 only determines the cancellation method, and the actual cancellation process is performed on the receiving device 53. In this example, the receiving device 53 is made to perform the cancellation process, which places a heavier load on the device, and this is effective when the receiving device 53 has more leeway in resources such as processing capacity and battery than the transmitting device 52.

[0241] First, the arm 241 touches the product OB91, which generates a tactile signal as a vibration. This tactile signal is acquired by the tactile sensor 242 provided on the arm 241, and the acquired tactile signal is input to the transmission device 52 via the tactile input unit 51.

[0242] The haptic signal input to the transmitting device 52 is input to the cancellation method determination unit 86, which determines the cancellation method using the method described above.

[0243] Specifically, the modalities are identified (determined) as vibrations from metadata that is set in advance in the system.

[0244] Next, since the tactile stimulus based on the tactile signal, i.e., the modal is identified as vibration, the frequency (vibration frequency) and energy that are the main components of the tactile signal, which is a vibration signal, are estimated using a known frequency analysis method.

[0245] Based on the frequency estimated by frequency analysis, tactile receptors that respond predominantly to the tactile signal (vibration signal) are selected.

[0246] It is known that which tactile receptor responds predominantly to vibration changes depending on the frequency. Therefore, for example, a threshold frequency may be set, and a tactile receptor may be selected by comparing the frequency with the threshold.

[0247] Specifically, for example, if the vibration frequency is 10 Hz or less, Merkel's disks may be selected, if the vibration frequency is greater than 10 Hz but less than or equal to 100 Hz, Meissner's corpuscles may be selected, and if the vibration frequency is greater than 100 Hz, Pacinian corpuscles may be selected.

[0248] In this case, if the estimated vibration frequency is 200 Hz, for example, the Pacinian corpuscles will be selected as the tactile receptors.

[0249] Next, a cancellation method is selected based on the selected tactile receptor. In this case, since the Pacinian corpuscles were selected, "Use low-latency IIR filter" is selected, i.e., IIR filter is selected as the cancellation method.

[0250] Then, for example, information indicating the selected cancellation method is converted into an ID and used as cancellation method information, and furthermore, characteristic data indicating the transmission characteristics of the arm 241 and tactile sensor 242, which are the sensing tools, is used as sensing tool characteristic information.

[0251] The cancellation technique information and sensing tool characteristic information are encoded by the encoding unit 81, and the resulting haptic encoded data is transmitted to the receiving device 53. This haptic encoded data includes the haptic signal, cancellation technique information, and sensing tool characteristic information.

[0252] The characteristic data indicating the transfer characteristics of the arm 241 and tactile sensor 242, which is taken as sensing tool characteristic information, may be the impulse response or frequency response of the transfer characteristic value itself, or may be ID information that allows reference to specific characteristic data depending on the type of arm 241 or tactile sensor 242. In the latter example, even if the arm 241 or tactile sensor 242 is replaced, only the ID information as sensing tool characteristic information (characteristic data) will be different.

[0253] Upon receiving the haptic coded data including the coded cancellation technique information and sensing tool characteristic information, the receiving device 53 first performs decoding in the decoding unit 113 to obtain the cancellation technique information and sensing tool characteristic information.

[0254] For example, if the sensing tool characteristic information is ID information, characteristic data corresponding to the ID information is read from the storage unit 115 .

[0255] The cancellation unit 119 then executes a cancellation process using the cancellation method indicated by the cancellation method information based on the read characteristic data. Here, "use of a low-delay IIR filter" corresponding to the ID information as the cancellation method information is selected as the cancellation method. The haptic signal obtained by the cancellation process is output from the haptic presentation device 54.

[0256] This allows a user in a remote location to sense the feel of the product OB91 with low latency, while canceling the transfer characteristics of the sensing tool, such as the arm 241, that touches the product OB91. Furthermore, by having the receiving device 53 perform the cancellation process, which involves a high processing load, the processing load on the transmitting device 52 can be reduced. This has the advantage of reducing power consumption and making it easier to power the arm 241, which moves widely around the product OB91, on a battery.

[0257] The system shown in Figure 12 above is characterized by selecting a tactile receptor according to the characteristics (vibration frequency) of the tactile signal, encoding and transmitting information indicating the selected cancellation method and the transfer characteristics of the sensing tool, and decoding the information indicating the selected cancellation method and the transfer characteristics of the sensing tool to cancel the transfer characteristics of the sensing tool that are convoluted in the tactile signal.

[0258] <Application Example 3 of the Present Technology> Furthermore, in the example described with reference to FIG. 12, for example, both the determination of the cancellation method and the cancellation process may be performed by the receiving device 53.

[0259] In such cases, by having the receiving device 53 perform not only the cancellation process but also the decision (selection) of the cancellation method, the load on the transmitting device 52 can be further reduced, especially when the signal analysis involves relatively high-load processing such as frequency analysis.

[0260] 12, for example, the transmitting device 52 encodes and transmits only the ID information of the characteristic data indicating the transmission characteristics of the arm 241, which is the sensing tool, and the tactile sensor 242. In other words, the tactile encoded data stores the tactile signal and the ID information of the characteristic data.

[0261] In the receiving device 53, the cancellation method determination unit 118 performs modal determination, presentation part estimation, tactile receptor selection, and cancellation method selection, and the cancellation unit 119 performs cancellation processing. That is, the receiving device 53 performs all of the cancellation method selection and cancellation processing.

[0262] <Application Example 4 of the Present Technology> As shown in FIG. 2 , the haptic transmission system 41 may be provided with a determination unit 55 that determines whether the determination of the cancellation method and the execution of the cancellation process should be performed by the transmitting device 52 or the receiving device 53.

[0263] The determination unit 55 determines the cancellation method and the device that will perform each process of the cancellation process, for example, based on the degree of processing resource availability and remaining battery power of each device in the entire tactile transmission system 41, i.e., the transmitting device 52 and the receiving device 53.

[0264] That is, it is determined whether the process of determining (selecting) the cancellation method is to be performed by the cancellation method determination unit 86 (cancellation method selection unit 154) of the transmitting device 52 or the cancellation method determination unit 118 of the receiving device 53. It is also determined whether the cancellation process is to be performed by the cancellation unit 87 of the transmitting device 52 or the cancellation unit 119 of the receiving device 53.

[0265] For example, the determination unit 55 may determine the device that will determine the cancellation method and perform each process of the cancellation process based on a user setting. The user setting here refers to a setting made by the user as to whether the transmission device 52 or the reception device 53 will perform the determination of the cancellation method and the cancellation process.

[0266] Deciding on a cancellation method involves high-load processing such as frequency analysis, but the cancellation process itself is essentially a filtering process and therefore requires a high load.

[0267] Therefore, the determination unit 55 first checks whether or not there is a user setting, and if there is a user setting, determines the device that will execute each process according to the user setting, and issues instructions to the transmitting device 52 and the receiving device 53 based on the determination result.

[0268] Specifically, for example, it is assumed that the user has set the decision on the cancellation method to be made by the transmitting device 52 and the cancellation process to be performed by the receiving device 53. In such a case, the decision unit 55 instructs the transmitting device 52 to decide on the cancellation method and instructs the receiving device 53 to execute the cancellation process.

[0269] If there is no user setting, the determination unit 55 determines the device that will determine the cancellation method and the device that will perform the cancellation process as follows.

[0270] For example, when the transmitting device 52 has sufficient processing resources and remaining battery power, the determination unit 55 causes the transmitting device 52 to perform both the decision on the cancellation method and the cancellation process. For example, such a decision may be made when the transmitting device 52 has more resources than the receiving device 53, or when it is determined that the transmitting device 52 has the resources to both decide on the cancellation method and perform the cancellation process.

[0271] Similarly, for example, when the receiving device 53 has sufficient processing resources and remaining battery power, the determination unit 55 can cause the receiving device 53 to perform both the determination of the cancellation method and the cancellation process.

[0272] In this case, such a decision can be made, for example, when the receiving device 53 has more capacity than the transmitting device 52, or when it is determined that the receiving device 53 has the capacity to both determine the cancellation method and perform the cancellation process.

[0273] Furthermore, for example, when neither the transmitting device 52 nor the receiving device 53 has sufficient processing resources and remaining battery capacity, the decision unit 55 can have the transmitting device 52 decide on the cancellation method and have the receiving device 53 perform the cancellation processing.

[0274] The cancellation method and the device that will perform the cancellation process may be determined based on the modality and tactile receptors used in the tactile transmission system 41.

[0275] For example, in the tactile transmission system 41, if it is known that both force (force stimulus) and temperature (temperature stimulus) are used as modals, the cancellation method for force (force stimulus) can be determined and the cancellation process can be executed on the transmitting device 52 side, and the cancellation method for temperature (temperature stimulus) can be determined and the cancellation process can be executed on the receiving device 53 side.

[0276] Furthermore, for example, the haptic encoded data, i.e., the bit stream, transmitted from the transmitting device 52 to the receiving device 53 may store flag information indicating which device, the transmitting device 52 or the receiving device 53, will determine (select) the cancellation method or perform the cancellation process.

[0277] For example, the decision on the cancellation method is made on the transmitting device 52 side, and the cancellation process is performed on the receiving device 53 side.

[0278] In such a case, for example, the encoding unit 81 determines the cancellation method (processing for selecting a cancellation method) on the transmitting device 52 side, and generates flag information indicating that the cancellation processing will be performed on the receiving device 53 side. The encoding unit 81 then encodes the flag information, haptic signal, and cancellation method information to generate haptic encoded data. At this time, sensing tool characteristic information may also be stored in the haptic encoded data.

[0279] In the receiving device 53, the flag information is obtained by decoding, and the cancellation unit 119 performs cancellation processing on the haptic signal using the cancellation method indicated by the cancellation method information in accordance with the flag information.

[0280] Alternatively, the cancellation method determination unit 86 (cancellation method selection unit 154) of the transmitting device 52 or the cancellation method determination unit 118 of the receiving device 53 may determine (select) the cancellation method or autonomously determine the device to perform the cancellation process. In other words, the device may determine whether or not to perform the process of selecting a cancellation method on its own side, or whether or not to perform the cancellation process on its own side.

[0281] In such a case, the decision as to which device will perform each process, such as the process of selecting a cancellation method, can be made based on at least one of the processing resources of at least one of the transmitting device 52 and the receiving device 53, the remaining battery power of at least one of the transmitting device 52 and the receiving device 53, a user setting, a haptic signal, user profile information, and the modality of the haptic signal. Furthermore, the device that will determine (select) the cancellation method or perform the cancellation process may be selected for each modality.

[0282] Specifically, for example, suppose that the transmitting device 52 determines a cancellation method and performs cancellation processing, but at a certain time, the cancellation method selection unit 154 determines that the transmitting device 52 has no more processing resources or remaining battery power left.

[0283] In this case, the cancellation method selection unit 154 may be configured to have the receiving device 53 perform both the decision on the cancellation method and the cancellation processing, or the decision on the cancellation method may be made by the transmitting device 52 but the cancellation processing may be performed by the receiving device 53.

[0284] For example, when the receiving device 53 is to perform both the decision on the cancellation method and the cancellation process, the control unit 82 notifies the receiving device 53 to that effect.

[0285] This notification may be sent directly to the receiving device 53 via the communication unit 84 by the control unit 82 upon receiving the notification from the cancellation method selection unit 154, or may be sent by the control unit 82 to the receiving device 53 via the communication unit 84 and the determination unit 55, or may be sent by the encoding unit 81 by storing the above-mentioned flag information in the tactile encoding data.

[0286] Conversely, receiving device 53 (cancellation method determination unit 118) may determine the cancellation method and the device to perform the cancellation process, depending on its own processing resources and remaining battery power. Even in this case, the notification of the decision on the cancellation method and the decision on the device to perform the cancellation process may be sent directly to transmitting device 52 via communication unit 116 by control unit 114, which has received the notification from cancellation method determination unit 118, or may be sent by control unit 114 to transmitting device 52 via communication unit 116 and determination unit 55.

[0287] Alternatively, for example, the cancellation method determination unit 86 (cancellation method selection unit 154) on the transmitting device 52 side may omit cancellation processing for modalities with low priority based on the supplied haptic signal, user profile information, predetermined priorities for each modal, etc. In other words, it may be determined (selected) whether or not to perform cancellation processing for each modal of haptic signals.

[0288] For example, suppose that the cancellation method determination unit 86 obtains "force" and "temperature" as modal determination results, and selects the cold nerve as the tactile receptor for the tactile signal of the modal "temperature." Also, suppose that the transmitting device 52 is short on processing resources and battery power.

[0289] In such a case, it is conceivable that the transmitting device 52 determines the cancellation method and performs the cancellation process only for the modal "temperature" haptic signal, but does not determine the cancellation method or perform the cancellation process for the modal "force" haptic signal, because it is known that a user's sense of force becomes dull when touching something cold.

[0290] In such a case, the decision on the cancellation method for the modal "force" haptic signal and the cancellation process may be performed on the receiving device 53 side, or the accuracy of the cancellation process for the modal "force" haptic signal may be reduced.

[0291] As described above, with the present technology, the transmitting device 52, the receiving device 53, or the determination unit 55 can determine a cancellation method or a device to perform cancellation processing based on at least one of the processing resources of at least one of the transmitting device 52 and the receiving device 53, the remaining battery power of at least one of the transmitting device 52 and the receiving device 53, a user setting, the haptic signal, the user profile information, a predetermined priority for each modal, and the modality of the haptic signal. In this case, it is possible to determine a cancellation method or a device to perform cancellation processing for each modal, or to determine whether to perform cancellation processing for each modal. Furthermore, the predetermined priority for each modal may be used to select a haptic receptor to be used for selecting (determining) a cancellation method.

[0292] In this way, the cancellation method can be determined and the cancellation process can be allocated to each device depending on the processing resources of the transmitting device 52 and the receiving device 53, thereby enabling more efficient tactile presentation.

[0293] <Application Example 5 of the Present Technology> Instead of selecting one tactile receptor for one tactile signal, multiple tactile receptors may be selected for one tactile signal.

[0294] As a specific example, in the case of modal "vibration," when analyzing the characteristics of a tactile signal, the tactile signal may be divided into a first band below 10 Hz, a second band between 10 Hz and 100 Hz, and a third band above 100 Hz.

[0295] In this case, the frequency (vibration frequency) and energy are estimated for each band of the tactile signal by known frequency analysis and energy calculation.

[0296] Based on the frequency and energy of each band, it is determined whether Merkel's disks respond in the first band, whether Meissner's corpuscles respond in the second band, and whether Pacinian corpuscles respond in the third band. For each tactile receptor, a vibration detection threshold, which indicates how much energy is required for a response to a vibration frequency, is known, and this vibration detection threshold can be used to determine whether a response will occur.

[0297] Then, for those bands determined to be responsive, a cancellation technique is selected based on the tactile receptors corresponding to those bands.

[0298] For example, it may be determined that the tactile receptors respond to a first band and a second band.

[0299] In this case, for the first frequency band (10 Hz or less), the cancellation method "low-delay FIR filter" defined for the Merkel corpuscles corresponding to the first frequency band is selected. Similarly, for the second frequency band (10 Hz to 100 Hz), the cancellation method "linear-phase FIR filter" defined for the Meissner corpuscles corresponding to the second frequency band is selected.

[0300] Furthermore, for the first band, a filtering process using a low-delay FIR filter is performed as a cancellation process on the signal of the first band component of the haptic signal obtained by band division. Similarly, for the second band, a filtering process using a linear-phase FIR filter is performed as a cancellation process on the signal of the second band component of the haptic signal obtained by band division.

[0301] The post-cancellation first band signal and the post-cancellation second band signal are then synthesized, and the resulting signal is used as the final post-cancellation haptic signal.

[0302] In this example, cancellation processing is performed for each band, such as the first band and the second band, or in other words, for each tactile receptor. Furthermore, in the third band (a band of 100 Hz or higher), tactile receptors (Pacinian corpuscles) do not respond (react) to tactile signals, so the third band is considered not to contribute to tactile sensation. Therefore, the signal components of the third band obtained by band division are discarded without being cancelled or transmitted to the receiving device 53.

[0303] <Application Example 6 of the Present Technology> FIG. 13 shows an example of haptic feedback in a remote arm in which multiple modal haptic signals (haptic information) are transmitted simultaneously.

[0304] In the example shown in FIG. 13, a user U101 operates a controller 281 and moves an arm 282 at a remote location in conjunction with the operation, thereby performing some operation on an object OB101 at a remote location using the arm 282.

[0305] At this time, the force, vibration, temperature, etc. applied from the object OB101 to the arm 282 are transmitted as tactile signals and presented to the user U101, which is tactile feedback. In this example, the tactile input unit 51 acquires tactile signals such as the force applied to the arm 282, and the tactile presentation device 54 provides tactile feedback, i.e., presents a tactile sensation, to the user U101.

[0306] In such an example, if the transfer characteristic of the arm 282 is not canceled, the tactile sensation perceived by the user U101 from the object OB101 will be inaccurate.

[0307] In this case, it is only necessary to select a tactile receptor and a cancellation method for each of the identified multiple modalities, and then execute the cancellation process.

[0308] For example, when transmitting two modal haptic signals, temperature and force, a cancellation method for temperature is selected and cancellation processing is executed first.

[0309] In the case of temperature, the receptor selector 153 analyzes the tactile signal and estimates the temperature represented by the tactile signal.

[0310] The estimated temperature is compared with a preset threshold, such as 35°C, to determine (select) whether the tactile signal will stimulate the cold nerve or the warm nerve. In other words, the tactile receptor that responds to the tactile signal is selected. For example, if the temperature indicated by the tactile signal is 40°C, the warm nerve is selected because it is higher than the threshold of 35°C.

[0311] Since the selected thermal nerve is sensitive to the rate of change of temperature, a "linear phase FIR filter" that maintains the phase is selected as the cancellation method so that the information on the rate of change can be well preserved.

[0312] Finally, the temperature haptic signal is cancelled using the selected cancellation technique.

[0313] Furthermore, for the force tactile signal, similarly to the case of temperature, selection of a tactile receptor, selection (decision) of a cancellation method, and execution of cancellation processing are performed.

[0314] Through the above processing, a temperature signal and a force signal are obtained as tactile signals in which the transmission characteristics of the sensing tool have been canceled, and these tactile signals are then encoded and multiplexed in the encoding unit 81, and the obtained tactile encoded data is transmitted to the receiving device 53.

[0315] Then, on the receiving device 53 side, tactile sensations are presented to the user U101 based on two tactile signals obtained from the received tactile encoded data, allowing the user U101 to operate the object OB101 while checking its hardness and temperature.

[0316] Here, a delay in the timing of tactile presentation not only leads to a delay in judgment regarding the operation, but can also change perceptions such as hardness, which are closely related to the temporal relationship between the timing of feeling force and the kinesthetic sensation of the arm.However, in this example, these delays and changes in perception can be prevented.

[0317] As described above, this technology allows for efficient tactile presentation by selecting a cancellation method that is sufficient for the characteristics of the tactile receptors. In other words, it is possible to prevent delays and increases in processing volume due to overly strict cancellation processing, and to avoid impairing the tactile experience.

[0318] In addition, since it is possible to select whether the transmitting device 52 or the receiving device 53 will determine the cancellation method and perform the cancellation process, it is possible to select the device that will execute the process depending on the processing resources, remaining battery power, etc., so as not to strain the processing resources more than necessary.

[0319] <Example of Computer Configuration> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs that constitute the software are installed on a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose computer, for example, that can execute various functions by installing various programs.

[0320] FIG. 14 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0321] In the computer, a CPU 501 , a ROM 502 , and a RAM 503 are interconnected by a bus 504 .

[0322] An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a storage unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

[0323] The input unit 506 includes a keyboard, mouse, microphone, image sensor, tactile sensor, etc. The output unit 507 includes a display, speaker, tactile presentation device, etc. The storage unit 508 includes a hard disk, non-volatile memory, etc. The communication unit 509 includes a network interface, etc. The drive 510 drives a removable storage medium 511 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

[0324] In a computer configured as described above, the CPU 501 performs the above-described series of processes by, for example, loading a program stored in the memory unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 and executing the program.

[0325] The program executed by the computer (CPU 501) can be provided by being stored in, for example, a removable storage medium 511 such as a package medium. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0326] In a computer, a program can be installed in the storage unit 508 via the input / output interface 505 by inserting a removable storage medium 511 into the drive 510. The program can also be received by the communication unit 509 via a wired or wireless transmission medium and installed in the storage unit 508. Alternatively, the program can be installed in the ROM 502 or the storage unit 508 in advance.

[0327] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0328] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0329] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0330] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0331] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0332] Furthermore, the present technology can also be configured as follows.

[0333] (1) An information processing device comprising: a cancellation method selection unit that adaptively selects a cancellation method for canceling transfer characteristics of a sensing tool used to sense a tactile signal, including one that has come into contact with a sensing target of the tactile signal. (2) The information processing device according to (1), wherein the cancellation method selection unit selects the cancellation method based on tactile receptors stimulated by the tactile signal. (3) The information processing device according to (2), further comprising: a receptor selection unit that selects the tactile receptors to be used in selecting the cancellation method based on at least one of the tactile signal, a modality of the tactile signal, a presentation site where a tactile sensation is presented by the tactile signal, a priority set for each modality, and a user profile. (4) The information processing device according to (3), wherein the receptor selection unit selects the tactile receptor that most dominantly responds to the tactile signal as the tactile receptor to be used in selecting the cancellation method. (5) The information processing device according to (3), wherein the receptor selection unit selects one or more of the tactile receptors that significantly respond to the tactile signal as the tactile receptors to be used in selecting the cancellation method. (6) The information processing device according to any one of (3) to (5), wherein the receptor selection unit determines the selection method for the tactile receptors in accordance with at least one of processing resources in the information processing device and at least one of a destination or source device of the tactile signal, a remaining battery level in the information processing device and at least one of a destination or source device of the tactile signal, a user setting, and an external instruction. (7) The information processing device according to any one of (3) to (6), further comprising a cancellation unit that performs a cancellation process on the tactile signal to cancel the transfer characteristic by the cancellation method selected by the cancellation method selection unit. (8) The information processing device according to (7), wherein the cancellation unit performs the cancellation process for each tactile receptor.(9) The information processing device according to (7) or (8), wherein the cancellation method selection unit determines whether to perform the cancellation process in accordance with at least one of processing resources in the information processing device and at least one of a destination or source device of the haptic signal, a remaining battery level in at least one of the information processing device and at least one of the destination or source device of the haptic signal, a user setting, an external instruction, the haptic signal, the user profile, a priority set for each modal, and the modal. (10) The information processing device according to (9), wherein the cancellation method selection unit determines whether to perform the cancellation process for each modal. (11) The information processing device according to any one of (2) to (10), wherein the cancellation method selection unit determines whether to perform processing to select the cancellation method according to at least one of processing resources in the information processing device and at least one of a device that is a transmission destination or a transmission source of the haptic signal, remaining battery levels in the information processing device and at least one of a device that is a transmission destination or a transmission source of the haptic signal, a user setting, an external instruction, the haptic signal, a user profile, priorities set for each modality, and a modality of the haptic signal. (12) The information processing device according to any one of (7) to (10), wherein the information processing device is a transmitting device and further includes a communication unit that transmits haptic data including either the haptic signal or a haptic signal obtained by the cancellation processing. (13) The information processing device according to (12), wherein the haptic data includes cancellation method information indicating the cancellation method selected by the cancellation method selection unit. (14) The information processing device according to (12) or (13), wherein the haptic data includes sensing tool characteristic information relating to the transfer characteristics. (15) The information processing device according to any one of (12) to (14), wherein the haptic data includes information indicating whether the process of selecting the cancellation method is to be performed in the transmitting device or in a device to which the haptic data is transmitted.(16) The information processing device according to any one of (12) to (15), wherein the haptic data includes information indicating whether the cancellation process should be performed in the transmitting device or in a device to which the haptic data is transmitted. (17) The information processing device according to any one of (7) to (10), wherein the information processing device is a receiving device and further includes a communication unit that receives haptic data including the haptic signal. (18) The information processing device according to (17), wherein, when the haptic data includes cancellation method information indicating the cancellation method, the cancellation unit performs the cancellation process using the cancellation method indicated by the cancellation method information. (19) The information processing device according to (17) or (18), wherein, when the haptic data includes sensing tool characteristic information related to the transfer characteristics, the cancellation unit performs the cancellation process based on the sensing tool characteristic information. (20) The information processing device according to any one of (17) to (19), wherein the haptic data includes information indicating whether the processing for selecting the cancellation method is to be performed in the receiving device or in a device that has transmitted the haptic data. (21) The information processing device according to any one of (17) to (20), wherein the haptic data includes information indicating whether the processing for selecting the cancellation method is to be performed in the receiving device or in a device that has transmitted the haptic data. (22) An information processing method, wherein an information processing device adaptively selects a cancellation method for canceling transfer characteristics of a sensing tool used to sense the haptic signal, including a sensing tool that has come into contact with a sensing target of the haptic signal. (23) A program that causes a computer to execute processing including the step of adaptively selecting a cancellation method for canceling transfer characteristics of a sensing tool used to sense the haptic signal, including a sensing tool that has come into contact with a sensing target of the haptic signal.(24) A transmission system having a transmitting device that transmits a haptic signal, a receiving device that receives the haptic signal, and a determination unit, wherein the transmitting device comprises: a first cancellation method selection unit that adaptively selects a cancellation method for canceling transfer characteristics of a sensing tool used to sense the haptic signal, including one that has come into contact with a sensing target of the haptic signal; and a first cancellation unit that performs a cancellation process on the haptic signal to cancel the transfer characteristics using the selected cancellation method; the receiving device comprises: a second cancellation method selection unit that adaptively selects the cancellation method; and a second cancellation unit that performs the cancellation process; and the determination unit determines whether the first cancellation method selection unit or the second cancellation method selection unit will perform the process of selecting the cancellation method, and whether the first cancellation unit or the second cancellation unit will perform the cancellation process. (25) The transmission system according to (24), wherein the determination unit determines a device that will perform the process of selecting the cancellation method and a device that will perform the cancellation process based on at least one of processing resources in at least one of the transmitting device and the receiving device, remaining battery power in at least one of the transmitting device and the receiving device, and user settings.

[0334] REFERENCE SIGNS LIST 11 haptic transmission system, 51 haptic input unit, 52 transmitting device, 53 receiving device, 54 haptic presentation device, 55 determining unit, 81 encoding unit, 82 control unit, 84 communication unit, 86 cancellation method determining unit, 87 cancellation unit, 113 decoding unit, 114 control unit, 116 communication unit, 118 cancellation method determining unit, 119 cancellation unit

Claims

1. An information processing device comprising a cancellation method selection unit that adaptively selects a cancellation method for canceling the transfer characteristics of a sensing tool used to sense a tactile signal, including one that has come into contact with a sensing target of the tactile signal.

2. The information processing device according to claim 1, wherein the cancellation method selection unit selects the cancellation method based on tactile receptors stimulated by the tactile signal.

3. The information processing device according to claim 2, further comprising a receptor selection unit that selects the tactile receptor to be used in selecting the cancellation method based on at least one of the tactile signal, the modality of the tactile signal, the presentation area where tactile presentation is performed by the tactile signal, the priority set for each modality, and a user profile.

4. The information processing device according to claim 3, wherein the receptor selection unit selects the tactile receptor that responds most dominantly to the tactile signal as the tactile receptor to be used in selecting the cancellation method.

5. The information processing device according to claim 3, wherein the receptor selection unit selects one or more of the tactile receptors that respond significantly to the tactile signal as the tactile receptors to be used in selecting the cancellation method.

6. The information processing device described in claim 3, wherein the receptor selection unit determines the selection method for the tactile receptor in accordance with at least one of the processing resources of the information processing device and at least one of the destination or source device of the tactile signal, the remaining battery power of the information processing device and at least one of the destination or source device, user settings, and external instructions.

7. The information processing device according to claim 3, further comprising a cancellation unit that performs a cancellation process on the haptic signal to cancel the transfer characteristic using the cancellation method selected by the cancellation method selection unit.

8. The information processing device according to claim 7, wherein the cancellation unit performs the cancellation process for each of the tactile receptors.

9. The information processing device according to claim 7, wherein the cancellation method selection unit determines whether to perform the cancellation process based on at least one of processing resources in the information processing device and at least one of a destination or source device of the haptic signal, remaining battery power in the information processing device and at least one of a destination or source device of the haptic signal, user settings, external instructions, the haptic signal, the user profile, priorities set for each modal, and the modal.

10. The information processing device according to claim 9, wherein the cancellation method selection unit determines whether or not to perform the cancellation process for each modality.

11. The information processing device according to claim 2, wherein the cancellation method selection unit determines whether to perform processing to select the cancellation method depending on at least one of processing resources in the information processing device and at least one of a device that is a destination or source of the haptic signal, remaining battery power in the information processing device and at least one of a device that is a destination or source of the haptic signal, user settings, external instructions, the haptic signal, a user profile, priorities set for each modal, and the modal of the haptic signal.

12. The information processing device according to claim 7, wherein the information processing device is a transmitting device and further comprises a communication unit that transmits haptic data including either the haptic signal or the haptic signal obtained by the cancellation process.

13. The information processing device according to claim 12, wherein the haptic data includes cancellation method information indicating the cancellation method selected by the cancellation method selection unit.

14. The information processing device according to claim 12, wherein the tactile data includes sensing tool characteristic information relating to the transfer characteristics.

15. The information processing device according to claim 12, wherein the haptic data includes information indicating whether the process of selecting the cancellation method is to be performed in the transmitting device or in a device to which the haptic data is to be transmitted.

16. The information processing device according to claim 12, wherein the haptic data includes information indicating whether the cancellation process should be performed in the transmitting device or in a device to which the haptic data is to be transmitted.

17. The information processing device according to claim 7, wherein the information processing device is a receiving device and further comprises a communication unit that receives haptic data including the haptic signal.

18. The information processing device according to claim 17, wherein, when the haptic data includes cancellation method information indicating the cancellation method, the cancellation unit performs the cancellation process using the cancellation method indicated by the cancellation method information.

19. The information processing device according to claim 17, wherein, when the tactile data includes sensing tool characteristic information relating to the transfer characteristics, the cancellation unit performs the cancellation process based on the sensing tool characteristic information.

20. The information processing device according to claim 17, wherein the haptic data includes information indicating whether the process of selecting the cancellation method is to be performed on the receiving device or on the device that transmitted the haptic data.

21. The information processing device according to claim 17, wherein the haptic data includes information indicating whether the cancellation process should be performed on the receiving device or on the device that transmitted the haptic data.

22. An information processing method in which an information processing device adaptively selects a cancellation technique for canceling the transfer characteristics of a sensing tool used to sense a tactile signal, including a sensing tool that has come into contact with a sensing target of the tactile signal.

23. A program that causes a computer to execute a process including a step of adaptively selecting a cancellation method for canceling the transfer characteristics of a sensing tool used to sense a tactile signal, including a sensing tool that has come into contact with a sensing target of the tactile signal.

24. A transmission system having a transmitting device that transmits a haptic signal, a receiving device that receives the haptic signal, and a determination unit, wherein the transmitting device comprises: a first cancellation method selection unit that adaptively selects a cancellation method for canceling transfer characteristics of a sensing tool used to sense the haptic signal, including one that has come into contact with a sensing target of the haptic signal; and a first cancellation unit that performs cancellation processing on the haptic signal to cancel the transfer characteristics using the selected cancellation method; the receiving device comprises: a second cancellation method selection unit that adaptively selects the cancellation method; and a second cancellation unit that performs the cancellation processing; and the determination unit determines whether the first cancellation method selection unit or the second cancellation method selection unit will perform the process of selecting the cancellation method, and whether the first cancellation unit or the second cancellation unit will perform the cancellation processing.

25. The transmission system according to claim 24, wherein the determination unit determines the device that will perform the process of selecting the cancellation method and the device that will perform the cancellation process based on at least one of the processing resources in at least one of the transmitting device and the receiving device, the remaining battery power in at least one of the transmitting device and the receiving device, and user settings.

Images