Decoding device, decoding method, and program

Abstract

To streamline a tactile reproduction-related system by reducing the amount of data in tactile signals while ensuring tactile reproducibility.SOLUTION: There is provided a decoding device comprising a decoding unit configured to decode tactile signals of a first part and a second part of a human body encoded by an encoding unit configured to encode the tactile signal of the first part and the tactile signal of the second part of the human body in such a way that a bit allocation of the tactile signal of the second part is differentiated from that of the tactile signal of the first part.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present technology relates to a decoding device, a decoding method, and a program, and in particular to a technical field related to encoding, decoding, and transmission of tactile signals for generating tactile stimulation. [Background technology]

[0002] In recent years, applications that provide tactile stimuli using tactile presentation devices that are in contact with human skin have been used in a variety of situations. Here, "tactile presentation" refers to the generation of tactile stimuli. For example, in mobile terminals equipped with a touch panel such as smartphones, the panel (or the housing) is vibrated when the panel is touched to provide a tactile stimulus to the finger, thereby simulating the sensation of touching a button. When listening to music, some headphones incorporate a tactile presentation device into the headphone housing, emphasizing the deep bass by providing tactile stimulation in parallel with the music playback. In the fields of computer games and VR (virtual reality), there are some games that enhance the user's sense of immersion by providing interactive tactile stimuli tailored to the scene in response to the user's operations using a tactile presentation device installed in the controller. In amusement facilities, such as movie theaters and theme parks, tactile presentation devices are installed in the seats to provide tactile stimulation according to the situation, thereby improving the sense of realism of visitors.

[0003] Also, in the research and development stage, when remotely operating a robot or the like, the vibrations received by the robot or the object being operated are fed back to a controller in the operator's hand, allowing the operator to intuitively sense the situation around the robot or object and to help predict danger (e.g. disaster response robots).<http: / / www.rm.is.tohoku.ac.jp / quince_mech / #_8> ) Furthermore, in the medical field, there is research being done to improve surgical precision by providing feedback to the operator on the feel (hardness) of the endoscopic forceps touching the organ when operating a surgical robot (e.g., the Da Vinci surgical robot).<http: / / techon.nikkeibp.co.jp / article / FEATURE / 20150217 / 404460 / ?P=2> )

[0004] On the other hand, with regard to tactile presentation devices, eccentric motors (ERMs) and linear actuators (LRAs) are widely used, and many of these devices are considered to have a resonant frequency at a frequency where humans have high tactile sensitivity (around several hundred Hz) (see, for example, Patent Document 1 below).

[0005] There are also examples of efforts to enhance the sense of realism by providing multiple tactile presentation devices that are worn all over the body to provide tactile stimulation. (Example: Synesthesia Suit)<http: / / rezinfinite.com / ja / synesthesia-suit / > ) [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2016-202486 A Summary of the Invention [Problem to be solved by the invention]

[0007] Here, when preparing multiple tactile presentation devices and applying tactile stimuli to multiple parts of the human body as described above, signals representing the pattern of tactile stimuli, i.e., tactile signals, are prepared and processed for each part on multiple channels. In this case, it is desirable to increase the number of parts to which tactile stimuli are applied in order to increase the sense of realism, but it is not desirable from the viewpoint of the system configuration to increase the amount of data required for tactile reproduction.

[0008] Furthermore, when transmitting a haptic signal for haptic reproduction, particularly when transmitting wirelessly, loss of encoded data may occur due to interference on the transmission path. When data loss occurs, the receiving side requests the transmitting side to resend the data, which causes a delay in data transmission, and this transmission delay may result in a decrease in the reproducibility of the haptic sensation. Specifically, it is assumed that the haptic sensation is reproduced in synchronization with content related to other senses, such as sound or video (hereinafter referred to as "sensory content"), but when the above-mentioned transmission delay occurs in the haptic signal, the haptic stimulation cannot be given at the appropriate timing, and synchronization with other sensory content cannot be achieved, which may result in a decrease in the reproducibility of the haptic sensation.

[0009] This technology is This was made in consideration of the above circumstances, and aims to improve the efficiency of systems related to tactile reproduction by reducing the amount of tactile signal data while ensuring the reproducibility of tactile sensations. The purpose is to [Means for solving the problem]

[0010] The decoding device according to the present technology is The first part of Tactile signals and a haptic signal of the first region and a haptic signal of the second region, the haptic signal of the first region being encoded by an encoding unit that performs encoding such that a bit allocation of the haptic signal of the second region is different from a bit allocation of the haptic signal of the first region. A decoder for decoding haptic signals Equipped It is something.

[0011] This means: For example, by taking advantage of the difference in tactile sensitivity between parts of the human body, the amount of data for tactile signals is reduced by taking into account the difference in tactile characteristics depending on the part of the human body, such as by reducing the data allocation for tactile signals from parts with low tactile sensitivity. It is possible that: Effect of the Invention

[0012] According to this technology, By reducing the amount of haptic signal data while ensuring the reproducibility of haptic sensation, we aim to improve the efficiency of systems related to haptic reproduction. It is possible to plan this. [Brief description of the drawings]

[0013] [Figure 1] 1 is a diagram showing an example of the configuration of a haptic reproduction system including a decoding device according to a first embodiment of the present technology. [Diagram 2] FIG. 2 is a diagram for explaining an example of the internal configuration of an encoding device according to a first embodiment; [Diagram 3] FIG. 2 is a diagram for explaining an example of the internal configuration of a decoding device according to a first embodiment; [Figure 4]FIG. 4 is an explanatory diagram of a vibration detection threshold curve. [Diagram 5] FIG. 1 shows an example of the distribution of neural firing for each receptor. [Figure 6] FIG. 13 is an explanatory diagram of vibration detection threshold curves for each receptor. [Figure 7] FIG. 2 is a diagram illustrating an example of an amplitude range and a frequency range covered by the digitization of a haptic signal. [Figure 8] 1A to 1C are diagrams showing examples of how a tactile presentation device is attached to each part of the human body and examples of data structures of tactile signals from each part. [Figure 9] FIG. 13 is a diagram illustrating an example of differences between body parts in frequency bands in which tactile stimulation can be perceived. [Figure 10] 13 is a diagram showing an example of the structure of encoded data when the data format of the haptic signal is made different for each part. FIG. [Figure 11] FIG. 2 is a functional block diagram showing the functional configuration of an encoding device according to a first embodiment. [Figure 12] 11 is a diagram for explaining an example of format conversion of a haptic signal. FIG. [Figure 13] 1A and 1B are diagrams illustrating an example of aliasing caused by oversampling and its removal. [Figure 14] FIG. 11 is an explanatory diagram of oversampling of a haptic signal. [Figure 15] FIG. 2 is a functional block diagram showing a functional configuration of a decoding device according to a first embodiment. [Figure 16] FIG. 13 is a diagram showing an example of the configuration of a tactile reproduction system according to a second embodiment. [Figure 17] FIG. 11 is a diagram illustrating an example of the internal configuration of a transmitting device according to a second embodiment. [Figure 18] FIG. 11 is a diagram for explaining an example of the internal configuration of a receiving device in a second embodiment. [Figure 19] 13 is a diagram showing an example of an arrangement order of frames for each body part when the tactile sensitivity is determined to be highest for the hands, followed by the face and then the feet. FIG. [Figure 20] FIG. 13 is an explanatory diagram of an example of providing redundancy to haptic signals according to the priority of parts. [Figure 21] FIG. 11 is a functional block diagram showing a functional configuration of a transmission device according to a second embodiment. [Figure 22] FIG. 11 is a functional block diagram showing a functional configuration of a receiving device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, with reference to the accompanying drawings, embodiments of the present technology will be described in the following order. <1. First embodiment> [1-1. Overview of the tactile reproduction system] [1-2. Encoding device configuration] [1-3. Configuration of the Decryption Device] [1-4. Tactile reproduction method as the first embodiment] (encoding method) (Functional configuration of the encoding side) (Decoding method) (Functional configuration of the decoding side) [1-5. Summary of the first embodiment] <2. Second embodiment> [2-1. Overview of the tactile reproduction system] [2-2. Configuration of the transmitting device] [2-3. Configuration of receiving device] [2-4. Tactile reproduction method as the second embodiment] (Sender functional configuration) (Receiving side functional configuration) [2-5. Summary of the second embodiment] 3. This Technology

[0015] In this specification, each term is defined as follows. Tactile stimulation: A physical phenomenon that allows a person to perceive touch, such as a vibration phenomenon. Tactile presentation: Producing tactile stimuli. Tactile Signal: A signal representing a pattern of tactile stimulation, e.g. a signal representing a vibration waveform. Haptic recipient: The person who receives the haptic presentation. Tactile characteristics: Characteristics related to human touch. They differ depending on the part of the body (hands, face, feet, etc.). Tactile sensitivity: The subjective sensitivity of a tactile stimulus. It varies depending on the receptors and parts of the human body. High tactile sensitivity means that it is easy to perceive tactile signals. Encoded data: Data that encodes a haptic signal. Sub-concepts include stream and frame. In addition, the "tactile sensitivity" referred to here is of two types: one related to the amplitude of the tactile stimulus, and the other related to the frequency of the tactile stimulus. In this specification, unless otherwise specified, "tactile sensitivity" does not distinguish between amplitude and frequency.

[0016] <1. First embodiment> [1-1. Overview of the tactile reproduction system] FIG. 1 shows an example of the configuration of a haptic reproduction system 1 including an encoding device (see 2) and a decoding device (see 3) according to a first embodiment of the present technology. The tactile reproduction system 1 includes an encoding device 2 to which multiple tactile sensors 5 are connected, a decoding device 3 configured to be able to communicate with the encoding device 2 via a predetermined network 4, and multiple tactile presentation devices 6 connected to the decoding device 3.

[0017] The tactile sensor 5 is a part or component that senses tactile stimuli, and in this example, a vibration sensor such as a piezo pickup or an acceleration sensor is used. The tactile sensor outputs vibrations or movements as voltage changes when it is brought into contact with the sensing target, that is, the human body (or a vibrating object) in this example. In this example, each tactile sensor 5 is connected by wire to the encoding device 2, and each contact sensor 5 is attached to a different part of the human body (or a vibrating object) as the target to sense the tactile stimulation occurring at each part.

[0018] The encoding device 2 is configured with a computer device such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), and encodes the detection signals (tactile signals) from each tactile sensor 5 in accordance with a predetermined data format, and transmits the encoded tactile signals, i.e., the encoded data, to the decoding device 3 via a predetermined network 4, such as the Internet.

[0019] The decoding device 3 is configured with a computer device such as a CPU or a DSP, decodes the encoded data received via the network 4, and drives each tactile presentation device 6 based on the decoded tactile signal.

[0020] The tactile presentation device 6 is a device that generates a tactile stimulus, and in this example, a device such as a vibrator or an actuator is used. In this example, each tactile presentation device 6 is attached to a different part of the human body of the touch receiver, and is adapted to reproduce the tactile stimulus sensed by the corresponding tactile sensor 5.

[0021] The tactile reproduction system 1 in this example is configured as a system that reproduces in a recipient the tactile sensation of each part of the body perceived by a person wearing a tactile sensor 5, and is capable of handling cases where the two are located remotely. Furthermore, according to the configuration of the tactile reproduction system 1 shown in Figure 1, it is possible to reproduce the tactile sensation in approximately real time by transmitting the tactile signal obtained by sensing with the tactile sensor 5 to the decoding device 3 via the network 4.

[0022] In the example of Figure 1, the number of tactile sensors 5 and tactile presentation devices 6, i.e., the number of parts of the human body that sense and reproduce tactile stimuli, is three, but the number of tactile sensors 5 and tactile presentation devices 6 is not limited to this.

[0023] [1-2. Encoding device configuration] Fig. 2 is a diagram for explaining an example of the internal configuration of the encoding device 2. Note that Fig. 2 shows the tactile sensors 5 shown in Fig. 1 together with the example of the internal configuration of the encoding device 2. As shown in the figure, the encoding device 2 includes a plurality of amplifiers 21, a plurality of A / D converters 22, a pre-processing unit 23, an encoding unit 24, a control unit 25, a storage unit 26, a communication unit 27, and a bus . As shown in the figure, a pre-processing unit 23, an encoding unit 24, a control unit 25, a storage unit 26, and a communication unit 27 are connected via a bus 28 and are capable of data communication with one another.

[0024] The detection signal of each tactile sensor 5 is input to a corresponding amplifier 21 and adjusted to an appropriate dynamic range, and then input to a corresponding A / D converter 22 and A / D converted (analog / digital converted). Each A / D converted detection signal (that is, the tactile signal for each part) is input to a pre-processing unit 23. In the pre-processing unit 23, various types of digital signal processing such as noise removal and calibration of the sensor characteristics of the tactile sensor 5 are performed. Each haptic signal that has been subjected to signal processing by the pre-processing unit 23 is input to the encoding unit 24.

[0025] The encoding unit 24 is composed of, for example, a DSP, and encodes each input haptic signal according to a predetermined data format.

[0026] The control unit 25 is configured with a microcomputer having, for example, a CPU, a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and performs overall control of the encoding device 2 by executing processing in accordance with a program stored in the ROM. For example, the control unit 25 performs data communication with an external device via the communication unit 27. The communication unit 27 is configured to be capable of performing data communication with an external device via the network 4, and the control unit 25 is capable of performing data communication with an external device (particularly the decoding device 3 in this example) connected to the network 4 via the communication unit 27. In particular, the haptic signal encoded by the encoding unit 24 can be transmitted to the decoding device 3 via the communication unit 27.

[0027] The storage unit 26 is a comprehensive representation of storage devices such as a hard disk drive (HDD) or a solid state drive (SSD), and is used to store various types of data in the encoding device 2. For example, the storage unit 26 stores data necessary for control by the control unit 25. Also, based on the control of the control unit 25, the storage unit 26 can store an encoded haptic signal.

[0028] [1-3. Configuration of the Decryption Device] FIG. 3 is a diagram for explaining an example of the internal configuration of the decoding device 3, and shows the internal configuration of the decoding device 3 together with each of the tactile presentation devices 6 shown in FIG. The decoding device 3 includes a plurality of amplifiers 31, a plurality of D / A converters 32, a post-processing unit 33, a decoding unit , a control unit 35, a storage unit , a communication unit 37, and a bus . The post-processing unit 33, the decoding unit 34, the control unit 35, the storage unit 36, and the communication unit 37 are connected via a bus 38 and are capable of data communication with one another.

[0029] The control unit 35 is configured with, for example, a microcomputer having a CPU, ROM, RAM, etc., and performs overall control of the decoding device 3 by executing processes according to a program stored in the ROM. For example, the control unit 35 performs data communication with an external device via the communication unit 37. The communication unit 37 is configured to be capable of performing data communication with an external device via the network 4, and the control unit 35 is capable of performing data communication with an external device (particularly, in this example, the encoding device 2) connected to the network 4 via the communication unit 37.

[0030] The control unit 35 causes the communication unit 37 to input the haptic signal (the encoded haptic signal) received from the encoding device 2 to the decoding unit .

[0031] The storage unit 36 ​​collectively represents storage devices such as an HDD or SSD, and is used to store various types of data in the decryption device 3. For example, the storage unit 36 ​​stores data necessary for control by the control unit 35.

[0032] The decoding unit 34 obtains a haptic signal for each body part by decoding the encoded haptic signal in accordance with a predetermined data format. The haptic signal for each body part obtained by the decoding unit 34 is input to the post-processing unit 33.

[0033] The post-processing unit 33 performs signal processing such as calibration of the tactile presentation device 6 and predetermined filtering, as necessary, on the input tactile signals for each part.

[0034] Each tactile signal that has passed through the post-processing unit 33 is input to a corresponding D / A converter 32 for D / A conversion (digital / analog conversion), then adjusted to an appropriate dynamic range by a corresponding amplifier 31, and output to a corresponding tactile presentation device 6. This allows each tactile presentation device 6 to be driven based on the tactile signal, making it possible to generate a tactile stimulus corresponding to each part of the user.

[0035] Although only haptic signals have been mentioned above, audio signals and video signals can also be transmitted to the decoding device 3 side together with the haptic signal to provide sounds and images to the haptic receiver.

[0036] [1-4. Tactile reproduction method as the first embodiment] (encoding method) A tactile sensation reproduction method according to a first embodiment will be described below. The tactile reproduction method according to the first embodiment focuses on the tactile characteristics of humans. As a guide to human tactile sensitivity, the vibration detection threshold curve shown in Figure 4 has been reported. In Figure 4, the horizontal axis represents frequency, and the vertical axis represents the amplitude of the tactile stimulus (vibration: here, the amplitude of displacement). The vibration detection threshold curve shown in Figure 4 is an example of an experiment to determine whether or not humans can sense a vibration as touch, that is, tactile sensitivity. Humans cannot sense vibrations smaller than this curve as touch.

[0037] It is generally known that there are a number of receptors under the human skin for sensing touch. Representative receptors include Meissner's, Merkel's, Ruffini's, and Pacinian's. Meissner and Pacini are also called "FA 1" and "FA 2", respectively, with FA being an abbreviation for "Fast Adapting". Merkel and Ruffini are also called "SA 1" and "SA 2", respectively, with SA being an abbreviation for "Slow Adapting".

[0038] Figure 5 shows the distribution of neural firing in each receptor when an object is gradually pressed against the skin, held there for a while, and then released. Merkel (SA 1) is believed to be the nerve that continues firing while the object is being pressed, and is responsible for detecting the strength (displacement, pressure). Meissner (FA 1) is believed to be responsible for detecting the speed of the object until the amount of pressure applied to it becomes constant. Pacinian (FA 2) is believed to be responsible for detecting the acceleration of the object as the amount of pressure applied to it changes.

[0039] The vibration detection threshold curves for each receptor are shown in Figure 6. The curves shown in Figure 4 do not represent the characteristics of a single receptor, but rather represent the combined characteristics of the tactile sensation obtained by multiple receptors, as shown in Figure 6.

[0040] The vibration detection threshold curves shown in Figures 4 and 6 indicate that humans can sense vibrations up to about 1 kHz as tactile stimuli. In addition, although values ​​above 1 kHz are not shown in these figures, it is known that humans can actually perceive vibrations with frequencies of only a few kHz as tactile stimuli, although sensitivity drops sharply above that range.

[0041] In most conventional tactile reproduction applications, vibrations up to about 200 Hz are targeted at the highest frequency, because the highest tactile sensitivity of humans is at about 200 Hz.

[0042] However, as mentioned above, various past experiments have revealed that humans can sense vibrations up to 1 kHz as tactile stimuli, and it must be said that it is difficult to reproduce a highly realistic tactile sensation using conventional applications. For example, the vibrations felt when removing a cork from a bottle actually contain high frequencies of several kHz. If these frequencies were only reproduced up to a few hundred Hz, the resulting tactile sensation would be completely different from reality.

[0043] Therefore, in this example, the haptic signal and the characteristics of the haptic presentation device 6 are made to have a broadband of about 1 kHz, thereby enhancing the sense of reality. Specifically, in this example, a method is adopted in which a tactile stimulus such as a vibration that actually occurs is sensed to obtain a tactile signal, and a tactile sensation is presented based on the tactile signal.

[0044] In recent years, all kinds of information are digitized and used, and we are considering digitizing and handling tactile signals in the same way. The amount of digitized data can be considered as the number of bits required per unit time, that is, the bit rate. For example, the region that humans can sense in the vibration detection threshold curve shown in Figure 4 is at least 50 dB (-20 dB to 30 dB) or more on the vertical axis (vibration) and about 1000 Hz on the horizontal axis. In this example, taking into account the distribution of tactile signals that humans actually sense, we will sense signals in the range of +20 dB from the threshold curve. Specifically, as shown in FIG. 7, it is assumed that the vibration range is 70 dB (−20 dB to 50 dB).

[0045] When this signal is digitized using LPCM (Linear Pulse Code Modulation), 1 bit can express 6 dB, so 12 bits are required on the vertical axis. In order to reproduce up to 1000 Hz, twice the sampling frequency, 2000 Hz (sample / sec), is required, so the required bit rate B0 can be calculated using the following [Equation 1]. B0=12bit / sample×2000sample / sec=24kbit / sec...[Formula 1]

[0046] This value itself is very small compared to, for example, the bit rate of CDs, which is a typical format for audio signals, at 700 kbps / ch. Therefore, it seems unlikely that incorporating this haptic signal as an additional component in some system would pose a major problem.

[0047] However, as mentioned above, it is known that the bandwidth of haptic signals that humans can sense extends up to several kHz. For example, if haptic signals are reproduced up to 2000 Hz, the bit rate will be 48 kbit / sec, twice that of [Equation 1].

[0048] Furthermore, unlike vision (two eyes) and hearing (two ears), touch is present everywhere on the surface of the human body. Just considering the fingertips on both hands, there are ten locations, and if we were to handle all of these tactile signals, the bit rate would be ten times higher, at 480 kbit / sec. If we were to add each finger joint, palm, and other locations, the bit rate would increase dramatically.

[0049] Furthermore, while tactile signals are essentially one-dimensional, the physical phenomenon of vibration can be captured in three axes (x, y, z). To handle all of this, a bit rate of 1440 kbit / sec, three times faster, would be required, exceeding the 1411 kbit / sec of an audio CD.

[0050] Thus, although the bit rate for one haptic signal is not particularly large, it becomes enormous when considering the tactile sensations that humans can sense, and will undoubtedly place a heavy load on a system that handles haptic signals.

[0051] Therefore, in this embodiment, attention is focused on the difference in tactile sensitivity between different parts of the human body. The type, distribution, and sensitivity of receptors in each part of the human body vary greatly, and these characteristics are represented by models known as the "somatosensory homunculus" or "sensory dwarf" (see, for example, http: / / web2.chubu-gu.ac.jp / web_labo / mikami / brain / 32 / index-32.html). These diagrams of the "somatosensory homunculus" and "sensory dwarf" quantitatively deform each part of the human body according to the size of the area of ​​the brain responsible for processing the sense of touch in that part of the body. However, this does not directly show the differences in the type, distribution, and sensitivity of the tactile receptors in each part of the body. Furthermore, the type, distribution, and sensitivity of the tactile receptors in each part of the human body have not yet been fully investigated and elucidated. However, since it is quite possible to qualitatively and quantitatively experience the difference in tactile sensitivity between different parts of the body in daily life, such as using the hands instead of the feet for delicate tasks that require touch, and the roughness of an object being difficult to sense on the belly or back as it is on the hand, it is easy to imagine that not all parts of the body have exactly the same tactile characteristics. As an easy-to-understand example, fingers have fingerprints, and there have been many reports and verifications that show that this increases tactile sensitivity. Given that there are no fingerprints on the back or belly, it is thought that it would be difficult to cover tactile signals with a single encoding method.

[0052] It is also widely known that when humans concentrate on one sense, their sensitivity to the other senses weakens. Specifically, when humans evaluate only sound, they are sensitive to deterioration in sound quality. However, when humans evaluate sound and video simultaneously, they are sensitive to deterioration in video but insensitive to deterioration in sound. This phenomenon occurs because vision is more dominant than hearing as a human perception mechanism.

[0053] Even when listening to a single sound, if you focus on a specific speaker among a mixture of various sound sources, you can hear that sound more clearly. This has long been known as the cocktail party effect.

[0054] Furthermore, even with video alone, humans do not recognize all objects in their field of vision. We constantly make choices, such as removing signs and buildings from a streetscape to find family and acquaintances in the crowd, and reading emotions by focusing on their faces (facial expressions).

[0055] This effect can also be said about the sense of touch alone. For example, when you concentrate on the sense of touch with your hands, the sensitivity of your hands is maximized, but the sensitivity of other parts of your body may decrease. For example, even if you can feel a gentle breeze on your feet when you are not doing anything, you may not feel the wind on your feet when you are concentrating on delicate work using your hands.

[0056] Thus, human perception of physical quantities is not absolute, but varies greatly depending on where the human consciousness is directed. Perceptual coding technologies for sound and video have become widespread, and these technologies can be said to utilize these human sensory characteristics. These technologies have been developed precisely because the mechanisms of hearing and vision have been largely elucidated, and they actively utilize this knowledge.

[0057] On the other hand, although there are many unknowns regarding the mechanism of tactile perception, particularly tactile receptors as sensors, it can be said that it is possible to realize unprecedented efficient encoding, transmission, and decoding of tactile signals by utilizing research into the areas of the brain that process tactile sensation, and by utilizing effects that we experience in our daily lives and take for granted. However, the technology for efficient encoding, transmission, and decoding of tactile signals that utilizes the human tactile characteristics similar to those of hearing and vision has not yet been established.

[0058] Since the sense of touch is distributed throughout the human body, it is conceivable that the tactile presentation device 6 will be attached to various positions on the human body, and multiple tactile presentation devices that vibrate independently will be used. As a concrete example, FIG. 8 is shown. In the example of Figure 8, a haptic playback device is attached to various parts of the body, such as the face, abdomen, and feet, other than the hands and fingertips, to generate vibrations of an object. Audio basically plays back two-channel (2ch) signals for both ears, but in some cases, 5.1ch, 7.1ch, or more channels of audio signals may be played back. In this case, it is necessary to decide which direction and position of the speaker each channel's audio signal will be played from, otherwise the intended playback will not be possible.

[0059] Like audio signals, tactile signals also require information indicating which channel will vibrate which part of the body. In addition, it is natural to think of haptic signals as digitized data, assuming that they will be transmitted over the Internet or wirelessly. In this case, the digitized haptic signals must be treated as data having a fixed time unit called a frame.

[0060] For this reason, a data structure such as that shown in FIG. 8 is required. In this example, the haptic signal (PCM data) for each body part is stored in an individual frame. As shown in the figure, the frame has an area for a frame header and an area for storing the actual data of the haptic signal. The frame header stores, as header information for the frame, at least information indicating which body part the haptic signal is for.

[0061] Furthermore, the frames for each part are integrated into a data unit called a stream. The stream has an area for the stream header and an area for storing the frames for each part. The stream header stores, as header information, information indicating the specifications of the digitized haptic signal (quantization bit length, sampling frequency, data size of the stream, etc.) and information on the number of frames included in the stream.

[0062] The haptic signals for each part are encoded in this manner, and when the haptic signals are transmitted, the encoded data is handled in a format in which each time stream is arranged on the time axis as shown in the figure. By performing the encoding as described above, it is possible to obtain high convenience in the transmission and reproduction of haptic signals.

[0063] As described above, when the number of parts to which a haptic signal is applied is increased, the bit rate of the haptic signal increases proportionally, placing a large load on the system. Therefore, in this embodiment, the haptic characteristics of each part are taken into consideration, and the bit allocation of the haptic signal is made different for each part, thereby reducing the bit rate when captured by the haptic signal of the entire part.

[0064] For the sake of simplicity, let us consider a case where tactile sensation is presented to one point each on the hand and foot of the human body. If the basic conditions are the same as those in [Equation 1], the total bit rate B(hand+foot) required in this case can be calculated as shown in [Equation 2] below. B(hand+foot)=12bit / sample×2000sample / sec×2=48kbit / sec...[Formula 2]

[0065] As can be seen from the above explanation, the ratio of brain areas that process the tactile sensation of the human hand and foot is significantly different. Although the tactile characteristics (e.g., sensitivity) of the hand and foot have not been quantified to a precise level of how many times they are different, it is clear from experience that the hand is superior to the foot in terms of tactile sensitivity, so the conversion coefficient (W) of the sensitivity of the foot (S-foot) when the sensitivity of the hand (S-hand) is used as the standard is expressed as the following [Equation 3]. S-foot=W×S-hand(W<1.0)...[Formula 3] In reality, the tactile sensitivity of each finger and palm of the hand differs, but for the sake of simplicity, we will consider this in terms of individual parts such as the hand and foot.

[0066] Regarding the above conversion coefficient W, if we assume that W = 0.25, for example, 1 bit corresponds to 6 dB (2 times), so the quantization bit length required for the hand tactile signal is 12 bits, while the quantization bit length required for the foot tactile signal is 10 bits, and the total bit rate B (hand + foot) can be calculated as follows: B(hand)=12bit / sample×2000sample / sec=24kbit / sec...[Formula 4] B(foot)=10bit / sample×2000sample / sec=20kbit / sec...[Formula 5] B(hand+foot)=44kbit / sec...[Formula 6]

[0067] In the above example, the bit rate can be reduced by only about 10% based on [Equation 2]. However, if we further assume that the feet can only be perceived at half the frequency of the hands, the total bit rate B(hand+foot) can be calculated as follows: B(hand)=12bit / sample×2000sample / sec=24kbit / sec...[Formula 7] B(foot)=10bit / sample×1000sample / sec=10kbit / sec...[Formula 8] B(hand+foot)=34kbit / sec...[Formula 9] This allows the bit rate to be reduced by approximately 30% based on [Equation 2].

[0068] Another example of differentiating the sampling frequency and quantization bit length for each part is shown below. In this example, consider the application of tactile stimulation to the hands, face, and feet. In this case, if the same sampling frequency and quantization bit length are set for the face and feet as for the hand, the total bit rate B(hand+face+foot) is B(hand+face+foot)=72kbit / sec...[Formula 10] This is expressed as:

[0069] For example, the sensitivity coefficients for the face and the feet are set as follows: S-face=W-face×S-hand(W=0.5)...[Formula 11] S-foot=W-foot×S-hand(W<0.25)...[Formula 12] In other words, if the quantization bit length required for the tactile signal of the hand is 12 bits, the quantization bit length for the face is 11 bits, and the quantization bit length for the feet is 10 bits.

[0070] It is also assumed here that the frequency bands in which tactile stimuli can be perceived differ for each part of the hands, face, and feet. Specifically, for example, as shown in FIG. 9, it is assumed that the hands can only perceive vibrations in the frequency bands up to 1 kHz (see FIG. 9A), the face up to 500 Hz (see FIG. 9B), and the feet up to 250 Hz (see FIG. 9C).

[0071] Under the above conditions, if the quantization bit length required for the tactile signal of the hand in this case is 12 bits, the total bit rate B(hand+face+foot) can be calculated as follows: B(hand)=12bit / sample×2000sample / sec=24kbit / sec...[Formula 13] B(face)=11bit / sample ×1000sample / sec=11kbit / sec...[Formula 14] B(foot)=10 bit / sample×500sample / sec=5kbit / sec...[Formula 15] B(hand+face+foot)=40kbit / sec...[Formula 16] This makes it possible to reduce the total bit rate by approximately 45% compared to the case of [Equation 10].

[0072] 10 shows an example of the structure of encoded data when the data format of the haptic signal is made different for each part as described above. As an example, the data structure of a frame and the data structure of a stream corresponding to the case of [Equation 9] are shown here. As shown in the figure, the header of the frame for each part stores values ​​indicating the quantization bit length and sampling frequency for the haptic signal for that part. This makes it possible to easily identify the data format of the haptic signal stored in the frame on the decoding device 3 side, even if the data format of the haptic signal differs for each part. In the case of audio signals, which are one-dimensional signals like haptic signals, even if the signal is composed of multiple frames (or channels), all of those frames (or channels) generally have the same format, so the quantization bit length and sampling frequency only need to be written in the stream header. When the data format of the haptic signal is made different for each part as described above, it becomes more complicated than that of an audio signal, but since all frames do not need to have the same data format, it is highly efficient and advantageous in terms of bit rate.

[0073] Here, in the above, it is assumed that the sampling frequency and quantization bit length are statically determined, i.e., the bit allocation for each part is statically determined, but it is also possible to dynamically change the bit allocation for each part in response to, for example, changes in conditions over time. As a specific example, consider a case where the bit rate on the transmission path is temporarily limited due to some factor, and the total bit rate of the haptic signals from each part cannot be accommodated within the limited bit rate. In this case, the sampling frequency or quantization bit length of a specific part is restricted so that the bit allocation to that part is reduced so that it fits within the restricted bit rate. In this case, it is desirable to restrict the parts to be parts with low tactile sensitivity, that is, the face and feet in the examples so far, in order to suppress a decrease in tactile reproducibility.

[0074] On the other hand, even if a tactile signal comes from a highly sensitive area, if the signal itself does not exist, that is, if the signal amplitude does not reach a perceptible amplitude, there is little need to transmit it. Therefore, it is possible to allocate fewer bits to tactile signals from areas with small signal amplitudes.

[0075] As an example, assume that the parts to which tactile stimulation is applied are the hands, face, and feet. In this case, the encoding device 2 monitors the amplitude value of the tactile signal for each part, and determines whether there are any parts where the signal amplitude is small, specifically, where the signal amplitude does not reach an amplitude value that can be perceived by humans. This determination is made, for example, for each frame in the stream. Note that the determination of whether the amplitude reaches a value that can be perceived by humans can be made based on the vibration detection threshold curve shown in FIG. 4. For a portion where it is determined that the amplitude value of the haptic signal does not reach a perceptible amplitude value, the bit allocation is reduced, specifically, set to zero. For example, if the only parts that were not deemed to reach a perceptible amplitude value were the face and feet, B(hand)=12bit / sample×2000sample / sec=24kbit / sec...[Formula 17] B(face)=0kbit / sec...[Formula 18] B(foot)=0kbit / sec...[Formula 19] B(hand+face+foot)=24kbit / sec...[Formula 20] Bit allocation will be as follows. Conversely, if the hand is the only part that is determined not to have reached a perceptible amplitude value, the following bit allocation is applied. B(hand)=0kbit / sec...[Formula 21] B(face)=11bit / sample ×2000sample / sec=22kbit / sec...[Formula 22] B(foot)=10 bit / sample×2000sample / sec20kbit / sec...[Formula 23] B(hand+face+foot)=42kbit / sec...[Formula 24] In addition, in [Equation 22] and [Equation 23], the sampling frequency of the face and feet is set to be the same as the sampling frequency of the hands, but it is also possible to use a lower sampling frequency (see [Equation 14] and [Equation 15]).

[0076] By using the bit allocation described above, it is possible to transmit a necessary haptic signal even under restrictive conditions.

[0077] (Functional configuration of the encoding side) FIG. 11 is a functional block diagram showing the functional configuration of the encoding device 2 for implementing the above-mentioned encoding method. As shown in the figure, the encoding device 2 has functions as an acquisition unit F21 and an encoding unit F22. In addition, the encoding unit F22 has a function as a format conversion unit F23. The acquisition unit F21 acquires a tactile signal for each part of the human body. In this example, the acquisition unit F21 corresponds to the amplifier 21 and the A / D converter 22.

[0078] The encoding unit F22 encodes the haptic signals for each body part acquired by the acquisition unit F21 so that the data formats differ between different body parts. In this example, the encoding unit F22 performs encoding by using the format conversion unit F23 to change the bit allocation to parts according to the tactile sensitivity. Specifically, encoding is performed to reduce the bit allocation to parts with low tactile sensitivity. That is, for example, as in [Equation 7], [Equation 8], [Equation 13], [Equation 14], and [Equation 15], the quantization bit length of the tactile signal for parts with low tactile sensitivity is shortened and the sampling frequency is lowered.

[0079] Furthermore, the encoding unit F22 of this example reduces bit allocation to areas with small signal amplitudes by the format conversion unit F23. Specifically, for areas where the amplitude value of the haptic signal does not reach a perceptible amplitude value, the quantization bit length of the haptic signal is shortened and the sampling frequency is lowered, for example, as shown in [Equation 7], [Equation 8], [Equation 13], [Equation 14], or [Equation 15]. Alternatively, for example, if there is a part where the amplitude value of the tactile signal does not reach a perceptible amplitude value, the bit allocation for the tactile signal of that part is set to zero, as in [Equation 17], [Equation 18], [Equation 19], [Equation 21], [Equation 22], or [Equation 23].

[0080] Furthermore, the encoding unit F22 of this example assigns index information indicating the type of body part to the haptic signal for each body part. Specifically, the information indicating the type of body part is stored in the frame header of the haptic signal as shown in FIG.

[0081] In this example, the function of the encoding unit F22 described above is realized by an encoding unit 24. To vary the quantization bit length and sampling frequency of the haptic signal depending on the area, for example, A / D converters 22 with the same quantization bit length and sampling frequency are used, and the encoding unit 24 performs a conversion process of the quantization bit length and sampling frequency for the haptic signals of areas that require conversion among the haptic signals after A / D conversion.

[0082] The function of varying the quantization bit length and sampling frequency of the haptic signal depending on the part can also be realized by using A / D converters 22 with different quantization bit lengths and sampling frequencies. In that case, each A / D converter 22 will have the function of the encoding unit F22.

[0083] (Decoding method) The decoding device 3 of this embodiment reproduces the haptic signals obtained by the encoding by the encoding device 2, that is, the haptic signals in different data formats for different parts of the human body, in accordance with the data format for each part.

[0084] Here, the haptic signal is a one-dimensional signal, and the data handling can basically be considered to be the same as that of an audio signal. The audio signal generally has the same quantization bit length and sampling frequency for each channel, and on the playback side, a D / A converter 32 with the same specifications can be used for each channel. On the other hand, in this embodiment, since the quantization bit length and sampling frequency differ depending on the channel, it is assumed that a D / A converter 32 with different specifications will be used between the channels. Therefore, there is a concern that the configuration of the decoding device 3 will become complicated and the associated cost will increase.

[0085] Therefore, in this example, when playing back the tactile signals of each body part, a format conversion process is performed on the tactile signals of at least one body part, as shown in Fig. 12. The example in Fig. 12 shows a case where the tactile signals of the hand have a quantization bit length of 12 bits and a sampling frequency of 2 kHz, while the tactile signals of the foot have a quantization bit length of 10 bits and a sampling frequency of 1 kHz, and the tactile signals of the foot are subjected to format conversion to match the data format of the tactile signals of the hand.

[0086] By performing such format conversion, the data formats of the tactile signals from each part can be unified, and D / A converters 32 of the same specifications can be used.

[0087] In this case, in order to easily adjust the sampling frequency to a constant value, it is preferable to set the ratio of the sampling frequency between the parts to an integer multiple on the encoding device 2 side. For example, if the sampling frequencies are set to 1000 sample / sec and 2000 sample / sec as in the example of Fig. 12, then double oversampling can be performed to match the former with the latter.

[0088] When oversampling an audio signal, for example from 8000 samples / sec to 16000 samples / sec, aliasing components occur in the playback frequency range of 4000Hz to 8000Hz, which can be heard, so a low pass filter (LPF) is essential. This is shown in Figure 13.

[0089] However, in the case of haptic signals, the range of frequencies that can be felt is said to be up to about 1 kHz (sampling frequency = 2 kHz). In addition, the reproduction range of the haptic presentation device 6 is often limited to that range in the first place. For this reason, unlike in the case of audio signals, oversampling of haptic signals does not necessarily require the provision of an LPF.

[0090] For example, since D / A converters 32 are often used for audio applications (4 kHz in terms of signal bandwidth, i.e. the lower limit of the sampling frequency is about 8 kHz), those that support a sampling frequency of 2 kHz are not common. For this reason, no matter how efficient the haptic signal is at a specification of 2 kHz, it will not be possible to reproduce it as is with a general D / A converter 32.

[0091] Therefore, in order to enable the use of a general D / A converter 32, the tactile signals from each part are subjected to an oversampling process for converting them into a predetermined sampling frequency, such as a sampling frequency of 8 kHz or higher. For example, if oversampling processing with a sampling frequency of 2 kHz to 8 kHz is performed using simple zero-value interpolation, aliasing will occur in the signal band of 1 kHz to 4 kHz. However, as mentioned above, this frequency band is an area with very low tactile sensitivity, so there is no problem even if the LPF is omitted. Also, if the tactile presentation device 6 itself cannot reproduce this aliasing, the LPF can be omitted as well. This concept is shown in Figure 14.

[0092] (Functional configuration of the decoding side) FIG. 15 is a functional block diagram showing the functional configuration of the decoding device 3. As shown in FIG. As shown in the figure, the decryption device 3 has functions as an acquisition unit F31 and a decryption unit F32. In addition, the decryption unit F32 has a function as a format conversion unit F33.

[0093] The acquisition unit F31 acquires haptic signals in different data formats for different parts of the human body. The acquisition unit F31 in this example corresponds to a part that acquires the haptic signal for each part transmitted from the encoding device 2, and in this example corresponds to the communication unit 37.

[0094] The acquisition unit F31 in this example acquires a haptic signal in which a data format for each part is defined so that bit allocation to the part is changed according to the haptic sensitivity. Specifically, the acquisition unit F31 acquires a haptic signal in which a data format for each part is defined so that bit allocation to parts with low tactile sensitivity is reduced. That is, for example, as in [Formula 7], [Formula 8], [Formula 13], [Formula 14], and [Formula 15], a haptic signal with a short quantization bit length and a low sampling frequency is acquired for parts with low tactile sensitivity.

[0095] The acquisition unit F31 of this example acquires a haptic signal with more bits allocated to areas with larger signal amplitude. Specifically, the acquisition unit F31 acquires haptic signals for each area, including haptic signals with shorter quantization bit lengths and lower sampling frequencies, for example, as in [Formula 7], [Formula 8], [Formula 13], [Formula 14], and [Formula 15], based on the magnitude of the signal amplitude based on whether or not the amplitude value of the haptic signal reaches a perceptible amplitude value. Alternatively, for example, when the amplitude value of the tactile signal does not reach a perceptible amplitude value, the bit allocation for some parts is set to zero, as in [Equation 17], [Equation 18], [Equation 19], [Equation 21], [Equation 22], or [Equation 23], the tactile signal is obtained for parts where the bit allocation is non-zero.

[0096] Furthermore, the acquisition unit F31 in this example acquires a haptic signal with index information indicating the type of part added for each part. Specifically, as shown in FIG. 10, the acquisition unit F31 acquires a haptic signal with information indicating the type of part stored in the frame header.

[0097] Furthermore, the decoding unit F32 decodes the haptic signal acquired by the acquisition unit F31. In the decoding device 3 of this example, the configuration for realizing this decoding unit F32 is a portion including at least a decoding unit . In the decoding unit F32 of this example, the format conversion unit F33 converts at least one of the quantization bit length and the sampling frequency for at least some of the haptic signals for each body part. Note that, in relation to the specifications of the D / A converter 32, if only the sampling frequency needs to be converted, conversion of the quantization bit length is not necessary, and conversely, if only the quantization bit length needs to be converted, conversion of the sampling frequency is not necessary. The function of the format conversion unit F33 is realized by the decoding unit 34.

[0098] As described above, the quantization bit length and sampling frequency of the haptic signal may change over time based on the determination result of the magnitude of the signal amplitude, etc. In this example, information on the quantization bit length and sampling frequency is stored in the frame header, so even if the quantization bit length and sampling frequency change over time in this way, the values ​​are indicated in the frame header. The format conversion unit F33 changes the content of the format conversion process, specifically the content of the process such as oversampling, for the tactile signals of each body part, based on the information on the quantization bit length and sampling frequency stored in the frame header. This makes it possible to perform appropriate D / A conversion of the haptic signal even if the quantization bit length or sampling frequency of the haptic signal changes over time.

[0099] Furthermore, the decoding unit F32 in this example decodes the haptic signal for each part based on index information indicating the part assigned to the haptic signal. Specifically, the decoding unit F32 (decoding unit 34) identifies the part of the haptic signal based on the information indicating the part stored in the frame header, and performs processing according to that part. For example, processing such as outputting the haptic signal to the output channel corresponding to that part is performed.

[0100] Although the above example shows the reproduction of haptics in almost real time, it is also possible to build a system in which the haptic signal encoded by the above-described method is stored in a predetermined storage medium and is read and reproduced as needed. In this case, the encoded data can be stored in the storage medium as a data file in a predetermined format. In addition, this data file can be processed and edited by the user, and additional information associated with the processing and editing can be tagged, etc. The above-mentioned storage medium may be, for example, a removable medium such as an optical disk or a memory card. In this case, it is possible to configure a decoding device 3, such as a personal computer, to read and play the haptic signal data file stored in the removable medium. As another example, a data file of a haptic signal stored in a cloud storage server may be acquired by the decoding device 3 as a client terminal via the network 4 and played back.

[0101] In addition, in the above, the data format of the haptic signal is different for each body part (i.e., the data format of the haptic signal is different for all body parts), but the data format may be the same for some body parts. In other words, it is sufficient that the data format of the haptic signal is different at least for different body parts.

[0102] Here, the functions of the encoding unit F22 and the decoding unit F32 described with reference to Fig. 11 and Fig. 15 can be realized as software processing by a CPU etc. The software processing is executed based on a program, and the program is stored in a storage device that can be read by a computer device such as a CPU.

[0103] [1-5. Summary of the first embodiment] As described above, the encoding device (F2) of the first embodiment includes an encoding unit (F22) that encodes haptic signals using different data formats for different parts of the human body.

[0104] This makes it possible to reduce the amount of data in tactile signals taking into account differences in tactile characteristics between parts of the human body, for example by taking advantage of differences in tactile sensitivity between parts of the human body and allocating less data to tactile signals from parts of the human body with low tactile sensitivity. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0105] Furthermore, in the encoding device according to the first embodiment, the encoding unit encodes the haptic signal using different data formats for parts of the human body having different tactile characteristics.

[0106] This makes it possible to reduce the amount of data in a tactile signal by utilizing differences in tactile sensitivity between parts of the human body. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0107] Furthermore, in the encoding device according to the first embodiment, the encoding unit encodes the haptic signal so as to change bit allocation to parts depending on the haptic sensitivity.

[0108] This makes it possible to reduce the amount of data in the tactile signal while taking into account differences in tactile characteristics between parts of the body. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0109] Furthermore, in the encoding device according to the first embodiment, the encoding unit encodes the haptic signal so that the quantization bit length differs between different parts.

[0110] This makes it possible to reduce the data amount of the tactile signal by shortening the quantization bit length for areas with low tactile sensitivity (low sensitivity to the amplitude of tactile stimulation). Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0111] Furthermore, in the encoding device according to the first embodiment, the encoding unit encodes the haptic signal so that the sampling frequency differs between different body parts.

[0112] This makes it possible to reduce the amount of data in the tactile signal by lowering the sampling frequency for areas with low tactile sensitivity (low sensitivity to the frequency of tactile stimulation). Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0113] Furthermore, in the encoding device according to the first embodiment, the encoding unit has a format conversion unit (F23) that converts at least one of the quantization bit length and the sampling frequency of at least one of the haptic signals for each body part.

[0114] In this way, by performing coding with different quantization bit lengths and / or different sampling frequencies, it is possible to reduce the amount of data.

[0115] Furthermore, in the encoding device according to the first embodiment, the encoding unit performs encoding such that more bits are allocated to portions of the haptic signal having a larger signal amplitude.

[0116] As a result, for tactile signals in areas where the signal amplitude is small, for example where the signal amplitude does not reach a perceptible amplitude, i.e., areas where it is estimated that it is difficult to perceive tactile stimulation in terms of the size of the signal amplitude, the bit allocation is reduced, including the option of not transmitting the signal itself. Therefore, the data amount of the tactile signal is reduced taking into account the differences in tactile characteristics of each part, so that the data amount of the tactile signal can be reduced while ensuring the reproducibility of the tactile sensation, thereby improving the efficiency of the system related to tactile reproduction.

[0117] Furthermore, in the encoding device according to the first embodiment, the encoding section performs encoding that adds index information indicating the location of a part to the haptic signal.

[0118] This makes it easier and more accurate to identify the part of the haptic signal, simplifies the configuration of the decoding device, reduces costs, and improves the accuracy of haptic reproduction for each part.

[0119] Moreover, the encoding method according to the first embodiment is an encoding method for encoding a haptic signal using different data formats for different parts of the human body.

[0120] With this encoding method as well, the same actions and effects as those of the encoding device according to the first embodiment can be obtained.

[0121] Furthermore, the encoding program according to the first embodiment is a program that causes an information processing device to realize an encoding function for encoding haptic signals using different data formats for different parts of the human body.

[0122] The encoding device according to the first embodiment described above can be realized by such an encoding program according to the first embodiment.

[0123] Moreover, the decoding device (F3) as the first embodiment includes a decoding unit (F32) that decodes haptic signals that have been coded in different data formats for different parts of the human body.

[0124] This makes it possible to reduce the amount of data in tactile signals taking into account differences in tactile characteristics between parts of the human body, for example by taking advantage of differences in tactile sensitivity between parts of the human body and allocating less data to tactile signals from parts of the human body with low tactile sensitivity. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0125] Furthermore, in the decoding device according to the first embodiment, the decoding unit decodes haptic signals in different data formats for parts of the human body having different tactile characteristics.

[0126] This makes it possible to reduce the amount of data in a tactile signal by utilizing differences in tactile sensitivity between parts of the human body. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0127] Furthermore, in the decoding device according to the first embodiment, the decoding section decodes a haptic signal in which a data format is defined for each part so that bit allocation to the part is changed according to the haptic sensitivity.

[0128] This makes it possible to reduce the amount of data in the tactile signal while taking into account differences in tactile characteristics between parts of the body. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0129] Furthermore, in the decoding device according to the first embodiment, the decoding section decodes haptic signals having different quantization bit lengths for different parts.

[0130] This makes it possible to reduce the data amount of the tactile signal by shortening the quantization bit length for areas with low tactile sensitivity (low sensitivity to the amplitude of tactile stimulation). Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0131] Furthermore, in the decoding device according to the first embodiment, the decoding unit decodes haptic signals having different sampling frequencies for different body parts.

[0132] This makes it possible to reduce the amount of data in the tactile signal by lowering the sampling frequency for areas with low tactile sensitivity (low sensitivity to the frequency of tactile stimulation). Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0133] Furthermore, in the decoding device of the first embodiment, the decoding unit has a format conversion unit (F33) that inputs haptic signals having different quantization bit lengths and / or sampling frequencies between the different parts, and converts at least one of the quantization bit lengths and / or sampling frequencies for at least one of the haptic signals for each part.

[0134] This eliminates the need to provide multiple types of D / A converters for D / A conversion of tactile signals according to differences in quantization bit length and sampling frequency depending on the part of the body. Therefore, the configuration of the decoding device can be simplified and the cost can be reduced.

[0135] Furthermore, in the decoding device according to the first embodiment, the decoding section decodes a haptic signal in which more bits are allocated to portions having larger signal amplitudes.

[0136] As a result, for tactile signals in areas where the signal amplitude is small, for example where the signal amplitude does not reach a perceptible amplitude, i.e., areas where it is estimated that it is difficult to perceive tactile stimulation in terms of the size of the signal amplitude, the bit allocation is reduced, including the option of not transmitting the signal itself. Therefore, the data amount of the tactile signal is reduced taking into account the differences in tactile characteristics of each part, so that the data amount of the tactile signal can be reduced while ensuring the reproducibility of the tactile sensation, thereby improving the efficiency of the system related to tactile reproduction.

[0137] Furthermore, in the decoding device according to the first embodiment, the decoding section receives a haptic signal to which index information indicating the type of body part is added, and decodes the haptic signal for each body part based on the index information.

[0138] This makes it easier and more accurate to identify the part of the haptic signal, simplifies the configuration of the decoding device, reduces costs, and improves the accuracy of haptic reproduction for each part.

[0139] Moreover, the decoding method as the first embodiment is a decoding method for decoding haptic signals encoded in different data formats between different parts of the human body.

[0140] With such a decoding method as the first embodiment, it is possible to obtain the same functions and effects as those of the decoding device as the first embodiment described above.

[0141] The decoding program according to the first embodiment is a program that causes an information processing device to realize a decoding function for decoding haptic signals that have been coded in different data formats for different parts of the human body.

[0142] Such a program can realize the decoding device according to the first embodiment described above.

[0143] <2. Second embodiment> [2-1. Overview of the tactile reproduction system] Next, a second embodiment will be described. The second embodiment is a measure to deal with the transmission delay of the haptic signal. FIG. 16 shows an example of the configuration of a tactile reproduction system 1A according to the second embodiment. In the following description, parts that are similar to parts that have already been described will be given the same reference numerals and description thereof will be omitted.

[0144] 16, a haptic reproduction system 1A differs from the haptic reproduction system 1 of the first embodiment in that a transmitting device 3A is provided instead of the decoding device 3, and a receiving device 40 is also provided. In the tactile reproduction system 1A, each tactile presentation device 6 worn by the tactile receiver is not connected to the transmitting device 3A by wire, and the transmission of tactile signals from the transmitting device 3A to the tactile presentation device 6 is performed via wireless communication. The receiving device 40 functions as a device that receives tactile signals transmitted by the transmitting device 3A by wireless communication and transmits them to the tactile presentation device 6. In this example, each tactile presentation device 6 is connected to the receiving device 40 by wire, and the part surrounded by the dashed line in the figure, i.e., the receiving device 40 and each tactile presentation device 6, is the part that is worn by the tactile receiver.

[0145] For example, in the case where the decoding device 3 and each tactile presentation device 6 are connected by wire as exemplified in the first embodiment, if the size of the decoding device 3 is large, it may cause annoyance to the haptic receiver wearing the tactile presentation device 6. This annoyance is expected to increase as the number of parts to which tactile stimulation is applied increases. The above-described configuration of the tactile reproduction system 1A makes it possible to arrange the receiving device 40, which is smaller in size than the decoding device 3, and thus makes it possible to prevent the tactile receiver from experiencing the above-described inconvenience.

[0146] [2-2. Configuration of the transmitting device] FIG. 17 is a diagram showing an example of the internal configuration of the transmission device 3A. The differences from the decoding device 3 shown in Figure 3 are that the amplifier 31, the D / A converter 32, the post-processing unit 33, and the decoding unit 34 are not provided, that a transmission data generating unit 51 and a wireless communication unit 52 are provided, and that a control unit 35A is provided instead of the control unit 35. As shown in the figure, a transmission data generation unit 51, a wireless communication unit 52, a control unit 35A, a storage unit 36, and a communication unit 37 are connected via a bus 38 and are capable of data communication with one another.

[0147] The transmission data generation unit 51 performs a predetermined encoding process on the encoded data of the haptic signal obtained from the encoding device 2 via the network 4 and the communication unit 37, to generate encoded data to be transmitted to the receiving device 40. The process performed by the transmission data generating unit 51 in this embodiment will be described later.

[0148] The wireless communication unit 52 performs short-distance wireless communication using a predetermined communication method such as Bluetooth (registered trademark).

[0149] The control unit 35A is configured to have, for example, a microcomputer, and performs overall control of the transmission device 3A. In particular, the control unit 35A performs output control of the coded data from the communication unit 37 to the transmission data generation unit 51, and output control of the coded data generated by the transmission data generation unit 51 to the wireless communication unit 52. This makes it possible to transmit tactile signals from each part to an external device via wireless communication unit 52.

[0150] In the second embodiment, the data format of the tactile signals for each part may be different between different parts as described in the first embodiment, or may be the same for all parts.

[0151] [2-3. Configuration of receiving device] FIG. 18 is a diagram for explaining an example of the internal configuration of the receiving device 40, and shows each of the tactile presentation devices 6 together with the example of the internal configuration of the receiving device 40. As shown in FIG. As shown in the figure, the receiving device 40 includes an amplifier 31, a D / A converter 32, a post-processing unit 33, and a decoding unit 34A, as well as a control unit 41, a storage unit 42, a wireless communication unit 43, and a bus 44. The post-processing unit 33, the decoding unit 34A, the control unit 41, the storage unit 42, and the wireless communication unit 43 are connected via the bus 44 and are capable of data communication with one another.

[0152] The wireless communication unit 43 performs short-distance wireless communication using a method such as Bluetooth that enables communication with the wireless communication unit 52 in the transmission device 3A. The wireless communication unit 43 receives the encoded data transmitted from the transmission device 3A.

[0153] The control unit 41 is configured to have, for example, a microcomputer, and performs overall control of the receiving device 40 . The storage unit 42 is a storage device similar to the storage units 26 and 36, and is used to store various data used by the control unit 40 and the like.

[0154] The decoding unit 34A performs the same processing as the decoding unit 34 described in the first embodiment on the encoded data input via the wireless communication unit 43. That is, in order to accommodate cases where the data format of the haptic signal differs for each part, the decoding unit 34A performs processing such as format conversion on the haptic signal of the necessary part. The decoding unit 34A also performs processing on the input encoded data to counter transmission delays, which will be described later.

[0155] [2-4. Tactile reproduction method as the second embodiment] Here, when data is transmitted over any section, whether wired or wireless, data loss may occur. It is possible to check for data loss and, if any loss occurs, to resend the data to compensate, but this results in extra data transmission, which increases the effective bit rate and causes transmission delays.

[0156] As a concrete example, wireless transmission using Bluetooth is affected by Wi-Fi (registered trademark), which uses the same carrier frequency. Bluetooth has weaker radio wave strength than Wi-Fi, so it can be said to be particularly susceptible to such effects. During transmission, encoded data is sent as data in predetermined transmission units called packets, but packet loss occurs frequently in the above-mentioned state. For example, in the transmission of audio signals using Bluetooth's A2DP (Advanced Audio Distribution Profile), a large stream buffer is prepared on the receiving side, with the premise that packets will be resent if packet loss occurs, and playback is started only after a certain amount of stream has accumulated in this buffer, preventing sound dropouts.

[0157] However, the amount of stored coded data is equivalent to the amount of delay, and if the amount of stored data becomes too large, problems will occur. For example, when watching a video on a display and listening to the corresponding audio, lip synchronization will be lost, which will be a major problem.

[0158] The transmission of haptic signals faces the same challenges as those for audio signals. For example, if you have a video of a baseball bat being swung and you want to provide a tactile stimulus to the recipient at the moment the bat hits the ball, if the stream buffer capacity is large or if packet loss occurs and retransmissions are repeated, there may be a clear mismatch between vision and touch, which may cause the recipient to feel very uncomfortable.

[0159] Thus, when considering synchronization between tactile sensation and other senses, it is desirable to minimize packet retransmissions and to make the receiving side stream buffer as small as possible. To achieve this, it is effective to use a data structure that does not cause major problems even if some packets are lost. Specifically, as mentioned earlier, humans have different tactile sensitivity depending on the part of the body, so a data structure that takes this into account is adopted.

[0160] As shown in Fig. 10, the stream has header information indicating the boundaries of frames, followed by the actual haptic signal. If the time granularity of the haptic signal in the frame is made too small, the proportion of the header in the transmission increases, decreasing efficiency. Conversely, if the time granularity is made too large, this alone can cause delay problems or be affected by interference, so it is desirable to encode the haptic signal for about a few milliseconds in the frame.

[0161] The frames of each part are transmitted by the aforementioned packets, and the longer the packet size, the longer the time required for transmission, making it more susceptible to interference probabilistically. Therefore, in the second embodiment, frames are packed in order of the areas with the highest tactile sensitivity from the head of the stream.

[0162] A specific example is shown in FIG. FIG. 19 shows an example of the order of frames for each body part when the tactile sensitivity is determined to be highest on the hands, followed by the face and then the feet. In this case, in the stream, frames (tactile signals) for each body part are arranged from the top in the order of hands, face, and feet.

[0163] In conventional transmission, the receiving side would return an ACK (acknowledgement) only if the packet was completely received successfully. In contrast, in the second embodiment, an ACK is returned if at least a frame with high tactile sensitivity that is part of the stream included in the packet is successfully received. For this reason, a parameter is recorded in the stream header that indicates the size of the successfully received frame that is allowed to return an ACK (see "Ack Allow Size" in FIG. 19). When the receiving side confirms that the part specified by this parameter has been successfully received, it returns an ACK to the sending side.

[0164] Doing this reduces the probability of interference, since it is equivalent to transmitting frames with high tactile sensitivity using short packets. Even if frames with low tactile sensitivity are lost due to interference, there is an advantage in that the impact on human tactile senses is relatively small if high tactile sensitivity data is reproduced. In other words, a certain degree of tactile reproducibility can be guaranteed.

[0165] In the above example, the priority order for arranging frames is based on haptic sensitivity, but the priority order can also be based on the amplitude of the haptic signal. That is, for example, frames of a portion where the amplitude value of the haptic signal has reached a perceptible amplitude value are preferentially arranged (placed at the beginning) in the stream.

[0166] In the above, an ACK is returned in response to reception of the frame with the highest priority, but it is also possible to return an ACK in response to reception of frames with the top n priorities (n is a natural number greater than or equal to 2 and less than the total number of parts in the stream), for example, by returning an ACK in response to reception of signals with the top two priorities. To achieve an effect equivalent to that achieved by transmitting using apparently short packets as described above, an ACK can be returned only when a haptic signal has been received from at least the area with the highest priority, excluding the area with the lowest priority.

[0167] Even if apparently short packets are used as described above, the interference rate per unit time is constant, so interference during the time it takes to transmit the packets will not reach zero. Therefore, as shown in the example of FIG. 20, frames with high tactile sensitivity (frames with high priority) are arranged in multiple locations in the stream and transmitted with redundancy.

[0168] In this case, the stream header stores information about which frames must be successfully received before an ACK can be returned, such as ID designation information that specifies the frame ID. If even one frame identified by this ID designation information has been successfully received, an ACK will be returned even if the others have been lost. FIG. 20 shows an example in which, when four frames with frame IDs 0 to 3 are arranged in one stream, the first frame with ID=0 is a hand frame, the second from the first frame with ID=1 is a face frame, the third from the first frame with ID=2 is a foot frame, and the fourth frame from the first with ID=3 is a hand frame again, and ID designation information specifying IDs 0 and 3 is stored in the stream header to correspond to this.

[0169] In this way, if the disturbance rate per unit time is constant, the resistance to loss of tactile sensitive frames can be improved many times over.

[0170] It should be noted that, in the above method as well, the priority order may be determined not based on the level of tactile sensitivity but based on the magnitude of the signal amplitude of the tactile signal. In addition, while the above example gives an example in which the parts having haptic signal redundancy are limited to the parts with the highest priority, it is also possible to give haptic signal redundancy to the parts with the top n priorities (n is a natural number greater than or equal to 2 and less than the total number of parts in the stream).

[0171] (Sender functional configuration) FIG. 21 is a functional block diagram showing the functional configuration of the transmission device 3A. As shown in the figure, the transmitting device 3A has the functions of an encoding unit F34 and a transmitting unit F35. The encoding unit F34 performs encoding to arrange the tactile signals for each part of the human body according to the priority of each part, and generates encoded data. The function of this encoding unit F34 is realized by the transmission data generation unit 51. Here, the encoding unit F34 arranges the haptic signals in order of the area with the highest priority. Specifically, for this function, the transmission data generation unit 51 generates encoded data having a stream structure in which frames are arranged from the top side in order of the area with the highest haptic sensitivity, as shown in Fig. 19. At this time, the transmission data generation unit 51 stores a parameter equivalent to "Ack Allow Size" shown in Fig. 19 in the stream header.

[0172] Furthermore, the encoding unit F34 provides redundancy to the haptic signals of the parts with high priority. With regard to this function, the transmission data generating unit 51 generates encoded data in which redundancy is provided only to the haptic signals of the parts with the highest tactile sensitivity, for example, as illustrated in Fig. 20. At this time, the transmission data generating unit 51 stores the above-mentioned ID designation information in the stream header.

[0173] The transmission unit F35 transmits the encoded data generated by the encoding unit F34. Specifically, the wireless communication unit 52 transmits the encoded data generated by the transmission data generation unit 51 to an external device (reception device 40) based on, for example, the control of the control unit 35A.

[0174] (Receiving side functional configuration) FIG. 22 is a functional block diagram showing the functional configuration of the receiving device 40. As shown in FIG. As shown in the figure, the receiving device 40 has the functions of a receiving unit F41 and a decoding unit F42. The receiving unit F41 receives coded data from a transmitting device that generates coded data by encoding the tactile signals for each part of the human body in order of priority for each part. That is, in this example, the receiving unit F41 receives coded data generated by the transmission data generating unit 51 of the transmitting device 3A. In this example, the receiving unit F41 corresponds to the wireless communication unit 43.

[0175] The decoding unit F42 decodes the encoded data received by the receiving unit F41 in accordance with the priority order for each part. In this example, the function of the decoding unit F42 is realized by the decoding unit 34A. Specifically, the decoding unit F42 performs the following process in response to the case where haptic signals are transmitted in order of priority: That is, the decoding unit F42 sends an affirmative response to the transmitting device on the condition that the haptic signal of at least the body part with the highest priority, excluding the body part with the lowest priority, has been received. As a function of this decoding unit F42, the decoding unit 34A refers to the above-mentioned parameters stored in the stream header of the encoded data generated by the transmission data generation unit 51 and received by the wireless communication unit 43, determines whether or not a frame identified from the parameters has been received (reception was successful), and returns an ACK to the transmitting device 3A side if it determines that the frame has been received. The decoding unit 34A repeats this process for each stream.

[0176] In addition, in a case where haptic signals are transmitted with redundancy given to a part with a high priority, the decoding unit F42 sends a positive response to the transmitting device in response to reception of at least one haptic signal for the part with the redundancy given. As a function of this decoding unit F42, the decoding unit 34A refers to the above-mentioned ID designation information stored in the stream header of the encoded data generated by the transmission data generation unit 51 and received by the wireless communication unit 43, determines whether or not at least one frame of the multiple frames indicated by the ID designation information has been received (reception has been successful), and returns an ACK to the transmitting device 3A side if it determines that it has been received. The decoding unit 34A repeats this process for each stream.

[0177] Here, the function of the decryption unit F42 described with reference to Fig. 22 can be realized as software processing by a CPU or the like. The software processing is executed based on a program, and the program is stored in a storage device that can be read by a computer device such as a CPU.

[0178] [2-5. Summary of the second embodiment] As described above, the transmission system of the second embodiment includes a transmitting device (3A) having an encoding unit (F34, transmission data generation unit 51) that generates encoded data by encoding the tactile signals for each part of the human body according to the priority of each part, and a transmitting unit (F35) that transmits the encoded data, a receiving device (40) having a receiving unit (F41, wireless communication unit 43) that receives the encoded data transmitted by the transmitting unit, and a decoding unit (F42, 34A) that decodes the encoded data received by the receiving unit according to the priority.

[0179] By transmitting the tactile signals from each area in an order according to priority as described above, it is possible to reduce data loss of tactile signals from areas that should be prioritized, such as areas with high tactile sensitivity.Furthermore, by decoding the tactile signals transmitted in this manner according to priority, even if data loss occurs on the transmission path, if the data is not a tactile signal from an area that should be prioritized, it is possible to ensure that only the tactile signal received from the area that should be prioritized is the subject of tactile reproduction. Therefore, it is possible to reduce the occurrence of transmission delays in haptic signals due to data loss on the transmission path, and to prevent degradation of haptic reproducibility due to transmission delays.

[0180] In addition, the receiving device (same 40) as the second embodiment is equipped with a receiving unit (same F41, wireless communication unit 43) that receives encoded data from a transmitting device (same 3A) that generates encoded data by encoding to arrange the tactile signals for each part of the human body according to the priority of each part, and a decoding unit (same F42, 34A) that decodes the encoded data received by the receiving unit according to the priority.

[0181] By having the transmitting device transmit the tactile signals from each area in an order according to priority, it is possible to reduce data loss of tactile signals from areas that should be prioritized, such as areas with high tactile sensitivity.Furthermore, by decoding the tactile signals transmitted in this manner according to priority, even if data loss occurs on the transmission path, if the data is not a tactile signal from an area that should be prioritized, only the tactile signal received from the area that should be prioritized will be subject to tactile reproduction. Therefore, it is possible to reduce the occurrence of transmission delays in haptic signals due to data loss on the transmission path, and to prevent degradation of haptic reproducibility due to transmission delays.

[0182] In addition, in the receiving device according to the second embodiment, the tactile signals for each part of the human body are tactile signals for parts having different tactile characteristics.

[0183] This makes it possible to reproduce the tactile sensation for each part of the body with different tactile characteristics. Therefore, appropriate tactile reproduction can be achieved according to the tactile characteristics of each part of the body.

[0184] Furthermore, in the receiving device of the second embodiment, the transmitting device transmits haptic signals in order of priority, and the decoding unit sends a positive response to the transmitting device on the condition that the haptic signal of at least the area with the highest priority, excluding the area with the lowest priority, has been received.

[0185] This is equivalent to sending tactile signals for areas with high priority using short packets. Therefore, it is possible to reduce the occurrence rate of data loss of tactile signals for areas with high priority, and to prevent a decrease in tactile reproducibility.

[0186] Furthermore, in the receiving device according to the second embodiment, the priority is determined based on the ease of perception of the tactile stimulus.

[0187] This makes it possible to reduce data loss of the tactile signal in areas that have a high priority in terms of ease of perception of tactile stimulation, such as high tactile sensitivity or large signal amplitude. Therefore, it is possible to prevent the deterioration of tactile reproducibility.

[0188] In the receiving device according to the second embodiment, the priority order is determined according to the level of tactile sensitivity.

[0189] This makes it possible to reduce data loss of tactile signals in areas that are more susceptible to perceiving tactile stimuli in terms of sensitivity to tactile stimuli. Therefore, it is possible to prevent the deterioration of tactile reproducibility.

[0190] Furthermore, in the receiving device according to the second embodiment, the priority is determined based on the magnitude of the amplitude of the haptic signal.

[0191] This makes it possible to reduce data loss of the tactile signal in areas where tactile stimulation is easily perceived in terms of the magnitude of the signal amplitude of the tactile signal. Therefore, it is possible to prevent the deterioration of tactile reproducibility.

[0192] Furthermore, in a receiving device of a second embodiment, the transmitting device transmits haptic signals with redundancy for areas with high priority, and the decoding unit sends a positive response to the transmitting device in response to receiving at least one haptic signal for an area with redundancy.

[0193] This makes it possible to reduce data loss in the haptic signal. Therefore, it is possible to prevent the deterioration of tactile reproducibility.

[0194] In addition, the decoding method as the second embodiment is a decoding method that performs decoding according to priority on encoded data received from a transmitting device that generates encoded data by encoding in which the tactile signals for each part of the human body are arranged according to the priority of each part.

[0195] With such a decoding method as the second embodiment, it is possible to obtain the same functions and effects as those of the receiving device as the second embodiment described above.

[0196] In addition, the program as the second embodiment is a program that causes an information processing device to realize the function of decoding, in accordance with priority, encoded data received from a transmitting device that generates encoded data by encoding the tactile signals for each part of the human body according to the priority of each part.

[0197] Such a program can realize the receiving device as the second embodiment described above.

[0198] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0199] 3. This Technology The present technology can also be configured as follows. (1) A decoder for decoding haptic signals encoded in different data formats between different parts of the human body. Decryption device. (2) The decoding unit is Decoding tactile signals in different data formats between different parts of the human body with different tactile characteristics The decoding device according to (1) above. (3) The decoding unit is A haptic signal having a data format determined for each part is decoded so that bit allocation to the part is changed according to the haptic sensitivity. The decoding device according to (1) or (2). (4) The decoding unit is Decoding haptic signals with different quantization bit lengths between different parts The decoding device according to (3) above. (5) The decoding unit is Decoding tactile signals with different sampling frequencies between different parts of the body The decoding device according to (3) or (4). (6) The decoding unit is A format conversion unit is provided for inputting haptic signals having different quantization bit lengths and sampling frequencies between the different parts of the body and converting at least one of the quantization bit lengths and sampling frequencies for at least one of the haptic signals for each of the parts. A decoding device according to any one of (1) to (5). (7) The decoding unit is The haptic signal to which index information indicating the type of the body part is added is input, and the haptic signal for each body part is decoded based on the index information. A decoding device according to any one of (1) to (6). (8) a receiving unit that receives encoded data from a transmitting device that generates encoded data by encoding the tactile signals for each part of the human body in accordance with the priority order of the parts; a decoding unit that performs decoding in accordance with the priority order on the encoded data received by the receiving unit. Receiving device. (9) The tactile signals for each part of the human body are tactile signals for each part of the human body having different tactile characteristics. The receiving device according to (8) above. (10) the transmitting device transmits the haptic signals in order of the areas with the highest priority, The decoding unit transmits an affirmative response to the transmitting device on condition that a haptic signal of at least the part having the highest priority has been received, excluding the part having the lowest priority. A receiving device according to (8) or (9). (11) The priority is a priority for the height of tactile sensitivity. The receiving device according to (10) above. (12) the transmitting device transmits a haptic signal with redundancy for the area with high priority; The decoding unit is and sending an acknowledgment to the transmitting device in response to at least one haptic signal being received for the haptic signals for the portion for which redundancy is provided. A receiving device according to any one of (8) to (11) above. [Explanation of symbols]

[0200] 1, 1A tactile reproduction system, 2 encoding device, 3 decoding device, 3A transmission device, 5 tactile sensor, 6 tactile presentation device, 22 A / D converter, 24 encoding unit, F21 acquisition unit, F22 encoding unit, 32 D / A converter, 34, 34A decoding unit, 35, 35A control unit, 37 communication unit, F31 acquisition unit, F32 playback unit, F23, F33 format conversion unit, F34 encoding unit, F35 transmission unit, 40 receiving device, 43 wireless communication unit, 51 transmission data transmission unit, 52 wireless communication unit, F41 receiving unit, F42 decoding unit

Claims

1. The encoding device includes a decoding unit that decodes the tactile signals of the first and second parts of the human body that have been encoded by an encoding unit that performs encoding such that a bit allocation of the tactile signal of the second part differs from a bit allocation of the tactile signal of the first part. Decryption device.

2. The decoding unit, Decoding a haptic signal having a different quantization bit length between the first portion and the second portion The decoding device according to claim 1 .

3. The decoding unit, Decoding a haptic signal having a different sampling frequency between the first portion and the second portion The decoding device according to claim 1 .

4. The decoding unit, a format conversion unit that receives haptic signals having different quantization bit lengths and sampling frequencies between the first and second regions and converts at least one of the quantization bit lengths and sampling frequencies of at least one of the haptic signals of the first region and the second region; The decoding device according to claim 1 .

5. The decoding unit, The haptic signal to which index information indicating the type of the body part is added is input, and the haptic signal for each body part is decoded based on the index information. The decoding device according to claim 1 .

6. A coding unit performs coding for a tactile signal of a first part and a tactile signal of a second part of a human body, the coding unit making a bit allocation of the tactile signal of the second part different from a bit allocation of the tactile signal of the first part, and decodes the tactile signals of the first part and the second part coded by the coding unit. Decryption method.

7. The information processing device is provided with a decoding function for decoding the haptic signals of the first and second parts of a human body that have been encoded by an encoding unit that performs encoding for the haptic signals of the first and second parts of the human body such that the bit allocation of the haptic signal of the second part differs from the bit allocation of the haptic signal of the first part. program.

Images