Decoding device, decoding method, and program

Abstract

To improve an ease and an accuracy of identification of a site of a tactile signal, to simply a configuration of a decoding apparatus and reduce a cost, and to improve an accuracy of tactile reproduction for each site.SOLUTION: An decoding apparatus according to the present technique includes a reception unit that receives a tactile signal for each site of a human body, and a decoding unit that decodes the tactile signal received by the reception unit. The tactile signal includes a header storing header information and a region storing real data of the tactile signal, and the header stores information indicating the site.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] This technology is , revenge Signaling device and method , and The present invention relates to a computer program product and, in particular, to a technical field related to the encoding, decoding, and transmission of tactile signals for generating tactile stimulation. [Background technology]

[0002] In recent years, applications that provide tactile stimuli using tactile presentation devices that are in contact with human skin have been used in a variety of situations. Here, "tactile presentation" refers to the generation of tactile stimuli. For example, in mobile devices equipped with touch panels such as smartphones, the panel (or the housing) is vibrated when the panel is touched, providing a tactile stimulus to the fingers, thereby creating a simulated sensation of touching a button. When listening to music, some headphones incorporate a tactile presentation device into the headphone housing, providing tactile stimulation in parallel with the music playback, thereby emphasizing deep bass sounds. In the fields of computer games and virtual reality (VR), there are devices that enhance the user's sense of immersion by providing interactive tactile stimulation tailored to the scene using a tactile presentation device installed in the controller in response to the user's operation. In amusement facilities, such as movie theaters and theme parks, tactile presentation devices are installed in the seats to provide tactile stimulation depending on the situation, thereby improving the sense of realism of visitors.

[0003] Also, at the research and development stage, when remotely operating a robot or the like, the vibrations received by the robot or the object being operated are fed back to a controller in the operator's hand, allowing the operator to intuitively sense the situation around the robot or object and help predict danger (e.g., disaster response robots).<http: / / www.rm.is.tohoku.ac.jp / quince_mech / #_8> ) Furthermore, in the medical field, research is being conducted to improve surgical precision by providing feedback to the operator about the feel (hardness) of the endoscopic forceps when they touch the organ during the operation of a surgical robot (e.g., the Da Vinci surgical robot).<http: / / techon.nikkeibp.co.jp / article / FEATURE / 20150217 / 404460 / ?P=2> )

[0004] On the other hand, tactile presentation devices widely use eccentric motors (ERMs) and linear actuators (LRAs), many of which are considered to have resonant frequencies at frequencies where humans have high tactile sensitivity (around several hundred Hz) (see, for example, Patent Document 1 below).

[0005] There are also examples of efforts to enhance the sense of realism by providing multiple tactile presentation devices that are worn all over the body to provide tactile stimulation (e.g., Synesthesia Suit).<http: / / rezinfinite.com / ja / synesthesia-suit / > ) [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-202486 Summary of the Invention [Problem to be solved by the invention]

[0007] Here, when multiple tactile presentation devices are used to provide tactile stimulation to multiple parts of the human body as described above, signals representing the patterns of tactile stimulation, i.e., tactile signals, are prepared and processed on multiple channels for each part. In this case, it is desirable to provide tactile stimulation to as many parts of the body as possible to enhance the sense of realism, but it is not desirable from a system configuration standpoint to increase the amount of data required for tactile reproduction.

[0008] Furthermore, when transmitting a haptic signal for haptic reproduction, particularly when transmitting wirelessly, loss of encoded data can occur due to interference on the transmission path. If data loss occurs, the receiving side will request the sending side to resend the data, which will cause a delay in data transmission, and this transmission delay may result in a decrease in the reproducibility of the haptic sensation. Specifically, haptic reproduction is expected to be synchronized with content related to other senses, such as sound or video (hereinafter referred to as "sensory content"), but if a transmission delay such as that described above occurs in the haptic signal, haptic stimulation cannot be provided at the appropriate timing, and synchronization with other sensory content cannot be achieved, which may result in a decrease in the reproducibility of the haptic sensation.

[0009] This technology is The present invention aims to increase the ease and accuracy of identifying the parts of tactile signals, thereby simplifying the configuration of a decoding device, reducing costs, and improving the accuracy of tactile reproduction for each part. [Means for solving the problem]

[0010] The decoding device according to the present technology comprises: The device includes a receiver that receives a haptic signal for each part of a human body, and a decoder that decodes the haptic signal received by the receiver, the haptic signal having a header that stores header information and an area that stores actual data of the haptic signal, and the header stores information that indicates the part. It is something.

[0011] This means: Increase the ease and accuracy of identifying the location of tactile signals It is possible to do this. [Effects of the Invention]

[0012] According to this technology, By increasing the ease and accuracy of identifying the parts of a haptic signal, it is possible to simplify the configuration of a decoding device, reduce costs, and improve the accuracy of haptic reproduction for each part. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an example of the configuration of a haptic reproduction system including a decoding device according to a first embodiment of the present technology. [Figure 2] FIG. 2 is a diagram illustrating an example of the internal configuration of an encoding device according to the first embodiment. [Figure 3] FIG. 2 is a diagram illustrating an example of the internal configuration of a decoding device according to a first embodiment. [Figure 4] FIG. 10 is an explanatory diagram of a vibration detection threshold curve. [Figure 5] FIG. 1 shows an example of the distribution of neural firing for each receptor. [Figure 6] FIG. 10 is an explanatory diagram of vibration detection threshold curves for each receptor. [Figure 7] FIG. 2 is a diagram illustrating an example of an amplitude range and a frequency range that are targeted in the digitization of a haptic signal. [Figure 8] 1A and 1B are diagrams showing examples of how a tactile presentation device is attached to each part of the human body and examples of data structures of tactile signals from each part. [Figure 9] FIG. 10 is a diagram illustrating an example of differences between body parts in frequency bands in which tactile stimuli can be perceived. [Figure 10] 10A and 10B are diagrams showing examples of the structure of coded data when the data format of the tactile signal is made different for each body part. [Figure 11] FIG. 1 is a functional block diagram showing the functional configuration of an encoding device according to a first embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of format conversion of a haptic signal. [Figure 13] 1A and 1B are diagrams illustrating an example of aliasing due to oversampling and its removal. [Figure 14] FIG. 10 is an explanatory diagram of oversampling of a haptic signal. [Figure 15] FIG. 2 is a functional block diagram showing the functional configuration of a decoding device according to a first embodiment. [Figure 16] FIG. 10 is a diagram illustrating an example of the configuration of a tactile reproduction system according to a second embodiment. [Figure 17] FIG. 10 is a diagram illustrating an example of the internal configuration of a transmission device according to a second embodiment. [Figure 18] FIG. 10 is a diagram illustrating an example of the internal configuration of a receiving device according to a second embodiment. [Figure 19] FIG. 10 is a diagram showing an example of the order of frames for each body part when the tactile sensitivity is highest for the hands, followed by the face and feet. [Figure 20] FIG. 10 is an explanatory diagram of an example in which redundancy is given to tactile signals according to the priority of parts. [Figure 21] FIG. 10 is a functional block diagram showing the functional configuration of a transmission device according to a second embodiment. [Figure 22]FIG. 10 is a functional block diagram showing the functional configuration of a receiving device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present technology will be described in the following order with reference to the accompanying drawings. <1. First embodiment> [1-1. Overview of the tactile reproduction system] [1-2. Encoding device configuration] [1-3. Configuration of the decoding device] [1-4. Tactile Reproduction Method as First Embodiment] (encoding method) (Encoding side functional configuration) (Decoding method) (Functional configuration on the decoding side) [1-5. Summary of the First Embodiment] 2. Second Embodiment [2-1. Overview of the tactile reproduction system] [2-2. Configuration of the transmitting device] [2-3. Configuration of receiving device] [2-4. Tactile Reproduction Method as Second Embodiment] (Sender's functional configuration) (Receiving side functional configuration) [2-5. Summary of the second embodiment] <3. This technology>

[0015] In this specification, the terms are defined as follows. Tactile stimulation: A physical phenomenon that allows a person to perceive touch, such as a vibration phenomenon. Tactile presentation: Producing tactile stimuli. Tactile signal: a signal representing a pattern of tactile stimulation, for example a signal representing a vibration waveform. Haptic recipient: The person receiving the haptic presentation. Tactile characteristics: Characteristics related to human tactile senses. They differ depending on the body part (hands, face, feet, etc.). Tactile sensitivity: The subjective sensitivity of tactile stimuli. It varies depending on the receptors and parts of the body. High tactile sensitivity means that tactile signals are easily perceived. Encoded data: Data that encodes haptic signals. Sub-concepts include streams and frames. It should be noted that the "tactile sensitivity" referred to here is of two types: one related to the amplitude of the tactile stimulus and the other related to the frequency of the tactile stimulus. In this specification, unless otherwise specified, "tactile sensitivity" does not distinguish between amplitude and frequency.

[0016] <1. First embodiment> [1-1. Overview of the tactile reproduction system] FIG. 1 shows an example of the configuration of a haptic reproduction system 1 including an encoding device (see 2) and a decoding device (see 3) as a first embodiment of the present technology. The tactile reproduction system 1 includes an encoding device 2 to which multiple tactile sensors 5 are connected, a decoding device 3 configured to be able to communicate with the encoding device 2 via a predetermined network 4, and multiple tactile presentation devices 6 connected to the decoding device 3.

[0017] The tactile sensor 5 is a part or component that senses tactile stimuli, and in this example, a vibration sensor such as a piezo pickup or an acceleration sensor is used. The tactile sensor outputs vibrations and movements as voltage changes when it is brought into contact with the sensing target, that is, a human body (or a vibrating object) in this example. In this example, each tactile sensor 5 is connected to the encoding device 2 by wire, and each contact sensor 5 is attached to a different part of the human body (or a vibrating object) as the target object to sense the tactile stimulation occurring in each part.

[0018] The encoding device 2 is configured with a computer device such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), and encodes the detection signals (tactile signals) from each tactile sensor 5 in accordance with a predetermined data format, and transmits the encoded tactile signals, i.e., the encoded data, to the decoding device 3 via a predetermined network 4, such as the Internet.

[0019] The decoding device 3 is configured with a computer device such as a CPU or a DSP, decodes the coded data received via the network 4, and drives each tactile presentation device 6 based on the decoded tactile signal.

[0020] The tactile presentation device 6 is a device that generates a tactile stimulus, and in this example, a device such as a vibrator or an actuator is used. In this example, each tactile presentation device 6 is attached to a different part of the human body of the touch receiver, and is configured to reproduce the tactile stimulus sensed by the corresponding tactile sensor 5.

[0021] The tactile reproduction system 1 in this example is configured as a system that reproduces the tactile sensations of each part perceived by a person wearing a tactile sensor 5 in a recipient, and is capable of handling cases where the two are located remotely. Furthermore, according to the configuration of the tactile reproduction system 1 shown in Figure 1, the tactile signal obtained by sensing with the tactile sensor 5 is transmitted to the decoding device 3 via the network 4, making it possible to reproduce the tactile sensation in approximately real time.

[0022] In the example of Figure 1, the number of tactile sensors 5 and tactile presentation devices 6, i.e., the number of parts of the human body that sense and reproduce tactile stimuli, is three, but the number of tactile sensors 5 and tactile presentation devices 6 is not limited to this.

[0023] [1-2. Encoding device configuration] 2 is a diagram illustrating an example of the internal configuration of the encoding device 2. Note that in FIG. 2, the tactile sensors 5 shown in FIG. 1 are also shown together with the example of the internal configuration of the encoding device 2. As shown in the figure, the encoding device 2 includes a plurality of amplifiers 21, a plurality of A / D converters 22, a preprocessing unit 23, an encoding unit 24, a control unit 25, a storage unit 26, a communication unit 27, and a bus . As shown in the figure, a preprocessing unit 23, an encoding unit 24, a control unit 25, a storage unit 26, and a communication unit 27 are connected via a bus 28 and are capable of data communication with one another.

[0024] The detection signal of each tactile sensor 5 is input to a corresponding amplifier 21 and adjusted to an appropriate dynamic range, and then input to a corresponding A / D converter 22 and subjected to A / D conversion (analog / digital conversion). Each A / D converted detection signal (i.e., tactile signal for each part) is input to the pre-processing unit 23. In the pre-processing unit 23, various types of digital signal processing such as noise removal and calibration of the sensor characteristics of the tactile sensor 5 are performed. Each haptic signal that has been subjected to signal processing by the pre-processing unit 23 is input to the encoding unit 24 .

[0025] The encoding unit 24 is configured by, for example, a DSP, and encodes each input haptic signal according to a predetermined data format.

[0026] The control unit 25 is configured with a microcomputer having, for example, a CPU, a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and performs overall control of the encoding device 2 by executing processing in accordance with a program stored in the ROM. For example, the control unit 25 performs data communication with an external device via the communication unit 27. The communication unit 27 is configured to be able to perform data communication with an external device via the network 4, and the control unit 25 is able to perform data communication with an external device (particularly the decoding device 3 in this example) connected to the network 4 via the communication unit 27. In particular, the haptic signal encoded by the encoding unit 24 can be transmitted to the decoding device 3 via the communication unit 27.

[0027] The storage unit 26 is a general term for storage devices such as a hard disk drive (HDD) or a solid state drive (SSD), and is used to store various types of data in the encoding device 2. For example, the storage unit 26 stores data necessary for control by the control unit 25. Furthermore, under the control of the control unit 25, the storage unit 26 can also store encoded haptic signals.

[0028] [1-3. Configuration of the decoding device] FIG. 3 is a diagram for explaining an example of the internal configuration of the decoding device 3, and shows the example of the internal configuration of the decoding device 3 together with each of the tactile presentation devices 6 shown in FIG. The decoding device 3 includes a plurality of amplifiers 31, a plurality of D / A converters 32, a post-processing unit 33, a decoding unit , a control unit 35, a storage unit , a communication unit 37, and a bus . The post-processing unit 33, the decoding unit 34, the control unit 35, the storage unit 36, and the communication unit 37 are connected via a bus 38 and are capable of data communication with one another.

[0029] The control unit 35 is configured with a microcomputer having, for example, a CPU, a ROM, a RAM, etc., and performs overall control of the decoding device 3 by executing processes in accordance with a program stored in the ROM. For example, the control unit 35 performs data communication with an external device via the communication unit 37. The communication unit 37 is configured to be able to perform data communication with an external device via the network 4, and the control unit 35 is configured to be able to perform data communication with an external device (particularly, in this example, the encoding device 2) connected to the network 4 via the communication unit 37.

[0030] The control unit 35 causes the communication unit 37 to input the haptic signal (encoded haptic signal) received from the encoding device 2 to the decoding unit .

[0031] The storage unit 36 ​​is a comprehensive representation of storage devices such as HDDs and SSDs, and is used to store various types of data in the decoding device 3. For example, the storage unit 36 ​​stores data necessary for control by the control unit 35.

[0032] The decoding unit 34 decodes the encoded haptic signal in accordance with a predetermined data format to obtain a haptic signal for each body part. The haptic signals for each body part obtained by the decoding unit 34 are input to the post-processing unit 33.

[0033] The post-processing unit 33 performs signal processing such as calibration of the tactile presentation device 6 and predetermined filtering, as necessary, on the input tactile signals for each part.

[0034] Each tactile signal that has passed through the post-processing unit 33 is input to a corresponding D / A converter 32 for D / A conversion (digital / analog conversion), then adjusted to an appropriate dynamic range by a corresponding amplifier 31, and output to a corresponding tactile presentation device 6. This allows each tactile presentation device 6 to be driven based on the tactile signal, and enables generation of a tactile stimulus corresponding to each part of the user.

[0035] Although only haptic signals have been mentioned above, audio signals and video signals can also be transmitted to the decoding device 3 along with the haptic signals to provide sounds and videos to the haptic receiver.

[0036] [1-4. Tactile Reproduction Method as First Embodiment] (encoding method) A tactile sensation reproduction method according to a first embodiment will be described below. The tactile sensation reproduction method according to the first embodiment is a method that focuses on the tactile characteristics of humans. As a guide to human tactile sensitivity, the vibration detection threshold curve shown in Figure 4 has been reported. In Figure 4, the horizontal axis represents frequency, and the vertical axis represents the amplitude of the tactile stimulus (vibration: here, the amplitude of displacement). The vibration detection threshold curve shown in Figure 4 is an example of an experiment that investigated whether or not humans can sense vibration as a touch, that is, tactile sensitivity. Humans cannot perceive vibrations smaller than this curve as touch.

[0037] It is generally known that there are multiple receptors under the human skin that perceive touch. Representative receptors include Meissner's, Merkel's, Ruffini's, and Pacini's. Meissner and Pacini are also called "FA 1" and "FA 2" respectively, where FA stands for "Fast Adapting." Merkel and Ruffini are also called "SA 1" and "SA 2" respectively, where SA stands for "Slow Adapting."

[0038] Figure 5 shows the distribution of neural firing in each receptor when an object is gradually pressed against the skin, held there for a while, and then released. Merkel (SA 1) continues firing while an object is being pressed, and is thought to detect strength (displacement, pressure). Meissner (FA 1) detects the period until the amount of pressure on the object becomes constant, that is, it detects velocity. Pacinian (FA 2) detects the period when the amount of pressure changes, that is, it detects acceleration.

[0039] The vibration detection threshold curves for each receptor are shown in Figure 6. The curves shown in Figure 4 do not represent the characteristics of a single receptor, but rather represent the combined characteristics of the tactile sensations obtained by multiple receptors, as shown in Figure 6.

[0040] The vibration detection threshold curves shown in Figures 4 and 6 indicate that humans can sense vibrations up to about 1 kHz as tactile stimuli. Furthermore, although these figures do not show values ​​above 1 kHz, it is known that humans can actually perceive vibrations as tactile stimuli even at frequencies of a few kHz, although sensitivity drops sharply above that range.

[0041] In most conventional tactile reproduction applications, vibrations up to about 200 Hz are targeted at the highest frequency, because the human tactile sensitivity is highest at around 200 Hz.

[0042] However, as mentioned above, various past experiments have shown that humans can sense vibrations up to 1 kHz as tactile stimuli, and it must be said that it is difficult to reproduce a highly realistic tactile sensation using conventional applications. For example, the vibrations produced when removing a cork from a bottle actually contain high frequencies of several kHz. If these frequencies were only reproduced up to a few hundred Hz, the resulting tactile sensation would be completely different from reality.

[0043] Therefore, in this example, the characteristics of the tactile signal and the tactile presentation device 6 are broadened to about 1 kHz to further enhance the sense of reality. Specifically, in this example, a method is adopted in which a tactile stimulus such as a vibration that actually occurs is sensed to obtain a tactile signal, and a tactile sensation is presented based on the tactile signal.

[0044] In recent years, all kinds of information have been digitized and used, and we are considering digitizing and handling tactile signals in the same way. The amount of digitized data can be thought of as the number of bits required per unit time, i.e., the bit rate. For example, the region that humans can sense on the vibration detection threshold curve shown in Figure 4 is at least 50 dB (-20 dB to 30 dB) or more on the vertical axis (vibration) and approximately 1000 Hz on the horizontal axis. In this example, taking into account the distribution of tactile signals that humans actually sense, we will sense signals in the range of +20 dB from the threshold curve. Specifically, as shown in FIG. 7, the vibration range is assumed to be 70 dB (−20 dB to 50 dB).

[0045] When this signal is digitized using LPCM (Linear Pulse Code Modulation), 1 bit can represent 6 dB, so the vertical axis is 12 bits, and to reproduce up to 1000 Hz, twice the sampling frequency, 2000 Hz (sample / sec), is required, so the required bit rate B0 can be calculated using the following [Equation 1]. B0=12bit / sample×2000sample / sec=24kbit / sec...[Formula 1]

[0046] This value itself is very small compared to, for example, the bit rate of CDs, a typical format for audio signals, which is 700 kbps / ch, so it seems unlikely that incorporating this tactile signal as an additional signal into some system would pose a major problem.

[0047] However, as mentioned above, it is known that the bandwidth of tactile signals that humans can sense extends up to several kHz. For example, if tactile signals are reproduced up to 2000 Hz, the bit rate will be 48 kbit / sec, double that of [Equation 1].

[0048] Furthermore, unlike vision (two eyes) and hearing (two ears), the sense of touch is present everywhere on the surface of the human body. Just considering the fingertips on both hands, there are ten locations, and if we were to handle all of these tactile signals, the bit rate would be ten times higher, at 480 kbit / sec. If we add each finger joint, palm, and other locations, the bit rate would increase dramatically.

[0049] Furthermore, while tactile signals are essentially one-dimensional, the physical phenomenon of vibration can be captured in three axes (x, y, z). To handle all of this requires a bit rate three times higher, at 1440 kbit / sec, which exceeds the 1411 kbit / sec of an audio CD.

[0050] Thus, although the bit rate for one haptic signal is not that large, it becomes enormous when considering the tactile sensations that humans can sense, and it is certain to place a heavy load on a system that handles haptic signals.

[0051] Therefore, in this embodiment, attention is focused on the difference in tactile sensitivity between different parts of the human body. The types, distribution, and sensitivity of receptors in each part of the human body vary greatly, and these characteristics are represented by models known as the "somatosensory homunculus" or "sensory dwarf" (see, for example, http: / / web2.chubu-gu.ac.jp / web_labo / mikami / brain / 32 / index-32.html). These diagrams of the "somatosensory homunculus" and "sensory dwarf" quantitatively deform each part of the human body according to the size of the brain area responsible for processing tactile sensations in that part of the body. However, this does not directly show the differences in the type, distribution, and sensitivity of tactile receptors in each part of the body. Furthermore, the type, distribution, and sensitivity of tactile receptors in each part of the human body have not yet been fully investigated or clarified. However, since it is possible to qualitatively and quantitatively experience the differences in tactile sensitivity between different parts of the body in everyday life, such as using the hands instead of the feet for delicate tasks that require touch, and the roughness of objects being difficult to detect on the belly or back as with the hands, it is easy to imagine that not all parts of the body have exactly the same tactile characteristics. A good example is the presence of fingerprints on the fingers, and many reports and verifications have suggested that this increases tactile sensitivity. Given that there are no fingerprints on the back or belly, it is thought that it would be difficult to capture tactile signals with a single encoding method.

[0052] It is also widely known that when humans concentrate on one sense, their sensitivity to the other senses weakens. Specifically, when humans evaluate only sound, they are sensitive to deterioration in sound quality. However, when evaluating sound and video simultaneously, they are sensitive to deterioration in video but insensitive to deterioration in sound. This phenomenon occurs because vision is more dominant than hearing as a human perceptual mechanism.

[0053] Even when listening to a single sound, if you focus on a specific speaker among a mixture of various sound sources, you can hear that sound more clearly. This has long been known as the cocktail party effect.

[0054] Furthermore, even with video alone, humans do not recognize all objects in their field of vision. They are constantly making choices, such as removing signs and buildings from a streetscape to find family and acquaintances in the crowd, and reading emotions by focusing particularly on their faces (expressions).

[0055] This effect can also be seen in the sense of touch alone: ​​for example, when you are concentrating on the sense of touch with your hands, the sensitivity of your hands is maximized, but the sensitivity of other parts of your body may decrease. For example, even if you can feel a gentle breeze on your feet when you are doing nothing, you may not feel the wind on your feet when you are concentrating on delicate work using your hands.

[0056] As such, human perception of physical quantities is not absolute, but varies greatly depending on where the human consciousness is directed. Perceptual coding technologies for sound and video have become widespread, and these technologies can be said to utilize these human sensory characteristics. These technologies have been developed precisely because the mechanisms of hearing and vision have been largely elucidated, and they actively utilize this knowledge.

[0057] On the other hand, although there are many unknowns regarding the mechanism of tactile perception, particularly regarding tactile receptors as sensors, it can be said that by researching the areas of the brain that process tactile sensations and by utilizing effects that we experience in our daily lives and take for granted, it is possible to achieve unprecedented efficient encoding, transmission, and decoding of tactile signals. However, the technology for efficient encoding, transmission, and decoding of tactile signals that utilizes the human tactile characteristics similar to those of hearing and vision has not yet been established.

[0058] Since the sense of touch is distributed all over the human body, it is conceivable that the tactile presentation device 6 will be attached to any position on the human body, and multiple tactile presentation devices that vibrate independently will be used. As a specific example, FIG. 8 is shown. In the example in Figure 8, haptic playback devices are attached to various parts of the body other than the hands and fingertips, such as the face, abdomen, and feet, to generate vibrations. Audio signals are generally played back in two channels for both ears, but in some cases audio signals of 5.1 channels, 7.1 channels, or even more channels may be played back. In this case, it is necessary to determine the direction and position of the speakers from which each channel's audio signal will be played in order to achieve the intended playback.

[0059] Like audio signals, tactile signals also require information indicating which channel will vibrate which part of the body. Furthermore, when it is assumed that haptic signals are transmitted via the internet or wirelessly, it is natural to think of them as digitized data. In this case, the digitized haptic signals must be treated as data with a fixed time unit called a frame.

[0060] For this reason, a data structure such as that shown in FIG. 8 is required. In this example, the haptic signals (PCM data) for each body part are stored in separate frames. As shown in the figure, the frame has an area for a frame header and an area for storing the actual data of the haptic signal. The frame header stores, as header information for the frame, at least information indicating which body part the haptic signal is for.

[0061] Furthermore, the frames for each part are integrated into a data unit called a stream. A stream has an area for the stream header and an area for storing the frames for each part. The stream header stores header information such as information indicating the specifications of the digitized haptic signal (quantization bit length, sampling frequency, stream data size, etc.) and information on the number of frames included in the stream.

[0062] The haptic signals for each part are coded in this way, and when the haptic signals are transmitted, the coded data is handled in a format in which the streams for each time are arranged on the time axis as shown in the figure. By performing the encoding as described above, it is possible to achieve high convenience in the transmission and reproduction of haptic signals.

[0063] As mentioned above, increasing the number of body parts to which haptic signals are applied increases the bit rate of the haptic signals proportionally, placing a heavy load on the system. Therefore, in this embodiment, the haptic characteristics of each body part are taken into consideration, and the bit allocation of the haptic signals is varied for each body part, thereby reducing the bit rate when captured using haptic signals from all body parts.

[0064] For simplicity, let us consider the case where tactile sensations are presented to one point each on the hand and foot of a human body. Assuming the basic conditions are the same as in [Equation 1], the total bit rate B(hand+foot) required in this case can be calculated as follows [Equation 2]. B(hand+foot)=12bit / sample×2000sample / sec×2=48kbit / sec...[Formula 2]

[0065] As can be seen from the previous explanation, the ratios of brain areas that process tactile sensations in the human hands and feet are significantly different. Although the tactile characteristics (e.g., sensitivity) of the hands and feet have not been quantified to an exact multiple, it is clear from experience that the hands have a superior tactile sensitivity compared to the feet. Therefore, the conversion coefficient (W) of the sensitivity of the feet (S-foot) relative to the sensitivity of the hand (S-hand) is expressed as the following [Equation 3]. S-foot=W×S-hand(W<1.0)...[Formula 3] Although the tactile sensitivity of each finger and palm of the hand is different, for the sake of simplicity we will consider it in terms of individual parts such as the hand and foot.

[0066] Regarding the above conversion coefficient W, if we assume W=0.25, for example, 1 bit corresponds to 6 dB (2 times), so the quantization bit length required for the hand tactile signal is 12 bits, while the quantization bit length required for the foot tactile signal is 10 bits, and the total bit rate B(hand+foot) can be calculated as follows: B(hand)=12bit / sample×2000sample / sec=24kbit / sec...[Formula 4] B(foot)=10bit / sample×2000sample / sec=20kbit / sec...[Formula 5] B(hand+foot)=44kbit / sec...[Formula 6]

[0067] In the above example, the bit rate can be reduced by only about 10% from [Equation 2]. However, if we further assume that the feet can only be perceived at half the frequency of the hands, the total bit rate B(hand+foot) can be calculated as follows: B(hand)=12bit / sample×2000sample / sec=24kbit / sec...[Formula 7] B(foot)=10bit / sample×1000sample / sec=10kbit / sec...[Formula 8] B(hand+foot)=34kbit / sec...[Formula 9] This reduces the bit rate by approximately 30% from [Equation 2].

[0068] Another example of varying the sampling frequency and quantization bit length for each part is shown below. In this example, consider the application of tactile stimulation to the hands, face, and feet. In this case, if the same sampling frequency and quantization bit length are set for the face and feet as for the hands, the total bit rate B(hand+face+foot) is B(hand+face+foot)=72kbit / sec...[Formula 10] It is expressed as:

[0069] Here, for example, the sensitivity coefficients for the face and the feet are set as follows: S-face=W-face×S-hand(W=0.5)...[Formula 11] S-foot=W-foot×S-hand(W<0.25)...[Formula 12] In other words, if the quantization bit length required for the tactile signal of the hand is 12 bits, the quantization bit length for the face is 11 bits, and the quantization bit length for the feet is 10 bits.

[0070] It is also assumed here that the frequency bands in which tactile stimuli can be perceived differ for each part of the body, such as the hands, face, and feet. Specifically, as shown in Figure 9, it is assumed that the hands can only perceive vibrations in the frequency bands up to 1 kHz (see Figure 9A), the face up to 500 Hz (see Figure 9B), and the feet up to 250 Hz (see Figure 9C).

[0071] Under the above conditions, if the quantization bit length required for the hand tactile signal in this case is 12 bits, the total bit rate B (hand+face+foot) can be calculated as follows: B(hand)=12bit / sample×2000sample / sec=24kbit / sec...[Formula 13] B(face)=11bit / sample ×1000sample / sec=11kbit / sec...[Formula 14] B(foot)=10 bit / sample×500sample / sec=5kbit / sec...[Formula 15] B(hand+face+foot)=40kbit / sec...[Formula 16] This allows the total bit rate to be reduced by approximately 45% compared to the case of [Equation 10].

[0072] 10 shows an example of the structure of encoded data when the data format of the haptic signal is different for each body part as described above. As an example, the data structure of a frame and the data structure of a stream corresponding to the case of [Equation 9] are shown here. As shown in the figure, the header of the frame for each body part stores values ​​indicating the quantization bit length and sampling frequency for the haptic signal for that body part. This makes it possible to easily identify the data format of the haptic signal stored in the frame on the decoding device 3 side, even if the data format of the haptic signal differs for each body part. In the case of audio signals, which are one-dimensional signals like haptic signals, even if the signal is composed of multiple frames (or channels), all of those frames (or channels) generally have the same format, so the quantization bit length and sampling frequency only need to be written in the stream header. While using different data formats for haptic signals for each part as described above is more complicated than for audio signals, it is advantageous in terms of bit rate efficiency because it is not necessary for all frames to have the same data format.

[0073] Here, in the above, it is assumed that the sampling frequency and quantization bit length are statically determined, i.e., the bit allocation for each part is statically determined, but it is also possible to dynamically change the bit allocation for each part, for example, in response to changes in conditions over time. As a specific example, consider a case where the bit rate on the transmission path is temporarily limited due to some factor, and the total bit rate of the haptic signals from each part of the body cannot be accommodated within the limited bit rate. In this case, the sampling frequency and quantization bit length of a specific part are restricted so that the bit allocation to that part is reduced so that it fits within the limited bit rate. In this case, it is desirable to restrict the parts to be parts with low tactile sensitivity, i.e., the face and feet in the examples so far, in order to prevent a decrease in tactile reproducibility.

[0074] On the other hand, even if a tactile signal is from a highly sensitive area, if the signal itself does not exist, i.e., if the signal amplitude does not reach a perceptible amplitude, there is little need to transmit it. Therefore, it is possible to reduce the bit allocation for tactile signals from areas with such small signal amplitudes.

[0075] As an example, suppose the parts to which tactile stimulation is applied are the hands, face, and feet. In this case, the encoding device 2 monitors the amplitude value of the tactile signal for each part and determines whether there are any parts where the signal amplitude is small, specifically, where the signal amplitude does not reach an amplitude value that can be perceived by humans. This determination is made, for example, for each frame in the stream. Note that the determination of whether the amplitude has reached a value that can be perceived by humans can be made based on the vibration detection threshold curve shown in Figure 4. As a result of the determination, for a portion where the amplitude value of the haptic signal does not reach a perceptible amplitude value, the bit allocation is reduced, specifically, set to zero. For example, if the only parts that did not reach a perceptible amplitude value were the face and feet, B(hand)=12bit / sample×2000sample / sec=24kbit / sec...[Formula 17] B(face)=0kbit / sec...[Formula 18] B(foot)=0kbit / sec...[Formula 19] B(hand+face+foot)=24kbit / sec...[Formula 20] Bit allocation will be as follows. Conversely, if the only part that is determined not to have reached a perceptible amplitude value is the hand, the following bit allocation is made: B(hand)=0kbit / sec...[Formula 21] B(face)=11bit / sample ×2000sample / sec=22kbit / sec...[Formula 22] B(foot)=10 bit / sample×2000sample / sec20kbit / sec...[Formula 23] B(hand+face+foot)=42kbit / sec...[Formula 24] In [Equation 22] and [Equation 23], the sampling frequency of the face and feet is the same as the sampling frequency of the hands, but it is also possible to use a lower sampling frequency (see [Equation 14] and [Equation 15]).

[0076] By allocating bits as described above, it is possible to transmit necessary haptic signals even under restricted conditions.

[0077] (Encoding side functional configuration) FIG. 11 is a functional block diagram showing the functional configuration of the encoding device 2 for implementing the above-described encoding method. As shown in the figure, the encoding device 2 has functions as an acquisition unit F21 and an encoding unit F22. The encoding unit F22 also has functions as a format conversion unit F23. The acquisition unit F21 acquires tactile signals for each part of the human body. In this example, the acquisition unit F21 corresponds to the amplifier 21 and the A / D converter 22.

[0078] The encoding unit F22 encodes the tactile signals for each body part acquired by the acquisition unit F21 so that the data formats differ between different body parts. In this example, the encoding unit F22 performs encoding using the format conversion unit F23 to change the bit allocation to areas depending on the tactile sensitivity. Specifically, encoding is performed to allocate fewer bits to areas with low tactile sensitivity. That is, the quantization bit length of the tactile signal for areas with low tactile sensitivity is shortened and the sampling frequency is lowered, for example, as in [Equation 7], [Equation 8], [Equation 13], [Equation 14], and [Equation 15].

[0079] Furthermore, the encoding unit F22 of this example reduces bit allocation to areas with small signal amplitudes using the format conversion unit F23. Specifically, for areas where the amplitude value of the haptic signal does not reach a perceptible amplitude value, the encoding unit F22 shortens the quantization bit length of the haptic signal and lowers the sampling frequency, for example, as in [Equation 7], [Equation 8], [Equation 13], [Equation 14], or [Equation 15]. Alternatively, for example, if there is a region where the amplitude value of the tactile signal does not reach a perceptible amplitude value, the bit allocation for the tactile signal of that region is set to zero, as in [Equation 17], [Equation 18], [Equation 19], [Equation 21], [Equation 22], and [Equation 23].

[0080] Furthermore, the encoding unit F22 in this example assigns index information indicating the type of body part to the haptic signal for each body part. Specifically, the information indicating the type of body part is stored in the frame header of the haptic signal as shown in FIG.

[0081] In this example, the function of the encoding unit F22 is realized by the encoding unit 24. To vary the quantization bit length and sampling frequency of the haptic signals depending on the region, for example, each A / D converter 22 is used that has the same quantization bit length and sampling frequency, and the encoding unit 24 performs conversion processing of the quantization bit length and sampling frequency for the haptic signals of the region that require conversion among the haptic signals after A / D conversion.

[0082] The function of varying the quantization bit length and sampling frequency of the tactile signal depending on the body part can also be achieved by using A / D converters 22 with different quantization bit lengths and sampling frequencies. In this case, each A / D converter 22 will perform the function of the encoding unit F22.

[0083] (Decoding method) The decoding device 3 of this embodiment reproduces the haptic signals obtained by the encoding device 2, i.e., the haptic signals in different data formats for different parts of the human body, in accordance with the data format for each part.

[0084] Here, a haptic signal is a one-dimensional signal, and its data can be handled basically in the same way as an audio signal. Audio signals generally have the same quantization bit length and sampling frequency for each channel, and on the playback side, D / A converters 32 with the same specifications can be used for each channel. In contrast, in this embodiment, since the quantization bit length and sampling frequency differ depending on the channel, it is assumed that D / A converters 32 with different specifications will be used for the channels. Therefore, there are concerns about the complexity of the configuration of the decoding device 3 and the resulting increase in costs.

[0085] Therefore, in this example, when playing back the tactile signals of each body part, format conversion processing is performed on the tactile signals of at least one body part, as shown in Fig. 12. The example in Fig. 12 shows a case where the tactile signals of the hand have a quantization bit length of 12 bits and a sampling frequency of 2 kHz, while the tactile signals of the foot have a quantization bit length of 10 bits and a sampling frequency of 1 kHz, and the tactile signals of the foot are subjected to format conversion to align with the data format of the tactile signals of the hand.

[0086] By performing such format conversion, the data formats of the tactile signals from each part can be unified, and D / A converters 32 with the same specifications can be used.

[0087] In this case, in order to easily adjust the sampling frequency to a constant value, it is desirable that the ratio of the sampling frequency between the parts is set to an integer multiple on the encoding device 2 side. For example, as in the example of FIG. p If we set the sampling rate to 1000 le / sec and 2000 sample / sec, we can simply oversample by a factor of 2 to match the former to the latter.

[0088] When oversampling an audio signal, for example from 8000 samples / sec to 16000 samples / sec, aliasing components occur in the playback frequency range of 4000 Hz to 8000 Hz, which can be heard, so an LPF (Low Pass Filter) is essential. This is shown in Figure 13.

[0089] However, in the case of haptic signals, the perceptible frequency range is said to be up to about 1 kHz (sampling frequency = 2 kHz). In addition, the playback range of the haptic presentation device 6 is often limited to this range. For this reason, unlike in the case of audio signals, oversampling of haptic signals does not necessarily require the installation of an LPF.

[0090] For example, D / A converters 32 are often used for audio applications (4 kHz in signal bandwidth, i.e., the minimum sampling frequency is about 8 kHz), so those that support a sampling frequency of 2 kHz are uncommon. For this reason, no matter how efficient the haptic signal is at a 2 kHz specification, it will not be possible to reproduce it as is with a typical D / A converter 32.

[0091] Therefore, in order to enable the use of a general D / A converter 32, the tactile signals from each part are subjected to oversampling processing to convert them into a predetermined sampling frequency, such as a sampling frequency of 8 kHz or higher. For example, if oversampling processing with a sampling frequency of 2 kHz to 8 kHz is performed using simple zero-value interpolation, aliasing will occur in the signal band of 1 kHz to 4 kHz. However, as mentioned above, this frequency band is an area with very low tactile sensitivity, so there is no problem even if the LPF is omitted. Furthermore, if the tactile presentation device 6 itself cannot reproduce this aliasing, the LPF can also be omitted. This is illustrated in Figure 14.

[0092] (Functional configuration on the decoding side) FIG. 15 is a functional block diagram showing the functional configuration of the decoding device 3. As shown in the figure, the decoding device 3 has functions as an acquisition unit F31 and a decoding unit F32. The decoding unit F32 also has functions as a format conversion unit F33.

[0093] The acquisition unit F31 acquires haptic signals in different data formats for different parts of the human body. In this example, the acquisition unit F31 corresponds to a part that acquires the haptic signals for each part transmitted from the encoding device 2, and in this example, corresponds to the communication unit 37.

[0094] The acquisition unit F31 in this example acquires haptic signals in which a data format for each region is defined so that bit allocation to the region is changed according to tactile sensitivity. Specifically, the acquisition unit F31 acquires haptic signals in which a data format for each region is defined so that bit allocation to regions with low tactile sensitivity is reduced. That is, for regions with low tactile sensitivity, haptic signals with a shorter quantization bit length and a lower sampling frequency are acquired, for example, as in [Equation 7], [Equation 8], [Equation 13], [Equation 14], and [Equation 15].

[0095] Furthermore, the acquisition unit F31 of this example acquires a haptic signal in which more bits are allocated to areas with larger signal amplitudes. Specifically, the acquisition unit F31 acquires haptic signals for each area, including haptic signals with a shorter quantization bit length and a lower sampling frequency, for example, as in [Equation 7], [Equation 8], [Equation 13], [Equation 14], or [Equation 15], based on the magnitude of the signal amplitude, for example, based on whether the amplitude value of the haptic signal reaches a perceptible amplitude value. Alternatively, for example, if the amplitude value of the tactile signal does not reach a perceptible amplitude value, and bit allocation is set to zero for some parts as in [Equation 17], [Equation 18], [Equation 19], [Equation 21], [Equation 22], or [Equation 23], the tactile signal is obtained for parts where bit allocation is non-zero.

[0096] Furthermore, the acquisition unit F31 in this example acquires, for each body part, a haptic signal to which index information indicating the type of body part is attached. Specifically, as shown in FIG. 10, the acquisition unit F31 acquires a haptic signal in which information indicating the type of body part is stored in the frame header.

[0097] Furthermore, the decoding unit F32 decodes the haptic signal acquired by the acquisition unit F31. In the decoding device 3 of this example, the configuration for realizing this decoding unit F32 is a portion including at least a decoding unit . In this example, the decoding unit F32 converts at least one of the quantization bit length and the sampling frequency for at least some of the haptic signals for each body part using the format conversion unit F33. Note that, in relation to the specifications of the D / A converter 32, if only the sampling frequency needs to be converted, conversion of the quantization bit length is not necessary, and conversely, if only the quantization bit length needs to be converted, conversion of the sampling frequency is not necessary. The function of the format conversion unit F33 is realized by the decoding unit .

[0098] As described above, the quantization bit length and sampling frequency of a haptic signal may change over time based on the determination results of the signal amplitude, etc. In this example, information about the quantization bit length and sampling frequency is stored in the frame header, so even if the quantization bit length and sampling frequency change over time, the values ​​are indicated in the frame header. The format conversion unit F33 changes the format conversion process content, specifically the process content such as oversampling, for the tactile signals of each body part based on the quantization bit length and sampling frequency information stored in the frame header. This allows the D / A conversion of the haptic signal to be performed appropriately even if the quantization bit length or sampling frequency of the haptic signal changes over time.

[0099] Furthermore, the decoding unit F32 in this example decodes the haptic signal for each part based on index information indicating the part assigned to the haptic signal. Specifically, the decoding unit F32 (decoding unit 34) identifies the part of the haptic signal based on the information indicating the part stored in the frame header, and performs processing according to that part. For example, processing such as outputting the haptic signal to the output channel corresponding to that part is performed.

[0100] While the above example illustrates the reproduction of haptic sensations in near real time, it is also possible to build a system in which haptic signals encoded using the above-described method are stored in a predetermined storage medium and read and played back as needed. In this case, the encoded data can be stored in the storage medium as a data file in a predetermined format. Furthermore, this data file can be processed and edited by the user, and additional information associated with the processing and editing can be tagged, etc. The storage medium may be a removable medium such as an optical disk or a memory card, in which case a decoding device 3 such as a personal computer may be configured to read and play the haptic signal data file stored on the removable medium. As another example, the decoding device 3 serving as a client terminal may acquire and play back a data file of a haptic signal stored in a cloud storage server via the network 4.

[0101] In addition, although the above example illustrates a case where the data format of the haptic signal is different for each body part (i.e., the data format of the haptic signal is different for all body parts), the data format may be the same for some body parts. In other words, it is sufficient that the data format of the haptic signal is different at least for different body parts.

[0102] Here, the functions of the encoding unit F22 and the decoding unit F32 described with reference to Fig. 11 and Fig. 15 can be realized as software processing by a CPU, etc. The software processing is executed based on a program, and the program is stored in a storage device that can be read by a computer device such as a CPU.

[0103] [1-5. Summary of the First Embodiment] As described above, the encoding device (F2) of the first embodiment includes an encoding unit (F22) that encodes haptic signals using different data formats for different parts of the human body.

[0104] This makes it possible to reduce the amount of data in tactile signals by taking into account differences in tactile characteristics between parts of the human body, for example by utilizing differences in tactile sensitivity between parts of the human body and allocating less data to tactile signals from parts with low tactile sensitivity. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0105] Furthermore, in the encoding device according to the first embodiment, the encoding unit encodes the haptic signal using different data formats for parts of the human body having different tactile characteristics.

[0106] This makes it possible to reduce the amount of data in tactile signals by utilizing differences in tactile sensitivity between parts of the human body. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0107] Furthermore, in the encoding device according to the first embodiment, the encoding unit encodes the haptic signal so as to change the bit allocation to the parts according to the haptic sensitivity.

[0108] This makes it possible to reduce the amount of data in the tactile signal while taking into account differences in tactile characteristics between parts. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0109] Furthermore, in the encoding device according to the first embodiment, the encoding unit encodes the haptic signal so that the quantization bit length differs between different parts.

[0110] This makes it possible to reduce the data amount of the tactile signal by shortening the quantization bit length for areas with low tactile sensitivity (low sensitivity to the amplitude of tactile stimulation). Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0111] Furthermore, in the encoding device according to the first embodiment, the encoding unit encodes the haptic signal so that the sampling frequency differs between different parts.

[0112] This makes it possible to reduce the amount of data in the tactile signal by lowering the sampling frequency for areas with low tactile sensitivity (low sensitivity to the frequency of tactile stimulation). Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0113] Furthermore, in the encoding device according to the first embodiment, the encoding unit has a format conversion unit (F23) that converts at least one of the quantization bit length and the sampling frequency of at least one of the haptic signals for each body part.

[0114] By performing encoding with different quantization bit lengths or different sampling frequencies in this way, it is possible to reduce the amount of data.

[0115] Furthermore, in the encoding device according to the first embodiment, the encoding unit performs encoding such that more bits are allocated to portions of the haptic signal where the signal amplitude is large.

[0116] As a result, for tactile signals from areas where the signal amplitude is small, for example, where the signal amplitude does not reach a perceptible amplitude, i.e., areas where it is estimated that it is difficult to perceive tactile stimulation in terms of the size of the signal amplitude, the bit allocation is reduced, including not transmitting the signal itself. Therefore, the data volume of the tactile signal is reduced taking into account the differences in tactile characteristics of each part, so that the data volume of the tactile signal can be reduced while ensuring the reproducibility of the tactile sensation, thereby improving the efficiency of the system related to tactile reproduction.

[0117] Furthermore, in the encoding device according to the first embodiment, the encoding unit performs encoding in such a way that index information indicating the location of a part is added to the haptic signal.

[0118] This increases the ease and accuracy of identifying the parts of the haptic signal, simplifies the configuration of the decoding device, reduces costs, and improves the accuracy of haptic reproduction for each part.

[0119] The encoding method according to the first embodiment is an encoding method that encodes haptic signals using different data formats for different parts of the human body.

[0120] This encoding method also provides the same effects and advantages as the encoding device of the first embodiment described above.

[0121] Furthermore, the encoding program according to the first embodiment is a program that causes an information processing device to realize an encoding function for encoding haptic signals using different data formats for different parts of the human body.

[0122] The encoding device according to the first embodiment can be realized by using the encoding program according to the first embodiment.

[0123] The decoding device (F3) as the first embodiment includes a decoding unit (F32) that decodes haptic signals encoded in different data formats for different parts of the human body.

[0124] This makes it possible to reduce the amount of data in tactile signals by taking into account differences in tactile characteristics between parts of the human body, for example by utilizing differences in tactile sensitivity between parts of the human body and allocating less data to tactile signals from parts with low tactile sensitivity. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0125] Furthermore, in the decoding device according to the first embodiment, the decoding unit decodes haptic signals in different data formats for parts of the human body having different tactile characteristics.

[0126] This makes it possible to reduce the amount of data in tactile signals by utilizing differences in tactile sensitivity between parts of the human body. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0127] Furthermore, in the decoding device according to the first embodiment, the decoding unit decodes a haptic signal in which a data format is defined for each part so that bit allocation to the part is changed according to the haptic sensitivity.

[0128] This makes it possible to reduce the amount of data in the tactile signal while taking into account differences in tactile characteristics between parts. Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0129] Furthermore, in the decoding device according to the first embodiment, the decoding unit decodes haptic signals having different quantization bit lengths for different parts.

[0130] This makes it possible to reduce the data amount of the tactile signal by shortening the quantization bit length for areas with low tactile sensitivity (low sensitivity to the amplitude of tactile stimulation). Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0131] Furthermore, in the decoding device according to the first embodiment, the decoding unit decodes haptic signals having different sampling frequencies for different parts of the body.

[0132] This makes it possible to reduce the amount of data in the tactile signal by lowering the sampling frequency for areas with low tactile sensitivity (low sensitivity to the frequency of tactile stimulation). Therefore, it is possible to reduce the amount of data of the haptic signal while ensuring the reproducibility of the haptic sensation, and to improve the efficiency of the system related to haptic sensation reproduction.

[0133] Furthermore, in the decoding device of the first embodiment, the decoding unit has a format conversion unit (F33) that inputs haptic signals that differ in at least one of the quantization bit length and sampling frequency between different parts and converts at least one of the quantization bit length and sampling frequency for at least one of the haptic signals for each part.

[0134] This eliminates the need to provide multiple types of D / A converters for D / A conversion of tactile signals according to differences in quantization bit length and sampling frequency depending on the part of the body. Therefore, the configuration of the decoding device can be simplified and the cost can be reduced.

[0135] Furthermore, in the decoding device according to the first embodiment, the decoding unit decodes a haptic signal in which more bits are allocated to portions with larger signal amplitudes.

[0136] As a result, for tactile signals from areas where the signal amplitude is small, for example, where the signal amplitude does not reach a perceptible amplitude, i.e., areas where it is estimated that it is difficult to perceive tactile stimulation in terms of the size of the signal amplitude, the bit allocation is reduced, including not transmitting the signal itself. Therefore, the data volume of the tactile signal is reduced taking into account the differences in tactile characteristics of each part, so that the data volume of the tactile signal can be reduced while ensuring the reproducibility of the tactile sensation, thereby improving the efficiency of the system related to tactile reproduction.

[0137] Furthermore, in the decoding device according to the first embodiment, the decoding unit receives haptic signals to which index information indicating the type of body part is attached, and decodes the haptic signals for each body part based on the index information.

[0138] This increases the ease and accuracy of identifying the parts of the haptic signal, simplifies the configuration of the decoding device, reduces costs, and improves the accuracy of haptic reproduction for each part.

[0139] The decoding method according to the first embodiment is a decoding method for decoding haptic signals encoded in different data formats between different parts of the human body.

[0140] The decoding method as the first embodiment can also provide the same functions and effects as the decoding device as the first embodiment described above.

[0141] The decoding program according to the first embodiment is a program that causes an information processing device to realize a decoding function for decoding haptic signals that have been coded in different data formats for different parts of the human body.

[0142] Such a program can realize the decoding device of the first embodiment described above.

[0143] 2. Second Embodiment [2-1. Overview of the tactile reproduction system] Next, a second embodiment will be described. The second embodiment addresses the transmission delay of haptic signals. FIG. 16 shows an example of the configuration of a tactile reproduction system 1A according to the second embodiment. In the following description, parts that are the same as parts that have already been described will be given the same reference numerals and description thereof will be omitted.

[0144] 16, the haptic reproduction system 1A differs from the haptic reproduction system 1 of the first embodiment in that a transmitting device 3A is provided instead of the decoding device 3, and a receiving device 40 is also provided. In the tactile reproduction system 1A, each tactile presentation device 6 worn by the haptic receiver is not connected to the transmitting device 3A by wire, but rather the transmission of tactile signals from the transmitting device 3A to the tactile presentation device 6 is performed via wireless communication. The receiving device 40 functions as a device that receives the tactile signals transmitted by the transmitting device 3A via wireless communication and transmits them to the tactile presentation device 6. In this example, each tactile presentation device 6 is connected to the receiving device 40 by wire, and the part surrounded by the dashed line in the figure, i.e., the receiving device 40 and each tactile presentation device 6, is the part worn by the haptic receiver.

[0145] For example, when the decoding device 3 and each tactile presentation device 6 are connected by wire as illustrated in the first embodiment, if the size of the decoding device 3 is large, it may cause annoyance to the haptic receiver wearing the tactile presentation device 6. This annoyance is expected to increase as the number of parts to which tactile stimulation is applied increases. The configuration of the tactile reproduction system 1A as described above makes it possible to arrange a receiving device 40 that is smaller in size than the decoding device 3, thereby preventing the tactile receiver from experiencing the above-mentioned inconvenience.

[0146] [2-2. Configuration of the transmitting device] FIG. 17 is a diagram showing an example of the internal configuration of the transmission device 3A. The differences from the decoding device 3 shown in Figure 3 are that the amplifier 31, D / A converter 32, post-processing unit 33, and decoding unit 34 are not provided, that a transmission data generation unit 51 and a wireless communication unit 52 are provided, and that a control unit 35A is provided instead of the control unit 35. As shown in the figure, a transmission data generation unit 51, a wireless communication unit 52, a control unit 35A, a storage unit 36, and a communication unit 37 are connected via a bus 38 and are capable of data communication with one another.

[0147] The transmission data generation unit 51 performs a predetermined encoding on the encoded data of the haptic signal obtained from the encoding device 2 via the communication unit 37 via the network 4, and generates encoded data to be transmitted to the receiving device 40. The processing performed by the transmission data generating unit 51 in this example will be explained later.

[0148] The wireless communication unit 52 performs short-distance wireless communication using a predetermined communication method such as Bluetooth (registered trademark).

[0149] The control unit 35A is configured to have, for example, a microcomputer, and performs overall control of the transmission device 3A. In particular, the control unit 35A controls the output of encoded data from the communication unit 37 to the transmission data generation unit 51, and controls the output of encoded data generated by the transmission data generation unit 51 to the wireless communication unit 52. This makes it possible to send tactile signals from each part to an external device via the wireless communication unit 52.

[0150] In the second embodiment, the data format of the tactile signals for each part may be different between different parts as described in the first embodiment, or may be the same for all parts.

[0151] [2-3. Configuration of receiving device] FIG. 18 is a diagram for explaining an example of the internal configuration of the receiving device 40, and shows each tactile presentation device 6 together with the example of the internal configuration of the receiving device 40. As shown in the figure, the receiving device 40 includes an amplifier 31, a D / A converter 32, a post-processing unit 33, and a decoding unit 34A, as well as a control unit 41, a storage unit 42, a wireless communication unit 43, and a bus 44. The post-processing unit 33, the decoding unit 34A, the control unit 41, the storage unit 42, and the wireless communication unit 43 are connected via the bus 44 and are capable of data communication with one another.

[0152] The wireless communication unit 43 performs short-range wireless communication using a method such as Bluetooth that enables communication with the wireless communication unit 52 in the transmission device 3A. The encoded data transmitted from the transmission device 3A is received by the wireless communication unit 43.

[0153] The control unit 41 is configured to have, for example, a microcomputer, and performs overall control of the receiving device 40 . The storage unit 42 is a storage device similar to the storage units 26 and 36, for example, and is used to store various data used by the control unit 40 and the like.

[0154] The decoding unit 34A performs the same processing as the decoding unit 34 described in the first embodiment on the coded data input via the wireless communication unit 43. That is, in cases where the data format of the haptic signal differs for each part, the decoding unit 34A performs processing such as format conversion on the haptic signal of the necessary part. The decoding unit 34A also performs processing on the input coded data to counter transmission delays, which will be described later.

[0155] [2-4. Tactile Reproduction Method as Second Embodiment] Here, when transmitting data over any section, whether wired or wireless, data loss may occur. It is possible to check for data loss and compensate for it by retransmitting the data if any loss occurs, but this results in an increase in the effective bit rate and transmission delays because extra data is transmitted.

[0156] As a specific example, consider wireless transmission over Bluetooth, which is affected by Wi-Fi (registered trademark), which uses the same carrier frequency. Bluetooth is particularly susceptible to interference because its radio wave strength is weaker than that of Wi-Fi. During transmission, coded data is sent in predetermined transmission units called packets, but packet loss occurs frequently under the conditions described above. For example, when transmitting audio signals using Bluetooth's A2DP (Advanced Audio Distribution Profile), a large stream buffer is prepared on the receiving side, with the assumption that packets will be resent if packet loss occurs. Playback is initiated only after a certain amount of stream has accumulated in this buffer, preventing audio dropouts.

[0157] However, the amount of stored coded data is equivalent to the amount of delay, and if the amount of stored data becomes too large, problems will occur. For example, when watching video on a display and listening to the corresponding audio, lip synchronization will become out of sync, which can be a major problem.

[0158] The transmission of haptic signals also faces the same problems as those for audio signals. For example, if you have a video of a baseball bat being swung and you want to give a tactile stimulus to the recipient at the moment the ball hits the bat, if the stream buffer capacity is large or if retransmissions are repeated due to packet loss, there is a risk that the visual and tactile sensations will be clearly out of sync, causing a significant sense of discomfort to the recipient.

[0159] Thus, when considering synchronization between tactile sensation and other senses, it is desirable to minimize packet retransmissions and to keep the stream buffer on the receiving side as small as possible. To achieve this, it is effective to use a data structure that does not cause major problems even if some packets are lost. Specifically, as mentioned earlier, humans have different tactile sensitivity depending on the body part, so a data structure that takes this into account is adopted.

[0160] As shown in Figure 10, the stream has header information indicating the boundaries of frames, followed by the actual haptic signal. If the time granularity of the haptic signal in a frame is made too small, the proportion of the header in the transmission increases, reducing efficiency. Conversely, if the time granularity is made too large, delay problems and interference can occur, so it is desirable to encode the haptic signal for a few milliseconds within a frame.

[0161] The frames of each part are transmitted by the aforementioned packets, but the longer the packet size, the longer the time required for transmission, making the packet more susceptible to interference stochastically. Therefore, in the second embodiment, frames are packed in order from the beginning of the stream in order of the area with the highest tactile sensitivity.

[0162] A specific example is shown in FIG. FIG. 19 shows an example of the order of frames for each body part when the tactile sensitivity is highest for the hands, followed by the face and then the feet. In this case, in the stream, frames (tactile signals) for each part are arranged in the order of hands, face, and feet from the beginning.

[0163] In conventional transmission, the receiving side returns an ACK (acknowledgment) only if the packet has been completely received successfully. In contrast, in the second embodiment, an ACK is returned if at least a frame with high tactile sensitivity that is part of the stream included in the packet is successfully received. To achieve this, a parameter is recorded in the stream header that specifies the size of data that must be successfully received before an ACK is returned (see "Ack Allow Size" in Figure 19). Once the receiving side confirms that the portion specified by this parameter has been successfully received, it returns an ACK to the sending side.

[0164] By doing this, from the perspective of frames with high tactile sensitivity, it appears as if they are being transmitted using short packets, which reduces the probability of interference. Even if frames with low tactile sensitivity are lost due to interference, there is an advantage in that as long as data with high tactile sensitivity is reproduced, the impact on human tactile senses is relatively small. In other words, a certain degree of tactile reproducibility can be guaranteed.

[0165] Although the above example shows a case where the priority for arranging frames is determined by tactile sensitivity, the priority can also be determined by the amplitude of the tactile signal. For example, frames from a portion where the amplitude of the tactile signal reaches a perceptible amplitude value can be preferentially arranged (placed at the beginning) within the stream.

[0166] In addition, in the above, an ACK is returned upon receiving the frame of the highest priority part, but it is also possible to return an ACK upon receiving frames of the top n priorities (n is a natural number greater than or equal to 2 and less than the total number of parts in the stream), for example, by returning an ACK upon receiving signals with the top two priorities. To achieve the same effect as transmitting using apparently short packets as described above, an ACK can be returned only when the haptic signal from at least the area with the highest priority has been received, excluding the area with the lowest priority.

[0167] Even if apparently short packets are used as described above, the interference rate per unit time is constant, so interference does not become zero within the time it takes to transmit the packets. Therefore, as shown in the example of FIG. 20, frames with high tactile sensitivity (frames with high priority) are arranged in multiple locations in the stream and transmitted with redundancy.

[0168] In this case, the stream header stores information about which frames must be successfully received before an ACK can be returned, such as ID specification information that specifies the frame ID.If even one frame identified by this ID specification information is successfully received, an ACK will be returned even if others are lost. Figure 20 shows an example in which four frames with frame IDs 0 to 3 are arranged in one stream, with the first frame with ID=0 representing a hand, the second frame from the top with ID=1 representing a face, the third frame from the top with ID=2 representing a foot, and the fourth frame from the top with ID=3 representing a hand again, and corresponding to this, ID designation information specifying ID=0 and 3 is stored in the stream header.

[0169] In this way, if the disturbance rate per unit time is constant, the tolerance to loss of frames with high tactile sensitivity can be improved many times over.

[0170] In the above method, the priority order may be determined not based on the level of tactile sensitivity but on the magnitude of the signal amplitude of the tactile signal. In addition, although the above example shows that the parts with redundancy of the haptic signal are limited to the parts with the highest priority, it is also possible to provide redundancy of the haptic signal for the parts with the top n priorities (n is a natural number greater than or equal to 2 and less than the total number of parts in the stream).

[0171] (Sender's functional configuration) FIG. 21 is a functional block diagram showing the functional configuration of the transmission device 3A. As shown in the figure, the transmission device 3A has the functions of an encoding unit F34 and a transmission unit F35. The encoding unit F34 generates encoded data by encoding the tactile signals for each part of the human body in order of priority for each part. The function of this encoding unit F34 is realized by the transmission data generation unit 51. Here, the encoding unit F34 arranges the haptic signals in order of priority. Specifically, for this function, the transmission data generation unit 51 generates encoded data having a stream structure in which frames are arranged from the beginning in order of the area with the highest haptic sensitivity, as shown in the example of Fig. 19. At this time, the transmission data generation unit 51 stores a parameter corresponding to the "Ack Allow Size" shown in Fig. 19 in the stream header.

[0172] Furthermore, the encoding unit F34 provides redundancy to the haptic signals of areas with high priority. For this function, the transmission data generation unit 51 generates encoded data in which redundancy is provided only to the haptic signals of areas with the highest tactile sensitivity, as illustrated in Fig. 20. At this time, the transmission data generation unit 51 stores the ID designation information described above in the stream header.

[0173] The transmission unit F35 transmits the coded data generated by the coding unit F34. Specifically, the wireless communication unit 52 transmits the coded data generated by the transmission data generation unit 51 to an external device (receiving device 40) under the control of, for example, the control unit 35A.

[0174] (Receiving side functional configuration) FIG. 22 is a functional block diagram showing the functional configuration of the receiving device 40. As shown in the figure, the receiving device 40 has the functions of a receiving unit F41 and a decoding unit F42. The receiving unit F41 receives coded data from a transmitting device that generates coded data by encoding haptic signals for each body part of the human body in order of priority for each part. In other words, in this example, the receiving unit F41 receives coded data generated by the transmission data generating unit 51 of the transmitting device 3A. In this example, the receiving unit F41 corresponds to the wireless communication unit 43.

[0175] The decoding unit F42 decodes the encoded data received by the receiving unit F41 in accordance with the priority order for each part. In this example, the function of the decoding unit F42 is realized by the decoding unit 34A. Specifically, the decoding unit F42 performs the following process in response to the case where haptic signals are transmitted in descending order of priority: That is, it sends an affirmative response to the transmitting device on the condition that the haptic signal of at least the highest priority area, excluding the area with the lowest priority, has been received. As a function of this decoding unit F42, the decoding unit 34A refers to the above-mentioned parameters stored in the stream header of the encoded data generated by the transmission data generation unit 51 and received by the wireless communication unit 43, determines whether the frame identified from the parameters has been received (reception was successful), and returns an ACK to the transmitting device 3A side if it determines that the frame has been received. The decoding unit 34A repeats this process for each stream.

[0176] In addition, in cases where tactile signals are transmitted with redundancy given to areas with high priority, the decoding unit F42 sends a positive response to the transmitting device in response to the reception of at least one tactile signal for the area with the redundancy given. As a function of this decoding unit F42, the decoding unit 34A refers to the above-mentioned ID designation information stored in the stream header of the encoded data generated by the transmission data generation unit 51 and received by the wireless communication unit 43, determines whether at least one frame out of the multiple frames indicated by the ID designation information has been received (reception was successful), and returns an ACK to the transmitting device 3A side if it determines that it has been received. The decoding unit 34A repeats this process for each stream.

[0177] Here, the function of the decoding unit F42 described with reference to Fig. 22 can be realized as software processing by a CPU or the like. The software processing is executed based on a program, and the program is stored in a storage device that can be read by a computer device such as a CPU.

[0178] [2-5. Summary of the second embodiment] As described above, the transmission system of the second embodiment includes a transmitting device (3A) having an encoding unit (F34, transmission data generating unit 51) that generates encoded data by encoding the tactile signals of each part of the human body in accordance with the priority of each part, and a transmitting unit (F35) that transmits the encoded data, a receiving device (40) having a receiving unit (F41, wireless communication unit 43) that receives the encoded data transmitted by the transmitting unit, and a decoding unit (F42, 34A) that decodes the encoded data received by the receiving unit in accordance with the priority.

[0179] By transmitting the tactile signals from each area in an order according to priority as described above, it is possible to reduce data loss of tactile signals from areas that should be prioritized, such as areas with high tactile sensitivity.Furthermore, by decoding the tactile signals transmitted in this manner according to priority, even if data loss occurs on the transmission path, if the data is not a tactile signal from an area that should be prioritized, only the tactile signal received from the area that should be prioritized will be the subject of tactile reproduction. Therefore, it is possible to reduce the occurrence of transmission delays of haptic signals due to data loss on the transmission path, and to prevent a decrease in haptic reproducibility due to transmission delays.

[0180] In addition, the receiving device (same 40) as the second embodiment is equipped with a receiving unit (same F41, wireless communication unit 43) that receives encoded data from a transmitting device (same 3A) that generates encoded data by encoding tactile signals for each part of the human body according to the priority of each part, and a decoding unit (same F42, 34A) that decodes the encoded data received by the receiving unit according to the priority.

[0181] By having the transmitting device transmit the tactile signals from each area in an order according to priority, it is possible to reduce data loss of tactile signals from areas that should be prioritized, such as areas with high tactile sensitivity.Furthermore, by decoding the tactile signals transmitted in this manner according to priority, even if data loss occurs on the transmission path, if the tactile signal is not from an area that should be prioritized, only the tactile signal received from the area that should be prioritized will be the subject of tactile reproduction. Therefore, it is possible to reduce the occurrence of transmission delays of haptic signals due to data loss on the transmission path, and to prevent a decrease in haptic reproducibility due to transmission delays.

[0182] In the receiving device according to the second embodiment, the tactile signals for each part of the human body are tactile signals for each part of the human body having different tactile characteristics.

[0183] This makes it possible to reproduce tactile sensations at different parts of the body with different tactile properties. Therefore, appropriate tactile reproduction can be performed according to the tactile characteristics of each part.

[0184] Furthermore, in the receiving device of the second embodiment, the transmitting device transmits haptic signals in order of priority, and the decoding unit sends a positive response to the transmitting device on the condition that the haptic signal of at least the area with the highest priority has been received, excluding the area with the lowest priority.

[0185] This is equivalent to sending tactile signals for areas with high priority using short packets. Therefore, it is possible to reduce the rate of data loss of tactile signals for areas with high priority, and to prevent a decrease in tactile reproducibility.

[0186] Furthermore, in the receiving device according to the second embodiment, the priority is determined based on the ease of perception of the tactile stimulus.

[0187] This makes it possible to reduce data loss of tactile signals in areas that have a high priority in terms of ease of perception of tactile stimulation, such as high tactile sensitivity or large signal amplitude. Therefore, it is possible to prevent a decrease in tactile reproducibility.

[0188] In the receiving device according to the second embodiment, the priority is determined based on the level of tactile sensitivity.

[0189] This makes it possible to reduce data loss of tactile signals in areas that are more susceptible to tactile stimuli in terms of sensitivity to tactile stimuli. Therefore, it is possible to prevent a decrease in tactile reproducibility.

[0190] Furthermore, in the receiving device according to the second embodiment, the priority is determined based on the magnitude of the amplitude of the haptic signal.

[0191] This makes it possible to reduce data loss of the tactile signal in areas where tactile stimulation is easily perceived in terms of the magnitude of the signal amplitude of the tactile signal. Therefore, it is possible to prevent a decrease in tactile reproducibility.

[0192] Furthermore, in the receiving device of the second embodiment, the transmitting device transmits tactile signals with redundancy for areas with high priority, and the decoding unit sends a positive response to the transmitting device in response to the reception of at least one tactile signal for the area with redundancy.

[0193] This makes it possible to reduce the occurrence of data loss in the haptic signal. Therefore, it is possible to prevent a decrease in tactile reproducibility.

[0194] In addition, the decoding method as the second embodiment is a decoding method that decodes coded data received from a transmitting device that generates coded data by encoding tactile signals for each part of the human body according to the priority of each part, in accordance with the priority order.

[0195] The decoding method according to the second embodiment can also provide the same functions and effects as those of the receiving device according to the second embodiment.

[0196] In addition, the program as the second embodiment is a program that causes an information processing device to realize the function of decoding, in accordance with priority, coded data received from a transmitting device that generates coded data by encoding tactile signals for each part of the human body according to the priority of each part.

[0197] Such a program can realize the receiving device of the second embodiment described above.

[0198] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0199] <3. This technology> The present technology can also be configured as follows. (1) A decoder is provided for decoding tactile signals encoded in different data formats between different parts of the human body. Decryption device. (2) The decoding unit Decoding tactile signals in different data formats between different parts of the human body with different tactile characteristics The decoding device according to (1) above. (3) The decoding unit A tactile signal having a data format determined for each of the regions is decoded so that bit allocation to the regions is changed according to tactile sensitivity. The decoding device according to (1) or (2). (4) The decoding unit Decoding tactile signals with different quantization bit lengths between different parts The decoding device according to (3) above. (5) The decoding unit Decoding tactile signals with different sampling frequencies between different parts The decoding device according to (3) or (4). (6) The decoding unit The device has a format conversion unit that receives tactile signals having different quantization bit lengths and sampling frequencies between the different parts and converts at least one of the quantization bit lengths and sampling frequencies for at least one of the tactile signals for each part. The decoding device according to any one of (1) to (5). (7) The decoding unit The haptic signal to which index information indicating the type of the body part is attached is input, and the haptic signal for each body part is decoded based on the index information. The decoding device according to any one of (1) to (6). (8) a receiving unit that receives coded data from a transmitting device that generates coded data by encoding the tactile signals for each human body part in order of priority for each part; a decoding unit that decodes the encoded data received by the receiving unit in accordance with the priority order. Receiving device. (9) The tactile signals for each part of the human body are tactile signals for each part of the human body having different tactile characteristics. The receiving device according to (8) above. (10) the transmitting device transmits the haptic signals in order of the parts with the highest priority, The decoding unit sends an affirmative response to the transmitting device on the condition that the haptic signal of at least the area with the highest priority has been received, excluding the area with the lowest priority. The receiving device according to (8) or (9). (11) The priority is a priority in terms of tactile sensitivity. The receiving device according to (10) above. (12) the transmitting device transmits the haptic signal with redundancy for the part with the higher priority; The decoding unit and transmitting an acknowledgment to the transmitting device in response to receiving at least one haptic signal from the redundancy-provided haptic signal. The receiving device according to any one of (8) to (11) above. [Explanation of symbols]

[0200] 1, 1A tactile reproduction system, 2 encoding device, 3 decoding device, 3A transmission device, 5 tactile sensor, 6 tactile presentation device, 22 A / D converter, 24 encoding unit, F21 acquisition unit, F22 encoding unit, 32 D / A converter, 34, 34A decoding unit, 35, 35A control unit, 37 communication unit, F31 acquisition unit, F32 playback unit, F23, F33 format conversion unit, F34 encoding unit, F35 transmission unit, 40 receiving device, 43 wireless communication unit, 51 transmission data transmission unit, 52 wireless communication unit, F41 receiving unit, F42 decoding unit

Claims

1. a receiving unit that receives tactile signals for each part of the human body; a decoding unit that decodes the haptic signal received by the receiving unit, the haptic signal has a header for storing header information and an area for storing actual data of the haptic signal; The header stores information indicating the part, The receiving unit receiving the coded data from a transmitting device that generates coded data by encoding the tactile signals for each body part of the human body in order of priority for each body part; The decoding unit The encoded data received by the receiving unit is decoded in accordance with the priority order. Decryption device.

2. The decoding unit Decoding the tactile signals with different bit allocations between the parts of the human body with different tactile characteristics The decoding device according to claim 1 .

3. The decoding unit The tactile signal, in which a data format for each part is determined so that bit allocation to the part is changed according to tactile sensitivity, is decoded. The decoding device according to claim 1 .

4. The decoding unit Decoding the haptic signals with different quantization bit lengths between different parts The decoding device according to claim 3 .

5. The decoding unit Decoding the tactile signals with different sampling frequencies between different parts The decoding device according to claim 3 .

6. The decoding unit The haptic signal processing device includes a format conversion unit that receives the haptic signals having different quantization bit lengths and sampling frequencies between the different parts and converts the quantization bit lengths and sampling frequencies of at least one of the haptic signals for each part. The decoding device according to claim 2 .

7. the transmitting device transmits the haptic signals in order of the parts with the highest priority, The decoding unit sends an affirmative response to the transmitting device on the condition that the haptic signal of at least the area with the highest priority has been received, excluding the area with the lowest priority. The decoding device according to claim 1 .

8. The priority is a priority in terms of tactile sensitivity. The decoding device according to claim 7.

9. the transmitting device transmits the haptic signal with redundancy for the part with the higher priority; The decoding unit and transmitting an acknowledgment to the transmitting device in response to receiving at least one haptic signal from the redundancy-provided haptic signal. The decoding device according to claim 1 .

10. The decoding device receiving the coded data from a transmitting device that generates coded data by encoding the haptic signals for each body part, the haptic signals having a header that stores header information including information indicating a body part of the human body and an area that stores actual data, in order of priority for each body part; The received encoded data is decoded in accordance with the priority order. Decryption method.

11. a receiving function for receiving the coded data from a transmitting device that generates coded data by encoding the haptic signals for each body part, the encoded data having a header that stores header information including information indicating a body part of the human body and an area that stores actual data, in order of priority for each body part; a decoding function for decoding the received encoded data in accordance with the priority order, program.

Images