Decoding device, decoding method, program, encoding device, and encoding method

Abstract

A decoding apparatus according to the present technology includes a first decoding unit configured to decode, for encoded data obtained by encoding a haptic signal, a first band signal that is a signal of a first band in the haptic signal, the encoded data including data obtained by encoding the first band signal and haptic intensity information indicating a haptic intensity of a second band in the haptic signal that is different from the first band; a second decoding unit configured to decode, based on the haptic intensity information in the encoded data, a second band signal that is a signal of the second band in the haptic signal; and a synthesis unit that synthesizes the first band signal decoded by the first decoding unit and the second band signal decoded by the second decoding unit.

Description

Technical Field

[0001] This technology relates to a decoding apparatus and method for decoding encoded tactile signals, a program for the decoding apparatus, an encoding apparatus and method for encoding tactile signals, and a program for the encoding apparatus. Background Technology

[0002] Technology is currently being developed in which a device worn by the user vibrates to apply tactile stimulation to the user. In this technology, tactile stimulation refers to physical phenomena such as vibration that allow the user to experience touch. Furthermore, the generation of tactile stimulation is called tactile presentation.

[0003] Such haptic presentation techniques are utilized in a variety of devices across different fields. In one example, a terminal device equipped with a touch panel (such as a smartphone) vibrates the touch panel in response to a user's touch operation to apply tactile stimulation to the user's fingers, thereby allowing the tactile sensation of contact with buttons or similar elements displayed on the touch panel to be presented. Furthermore, for example, devices for listening to music (such as headphones) can apply haptic stimulation while playing music to emphasize deep bass sounds in the music. Additionally, for example, devices providing computer games, virtual reality (VR), etc., can vibrate controllers to apply haptic stimulation in response to controller operation or the scene of the content, enhancing the user's immersive experience of the content.

[0004] Furthermore, techniques have been developed for applying tactile stimulation to a user based on tactile signals received from an external device. In one example, Patent Document 1, mentioned below, discloses a technique for applying tactile stimulation to a user while changing the frequency and amplitude of vibration based on the received signal.

[0005] Reference List

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2016-202486 Summary of the Invention

[0008] The problem to be solved by the present invention

[0009] In this reference, it is envisioned that a tactile reproduction system for reproducing tactile information provides multiple tactile presentation devices, and that tactile stimuli are applied to multiple parts of the human body.

[0010] However, increasing the number of sites to which tactile stimulation is applied will increase the number of channels used for tactile signals, resulting in an increase in data volume. This increase in tactile signal data volume undesirably leads to an increase in processing load, transmission delay, and other factors associated with tactile reproduction.

[0011] This technology was developed with the above considerations in mind, and aims to enhance tactile reproducibility while reducing the amount of data in the tactile signal.

[0012] Solution to the problem

[0013] The decoding apparatus according to the present technology includes: a first decoding unit configured to decode a first frequency band signal, which is a signal of a first frequency band in the tactile signal, for encoded data obtained by encoding the tactile signal, the encoded data including data obtained by encoding the first frequency band signal and tactile intensity information, the tactile intensity information indicating the tactile intensity of a second frequency band in the tactile signal that is different from the first frequency band; a second decoding unit configured to decode the second frequency band signal, which is a signal of a second frequency band in the tactile signal, based on the tactile intensity information in the encoded data; and a synthesis unit for synthesizing the first frequency band signal decoded by the first decoding unit and the second frequency band signal decoded by the second decoding unit.

[0014] For encoded data generated by the encoding device according to the present technology, this configuration enables the decoding of tactile signals related to the second frequency band, while reducing or preventing the degradation of tactile reproducibility.

[0015] In the decoding apparatus according to the present technology described above, it is conceivable that the first frequency band is set to a frequency band in a frequency range lower than that of the second frequency band.

[0016] For tactile signals at a predetermined frequency or in a larger frequency band, the human body can perceive changes in tactile intensity relatively easily, but it is relatively difficult to perceive changes in frequency.

[0017] In the decoding apparatus according to the present technology described above, it is conceivable that tactile intensity information is calculated for each frequency in the second frequency band, and the second decoding unit decodes the second frequency band signal based on the total value of tactile intensity indicated by the tactile intensity information calculated for each frequency.

[0018] This configuration allows for the reproduction of the appropriate tactile intensity as the tactile intensity of the second frequency band based on the tactile intensity of each frequency in the second frequency band signal before encoding.

[0019] In the decoding apparatus according to the present technology described above, it is conceivable that tactile intensity information is calculated for one or more main frequencies in the second frequency band, and the second decoding unit decodes the second frequency band signal based on the tactile intensity indicated by the tactile intensity information calculated from the main frequencies.

[0020] This configuration allows for the reproduction of the appropriate tactile intensity as the tactile intensity of the second frequency band based on the tactile intensity of the dominant frequency in the second frequency band signal before encoding.

[0021] In the decoding device according to the present technology described above, it is conceivable that the second decoding unit converts the tactile intensity indicated by the tactile intensity information into a signal amplitude and generates a periodic signal with the converted signal amplitude as a decoding signal of the second frequency band signal, and the signal frequency of the periodic signal is set to a frequency in the second frequency band.

[0022] For encoded data generated by the encoding device according to the present technology, this configuration enables the decoding of second frequency band signals to reduce or prevent degradation of tactile reproducibility.

[0023] In the decoding device according to the present technology described above, it is conceivable that the periodic signal has a frequency that substantially matches the resonant frequency of the tactile presentation device.

[0024] This configuration enables improved efficiency in driving haptic rendering devices. Furthermore, the haptic rendering device referred to herein is a device that performs haptic rendering based on haptic signals obtained by a decoder according to this technology.

[0025] In the decoding device according to the present technology described above, it is conceivable that the periodic signal has a frequency set based on the tactile sensitivity characteristics of the human body.

[0026] This configuration allows the frequency of the periodic signal to be set to a frequency that the human body is highly sensitive to touch, thereby improving the efficiency of driving the haptic presentation device.

[0027] In the decoding device according to the present technology described above, it is conceivable that the periodic signal has a frequency set based on the characteristics of human auditory sensitivity.

[0028] This configuration allows the frequency of the periodic signal to be set to a frequency that the human body has low auditory sensitivity to, thereby reducing noise generated when driving the haptic presentation device.

[0029] In the decoding apparatus according to the present technology described above, it is conceivable that the second decoding unit sets the frequency of the periodic signal based on the operation settings.

[0030] This configuration allows users to optionally select the frequency of the periodic signal.

[0031] Furthermore, the decoding method according to this technology includes: a first decoding step, for encoded data obtained by encoding a tactile signal, decoding a first frequency band signal that is a signal of a first frequency band in the tactile signal, the encoded data including data obtained by encoding the first frequency band signal and tactile intensity information, the tactile intensity information indicating the tactile intensity of a second frequency band in the tactile signal that is different from the first frequency band; a second decoding step, decoding a second frequency band signal that is a signal of a second frequency band in the tactile signal based on the tactile intensity information in the encoded data; and a synthesis step, synthesizing the first frequency band signal decoded in the first decoding step and the second frequency band signal decoded in the second decoding step.

[0032] This decoding method can even achieve operations similar to those obtained by a decoder according to the present technology described above.

[0033] Furthermore, according to the first procedure of this technology, the information processing device implements the following procedure: a first decoding function, for encoded data obtained by encoding the tactile signal, decoding a first frequency band signal that is a signal of a first frequency band in the tactile signal, the encoded data including data obtained by encoding the first frequency band signal and tactile intensity information indicating the tactile intensity of a second frequency band in the tactile signal that is different from the first frequency band; a second decoding function, decoding a second frequency band signal that is a signal of a second frequency band in the tactile signal based on the tactile intensity information in the encoded data; and a synthesis function, synthesizing the first frequency band signal decoded by the first decoding function and the second frequency band signal decoded by the second decoding function.

[0034] This first procedure according to the present technology allows for the implementation of the decoding device according to the present technology described above.

[0035] The encoding apparatus according to the present technology includes: a frequency band segmentation unit configured to segment a tactile signal into a first frequency band signal as a first frequency band signal and a second frequency band signal as a second frequency band signal different from the first frequency band signal; a first encoding unit configured to encode the first frequency band signal; a tactile intensity calculation unit configured to calculate tactile intensity information indicating the tactile intensity of the second frequency band based on the second frequency band signal; and an encoded data generation unit configured to generate encoded data including the tactile intensity information and data obtained by encoding the first frequency band signal.

[0036] This configuration allows the use of human sensitivity characteristics to improve the compression efficiency of tactile signal information. Sensitivity characteristics refer to the ease with which both tactile intensity and frequency changes are perceived within a predetermined frequency band, but in another frequency band, tactile intensity changes are easily perceived while frequency changes are difficult to perceive.

[0037] In the encoding apparatus according to the present technology described above, it is conceivable that the first frequency band is set to a frequency band in a frequency range lower than that of the second frequency band.

[0038] For tactile signals at a predetermined frequency or in a larger frequency band, the human body can perceive changes in tactile intensity relatively easily, but it is relatively difficult to perceive changes in frequency.

[0039] In the encoding apparatus according to the present technology described above, it is conceivable that the first frequency band and the second frequency band have a variable division frequency between the first frequency band and the second frequency band.

[0040] Making the segmentation frequency variable allows for a balance between the effect of data reduction caused by encoding and the effect of reducing or preventing the decrease in tactile reproducibility.

[0041] In the encoder according to the present technology described above, it is conceivable that the frequency band segmentation unit changes the segmentation frequency based on the stability of communication with external devices.

[0042] This configuration allows for adjusting the segmentation frequency to enhance data reduction when communication with external devices transmitting encoded data is unstable; that is, by reducing the bit rate of the communication data. Conversely, during stable communication, the segmentation frequency can be adjusted to improve tactile reproducibility.

[0043] In the coding apparatus according to the present technology described above, it is conceivable that the frequency band segmentation unit reduces the segmentation frequency when communication stability is low rather than when communication stability is high.

[0044] This configuration allows for altering the segmentation frequency to enhance data reduction, i.e., reducing the bit rate of communication data during unstable communication. Conversely, during stable communication, the segmentation frequency can be altered to improve haptic reproduction.

[0045] In the encoding device according to the present technology described above, it is conceivable that the tactile intensity calculation unit calculates tactile intensity information for each frequency in the second frequency band.

[0046] This configuration allows for the reproduction of the appropriate tactile intensity as the tactile intensity of the second frequency band based on the tactile intensity of each frequency in the second frequency band signal before encoding.

[0047] In the encoding device according to the present technology described above, it is conceivable that the tactile intensity calculation unit calculates tactile intensity information of only one or more main frequencies in the second frequency band.

[0048] This configuration allows for the reproduction of the appropriate tactile intensity as the tactile intensity of the second frequency band based on the tactile intensity of the dominant frequency in the second frequency band signal before encoding.

[0049] Furthermore, the encoding method according to this technology includes: a frequency band segmentation step, dividing the tactile signal into a first frequency band signal as a first frequency band signal and a second frequency band signal as a second frequency band signal different from the first frequency band; a first encoding step, encoding the first frequency band signal; a tactile intensity calculation step, calculating tactile intensity information indicating the tactile intensity of the second frequency band based on the second frequency band signal; and an encoded data generation step, generating encoded data including the tactile intensity information and the data obtained by encoding the first frequency band signal.

[0050] This encoding method can even achieve operations similar to those obtained by the encoding device according to the above-described technology.

[0051] Furthermore, according to the second procedure of this technology, the information processing device implements the following procedure: a frequency band segmentation function, which divides the tactile signal into a first frequency band signal as a signal of a first frequency band and a second frequency band signal as a signal of a second frequency band different from the first frequency band; a first encoding function, which encodes the first frequency band signal; a tactile intensity calculation function, which calculates tactile intensity information indicating the tactile intensity of the second frequency band based on the second frequency band signal; and an encoded data generation function, which generates encoded data including the tactile intensity information and the data obtained by encoding the first frequency band signal.

[0052] This second procedure according to the present technology allows for the implementation of the encoder described above according to the present technology. Attached Figure Description

[0053] Figure 1 An exemplary configuration of a tactile reproduction system including an encoder and a decoder according to an embodiment of the present technology is shown.

[0054] Figure 2 A diagram illustrating an exemplary internal configuration of an encoder as an implementation.

[0055] Figure 3 This is a diagram illustrating an exemplary internal configuration of a playback device as a first embodiment.

[0056] Figure 4 A diagram illustrating an exemplary internal configuration of a decoder as an implementation.

[0057] Figure 5 This is a diagram illustrating a usage example of a tactile reproduction system.

[0058] Figure 6 This is a graph illustrating the vibration detection threshold curve.

[0059] Figure 7 A functional block diagram illustrating the function of the encoding unit in the embodiment.

[0060] Figure 8 It is a diagram showing the tactile sensitivity characteristics of each sensory receptor in the human body.

[0061] Figure 9 This is a diagram illustrating the intensity of tactile sensation.

[0062] Figure 10 This is a diagram illustrating the main frequency.

[0063] Figure 11 A diagram illustrating an example of an encoded data format.

[0064] Figure 12This is a diagram illustrating an exemplary data structure that describes tactile intensity information.

[0065] Figure 13 This is a flowchart illustrating an exemplary processing procedure for implementing an encoding technique according to an embodiment.

[0066] Figure 14 This is a diagram illustrating an example of a machine learning technique that uses a DNN model to specify the dominant frequency.

[0067] Figure 15 This is a functional block diagram illustrating the function of the decoding unit according to an embodiment.

[0068] Figure 16 This is a flowchart illustrating an exemplary processing procedure for implementing a decoding technique according to an embodiment.

[0069] Figure 17 This is a diagram illustrating an example of responding to the frequency of an operation setting periodic signal.

[0070] Figure 18 This is a diagram illustrating an example of a split frequency control that depends on communication stability. Detailed Implementation

[0071] Embodiments according to the present technology will now be described in the following order with reference to the accompanying drawings.

[0072] <1. Overview of Tactile Reproduction Systems>

[0073] <2. Encoder Configuration>

[0074] <3. Configuration of playback device>

[0075] <4. Decoder Configuration>

[0076] <5. Examples of the Use of Tactile Reproduction Systems>

[0077] <6. Tactile Reproduction Technology According to the Implementation Method>

[0078] [6-1. Challenges related to tactile signal transmission]

[0079] [6-2. Encoding Techniques]

[0080] [6-3. Decoding Technology]

[0081] [6-4. Control of frequency division based on communication stability]

[0082] <7. Overview of Implementation Methods>

[0083] <8. This technology>

[0084] The terms used in this article are now defined as follows:

[0085] - Tactile stimulation: Physical phenomena that cause people to perceive touch, such as vibration.

[0086] -Tactile presentation: Produces tactile stimulation.

[0087] - Tactile information: Information perceived by touch, such as vibration information.

[0088] - Tactile signals: Signals that represent patterns of tactile stimulation, such as signals that represent vibration waveforms.

[0089] -Tactile receiver: A person who experiences tactile sensation.

[0090] - Tactile sensitivity: The sensitivity to the subjective perception of the intensity of tactile stimuli. It depends on the receptors and locations on the body.

[0091] - Tactile sensitivity characteristics: Properties related to a person's tactile sensitivity. These depend on the body part (hands, face, feet, etc.).

[0092] - Encoded data: Data generated through encoded signals. It has streams and frames as more specific concepts.

[0093] - Encoded tactile data: Data generated by encoding tactile signals.

[0094] <1. Overview of Tactile Reproduction Systems>

[0095] Figure 1 An exemplary configuration of a tactile reproduction system 1 including an encoder (encoder 2) and a decoder (decoder 3) according to an embodiment of the present technology is shown.

[0096] According to this embodiment, the environment for realizing tactile reproduction can be classified into a recording environment and a reproduction environment. The recording environment encodes the tactile signal obtained by sensing tactile information (tactile stimulus) as a target and records the encoded tactile data Dc obtained through encoding. The reproduction environment reproduces the tactile information based on the tactile signal obtained by decoding the encoded tactile data Dc.

[0097] As shown in the figure, the tactile reproduction system 1 includes multiple tactile sensors 5 and an encoder 2 in the recording environment, and also includes a playback device 4, a decoder 3, and multiple tactile presentation devices 6 in the reproduction environment. The encoder 2 is connected to the tactile sensors 5. The playback device 4 can acquire the encoded tactile data Dc. The decoder 3 can communicate wirelessly with the playback device 4. The multiple tactile presentation devices 6 are connected to the decoder 3.

[0098] Tactile sensor 5 senses tactile stimuli, and in this example, a vibration sensor, such as a piezoelectric pickup or an accelerometer, is used. The tactile sensor comes into contact with the target object to be sensed (i.e., the human body in this example) to output vibration or motion in the form of a voltage change.

[0099] In this example, each tactile sensor 5 is connected to the encoder 2 via a line and attached to different parts of the human body, which is the target object, to sense tactile stimuli occurring at each part.

[0100] The encoder 2 includes, for example, a computer processor, such as a central processing unit (CPU) or a digital signal processor (DSP). The encoder 2 encodes the detection signals (tactile signals) obtained by each tactile sensor 5 according to a predetermined data format and stores the resulting encoded tactile data Dc in, for example, an internal storage device.

[0101] Playback device 4 includes a computer processor such as a CPU or DSP and sends the acquired encoded haptic data Dc to decoder 3. In one example, playback device 4 acquires the encoded haptic data Dc recorded in the recording environment via a desired network such as the Internet, home network, local area network (LAN), and satellite communication network. The encoded haptic data Dc can also be acquired by playback device 4 in the form of being recorded on a portable recording medium.

[0102] Decoder 3 decodes the encoded haptic data Dc received from playback device 4 and drives each haptic presentation device 6 based on the haptic signals obtained through decoding.

[0103] The tactile presentation device 6 generates tactile stimulation, and in this example, a vibrating device such as a vibrator or actuator is used.

[0104] In this example, the corresponding tactile presentation device 6 is attached to different parts of the human body of the tactile receiver and reproduces the tactile stimulation sensed by the corresponding tactile sensor 5.

[0105] In this example, each haptic presentation device 6 is wired to the decoder 3. The portion enclosed by the dashed line in the figure (i.e., the portion including the decoder 3 and the haptic presentation device 6) is attached to the haptic receiver.

[0106] The tactile reproduction system 1 can be configured such that the playback device 4 has the function of the decoder 3, and the playback device 4 is connected to the corresponding tactile presentation device 6 via a wire. However, in this case, there is a possibility of causing annoyance to the tactile receiver wearing the tactile presentation device 6. This annoyance tends to increase with the number of parts of the body receiving tactile stimulation.

[0107] Figure 1 The configuration of the tactile reproduction system 1 shown makes it possible to prevent such annoyance to the tactile receiver.

[0108] exist Figure 1 The tactile reproduction system 1 shown reproduces the tactile sensation of each body part perceived by a person wearing tactile sensors 5 to the tactile receiver, so it is usable even when two people are separated from each other.

[0109] Furthermore, in this embodiment, the number of tactile sensors 5 and tactile presentation devices 6 (i.e., the number of human body parts that perceive and reproduce tactile stimuli) is at least three or more.

[0110] <2. Encoder Configuration>

[0111] Figure 2 A diagram illustrating an exemplary internal configuration of encoder 2 is provided. In addition to the exemplary internal configuration of encoder 2, Figure 2 It also shows Figure 1 5. Tactile sensor.

[0112] As shown in the figure, encoder 2 includes multiple amplifiers 21, multiple analog-to-digital (A / D) converters 22, a preprocessor 23, an encoding unit 24, a control unit 25, a storage unit 26, a communication unit 27, and a bus 28. The preprocessor 23, encoding unit 24, control unit 25, storage unit 26, and communication unit 27 are connected via bus 28, enabling them to communicate with each other.

[0113] The signal detected by the tactile sensor 5 is input to the corresponding amplifier 21, where the detected signal is adjusted to have an appropriate dynamic range. The resulting signal is then input to the respective A / D converter 22, where the signal is converted from analog to digital (A / D).

[0114] The detected signals after A / D conversion (i.e., tactile signals obtained from each body part) are input to the preprocessor 23. The preprocessor 23 performs various digital signal processing operations, such as noise reduction or calibration of the sensor characteristics of the tactile sensor 5.

[0115] Each tactile signal, obtained after signal processing in preprocessor 23, is input to encoding unit 24.

[0116] The encoding unit 24 includes, for example, a DSP. The encoding unit 24 encodes the input tactile signal according to a predetermined data format to obtain the encoded tactile data Dc.

[0117] Furthermore, the encoding of tactile signals according to this embodiment will be described later.

[0118] The control unit 25 includes, for example, a microcomputer with a CPU, read-only memory (ROM), random access memory (RAM), etc. The control unit 25 performs processing according to the program stored in the ROM to control the entire encoder 2.

[0119] In one example, the control unit 25 communicates data with an external device via the communication unit 27.

[0120] The communication unit 27 is capable of communicating data with external devices via a network such as the Internet. The control unit 25 is capable of communicating data with external devices connected to the network via the communication unit 27. Specifically, the encoded tactile data Dc obtained by the encoding unit 24 can be transmitted to the external device via the communication unit 27.

[0121] Storage unit 26 includes a storage device such as a hard disk drive (HDD) or a solid-state drive (SSD) and is arranged within encoder 2 for storing various types of data. In one example, storage unit 26 stores data required for control by control unit 25. Furthermore, under the control of control unit 25, encoded tactile data Dc obtained by encoding unit 24 can be stored in storage unit 26.

[0122] <3. Configuration of playback device>

[0123] Figure 3 This is a diagram illustrating an exemplary internal configuration of playback device 4.

[0124] As shown in the figure, the playback device 4 includes a control unit 41, a communication unit 42, a media driver 43, a storage unit 44, and a wireless communication unit 45, and also includes a bus 46 that connects these components to communicate with each other.

[0125] The control unit 41 includes, for example, a microcomputer with a CPU, ROM, RAM, etc., and controls the entire playback device 4.

[0126] The communication unit 42 is capable of communicating data with external devices via a network such as the Internet. The control unit 41 is capable of communicating data with external devices connected to the network via the communication unit 42. Specifically, encoded tactile data Dc can be received by the communication unit 42 from external devices (such as server devices on a network).

[0127] Media drive 43 is a removable portable recording medium and a reader / writer unit capable of writing data to or reading data from an attached recording medium. Examples of recording media supported by media drive 43 may include memory cards (e.g., portable flash memory), optical disc recording media, etc.

[0128] The media driver 43 enables the reading of encoded tactile data Dc recorded on a portable recording medium.

[0129] Storage unit 44 includes a storage device such as an HDD or SSD and is arranged in playback device 4 for storing various types of data. In one example, storage unit 44 stores data required by control unit 41 to perform control. Furthermore, under the control of control unit 41, encoded haptic data Dc read by media drive 43 or encoded haptic data Dc received from external devices by communication unit 42 can be stored in storage unit 44.

[0130] The wireless communication unit 45 performs short-range wireless communication using a predetermined communication scheme such as Bluetooth (registered trademark).

[0131] In this regard, as part of the overall control described above, the control unit 41 controls the communication unit 42 to receive encoded tactile data Dc, or controls the media driver 43 to read encoded tactile data Dc. Furthermore, the control unit 41 controls the wireless communication unit 45 to transmit the encoded tactile data Dc obtained by the communication unit 42 or the media driver 43 to the decoder 3.

[0132] <4. Decoder Configuration>

[0133] Figure 4 The diagram illustrates an exemplary internal configuration of the decoder 3, and in addition to the exemplary internal configuration of the decoder 3, each haptic presentation device 6 is also shown.

[0134] As shown in the figure, decoder 3 includes multiple amplifiers 31, multiple digital-to-analog (D / A) converters 32, a post-processor 33, and a decoding unit 34. Decoder 3 also includes a control unit 35, a wireless communication unit 36, a storage unit 37, an operation unit 38, a display unit 39, and a bus 30. The post-processor 33, decoding unit 34, control unit 35, wireless communication unit 36, and storage unit 37 are connected via bus 30, allowing data to communicate with each other.

[0135] The control unit 35 includes, for example, a microcomputer with a CPU, ROM, RAM, etc., and controls the entire decoder 3.

[0136] The wireless communication unit 36 ​​uses a communication scheme such as Bluetooth to conduct short-range wireless communication with the wireless communication unit 45 of the playback device 4. The wireless communication unit 36 ​​receives encoded tactile data Dc transmitted from the playback device 4.

[0137] Storage unit 37 is, for example, a storage device similar to storage unit 26, storage unit 44, etc., and various data used by storage control unit 35, etc.

[0138] The operation unit 38 includes various operation elements or operators such as buttons, keys and touch panels (touch sensors) disposed in the decoder 3, and outputs operation input information corresponding to the operation input to the control unit 35.

[0139] Display unit 39 includes a display device such as a liquid crystal display (LCD) or an organic electroluminescent (EL) display, and displays various types of information such as image information based on instructions from control unit 35.

[0140] The decoding unit 34 uses the techniques described later to decode the encoded tactile data Dc input via the wireless communication unit 36 ​​to obtain tactile signals for each body part. The tactile signals for each body part obtained by the decoding unit 34 are input to the post-processor 33.

[0141] The post-processor 33 performs signal processing on the input tactile signals for each body part, such as calibration or predetermined filtering of the tactile presentation device 6 (as needed).

[0142] The tactile signal processed by the post-processor 33 is input to the corresponding D / A converter 32, where the signal undergoes digital-to-analog (D / A) conversion. The signal is then adjusted to have an appropriate dynamic range in the corresponding amplifier 31 and output to the corresponding tactile presentation device 6.

[0143] This configuration enables the corresponding tactile presentation device 6 to be driven based on tactile signals, thereby applying the tactile stimulus to be sensed to the tactile receiver in the detection environment (i.e., reproducible tactile information).

[0144] In addition, while tactile signals are the subject of the above description, audio or video signals can also be recorded together with tactile signals to provide sound or images along with tactile information to the tactile receiver.

[0145] <5. Examples of the Use of Tactile Reproduction Systems>

[0146] It is conceivable that, in addition to video, content presented through tactile feedback could also be reproduced.

[0147] Figure 5 This is a diagram illustrating an example of the use of the tactile reproduction system 1.

[0148] exist Figure 5In this process, encoded data is recorded as content Cnt by recording synchronously with video. This encoded data is obtained by encoder 2, which encodes tactile signals recorded by tactile sensors 5 attached to person Hm1 (tactile sensor 5b of the torso, tactile sensor 5h of the fingers, and tactile sensor 5f of the feet in the figure), recording tactile sensations in addition to the video at the time of content generation. During playback, the recorded content Cnt is transmitted to decoder 3 using, for example, wireless communication, and decoding unit 34 of decoder 3 decodes the received content Cnt. This configuration allows tactile presentation to be performed based on the corresponding tactile signals by tactile presentation devices 6 (tactile presentation devices 6b of the torso, tactile presentation devices 6h of the fingers, and tactile presentation devices 6f of the feet in the figure) worn by person Hm2 as a tactile receiver.

[0149] Examples of scenes in the video that actually depict tactile sensations include a character striking (being struck), being shot (being hit), being hit by a storm, or feeling the ground shaking.

[0150] The diagram illustrates waveforms of tactile signals from the torso, hands, and feet as actual examples of tactile signals within the content. Described as a time series, this example begins with a scenario where person Hm1 shoots an opponent; the vibrations caused by the recoil from the bullet occur in the fingers. Then, in a scenario where the opponent also shoots person Hm1 in the torso (his body), vibrations occur in the torso caused by the impact of the shot. Subsequently, in a scenario of an earthquake, the ground vibrations gradually propagate to the feet, torso, and fingers.

[0151] In one instance, in addition to video and audio, playing such content also reproduces tactile sensations through vibration, allowing the person Hm2, as a tactile receiver, to experience high-quality reality.

[0152] <6. Tactile Reproduction Technology According to the Implementation Method>

[0153] [6-1. Challenges related to tactile signal transmission]

[0154] A description of a tactile reproduction technique according to an embodiment is now given.

[0155] The first thing to describe is a tactile reproduction technology based on an embodiment, focusing on the tactile characteristics of humans.

[0156] Figure 6 The vibration detection threshold curve shown is known as a reference for human tactile sensitivity. Furthermore, in Figure 6 In the diagram, the horizontal axis represents frequency, and the vertical axis represents the amplitude of the tactile stimulus (in this example, vibration is represented as displacement). Figure 6The vibration detection threshold curve is based on experimental results described in the article entitled "Four Cahnnesl Mediate the Mechanical Aspects of Touch", SJ, Bolanowski, 1988.

[0157] exist Figure 6 The vibration detection threshold curve shown is an example of experimentally examining whether a person perceives vibration as touch (i.e., tactile sensitivity). Humans failed to perceive vibrations smaller than those shown in the curve as touch.

[0158] Figure 6 The frequency at which humans have the highest tactile sensitivity is typically around 200 Hz. Therefore, devices or applications that generate vibrations are often designed to produce vibrations up to around 200 Hz.

[0159] On the other hand, although Figure 6 The results are not shown, but it is well known that humans can perceive vibrations with frequencies up to about 1 kHz as tactile sensations. Humans can perceive vibrations with a frequency component of about 1 kHz and vibrations without a frequency component of about 1 kHz as different tactile sensations.

[0160] In one instance, the vibrations generated when the cork is pulled from a bottle include vibrations with frequencies up to several kHz. When such vibrations are transmitted to the user as vibrations with frequencies up to several hundred Hz from a device that provides tactile feedback, the user cannot feel a satisfyingly realistic tactile vibration when the cork is pulled.

[0161] Therefore, it is necessary to use vibrations with frequencies up to about 1 kHz to present a tactile sensation in order to provide users with a more realistic tactile experience.

[0162] However, widening the bandwidth of the frequencies included in the signal increases the amount of data in the signal, making delays more likely to occur during signal transmission or reception. In other words, improved haptic quality can cause situations where the haptic sensation fails to be presented at the proper timing.

[0163] Describe the delays in signal transmission or reception and tactile presentation using specific examples.

[0164] First, a description of the tactile signal data volume is given. In the case of transmitting tactile signals between devices, the tactile signals are first converted into digital data. The capacity of digital data is expressed as the bit depth transmitted or processed per unit of time, i.e., the bit rate (B). At this point, tactile sensitivity depends not only on the vibration frequency but also on the amplitude. In one example, the above... Figure 6Experimental results show that the amplitude of vibration perceived by humans is approximately 50 dB (range -20 dB to 30 dB) or greater, and the frequency is approximately 1000 Hz. Furthermore, considering the distribution of tactile information actually felt by humans, the amplitude of the vibration is estimated to be approximately 70 dB.

[0165] When converting the tactile signal TS into digital data using linear pulse code modulation (LPCM), the vibration amplitude can be represented by 1 bit, which is 6 dB. In other words, a vibration amplitude of 70 dB requires 12 bits. On the other hand, with a vibration frequency of 1000 Hz, the sampling frequency is doubled to 2000 Hz, and the bit rate B0 is given by the following formula (1):

[0166] B0 = 12 bits / sample × 2000 samples / second = 24kb / s... Formula (1)

[0167] This value is relatively very small, less than, for example, the bit rate per channel of a CD = 700 kbps (kbps / ch). CDs are a representative format for audio signals, so this tactile signal seems unlikely to cause significant problems when incorporated into any other system.

[0168] However, as mentioned above, the bandwidth of tactile signals that are known to be perceptible to humans extends up to several kHz. In one instance, when reproducing tactile signals up to 2000 Hz, the bit rate becomes 48 kb / s, which is twice that of Equation (1).

[0169] Furthermore, unlike vision (both eyes) and hearing (both ears), touch is distributed across the entire surface of the human body. Considering only the fingertips of both hands, there are ten points to be sensed, and if tactile signals from all fingertips were to be processed, the bit rate would increase tenfold to 480 kb / s. The increased number of body parts to be sensed in the joints of the palms and fingers would significantly increase the bit rate.

[0170] Furthermore, tactile signals are essentially one-dimensional signals. However, the physical phenomenon of vibration can be captured on three axes (x, y, z). To handle all of this, a bit rate of 1440 kb / s is required, that is, three times higher, and this value is much larger than the 1411 kb / s bit rate of an audio CD.

[0171] In this way, the total amount of tactile signal data to be used increases as the reproducibility of the tactile stimulus improves and the number of user body parts applying the tactile stimulus increases. This increase in total data volume then leads to a heavy load on the network system transmitting the tactile signals.

[0172] In addition to the total amount of data mentioned above, other factors besides the total amount of data can be considered as factors contributing to the delay.

[0173] In one instance, when using wireless communication to transmit haptic signals, the encoded data of the haptic signals may be lost due to interference on the transmission line. If data loss occurs, the data is retransmitted from the transmitting device, and there may be a delay in the time it takes for the receiving side to complete the data reception. In other words, the increased amount of data to be retransmitted increases the time spent retransmitting data in response to data loss, resulting in a further delay until the transmission or reception of the haptic signal is successfully completed.

[0174] In this way, if a delay occurs until the transmission of the tactile signal is complete, tactile reproducibility may be reduced in some cases. Specifically, tactile stimuli may become out of sync with other sensory content (such as images and sounds) because the tactile stimulus is not provided to the user at the appropriate time.

[0175] Now, a description of an application of a specific wireless communication mode considering the above situation is given.

[0176] Devices that perform haptic feedback are installed, come into contact with the user, and, from the perspective of device weight, are generally expected to communicate wirelessly with other devices. However, in the case of using broadband wireless communication such as Wi-Fi, the device's battery size increases from a power consumption perspective, potentially reducing user convenience. Furthermore, with Wi-Fi, there is typically processing time from the signal transmission request on the transmitting side to the processing of the received signal on the receiving side, thus leading to potentially greater latency compared to other wireless communications.

[0177] On the other hand, short-range wireless communication, such as Bluetooth (registered trademark), can perform communication with low power consumption and low latency compared to other wireless communications, and is therefore considered suitable for the transmission of tactile signals. However, short-range wireless communication allows for a smaller amount of data to be transmitted at a time compared to other wireless communications. In one example, in the transmission of content that applies tactile stimulation to a user simultaneously with video or sound, one can imagine a situation where the communication capacity allocated to the transmission of tactile signals is insufficient.

[0178] Furthermore, aside from services used for streaming video or audio over the Internet, in cases where the aim is to transmit tactile sensations to the user, for example, the communication capacity allocated for transmitting tactile signals is easily insufficient due to Quality of Service (QoS) features corresponding to the network's line conditions.

[0179] In view of the above, this embodiment aims to reduce the delay of tactile signals and prevent the degradation of tactile reproducibility by reducing the amount of data transmitted without compromising tactile reproducibility as much as possible.

[0180] [6-2. Encoding Techniques]

[0181] Figure 7 This is a functional block diagram illustrating the function of the encoding unit 24.

[0182] As shown in the figure, the encoding unit 24 includes a signal input unit 24a, a frequency band segmentation unit 24b, a low-frequency signal encoding unit 24c, a tactile intensity calculation unit 24d, a multiplexer 24e, and a signal output unit 24f.

[0183] The signal input unit 24a receives a predetermined number of channels of tactile signals as input for each signal by sampling a fixed number of times. The following description illustrates the case where similar processing is performed for each channel. Furthermore, the tactile signals to be input are set to have a frequency band of 2 kHz and a sampling frequency of 4 kHz.

[0184] The signal input unit 24a divides the input tactile signal into appropriate processing blocks (e.g., the number of samples within a 5-millisecond (ms) time period).

[0185] This segmented processing block is referred to as a "frame" below. The processing following signal input unit 24a is performed on a frame-by-frame basis.

[0186] The frequency band division unit 24b performs frequency band division processing on the input signal from the signal input unit 24a. Specifically, the input signal is divided into a low-frequency signal and a high-frequency signal. Furthermore, the division frequency associated with this frequency band division processing will be described later.

[0187] The low-frequency signal encoding unit 24c receives the low-frequency signal obtained by undergoing frequency band segmentation processing in the frequency band segmentation unit 24b as input, and performs encoding processing on the low-frequency signal using a predetermined encoding technique.

[0188] For this purpose, low-frequency signal encoding techniques can employ various methods commonly used for encoding audio signals, which are one-dimensional signals similar to tactile signals. Examples include MPEG-1 Audio Layer-III (MP3), Advanced Audio Coding (AAC), and lossless audio compression coding (FLAC) as a lossless coding technique. Furthermore, considering the resource constraints of arithmetic operations, adaptive differential pulse code modulation (ADPCM) can be used.

[0189] As mentioned above, if possible, the encoding technique for low-frequency signals employs a waveform-preserving encoding technique.

[0190] The tactile intensity calculation unit 24d receives the high-frequency signal obtained by the frequency band division unit 24b as input and calculates the tactile intensity based on the high-frequency signal.

[0191] Furthermore, the intensity of touch will be described later.

[0192] Multiplexer 24e receives encoded low-frequency signal data (hereinafter referred to as "encoded low-frequency data") obtained by low-frequency signal encoding unit 24c and tactile intensity calculated by tactile intensity calculation unit 24d as input. Multiplexer 24e follows the method described later. Figure 11 The encoded data format encodes various types of information, such as encoded low-frequency data or tactile intensity, into a bitstream. This encoding results in the acquisition of encoded tactile data Dc.

[0193] The signal output unit 24f outputs the encoded tactile data Dc obtained in the multiplexer 24e.

[0194] Detailed description of frequency band segmentation unit 24b.

[0195] Figure 8 It is a diagram showing the tactile sensitivity characteristics of each sensory receptor in the human body.

[0196] like Figure 8 As shown, four types of sensory receptors are known to allow the human body to feel touch. Among them, Meissner corpuscles are called velocity-detecting receptors and are highly sensitive to low frequency changes. Therefore, frequency components from a few hertz (Hz) to approximately 100 Hz (where Meissner corpuscles are most likely to be excited) are encoded to preserve their waveform as much as possible. Consequently, the frequency band segmentation unit 24b divides the tactile signal frequency band into a low-frequency band below 100 Hz and a high-frequency band above 100 Hz.

[0197] Furthermore, the breakpoint used between the low-frequency and high-frequency bands can be divided into any frequency range depending on the circumstances. Details will be described later.

[0198] Now, let's describe the tactile intensity calculation unit 24d.

[0199] Regarding human touch, from Figure 6 The vibration detection thresholds shown in the figure demonstrate that the intensity perceived by the human body varies with frequency. This intensity is quantified and modeled as a tactile intensity model, which has been reported in this field.

[0200] In this specification, tactile intensity (tactile strength) refers to the intensity of the tactile sensation felt by the human body when a tactile stimulus of a given amplitude is applied.

[0201] Figure 9 This is a diagram illustrating the intensity of tactile sensation, where Figure 9 (A) is a graph of T(f), and Figure 9 (B) is a graph of a(f). In this graph, T(f) represents the vibration detection threshold T relative to the frequency f (e.g., the threshold T is a reference). Figure 6The vibration detection threshold is described, and a(f) represents the coefficient a relative to the frequency f.

[0202] Based on T(f), a(f), and amplitude A(f) at frequency f of the tactile signal, the tactile intensity I(f) at frequency f is given by the following formula (2):

[0203] [Mathematical Expression 1]

[0204]

[0205] A(f) is the amplitude of the signal at frequency f.

[0206] When the tactile intensity I is the same signal, even if their frequencies are different, the vibration intensity is felt equally. Furthermore, in Figure 8 Among the tactile receivers shown, the Pacinian corpuscles are primarily known to be less sensitive to changes in vibration frequency than to changes in vibration intensity. Therefore, by following changes in tactile intensity I, perception at high frequencies equivalent to 100 Hz or higher can be obtained, where the Pacinian corpuscles are most likely to be stimulated.

[0207] Therefore, for high-frequency signals of 100Hz or higher, the tactile intensity I(f) value for each frequency is recorded instead of the waveform itself.

[0208] Furthermore, the lower cutoff frequency for calculating its tactile intensity can be determined based on the aforementioned frequency band segmentation.

[0209] When calculating and recording the tactile intensity I of a frame, the tactile intensity I of the high-frequency signal in the tactile signal of a frame is given as a finite integral of frequency f by the following formula (3):

[0210] [Mathematical Expression 2]

[0211]

[0212] Considering the above-mentioned processing as digital signal processing, firstly, the input signal undergoes a discrete Fourier transform to obtain the spectrum. Then, the tactile intensity I(Fbin) for each frequency window Fbin is given by formula (4).

[0213] Where AS(Fbin) is the amplitude spectrum value relative to Fbin, and fFbin is the center frequency of Fbin, as follows:

[0214] [Mathematical Expression 3]

[0215]

[0216] Then, the tactile intensity I of the high-frequency signal is obtained by summing I(Fbin) from the minimum value FbinMin of the frequency bin to the maximum value FbinMax of the frequency bin within the tactile intensity calculation range using the following formula (5):

[0217] [Mathematical Expression 4]

[0218]

[0219] Furthermore, in the above examples, representative tactile intensities were obtained for all frequencies from 100Hz to 2kHz. However, if the dominant frequency important for tactile reproduction in the input signal is known, the tactile intensity I can be obtained only for the component of this dominant frequency.

[0220] As used in this article, the term "dominant frequency" refers to the frequency from which the dominant spectrum is obtained. The dominant spectrum is the spectrum that at least does not include the spectrum as noise.

[0221] Provide a description of an example of obtaining the dominant frequency using frequency analysis.

[0222] First, a discrete Fourier transform is performed on the input tactile signal to obtain the spectrum. Then, the main frequency is determined by examining the peak values ​​of the amplitude spectrum above 100Hz.

[0223] In one instance, if the peak appears at the frequency components of 500Hz, 800Hz, and 1000Hz, such as Figure 10 As shown, by performing a Fourier transform, these frequency components of 500Hz, 800Hz, and 1000Hz are designated as the dominant frequencies.

[0224] In this respect, peaks can be detected, for example, as a determination of whether the amplitude spectrum is equal to or higher than a predetermined threshold. Furthermore, the methods for detecting peaks are not limited to this example, and various known techniques can be employed, for example.

[0225] If a dominant frequency is specified, the tactile intensity I of the high-frequency signal is calculated based on the amplitude A of the dominant frequency. In one example, in... Figure 10 Given the specified dominant frequency, the tactile intensity I is given by the following formula (6):

[0226] [Mathematical Expression 5]

[0227]

[0228] On the other hand, if the dominant frequency is not specified, the tactile intensity I of the high-frequency signal is given by formula (5).

[0229] Figure 11 This is a diagram illustrating an example of the data format of the encoded tactile data Dc. Specifically, Figure 11The data format of a frame of encoded tactile data Dc is shown.

[0230] In this format, the synchronization word is an identifier used to indicate the start of a frame, and a pattern is stored for the synchronization word that is not included in other data as much as possible.

[0231] For sampling frequency ID, record the pattern ID of the sampling frequency of the tactile signal.

[0232] For the number of channels, record the total number of tactile signals.

[0233] For frame size, the size of the haptic data Dc encoded for a frame is recorded in bytes.

[0234] For the first channel data, stored is coded data related to the tactile signal of the first channel. Specifically, stored is the segmentation frequency, coded low-frequency data, and tactile intensity information.

[0235] The segmented frequency is the frequency used by the band segmentation unit 24b, and for example, a value indicating the frequency [Hz] is stored for segmenting the frequency.

[0236] For encoded low-frequency data, the encoded data of the low-frequency signal encoded by the low-frequency signal encoding unit 24c is stored.

[0237] Tactile intensity information in Figure 12 It is shown in detail in the text.

[0238] As shown in the figure, the sign has a storage area for storing tactile intensity information.

[0239] In one example, when a dominant frequency is specified based on the peak detection described above, i.e., when the frequency at which the tactile intensity is calculated is limited, the flag is set to "1". The area following the flag stores, in pairs, an identifier for the frequency at which the tactile intensity is calculated (indicated in the figure as an identifier for frequency A and an identifier for frequency B) and information about the tactile intensity I for that frequency.

[0240] On the other hand, if no dominant frequency is specified and the specific frequency at which the tactile intensity is calculated is not restricted, the flag is set to "0". In the subsequent region, the tactile intensity I for each frequency (each frequency window) in the high-frequency region (in this example, from the segmented frequency to the Nyquist frequency: 100Hz to 2kHz) is stored. Furthermore, for the tactile intensity I with the flag = 0, the value for each frequency can be stored, or the total value for each frequency calculated in the above formula (5) can be stored.

[0241] For the second channel data, the encoded data associated with the tactile signals of the second channel is stored in the same format as the first channel data.

[0242] Other channels (if any) are added, and information is stored similarly.

[0243] refer to Figure 13 The flowchart in the document describes an exemplary processing procedure for implementing the encoding technique according to the above embodiments.

[0244] Furthermore, this process illustrates an example of the encoding unit 24 performing processing for implementing the encoding technique according to the embodiment as software processing, but all or part of the processing described below can be implemented in hardware.

[0245] exist Figure 13 The diagram illustrates the process of generating encoded haptic data Dc for a single frame, and this process is repeated for each frame.

[0246] In step S101, firstly, the encoding unit 24 receives a signal as input obtained by dividing the tactile signal of a channel to be encoded into the number of samples of a frame.

[0247] In step S102, following step S101, the encoding unit 24 performs band segmentation processing on the input signal using the segmentation frequency to obtain low-frequency and high-frequency signals.

[0248] In step S103 following step S102, encoding unit 24 uses a predetermined encoding technique (such as MP3, AAC, FLAC or ADPCM as described above) to encode the low-frequency signal obtained through the frequency band segmentation process in step 102 to generate encoded low-frequency data.

[0249] In step 104, following step S103, encoding unit 24 determines whether the tactile intensity range is the entire high-frequency band. In other words, it determines whether the tactile intensity has been calculated for the entire frequency band of the high-frequency signal, specifically, whether the dominant frequency has been specified as described above. This determination is based on the results obtained by analyzing the high-frequency signal obtained through the frequency band segmentation process in step 102.

[0250] In step 104, for example, if no dominant frequency is specified and the tactile intensity range is a positive result (yes) covering the entire high-frequency band, the process proceeds to step 105. In this step, the encoding unit 24 calculates the tactile intensity of the entire high-frequency signal band (see formula (5) expressed above), and then the process proceeds to step S107.

[0251] On the other hand, in step 104, for example, if a dominant frequency is specified and a negative result (No) is obtained that the tactile intensity range is not the entire high-frequency band, the process proceeds to step 106. In this step, the encoding unit 24 calculates the tactile intensity of the dominant frequency (see formula (6) expressed above), and then the process proceeds to step S107. Meanwhile, if a dominant frequency is specified, the encoding unit 24 stores the identifier of the specified dominant frequency.

[0252] In step S107, the encoding unit 24 stores the flag of the current channel (the channel currently being processed). Specifically, if the tactile intensity range obtained in step S104 is a positive result of the entire high-frequency band, the encoding unit 24 stores "0" as the flag of the current channel; otherwise, the encoding unit 24 stores "1" as the flag of the current channel.

[0253] In step S108 following step S107, encoding unit 24 determines whether the processing of all channels is complete, that is, whether the processing of all channels of the tactile signal from steps S101 to S107 is complete.

[0254] If processing for all channels is not completed, the processing in encoding unit 24 returns to step S101. This return allows similar processing of a frame to be performed on the next channel.

[0255] On the other hand, if the processing of all channels is completed, the encoding unit 24 performs bitstream generation processing in step S109, and then ends the process. Figure 13 The series of processing steps shown in the figure.

[0256] In the bit stream generation process of step S109, according to the reference Figure 11 and Figure 12 The described encoded data format is multiplexed to produce a bitstream of tactile data Dc for encoding.

[0257] Furthermore, although the above example of specifying the dominant frequency through peak detection has been given, the dominant frequency can also be specified by using a deep neural network (DNN).

[0258] Reference Figure 14 Describe specific examples.

[0259] Figure 14 This illustrates a DNN model that performs supervised learning using a pre-existing training dataset. For the training dataset, the training input dataset is used as the training input signal.

[0260] The learning input dataset is the amplitude spectrum of the tactile signal. In the DNN model, the input layers I1 to In are the center frequencies of the corresponding frequency windows, and the input is the amplitude spectrum value for each frequency window.

[0261] In the DNN model, the output layers A1 to An are combinations of the dominant frequencies used for tactile reproduction, and the training dataset is a supervised dataset corresponding to each data point in the learning input dataset and is a pre-labeled combination of the dominant frequencies used for tactile reproduction.

[0262] When learning a DNN model, the weight coefficients of each edge of the DNN model are updated by the probability (likelihood) of the output layer A1 to An when learning from the input data and by backpropagation of the error from learning the training data.

[0263] When using a DNN model learned in the manner described above to specify the dominant frequency from an actual tactile signal, the input tactile signal (high-frequency signal) is subjected to a Discrete Fourier Transform and subsequently converted into a spectrum. The amplitude spectrum is then input into the learned DNN model, and the outputs (frequency combination patterns) with the highest probability among the output layers A1 to An are used.

[0264] [6-3. Decoding Technology]

[0265] Figure 15 This is a functional block diagram illustrating the function of the decoding unit 34.

[0266] As shown in the figure, the decoding unit 34 includes an encoded data input unit 34a, a demultiplexer 34b, a low-frequency signal decoding unit 34c, a tactile intensity acquisition unit 34d, a high-frequency signal generation unit 34e, a frequency band synthesis unit 34f, and a signal output unit 34g.

[0267] The encoded data input unit 34a receives encoded tactile data Dc of one frame as input. Furthermore, the start of a frame can be detected from the aforementioned synchronization word.

[0268] Demultiplexer 34b according to in Figure 11 and Figure 12 The encoded data format shown is demultiplexed to obtain various types of information stored in the encoded tactile data Dc.

[0269] The low-frequency signal decoding unit 34c decodes the encoded low-frequency signal data obtained by the demultiplexer 34b to obtain the low-frequency signal.

[0270] The tactile intensity acquisition unit 34d performs flag value determination processing based on the tactile intensity information acquired by the demultiplexer 34b, or performs tactile intensity acquisition processing according to the determined flag value.

[0271] The high-frequency signal generation unit 34e generates a high-frequency signal based on the tactile intensity acquired by the tactile intensity acquisition unit 34d. Furthermore, the method of generating the high-frequency signal will be described later.

[0272] The frequency band synthesis unit 34f synthesizes the low-frequency signal obtained by the low-frequency signal decoding unit 34c and the high-frequency signal obtained by the high-frequency signal generation unit 34e.

[0273] The signal output unit 34g outputs the tactile signal obtained by synthesis in the frequency band synthesis unit 34f.

[0274] Explain the processing of the high-frequency signal generation unit 34e.

[0275] The processing of the high-frequency signal generation unit 34e varies depending on the value of the flag.

[0276] With flag = 0, the tactile intensity of the entire high-frequency signal band is obtained, and a periodic signal with any selectable frequency fx in the high-frequency band is generated as a high-frequency signal based on the tactile intensity of the entire band.

[0277] In this specification, a periodic signal refers to a signal whose amplitude changes within a predetermined period (time signal), and examples of such signals include sine wave signals, etc.

[0278] When the flag = 0, the high-frequency signal generation unit 34e generates a high-frequency signal based on the tactile intensity I of the high-frequency signal (i.e., the total value of the tactile intensity calculated for each frequency in the high-frequency band).

[0279] Specifically, the amplitude A(fx) can be calculated from the obtained tactile intensity I and any selectable frequency fx based on formula (2), and thus the high-frequency signal S1(t) is given in this case by the following formula (7):

[0280] [Mathematical Expression 6]

[0281]

[0282] As described above, when the flag = 0, the high-frequency signal generation unit 34e converts the tactile intensity into a signal amplitude and generates a periodic signal as a high-frequency signal. The periodic signal has a converted signal amplitude and a signal frequency fx, which is a frequency in the high-frequency band.

[0283] In this example, when the flag = 0, the selectable frequency fx can be uniquely determined in advance in the decoding unit 34.

[0284] In one example, the frequency fx can be considered to be set to a frequency substantially consistent with the resonant frequency of the haptic presentation device 6. The term "substantially consistent" as used herein is a concept encompassing values ​​within a range considered to be identical. For example, it refers to a frequency range from the front-side frequency to the rear-side frequency that falls within the peak portion centered on the resonant frequency in the frequency response curve of the haptic presentation device 6.

[0285] Setting the frequency fx to be basically consistent with the resonant frequency of the haptic presentation device 6 can improve the driving efficiency of the haptic presentation device, thereby achieving energy saving of the haptic presentation device 6.

[0286] Furthermore, the frequency fx can be set based on the tactile sensitivity characteristics of the human body. Specifically, it can be envisioned that the frequency fx is set to a frequency at which the tactile sensitivity is equal to or higher than a predetermined value.

[0287] This allows the frequency fx to be set to a frequency at which the human body has high tactile sensitivity, thereby improving the efficiency of driving the haptic presentation device 6. Therefore, energy saving can be achieved in the haptic presentation device 6.

[0288] Furthermore, the frequency fx can be set based on the characteristics of human auditory sensitivity. Specifically, it can be envisioned that the frequency fx is set to a frequency where the auditory sensitivity is equal to or lower than a predetermined value.

[0289] This allows the frequency fx to be set to a frequency to which the human body has low auditory sensitivity, resulting in a reduction of noise generated by driving the haptic presentation device 6. This, in turn, enhances the user's immersion in the haptic content.

[0290] Explain the handling when the flag is 1.

[0291] When the flag = 1, the tactile intensity acquisition unit 34d obtains the dominant frequency and the tactile intensity of that dominant frequency.

[0292] In this case, the high-frequency signal generation unit 34e generates a high-frequency signal based on the tactile intensity of the dominant frequency.

[0293] Specifically, for example, when the main frequencies are frequency A and frequency B, a high-frequency signal S2 is given based on frequency A (fa), tactile intensity of frequency A (I(fa)), frequency B (fb), and tactile intensity of frequency B (I(fb)), as shown in the following formula (8):

[0294] [Mathematical Expression 7]

[0295]

[0296] Reference Figure 16 The flowchart in the document describes an exemplary processing procedure for implementing the decoding technique according to the above embodiments.

[0297] An example is now given, in which, similar to the encoding case, the decoding unit 34 uses software processing to perform the processing for implementing the decoding technique according to the embodiment. However, all or part of the processing described below can also be implemented in hardware.

[0298] Figure 16The process shown is to decode the bitstream for one frame (encoded haptic data Dc) to obtain the haptic signal, and to repeat the process frame by frame.

[0299] In step S201, firstly, the decoding unit 34 analyzes the encoded tactile data Dc for a frame according to the encoded data format. This analysis process allows obtaining the number of channels, frame size, and data for each channel by specifying the sampling frequency according to the sampling frequency ID.

[0300] In step 202, following step S201, decoding unit 34 decodes the encoded low-frequency data included in the target channel data to obtain a low-frequency signal.

[0301] In step 203, following step S202, the decoding unit 34 determines whether the value of the flag included in the target channel data is 0.

[0302] If the flag = 0, the processing in the decoding unit 34 proceeds to step S204 to obtain the tactile intensity of the entire high-frequency band included in the target channel data and then proceeds to step S206.

[0303] On the other hand, if the flag is not 0, the process proceeds to step 205, where the decoding unit 34 acquires the frequency identifier of the main frequency and the tactile intensity of the main frequency included in the target channel data. Then, the process proceeds to step S206.

[0304] In step S206, the decoding unit 34 performs high-frequency signal generation processing. In other words, when the flag = 0, a high-frequency signal is generated based on the tactile intensity of the entire high-frequency band obtained in step S204 using the formula (7) above. On the other hand, when the flag = 1, a high-frequency signal is generated based on the frequency identifier of the main frequency and the tactile intensity obtained in step S205 using the formula (8) above.

[0305] In step S207, following step S206, the decoding unit 34 performs a bandgap synthesis process, which combines the low-frequency signal obtained in step S202 with the high-frequency signal obtained in step S206.

[0306] Then, in step S208, the decoding unit 34 determines whether the processing of all channels is complete. If the processing of all channels is not complete, it returns to step S202. This return allows the processing from steps S202 to S207 to be performed on the next channel. On the other hand, if the processing of all channels is complete, then the decoding unit 34 terminates. Figure 16 The series of processing steps shown.

[0307] In this regard, the above provides a description of an example of setting the predetermined frequency to frequency fx when the flag = 0, but the frequency fx can also be set in response to this operation.

[0308] In one instance, the decoding unit 34 may respond to... Figure 4 The frequency fx is set by the operation input of the operation unit 38 shown in the figure.

[0309] Figure 17 An example of a graphical user interface (GUI) for setting the frequency fx is shown.

[0310] Figure 17 The example demonstrates a GUI that allows multiple frequencies, instead of a single frequency, to be set as frequencyfx.

[0311] In this case, as shown, a setting screen is displayed on the screen 39a of the display unit 39 to receive the setting operation of the frequency fx. The setting screen provides a frequency setting area for each channel of the tactile signal. In the frequency setting area of ​​each channel, a frequency display window 50 (in the figure, display windows 50a, 50b, and 50c for each corresponding frequency A, B, and C) and sliders 51 for multiple frequencies to be set (in the figure, sliders 51a, 51b, and 51c for each corresponding frequency A, B, and C) are provided. The sliders are used to specify the frequency value.

[0312] When the user changes the position of slider 51, the value in the frequency display window 50 corresponding to slider 51 changes. This arrangement allows the user to specify any selectable frequency by manipulating slider 50 while checking the value displayed in the frequency display window 50.

[0313] In this example, when multiple frequencies fx are set, for example, the decoding unit 34 generates a periodic signal for each of these frequencies and generates a high-frequency signal by synthesizing the periodic signal.

[0314] In this case, the tactile intensity of each frequency in the high-frequency band is preset to be stored as tactile intensity information in the encoded tactile data Dc when the flag = 0. Then, the decoding unit 34 performs a calculation similar to the above formula (8) based on the respective frequencies fx set in the high-frequency signal generation process and each tactile intensity at those frequencies fx to generate a high-frequency signal.

[0315] Furthermore, the above example is based on setting the frequency fx of the periodic signal through the operation input of the operation unit 38 provided in the decoder 3. However, the frequency fx can also be set based on the operation input of the operation unit provided in a device other than the decoder 3. In one example, a method is conceivable in which an operation unit is installed in the playback device 4 and the frequency of the periodic signal is set in response to the operation input to the playback device 4.

[0316] [6-4. Control of frequency division based on communication stability]

[0317] In this regard, the above provides a description of an example of fixing the split frequency between the low-frequency band and the high-frequency band in the frequency band splitting process during encoding, but the split frequency can also be variable.

[0318] It is conceivable that the segmentation frequency increases when the intention is to reproduce low-frequency signals with higher quality, and decreases when the intention is to reduce the amount of encoded tactile data Dc. In one instance, in the case of deteriorating communication stability, such as deterioration of radio wave conditions during communication, it is conceivable to improve communication stability by dynamically changing the segmentation frequency as QoS by reducing signal quality.

[0319] In this regard, communication stability refers to metrics related to the stability of communication. Examples may include communication rate (communication bit rate), latency values ​​such as ping values ​​(round-trip time of communication when a communication partner returns a response packet), radio wave strength in wireless communication, and packet loss rate in packet communication.

[0320] refer to Figure 18 Describe an example of changing the segmentation frequency based on communication stability.

[0321] exist Figure 18 In this context, the transmission buffer refers to the buffer used for transmission by the communication unit 27 in the encoder 2. When the encoded tactile data Dc obtained by the encoding unit 24 is sent to an external device, the encoded tactile data Dc is sequentially buffered in the transmission buffer on a per-data-packet basis.

[0322] In this example, communication stability is determined based on the number of empty time slots in the transmission buffer. Specifically, if a large amount of data packets accumulates in the transmission buffer, the communication state deteriorates, and multiple data retransmission requests are sent from the receiving device side (the receiving side of the encoded haptic data Dc, such as playback device 4). Therefore, it can be determined that the transmission is not seamless. In other words, low communication stability (deterioration) can be determined.

[0323] Figure 18 This indicates that communication stability is normal ( Figure 18 (A) , good ( Figure 18 (B) and difference ( Figure 18 In case (C), each corresponding segmentation frequency. The low-frequency band determined by the segmentation frequency is the band where the waveform is encoded almost exactly as is, so it can be called the quality-oriented coding range. Meanwhile, the high-frequency band is the band where only tactile intensity is used instead of the waveform itself, so it can be called the efficiency-oriented coding range.

[0324] In other words, when communication stability is good or satisfactory, data interruptions are unlikely to occur even at high bit rates. Therefore, the segmentation frequency is set high (e.g., 500 Hz) to widen the quality-focused coding range (low frequency band) as much as possible and improve quality. Conversely, when communication stability is poor or deteriorated, the focus is on preventing data interruptions, and the segmentation frequency is set low (e.g., 100 Hz) to widen the efficiency-focused coding range (high frequency band) and reduce the bit rate.

[0325] As described above, when the segmentation frequency is set to be variable based on communication stability, the coding unit 24 in Figure 13 In the frequency band division process shown in step S102, communication stability determination process (e.g., determination of three states: good, normal, and poor) and frequency division setting process are performed based on the result obtained by determining the number of idle time slots in the transmission buffer in the communication unit 27.

[0326] <7. Overview of Implementation Methods>

[0327] As described above, the decoder (decoder 3) in this embodiment includes: a first decoding unit (low-frequency signal decoding unit 34c), configured to decode a first frequency band signal, which is a signal of a first frequency band in the tactile signal, for encoded data (encoded tactile data Dc) obtained by encoding the tactile signal, wherein the encoded data includes data obtained by encoding the first frequency band signal and tactile intensity information indicating tactile intensity (tactile intensity) of a second frequency band in the tactile signal that is different from the first frequency band; a second decoding unit (high-frequency signal generation unit 34e), configured to decode a second frequency band signal, which is a signal of a second frequency band in the tactile signal, based on the tactile intensity information in the encoded data; and a synthesis unit (frequency band synthesis unit 34f), for synthesizing the first frequency band signal decoded by the first decoding unit and the second frequency band signal decoded by the second decoding unit.

[0328] For the encoded data generated by the encoder according to the embodiment, this configuration enables the decoding of tactile signals related to the second frequency band, while reducing or preventing the degradation of tactile reproducibility.

[0329] Therefore, the amount of tactile signal data can be reduced, while reducing or preventing tactile reproducibility degradation.

[0330] Furthermore, in the decoder according to the embodiment, the first frequency band is set to a frequency band within a frequency range lower than the second frequency band.

[0331] For tactile signals at a predetermined frequency or in a larger frequency band, the human body can perceive changes in tactile intensity relatively easily, but it is relatively difficult to perceive changes in frequency.

[0332] Therefore, the above configuration allows the use of this tactile sensitivity characteristic of the human body to achieve information compression of tactile signals, thereby reducing or preventing the degradation of tactile reproducibility while reducing the amount of tactile signal data.

[0333] Furthermore, in the decoder as an implementation, tactile intensity information is calculated for each frequency in the second frequency band, and the second decoding unit decodes the second frequency band signal based on the total value of the tactile intensity indicated by the tactile intensity information calculated for each frequency (see [link]). Figure 16 Steps S204 to S206 in the process.

[0334] This configuration allows for the reproduction of the appropriate tactile intensity as the tactile intensity of the second frequency band based on the tactile intensity of each frequency in the second frequency band signal before encoding.

[0335] Therefore, it can reduce or prevent the deterioration of tactile reproducibility.

[0336] Furthermore, in the decoder as an implementation, tactile intensity information is calculated for one or more main frequencies in the second frequency band, and the second decoding unit decodes the second frequency band signal based on the tactile intensity indicated by the tactile intensity information calculated from the main frequencies (see [link]). Figure 16 Steps S205 to S206 in the process.

[0337] This configuration allows for the reproduction of the appropriate tactile intensity as the tactile intensity of the second frequency band based on the tactile intensity of the dominant frequency in the second frequency band signal before encoding.

[0338] Therefore, it can reduce or prevent the deterioration of tactile reproducibility.

[0339] Furthermore, in the decoder of the embodiment, the second decoding unit converts the tactile intensity indicated by the tactile intensity information into a signal amplitude and generates a periodic signal with the converted signal amplitude as the decoding signal of the second frequency band signal. The signal frequency of the periodic signal is set to the frequency in the second frequency band (see formula (7)).

[0340] This configuration enables the decoding of second-band signals for encoded data generated by the encoder according to this technology, thereby reducing or preventing degradation of tactile reproducibility.

[0341] Therefore, the amount of tactile signal data can be reduced, while reducing or preventing tactile reproducibility degradation.

[0342] Furthermore, in the decoder used as an example, the periodic signal has a frequency that substantially matches the resonant frequency of the haptic presentation device.

[0343] This configuration enables improved efficiency in driving haptic display devices.

[0344] Therefore, energy saving can be achieved in haptic presentation devices.

[0345] Furthermore, in the decoder used as an implementation, the periodic signal has a frequency set based on the tactile sensitivity characteristics of the human body.

[0346] This configuration allows the frequency of the periodic signal to be set to a frequency that the human body is highly sensitive to touch, thereby improving the efficiency of driving the haptic presentation device.

[0347] Therefore, energy saving can be achieved in haptic presentation devices.

[0348] Furthermore, in the decoder used as an implementation, the periodic signal has a frequency set based on the auditory sensitivity characteristics of the human body.

[0349] This configuration allows the frequency of the periodic signal to be set to a frequency that the human body has low auditory sensitivity to, thereby reducing noise generated when driving the haptic presentation device.

[0350] This enhances the user's immersion in tactile content.

[0351] Furthermore, in the decoder implemented as an example, the second decoding unit sets the frequency of the periodic signal based on the operation (see...). Figure 17 ).

[0352] This configuration allows users to optionally select the frequency of the periodic signal.

[0353] Furthermore, the decoding method in this embodiment includes: a first decoding step, decoding a first frequency band signal that is a signal of a first frequency band in the tactile signal by encoding encoded data obtained by encoding the tactile signal, the encoded data including data obtained by encoding the first frequency band signal and tactile intensity information, the tactile intensity information indicating the tactile intensity of a second frequency band in the tactile signal that is different from the first frequency band; a second decoding step, decoding the second frequency band signal that is a signal of a second frequency band in the tactile signal based on the tactile intensity information in the encoded data; and a synthesis step, synthesizing the first frequency band signal decoded in the first decoding step and the second frequency band signal decoded in the second decoding step.

[0354] Even so, this decoding method can achieve similar operations and effects as those obtained by the decoder according to the above-described embodiments.

[0355] Furthermore, the encoder in this embodiment includes: a frequency band segmentation unit (frequency band segmentation unit 24b) configured to segment the tactile signal into a first frequency band signal as a signal of a first frequency band and a second frequency band signal as a signal of a second frequency band different from the first frequency band; a first encoding unit (low-frequency signal encoding unit 24c) configured to encode the first frequency band signal; a tactile intensity calculation unit (tactile intensity calculation unit 24d) configured to calculate tactile intensity information indicating the tactile intensity of the second frequency band based on the second frequency band signal; and an encoded data generation unit (multiplexer 24e) configured to generate encoded data (encoded tactile data Dc) including tactile intensity information and data obtained by encoding the first frequency band signal.

[0356] This configuration allows the use of human sensitivity characteristics to improve the compression efficiency of tactile signal information. Sensitivity characteristics readily perceive both tactile intensity and frequency changes within a predetermined frequency band, but in another frequency band, tactile intensity changes are readily perceived while frequency changes are difficult to perceive.

[0357] Therefore, the amount of tactile signal data can be reduced, while reducing or preventing tactile reproducibility degradation.

[0358] Furthermore, in the decoder according to the embodiment, the first frequency band is set to a frequency band within a frequency range lower than the second frequency band.

[0359] For tactile signals at a predetermined frequency or in a larger frequency band, the human body can perceive changes in tactile intensity relatively easily, but it is relatively difficult to perceive changes in frequency.

[0360] Therefore, the above configuration allows the use of this tactile sensitivity characteristic of the human body to achieve information compression of tactile signals, thereby reducing the amount of data in the tactile signals and reducing or preventing the degradation of tactile reproducibility.

[0361] Furthermore, in the decoder implemented as an example, the first frequency band and the second frequency band have a variable split frequency between the first frequency band and the second frequency band.

[0362] Making the segmentation frequency variable allows for a balance between the effect of data reduction caused by encoding and the effect of reducing or preventing the decrease in tactile reproducibility.

[0363] Therefore, even when the effect of data reduction is intended to be more important than tactile reproducibility for some reason, or conversely when tactile reproducibility is intended to be more important than the effect of data reduction, the balance between the effect of data reduction and the effect of preventing the degradation of tactile reproducibility can be optimized according to various situations.

[0364] Furthermore, in the decoder implemented as an example, the band segmentation unit changes the segmentation frequency based on the stability of communication with external devices (see...). Figure 18 ).

[0365] This configuration allows for adjusting the segmentation frequency to enhance data reduction when communication with the external device to which the encoded data is to be transmitted is unstable; that is, by reducing the bit rate of the communication data. Conversely, during stable communication, the segmentation frequency can be adjusted to improve tactile reproducibility.

[0366] Therefore, a balance can be appropriately adjusted between the effect of reducing data and the effect of preventing degradation of tactile reproducibility, based on communication stability.

[0367] Furthermore, in the decoder implemented as an example, the band segmentation unit reduces the segmentation frequency when communication stability is low, compared to the case of high communication stability.

[0368] This configuration allows for altering the segmentation frequency to enhance data reduction, i.e., reducing the bit rate of communication data during unstable communication. Conversely, during stable communication, the segmentation frequency can be altered to improve haptic reproduction.

[0369] Therefore, a balance can be appropriately adjusted between reducing or preventing communication interruptions and reducing or preventing tactile reproducibility degradation.

[0370] Furthermore, in the decoder used as an implementation, the tactile intensity calculation unit calculates tactile intensity information for each frequency in the second frequency band (see [link]). Figure 13 Step S105 in the process.

[0371] This configuration allows for the reproduction of the appropriate tactile intensity as the tactile intensity of the second frequency band based on the tactile intensity of each frequency in the second frequency band signal before encoding.

[0372] Therefore, it can reduce or prevent the deterioration of tactile reproducibility.

[0373] Furthermore, in the decoder implemented as an example, the tactile intensity calculation unit calculates tactile intensity information only for one or more dominant frequencies in the second frequency band (see [link]). Figure 13 Step S106 in the process.

[0374] This configuration allows for the reproduction of the appropriate tactile intensity as the tactile intensity of the second frequency band based on the tactile intensity of the dominant frequency in the second frequency band signal before encoding.

[0375] Therefore, it can reduce or prevent the deterioration of tactile reproducibility.

[0376] Furthermore, the encoding method in this embodiment includes: a frequency band segmentation step, dividing the tactile signal into a first frequency band signal that is a signal of a first frequency band and a second frequency band signal that is a signal of a second frequency band different from the first frequency band; a first encoding step, encoding the first frequency band signal; a tactile intensity calculation step, calculating tactile intensity information indicating the tactile intensity of the second frequency band based on the second frequency band signal; and an encoded data generation step, used to generate encoded data including the tactile intensity information and the data obtained by encoding the first frequency band signal.

[0377] Even this encoding method according to the embodiments can achieve similar operations and effects as those obtained by the decoder according to the above embodiments.

[0378] Here, the functions of the aforementioned encoding unit (24) and decoding unit (34) can be implemented as software processing by a CPU or the like. Software processing is performed based on program execution.

[0379] The first procedure of the embodiment is to enable the information processing device to perform the following procedure: a first decoding function, which decodes a first frequency band signal that is a signal of a first frequency band in the tactile signal, by encoding data obtained by encoding the tactile signal, wherein the encoded data includes data obtained by encoding the first frequency band signal and tactile intensity information indicating the tactile intensity of a second frequency band in the tactile signal that is different from the first frequency band; a second decoding function, which decodes the second frequency band signal that is a signal of a second frequency band in the tactile signal based on the tactile intensity information in the encoded data; and a synthesis function, which synthesizes the first frequency band signal decoded by the first decoding function and the second frequency band signal decoded by the second decoding function.

[0380] Using such a first procedure makes it possible to implement the decoder according to the above embodiments.

[0381] Furthermore, as a second procedure in the embodiment, the information processing device implements the following procedure: a frequency band segmentation function that divides the tactile signal into a first frequency band signal that is a signal of a first frequency band and a second frequency band signal that is a signal of a second frequency band different from the first frequency band; a first encoding function that encodes the first frequency band signal; a tactile intensity calculation function that calculates tactile intensity information indicating the tactile intensity of the second frequency band based on the second frequency band signal; and an encoded data generation function that generates encoded data including the tactile intensity information and the data obtained by encoding the first frequency band signal.

[0382] Using such a second procedure makes it possible to implement the encoder according to the above embodiments.

[0383] The first or second program mentioned above can be pre-recorded in a recording medium such as ROM in a computer device or a microcomputer with a CPU.

[0384] Alternatively, such programs can be temporarily or permanently stored (recorded) on removable recording media, such as floppy disks, CD-ROMs, magneto-optical (MO) disks, DVDs, Blu-ray discs (registered trademark), hard disks, semiconductor memory, memory cards, etc. These removable recording media can be used as so-called packaged software.

[0385] In addition, the first or second program can be installed on a personal computer or the like from a removable recording medium, or downloaded from a download site via a network such as the Internet or a local area network (LAN).

[0386] Furthermore, the first or second program is suitable for a wide range of settings for the decoder or encoder according to the embodiment. For example, the program can be downloaded to a personal computer, portable information processing device, mobile phone, game console, audiovisual (AV) device, etc., enabling the personal computer or the like to be used as a decoder or encoder of this technology.

[0387] It should be noted that the effects described in this specification are merely illustrative and not limited, and other effects may exist.

[0388] <8. This technology>

[0389] In addition, this technology can also be configured as follows.

[0390] (1) A decoder, comprising:

[0391] The first decoding unit is configured to: decode the first frequency band signal, which is a signal of the first frequency band in the tactile signal, for the encoded data obtained by encoding the tactile signal, the encoded data including the data obtained by encoding the first frequency band signal and tactile intensity information, the tactile intensity information indicating the tactile intensity of a second frequency band in the tactile signal that is different from the first frequency band;

[0392] The second decoding unit is configured to decode a second-frequency band signal as a second-frequency band signal in the tactile signal based on tactile intensity information in the encoded data; and

[0393] The synthesis unit is configured to synthesize the first frequency band signal decoded by the first decoding unit and the second frequency band signal decoded by the second decoding unit.

[0394] (2) According to the decoder in (1), where,

[0395] The first frequency band is set to be a frequency band in a lower frequency range than the second frequency band.

[0396] (3) Based on the decoder of (1) or (2), where,

[0397] Tactile intensity information is calculated for each frequency in the second frequency band, and

[0398] The second decoding unit decodes the second frequency band signal based on the total value of the tactile intensity indicated by the tactile intensity information calculated for each frequency.

[0399] (4) A decoder based on either (1) or (2), wherein,

[0400] Calculate tactile intensity information for one or more dominant frequencies in the second frequency band, and

[0401] The second decoding unit decodes the second frequency band signal based on the tactile intensity indicated by the tactile intensity information calculated for the main frequency.

[0402] (5) The decoder based on any one of (1) to (3), where,

[0403] The second decoding unit converts the tactile intensity represented by the tactile intensity information into a signal amplitude; and generates a periodic signal with the converted signal amplitude as the decoding signal of the second frequency band signal, the signal frequency of the periodic signal being set to a frequency in the second frequency band.

[0404] (6) According to the decoder in (5), where,

[0405] The periodic signal has a frequency that essentially matches the resonant frequency of the haptic presentation device.

[0406] (7) According to the decoder of (5) or (6), where,

[0407] The periodic signal has a frequency set based on the human body's tactile sensitivity characteristics.

[0408] (8) A decoder based on any one of (5) to (7), wherein,

[0409] Periodic signals have frequencies set based on the characteristics of human auditory sensitivity.

[0410] (9) A decoder based on any one of (5) to (8), wherein,

[0411] The second decoding unit sets the frequency of the periodic signal based on the operation settings.

[0412] (10) A decoding method, comprising:

[0413] The first decoding step involves decoding the first frequency band signal, which is a signal of the first frequency band in the tactile signal, for the encoded data obtained by encoding the tactile signal. The encoded data includes data obtained by encoding the first frequency band signal and tactile intensity information, which indicates the tactile intensity of a second frequency band in the tactile signal that is different from the first frequency band.

[0414] The second decoding step involves decoding the second frequency band signal, which is the second frequency band signal in the tactile signal, based on the tactile intensity information in the encoded data; and

[0415] The synthesis step synthesizes the first frequency band signal decoded in the first decoding step and the second frequency band signal decoded in the second decoding step.

[0416] (11) A program that enables an information processing device to perform the following:

[0417] The first decoding function decodes the first frequency band signal, which is a signal of the first frequency band in the tactile signal, for the encoded data obtained by encoding the tactile signal. The encoded data includes data obtained by encoding the first frequency band signal and tactile intensity information, which indicates the tactile intensity of the second frequency band in the tactile signal that is different from the first frequency band.

[0418] The second decoding function decodes the second frequency band signal, which is the second frequency band signal in the tactile signal, based on the tactile intensity information in the encoded data; and

[0419] The synthesis function synthesizes a first frequency band signal decoded by the first decoding function and a second frequency band signal decoded by the second decoding function.

[0420] (12) An encoder, comprising:

[0421] The frequency band segmentation unit is configured to segment the tactile signal into a first frequency band signal as a first frequency band signal and a second frequency band signal as a second frequency band signal different from the first frequency band;

[0422] The first coding unit is used to encode the first frequency band signal;

[0423] The tactile intensity calculation unit is configured to calculate tactile intensity information representing the tactile intensity of the second frequency band based on the second frequency band signal; and

[0424] The encoded data generation unit is configured to generate encoded data that includes tactile intensity information and data obtained by encoding a first frequency band signal.

[0425] (13) According to the encoder in (12), where

[0426] The first frequency band is set to a frequency range lower than the second frequency band.

[0427] (14) The encoder according to (12) or (13), where

[0428] The first frequency band and the second frequency band have a segmentation frequency that can vary between the first frequency band and the second frequency band.

[0429] (15) According to the encoder in (14), where

[0430] The frequency band division unit changes the division frequency based on the stability of communication with external devices.

[0431] (16) According to the encoder in (15), where,

[0432] Compared to situations with high communication stability, when communication stability is low, the frequency band segmentation unit reduces the segmentation frequency.

[0433] (17) An encoder based on any one of (12) to (16), wherein,

[0434] The tactile intensity calculation unit calculates tactile intensity information for each frequency in the second frequency band.

[0435] (18) An encoder based on any one of (12) to (16), wherein,

[0436] The tactile intensity calculation unit calculates tactile intensity information only for one or more dominant frequencies in the second frequency band.

[0437] (19) An encoding method, comprising:

[0438] The frequency band segmentation step divides the tactile signal into a first frequency band signal, which is a signal of the first frequency band, and a second frequency band signal, which is a signal of the second frequency band different from the first frequency band.

[0439] The first encoding step is to encode the signal in the first frequency band;

[0440] The tactile intensity calculation steps involve calculating tactile intensity information representing the tactile intensity of the second frequency band based on the second frequency band signal; and

[0441] The encoded data generation step generates encoded data that includes tactile intensity information and data obtained by encoding a first frequency band signal.

[0442] (20) A program that enables an information processing device to perform the following:

[0443] The frequency band segmentation function divides the tactile signal into a first frequency band signal, which is a signal of the first frequency band, and a second frequency band signal, which is a signal of the second frequency band, which is different from the first frequency band.

[0444] The first encoding function encodes the signal in the first frequency band;

[0445] The tactile intensity calculation function calculates tactile intensity information representing the tactile intensity of the second frequency band based on the second frequency band signal; and

[0446] The encoded data generation function generates encoded data that includes tactile intensity information and data obtained by encoding the first frequency band signal.

[0447] List of reference numerals

[0448] 1. Tactile Reproduction System

[0449] 2 Encoders

[0450] 3 Decoders

[0451] 5. Tactile sensors

[0452] 6. Haptic Presentation Devices

[0453] DC-encoded tactile data

[0454] 24 coding units

[0455] 25 Control Unit

[0456] 26 storage units

[0457] 27 Communication Unit

[0458] 34 Decoding Units

[0459] 35 Control Unit

[0460] 38 operating units

[0461] 39 display units

[0462] 41 Control Unit

[0463] 42 Communication Units

[0464] 24a Signal Input Unit

[0465] 24b Bandwidth Segmentation Unit

[0466] 24c Low-frequency signal encoding unit

[0467] 24d tactile intensity calculation unit

[0468] 24e multiplexer

[0469] 34a Encoded Data Input Unit

[0470] 34b Demultiplexer

[0471] 34c Low-frequency signal decoding unit

[0472] 34d tactile intensity acquisition unit

[0473] 34e High-frequency signal generation unit

[0474] 34f bandgap synthesis unit

Claims

1. A decoding device, comprising: The first decoding unit is configured to decode, for encoded data obtained by encoding the tactile signal, a first frequency band signal that is a signal of a first frequency band in the tactile signal, the encoded data including data obtained by encoding the first frequency band signal and tactile intensity information, the tactile intensity information indicating the tactile intensity of a second frequency band in the tactile signal that is different from the first frequency band; The second decoding unit is configured to decode a second frequency band signal as a signal of the second frequency band in the tactile signal based on the tactile intensity information in the encoded data; and The synthesis unit is configured to synthesize the first frequency band signal decoded by the first decoding unit and the second frequency band signal decoded by the second decoding unit. The tactile intensity refers to the intensity of the tactile sensation felt by the human body when a tactile stimulus of a given amplitude is applied.

2. The decoding device according to claim 1, wherein, The first frequency band is set to a frequency band in a frequency range lower than that of the second frequency band.

3. The decoding device according to claim 1, wherein, The tactile intensity information is calculated for each frequency in the second frequency band, and The second decoding unit decodes the second frequency band signal based on the total value of the tactile intensity indicated by the tactile intensity information calculated for each frequency.

4. The decoding device according to claim 1, wherein, The tactile intensity information is calculated for one or more dominant frequencies in the second frequency band, and the second decoding unit decodes the second frequency band signal based on the tactile intensity indicated by the tactile intensity information calculated for the dominant frequencies.

5. The decoding device according to claim 1, wherein, The second decoding unit converts the tactile intensity indicated by the tactile intensity information into a signal amplitude; and generates a periodic signal with the converted signal amplitude as the decoding signal of the second frequency band signal, wherein the signal frequency of the periodic signal is set to a frequency in the second frequency band.

6. The decoding apparatus according to claim 5, wherein, The periodic signal has a frequency that matches the resonant frequency of the tactile presentation device.

7. The decoding apparatus according to claim 5, wherein, The periodic signal has a frequency set based on the tactile sensitivity characteristics of the human body.

8. The decoding apparatus according to claim 5, wherein, The periodic signal has a frequency set based on the characteristics of human auditory sensitivity.

9. The decoding apparatus according to claim 5, wherein, The second decoding unit sets the frequency of the periodic signal based on the operation.

10. A decoding method, comprising: The first decoding step involves decoding the first frequency band signal, which is a signal of the first frequency band in the tactile signal, for the encoded data obtained by encoding the tactile signal. The encoded data includes data obtained by encoding the first frequency band signal and tactile intensity information, the tactile intensity information indicating the tactile intensity of a second frequency band in the tactile signal that is different from the first frequency band. The second decoding step involves decoding the second frequency band signal, which is the second frequency band signal in the tactile signal, based on the tactile intensity information in the encoded data; and The synthesis step combines the first frequency band signal decoded in the first decoding step and the second frequency band signal decoded in the second decoding step. The tactile intensity refers to the intensity of the tactile sensation felt by the human body when a tactile stimulus of a given amplitude is applied.

11. A computer-readable storage medium storing a program that causes an information processing device to perform the following: The first decoding function decodes a first frequency band signal, which is a signal of a first frequency band in the tactile signal, for encoded data obtained by encoding the tactile signal. The encoded data includes data obtained by encoding the first frequency band signal and tactile intensity information, the tactile intensity information indicating the tactile intensity of a second frequency band in the tactile signal that is different from the first frequency band. The second decoding function decodes the second frequency band signal, which is the second frequency band signal in the tactile signal, based on the tactile intensity information in the encoded data; and The synthesis function synthesizes the first frequency band signal decoded by the first decoding function and the second frequency band signal decoded by the second decoding function. in, The tactile intensity indicates the intensity of the tactile sensation felt by the human body when a tactile stimulus of a given amplitude is applied.

12. An encoding device, comprising: The frequency band segmentation unit is configured to segment the tactile signal into a first frequency band signal as a first frequency band signal and a second frequency band signal as a second frequency band signal different from the first frequency band; The first encoding unit encodes the first frequency band signal; The tactile intensity calculation unit is configured to calculate tactile intensity information representing the tactile intensity of the second frequency band based on the second frequency band signal; as well as The encoded data generation unit is configured to generate encoded data including the tactile intensity information and data obtained by encoding the first frequency band signal. The tactile intensity refers to the intensity of the tactile sensation felt by the human body when a tactile stimulus of a given amplitude is applied.

13. The encoding device according to claim 12, wherein, The first frequency band is set to a frequency band in a frequency range lower than that of the second frequency band.

14. The encoding device according to claim 12, wherein, The first frequency band and the second frequency band have a segmentation frequency that can vary between the first frequency band and the second frequency band.

15. The encoding device according to claim 14, wherein, The frequency band segmentation unit changes the segmentation frequency based on the stability of communication with external devices.

16. The encoding device according to claim 15, wherein, Compared to a situation with high communication stability, in a situation with low communication stability, the frequency band segmentation unit reduces the segmentation frequency.

17. The encoding device according to claim 12, wherein, The tactile intensity calculation unit calculates the tactile intensity information for each frequency in the second frequency band.

18. The encoding device according to claim 12, wherein, The tactile intensity calculation unit calculates the tactile intensity information only for one or more dominant frequencies in the second frequency band.

19. An encoding method, comprising: The frequency band segmentation step divides the tactile signal into a first frequency band signal, which is a signal of the first frequency band, and a second frequency band signal, which is a signal of the second frequency band different from the first frequency band. The first encoding step is to encode the first frequency band signal; The tactile intensity calculation step involves calculating tactile intensity information indicating the tactile intensity of the second frequency band based on the second frequency band signal. as well as The encoded data generation step generates encoded data that includes the tactile intensity information and data obtained by encoding the first frequency band signal. The tactile intensity refers to the intensity of the tactile sensation felt by the human body when a tactile stimulus of a given amplitude is applied.

20. A computer-readable storage medium storing a program that causes an information processing device to perform the following: The frequency band segmentation function divides the tactile signal into a first frequency band signal, which is a signal of the first frequency band, and a second frequency band signal, which is a signal of the second frequency band, which is different from the first frequency band. The first encoding function encodes the signal in the first frequency band; The tactile intensity calculation function calculates tactile intensity information representing the tactile intensity of the second frequency band based on the second frequency band signal; as well as The encoded data generation function generates encoded data that includes the tactile intensity information and data obtained by encoding the first frequency band signal. The tactile intensity refers to the intensity of the tactile sensation felt by the human body when a tactile stimulus of a given amplitude is applied.

Images