Decoding device, decoding method, program, encoding device, encoding method

JP7593399B2Active Publication Date: 2024-12-03SONY GROUP CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2022517551
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2020-04-29
Filing Date
2021-03-19
Publication Date
2024-12-03
Estimated Expiration
2041-03-19

Smart Images

  • Figure 0007593399000003
    Figure 0007593399000003
  • Figure 0007593399000004
    Figure 0007593399000004
  • Figure 0007593399000005
    Figure 0007593399000005
Patent Text Reader

Abstract

The purpose of this invention is to reduce the amount of transmitted data while ensuring haptics reproducibility. A decoding device according to this technology comprises: a first decoding unit which decodes first encoded data obtained by encoding with a first bit rate, such encoding carried out for a first signal section that is a partial section of a contact signal section which is a signal section, in a tactile signal, indicating a contact state with a target object, and that is a signal section including a boundary between a contact state and a non-contact state of the target object; and a second decoding unit which decodes second encoded data obtained by encoding with a bit rate lower than the first bit rate, such decoding carried out for a second signal section that is a signal section obtained by excluding the first signal section from the contact signal section.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present technology relates to a decoding device and a decoding method for decoding an encoded haptic signal, and a program for the decoding device, and a coding device and a coding method for encoding a haptic signal, and a program for the coding device. [Background technology]

[0002] In recent years, technology has been developed that provides tactile stimulation to a user by vibrating a device worn by the user. Here, the tactile stimulation refers to a physical phenomenon that causes a user to feel a sense of touch through a vibration phenomenon or the like. In addition, the generation of the tactile stimulation is called tactile presentation.

[0003] Tactile presentation technology is used in devices in various fields. For example, in a terminal device equipped with a touch panel such as a smartphone, the touch panel vibrates in response to a touch operation by a user, and a tactile stimulus is given to the user's finger, making it possible to express the feeling of touching a button or the like displayed on the touch panel. In addition, in a music listening device such as a headphone, it is possible to emphasize the deep bass of the music being played by giving a tactile stimulus in accordance with the music being played. In addition, in a device that provides a computer game or VR (Virtual Reality), it is possible to improve the user's immersion in the content by vibrating the controller or the like and giving a tactile stimulus in response to the operation using the controller or the scene of the content.

[0004] In addition, a technology has been developed that provides a tactile stimulus to a user based on a tactile signal received from an external device. For example, Patent Document 1 below discloses a technology that provides a tactile stimulus to a user while changing the frequency and amplitude of vibration based on a received signal. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2016-202486 A Summary of the Invention [Problem to be solved by the invention]

[0006] Here, when presenting tactile sensations to multiple parts of the user's body, it is necessary to prepare a tactile signal for each channel corresponding to each part of the user. In this case, it is expected that the total amount of data required for the tactile signal will increase as the number of parts to which tactile stimulation is applied increases. Also, the total amount of data required for the tactile signal will increase even if the range of vibration frequencies contained in the tactile signal is widened. Therefore, delays in the transmission and reception of tactile signals may occur due to congestion of the network used to transmit and receive tactile signals.

[0007] The present technology has been made in consideration of the above circumstances, and aims to achieve both a reduction in the amount of transmitted data and ensuring tactile reproducibility. [Means for solving the problem]

[0008] The decoding device according to the present technology is a first signal section which is a part of a contact signal section which is a signal section indicating a contact state with an object in a haptic signal, the first signal section being a signal section including a boundary between a contact state with the object and a non-touch state. Regarding tactile signals, a first decoding unit that decodes first encoded data obtained by encoding at a first bit rate; and a second signal section that is a signal section excluding the first signal section from the contact signal section. Regarding tactile signals, and a second decoding unit configured to decode second encoded data obtained by performing encoding at a bit rate lower than the first bit rate. This allows tactile encoded data to be properly decoded, in which only important sections of the contact signal are encoded at a high bit rate, and other sections are encoded at a low bit rate, making it possible to achieve both a reduction in data volume and ensuring tactile reproducibility.

[0009] The decoding device according to the present technology described above may include a combining unit that combines first waveform data obtained by decoding the first encoded data and second waveform data obtained by decoding the second encoded data. As a result, the waveform data are integrated into one waveform data.

[0010] In the above-described decoding device according to the present technology, it is considered that the combining unit performs a cross-fade process on a combined portion of the first waveform data and the second waveform data. This allows the first waveform data and the second waveform data to be smoothly combined.

[0011] The decoding method according to the present technology further comprises decoding a first signal section which is a part of a contact signal section that is a signal section in a haptic signal indicating a contact state with an object and which includes a boundary between a contact state with the object and a non-contact state. Regarding tactile signals, a first coded data obtained by coding at a first bit rate, and a second signal section which is a signal section excluding the first signal section from the contact signal section; Regarding tactile signals, The decoding method is for decoding second coded data obtained by performing coding at a bit rate lower than the first bit rate. With such a decoding method, the same effect as that of the decoding device according to the present technology described above can be obtained.

[0012] Furthermore, a first program according to the present technology includes a first signal section that is a part of a contact signal section that is a signal section that indicates a contact state with an object in a haptic signal and that includes a boundary between a contact state with the object and a non-contact state. Regarding tactile signals, a function of decoding first encoded data obtained by encoding at a first bit rate; and a function of decoding a second signal section which is a signal section excluding the first signal section from the contact signal section. Regarding tactile signals, and a function of decoding second coded data obtained by performing coding at a bit rate lower than the first bit rate. The above-mentioned decoding device according to the present technology is realized by the first program according to the present technology.

[0013] The encoding device according to the present technology includes a determination unit that determines a first signal section, which is a part of a contact signal section that is a signal section in a haptic signal indicating a contact state with an object and that includes a boundary between a contact state with the object and a non-contact state, and a second signal section, which is a signal section of the contact signal section excluding the first signal section; About tactile signals a first encoding unit that performs encoding at a first bit rate; About tactile signals and a second encoding unit that performs encoding at a bit rate lower than the first bit rate. As a result, only important sections in the contact signal section are coded at a high bit rate, and other sections are coded at a low bit rate.

[0014] In the encoding device according to the present technology described above, it is considered that the determination unit determines the first signal section and the second signal section based on division information of the first signal section and the second signal section added to the haptic signal. This eliminates the need for waveform analysis processing in determining whether the signal is the first signal section or the second signal section.

[0015] In the encoding device according to the present technology described above, it is considered that the determination unit determines the first signal section and the second signal section based on a result of a waveform analysis of the haptic signal. This makes it unnecessary to add information for distinguishing the first signal section from the second signal section to the contact signal when encoding the first signal section and the second signal section separately.

[0016] In the encoding device according to the present technology described above, it is considered that the determination unit determines the first signal section and the second signal section based on an amplitude change rate of the haptic signal. This makes it possible to appropriately determine the first signal section that involves a steep change in the waveform of the haptic signal.

[0017] In the encoding device according to the present technology described above, it is considered that the second encoding unit performs encoding for the second signal section using an encoding method having a longer transform length than encoding for the first signal section. This reduces the amount of second encoded data in the second signal section.

[0018] In the encoding device according to the present technology described above, it is considered that the second encoding unit performs parametric encoding on the second signal section. This reduces the amount of second encoded data in the second signal section.

[0019] In the encoding device according to the present technology described above, it is conceivable that the encoding device includes a first buffer memory used to determine the first signal section and the second signal section, and a second buffer memory used to encode the second signal section depending on the result of the determination. This makes it possible for the second buffer processing unit to buffer the haptic signal in the second signal section, which has a longer transform length than the encoding length for the first signal section, that cannot be fully buffered in the first buffer processing unit.

[0020] Also, the encoding method according to the present technology includes determining a first signal section, which is a part of a contact signal section that is a signal section in a haptic signal indicating a contact state with an object and that includes a boundary between a contact state with the object and a non-contact state, and a second signal section, which is a signal section of the contact signal section excluding the first signal section, About tactile signals , encoding at a first bit rate, About tactile signals The encoding method performs encoding at a bit rate lower than the first bit rate. With such an encoding method, the same effect as that of the encoding device according to the present technology described above can be obtained.

[0021] Further, a second program according to the present technology includes a function for determining a first signal section, which is a part of a contact signal section that is a signal section in a haptic signal indicating a contact state with an object and includes a boundary between a contact state with the object and a non-contact state, and a second signal section, which is a signal section of the contact signal section excluding the first signal section; About tactile signals between , a function of performing encoding at a first bit rate, and About tactile signals and a function of performing encoding at a bit rate lower than the first bit rate. The above-mentioned encoding device according to the present technology is realized by such a second program according to the present technology. [Brief description of the drawings]

[0022] [Figure 1] 1 is a diagram illustrating an example of the configuration of a tactile reproduction system according to an embodiment of the present technology. [Diagram 2] FIG. 1 illustrates an example of the configuration of an encoding device according to an embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of the configuration of a playback device according to an embodiment. [Figure 4] FIG. 2 illustrates an example of a configuration of a decoding device according to an embodiment. [Diagram 5] 1A to 1C are diagrams illustrating an example of use of a tactile reproduction system according to an embodiment. [Figure 6] 1 is a diagram showing a vibration detection threshold curve serving as a measure of human tactile sensitivity according to an embodiment; [Figure 7] 5A and 5B are diagrams illustrating vibration waveforms of haptic signals according to an embodiment. [Figure 8] FIG. 2 is a diagram illustrating an example of a functional configuration of an encoding unit according to the embodiment. [Figure 9] 4 is a diagram illustrating an example of the configuration of a data format of haptic encoded data according to an embodiment. FIG. [Figure 10] 1 is a flowchart showing a process for implementing an encoding method according to an embodiment. [Figure 11]4 is a diagram illustrating an example of the configuration of a data format of haptic encoded data according to an embodiment. FIG. [Figure 12] 1 is a flowchart showing a process for implementing an encoding method according to an embodiment. [Figure 13] 11A and 11B are explanatory diagrams of cross-fade processing by a combining unit according to an embodiment. [Figure 14] 1 is a flowchart showing a process for implementing a decoding method according to an embodiment. [Figure 15] 11A and 11B are diagrams illustrating details of a cross-fade process performed by a combining unit according to an embodiment. [Figure 16] 11A and 11B are diagrams illustrating details of a cross-fade process performed by a combining unit according to an embodiment. [Figure 17] FIG. 1 is a diagram illustrating a configuration example of a first application example of an embodiment. [Figure 18] FIG. 11 is a diagram illustrating a configuration example of a second application example of the embodiment. [Figure 19] FIG. 13 is a diagram illustrating a configuration example of a third application example of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The embodiments of the present technology will be described with reference to the above drawings. In the description of the drawings, the same reference numerals may be used for configurations that have been described once, and the description may be omitted. In addition, the drawings are schematic, and the essential parts and their surrounding configurations necessary for describing the present technology are extracted and shown. Note that the relationship between the thickness and planar dimensions of each structure described in the drawings, ratios, etc. are merely examples, and various changes can be made according to the design, etc., as long as they do not deviate from the technical idea of ​​the present technology.

[0024] The embodiments will be described below in the following order. <1. Overview of the tactile reproduction system> <2. Encoding device configuration> <3. Configuration of playback device> <4. Configuration of the Decryption Device> <5. Example of use of tactile reproduction system> <6. Tactile reproduction method as an embodiment> [6-1. Issues related to the transmission of tactile signals] [6-2. Encoding method] [6-3. Decryption method] <7. First application example> <8. Second application example> <9. Third Application Example> <10. Summary> <11. This Technology>

[0025] In this specification, each term is defined as follows. Tactile stimulation: A physical phenomenon that allows a person to perceive touch, such as a vibration phenomenon. Tactile presentation: Producing tactile stimuli. Tactile Signal: A signal representing a pattern of tactile stimulation, e.g. a signal representing a vibration waveform. Haptic recipient: The person who receives the haptic presentation. Tactile characteristics: Characteristics related to human touch. They differ depending on the part of the body (hands, face, feet, etc.). Tactile sensitivity: The subjective sensitivity of a tactile stimulus. It varies depending on the receptors and parts of the human body. High tactile sensitivity means that it is easy to perceive tactile signals. Haptic encoded data: Data obtained by encoding a haptic signal. Sub-concepts include stream and frame. Haptic encoded data includes first encoded data and second encoded data, which will be described later.

[0026] <1. Overview of the tactile reproduction system> FIG. 1 shows an example of the configuration of a haptic reproduction system 1 including an encoding device 2 and a decoding device 3 as an embodiment of the present technology.

[0027] First, in this embodiment, the environment for realizing tactile reproduction is divided into a recording environment in which the tactile signal obtained by sensing the target tactile information (tactile stimulation) is encoded and the tactile encoded data Dc obtained by the encoding is recorded, and a reproduction environment in which tactile information is reproduced based on the tactile signal obtained by decoding the tactile encoded data Dc.

[0028] As shown in the figure, the tactile reproduction system 1 comprises, in a recording environment, a plurality of tactile sensors 5 and an encoding device 2 to which the tactile sensors 5 are connected, and, in a reproduction environment, a playback device 4 configured to be capable of acquiring tactile encoded data Dc, a decoding device 3 configured to be capable of wireless communication with the playback device 4, and a plurality of tactile presentation devices 6 connected to the decoding device 3.

[0029] The tactile sensor 5 is a sensor that senses tactile stimuli, and in this example, a vibration sensor such as a piezo pickup or an acceleration sensor is used. The tactile sensor outputs vibrations and movements as voltage changes when it is brought into contact with an object to be sensed, that is, a human body in this example. In this example, each tactile sensor 5 is connected to the encoding device 2 by wire, and each tactile sensor 5 is attached to a different part of the human body as the target to sense the tactile stimulus occurring at each part. The parts on which the tactile sensors 5 are attached are not limited to the human body, and they may be attached to tools or the like used in the tactile reproduction system 1. In this case, each tactile sensor 5 senses the tactile stimulus occurring at each part attached to the tool as the target object.

[0030] The encoding device 2 is configured with a computer device such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), and encodes the detection signals (tactile signals) from each tactile sensor 5 in accordance with a predetermined data format, and stores the tactile encoded data Dc obtained by the encoding in, for example, a storage device provided internally.

[0031] The playback device 4 is configured with a computer device such as a CPU and a DSP, and transmits the acquired haptic encoded data Dc to the decoding device 3. For example, the haptic encoded data Dc recorded in the recording environment is acquired by the playback device 4 via a required network such as the Internet. Alternatively, the haptic encoded data Dc can be recorded on a portable recording medium, and the playback device 4 can acquire the haptic encoded data Dc via the recording medium.

[0032] The decoding device 3 decodes the haptic encoded data Dc received from the playback device 4, and drives each haptic presentation device 6 based on the haptic signal obtained by the decoding.

[0033] The tactile presentation device 6 is a device that generates a tactile stimulus, and in this example, a vibration device such as a vibrator or actuator is used. In this example, each tactile presentation device 6 is attached to a different part of the human body of the touch receiver, and is adapted to reproduce the tactile stimulus sensed by the corresponding tactile sensor 5.

[0034] Here, in this example, each tactile presentation device 6 is connected to the decoding device 3 by wire, and the part surrounded by the dashed line in the figure, i.e., the decoding device 3 and each tactile presentation device 6, is the part that is worn by the haptic receiver.

[0035] The tactile reproduction system 1 can be configured such that the playback device 4 is provided with the functionality of the decoding device 3 and the playback device 4 is connected to each tactile presentation device 6 by wire, but in that case, there is a risk of causing annoyance to the haptic receiver wearing the tactile presentation device 6. This annoyance is expected to increase as the number of parts to which tactile stimulation is applied increases. The configuration of the tactile reproduction system 1 shown in FIG. 1 makes it possible to prevent such annoyance from being caused to the tactile receiver.

[0036] The tactile reproduction system 1 shown in Figure 1 is configured as a system that reproduces the tactile sensation of each part of the body perceived by a person wearing a tactile sensor 5 in a recipient, and is also capable of handling cases where the two are located remotely.

[0037] <2. Encoding device configuration> Fig. 2 is a diagram for explaining an example of the internal configuration of the encoding device 2. Note that Fig. 2 also shows the tactile sensors 5 shown in Fig. 1 together with the example of the internal configuration of the encoding device 2.

[0038] As shown in the figure, the encoding device 2 includes a plurality of amplifiers 21, a plurality of A / D (Analog / Digital) converters 22, a pre-processing unit 23, an encoding unit 24, a control unit 25, a storage unit 26, a communication unit 27, and a bus 28. The pre-processing unit 23, the encoding unit 24, the control unit 25, the storage unit 26, and the communication unit 27 are connected via the bus 28 and are capable of data communication with each other.

[0039] The detection signal of each tactile sensor 5 is input to a corresponding amplifier 21 and adjusted to an appropriate dynamic range, and then input to a corresponding A / D converter 22 and A / D converted (analog / digital converted).

[0040] Each A / D converted detection signal (i.e., the tactile signal for each part) is input to a pre-processing unit 23. In the pre-processing unit 23, various types of digital signal processing such as noise removal and calibration of the sensor characteristics of the tactile sensor 5 are performed. Each tactile signal that has been subjected to signal processing by the pre-processing unit 23 is input to an encoding unit 24.

[0041] The encoding unit 24 is composed of, for example, a DSP, and encodes each input haptic signal according to a predetermined data format to obtain the above-mentioned encoded haptic data Dc. The details of encoding the haptic signal according to this embodiment will be described later.

[0042] The control unit 25 is configured with a microcomputer having, for example, a CPU, a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and performs overall control of the encoding device 2 by executing processing in accordance with a program stored in the ROM. For example, the control unit 25 performs data communication with an external device via the communication unit 27.

[0043] The communication unit 27 is configured to be capable of performing data communication with an external device via a network such as the Internet, and the control unit 25 is capable of performing data communication with an external device connected to the network via the communication unit 27. In particular, the haptic encoded data Dc obtained by the encoding unit 24 can be transmitted to the external device via the communication unit 27.

[0044] The storage unit 26 is a general representation of storage devices such as a hard disk drive (HDD) or a solid state drive (SSD), and is used to store various types of data in the encoding device 2. For example, the storage unit 26 stores data necessary for control by the control unit 25. Also, under the control of the control unit 25, the haptic encoded data Dc obtained by the encoding unit 24 can be stored in the storage unit 26.

[0045] <3. Configuration of playback device> 3 is a diagram showing an example of the internal configuration of the playback device 4. As shown in the figure, the playback device 4 includes a control unit 41, a communication unit 42, a media drive 43, a storage unit 44, and a wireless communication unit 45, as well as a bus 46 that connects these units to each other so that they can communicate data with each other.

[0046] The control unit 41 is configured with, for example, a microcomputer having a CPU, ROM, RAM, etc., and performs overall control of the playback device 4.

[0047] The communication unit 42 is configured to be capable of performing data communication with an external device via a network such as the Internet.

[0048] The control unit 41 is capable of performing data communication with an external device connected to the network via the communication unit 42. In particular, it is possible to cause the communication unit 42 to receive haptic encoded data Dc from an external device such as a server device on the network.

[0049] The media drive 43 is configured to allow a portable recording medium to be detachably attached, and is configured as a reader / writer unit capable of writing and reading data to and from the attached recording medium. Examples of recording media compatible with the media drive 43 include memory cards (e.g., portable flash memory) and optical disk recording media. The media drive 43 allows the reading of haptic encoded data Dc recorded on the portable recording medium.

[0050] The storage unit 44 is a comprehensive representation of storage devices such as an HDD or SSD, and is used to store various types of data in the playback device 4. For example, the storage unit 44 stores data necessary for control by the control unit 41. Furthermore, based on the control of the control unit 41, the storage unit 44 can also store haptic encoded data Dc read by the media drive 43 and haptic encoded data Dc received from an external device by the communication unit 42.

[0051] The wireless communication unit 45 performs short-distance wireless communication using a predetermined communication method such as Bluetooth (registered trademark).

[0052] Here, as part of the overall control described above, the control unit 41 controls the communication unit 42 to receive the haptic encoded data Dc and the media drive 43 to read it. The control unit 41 also controls the wireless communication unit 45 to transmit the haptic encoded data Dc obtained via the communication unit 42 and the media drive 43 to the decoding device 3.

[0053] <4. Configuration of the Decryption Device> FIG. 4 is a diagram for explaining an example of the internal configuration of the decoding device 3, and shows each of the tactile presentation devices 6 together with the example of the internal configuration of the decoding device 3. As shown in FIG.

[0054] As shown in the figure, the decoding device 3 includes a plurality of amplifiers 31, a plurality of D / A (Digital / Analog) converters 32, a combining unit 33, a decoding unit 34, a control unit 35, a wireless communication unit 36, a storage unit 37, and a bus 38. The combining unit 33, the decoding unit 34, the control unit 35, the wireless communication unit 36, and the storage unit 37 are connected via the bus 38 and are capable of data communication with one another.

[0055] The control unit 35 is configured with, for example, a microcomputer having a CPU, ROM, RAM, etc., and performs overall control of the decoding device 3.

[0056] The wireless communication unit 36 ​​performs short-range wireless communication using a method such as Bluetooth that enables communication with the wireless communication unit 45 in the playback device 4. The haptic encoded data Dc transmitted from the playback device 4 is received by the wireless communication unit 36.

[0057] The storage unit 37 is a storage device similar to the storage unit 26 of the encoding device 2 and the storage unit 44 of the playback device 4, and is used to store various data used by the control unit 35 and the like.

[0058] The decoding unit 34 decodes the haptic encoded data Dc input via the wireless communication unit 36 ​​using a method described below to obtain a haptic signal for each body part. The haptic signals for each body part obtained by the decoding unit 34 are input to the combining unit 33.

[0059] The combining unit 33 performs signal processing such as calibration of the tactile presentation device 6 and predetermined filtering, as necessary, on the input tactile signals for each part. The details of the decoding of the haptic encoded data Dc according to this embodiment will be described later.

[0060] Each tactile signal that passes through the connection unit 33 is input to a corresponding D / A converter 32 and subjected to D / A conversion (digital / analog conversion), then adjusted to an appropriate dynamic range by a corresponding amplifier 31 and output to a corresponding tactile presentation device 6. As a result, each tactile presentation device 6 is driven based on the tactile signal, and the tactile stimulus that is the subject of sensing in the detection environment can be given to the touch recipient (that is, tactile information can be reproduced).

[0061] Although only haptic signals have been mentioned above, audio and video signals may be recorded together with the haptic signals, so that sounds and videos may be provided to the touch recipient along with the haptic information.

[0062] <5. Example of use of tactile reproduction system> Consider a case where content that provides tactile feedback in addition to video is played back. FIG. 5 is an explanatory diagram of an example of use of the tactile feedback system 1.

[0063] In Figure 5, when producing content, in addition to the video, tactile signals (tactile signal 62 of the torso, tactile signal 63 of the fingers, and tactile signal 64 of the feet) recorded by tactile sensors 5 (in the figure, tactile sensor 5b for the torso, tactile sensor 5h for the fingers, and tactile sensor 5f for the feet) attached to a tactile recorder 61 are coded by encoding device 2 to produce tactile coded data Dc, which are recorded in synchronization with the video and stored in the memory unit 44 of the playback device 4 as content 65.

[0064] During playback, the content 65 is transmitted from the storage unit 44 to the decoding device 3 via the wireless communication unit 45, and the content 65 received by the wireless communication unit 36 ​​is decoded by the decoding unit 34. This makes it possible to present tactile signals (a tactile signal 66 for the trunk, a tactile signal 67 for the fingers, and a tactile signal 68 for the feet) corresponding to the tactile presentation devices 6 worn by the touch receiver 69 (in the figure, a tactile presentation device 6b for the trunk, a tactile presentation device 6h for the fingers, and a tactile presentation device 6f for the feet) through the tactile presentation devices 6.

[0065] Examples of scenes in which haptics are actually presented within the video include scenes in which characters punch (or are punched), shoot (or are shot), are hit by an explosion, or feel the ground shaking. In the figure, waveform examples 50 of tactile signals for the torso, fingers, and feet are shown as examples of tactile signals in actual content 65. Waveform example 50 shows tactile signals obtained by tactile sensors 5 (tactile sensor 5b, tactile sensor 5h, tactile sensor 5f) attached to three parts of the torso, fingers, and feet of tactile recorder 61.

[0066] To explain in chronological order, first, in the scene where haptic recorder 61 shoots someone with a gun, the recoil of the bullet causes vibrations in his fingers, then in the scene where the other person also shoots at the haptic recorder's torso, vibrations occur in the torso due to the impact of the gunshot, and then in the scene where an earthquake occurs, the vibrations of the ground gradually propagate to the feet, torso, and fingers. For example, by playing such content, haptic receiver 69 can feel high-quality reality through the reproduction of haptics through vibrations in addition to video and audio.

[0067] <6. Tactile reproduction method as an embodiment> [6-1. Issues related to the transmission of tactile signals] A tactile reproduction method according to an embodiment will be described below. First, the tactile reproduction method according to the embodiment is a method that focuses on the tactile characteristics of humans. The vibration detection threshold curve shown in Figure 6 has been reported as a guide to human tactile sensitivity. In the figure, the horizontal axis represents frequency, and the vertical axis represents the magnitude of the tactile stimulus (vibration: here, displacement). The vibration detection threshold curve in Figure 6 is based on the experimental results in "Four channels mediate the mechanical aspects of touch SJ, Bolanowski 1988". The vibration detection threshold curve shown in the figure is an example of an experiment to determine whether or not humans can sense a vibration as a touch, that is, tactile sensitivity. Humans cannot sense vibrations smaller than this curve as touch.

[0068] As can be seen from Figure 6, the frequency at which humans have the highest tactile sensitivity is generally considered to be around 200 Hz. Therefore, devices and applications that generate vibrations are often designed to generate vibrations up to around 200 Hz.

[0069] On the other hand, although it is not shown in the results of Figure 6, it is commonly known that humans can sense vibrations with frequencies up to about 1 kHz as touch. Humans can sense vibrations that contain frequency components of about 1 kHz as different tactile sensations from vibrations that do not. For example, the vibrations generated when removing a cork from a bottle include vibrations with frequencies up to several kHz. If the vibrations are transmitted from a haptic device at frequencies up to several hundred Hz, the user cannot feel the vibrations with a sufficient sense of reality as the haptic sensation of removing the cork from the bottle. Therefore, in order to provide a user with a more realistic tactile sensation, it is necessary to present the tactile sensation using vibrations including frequencies up to about 1 kHz.

[0070] However, as the frequency range of a signal becomes wider, the amount of data in the signal increases, and this increases the possibility of delays in signal transmission and reception. In other words, if the quality of the tactile sensation is improved, a situation may arise in which the tactile sensation cannot be presented at the appropriate time.

[0071] The delay in signal transmission and reception and in tactile presentation will be explained using specific examples. First, the data volume of a haptic signal will be explained. When a haptic signal is transmitted between devices, it is first converted into digital data. The capacity of digital data is expressed as the number of bits required per unit time, i.e., bit rate B. Incidentally, haptic sensitivity depends not only on the frequency but also on the amplitude of vibration. For example, the experimental results in FIG. 6 above show that the amplitude of vibration felt by humans is about 50 dB (-20 dB to 30 dB) or more, and the frequency is about 1000 Hz. In the following, taking into account the distribution of haptic information actually felt by humans, the amplitude of vibration is considered to be about 70 dB.

[0072] When haptic signals are digitized using Linear Pulse Code Modulation (LPCM), the vibration amplitude that can be expressed by 1 bit is 6 dB. In other words, a vibration amplitude of 70 dB requires 12 bits. On the other hand, if the vibration frequency is 1000 Hz, the sampling frequency must be doubled to 2000 Hz, and the bit rate B0 can be calculated using the following formula (1). B0=12bit / sample×2000sample / sec=24kbit / sec...[Formula 1]

[0073] This value itself is extremely small compared to, for example, the bit rate of CD (Compact Disc), a typical format for audio signals, which is 700 kbps / ch. Therefore, it seems unlikely that incorporating this haptic signal as an additional component in some system would pose a major problem.

[0074] However, as mentioned above, it is known that the bandwidth of haptic signals that humans can sense extends up to several kHz. For example, if haptic signals are reproduced up to 2000 Hz, the bit rate will be 48 kbit / sec, twice that of [Equation 1].

[0075] Furthermore, unlike vision (two eyes) and hearing (two ears), touch is present everywhere on the surface of the human body. Just considering the fingertips on both hands, there are ten locations, and if we were to handle all of these tactile signals, the bit rate would be ten times higher, at 480 kbit / sec. If we were to add each finger joint, palm, and other locations, the bit rate would increase dramatically.

[0076] Furthermore, while tactile signals are essentially one-dimensional, the physical phenomenon of vibration can be captured in three axes (x, y, z). To handle all of this, a bit rate of 1440 kbit / sec, three times faster, would be required, exceeding the 1411 kbit / sec of an audio CD.

[0077] In this way, the total amount of data required for haptic signals increases with improvements in the reproducibility of haptic stimulation and with an increase in the number of parts of the user to which haptic stimulation is applied, and this increase in the total amount of data places a heavy load on the network system that transmits the haptic signals.

[0078] Here, we consider the vibrations generated by the action of "tracing an object with one's hand," an important part of tactile sensation. This action can be divided into three stages: a person touches an object, then moves the hand along the object's surface, and finally removes the hand from the object. If we focus on the vibration waveforms generated by this series of actions, we can see that the "moment of contact" and the "moment of removal" are very short in time, but are accompanied by a steep change in intensity. On the other hand, the vibration waveform while the hand is moving along the object's surface has a relatively long time scale and is steady.

[0079] In this specification, a contact state refers to a state in which the part from which a tactile signal is to be obtained is in contact with the object, such as when a person touches an object, then moves their hand along the surface of the object, and finally removes their hand from the object. A non-contact state refers to a state in which the part from which a tactile signal is to be obtained is not in contact with the object.

[0080] Also, a section of a tactile signal that occurs in conjunction with a series of actions such as "tracing an object with a hand", that is, a signal section in the tactile signal that indicates a contact state with an object, is defined as a contact signal section. In this case, the signal section including the boundary between the contact state and the non-contact state with the object, such as "the moment the hand touches the object" or "the moment the hand leaves the object", is defined as the first signal section. In addition, the signal section excluding the first signal section, such as "the state where the hand is placed on the object" or "the state where the hand is moving along the surface of the object", is defined as the second signal section.

[0081] FIG. 7 is a diagram showing a vibration waveform 70 of a tactile signal when a human traces an object 79 with a hand 78. In vibration waveform 70, a steep pulse waveform is observed as in signal section 73 at the moment when hand 78 in scene 72 touches object 79 after being released from object 79 in scene 71. It is known that the vibration intensity, timing, pitch, and waveform decay rate at the "moment of contact" affect perception of the hardness of an object and the speed at which one touches object 79. The signal section 73 including the boundary between the contact state and the non-contact state of the hand 78 and the object 79 is set as a first signal section. This signal section 73 is very short in time.

[0082] On the other hand, as shown in scene 75, a waveform of signal section 74 accompanied by a gradual change in amplitude is observed as a result of moving hand 78 over object 79. The vibration during this period affects the texture perception of object 79, such as smoothness or roughness, and the pitch and amplitude of the waveform are considered important. A signal section 74 in which the contact state between the hand 78 and the object 79 is maintained in this manner is defined as a second signal section.

[0083] Finally, as shown in scene 76, when hand 78 is removed from object 79, the waveform of corresponding signal section 77 is accompanied by a short and steep change in intensity, and is presumed to have properties similar to the waveform at the time of contact in signal section 73. Signal section 77, which includes the boundary between the contact state and non-contact state of hand 78 and object 79, is defined as the first signal section.

[0084] A series of signal sections including the signal section 73, the signal section 74, and the signal section 77 is defined as a contact signal section 200 that indicates a contact state with the object 79 in the haptic signal.

[0085] Here, when the signal is divided into signal sections 73 and 77 (first signal section) where there are sudden changes in intensity, and signal section 74 (second signal section), the first signal section has more and more complex features that need to be preserved due to the nature of perception, so it is desirable to employ a high-quality encoding method with a relatively high bit rate, such as waveform encoding such as AAC (Advanced Audio Codec) or lossless compression encoding such as FLAC (Free Lossless Audio Codec).

[0086] On the other hand, the steady second signal section has a gradual change over time, so even if a long signal is encoded at once, there is little information loss, and the features to be preserved are few and simple. Therefore, it is expected that humans will not notice the degradation easily even with low-quality encoding methods with lower bit rates than high-quality encoding methods such as CELP (Code Excited Linear Prediction) and HVXC (Harmonic Vector Excitation Coding), which have a high compression rate and place emphasis on reproducing the energy of the signal, or the parametric encoding that synthesizes an equivalent signal from the features to be preserved mentioned above.

[0087] Therefore, if even the longer signal section 74 is coded by a high-quality coding method in order to accurately reproduce shorter sections such as signal section 73 and signal section 77, the haptic signal as a whole will have redundant information and the compression efficiency will decrease. The increase in the total data volume due to such a decrease in compression efficiency will also be a factor in causing delays in the transmission and reception of haptic signals.

[0088] Further, factors other than the total data volume as described above may also be considered as causes of delays. For example, when a haptic signal is transmitted by wireless communication, it is assumed that a loss of haptic encoded data Dc of the haptic signal occurs due to interference on the transmission path. When data loss occurs, the transmitting device retransmits the data, and a delay may occur in the time until the receiving device completes receiving the data. In other words, as the amount of data to be retransmitted increases, the time required to retransmit the data corresponding to the data loss increases, and as a result, the time until the transmission and reception of the haptic signal is normally completed may be further delayed.

[0089] In this way, if there is a delay in the time until the transmission of the haptic signal is completed, the reproducibility of the haptic sensation may be reduced. Specifically, if the haptic stimulation is not given to the user (the receiver) at the appropriate time, a situation may occur in which the haptic stimulation is not synchronized with content related to other senses, such as video and sound.

[0090] Next, a specific application of a wireless communication scheme will be described taking into consideration the above-mentioned circumstances. Since a device that presents haptic sensation is installed so as to come into contact with a user, it is generally desirable to communicate with other devices wirelessly in terms of device weight. However, when using a broadband wireless method such as Wi-Fi (registered trademark), the battery of the device becomes large in terms of power consumption, and this is expected to reduce user convenience. In addition, when using Wi-Fi, a processing time is generally required for the procedure from a signal transmission request of a transmitting device to reception processing of a receiving device, so a larger delay may occur compared to other wireless methods.

[0091] On the other hand, short-range wireless systems such as Bluetooth are capable of communication with lower power consumption and lower latency than other wireless systems, and are therefore considered suitable for transmitting haptic signals. However, short-range wireless systems have a smaller capacity for transmitting data at one time than other wireless systems. For example, in transmitting content that provides a user with a haptic stimulus while synchronizing with video, sound, etc., it is conceivable that the communication capacity allocated to transmitting the haptic signal is insufficient.

[0092] Furthermore, in a service that streams video and audio over the Internet, when haptic sensations are also conveyed to the user, it is conceivable that the communication capacity allocated to the transmission of haptic signals may become insufficient due to, for example, Quality of Service (QoS) functions that respond to network line conditions.

[0093] In this embodiment, in consideration of the above circumstances, the objective is to achieve both low latency of haptic signals and suppression of degradation of haptic reproducibility by reducing the amount of transmitted data without impairing haptic reproducibility as much as possible.

[0094] [6-2. Encoding method] 8 is a functional block diagram showing functions of the encoding unit 24. As shown in the figure, the encoding unit 24 has a signal input unit 80, a first buffer processing unit 86, a determination unit 82, a first encoding unit 83, a second buffer processing unit 87, a second encoding unit 84, and an encoded signal output unit 85.

[0095] A haptic signal 81 to be encoded is input to the signal input unit 80 from the pre-processing unit 23 . The first buffer processing unit 86 has a first buffer memory used to determine the first signal section and the second signal section, and buffers the haptic signal 81 input to the signal input unit 80.

[0096] As shown in FIG. 7, for example, the judgment unit 82 judges a first signal section (signal section 73, 77) which is a portion of a contact signal section 200 in the tactile signal and which includes the boundary between a contact state and a non-contact state between an object 79 and a hand 78, and a second signal section (signal section 74) which is a signal section of the contact signal section 200 excluding the first signal section.

[0097] The determination unit 82 determines a first signal section based on, for example, the result of waveform analysis of the haptic signal 81. The determination unit 82 also determines a signal section other than the first signal section as a second signal section.

[0098] A representative waveform analysis for determining the first signal section is detection of a rising edge of a signal based on the rate of change of the amplitude of the haptic signal, which is performed by a known signal processing method.

[0099] Here, the determination unit 82 can determine the first signal section by detecting the rate of change in signal strength, frequency spectrum, or other attacks known in the field of signal processing. The determination unit 82 can also determine the first signal section based on the recognition by various other sensors of the distance between the object and the tactile sensor or the presence or absence of contact. The determining unit 82 may also determine the encoding method using metadata (encoding quality flagging) or the like added by a creator (worker). Furthermore, as the determination unit 82, a determination device that performs machine learning using a large number of vibration signals and can automatically determine the target section from the input waveform may be used.

[0100] The determination of the first signal section by the determination unit 82 can extend the effect of the present technology to general signals corresponding to the moment of contact and the moment of separation between a human and an object 79 in touch. If it is determined that the entire or part of the haptic signal 81 is the first signal section, the determining unit 82 sends that section to the first encoding unit 83 if it is a part, or sends the entire haptic signal 81 if it is the entirety. The determining unit 82 also sends the second signal section of the haptic signal 81 that was not sent to the first encoding unit 83 to the second buffer processing unit 87 .

[0101] The first encoding unit 83 performs encoding using a high bit rate encoding method on the haptic signal of the first signal section received from the determination unit 82, and generates first encoded data (haptic encoded data Dc). In the following description, the high bit rate encoding method is also referred to as a high quality encoding method. The high-quality encoding method used in the first signal section should preferably be a relatively high-bitrate method that can restore complex waveform information, such as the pulse height, width, timing, and rate of energy change of the signal, with high time resolution. For example, waveform encoding such as AAC or lossless compression encoding such as FLAC are suitable.

[0102] The second buffer processing unit 87 has a second buffer memory, and buffers the input signal according to the result of the signal section determination by the determination unit 82 . The second encoding unit 84 encodes the signal of the second signal section buffered by the second buffer processing unit 87 using an encoding method with a bit rate lower than the high-quality encoding method for the first signal section, and generates second encoded data (haptic encoded data Dc). In the following description, the encoding method with a bit rate lower than the high-quality encoding method is also referred to as a low-quality encoding method.

[0103] For the low-quality coding method used in the second signal section, it is desirable to code a small number of parameters such as energy and pitch using a long time frame. Therefore, methods that emphasize signal energy and pitch reproduction such as CELP and HVXC with high compression rates, and parametric coding methods that synthesize equivalent signals from the features to be preserved as mentioned above are suitable. Regarding these parameters, for example, the energy can be obtained by calculating the effective value of the signal f[n] as shown in the following [Equation 2].

[0104]

number

[0105] A typical method for acquiring the pitch is to find the peak point m_0 of the autocorrelation function of the signal as shown in the following [Equation 3], and use the reciprocal of the corresponding time as the pitch frequency.

[0106]

number

[0107] The coded signal output unit 85 outputs the first coded data generated by the first coding unit 83 and the second coded data generated by the second coding unit 84.

[0108] Fig. 9 shows an example of the data format of the haptic encoded data Dc. Fig. 9 shows an example of the data format of one frame of the haptic encoded data Dc.

[0109] An identifier 100 is stored at the beginning of the frame header which is added to the signal as additional information, the signal sampling frequency is recorded in area 101, and the signal quantization accuracy is recorded in area 102. An encoding method ID (Identification) which identifies whether the data was encoded by a high-quality encoding method or a low-quality encoding method is stored in area 103. For example, a "0" is stored for a high-quality encoding method, and a "1" is stored for a low-quality encoding method. Area 104 stores the number of samples of the signal contained in this frame, and area 105 stores the signal data.

[0110] An example of a processing procedure for implementing the encoding method according to the embodiment described above will be described with reference to the flowchart in Fig. 10. The processing shown in Fig. 10 is processing for generating one frame of haptic encoded data Dc, and is repeatedly executed for each frame. The process for implementing the encoding method by the encoding unit 24 can be implemented as software processing, or as hardware, or a combination of software and hardware.

[0111] First, the encoding unit 24 acquires the haptic signal to be encoded from the pre-processing unit 23 in step S110. In step S111 following step S110, the encoding unit 24 buffers the acquired haptic signal in order to determine the first signal section and the second signal section. This corresponds to the processing of the first buffer processing unit 86 described above.

[0112] In step S112 following step S111, the encoding unit 24 determines whether the signal section of the acquired haptic signal is a first signal section to be encoded using a high-quality encoding method or a second signal section to be encoded using a low-quality encoding method.

[0113] At this time, the encoding unit 24 determines whether the signal rises as shown in signal section 73 in Fig. 7 or whether the signal falls as shown in signal section 77, for example, based on a comparison between the amplitude change rate of the haptic signal and a predetermined threshold. The encoding unit 24 determines a signal section in which such a signal rise or fall is detected as a first signal section. Moreover, the encoding unit 24 determines a signal section in which no signal rise or fall is detected as a second signal section.

[0114] The encoding unit 24 can also use the frequency spectrum of the haptic signal to determine rising and falling edges of the signal based on the intensity distribution of the haptic signal decomposed into each frequency component.

[0115] The encoding unit 24 can also determine the first signal section and the second signal section based on division information of the first signal section and the second signal section added to the haptic signal 81. The division information is added as metadata by the creator, for example, as described above.

[0116] Here, an example will be described in which metadata added to the haptic signal is used to determine the first signal section and the second signal section. Fig. 11 shows an example of the data format of the haptic encoded data Dc. Fig. 11 shows an example of the data format of one frame of the haptic signal input to the encoding unit 24. In addition, a WAVE file is used here as an example of the data format.

[0117] In a WAVE file, header information including format information such as sampling frequency, quantization precision, and number of channels is stored in a first area 130, and signal data is stored in an area 131.

[0118] In this embodiment, metadata is added to the end of this data to determine whether the first signal section is to be encoded by a high-quality encoding method or the second signal section is to be encoded by a low-quality encoding method. In the following description, encoding by the high-quality encoding method is also referred to as high-quality encoding, and encoding by the low-quality encoding method is also referred to as low-quality encoding.

[0119] Here, an example is shown in which the number of samples is assigned from the beginning of the file at the start and end points of the first signal section for the channel to be subjected to high quality encoding. The channel ID to be subjected to high quality encoding is stored in area 132, the starting sample number is stored in area 133, and the ending sample number to be subjected to high quality encoding is stored in area 134. If there are multiple first signal sections to be subjected to high quality encoding, sets of areas 132, 133, 134 may be added for the number of sections.

[0120] At this time, in step S112 of Figure 10, the encoding unit 24 checks in advance the end data (areas 132, 133, 134) of the signal input as the target for encoding, and determines whether the currently buffered signal section is the signal section specified by the metadata.

[0121] If it is determined that the first signal section is to be coded using high quality coding, the coding unit 24 advances the process from step S112 to step S113, and codes the first signal section using high quality coding, i.e., the coding unit 24 codes the first signal section using a coding method with a higher bit rate than the low quality coding.

[0122] In step S114 following step S113, the encoding unit 24 generates first encoded data to which an encoding method ID corresponding to the high-quality encoding method is assigned. Thereafter, the encoding unit 24 advances the process to step S118.

[0123] On the other hand, if it is determined in step S112 that the second signal section is other than the first signal section for which high-quality encoding is to be performed, the encoding unit 24 proceeds to step S115 and buffers the haptic signal until low-quality encoding can be performed.

[0124] In this embodiment, the high-quality encoding has a shorter transformation length, and the low-quality encoding has a longer transformation length than the high-quality encoding. Therefore, at the stage when it is determined that high-quality encoding is not necessary, the haptic signal may not yet be buffered to the extent that low-quality encoding is possible. Therefore, in step S115, the encoding unit 24 buffers the haptic signal with the number of samples corresponding to the low-quality encoding. This corresponds to the processing performed by the second buffer processing unit 87 described above.

[0125] In step S116 following step S115, the encoder 24 performs low-quality encoding on the second signal section, that is, the encoder 24 performs encoding using an encoding method with a lower bit rate than the high-quality encoding.

[0126] In step S117 following step S116, the encoding unit 24 generates second encoded data to which an encoding method ID corresponding to the encoding method for low-quality encoding has been assigned. Thereafter, the encoding unit 24 advances the process to step S118.

[0127] In step S118, the encoding unit 24 determines whether or not the entire signal has been encoded, that is, whether or not the process of generating the first encoded data and the second encoded data for all channels of the haptic signal has been completed.

[0128] If the processing of all signal intervals has not been completed, the encoding unit 24 returns to step S111, whereby the same processing as for one frame is executed for the next interval. On the other hand, when the processing for all signal sections is completed, the encoding unit 24 ends the series of processing shown in Fig. 11. When there are multiple channels, the encoding unit 24 executes the processing shown in Fig. 11 for all channels.

[0129] In this embodiment, an example has been described in which the transformation length of the low-quality encoding method is longer than that of the high-quality encoding method, but this technology can also be applied to cases in which the transformation lengths of the low-quality encoding method and the high-quality encoding method are equal.

[0130] In this case, the encoding unit 24 always performs the buffering in step S111 with the number of samples equal to the transform length, and applies only one type of encoding quality to each signal buffered at one time. This makes it unnecessary to perform the buffering in step S115 before encoding by low-quality encoding in step S116, as shown in Fig. 12. By making the transform lengths of low-quality encoding and high-quality encoding equal in this way, the processing load of the encoding unit 24 is reduced.

[0131] [6-3. Decryption method] The functions of the combining unit 33 and the decoding unit 34 in the decoding device 3 will be described with reference to FIG. The decoding unit 34 includes a first decoding unit 34a and a second decoding unit 34b.

[0132] The first decoding unit 34a decodes the first encoded data obtained by performing high-quality encoding on the first signal section, which is a part of the contact signal section. The first decoding unit 34a transmits the first waveform data obtained by decoding the first encoded data to the combining unit 33.

[0133] The second decoding unit 34b decodes the second coded data obtained by performing low-quality coding on the second signal section, which is a signal section excluding the first signal section. The second decoding unit 34b transmits the second waveform data obtained by decoding the second coded data to the combining unit 33.

[0134] The combining unit 33 combines the first waveform data obtained by decoding the first encoded data with the second waveform data obtained by decoding the second encoded data.

[0135] The combining unit 33 determines whether the waveform data currently being output is encoded by a different encoding method from that of the currently decoded waveform data waiting to be output. If the encoding methods are the same, the combining unit 33 appropriately post-processes the signal in accordance with the technique of the encoding method and outputs it. Specifically, in this post-processing, the waveform data waiting to be output and the waveform data being output are combined in a normal manner for the encoding method, but the combining process involving cross-fading described below is not performed.

[0136] On the other hand, if the encoding methods are different, there is a possibility that the number of signal samples or the handling of edges may differ between the two, and an unpleasant discontinuous waveform may be generated if the two are output as is. Therefore, when outputting decoded data following waveform data that has been encoded using a different encoding method, the combining unit 33 performs signal processing so that the two are smoothly connected.

[0137] The combining unit 33 performs cross-fade processing on a combined portion of the first waveform data and the second waveform data that have been encoded by different encoding methods, for example. 13, when first waveform data 91 that has been coded in high quality and second waveform data 92 that has been coded in low quality are output consecutively, a section 93 where the first waveform data 91 and the second waveform data 92 overlap is provided, and by multiplying and adding a coefficient such that the sum of the section becomes "1", it is possible to smoothly connect the first waveform data 91 and the second waveform data 92 as an output waveform 94. This is also true when second waveform data 92 that has been coded in low quality and first waveform data 91 that has been coded in high quality are output consecutively.

[0138] The combining unit 33 may use a known process other than the cross-fade process to combine the first encoded data and the second waveform data.

[0139] An example of a processing procedure for implementing the decoding method according to the embodiment described above will be described with reference to the flowchart in Fig. 14. The processing shown in Fig. 14 is processing for obtaining a haptic signal by decoding one frame of haptic encoded data Dc, and is repeatedly executed for each frame. The process for implementing the encoding method by the decoding device 3 can be implemented as software processing, or can be implemented by hardware, or a combination of software and hardware.

[0140] First, in step S140, the decoding unit 34 of the decoding device 3 obtains the haptic encoded data Dc. In step S141 following step S140, the decoding unit 34 obtains the encoding method ID from the haptic encoded data Dc.

[0141] In step S142 following step S141, the decoding unit 34 determines, based on the acquired encoding method ID, whether the haptic encoded data Dc is first encoded data encoded by a high-quality encoding method or second encoded data encoded by a low-quality encoding method.

[0142] If the encoding method ID indicates encoding by the high-quality encoding method, the decoding unit 34 proceeds from step S142 to step S143, and decodes the first encoded data by a decoding method corresponding to the high-quality encoding method. The decoding unit 34 transmits the first waveform data obtained by decoding the first encoded data to the combining unit 33. Thereafter, the decoding unit 34 advances the process from step S143 to step S145.

[0143] On the other hand, if the coding method ID indicates coding by a low-quality coding method, the decoding unit 34 proceeds from step S142 to step S144, and decodes the second coded data by a decoding method corresponding to the low-quality coding method. The decoding unit 34 transmits the second waveform data obtained by decoding the second coded data to the combining unit 33. Thereafter, the decoding unit 34 advances the process from step S144 to step S145.

[0144] In step S145, the combining unit 33 of the decoding device 3 determines whether the encoding method of the waveform data currently waiting to be output and decoded by the decoding unit 34 is different from that of the waveform data currently being output. If the encoding methods are different, the combining unit 33 advances the process from step S145 to step S146, and performs the cross-fade process as shown in FIG. 9 on the combined portion of each of the waveform data.

[0145] The cross-fade process executed by the combining unit 33 will now be described in detail with reference to FIGS. As shown in FIG. 15, the combining unit 33 passes the decoded signal 150 decoded in step S143 or step S144 in FIG. 14 to a block 151 that delays the signal by one frame by buffering or the like.

[0146] Here, the combiner 33 compares the delayed signal (waveform data waiting to be output) obtained by passing through the block 151 with the latest decoded signal 150 (waveform data currently being output). When the delayed signal and the decoded signal 150 have the same coding scheme, the combining unit 33 combines the latest decoded signal 150 with the delayed signal by normal combining processing as shown in Fig. 15 to generate the haptic output 147. The adder 155 in the figure means combining the decoded signal 150 with the delayed signal in this manner.

[0147] Here, when the coding methods of the delayed signal and the decoded signal 150 are different, the combining unit 33 performs a cross-fade process shown in Fig. 16. The delayed signal and the decoded signal 150 have overlapping portions between frames provided as blank spaces. These blank spaces are used as blank spaces for combining.

[0148] In the cross-fade process, a cross-fade function 152 that is multiplied by the decoded signal 150 and a cross-fade function 153 that is multiplied by the delayed signal are used. Any window function whose sum is "1", such as a triangular window or a Hanning window, can be used for the fade-in and fade-out parts of the cross-fade functions 152 and 153. The cross-fade function 152 is a function that transitions from "0" to "1" as time passes in the joining margin part and maintains "1" after the joining margin part. The cross-fade function 153 is a function that transitions from "1" to "0" as time passes in the joining margin part.

[0149] The combining unit 33 multiplies the decoded signal 150 by a cross-fade function 152 in a multiplier 156, and multiplies the delayed signal by a cross-fade function 153 in a multiplier 157. The combining unit 33 generates the haptic output 147 by adding these multiplication results together in an adder 158.

[0150] After performing the cross-fade process in step S146 of FIG. 14, the combining unit 33 advances the process to step S147.

[0151] If the encoding formats are not different, that is, if the currently received waveform data and the next waveform data to be received have the same encoding format, the combining unit 33 proceeds to step S147 without performing the cross-fade process in step S146. This is because encoding formats are generally designed to smoothly connect signals even at the connections between transform units, and there is no need to perform additional processing to connect signals.

[0152] In step S147, the combining unit 33 outputs the decoded haptic signal. In step S148 following step S147, the decoding unit 34 determines whether or not all of the haptic encoded data Dc has been decoded, that is, whether or not the decoding process has been completed for the haptic encoded data Dc for all signal sections of the haptic signal.

[0153] If the processing of all signal intervals has not been completed, the decoding unit 34 returns to step S140, whereby the same processing as for one frame is executed for the remaining intervals. On the other hand, when the processing of all signal sections has been completed, the decoding unit 34 ends the series of processing shown in Fig. 14. When there are multiple channels, the decoding unit 34 executes the processing shown in Fig. 14 for all channels.

[0154] <7. First application example> As a first application example of this technology, consider a haptic movie in which vibrations are presented to the viewer as a haptic sensation in addition to video and audio, as shown in FIG.

[0155] First, in the production stage, in addition to video and audio, vibrations are also recorded. For this recording, for example, a tactile sensor 161 (tactile sensor 5) is attached to the body of an actor 160 to record the vibrations felt by the characters in the movie. In another example, a tactile sensor is installed on the floor of the recording site to record the vibrations occurring in the environment.

[0156] The vibration waveform thus recorded is coded in a coding device 162 (coding device 2). The details of the coding are as described with reference to Figs. 8, 10, and 12, and through control of coding quality involving signal analysis and the use of meta-information, sections with sudden changes in signal strength are coded using a high-quality coding method, and other sections are coded using a low-quality coding method. After the video and audio are also coded using a known method, the coded data of the video, audio, and vibration is stored in a storage device 163, such as a DVD (Digital Versatile Disc).

[0157] For example, when a character runs his or her hand along a wall, the moment of contact is a vibration waveform that changes significantly over time, so high-quality coding is desirable, but low-quality coding is sufficient for the period after that when the character's hand is moving along the wall.

[0158] Therefore, when a haptic signal is input by tactile sensor 161 attached to the hand of actor 160, a known signal processing method is applied to the haptic signal to detect the rising edge, and high-quality encoding is performed only for that section (first signal section). Alternatively, when a creator wants to make the viewer feel the moment when the wall is touched while looking at a captured image, such meta-information may be added to the haptic signal. In this case, this meta-information is detected by decision unit 82 in Fig. 8, and high-quality encoding is performed only for the haptic signal at that point. This produces tactile encoded data Dc that is coded with high quality only at the moment the wall is touched, and low quality at other times.

[0159] As another example, when a movie is shot, the vibration caused by touching a wall is not recorded, but the creator may design a waveform that gives the impression of touching a wall. Even in this case, the encoding method can be determined by the signal processing method and meta-information added separately by the creator, as in the above case.

[0160] A viewer 164 watches a movie from this storage 163 via a player 165. At this time, the player 165 decodes and outputs the video and audio from a known video and audio output device using a known method.

[0161] 14, the vibration signal is decoded in the decoding device 166 (decoding device 3), and the vibration signal is output from the vibration output device 167. That is, the signal is decoded for each section according to the encoding method ID assigned at the time of encoding, and signal switching processing is performed as necessary to make the signal smooth, and the signal is output as a decoded signal.

[0162] In this case, the number of systems of the vibration signals to be output may be multiple according to the number of systems stored in the storage, or may be adjusted according to the number of devices owned by the viewer 164. A known channel number conversion method can be used for this adjustment.

[0163] In this embodiment, it is possible to save the storage required for saving vibration data, which has the effects of reducing storage costs, realizing further multi-channel and high-resolution vibration, and increasing the storage available for video and audio.

[0164] <8. Second application example> As a second application example of this technology, consider a cloud game as shown in FIG. 18, in which all operations by a user 174 are transmitted from an edge 176 to a cloud 172, and the resulting changes in video, audio, and haptics are all calculated and processed on the cloud 172, and then transmitted to the edge 176 (where the user 174 is located) and played.

[0165] A typical home game, in other words, an example in which a machine is held in the user's hand and button operations by the user 174 are sent to the machine, and various signals processed there are transmitted to various output devices via wire or very short-distance wireless communication, has the same problems, configuration, and effects as the haptic movie described in the first application example above, so a description thereof will be omitted.

[0166] In cloud gaming, since there is no high-performance computer on the edge 176, the interpretation of complex user operations and the synthesis and adjustment of the resulting feedback are all performed in the cloud 172, and the resulting large amounts of data such as video, audio, and haptics are transmitted from the cloud 172 to the edge 176.

[0167] In this case, if the communication bandwidth from the cloud 172 to the edge 176 is not sufficiently secured, noise and signal interruptions will occur on the edge 176 side, significantly impairing the user experience. In addition, tactile vibrations change depending on the way of touching the same object, so storing them as separate signals would require a huge amount of storage. Although a method of storing only the reference signal and modulating it according to the object and the way of touching is also conceivable, it is expected that the former method will output a higher quality signal.

[0168] A second application example will now be described. It is assumed that video and audio are handled using known methods, and only haptics will be described in this application example. First, at the game creation stage, a recorded or synthesized vibration signal 170 (haptic signal) is prepared. For example, this may be the vibration caused by firing a gun, opening a door, tracing a wall, or being tapped on the shoulder. It may also be a vibration that is difficult to actually perceive, such as a heartbeat. Each of these signals is coded in coding device 171 (coding device 2).

[0169] As described above, this encoding involves quality control and application of an appropriate encoding method for each section. The haptic encoded data Dc obtained in this manner is stored in the storage 173 in the cloud 172.

[0170] Next, when a user 174 plays this game, first, an operation by the user 174 is input from a controller 175 and transmitted from an edge 176 to the cloud 172. In the cloud 172, the operation by the user 174 is received by a transmitting / receiving unit 177, recognized by a control unit 178, and haptic encoded data Dc of the vibration signal to be output from a storage 173 is selected and read out. The read haptic encoded data Dc is transmitted as is to the edge 176 , where it is decoded by a decoding device 179 (decoding device 3 ) and output from a vibration output device 1710 .

[0171] This reduces the amount of information in the recorded haptic encoded data Dc, making it possible to reduce the storage capacity on the cloud 172. Also, by communicating the data in an encoded state and decoding it on the edge 176 side, the amount of communication data can be reduced, and even if the communication bandwidth between the cloud 172 and the edge 176 is narrow, the user 174 can enjoy a realistic haptic experience through vibration signals in cloud games. This application example is simple since the decoding process involves only reading the encoding method ID and decoding accordingly, and has the advantage that it does not require a high-performance computer on the edge 176 side.

[0172] As a modification of the second application example, the encoding quality can be adjusted according to the communication bandwidth. Before the haptic encoded data Dc read from the storage 173 is transmitted directly from the transmitting / receiving unit 177, the control unit 178 acquires communication bandwidth information between the cloud 172 and the edge 176.

[0173] If the current communication environment is good, the haptic encoded data Dc is transmitted as is from the transmission / reception unit 177. On the other hand, if the communication environment is poor, the read haptic encoded data Dc is re-encoded by the encoding device 171. At this time, the control unit 178 provides new haptic encoded data Dc that is appropriate for the communication environment.

[0174] For example, if the communication environment is determined to be poor when transmitting tactile encoding data Dc of a vibration signal tracing a wall, it is possible to further reduce the amount of data, even at the expense of some quality, by using a technique such as encoding over a longer time frame and reducing the frequency of parameter updates in the second signal section, in which low-quality encoding is performed while the hand is moving along the wall.

[0175] In the second application example of the present technology, the first signal section for high quality coding and the second signal section for low quality coding are divided based on human perception characteristics, so that such adjustments can be easily made while maintaining human perception. For example, the above adjustments are targeted at sections for which time resolution is not considered to be important, so that the impact of adjusting the parameter update frequency is considered to be limited.

[0176] <9. Third Application Example> As a third application example of the present technology, consider an example in which the texture of a product 180 is reproduced as vibrations in a mail-order business via the Internet as shown in Fig. 19, allowing a user 186 to feel tactile information of the product 180. For example, a user 186 searches for a product on a shopping site, and outputs vibrations from a vibrating device 1810 in his / her hand as detailed information, allowing the user 186 to confirm the texture of the product 180.

[0177] First, in order to record the feel of the product 180, a vibration sensor 181 (tactile sensor 5) is used to record vibrations when touched in various ways. At this time, a robot arm 182, for example, may be used to repeat the touching action.

[0178] The vibration signal data acquired by the vibration sensor 181 is coded by the coding device 183 according to the touching method. The coding method is as described above, and in this application example, a separate contact sensor 184 (tactile sensor 5) is attached to the robot arm 182, and the coding quality is controlled based on the information from the contact sensor. For example, the moment when the robot arm 182 touches the product 180 and the moment when the robot arm 182 leaves the product 180 can be identified by the contact sensor 184.

[0179] Since it is predicted that the signal will change rapidly over a certain period of time from that moment, this period is designated as the first signal period for high-precision encoding. In this case, quality control is completed using information from the contact sensor 184, so no signal processing is required. These haptic encoded data Dc are then stored in the storage 185.

[0180] Next, a playback request from user 186 is received by transceiver 187, and appropriate haptic encoded data Dc is read from storage 185 by control unit 188 and transmitted to user 186. On the user 186 side, the received haptic encoded data Dc is decoded by decoding device 189 (decoding device 3), and vibrations are reproduced from a vibration device 1810 held by user 186.

[0181] This allows the user 186 to feel the texture of the product 180 over the Internet. In addition, due to the effect of this application example, the data volume is reduced, so the vibration data can be downloaded in a short time and the load on the web page is also reduced, so the quality of the user experience is less likely to be impaired.

[0182] <10. Summary> The decoding device 3 in the above embodiment includes a first decoding unit 34a that decodes first encoded data obtained by performing encoding (high-quality encoding) at a first bit rate for a first signal section (e.g., signal sections 73, 77), which is a portion of a contact signal section 200 that is a signal section that indicates a contact state with an object in a haptic signal and that includes the boundary between a contact state and a non-contact state with the object, and a second decoding unit 34b that decodes second encoded data obtained by performing encoding (low-quality encoding) at a bit rate lower than the first bit rate for a second signal section (e.g., signal section 74), which is a signal section of the contact signal section 200 excluding the first signal section (see S140 to S144 in Figures 4, 7, and 14). By decoding tactile encoded data Dc in which only important sections of the contact signal are encoded at a high bit rate (high quality encoding) and the other sections are encoded at a low bit rate (low quality encoding), it is possible to achieve both a reduction in data volume and ensuring tactile reproducibility. Therefore, data in which the overall amount of transmission data of the haptic encoded data Dc is reduced can be received with low delay without impairing the haptic reproducibility as much as possible, so that the haptic receiver can experience high-quality haptics without feeling any delay.

[0183] The decoding device 3 of the embodiment includes a combining unit 33 that combines first waveform data obtained by decoding the first encoded data and second waveform data obtained by decoding the second encoded data (see S145 and S146 in Figures 4 and 14). As a result, the waveform data are integrated into one waveform data. Therefore, it is possible to prevent the occurrence of unpleasant discontinuous waveforms caused by separately outputting each waveform data, and it is possible to improve the reproducibility of tactile sensations.

[0184] In the decoding device 3 of the embodiment, the combining unit 33 performs cross-fade processing on the combined portion of the first waveform data and the second waveform data (see S146 in FIG. 14, and FIGS. 15 and 16). This allows the first waveform data and the second waveform data to be smoothly combined. Therefore, it is possible to prevent the occurrence of unpleasant discontinuous waveforms caused by separately outputting each waveform data, and it is possible to improve the reproducibility of tactile sensations.

[0185] In addition, the decoding method in the embodiment is a decoding method that decodes first encoded data obtained by encoding (high quality encoding) at a first bit rate for a first signal section (e.g., signal sections 73, 77), which is a portion of a contact signal section that is a signal section that indicates a contact state with an object in a haptic signal and includes a boundary between a contact state and a non-contact state with the object, and decodes second encoded data obtained by encoding (low quality encoding) at a bit rate lower than the first bit rate for a second signal section (e.g., signal section 74), which is a signal section of the contact signal section excluding the first signal section.

[0186] The decoding method according to this embodiment can also achieve the same effects and advantages as the decoding device 3 according to the above embodiment.

[0187] Furthermore, the encoding device 2 in the embodiment includes a determination unit 82 that determines a first signal section (e.g., signal sections 73, 77) which is a portion of contact signal section 200, which is a signal section that indicates a contact state with an object in a haptic signal and includes the boundary between a contact state and a non-contact state with the object, and a second signal section (e.g., signal section 74) which is a signal section of contact signal section 200 excluding the first signal section, a first encoding unit 83 that performs encoding at a first bit rate (high quality encoding) for the first signal section, and a second encoding unit 84 that performs encoding at a bit rate lower than the first bit rate (low quality encoding) for the second signal section (see Figures 7, 8, and 10). By encoding only important sections of the contact signal at a high bit rate, and encoding the other sections at a low bit rate, it is possible to achieve both a reduction in data volume and ensuring tactile reproducibility. Therefore, the overall amount of transmitted data of the haptic encoded data Dc can be reduced without impairing the haptic reproducibility as much as possible, and the haptic encoded data Dc can be transmitted with low delay while maintaining the perceptual quality of the haptic signal.

[0188] In the encoding device 2 of the embodiment, the determining unit 82 determines the first and second signal sections based on the division information of the first and second signal sections added to the haptic signal (S112 in FIG. 10, see FIG. 11). This eliminates the need for processing such as waveform analysis when determining whether the signal is the first or second signal section. This reduces the processing load on the encoding device 2 for determining whether the first signal section or the second signal section is involved, thereby enabling more efficient encoding processing.

[0189] In the encoding device 2 of the embodiment, the determining unit 82 determines whether the signal section is a first signal section or a second signal section based on the result of the waveform analysis of the haptic signal (see S112 in FIG. 10). This makes it unnecessary to add information for distinguishing the first signal section from the second signal section to the contact signal when encoding the first signal section and the second signal section separately. Therefore, the burden on the worker who creates a contact signal to which division information is added for each signal section can be reduced.

[0190] In the encoding device 2 of the embodiment, the determining unit 82 determines whether the signal section is the first signal section or the second signal section based on the amplitude change rate of the haptic signal (see S112 in FIG. 10). This makes it possible to appropriately determine the first signal section that involves a steep change in the waveform of the haptic signal. Therefore, it is possible to appropriately achieve both a reduction in the amount of data and an assurance of tactile reproducibility.

[0191] In the encoding device 2 of the embodiment, the second encoding unit 84 performs encoding for the second signal section using an encoding method with a longer transform length than that used for encoding the first signal section (see S116 in FIG. 10). This reduces the amount of second encoded data in the second signal section. Therefore, the overall amount of transmitted data of the haptic encoded data Dc can be reduced.

[0192] In the encoding device 2 according to the embodiment, the second encoding unit 84 performs parametric encoding on the second signal section (see S116 in FIG. 10). This reduces the amount of second encoded data in the second signal section. Therefore, the overall amount of transmitted data of the haptic encoded data Dc can be reduced.

[0193] The encoding device 2 of the embodiment is provided with a first buffer memory used to determine the first signal section and the second signal section, and a second buffer memory used to encode the second signal section depending on the result of the determination (see S111 and S115 in Figures 8 and 10). This makes it possible for the second buffer processing unit 87 to buffer the haptic signal of the second signal section, which has a longer transform length than the encoding for the first signal section, that cannot be completely buffered by the first buffer processing unit 86. Therefore, since there is no need to adjust the buffering period to the second signal section having a longer transform length, it is possible to prevent delays in the processing time for high-quality encoding of the haptic signal of the first signal section, thereby improving the efficiency of the encoding process.

[0194] Furthermore, the encoding method in the embodiment is an encoding method that determines a first signal section (e.g., signal sections 73, 77) which is a portion of contact signal section 200, which is a signal section that indicates a contact state with an object in a haptic signal, and which includes the boundary between a contact state and a non-contact state with the object, and a second signal section (e.g., signal section 74) which is a signal section of contact signal section 200 excluding the first signal section, and performs encoding at a first bit rate (high quality encoding) for the first signal section, and performs encoding at a bit rate lower than the first bit rate (low quality encoding) for the second signal section.

[0195] The encoding method as this embodiment can also achieve the same effects and advantages as the encoding device 2 as the above embodiment.

[0196] Here, the functions of the decoding device 3 and the encoding device 2 described above can be realized as software processing by, for example, a CPU, a DSP, etc. The software processing is executed based on a program.

[0197] A first program as an embodiment is a program that causes an information processing device to execute a function of decoding first encoded data obtained by encoding, at a first bit rate, a first signal section, which is a part of a contact signal section that is a signal section that indicates a contact state with an object in a tactile signal and includes a boundary between a contact state and a non-contact state with the object, and a function of decoding second encoded data obtained by encoding, at a bit rate lower than the first bit rate, a second signal section, which is a signal section of the contact signal section excluding the first signal section. The first program described above can realize the decoding device 3 according to the embodiment.

[0198] A second program as an embodiment is a program that causes an information processing device to execute a function of determining a first signal section, which is a part of a contact signal section that is a signal section in a tactile signal that indicates a contact state with an object and that includes the boundary between a contact state and a non-contact state with the object, and a second signal section, which is a signal section of the contact signal section excluding the first signal section; a function of encoding the first signal section at a first bit rate; and a function of encoding the second signal section at a bit rate lower than the first bit rate. The encoding device 2 according to the above embodiment can be realized by the second program.

[0199] The above-mentioned first and second programs can be recorded in advance on a recording medium built into a device such as a computer device, or on a ROM or the like in a microcomputer having a CPU. Alternatively, the software may be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magnet optical) disk, a DVD, a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, a memory card, etc. Such removable recording media may be provided as a so-called package software. The first and second programs can be installed into a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as the Internet or a LAN (Local Area Network).

[0200] In addition, the first and second programs are suitable for providing the decoding device and encoding device of the embodiment in a wide range. For example, by downloading the programs to a personal computer, a portable information processing device, a mobile phone, a game device, an AV (Audio Visual) device, etc., the personal computer can function as the decoding device or encoding device of the present technology.

[0201] <11. This Technology> The present technology can also be configured as follows. (1) a first decoding unit that decodes first encoded data obtained by encoding, at a first bit rate, a first signal section, which is a part of a contact signal section that is a signal section in a haptic signal that indicates a contact state with an object, and which is a signal section that includes a boundary between a contact state and a non-contact state with the object; a second decoding unit that decodes second encoded data obtained by encoding a second signal section, which is a signal section excluding the first signal section, of the contact signal section at a bit rate lower than the first bit rate. Decryption device. (2) a combining unit that combines first waveform data obtained by decoding the first encoded data with second waveform data obtained by decoding the second encoded data. The decoding device according to (1) above. (3) The combining unit performs a cross-fade process on a combined portion of the first waveform data and the second waveform data. The decoding device according to (2) above. (4) decoding first coded data obtained by encoding, at a first bit rate, a first signal section which is a part of a contact signal section which is a signal section in a haptic signal that indicates a contact state with an object and which includes a boundary between a contact state and a non-contact state with the object; A second signal section, which is a signal section of the contact signal section excluding the first signal section, is encoded at a bit rate lower than the first bit rate to obtain second encoded data, which is decoded. Decryption method. (5) a function of decoding first coded data obtained by coding, at a first bit rate, a first signal section, which is a part of a contact signal section that is a signal section in a haptic signal that indicates a contact state with an object and includes a boundary between a contact state and a non-contact state with the object; and decoding second encoded data obtained by encoding a second signal section, which is a signal section excluding the first signal section, at a bit rate lower than the first bit rate, in the contact signal section. program. (6) a determination unit that determines a first signal section, which is a part of a contact signal section that is a signal section in a haptic signal indicating a contact state with an object and that includes a boundary between a contact state with the object and a non-contact state, and a second signal section, which is a signal section of the contact signal section excluding the first signal section; a first encoding unit that performs encoding at a first bit rate on the first signal section; a second encoding unit that performs encoding on the second signal section at a bit rate lower than the first bit rate. Encoding device. (7) The determination unit determines the first signal section and the second signal section based on division information of the first signal section and the second signal section added to the haptic signal. The encoding device according to (6) above. (8) The determination unit determines the first signal section and the second signal section based on a result of a waveform analysis of the haptic signal. An encoding device according to (6) or (7) above. (9) The determination unit determines the first signal section and the second signal section based on an amplitude change rate of the haptic signal. An encoding device according to any one of (6) to (8) above. (10) The second encoding unit performs encoding for the second signal section using an encoding method having a longer transform length than encoding for the first signal section. An encoding device according to any one of (6) to (9) above. (11) The second encoding unit performs parametric encoding on the second signal section. An encoding device according to any one of (6) to (10) above. (12) a first buffer memory used to determine the first signal section and the second signal section; a second buffer memory used to encode the second signal section according to the result of the determination. The encoding device according to (10) above. (13) A first signal section is a part of a contact signal section which is a signal section in a haptic signal that indicates a contact state with an object and includes a boundary between a contact state and a non-contact state with the object, and a second signal section is a signal section of the contact signal section excluding the first signal section, and the first signal section is encoded at a first bit rate, and the second signal section is encoded at a bit rate lower than the first bit rate. Encoding method. (14) a function of determining a first signal section, which is a part of a contact signal section that is a signal section in a haptic signal that indicates a contact state with an object and that includes a boundary between a contact state with the object and a non-contact state, and a second signal section, which is a signal section of the contact signal section excluding the first signal section; a function of encoding the first signal section at a first bit rate; and a function of encoding the second signal section at a bit rate lower than the first bit rate. program.

[0202] Finally, the effects described in this disclosure are exemplary and not limiting, and other effects may be achieved, or only a portion of the effects described in this disclosure may be achieved. In addition, the embodiment described in this disclosure is merely an example, and the present technology is not limited to the above-mentioned embodiment. Therefore, even if it is not the above-mentioned embodiment, various modifications are possible according to the design, etc., as long as it does not deviate from the technical idea of ​​the present technology. It is not necessarily true that all combinations of the configurations described in the embodiment are essential to solving the problem. [Explanation of symbols]

[0203] 1. Tactile reproduction system 2 Encoding device 3. Decryption Device 24 Encoding section 33 Joint part 34 Decoding section 34a First Decoding Unit 34b Second Decoding Unit 82 Judgment section 83 1st encoding section 84 Second encoding section 86 First buffer processing section 87 Second buffer processing section 200 Contact signal section

Claims

1. a first decoding unit that decodes first encoded data obtained by encoding, at a first bit rate, a haptic signal in a first signal section, the first signal section being a part of a contact signal section that is a signal section in the haptic signal that indicates a contact state with an object and that includes a boundary between a contact state and a non-contact state with the object; a second decoding unit that decodes second encoded data obtained by encoding a haptic signal in a second signal section, which is a signal section excluding the first signal section, at a bit rate lower than the first bit rate. Decryption device.

2. a combining unit that combines first waveform data obtained by decoding the first encoded data with second waveform data obtained by decoding the second encoded data. The decoding device according to claim 1 .

3. The combining unit performs a cross-fade process on a combined portion of the first waveform data and the second waveform data. The decoding device according to claim 2.

4. decoding first encoded data obtained by encoding, at a first bit rate, a haptic signal in a first signal section, which is a part of a contact signal section that is a signal section in the haptic signal that indicates a contact state with an object and includes a boundary between a contact state and a non-contact state with the object; and decoding second encoded data obtained by encoding the haptic signal in a second signal section, which is a signal section of the contact signal section excluding the first signal section, at a bit rate lower than the first bit rate. Decryption method.

5. a function of decoding first encoded data obtained by encoding, at a first bit rate, a haptic signal in a first signal section, the first signal section being a part of a contact signal section that is a signal section in the haptic signal that indicates a contact state with an object and that includes a boundary between a contact state and a non-contact state with the object; and decoding second encoded data obtained by encoding the haptic signal in a second signal section, which is a signal section excluding the first signal section among the contact signal sections, at a bit rate lower than the first bit rate. program.

6. a determination unit that determines a first signal section, which is a part of a contact signal section that is a signal section in a haptic signal indicating a contact state with an object and that includes a boundary between a contact state with the object and a non-contact state, and a second signal section, which is a signal section of the contact signal section excluding the first signal section; a first encoding unit that encodes the haptic signal in the first signal section at a first bit rate; a second encoding unit that encodes the haptic signal in the second signal section at a bit rate lower than the first bit rate. Encoding device.

7. The determination unit determines the first signal section and the second signal section based on division information of the first signal section and the second signal section added to a haptic signal. The encoding device according to claim 6.

8. The determination unit determines the first signal section and the second signal section based on a result of a waveform analysis of a haptic signal. The encoding device according to claim 6.

9. The determination unit determines the first signal section and the second signal section based on an amplitude change rate of a haptic signal. The encoding device according to claim 6.

10. The second encoding unit performs encoding for the second signal section using an encoding method having a longer transform length than encoding for the first signal section. The encoding device according to claim 6.

11. The second encoding unit performs parametric encoding on the second signal section. The encoding device according to claim 6.

12. a first buffer memory used to determine the first signal section and the second signal section; a second buffer memory used to encode the second signal section according to the result of the determination. The encoding device according to claim 10.

13. A first signal section is a part of a contact signal section which is a signal section in a haptic signal indicating a contact state with an object and which includes a boundary between a contact state with the object and a non-contact state, and a second signal section is a signal section of the contact signal section excluding the first signal section, and the haptic signal in the first signal section is encoded at a first bit rate, and the haptic signal in the second signal section is encoded at a bit rate lower than the first bit rate. Encoding method.

14. a function of determining a first signal section, which is a part of a contact signal section that is a signal section in a haptic signal indicating a contact state with an object and includes a boundary between a contact state with the object and a non-contact state, and a second signal section, which is a signal section of the contact signal section excluding the first signal section; a function of encoding the haptic signal in the first signal section at a first bit rate; and a function of encoding the haptic signal in the second signal section at a bit rate lower than the first bit rate. program.

Citation Information

Patent Citations

  • Contact presenting device and method

    JP2008123431A

  • Method and system for coding and streaming tactile sense data

    JP2014239430A

  • Vibration signal creation program, vibration signal creation system, vibration signal creation device, vibration signal creation method and data output program

    JP2016202486A

  • Tactile sensation reproduction method, device, computer program, and recording medium having a computer program recorded thereon

    WO2011027535A1