Information processing device, information processing method, server device, and program

Abstract

The present technology pertains to an information processing device, an information processing method, a server device, and a program, with which an existing data format can be used to present tactile sensation information. In the present technology a smartphone is provided with a data processing unit for supplying a tactile sensation control signal to a tactile sensation presentation device on the basis of tactile sensation control information embedded in image data in a prescribed file format. For example, the present technology can be applied to a terminal device having a display that displays an image and on which a user carries out a touch operation.

Description

[Technical field]

[0001] The present technology relates to an information processing device, an information processing method, a server device, and a program, and in particular to an information processing device, an information processing method, a server device, and a program that are capable of presenting tactile information by utilizing an existing data format. [Background technology]

[0002] Conventionally, various techniques have been proposed for presenting tactile stimuli such as vibrations to a user. For example, a technique has been disclosed in which a vibration waveform is generated based on features extracted from a stereo audio signal, and a tactile stimulus is presented (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-64264 A Summary of the Invention [Problem to be solved by the invention]

[0004] There are also cases where it is desired to present tactile information corresponding to the texture of an image, for example, to a terminal from a sending side such as a server, rather than generating the information on the terminal itself.

[0005] However, if a new data format including tactile information is defined, the number of devices that can support it will be limited.

[0006] The present technology has been made in view of these circumstances, and makes it possible to present tactile information by utilizing an existing data format. [Means for solving the problem]

[0007] The information processing device according to the first aspect of the present technology includes a data processing unit that supplies a haptic control signal to a haptic presentation device based on haptic control information embedded in image data in a predetermined file format. The haptic control information includes haptic identification information that identifies the haptic control signal, and the predetermined file format is a file format having an alpha channel, the haptic identification information is stored in the alpha channel, and a flag indicating whether the haptic identification information or transparency is stored in the alpha channel is included. .

[0008] The information processing method according to the first aspect of the present technology includes: an information processing device providing a haptic control signal to a haptic presentation device based on haptic control information embedded in image data in a predetermined file format. the haptic control information includes haptic identification information for identifying the haptic control signal, the predetermined file format is a file format having an alpha channel, the haptic identification information is stored in the alpha channel, and a flag indicating whether the haptic identification information or transparency is stored in the alpha channel is included. .

[0009] A program according to a first aspect of the present technology causes a computer to function as a data processing unit that supplies a haptic control signal to a haptic presentation device based on haptic control information embedded in image data in a predetermined file format. The haptic control information includes haptic identification information that identifies the haptic control signal, and the predetermined file format is a file format having an alpha channel, the haptic identification information is stored in the alpha channel, and a flag indicating whether the haptic identification information or transparency is stored in the alpha channel is included. It is something.

[0010] In a first aspect of the present technology, a haptic control signal is supplied to a haptic presentation device based on haptic control information embedded in image data in a predetermined file format. The tactile control information includes tactile identification information that identifies the tactile control signal, and the specified file format is a file format having an alpha channel, in which the tactile identification information is stored, and which has a flag indicating whether the tactile identification information or transparency is stored in the alpha channel.

[0011] A server device according to a second aspect of the present technology includes a storage unit configured to store image data in a predetermined file format in which haptic control information is embedded, and a communication unit configured to transmit the image data to a predetermined information processing device in response to a request from the predetermined information processing device, and the haptic control information is Haptic Control Signal The predetermined file format includes tactile identification information that identifies the tactile identification information, and the predetermined file format is a file format having an alpha channel, in which the tactile identification information is stored, and a flag indicating whether the tactile identification information or transparency is stored in the alpha channel.

[0012] In a second aspect of the present technology, image data in a predetermined file format in which haptic control information is embedded is stored, and the image data is transmitted to a predetermined information processing device in response to a request from the predetermined information processing device. Haptic Control SignalThe predetermined file format includes tactile identification information that identifies the tactile identification information, and the predetermined file format is a file format having an alpha channel, in which the tactile identification information is stored, and a flag indicating whether the tactile identification information or transparency is stored in the alpha channel.

[0013] The program can be provided by transmitting it via a transmission medium or by recording it on a recording medium.

[0014] The information processing device and the server device may be independent devices, or may be internal blocks constituting a single device. [Brief description of the drawings]

[0015] [Figure 1] 1 is a block diagram showing an example of the configuration of a first embodiment of an image processing system to which the present technology is applied. [Diagram 2] FIG. 13 is a diagram showing an example of image data with tactile information. [Diagram 3] FIG. 1 is a cross-sectional view showing a tactile presentation mechanism of a smartphone. [Figure 4] FIG. 1 is a diagram illustrating an operation of a smartphone. [Diagram 5] FIG. 2 is a functional block diagram of a smartphone. [Figure 6] 10 is a diagram illustrating a file format of image data in which haptic data is embedded. FIG. [Figure 7] 1A and 1B are diagrams illustrating control of a tactile presentation device based on tactile data. [Figure 8] 1A and 1B are diagrams illustrating control of a tactile presentation device based on tactile data. [Figure 9] 1A and 1B are diagrams illustrating control of a tactile presentation device based on tactile data. [Figure 10] FIG. 13 is a diagram illustrating a process when an image is enlarged. [Figure 11] 13 is a flowchart illustrating a haptic control process. [Figure 12] FIG. 13 is a diagram showing a modified example of the tactile presentation device. [Figure 13] FIG. 13 is a diagram showing yet another modified example of the tactile presentation device. [Figure 14] FIG. 11 is a diagram illustrating a configuration example of a second embodiment of an image processing system to which the present technology is applied. [Figure 15] FIG. 11 is a functional block diagram of a second embodiment. [Figure 16] FIG. 2 is a block diagram showing an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, a description will be given of an embodiment of the present technology with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and duplicated description will be omitted. The description will be given in the following order. 1. Example of configuration of the first embodiment of the image processing system 2.Function block diagram of smartphone 3. File format of image data with tactile information 4. Control of the tactile presentation device 5. Flowchart of haptic control process 6. Variations 7. Example of configuration of the second embodiment of the image processing system 8. Computer configuration examples

[0017] <1. Configuration Example of First Embodiment of Image Processing System> FIG. 1 shows an example of the configuration of a first embodiment of an image processing system to which the present technology is applied.

[0018] The image processing system 1 shown in FIG. 1 includes a smartphone 11 and a server device 12, and the smartphone 11 and the server device 12 are connected to each other via a network 13.

[0019] The smartphone 11 is a terminal device (information processing device) operated by a user. Based on the user's operation, the smartphone 11 accesses the server device 12 via the network 13, acquires image data stored in the server device 12, and displays the image on its display. The image displayed on the display may be either a moving image or a still image.

[0020] Server device 12 has a storage unit 14 and a communication unit 15. Storage unit 14 stores image data of one or more moving images and image data of one or more still images. In response to a request from smartphone 11, communication unit 15 transmits image data of the moving images or still images stored in storage unit 14 to smartphone 11. Note that server device 12 may store only either image data of moving images or image data of still images.

[0021] The network 13 is any communication network, and may be a wired communication network, a wireless communication network, or both. The network 13 may be configured with one communication network, or may be configured with multiple communication networks. The network 13 may include, for example, the Internet, a public telephone line network, a wide area communication network for wireless mobile devices such as a so-called 4G line or 5G line, a Wide Area Network (WAN), or a Local Area Network (LAN). The network 13 may also include a dedicated line network such as an Internet Protocol-Virtual Private Network (IP-VPN). The network 13 may also include a communication network or communication path of any communication standard, such as a wireless communication network that performs communication in accordance with the Bluetooth (registered trademark) standard, a communication path for short-distance wireless communication such as NFC (Near Field Communication), a communication path for infrared communication, a wired communication network in accordance with standards such as HDMI (registered trademark) (High-Definition Multimedia Interface) or USB (Universal Serial Bus), etc.

[0022] 1 is configured to focus on one user, and therefore, only one smartphone 11 and one server device 12 are connected via the network 13, but the image processing system 1 may include a plurality of smartphones 11. The server device 12 may be configured to be distributed to two or more server devices according to the type of image data to be stored or for each connected device. The server device 12 may be any device that has a function of storing image data and a communication function and can realize a server function, and may be, for example, a personal computer, a smartphone, or the like.

[0023] Haptic control information is embedded in image data of moving images and image data of still images stored in the server device 12. Hereinafter, image data in a predetermined file format in which haptic control information is embedded and stored in the server device 12 will be referred to as image data with haptic information.

[0024] FIG. 2 shows an example of image data with tactile information.

[0025] In the image data with tactile information supplied from the server device 12, for example, as shown in FIG. 2, RGB texture information (color information) and tactile data A as tactile control information are stored for each pixel. The tactile data A represents a tactile ID (tactile identification information) that identifies a tactile control signal. For example, when the tactile data A stored in a certain pixel is 0 (A=0), it represents that a tactile control signal with tactile ID=0 is used. Also, for example, when the tactile data A stored in a certain pixel is 1 (A=1), it represents that a tactile control signal with tactile ID=1 is used. The tactile control signal is, for example, a signal that drives a tactile presentation device such as a piezoelectric element, and corresponds to a vibration waveform for a predetermined period of time.

[0026] FIG. 3 is a cross-sectional view showing a tactile sensation providing mechanism of the smartphone 11. As shown in FIG.

[0027] The smartphone 11 includes a touch sensor-equipped display 22 (hereinafter also simply referred to as a display) disposed on a predetermined surface of a main body (housing) 21, and a piezoelectric actuator 23 attached to a rear surface opposite to a user operation surface of the display 22. The haptic control signal is, for example, a signal that drives the piezoelectric actuator 23 as a haptic presentation device.

[0028] The smartphone 11 displays an image on the display 22 based on the texture information of the image data with tactile information, and vibrates the display 22 by supplying a tactile control signal corresponding to the tactile data A to the piezoelectric actuator 23 depending on the position (pixel position) of the image touched by the user.

[0029] As a result, as shown in FIG. 4, when a user touches or traces a specific position of an image displayed on display 22, the user can feel the tactile sensation, such as the texture of the image displayed on display 22.

[0030] The tactile sensation generating method for generating a tactile sensation may be a method for generating a tactile sensation by vibration, such as the above-mentioned piezoelectric actuator 23, or a method for generating a tactile sensation by electrical stimulation.

[0031] <2. Smartphone functional block diagram> FIG. 5 is a functional block diagram of the smartphone 11 relating to haptic control for displaying an image on the display 22 and allowing the user to perceive the tactile sensation of the image.

[0032] The smartphone 11 includes a communication unit 41, a data processing unit 42, a storage unit 43, a sensor unit 44, an image display unit 45, and a tactile presentation unit 46. The communication unit 41, the data processing unit 42, the storage unit 43, the sensor unit 44, the image display unit 45, and the tactile presentation unit 46 are connected to each other via a predetermined bus 47.

[0033] The communication unit 41 is configured, for example, by a network interface connectable to the network 13 , and communicates with the server device 12 .

[0034] The communication unit 41 connects to the server device 12 via the network 13 under the control of the data processing unit 42, and acquires (receives) image data in a predetermined file format in which haptic data is embedded. The acquired image data is supplied to the data processing unit 42 and / or the storage unit 43.

[0035] Furthermore, under the control of the data processing unit 42, the communication unit 41 can also transmit the image data with tactile information stored in the storage unit 43 to the server device 12 via the network 13.

[0036] A user can upload image data with tactile information that has been pre-stored in the user's smartphone 11, image data with tactile information obtained from another device (another user), etc., to the server device 12. If the smartphone 11 has a function for creating image data with tactile information, the user may upload image data with tactile information that the user has created. The server device 12 receives image data with tactile information transmitted from each of a plurality of devices including the smartphone 11, and stores the image data in the storage unit 14 (FIG. 1).

[0037] The data processing unit 42 is composed of an arithmetic processing device and a data storage device, such as a Central Processing Unit (CPU), a Read Only Memory (ROM), and a Random Access Memory (RAM).

[0038] The data processing unit 42 supplies a tactile control signal to the tactile presentation unit 46 based on the tactile data embedded in the image data with tactile information in a predetermined file format. More specifically, the data processing unit 42 acquires from the storage unit 43 a tactile control signal based on the tactile data A corresponding to the user's touch position on the image displayed based on the image data with tactile information, and supplies the signal to the tactile presentation unit 46. The user's touch position on the image is supplied from the sensor unit 44 as the touch position on the image display unit 45 (display 22).

[0039] The storage unit 43 is configured with a storage medium such as a hard disk or a non-volatile memory.

[0040] The storage unit 43 stores the image data with tactile information acquired from the server device 12. Furthermore, the storage unit 43 may store image data with tactile information in advance, in addition to the data acquired from the server device 12.

[0041] Furthermore, storage unit 43 pre-stores a haptic control signal corresponding to the haptic ID embedded in the haptic information-attached image data as haptic data A. The haptic control signal corresponding to the haptic data A (tactile ID) included in the haptic information-attached image data is stored in server device 12, and is acquired in advance when acquiring the image data with haptic information, or at a timing separate from the timing of acquiring the image data with haptic information, and stored in storage unit 43. Alternatively, the haptic control signal corresponding to the haptic data A (tactile ID) may be stored in a server device other than server device 12.

[0042] The sensor unit 44 senses the user's touch position on the image displayed on the image display unit 45. The sensor unit 44 is configured, for example, with a touch sensor superimposed on the display 22. The sensor unit 44 senses the user's touch position on the display 22 and supplies the sensing result to the data processing unit 42.

[0043] The image display unit 45 is configured by, for example, a liquid crystal display (LCD), an OLED (Organic Light Emitting Diode) display, etc. The image display unit 45 displays an image corresponding to the image data supplied from the data processing unit .

[0044] The tactile sense providing unit 46 is configured by, for example, a vibration generating device of the piezo actuator 23 including a piezo element. The tactile sense providing unit 46 is a tactile sense providing device that provides a tactile sense to the user.

[0045] The touch sensor display 22 and the piezoelectric actuator 23 shown in FIG. 3 correspond to the sensor unit 44, the image display unit 45, and the tactile sense providing unit 46 in FIG.

[0046] <3. File format of image data with tactile information> FIG. 6 shows an example of a file format for image data with embedded haptic data.

[0047] The image data with tactile information can be transmitted, for example, using a PNG (Portable Network Graphics) file format.

[0048] In the PNG file format, a parameter called an alpha channel that specifies transparency can be stored for each pixel. This alpha channel area is used as a parameter to store haptic data A, not transparency. Since transparency is not necessarily a parameter that is used, this parameter can be used to transmit haptic data A.

[0049] Specifically, as shown in A of Fig. 6, a PNG file is composed of a header section 61 and a data section 62, and RGB texture information (color information) and haptic data A are stored in the data section 62 on a pixel-by-pixel basis. The haptic data A is stored in the alpha channel area. The alpha channel is not limited to the binary values ​​of "0" or "1" but can be defined in 8 bits (256 gradations). By embedding the haptic data A as haptic control information for controlling a haptic presentation device for each pixel, it is possible to reproduce delicate tactile sensations.

[0050] The header section 61 can store a haptic data flag HAP_FLG, which indicates whether the data stored in the alpha channel area is transparency or haptic data, as metadata describing information about the data. For example, when HAP_FLG=1, it indicates that the data stored in the alpha channel area is haptic data, and when HAP_FLG=0, it indicates that the data stored in the alpha channel area is transparency. This allows the playback application (program) that has acquired the file to determine whether to process the alpha channel as transparency or haptic data.

[0051] 6B, a haptic usage flag AUSE_FLG that specifies whether or not to use the data stored in the alpha channel area may be provided in the header section 61. In this way, if the playback application that has acquired the file does not support control using haptic data, the haptic usage flag AUSE_FLG can be set to prevent the haptic data stored in the alpha channel from being used.

[0052] Furthermore, as shown in C of FIG. 6, the header section 61 may store both a haptic data flag HAP_FLG indicating either transparency or haptic data, and a haptic usage flag AUSE_FLG specifying whether the data stored in the alpha channel area is to be used.

[0053] In the above explanation, PNG has been used as an example of a file format having an alpha channel, but the alpha channel area can also be used as an area for storing haptic data in file formats having an alpha channel other than PNG, such as AVI (Audio Video Interleave) and QuickTime. By adopting an existing file format having an alpha channel as the file format for transmitting image data with haptic information, it is not necessary to prepare a special data format or develop a new data format, and haptic information can be presented using an existing data format. Using the mechanism of an existing file format facilitates data exchange between devices, such as distribution from server device 12.

[0054] It should be noted that the present technology is not limited to a file format having an alpha channel as the file format of image data with haptic information. Even in a file format having an alpha channel, for example, as shown in D of FIG. 6, haptic data A may not be stored in the alpha channel area, but may be stored as haptic data (HAP_DATA) 64 for all pixels by adding it to the end of pixel data 63 stored in an existing format in the data section 62. Alternatively, the haptic data 64 may be placed before the pixel data 63 stored in an existing format in the data section 62, or before the header section 61. Furthermore, the haptic data may be stored in an area where an application can freely write (free writing area), such as an application extension area of ​​the header section 61. It goes without saying that the haptic data storage method described with reference to D of FIG. 6 can also be applied to a file in a file format that does not have an alpha channel.

[0055] In the above example, the tactile data A is described as being stored in pixel units, but the tactile data A may be stored in units of multiple pixels in both the horizontal and vertical directions (for each multiple pixels) rather than in units of one pixel. When a pixel that does not have tactile data A is touched by the user, the data processing unit 42 can supply a tactile control signal to the tactile presentation unit 46 using the tactile data A of one or more pixels that have tactile data A around the touch position.

[0056] Furthermore, when the image data with haptic information is image data of a moving image, the haptic data A may be stored in units of frames, or in units of multiple frames. In this case, the haptic data A of an image in which no haptic data A is stored can use the haptic data A of an adjacent image.

[0057] Furthermore, the haptic data A may be stored in units of multiple frames, and the image in which the haptic data A is stored may also be stored in units of multiple pixels.

[0058] In the above example, a haptic ID that identifies a haptic control signal is embedded in the alpha channel of each pixel, but the haptic control signal itself may also be embedded. The haptic control signal embedded in the alpha channel may not be a waveform with a time width as shown in Figure 2, but may be an instantaneous value, for example, data such as a haptic stimulus value that quantifies the intensity of a haptic stimulus. Furthermore, a haptic ID may be embedded in the alpha channel of each pixel, and the haptic control signal may be embedded in another location, such as the header section 61 or the haptic data 64 following the pixel data 63 in D of Figure 6.

[0059] The haptic data A (haptic ID) embedded in the α channel of each pixel is not limited to one, and multiple haptic data A (haptic IDs) may be embedded. For example, it is possible to embed three haptic data A1 to A3 corresponding to three vibration modes, strong, medium, and weak, in the α channel, obtain a haptic control signal of one of the haptic data A1 to A3 according to the vibration mode set by the user, and supply it to the haptic presentation device.

[0060] <4. Control of tactile presentation device> Control of the tactile presentation device based on the tactile data A will be described with reference to FIGS.

[0061] 7 and 8 show an example of control of the tactile provision device by the data processing unit 42 when the user performs an operation of tracing (sliding) an image displayed on the display 22. FIG.

[0062] FIG. 7A shows how the user performs an operation of tracing an image displayed on display 22, and tactile data A (tactile ID) of the pixel touched by the user changes from "0" to "1."

[0063] As the most basic control, when the tactile data A (tactile ID) of the pixel touched by the user changes from "0" to "1", the data processing unit 42 can perform control to switch the tactile control signal corresponding to the tactile ID at the moment the tactile ID changes, as shown in B of Figure 7.

[0064] Also, for example, as shown in C of Fig. 7, data processing unit 42 can control the output level (amplitude) of the head of the next tactile control signal (of the tactile ID) in accordance with the finger movement speed at the moment of switching, thereby controlling the switching of the tactile control signal corresponding to the tactile ID. Specifically, data processing unit 42 increases the output level of the head of the tactile control signal when tactile data A (tactile ID) changes from "0" to "1" while the finger movement speed changes quickly, and decreases the output level of the head of the tactile control signal when tactile data A (tactile ID) changes from "0" to "1" while the finger movement speed changes slowly. C of Fig. 7 shows a graph of an example in which the output level is changed significantly.

[0065] When the moving speed of the finger is high, the time for touching the pixel is short. Therefore, according to the moving speed, the pitch (reproduction speed) of the output level of the tactile control signal may be controlled. Specifically, when the moving speed of the finger is high, the data processing unit 42 increases the pitch (reproduction speed) of the output level of the tactile control signal. On the other hand, when the moving speed of the finger is low, the data processing unit 42 decreases the pitch (reproduction speed) of the output level of the tactile control signal. The control of the pitch (reproduction speed) will also be described later in B of FIG. 8.

[0066] When the moving speed of the finger is high, if the output level (amplitude) of the tactile control signal is decreased according to the moving speed, it may fall below the user's perception level and may not provide sufficient tactile sensation. Therefore, a lower limit value can be provided for the output level of the tactile control signal.

[0067] Also, for example, as shown in D of FIG. 7, the data processing unit 42 can perform control to switch the tactile control signal by fading out the tactile control signal before the change from "0" to "1" and fading in the tactile control signal after the change.

[0068] A of FIG. 8 shows a state where the user traces an image displayed on the display 22 and the tactile data A (tactile ID) of the pixel touched by the user remains unchanged at "1".

[0069] In this case, for example, the data processing unit 42 can perform control to switch the tactile control signal by controlling the output level (amplitude) and pitch (reproduction speed) at the beginning of the next tactile control signal according to the moving speed of the finger when crossing the pixel.

[0070] For example, as shown in B of FIG. 8, it is assumed that an operation of tracing five pixels where the tactile data A (tactile ID) is "1" in order is detected, and the speeds when crossing the pixels are in the order of V1, V2, V3, V2, V1 (V1 < V2 < V3).

[0071] When the moving speed of the finger when crossing a pixel is high, the data processing unit 42 increases the output level (amplitude) of the tactile control signal and raises the pitch (reproduction speed). On the other hand, when the moving speed of the finger when crossing a pixel is low, the data processing unit 42 decreases the output level (amplitude) of the tactile control signal and lowers the pitch (reproduction speed).

[0072] In the example of the tactile control signal in B of FIG. 8, the section of one tactile control signal corresponding to the pitch is T1, T2, T3, T2, and T1 (T3 < T2 < T1) corresponding to the speeds V1, V2, V3, V2, and V1, respectively. Also, the output level (amplitude) is also speed AMP1, AMP2, AMP3, AMP2, and AMP1 (AMP1 < AMP2 < AMP3) corresponding to the speeds V1, V2, V3, V2, and V1, respectively.

[0073] The operation example shown in FIG. 7 was an example in which the user traced the image displayed on the display 22, that is, the tactile data A (tactile ID) of the pixel changed by the active operation of the user. However, the tactile data A (tactile ID) of the pixel may be changed passively. Specifically, when the image data with tactile information is a moving image, even when the user touches the same pixel position without moving the finger, the image may be changed and the tactile data A (tactile ID) stored in the pixel at the touch position may change.

[0074] FIG. 9A shows how the tactile data A (tactile ID) stored in the pixel at the touch position changes from "0" to "1" when the image is changed.

[0075] In FIGS. 7A and 8A, the horizontal axis direction represents the X position on the display 22, but in FIG. 9A, the horizontal axis direction represents the time direction. When the frame rate is 30 fps, the image is updated approximately every 33.3 msec.

[0076] When the image is changed and the tactile data A (tactile ID) stored in the pixel at the touch position changes from "0" to "1," the most basic control is that the data processing unit 42 can perform control to switch the tactile control signal corresponding to the tactile ID at the moment the tactile ID changes, as shown in B of Figure 9.

[0077] Alternatively, the data processing unit 42 can perform control to switch the haptic control signal by fading out the haptic control signal before changing from "0" to "1" while fading in the haptic control signal after changing, as shown in C of Figure 9.

[0078] FIG. 10 shows the processing of haptic data A (haptic ID) when an image displayed on the display 22 is enlarged by the user performing a pinch-out operation on the image displayed on the display 22, for example.

[0079] When an image is enlarged, the haptic data A for each pixel after enlargement can be generated using standard image enlargement processing of the drawing application as is, just like the alpha channel. When a new pixel is generated by interpolation between a pixel with haptic data A="0" and a pixel with haptic data A="1" due to image enlargement, the haptic data A of the pixel generated by interpolation is generated by interpolation using haptic data A="0" and haptic data A="1", for example, as haptic data A="0.5".

[0080] When a pixel with tactile data A="0.5" is touched by a user, the data processing unit 42 generates a tactile control signal that superimposes a tactile control signal with tactile ID="0" and a tactile control signal with tactile ID="1" as a tactile control signal for the pixel with tactile data A="0.5" and supplies it to the tactile presentation device.

[0081] Furthermore, when an image is reduced by a user's pinch-in operation or the like, the haptic data A for each reduced pixel can be generated using standard image reduction processing of a drawing application as is, similar to the alpha channel. For example, when multiple pixels are reduced to one pixel, thinned-out pixels can be omitted, or the average value of the haptic data A of the aggregated multiple pixels can be used as the haptic data A of the aggregated pixel.

[0082] When the tactile data A of a pixel becomes third tactile data A3 between the first tactile data A1 and the second tactile data A2 in response to enlargement or reduction of the image, the data processing unit 42 can generate a signal that superimposes a tactile control signal corresponding to the first tactile data A1 and a tactile control signal corresponding to the second tactile data A2, and supply the signal to the tactile presentation device.

[0083] By using the alpha channel of a pixel as tactile data A for controlling the tactile presentation device, no special processing is required when enlarging or reducing an image; the standard image enlargement processing of a drawing application can be used as is, making it easy to apply.

[0084] <5. Flowchart of haptic control process> 11, a haptic control process for making the user perceive haptic information corresponding to a touch position on an image by the smartphone 11 will be described. This process is started, for example, when the user instructs the smartphone 11 to perform an operation to acquire image data with haptic information stored in the server device 12.

[0085] It is assumed that a haptic control signal corresponding to haptic data A (haptic ID) that can be stored in an image is acquired and stored in the storage unit 43 in advance before the start of the processing in FIG.

[0086] First, in step S1, under the control of the data processing unit 42, the communication unit 41 requests image data with tactile information from the server device 12 and acquires (receives) the image data with tactile information provided by the server device 12. The acquired image data with tactile information is provided to the data processing unit 42. The user's operation instructs which image data with tactile information to acquire. Note that if the user designates image data with tactile information stored in the storage unit 43 as the image to be displayed, in step S1, the image data with tactile information stored in the storage unit 43 is acquired without acquiring the image data with tactile information from the server device 12.

[0087] In step S2, data processing unit 42 analyzes the metadata in the header portion 61 of the image data with haptic information. Specifically, data processing unit 42 references the haptic data flag HAP_FLG stored in the metadata and determines whether haptic data A is stored in the acquired image data with haptic information.

[0088] In step S3, the data processing unit 42 supplies an image signal based on the texture information of the image data with tactile information to the display 22, causing the display 22 to display the image. The display 22 displays the image based on the image signal supplied from the data processing unit 42. The image may be either a moving image or a still image. If the image to be displayed is a moving image, after the start of processing in step S3, one or more images constituting the moving image are displayed on the display 22 sequentially at a predetermined frame rate.

[0089] In step S4, the data processing unit 42 determines whether or not haptic data A is included in the image data with haptic information of the image displayed on the display 22, based on the results of the analysis of the metadata in step S2.

[0090] If it is determined in step S4 that the haptic data A is not included in the image data with haptic information of the image displayed on the display 22, the haptic control process in FIG. 11 ends.

[0091] On the other hand, if it is determined in step S4 that the image data with tactile information of the image displayed on the display 22 contains tactile data A, the process proceeds to step S5.

[0092] In step S5, the data processing unit 42 determines whether a user's touch on the display 22 has been detected based on the sensing result from the sensor unit 44 (touch sensor), and repeats the process of step S5 until it is determined that a touch has been detected.

[0093] Then, if it is determined in step S5 that a user's touch has been detected, the processing proceeds to step S6, where the data processing unit 42 identifies the tactile data A stored in the alpha channel of the pixel at the user's touch position, and obtains from the memory unit 43 a tactile control signal corresponding to the tactile ID indicated by the tactile data A.

[0094] In step S7, the data processing unit 42 modulates a tactile control signal corresponding to a single or multiple tactile IDs based on a user's touch operation, such as a tracing operation on the display 22 or an image zoom operation, and supplies it to the tactile presentation unit 46 (tactile presentation device).

[0095] In step S8, the tactile sensation providing unit 46 provides a tactile sensation based on the tactile control signal supplied from the data processing unit 42. Specifically, the piezo actuator 23 serving as a tactile sensation providing device vibrates the surface of the display 22 that is being touched by the user.

[0096] In reality, the processes in steps S6 to S8 are partly executed in parallel.

[0097] In step S9, the data processing unit 42 determines whether or not to end the haptic control process. For example, when an instruction to end the application executing the haptic control process is received or when an operation to end the display of the image being displayed on the display 22 is received, the data processing unit 42 determines to end the haptic control process.

[0098] If it is determined in step S9 that the haptic control process should not be ended again, the data processing unit 42 returns the process to step S5 and repeatedly executes steps S5 to S9 described above.

[0099] On the other hand, if it is determined in step S9 that the haptic control process is to be ended, the haptic control process of FIG. 11 is ended.

[0100] The haptic control process by the data processing unit 42 of the smartphone 11 is executed as described above.

[0101] In addition, in the above-mentioned tactile control processing, the tactile control signal corresponding to the tactile data A (tactile ID) stored in the alpha channel of each pixel of the image displayed on the display 22 is acquired in advance and stored in the memory unit 43 before the image is displayed.

[0102] However, in the processing of step S1 in which image data with tactile information specified by the user is obtained from server device 12, a tactile control signal corresponding to tactile data A (tactile ID) contained in the image data with tactile information may be obtained from server device 12 at the same time as the image data with tactile information and stored in memory unit 43.

[0103] Alternatively, in cases where there is sufficient communication bandwidth in network 13 and high-speed communication is possible between smartphone 11 and server device 12, it is possible to access server device 12 and obtain a haptic control signal corresponding to the haptic ID of the pixel at the touch position when the pixel at the touch position is identified, without obtaining a haptic control signal in advance.

[0104] Specifically, a method may be adopted in which a URL such as "http: / / tactile.com / ?id=" that is the access point of the server device 12 is stored in the metadata of the header section 61, and if the tactile data A (tactile ID) stored in the alpha channel of the pixel touched by the user is "2", "http: / / tactile.com / ?id=2" is accessed to obtain a tactile control signal corresponding to the tactile ID.

[0105] When the user performs a tracing operation, the data processing unit 42 can predict (read ahead) the destination of the tracing operation and obtain a tactile control signal corresponding to the tactile ID of the predicted pixel from the memory unit 43 or the server device 12.

[0106] If a haptic control signal corresponding to a haptic ID cannot be obtained in real time due to limitations such as communication speed and communication costs (communication time), a haptic control signal obtained earlier and stored in memory unit 43 or a cache may be used.

[0107] In the alpha channel of each pixel, not only haptic data A (haptic ID) for controlling a haptic control device, but also an audio ID (audio control identification information) for controlling an audio device that outputs audio may be embedded. For example, 8-bit alpha channel data may be embedded with 0 to 127 assigned as a haptic ID and 128 to 255 assigned as an audio ID. Alternatively, values ​​corresponding to combinations of haptic IDs and audio IDs may be embedded. For example, data corresponding to predefined combinations can be embedded, such that alpha channel data of "1" represents haptic ID="1" and audio ID="1", and alpha channel data of "5" represents haptic ID="4" and audio ID="1".

[0108] <6. Variations> In the above-described first embodiment, the following modifications are possible.

[0109] <Example of a configuration with a tactile presentation device for each pixel> In the first embodiment described above, as shown in FIG. 3, one piezoelectric actuator 23 as a tactile presentation device is provided for the display 22, and different or the same tactile control signal is supplied to the piezoelectric actuator 23 depending on the position (pixel position) of the image touched by the user, thereby vibrating the display 22 and providing the user with a tactile sensation.

[0110] 12, a device configuration is also possible in which a piezoelectric actuator 23P that generates vibrations is provided for each pixel of the display 22. In this case, the data processing unit 42 can supply a tactile control signal corresponding to the tactile ID of each pixel to the piezoelectric actuator 23P independently for each of the multiple pixels including the touch position, to vibrate the pixels. Of course, the piezoelectric actuator 23P may be provided for each set of multiple pixels, rather than for each pixel.

[0111] <Example of configuration with a tactile presentation device in addition to a display> 13, the smartphone 11 may have a configuration including another piezo actuator 71 in addition to the piezo actuator 23 attached to the back surface of the display 22. The piezo actuator 71 is attached to the main body (housing) 21 of the smartphone 11, for example.

[0112] Generally, a user grasps and holds the main body 21 of the smartphone 11 with one hand, and uses the other hand to perform a touch operation on the display 22. Therefore, when the piezo actuator 71 is attached to the main body 21, the data processing unit 42 can provide a tactile sensation different from the tactile sensation corresponding to the touch operation on the display 22 to the hand grasping the main body 21.

[0113] For example, when the image displayed on the display 22 is a moving image, the piezo actuator 71 of the main body 21 can be caused to provide a tactile sensation in accordance with the sound data (audio data) reproduced together with the moving image.

[0114] As described above, tactile data for providing a tactile sensation according to the touch position when an image displayed on the display 22 is touched is embedded in the alpha channel of each pixel of the data section 62 of the image data of the moving image, or in the free description area of ​​the header section 61. Meanwhile, tactile data for providing a tactile sensation in accordance with audio data (audio data) played together with the moving image can be embedded in the file of the audio data.

[0115] <Touch display function on / off> The data processing unit 42 can turn on or off the haptic presentation function based on the haptic control information embedded in the α channel according to the setting conditions set by a setting unit (not shown). In addition to turning on and off by user designation, an automatic save function can be provided to automatically turn off the function when the communication speed when the communication unit 41 communicates via the network 13 or the remaining battery power of the smartphone 11 falls below a specified reference value. When the haptic presentation function is off, the process of acquiring a haptic control signal corresponding to the haptic ID from the server device 12, the process of supplying the haptic control signal to the haptic presentation device, the process of analyzing metadata of image data with haptic information, and the like are stopped.

[0116] <7. Configuration Example of the Second Embodiment of the Image Processing System> In the first embodiment described above, the sensor unit 44 that senses the user's touch position on an image, the image display unit 45 that displays an image of image data with tactile information, and the tactile presentation unit 46 that presents tactile sensations to the user were all provided in a single information processing device, the smartphone 11.

[0117] However, the sensor unit 44, the image display unit 45, and the tactile sense providing unit 46 do not necessarily need to be integrated, and at least one of them may be configured as a plurality of separate devices.

[0118] FIG. 14 shows an example of the configuration of a second embodiment of an image processing system to which the present technology is applied.

[0119] In the second embodiment, as shown in FIG. 14, a device equivalent to the smartphone 11 in the first embodiment is configured to be separated into three devices: an information processing device 81, a head-mounted display 82, and a glove 83.

[0120] The information processing device 81 is composed of, for example, a personal computer, a smartphone, a dedicated data processing device, etc. The information processing device 81 includes the above-mentioned data processing unit 42, acquires image data with haptic information in a predetermined file format, and analyzes the metadata. The information processing device 81 then supplies a haptic control signal to the glove 83 in response to the user's hand movement transmitted from the head mounted display 82.

[0121] The head mounted display 82 includes an image display unit 45 (not shown in FIG. 14) and a sensor unit 44.

[0122] The image display unit 45 is composed of a liquid crystal display, an OLED display, or the like, and displays an image 84 supplied from the information processing device 81 and presents it to the user. The image 84 in FIG. 14 is an image corresponding to the image data with tactile information, and is an image diagram displayed on the image display unit 45 of the head mounted display 82.

[0123] The sensor unit 44 is composed of, for example, an image sensor that captures an image of a subject in front, a distance measuring sensor (ToF sensor) that measures the distance to a subject in front, etc. The sensor unit 44 senses the movement of the hand of the user wearing the glove 83, and supplies the sensing result to the information processing device 81.

[0124] The glove 83 is a tactile sensation providing device that includes the tactile sensation providing unit 46 and provides a tactile sensation to the user based on a tactile control signal supplied from the information processing device 81.

[0125] The information processing device 81 communicates with the server device 12 and receives image data with tactile information and a tactile control signal corresponding to the tactile ID. The information processing device 81 transmits an image signal of an image 84 corresponding to the image data with tactile information to the head mounted display 82. The head mounted display 82 displays the image 84 on the image display unit 45 based on the received image signal. When the user moves the hand wearing the glove 83 with respect to the image 84 displayed on the head mounted display 82, the movement is detected by the sensor unit 44 and transmitted to the information processing device 81. The information processing device 81 transmits to the glove 83 a tactile control signal corresponding to the tactile data A (tactile ID) of the pixel corresponding to the position of the user's hand in the image 84. The glove 83 presents a tactile sensation to the user based on the tactile control signal transmitted from the information processing device 81.

[0126] The image display unit 45 may be a 3D display that displays an image for the left eye and an image for the right eye, and allows the user to perceive the image three-dimensionally by the parallax between the two images. The head mounted display 82 realizes a world called VR (Virtual Reality) or AR (Augmented Reality).

[0127] There are various formats for image data of 3D objects, including the first format, which transmits two images, one for the left eye and one for the right eye, and a depth image representing the depth direction; the second format, which represents the three-dimensional position of an object as a collection of points (point cloud), and holds color information of the object corresponding to each point; and the third format, which represents the three-dimensional position of an object as vertices called a polygon mesh and the connections between the vertices, and holds color information of the object corresponding to each polygon mesh as a texture image in the UV coordinate system.

[0128] In the first format, for example, similar to a two-dimensional image, tactile control information (e.g., tactile data A) for controlling a tactile presentation device can be embedded in the alpha channel of each pixel of either the left eye or the right eye image.

[0129] In the second and third formats, haptic control information (for example, haptic data A) for controlling a haptic presentation device can be embedded in correspondence with points or polygon meshes that represent the three-dimensional position of an object.

[0130] FIG. 15 illustrates the correspondence between the information processing device 81, the head mounted display 82, and the glove 83 in FIG. 14 and the functional blocks in FIG.

[0131] 15, the sensor unit 44 and the image display unit 45 are configured as one device (head mounted display 82), but the sensor unit 44 and the image display unit 45 may be configured as separate devices. For example, the sensor unit 44 can be an imaging device such as a stereo camera that images a user wearing a glove 83 and the head mounted display 82 from different viewpoints.

[0132] Alternatively, the sensor unit 44 may be integrated with the glove 83 having the tactile sense providing unit 46. Specifically, the sensor unit 44 is configured with an acceleration sensor, a gyro sensor, or the like that estimates the attitude of the glove 83 itself, and can be configured as a part of the glove 83.

[0133] A predetermined bus 47 connecting the information processing device 81, the head mounted display 82, and the glove 83 includes a network. The information processing device 81, the head mounted display 82, and the glove 83 can transmit and receive sensing data, haptic control signals, image signals, and the like, by wireless communication such as Bluetooth (registered trademark) or Wi-fi (registered trademark). This allows the user to feel a greater sense of realism by adding haptic information to the information obtained visually.

[0134] When the image display unit 45 is composed of a device different from the sensor unit 44 and the tactile presentation unit 46 and only needs to have the function of displaying images, the image display unit 45 can be, for example, a display of a television receiver or the like, AR glasses, or the like.

[0135] <8. Computer configuration examples> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a microcomputer built into dedicated hardware, and a general-purpose personal computer, for example, capable of executing various functions by installing various programs.

[0136] FIG. 16 is a block diagram showing an example of the hardware configuration of a computer in the case where the information processing device 81 is configured by a computer.

[0137] In the computer, a central processing unit (CPU) 101, a read only memory (ROM) 102, and a random access memory (RAM) 103 are interconnected via a bus 104.

[0138] An input / output interface 105 is further connected to the bus 104. An input unit 106, an output unit 107, a storage unit 108, a communication unit 109, and a drive 110 are connected to the input / output interface 105.

[0139] The input unit 106 includes a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 107 includes a display, a speaker, an output terminal, etc. The storage unit 108 includes a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 109 includes a network interface, etc. The drive 110 drives a removable recording medium 111 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0140] In the computer configured as above, the CPU 101 performs the above-mentioned series of processes by, for example, loading a program stored in the storage unit 108 into the RAM 103 via the input / output interface 105 and the bus 104 and executing the program. The RAM 103 also stores data and the like necessary for the CPU 101 to execute various processes as appropriate.

[0141] The program executed by the computer (CPU 101) can be provided by being recorded on a removable recording medium 111 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0142] In the computer, the program can be installed in the storage unit 108 via the input / output interface 105 by mounting the removable recording medium 111 in the drive 110. The program can also be received by the communication unit 109 via a wired or wireless transmission medium and installed in the storage unit 108. Alternatively, the program can be pre-installed in the ROM 102 or the storage unit 108.

[0143] In addition, the program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0144] The server device 12 in FIG. 1 can also be configured in the same manner as the example of the hardware configuration in FIG.

[0145] The tactile presentation technology using image data with tactile information of the present disclosure is not limited to the above-described embodiment, but can be applied to any terminal device having a display that displays an image and allows a user to perform a touch operation. For example, the technology can be applied to information terminals installed in stores such as convenience stores, bank ATMs (automatic teller machines), and the like.

[0146] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0147] For example, it is possible to adopt a form in which all or part of the above-described multiple embodiments are combined.

[0148] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0149] The steps described in the above flowchart may of course be performed chronologically in the order described, but they do not necessarily have to be processed chronologically, and may be performed in parallel or at any required timing, such as when a call is made.

[0150] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0151] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices.

[0152] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device in which multiple modules are housed in a single housing, are both systems.

[0153] It should be noted that the effects described in this specification are merely examples and are not limiting, and there may be effects other than those described in this specification.

[0154] The present technology can have the following configurations. (1) A data processing unit that supplies a haptic control signal to the haptic presentation device based on haptic control information embedded in image data in a predetermined file format. An information processing device comprising: (2) The data processing unit supplies the haptic control signal based on the haptic control information corresponding to a touch position of a user on an image displayed based on the image data to the haptic presentation device. The information processing device according to (1). (3) The data processing unit performs a fade-out / fade-in process of the haptic control signal when the haptic control information corresponding to the touch position is changed due to a change in an image or a change in a touch position. The information processing device according to (2). (4) The data processing unit controls at least one of an output level and a pitch of the haptic control signal in accordance with a moving speed of the touch position. The information processing device according to (2) or (3). (5) The haptic control information is haptic identification information that identifies the haptic control signal. The information processing device according to any one of (1) to (4). (6) the predetermined file format is a file format having an alpha channel, The tactile discrimination information is stored in the alpha channel. The information processing device according to (5). (7) a storage unit for storing the tactile control signal corresponding to the tactile identification information; The information processing device according to (5) or (6). (8) A communication unit for communicating with other devices is further provided. The data processing unit acquires the tactile control signal corresponding to the tactile identification information from the other device via the communication unit. The information processing device according to any one of (5) to (7). (9) The haptic control information is stored in units of pixels of the image data. The information processing device according to any one of (1) to (8). (10) When the haptic control information of a pixel becomes a third value between a first value and a second value in response to enlargement or reduction of an image displayed based on the image data, the data processing unit supplies to the haptic presentation device a signal in which the haptic control signal corresponding to the first value and the haptic control signal corresponding to the second value are superimposed. The information processing device according to (9). (11) When the image data in the predetermined file format is image data of a moving image, other haptic control information is also embedded in the audio data of the moving image. The information processing device according to any one of (1) to (10). (12) The haptic control information is stored in units of one or more frames of the image data. The information processing device according to any one of (1) to (11). (13) The haptic control information is stored in units of two or more pixels of the image data. 13. The information processing device according to any one of (1) to (12). (14) the haptic control information is a haptic stimulus value that quantifies an intensity of the haptic stimulus; The data processing unit supplies the haptic control signal corresponding to the haptic stimulus value to the haptic presentation device. The information processing device according to any one of (1) to (13). (15) A sensor unit that senses a touch position of the user; an image display unit that displays an image based on the image data; The tactile presentation device; Further equipped The information processing device according to any one of (2) to (14). (16) A storage unit that stores image data in a predetermined file format in which haptic control information is embedded; a communication unit that transmits the image data to a predetermined information processing device in response to a request from the predetermined information processing device; A server device comprising: (17) the haptic control information is haptic identification information for identifying a haptic control signal to be supplied to a haptic presentation device; The storage unit also stores the tactile control signal corresponding to the tactile identification information; The communication unit transmits the haptic control signal to the predetermined information processing device in response to a request from the predetermined information processing device. The server device according to (16) above. (18) the communication unit receives image data in the predetermined file format transmitted from another information processing device; The storage unit stores the received image data in the predetermined file format. The server device according to (16) or (17). (19) An information processing device, Supplying a haptic control signal to a haptic presentation device based on haptic control information embedded in image data in a predetermined file format. An information processing method comprising: (20) Computer, A data processing unit that supplies a haptic control signal to the haptic presentation device based on haptic control information embedded in image data in a predetermined file format. A program to function as a [Explanation of symbols]

[0155] 1 image processing system, 11 smartphone, 12 server device, 13 network, 14 memory unit, 15 communication unit, 21 main body (housing), 22 display, 23 piezo actuator, 41 communication unit, 42 data processing unit, 43 memory unit, 44 sensor unit, 45 image display unit, 46 tactile presentation unit, 61 header unit, 62 data unit, 63 pixel data, 64 tactile data, 71 piezo actuator, 81 information processing device, 82 head mounted display, 83 glove, 101 CPU, 102 ROM, 103 RAM, 106 input unit, 107 output unit, 108 memory unit, 109 communication unit, 110 drive

Claims

1. A data processing unit that supplies a haptic control signal to the haptic presentation device based on haptic control information embedded in image data in a predetermined file format. Equipped with the haptic control information includes haptic identification information that identifies the haptic control signal; The predetermined file format is a file format having an alpha channel, the tactile identification information is stored in the alpha channel, and a flag indicating whether the tactile identification information or transparency is stored in the alpha channel is included. Information processing device.

2. The data processing unit supplies the haptic control signal based on the haptic control information corresponding to a touch position of a user on an image displayed based on the image data to the haptic presentation device. The information processing device according to claim 1 .

3. The data processing unit performs a fade-out / fade-in process of the haptic control signal when the haptic control information corresponding to the touch position is changed due to a change in an image or a change in a touch position. The information processing device according to claim 2 .

4. The data processing unit controls at least one of an output level and a pitch of the haptic control signal in accordance with a moving speed of the touch position. The information processing device according to claim 2 .

5. the tactile control information includes a plurality of the tactile identification information; Depending on the set mode, one of the plurality of pieces of tactile identification information is selected. The information processing device according to claim 1 .

6. a storage unit for storing the tactile control signal corresponding to the tactile identification information; The information processing device according to claim 1 .

7. A communication unit for communicating with other devices is further provided. The data processing unit acquires the tactile control signal corresponding to the tactile identification information from the other device via the communication unit. The information processing device according to claim 1 .

8. The haptic control information is stored in units of pixels of the image data. The information processing device according to claim 1 .

9. When the haptic control information of a pixel becomes a third value between a first value and a second value in response to enlargement or reduction of an image displayed based on the image data, the data processing unit supplies to the haptic presentation device a signal in which the haptic control signal corresponding to the first value and the haptic control signal corresponding to the second value are superimposed. The information processing device according to claim 8.

10. When the image data in the predetermined file format is image data of a moving image, other haptic control information is also embedded in the audio data of the moving image, The data processing unit supplies the haptic control signal to the first haptic presentation device based on the haptic control information embedded in the image data, and supplies the haptic control signal to the second haptic presentation device based on the other haptic control information embedded in the acoustic data. The information processing device according to claim 1 .

11. The haptic control information is stored in units of one or more frames of the image data. The information processing device according to claim 1 .

12. The haptic control information is stored in units of two or more pixels of the image data. The information processing device according to claim 1 .

13. The haptic control information includes a haptic stimulus value that quantifies an intensity of the haptic stimulus, The data processing unit supplies the haptic control signal corresponding to the haptic stimulus value to the haptic presentation device. The information processing device according to claim 1 .

14. A sensor unit that senses a touch position of the user; an image display unit that displays an image based on the image data; The tactile presentation device; Further equipped The information processing device according to claim 2 .

15. A storage unit that stores image data in a predetermined file format in which haptic control information is embedded; a communication unit that transmits the image data to a predetermined information processing device in response to a request from the predetermined information processing device; Equipped with The haptic control information includes haptic identification information for identifying a haptic control signal, and the predetermined file format is a file format having an alpha channel, in which the haptic identification information is stored, and which has a flag indicating whether the haptic identification information or transparency is stored in the alpha channel. Server device.

16. The storage unit also stores the tactile control signal corresponding to the tactile identification information; The communication unit transmits the haptic control signal to the predetermined information processing device in response to a request from the predetermined information processing device. The server device according to claim 15.

17. the communication unit receives image data in the predetermined file format transmitted from another information processing device; The storage unit stores the received image data in the predetermined file format. The server device according to claim 15.

18. An information processing device, Supplying a haptic control signal to a haptic presentation device based on haptic control information embedded in image data in a predetermined file format. Including, the haptic control information includes haptic identification information that identifies the haptic control signal; The predetermined file format is a file format having an alpha channel, the tactile identification information is stored in the alpha channel, and a flag indicating whether the tactile identification information or transparency is stored in the alpha channel is included. Information processing methods.

19. Computer, A data processing unit that supplies a haptic control signal to the haptic presentation device based on haptic control information embedded in image data in a predetermined file format. Function as a the haptic control information includes haptic identification information that identifies the haptic control signal; The predetermined file format is a file format having an alpha channel, the tactile identification information is stored in the alpha channel, and a flag indicating whether the tactile identification information or transparency is stored in the alpha channel is included. program.

Images