Information Processing Apparatus, Information Processing Method, Program, and Information Processing System

Abstract

Provided is an information processing device (300) comprising: a first acquisition unit (308) that acquires, in response to input from a first user, control commands that include presentation unit information indicating a presentation unit that presents a tactile stimulus by means of a tactile presentation device and that include form information indicating a form of the tactile stimulus; a generation unit (302) that generates tactile control signals for presenting the tactile stimulus to the presentation unit in response to the control commands; and a first distribution unit (302) that distributes the tactile control signals to the tactile presentation device mounted on the body of a second user.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, a program, and an information processing system.

Background Art

[0002] In recent years, consumers' consumption activities have shifted from "product consumption" which is merely the purchase of goods to "experience consumption" where they pay for experiences with high added value. For example, in the entertainment field such as music and animation, consumers not only seek one-way experiences such as watching live performances of artists (distributors), but also strongly demand more real-time, interactive, and high-added-value experiences such as exchanges between artists and fans.

[0003] For example, as an example of such an experience, "tipping" can be cited, where viewers of content distributed on the Internet send data such as illustrations and texts, or send money along with these, to artists or the like. By communicating between the artist and the viewer through such "tipping", the viewer can obtain a more high-added-value experience. As a result, the viewer's satisfaction with the content will be further enhanced, or the willingness to purchase such "experiences" will increase.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, conventionally, devices for presenting tactile stimuli such as vibrations to users have been proposed. For example, as an example, a jacket-type tactile presentation device described in Patent Document 1 can be cited. Such a tactile presentation device can be worn by a user in a movie theater, an attraction in a theme park, etc., and is controlled to be synchronized with the reproduced content that the user views, thereby amplifying the sense of presence of the reproduced content provided.

[0006] Therefore, in the present disclosure, an information processing device, an information processing method, a program, and an information processing system are proposed that can use such a tactile presentation device to provide a viewer with a real-time and highly interactive added-value experience.

Means for Solving the Problem

[0007] According to the present disclosure, a first acquisition unit that acquires each control command including presentation unit information that designates a presentation unit that presents a tactile stimulus by a tactile presentation device in response to an input from a first user and form information that designates the form of the tactile stimulus, a generation unit that generates each tactile control signal for presenting the tactile stimulus to the presentation unit in response to each control command, and a first distribution unit that distributes each tactile control signal to the tactile presentation device worn on the body of a second user are provided. The first acquisition unit acquires a new control command different from each of the control commands corresponding to each of the inputs from the plurality of first users when the inputs from the plurality of first users satisfy a predetermined condition. An information processing device is provided.

[0008] Also, according to the present disclosure, a first acquisition unit that acquires each identification information for specifying presentation unit information that specifies a presentation unit that presents a tactile stimulus by a tactile presentation device and form information that specifies the form of the tactile stimulus in response to an input from a first user, a generation unit that generates each tactile control signal for presenting the tactile stimulus to the presentation unit according to each identification information, and a first distribution unit that distributes each tactile control signal to the tactile presentation device worn on the body of a second user are provided. When inputs from a plurality of the first users satisfy a predetermined condition, the first acquisition unit acquires other identification information in which the presentation unit information and the form information different from the presentation unit information and the form information specified by each of the identification information corresponding to each of the inputs from the plurality of first users are specified. An information processing apparatus is provided.

[0009] Also, according to the present disclosure, an information processing method includes acquiring, by an information processing apparatus, each control command including presentation unit information that specifies a presentation unit that presents a tactile stimulus by a tactile presentation device and form information that specifies the form of the tactile stimulus in response to an input from a first user, generating, according to each control command, each tactile control signal for presenting the tactile stimulus to the presentation unit, and distributing each tactile control signal to the tactile presentation device worn on the body of a second user. When inputs from a plurality of the first users satisfy a predetermined condition, new control commands different from each of the control commands corresponding to each of the inputs from the plurality of first users are acquired.

[0010] Also, according to the present disclosure, a program is provided that causes a computer to have a function of acquiring each control command including presentation unit information that designates a presentation unit that presents a tactile stimulus by a tactile presentation device and form information that designates the form of the tactile stimulus in response to an input from a first user, a function of generating each tactile control signal for presenting the tactile stimulus to the presentation unit according to each control command, and a function of distributing each tactile control signal to the tactile presentation device worn on the body of a second user. When inputs from a plurality of the first users satisfy a predetermined condition, the program acquires a new control command different from each of the control commands corresponding to each of the inputs from the plurality of the first users.

[0011] Furthermore, according to the present disclosure, there is provided an information processing system including an information processing device and a distribution device. The information processing device includes a first acquisition unit that acquires each control command including presentation unit information that designates a presentation unit that presents a tactile stimulus by a tactile presentation device and form information that designates the form of the tactile stimulus in response to an input from a first user, a generation unit that generates each tactile control signal for presenting the tactile stimulus to the presentation unit according to each control command, and a first distribution unit that distributes each tactile control signal to the tactile presentation device worn on the body of a second user. When inputs from a plurality of the first users satisfy a predetermined condition, the first acquisition unit acquires a new control command different from each of the control commands corresponding to each of the inputs from the plurality of the first users. The distribution device includes an image generation unit that superimposes a predetermined image generated based on the input on a real-space image distributed to the first user.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Figure 47

Figure 48

Figure 49

Figure 50

Figure 51

Figure 52

Figure 53

Figure 54

Figure 55

Figure 56

Figure 57

Mode for Carrying Out the Invention

[0013] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted. Also, in the present specification and drawings, similar components in different embodiments may be distinguished by attaching different alphabets after the same reference numeral. However, when it is not particularly necessary to distinguish each of the similar components, only the same reference numeral is attached.

[0014] The description will be made in the following order. 1. Overview of Embodiments of the Present Disclosure 2. Overview of Information Processing System 10 of the Present Disclosure 2.1 Overview of Information Processing System 10 2.2 Detailed Configuration of Tactile Presentation Device 100 2.3 Detailed Configuration of Haptics Server 300 2.4 Detailed Configuration of Distribution Data Editing Server 400 2.5 Detailed Configuration of Live Streaming Server 500 2.6 Detailed Configuration of User Terminal 700 3. First Embodiment 4. Second Embodiment 5. Third Embodiment 6. Fourth Embodiment 7. Fifth Embodiment 8. Sixth Embodiment 9. Seventh Embodiment 10. Eighth Embodiment 11. Ninth Embodiment 12. Summary 13. Modification Example 1 of Information Processing System 10 of the Present Disclosure 14. Modification Example 2 of Information Processing System 10 of the Present Disclosure 15. Output Method of Tactile Stimulation 16. Regarding Modification Examples of Stamp Display 17. Regarding Hardware Configuration 18. Supplementary Explanation

[0015] <<1. Overview of Embodiments of the Present Disclosure>> First, before explaining the details of the embodiments of the present disclosure, with reference to FIGS. 1 and 2, an overview of the embodiments of the present disclosure created by the inventors will be described. FIGS. 1 and 2 are explanatory diagrams for explaining the overview of the embodiments of the present disclosure.

[0016] As described above, in recent years, consumers' consumption activities have shifted from "product consumption" to "service consumption." In particular, in the entertainment field, consumers not only seek one-way experiences such as watching live performances by artists, etc., but also strongly demand more real-time, interactive, and higher-value-added experiences such as communicating with artists.

[0017] Therefore, in view of such a situation, the inventors have been earnestly studying whether it is possible to provide viewers with a higher-value-added experience, and have conceived of using a tactile presentation device (haptics device) that presents tactile stimuli such as vibrations to the wearer. The inventors believe that by using such a tactile presentation device, viewers can obtain a higher-value-added experience.

[0018] In an embodiment of the present disclosure created based on such an idea, as shown in FIG. 1, for example, a best-type tactile presentation device 100 is worn by a distributor 800 who watches or distributes performances and content. The tactile presentation device 100 has a plurality of tactile stimulation units (for example, actuators, etc.) inside. Further, when the tactile stimulation unit receives a predetermined signal, it can present a tactile stimulus to the wearer, the distributor 800. Next, as shown in FIG. 2, the viewer 900 selects one stamp 850 with a tactile stimulation effect from a plurality of stamps 850 with tactile stimulation effects displayed on the display unit 702 of the user terminal 700, and transmits the selected stamp 850 with a tactile stimulation effect. Then, by transmitting the selected stamp 850 with a tactile stimulation effect, a predetermined tactile control signal corresponding to the tactile stimulus given to the stamp 850 is transmitted to the tactile stimulation unit (so-called, performing "tossing money" with a tactile stimulation effect). Further, the tactile stimulation unit presents a tactile stimulus corresponding to the selected stamp 850 with a tactile stimulation effect to the wearer based on the received predetermined tactile control signal.

[0019] Therefore, if the distributor 800 perceives the presented tactile stimulus and takes an action, the viewer 900 can confirm in real time the action induced by the tactile stimulus presented by the stamp 850 sent by the viewer himself / herself. And if the viewer 900 can confirm in real time the action induced by the tactile stimulus related to the stamp 850 sent by the viewer himself / herself, the viewer 900 can feel as if directly interacting with the distributor 800, that is, can obtain a more value-added experience.

[0020] Thus, in the embodiment of the present disclosure created by the present inventors, not only visual information such as illustrations, animations, texts, etc. and auditory information such as music can be sent from the viewer 900 to the distributor 800, but also tactile information can be sent. Therefore, according to the embodiment of the present disclosure, it is possible to provide a highly interactive and value-added experience to the viewer 900 in real time, which can further enhance the viewer 900's satisfaction with the performance and content, or increase the viewer 900's willingness to purchase such "things". Hereinafter, the details of the embodiment of the present disclosure created by the present inventors will be sequentially described.

[0021] <<2. Overview of the information processing system 10 of the present disclosure>> <2.1 Overview of the information processing system 10> First, with reference to FIG. 3, the overview of the information processing system 10 according to the embodiment of the present disclosure will be described. FIG. 3 is a system diagram showing a schematic configuration example of the information processing system 10 according to the embodiment of the present disclosure. In the following description, it is assumed that the distributor 800 and the viewer 900 exist in different spaces or the same space. That is, in the information processing system 10, the viewer 900 may, for example, directly view the appearance of the distributor 800 or view the video of the distributed distributor 800.

[0022] Specifically, as shown in FIG. 3, in the information processing system 10 according to the present embodiment, for example, on the distributor 800 side, there are arranged a tactile presentation device 100, a drive amplifier / interface 200, a speaker 202, a monitor 204, a microphone (hereinafter referred to as a mic) 206, and a camera 208. Further, in the information processing system 10, between the distributor 800 side and the viewer 900 side, for example, there are arranged a haptics server (information processing device) 300, a distribution data editing server 400, and a live distribution server 500 (other information processing device). Furthermore, in the information processing system 10, on the viewer 900 side, there are arranged a smartphone or a tablet as an example of the user terminal 700. Each device included in the information processing system 10 can transmit and receive data to and from each other via various communication networks such as, for example, a wired / wireless LAN (Local Area Network), or Wi-Fi (registered trademark), Bluetooth (registered trademark), a mobile communication network (LTE (Long Term Evolution), 5G (fifth-generation mobile communication system)), etc. Note that the number of each device included in the information processing system 10 is not limited to the number shown in FIG. 3, and may be even more. Also, the information processing system 10 may include devices not shown in FIG. 3. For example, the information processing system 10 can include a general-purpose PC (Personal Computer), a game console, a mobile phone, a portable media player, a speaker, a projector, a display (digital signage, etc.), headphones, smart glasses, wearable devices such as a smartwatch, etc.

[0023] In the information processing system 10 shown in FIG. 3, for example, the haptics server 300 that manages the presentation of tactile stimuli, the distribution data editing server 400 with an application for editing video and audio distributed to the viewer 900, etc., and the live distribution server 500 that manages the distribution to the viewer 900 may be operated by different operators. That is, in the present embodiment, the operator who manages and operates each server is not particularly limited, and the operators who operate each server may all be different, or for some or all of the servers, they may be operated by a common operator. Hereinafter, an outline of each device included in the information processing system 10 according to the present embodiment will be described.

[0024] (Tactile presentation device 100) The tactile presentation device 100 is configured by, for example, a device that can be worn on the body. In the present embodiment, the tactile presentation device 100 is assumed to be, for example, a vest-type (jacket shape without sleeves) device worn by the distributor 800. As described above, the vest-type tactile presentation device 100 has a plurality of tactile stimulation units (not shown) inside. The tactile stimulation units can be arranged, for example, a predetermined number (e.g., 6) each on the front side and the back side of the distributor 800 inside the tactile presentation device 100. As an example, the individual tactile stimulation units arranged on the front side and the individual tactile stimulation units arranged on the back side are arranged in a facing position relationship.

[0025] Note that in the present embodiment, the shape of the tactile presentation device 100 is not limited to the vest type, and it may be a jacket shape with sleeves. In this case, not only the chest and abdomen of the distributor 800 but also at least one tactile stimulation unit may be arranged at positions corresponding to both arms of the distributor 800. Also, in the present embodiment, the tactile presentation device 100 is not limited to the jacket shape, and it may be in the shape of pants, shoes, a belt, a hat, gloves, a mask, or the like.

[0026] Also, one microphone 206 may be arranged on each of the left and right shoulder portions of the tactile presentation device 100, or one microphone 206 may be arranged on either the left or the right, or three or more microphones may be arranged. Further, the microphone 206 may be arranged around the distributor 800 as another device independent of the tactile presentation device 100. Further, the tactile presentation device 100 may incorporate a wearing state detection sensor (e.g., a fastener type sensor, a pressure sensor, etc.) (not shown) for detecting the wearing state of the tactile presentation device 100, an inertial sensor (Inertial Measurement Unit: IMU) (not shown) for detecting the motion and posture of the distributor 800, a biological information sensor (e.g., a sensor for sensing heartbeat, pulse, brain wave, respiration, sweating, myoelectric potential, skin temperature, skin electrical resistance, eye movement, or pupil diameter, etc.) (not shown) for detecting the biological information of the distributor 800, and the like. The detailed configuration of the tactile presentation device 100 will be described later.

[0027] (Drive Amplifier / Interface 200) The drive amplifier / interface 200 is an interface for transmitting and receiving tactile control signals between the tactile presentation device 100 and the haptics server 300. For example, the drive amplifier / interface 200 can acquire the profile information (function information, etc.) of the tactile presentation device 100 from the tactile presentation device 100, or signal-convert and amplify the tactile control signal generated by the haptics server 300 and transmit it to the tactile presentation device 100.

[0028] (Monitor 204) The monitor 204 can display, for example, the video of the distributor 800 captured by the camera 208 described later towards the distributor 800. Further, the monitor 204 can superimpose and display text, icons, animations, etc. on the image of the distributor 800. For example, the monitor 204 is realized by a liquid crystal display (LCD) device, an organic light emitting diode (OLED) device, etc. Also, in the present embodiment, the display unit (not shown) of the monitor 204 may be provided as a unit integrated with the input unit (not shown). In this case, the input unit is realized by, for example, a touch panel superimposed on the display unit. Further, in the present embodiment, the monitor 204 may be provided with a speaker 202 that outputs voice to the distributor 800.

[0029] (Camera 208) The camera 208 is one or a plurality of visible light cameras that image the distributor 800 from one viewpoint or multiple viewpoints. The video captured by the camera 208 is transmitted to the user terminal 700 on the viewer 900 side via the haptics server 300, the distribution data editing server 400, and the live distribution server 500. Note that the camera 208 may image the surroundings of the distributor 800 or real objects existing in the surroundings. Specifically, the camera 208 includes a lens system composed of an imaging lens, an aperture, a zoom lens, a focus lens, etc., and a drive system that causes the lens system to perform a focus operation and a zoom operation. Further, the camera 208 each has a solid-state imaging device array that photoelectrically converts the imaging light obtained by the lens system to generate an imaging signal. Note that the solid-state imaging device array may be realized by, for example, a charge coupled device (CCD) sensor array or a complementary metal oxide semiconductor (CMOS) sensor array.

[0030] (Haptics Server 300) The haptics server 300 can receive a stamp (control command) 850 with a haptic stimulation effect input from the viewer 900 via the live distribution server 500, generate a haptic control signal according to the stamp 850, and transmit the generated haptic control signal to the haptic presentation device 100. Each stamp 850 with a haptic stimulation effect is associated with each predetermined control command. Each control command includes information (position information) specifying the perception position for presenting a vibration stimulation (specifically, for example, information specifying the haptic stimulation part for presenting the haptic stimulation, etc.), information (form information) specifying the waveform type, intensity, etc. of the vibration stimulation, and the like. Further, the control command may include identification information (ID) specifying the haptic stimulation part (not shown) provided in the haptic presentation device 100 and the above form information. Specifically, the haptics server 300 generates a haptic control signal (waveform data) to be input to each individual haptic stimulation part provided in the haptic presentation device 100 so as to present a vibration stimulation with the specified intensity and the specified waveform at the specified perception position according to the control command, and transmits it to the haptic presentation device 100. Furthermore, the control command associated with each stamp 850 with a haptic stimulation effect may include only the identification information (ID) of the stamp 850. In this case, the haptics server 300 may generate a haptic control signal (waveform data) corresponding to the stamp 850 based on the data associated with the received identification information of the stamp 850 by referring to the pre-stored data. In this specification, the "perception position" shall also include the movement path of the perception position and the perception range having a predetermined width. The detailed configuration of the haptics server 300 will be described later.

[0031] (Distribution data editing server 400) The distribution data editing server 400 can edit the video received by the camera 208 via the haptics server 300, or edit the audio received by the microphone 206 via the haptics server 300. Furthermore, the distribution data editing server 400 can transmit the edited video and audio data to the user terminal 700 via the live distribution server 500, or output it to the speaker 202 and the monitor 204 via the haptics server 300. For example, the distribution data editing server 400 can generate video data for distribution by superimposing the image of the stamp 850 input from the viewer 900 or the video effect associated with the stamp 850 on the image of the distributor 800 captured by the camera 208. The detailed configuration of the distribution data editing server 400 will be described later.

[0032] (Live distribution server 500) The live distribution server 500 can distribute to the user terminal 700 images and the like for selecting images of the distributor 800 and the like, or stamps 850 with tactile stimulation effects. For example, the live distribution server 500 can perform authentication via a Web API (Application Programming Interface) and monitor stamps 850 with tactile stimulation effects transmitted from the viewer 900. The detailed configuration of the live distribution server 500 will be described later.

[0033] Furthermore, in this embodiment, the haptics server 300, the distribution data editing server 400, and the live distribution server 500 may be implemented by a single device, or may be implemented by a plurality of devices, and are not particularly limited. This will be described in detail later.

[0034] (User terminal 700) The user terminal 700 is used by the viewer 900 or installed near the viewer 900. For example, it is a terminal for the viewer 900 to input the stamp 850 with the tactile stimulation effect. The user terminal 700 receives the stamp (control command) 850 with the tactile stimulation effect input from the viewer 900 and transmits the received stamp 850 to the haptics server 300 via the live distribution server 500. Also, the user terminal 700 can receive and display, via the live distribution server 500, the video of the distributor 800 to whom a video effect associated with the received stamp 850 is applied and the tactile stimulation associated with the stamp 850 is presented. At that time, the user terminal 700 may receive the identification information (ID) of the stamp 850, read out the information of the video effect associated with the identification information from its own storage unit (not shown), and perform processing by itself, or may receive the video to which the video effect has already been applied. For example, the user terminal 700 can be a wearable device such as a smartphone, a tablet PC (Personal Computer), a mobile phone, a laptop PC, an HMD (Head Mounted Display). Also, the user terminal 700 may be a dedicated device installed in a room provided by a business operator such as a karaoke box. The detailed configuration of the user terminal 700 will be described later.

[0035] <2.2 Detailed Configuration of the Tactile Presentation Device 100> Next, with reference to FIGS. 4 and 5, the detailed configuration of the tactile presentation device 100 will be described. FIG. 4 is a diagram showing an example of the external configuration of the tactile presentation device 100 according to the present embodiment, and FIG. 5 is a diagram showing an example of the functional configuration of the tactile presentation device 100 according to the present embodiment. As described above, the tactile presentation device 100 is a device that is worn on a part of the body of the distributor 800 and gives a tactile stimulation to the distributor 800 by vibrating or the like according to the stamp (control command) 850.

[0036] As shown in FIG. 4, for example, the vest-type tactile presentation device 100 has a plurality of tactile stimulation units 106 inside, as described above. The tactile stimulation unit 106 is composed of, for example, an actuator, and is driven by a tactile control signal generated by the haptics server 300 to generate vibration, and presents such vibration as a tactile stimulation. As the actuator, for example, an eccentric motor, a linear vibrator, a piezoelectric element, etc. can be used.

[0037] Furthermore, as shown in FIG. 5, the tactile presentation device 100 has a communication unit 102, a control unit 104, the above-described tactile stimulation unit 106, and an operation unit 108. Each functional block of the tactile presentation device 100 will be sequentially described below.

[0038] (Communication Unit 102) The communication unit 102 is wirelessly connected to the haptics server 300 via the drive amplifier / interface 200 and can transmit and receive information to and from the haptics server 300.

[0039] (Control Unit 104) The control unit 104 is a controller and can drive the tactile stimulation unit 106 based on the tactile control signal input via the above-described communication unit 102. The control unit 104 is realized, for example, by various programs stored in the ROM (Read Only Memory) inside the tactile presentation device 100 being executed with the RAM (Random Access Memory) as a working area by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc. Also, the control unit 104 can be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0040] (Operation Unit 108) The operation unit 108 is an operation device such as a touch sensor, a pressure sensor, a proximity sensor, a button, a switch, and a lever that is operated by the distributor 800. For example, the distributor 800 can weaken the intensity of the presented tactile stimulus or stop the presentation of the tactile stimulus.

[0041] As described above, the detailed configuration of the tactile presentation device 100 according to the present embodiment has been specifically described. However, the detailed configuration of the tactile presentation device 100 according to the present embodiment is not limited to the examples shown in FIGS. 4 and 5.

[0042] Further, the tactile presentation device 100 is not limited to the best type of wearable device as described above, and may be a wearable device that can be worn on a part of the user's body (such as the earlobe, neck, arm, wrist, ankle, etc.). More specifically, examples of the wearable device include wearable devices of various methods / forms such as ear device type, anklet type, ring type, glove type, bracelet (wristband) type, necklace type, eyewear type, headgear type, pad type, patch type, and clothing type. In addition, the tactile presentation device 100 may be configured as an on-hand type mounted on a device held by the distributor 800, such as a smartphone, a tablet, a camera, a game controller, a portable music player, etc., and may be a pen type, a stick type, or a handle type. Furthermore, the tactile presentation device 100 is not limited to the wearable type or the on-hand type, and may be configured as a slate / floor type mounted on furniture or various equipment such as a bed, a chair, and a table.

[0043] Furthermore, in the embodiments of the present disclosure, although it is described that the tactile presentation device 100 presents a tactile stimulus (vibration) to the distributor 800, the present disclosure is not limited thereto. Instead of the tactile stimulus, or together with the tactile stimulus, wind, electrical stimulus, ultrasonic waves, force sensation, heat, moisture, fragrance, etc. may be given to the distributor 800. Also, in the embodiments of the present disclosure, it is not limited to presenting a tactile stimulus to the distributor 800, and a tactile stimulus or the like may be presented to the viewer 900 who has selected the stamp 850 or the viewer 900 who simply enjoys the distribution of the distributor 800.

[0044] <2.3 Detailed Configuration of the Haptics Server 300> Next, with reference to FIG. 6, the detailed configuration of the haptics server 300 according to the embodiments of the present disclosure will be described. FIG. 6 is a diagram showing a functional configuration example of the haptics server 300 according to the embodiments of the present disclosure. As shown in FIG. 6, the haptics server 300 mainly includes a communication unit 302, a camera image acquisition unit 304, a microphone audio acquisition unit 306, a stamp acquisition unit 308, a tactile signal generation unit 310, a distributor state acquisition unit 312, an output image acquisition unit 314, an output audio acquisition unit 316, and a storage unit 318. Hereinafter, each functional block of the haptics server 300 will be sequentially described.

[0045] (Communication Unit 302) The communication unit (distribution unit) 302 can transmit and receive information between the tactile presentation device 100, the speaker 202, the monitor 204, and the camera 208. The communication unit 302 is a communication interface having a function of transmitting and receiving data, and is realized by communication devices (not shown) such as a communication antenna, a transmission and reception circuit, and a port. Specifically, the communication unit 302 can transmit a tactile control signal to the tactile presentation device 100, transmit video data (including, for example, text information) from the distribution data editing server 400 to the monitor 204, and transmit audio data (predetermined audio) from the distribution data editing server 400 to the speaker 202.

[0046] Next, the camera image acquisition unit 304, the microphone audio acquisition unit 306, the stamp acquisition unit 308, the tactile signal generation unit 310, the distributor state acquisition unit 312, the output image acquisition unit 314, and the output audio acquisition unit 316 will be described. These functional units are realized, for example, by various programs stored in a ROM or the like inside the haptics server 300 being executed by a CPU, MPU, or the like with the RAM as a work area.

[0047] (Camera Image Acquisition Unit 304) The camera image acquisition unit 304 can acquire an image of the real space on the distributor 800 side or an image of the distributor 800 from the camera 208 and transmit it to the distribution data editing server 400 via the communication unit 302.

[0048] (Microphone Audio Acquisition Unit 306) The microphone audio acquisition unit 306 can acquire the ambient sound of the real space on the distributor 800 side or the voice of the distributor 800 from the microphone 206 and transmit it to the distribution data editing server 400 via the communication unit 302.

[0049] (Stamp Acquisition Unit 308) The stamp acquisition unit 308 can acquire a control command including position information (presentation unit information) specifying a presentation position for presenting a tactile stimulus by the tactile presentation device 100 and form information specifying the form of the tactile stimulus, which are associated with the stamp 850 with a tactile stimulus effect input from the viewer 900, and output it to the tactile signal generation unit 310 described later. Further, the stamp acquisition unit 308 may acquire a control command including only the identification information (ID) associated with the stamp 850 with a tactile stimulus effect and output it to the tactile signal generation unit 310 described later.

[0050] (Tactile Signal Generation Unit 310) The tactile signal generation unit 310 can generate a tactile control signal for controlling the tactile presentation device 100 based on a control command associated with the stamped image 850 with tactile stimulus effects input from the viewer 900. Specifically, the tactile signal generation unit 310, based on a control command including the perceived position (position information) of the vibration stimulus, the waveform type of the vibration stimulus, and the intensity information (form information) of the vibration stimulus, etc., at the specified perceived position, at the specified intensity, generates waveform data for input to the individual tactile stimulus units 106 (specifically, vibration actuators) provided in the tactile presentation device 100 so as to present a vibration stimulus with the specified waveform. Note that the tactile control signal may include information instructing to present a vibration stimulus at the presentation timing, frequency, interval, and presentation time of the tactile stimulus based on the above control command. Further, the tactile signal generation unit 310 may refer to a control command including only the identification information (ID) associated with the stamped image 850 with tactile stimulus effects input from the viewer 900, and generate a tactile control signal (including the perceived position, waveform type, intensity information, etc.) corresponding to the stamped image 850 based on the data associated with the stored identification information in advance.

[0051] Furthermore, if the frequency or intensity of the vibration stimulus specified by the control command has a wide band (for example, 50 to 500 Hz), the tactile signal generation unit 310 may compress it to a narrow band (for example, 100 Hz) according to the function of the tactile presentation device 100 to generate waveform data. Further, the tactile signal generation unit 310 may adjust the tactile control signal according to the wearing status of the tactile presentation device 100 of the distributor 800 or the profile (functions, etc.) of the tactile presentation device 100.

[0052] Also, the tactile signal generation unit 310 can transmit the generated tactile control signal to the tactile presentation device 100 worn on the body of the distributor 800 via the communication unit 302.

[0053] (Distributor state acquisition unit 312) The distributor status acquisition unit 312 can acquire, for example, sensing data obtained by a wearing state detection sensor (not shown) for detecting the wearing state of the tactile presentation device 100, and output it to the above-described tactile signal generation unit 310.

[0054] (Output image acquisition unit 314) The output image acquisition unit 314 can acquire the edited video data (predetermined images, text information) from the distribution data editing server 400 and transmit it to the monitor 204 via the communication unit 302.

[0055] (Output audio acquisition unit 316) The output audio acquisition unit 316 can acquire the edited audio data and the like from the distribution data editing server 400 and transmit it to the speaker 202 via the communication unit 302.

[0056] (Memory unit 318) The memory unit 318 is realized by a storage device such as a ROM that stores programs, arithmetic parameters, etc. used in the processing of the tactile signal generation unit 310, a RAM that temporarily stores parameters that change as appropriate, and an HDD that stores various databases (DBs). For example, the memory unit 318 stores a pre-generated tactile data library (perception position, vibration waveform pattern, etc.) associated with the identification information (ID) of each stamp 850, and using this, the above-described tactile signal generation unit 310 can also generate a tactile control signal. Further, the memory unit 318 may store information such as the number, position, frequency characteristics, maximum input voltage, etc. of the tactile stimulation units 106 of the tactile presentation device 100 as profile information of the tactile presentation device 100. The tactile signal generation unit 310 can adjust the tactile control signal with reference to such profile information.

[0057] As described above, the detailed configuration of the haptics server 300 according to the present embodiment has been specifically described, but the detailed configuration of the haptics server 300 according to the present embodiment is not limited to the example shown in FIG. 6.

[0058] <2.4 Detailed Configuration of the Distribution Data Editing Server 400> Next, with reference to FIG. 7, the detailed configuration of the distribution data editing server 400 according to an embodiment of the present disclosure will be described. FIG. 7 is a diagram showing a functional configuration example of the distribution data editing server 400 according to an embodiment of the present disclosure. As shown in FIG. 7, the distribution data editing server 400 mainly includes a communication unit 402, a camera image acquisition unit 404, an image generation unit 406, a stamp acquisition unit 408, a microphone audio acquisition unit 410, an audio generation unit 412, an output audio acquisition unit 414, and a storage unit 416. Each functional block of the distribution data editing server 400 will be sequentially described below.

[0059] (Communication Unit 402) The communication unit 402 can transmit and receive information between the haptics server 300 and the live distribution server 500. The communication unit 402 is a communication interface having a function of transmitting and receiving data, and is realized by communication devices (not shown) such as a communication antenna, a transmission and reception circuit, and a port.

[0060] Next, the camera image acquisition unit 404, the image generation unit 406, the stamp acquisition unit 408, the microphone audio acquisition unit 410, the audio generation unit 412, and the output audio acquisition unit 414 will be described. These functional units are realized, for example, by various programs stored in the internal ROM of the distribution data editing server 400, such as a CPU or an MPU, being executed with the RAM as a work area.

[0061] (Camera Image Acquisition Unit 404) The camera image acquisition unit 404 can acquire an image of the real space on the side of the distributor 800 and an image of the distributor 800 from the camera 208 via the haptics server 300 and output them to the image generation unit 406 described later.

[0062] (Image Generation Unit 406) The image generation unit 406 can generate image data for presenting to the viewer 900 or the distributor 800. For example, the image generation unit 406 can generate video data that superimposes virtual objects such as icons, animations, and text on the image of the distributor 800. More specifically, the virtual object can be an animation such as an exploded bomb that is displayed simultaneously with the presentation of a tactile stimulus. Further, the virtual object may be an image corresponding to the tactile stimulus effect stamp 850 input from the viewer 900. For example, in the case of a heart-shaped stamp 850, the image generation unit 406 can generate video data that superimposes the heart-shaped stamp 850 on the image of the distributor 800. Additionally, the image generation unit 406 may apply a video effect associated with the stamp 850 input from the viewer 900 to the image of the distributor 800, for example. Also, when the tactile presentation device 100 is not attached to the body of the distributor 800 and thus the distributor 800 cannot be presented with a tactile stimulus, the image generation unit 406 can generate video data to be presented to the viewer 900 or the like instead of the tactile stimulus.

[0063] (Stamp acquisition unit 408) The stamp acquisition unit 408 can acquire information on the tactile stimulus effect stamp 850 input from the viewer 900 and output it to the image generation unit 406 and the audio generation unit 412.

[0064] (Microphone audio acquisition unit 410) The microphone audio acquisition unit 410 can acquire the voice of the distributor 800 from the microphone 206 via the haptics server 300 and output it to the audio generation unit 412 described later.

[0065] (Audio generation unit 412) The voice generation unit 412 can generate voice data for presentation to the viewer 900 or the distributor 800. For example, the voice generation unit 412 can generate voice data such as the sound of an exploding bomb that is output simultaneously with the presentation of a tactile stimulus. Further, the voice generation unit 412 may generate voice data of a voice corresponding to the stamp 850 with a tactile stimulus effect input from the viewer 900. For example, in the case of a stamp 850 in the shape of a wild bird, the voice generation unit 412 generates voice data in which the chirping sound of the wild bird is superimposed on the singing voice of the distributor 800 or the like. Furthermore, the voice generation unit 412 may generate voice data in which the voice of the viewer 900 acquired by the user terminal 700 is superimposed on the singing voice of the distributor 800 or the like. Also, for example, when the tactile presentation device 100 is not worn on the body of the distributor 800 and thus a tactile stimulus cannot be presented to the distributor 800, the voice generation unit 412 can also generate voice data that is output to the viewer 900 or the like instead of the tactile stimulus.

[0066] (Output voice acquisition unit 414) The output voice acquisition unit 414 can acquire the voice input from the viewer 900 using the user terminal 700 and output it to the voice generation unit 412.

[0067] (Storage unit 416) The storage unit 416 is realized from a storage device such as a ROM that stores programs, arithmetic parameters, etc. used in the processing by the image generation unit 406 and the output voice acquisition unit 414, a RAM that temporarily stores parameters that change as appropriate, and an HDD that stores various databases (DB). For example, the storage unit 416 stores, in association with the identification information (ID) of each stamp 850, pre-generated images and a voice data library, and using this, the image generation unit 406 and the output voice acquisition unit 414 generate video data and voice data.

[0068] As described above, the detailed configuration of the distribution data editing server 400 according to the present embodiment has been specifically described, but the detailed configuration of the distribution data editing server 400 according to the present embodiment is not limited to the example shown in FIG. 7.

[0069] <Detailed Configuration of Live Streaming Server 500> Next, with reference to FIG. 8, the detailed configuration of the live streaming server 500 according to the embodiment of the present disclosure will be described. FIG. 8 is a diagram showing a functional configuration example of the live streaming server 500 according to the embodiment of the present disclosure. As shown in FIG. 8, the live streaming server 500 mainly includes a communication unit 502, a GUI control unit 504, a stamp acquisition unit 506, an audio data acquisition unit 508, an image data acquisition unit 510, a tactile signal acquisition unit 512, a viewer information acquisition unit 514, a distribution control unit 516, and a storage unit 518. Each functional block of the live streaming server 500 will be sequentially described below.

[0070] (Communication Unit 502) The communication unit 502 can transmit and receive information to and from the user terminal 700, the distribution data editing server 400, and the haptics server 300. The communication unit 502 is a communication interface having a function of transmitting and receiving data, and is realized by communication devices (not shown) such as a communication antenna, a transmission and reception circuit, and a port.

[0071] Next, the GUI control unit 504, the stamp acquisition unit 506, the audio data acquisition unit 508, the image data acquisition unit 510, the tactile signal acquisition unit 512, the viewer information acquisition unit 514, and the distribution control unit 516 will be described. These functional units are realized, for example, by various programs stored in the internal ROM of the live streaming server 500, such as a CPU or an MPU, being executed with the RAM as a work area.

[0072] (GUI Control Unit 504) The GUI (Graphical User Interface) control unit 504 can control the user terminal 700 of the viewer 900 to display a screen for inputting a stamp with a tactile stimulation effect (control command) 850. Specifically, the GUI control unit 504 causes the display unit 702 of the user terminal 700 to display a screen as shown in FIG. 2 as a screen for selecting the stamp 850 with a tactile stimulation effect. A large number of stamps 850 with a tactile stimulation effect are displayed on the selection screen (stamp selection screen). Each stamp 850 with a tactile stimulation effect is associated with a control command for the tactile presentation device 100, and the viewer 900 can input a control command by selecting the stamp 850 displayed on the selection screen. In this embodiment, it is preferable that the image of the stamp 850 with a tactile stimulation effect intuitively reminds the viewer 900 of tactile stimulations, thoughts (messages), etc. that the viewer 900 wants to send to the distributor 800. Also, as will be described later, the GUI control unit 504 may cause the display unit 702 of the user terminal 700 to display an image that can also select a stamp without a tactile stimulation effect in addition to the stamp 850 with a tactile stimulation effect.

[0073] (Stamp acquisition unit 506) The stamp acquisition unit 506 can acquire information (e.g., ID) of the stamp 850 with a tactile stimulation effect input from the viewer 900 and transmit it to the distribution data editing server 400 and the haptics server 300 via the communication unit 502. Furthermore, the stamp acquisition unit 506 may acquire command information such as a tactile control signal and an image table control signal corresponding to the above input from the user terminal 700.

[0074] (Audio data acquisition unit 508) The audio data acquisition unit 508 can acquire audio data from the distribution data editing server 400 via the communication unit 502 and transmit it to the user terminal 700.

[0075] (Image data acquisition unit 510) The image data acquisition unit 510 can acquire image data from the distribution data editing server 400 via the communication unit 502 and transmit it to the user terminal 700.

[0076] (Tactile signal acquisition unit 512) The tactile signal acquisition unit 512 can acquire a tactile control signal from the haptics server 300 and transmit it to the user terminal 700. For example, when the user terminal 700 is equipped with a vibration device (not shown), the vibration device may reproduce a tactile stimulus corresponding to the stamp 850 selected by the viewer 900.

[0077] (Viewer information acquisition unit 514) The viewer information acquisition unit 514 can, for example, acquire the identification information (e.g., ID) of the viewer 900 transmitted from the user terminal 700 and authenticate the viewer 900.

[0078] (Distribution control unit 516) The distribution control unit 516 can control the transmission of data from the audio data acquisition unit 508, the image data acquisition unit 510, and the tactile signal acquisition unit 512 to the user terminal 700 of the viewer 900 authenticated by the viewer information acquisition unit 514.

[0079] (Memory unit 518) The memory unit 518 is realized from a storage device such as a ROM that stores programs, arithmetic parameters, etc. used in the processing of the live distribution server 500, a RAM that temporarily stores parameters that change as appropriate, and an HDD that stores various databases (DB). For example, the memory unit 518 may store the identification information of the viewer 900, and further, the information of the control commands acquired by the viewer 900 in the past and the stamps (control commands) 850 used may be stored in association with the identification information of the viewer 900.

[0080] As described above, the detailed configuration of the live distribution server 500 according to the present embodiment has been specifically described. However, the detailed configuration of the live distribution server 500 according to the present embodiment is not limited to the example shown in FIG. 8. For example, the live distribution server 500 may further include a recommendation unit (not shown) that refers to the information stored in the storage unit 518, selects a stamp (control command) 850 that is frequently used by the viewer 900 and, conversely, a stamp (control command) 850 that is used infrequently, and recommends the selected stamp 850 with a tactile stimulation effect to the viewer 900. Further, the recommendation unit may recommend to the viewer 900 the input of the stamp 850 to the distributor 800 who has not obtained a predetermined number or more of stamps 850 in the past.

[0081] <2.6 Detailed Configuration of User Terminal 700> Next, with reference to FIG. 9, the detailed configuration of the user terminal 700 according to the embodiment of the present disclosure will be described. FIG. 9 is a diagram showing a functional configuration example of the user terminal 700 according to the present embodiment. As shown in FIG. 9, the user terminal 700 mainly includes a display unit 702, an operation input unit 704, a speaker 706, a communication unit 708, a control unit 710, a storage unit 712, and a sensor unit 720. Each functional block of the user terminal 700 will be sequentially described below.

[0082] (Display Unit 702) The display unit 702 can display, for example, a selection screen for selecting a stamp (control command) 850 with a tactile stimulation effect, an image of the distributor 800, etc. to the viewer 900. Further, the display unit 702 can superimpose and display text, an icon, an animation, etc. on the image of the distributor 800. The display unit 702 is realized by a liquid crystal display device, an OLED device, etc.

[0083] (Operation Input Unit 704) The operation input unit 704 receives the selection result of the stamp 850 with haptic stimulus effects associated with a control command including position information specifying the presentation position for presenting a haptic stimulus and form information specifying the form of the haptic stimulus, for example, by the viewer 900. Alternatively, the operation input unit 704 may directly receive the control command. For example, the operation input unit 704 is implemented by a switch, a button, a touch panel, a lever, or the like. Also, the content of the operation input by the operation input unit 704 may be displayed by the above-described display unit 702. Further, the operation input unit 704 is provided so as to overlap the above-described display unit 702, and can receive information (for example, position information) of an input operation from the viewer 900 with respect to a position selection screen for specifying the presentation position displayed by the display unit 702.

[0084] (Speaker 706) The speaker 706 can reproduce sound according to the control of a control unit 710 described later. Note that the speaker 706 may be provided in the user terminal 700, or may be a device separate from the user terminal 700, such as a pair of earphone speakers (not shown).

[0085] (Communication unit 708) The communication unit 708 can transmit and receive information to and from the live distribution server 500. Specifically, the communication unit 708 can transmit information on the stamp 850 with haptic stimulus effects (control command) input by the viewer 900 to the live distribution server 500. Also, the communication unit 708 can receive information transmitted from the live distribution server 500. For example, the communication unit 708 is a communication interface having a function of transmitting and receiving data, and is implemented by communication devices (not shown) such as a communication antenna, a transmission and reception circuit, and a port.

[0086] (Control unit 710) The control unit 710 is a controller of the user terminal 700, and is realized, for example, by various programs stored in a ROM or the like inside the user terminal 700 being executed with a RAM as a work area by a CPU, an MPU, or the like.

[0087] (Memory unit 712) The memory unit 712 is realized from a storage device such as a ROM that stores programs, arithmetic parameters, etc. used in the processing of the above-described control unit 710, and a RAM that temporarily stores parameters that change as appropriate, etc.

[0088] (Sensor unit 720) The sensor unit 720 can acquire sensing data regarding operations from the viewer 900 (such as vibrations applied to the user terminal 700). For example, as shown in FIG. 9, the sensor unit 720 mainly includes a camera 722, a microphone 724, a gyro sensor 726, and an acceleration sensor 728. Note that the above-described sensors are merely examples, and the present embodiment is not limited thereto.

[0089] The camera 722, for example, captures the actions of the viewer 900 and outputs the captured image to the control unit 710 described above. Then, the control unit 710 can extract a predetermined action by the viewer 900 from the image captured by the camera 722 and obtain a control command related to the extracted action. Specifically, the camera 722 includes a lens system composed of an imaging lens, an aperture, a zoom lens, a focus lens, etc., and a drive system that causes the lens system to perform a focus operation or a zoom operation. Further, the camera 722 respectively has a solid-state imaging device array that photoelectrically converts the imaging light obtained by the lens system to generate an imaging signal. Note that the solid-state imaging device array may be realized by, for example, a CCD sensor array, a CMOS sensor array, or the like. Also, the camera 722 may include a ToF (Time of Flight) sensor (not shown). The ToF sensor can irradiate a subject with irradiation light having a predetermined period, detect the reflected light reflected by the subject, and detect the phase difference or time difference between the irradiation light and the reflected light, thereby obtaining depth information of the subject. Here, the depth information of the subject is depth information, which is one of the information on the distances from the ToF sensor to each point on the subject surface. By aggregating the depth information of a plurality of points on the subject surface, shape information regarding the uneven shape of the subject surface, that is, the external shape information of the subject can be obtained.

[0090] The microphone 724 collects the voice of the viewer 900 and outputs the collected voice data to the control unit 710 described above. Then, the control unit 710 may extract a voice pattern by the viewer 900 from the voice collected by the microphone 724 and obtain a control command related to the extracted voice pattern (for example, specifying the stamp 850 by the voice of the viewer 900).

[0091] The gyro sensor 726 is realized by, for example, a three-axis gyro sensor, and detects the angular velocity (rotation speed) of the movement of the user terminal 700 by the viewer 900. Further, the acceleration sensor 728 is realized by, for example, a three-axis acceleration sensor (also called a G sensor), and detects the acceleration of the movement of the user terminal 700 by the viewer 900. In the present embodiment, based on the sensing data from these sensors, the operation performed by the viewer 900 on the user terminal 700 can be recognized, and further, a control command related to the recognized operation can be obtained.

[0092] As described above, the detailed configuration of the user terminal 700 according to the present embodiment has been specifically described. However, the detailed configuration of the user terminal 700 according to the present embodiment is not limited to the example shown in FIG. 9. For example, it may further include a vibration device (vibrator) that notifies the viewer 900 by vibrating. That is, the user terminal 700 may be equipped with a vibration device that reproduces a tactile stimulus, although not shown in FIG. 9, or may be a tactile presentation device 100 attached to the body of the viewer 900.

[0093] <<3. First Embodiment>> First, referring to FIGS. 10 to 18, the first embodiment of the present disclosure will be described. FIGS. 10, 12, 15 to 18 are explanatory diagrams for explaining display examples according to the present embodiment, and FIGS. 11, 13, 14 are explanatory diagrams for explaining tactile stimulus presentation examples according to the present embodiment.

[0094] In the example shown in FIG. 10, it is assumed that the viewer 900 selects a stamp 850 with a tactile stimulus effect, which consists of a plurality of heart shapes. In such a case, in the user terminal 700 shown in FIG. 10, the input stamp 850 may be displayed beside the comment 860 of the viewer 900. Then, as shown in FIG. 11, the tactile stimulation unit 106 at the tactile position specified by the control command associated with the selected stamp 850 of the tactile presentation device 100 attached to the body of the distributor 800 vibrates, and thus a tactile stimulus is presented to the distributor 800.

[0095] At this time, as shown in FIG. 10, the information processing system 10 may distribute to the viewer 900 an image in which the image of the stamp 850 is superimposed on the image of the distributor 800 (Augmented Reality; AR display). Specifically, the position where the stamp 850 is superimposed may be the perceived position specified by the control command associated with the stamp 850. At this time, the information processing system 10 may distribute the comment 860 of the viewer 900 who views the image of the same distributor 800 together with the above image. Further, in the present embodiment, based on the comment input by the viewer 900, the information processing system 10 may automatically select the stamp 850, and a tactile stimulus associated with the automatically selected stamp 850 may be presented to the distributor 800.

[0096] Also, in the present embodiment, as shown in FIG. 12, the virtual object superimposed on the image of the distributor 800 may be an animated image. In the example shown in FIG. 12, the stamp 852 input by the viewer 900 is a fireworks, and according to the stamp 852, an animated image of fireworks (an image in which light spreads from the center to the outer periphery) 852 is superimposed on the image of the distributor 800. And in this case, as shown in FIG. 13, according to the control command associated with the selected stamp 852, in synchronization with the change of the animated image 852, the tactile position (that is, the tactile stimulation part 106 where the tactile stimulation is presented) of the tactile presentation device 100 worn on the body of the distributor 800 changes. Specifically, in synchronization with the image in which light spreads from the center to the outer periphery of the animated image 852 of the fireworks, the tactile stimulation part 106 of the tactile presentation device 100 that presents the tactile stimulation changes from the tactile stimulation part 106 located at the center to the tactile stimulation part 106 located at the outer periphery. Note that in the present embodiment, the change of the animated image 852 is not limited to being synchronized with the change of the tactile position, and may be synchronized with any of the changes in the frequency, waveform, and intensity of the tactile control signal, for example.

[0097] Furthermore, as shown in FIG. 14, in the above-described embodiment, in synchronization with the change of the animation image 852, the tactile position (i.e., the tactile stimulation unit 106 where the tactile stimulation is presented) of the tactile presentation device 100 worn on the body of the distributor 800 changes, and this may be presented to the viewer 900 by showing specific information. For example, on the user terminal 700 of the viewer 900 shown in FIG. 14, a display 842 indicating which part (tactile stimulation unit 106) is vibrating is displayed. In the example of FIG. 14, since the tactile position changes as the animation image 852 changes, the display of the display 842 also changes. Also, in the display 842, the color, shade, brightness, etc. of the vibrating part may change according to the change in the intensity of the tactile stimulation presented at each tactile position. Furthermore, in the example of FIG. 14, a specific waveform image 840 of the tactile stimulation during playback is also displayed. Note that the waveform image may be displayed in a deformed manner, and the color, shade, brightness, line thickness, etc. of the waveform may change according to the change in the intensity of the tactile stimulation.

[0098] Also, as shown in FIG. 15, in the present embodiment, the virtual objects superimposed on the stamp 850 or the image of the distributor 800 are not limited to those prepared in advance, and may be composed of a trajectory drawn by the operation of the viewer 900 or the like. Specifically, the viewer 900 may form a stamp 850 indicating one phrase by arranging a plurality of stamps 850a along a trajectory determined by himself / herself (in FIG. 15, a stamp 850 consisting of the phrase "LOVE" is formed). Also, the stamp 850 in FIG. 15 may be an animation image in which a plurality of small stamps 850a appear in the order along the trajectory drawn by the viewer 900 to create the characters "LOVE".

[0099] Also, as shown in FIG. 16, in the present embodiment, the pre-prepared stamp 850 and virtual object may be deformed according to the shape of the object in the real space on which they are superimposed. For example, in the example shown in FIG. 16, according to the shape and size of the face of the distributor 800, the shape and size of the glasses-type stamp 850 are changed, and the stamp 850 is superimposed on the image of the face of the distributor 800.

[0100] Furthermore, in this embodiment, as a result of a lottery, a stamp 850 itself or coins for purchasing the stamp 850 may be generated as a gift for viewer 900 based on a login bonus (visiting bonus), a subscription service, a win / loss prediction, etc.

[0101] For example, in the example shown in FIG. 17, the distributor 800 prepares prize coins and holds a lottery by specifying conditions (specifically, for example, entering a password or an arbitrary comment, etc.) under which the lottery can be conducted. Then, when viewer 900 meets the specified conditions, viewer 900 can participate in the above lottery and win coins by lottery. Then, viewer 900 can purchase the stamp 850 with tactile stimulation effect with the obtained coins.

[0102] Also, for example, in the example shown in FIG. 18, when two distributors 800a and 800b play a predetermined game, viewer 900 predicts the win / loss result. Then, when the prediction is correct, viewer 900 can obtain the stamp 850 with tactile stimulation effect as a gift.

[0103] Furthermore, in this embodiment, for example, by logging in to a predetermined application or website once a day, viewer 900 can obtain a bonus. The bonus can be the stamp 850 with tactile stimulation effect or virtual currency that can purchase the stamp 850. Also, in this embodiment, not only logging in to a predetermined application, etc., but viewer 900 may also obtain a bonus when visiting a predetermined store (detected by a position sensor mounted on the user terminal 700). Also, by continuously logging in, viewer 900 may be able to obtain more stamps 850 or high-value stamps (rare stamps) 850.

[0104] Furthermore, in the present embodiment, the distributor 800 may provide, as a privilege limited to subscribers of the subscription service, a stamp 850 with a tactile stimulation effect or virtual currency that can be used to purchase the stamp 850 by releasing the subscription function. By doing so, the viewer 900 can support the distributor 800 by subscribing to the subscription service of the favorite distributor 800 and can use the above-mentioned subscriber-exclusive privilege.

[0105] As described above, in the first embodiment, by devising the method of presenting the tactile stimulation corresponding to the stamp 850 with a tactile stimulation effect selected by the viewer 900 to the distributor 800 and the image presented to the viewer 900 and the distributor 800 when presenting the tactile stimulation, the viewer 900 can feel as if directly interacting with the distributor 800. That is, the viewer 900 can obtain a real-time and highly interactive added-value experience.

[0106] <<4. Second Embodiment>> In addition, the viewer 900 may enjoy the distributions from a plurality of distributors 800 simultaneously. The second embodiment of the present disclosure applied to such a case will be described with reference to FIGS. 19 and 20. FIG. 19 is an explanatory diagram for explaining this embodiment, and FIG. 20 is an explanatory diagram for explaining an example of display according to this embodiment. For example, as shown in FIG. 19, one viewer 900 may enjoy the distributions from a plurality of distributors 800a, 800b, and 800c simultaneously. In the present embodiment, a method for the viewer 900 to select the distributor 800 that transmits the stamp (control command) 850 with a tactile stimulation effect and a method for selecting which distributor 800's presented tactile stimulation to experience will be described.

[0107] For example, in the present embodiment, as shown in FIG. 19, when viewer 900 is displaying the distribution screens from multiple distributors 800a, 800b, and 800c on the display unit 702 of user terminal 700, viewer 900 may simultaneously transmit the same stamp 850 to all or some of these distributors 800a, 800b, and 800c. Alternatively, in the present embodiment, viewer 900 may determine the distributor 800 to which stamp 850 is to be transmitted by selecting the distribution screen of the distributor 800 for which a tactile stimulus is desired or an icon (not shown) superimposed on the distribution screen. More specifically, for example, after viewer 900 touches the distribution screen of the distributor 800 to be transmitted, and then touches stamp 850, viewer 900 can transmit stamp 850 with a tactile stimulus effect to distributor 800.

[0108] Also, in the present embodiment, voice input may be used as a means for selecting the distributor 800 to be transmitted. For example, calling the name of distributor 800, speaking the position of the distribution screen (right side, lower side, etc.), speaking information about an object located in the real space included in the distribution image of distributor 800 (for example, a chair, the color of the clothes of distributor 800, accessories worn by distributor 800, etc.), or speaking information about a virtual object superimposed on the image of distributor 800 may be done. Also, in the present embodiment, the position of the viewer 900's line of sight may be used as a means for selecting the distributor 800 to be transmitted.

[0109] Also, in the present embodiment, as shown in FIG. 20, for example, it is preferable that an icon 870 indicating whether a tactile presentation can be received is displayed in the upper right of each distribution screen. For example, icon 870a indicates that distributor 800b can receive a tactile control signal, while icon 870b indicates that distributor 800b cannot receive a tactile control signal. Therefore, viewer 900 can determine the distributor 800 to which stamp 850 is to be transmitted by referring to such an icon 870.

[0110] Also, as shown in FIG. 20, the viewer 900 can set whether the tactile presentation device 100 worn by the viewer 900 can receive a tactile stimulus by operating on the icon 870 shown beside the viewer's own comment display 860. Specifically, for example, the viewer 900 can switch and set whether to be able to receive a tactile control signal by long-tapping on the icon 870b shown beside the viewer's own comment display 860. Further, in the present embodiment, the viewer 900 can set whether the tactile presentation device 100 worn by the viewer 900 can receive a tactile stimulus by operating on the icon 870 shown beside the comment display 860 of another viewer 900, so as to set the tactile control signal input by the said other viewer 900.

[0111] Also, in the present embodiment, when the viewer 900 wears the tactile presentation device 100 or a vibration device (not shown) is mounted on the user terminal 700 owned by the viewer 900, the viewer 900 can experience a tactile stimulus corresponding to the stamp 850 transmitted to the distributor 800 by the tactile presentation device 100 or the said vibration device. Note that the tactile stimulus experienced by the viewer 900 may be a tactile stimulus corresponding to the stamp 850 selected by the viewer 900 itself, or may be a tactile stimulus corresponding to the stamp 850 transmitted from another viewer 900 by the distributor 800 selected by the viewer 900. In this case, the viewer 900 can pre-select the distributor 800 corresponding to the tactile stimulus that the viewer wishes to experience by operating on the icon 870 at the upper right of each distribution screen shown in FIG. 20.

[0112] In addition, in the present embodiment, when viewer 900 selects a plurality of distributors 800, the tactile stimuli presented to each distributor 800 may be superimposed (specifically, the waveforms of the respective tactile control signals are superimposed) and presented to the viewer 900. Also, in the present embodiment, the ratio of superimposing the tactile stimuli may be determined according to the degree of interest of the viewer 900 in each distribution (the number of stamps 850 sent, the number of comments 860 sent, the ratio of the time during which the line of sight stays, etc.). Further, in the present embodiment, according to the degree of interest of the viewer 900 in each distribution (the number of stamps 850 sent, the number of comments sent, the ratio of the time during which the line of sight stays, etc.), the intensity, presentation time, etc. of the tactile stimuli presented to the distributor 800 may change.

[0113] Also, in the present embodiment, when the distributor 800 is not wearing the tactile presentation device 100, viewers 900 may be able to send stamps 850 with tactile stimulation effects to each other. In this case, the stamp 850 selected by one viewer 900 may be sent only to the other viewer 900 set by the one viewer 900. Or, only the stamp 850 from one viewer 900 set by the other viewer 900 may be sent to the other viewer 900. Also, in the present embodiment, the viewer 900 may set in advance whether it can receive the tactile control signal from another viewer 900. Specifically, for example, as shown in FIG. 20, the viewer 900 can switch and set whether it can receive the tactile control signal by long-tapping on the icon 870b shown beside its comment display 860. Or, in the present embodiment, the viewer 900 may perform an operation on the icon 870 shown beside the comment display 860 of another viewer 900 to set whether the tactile presentation device 100 worn by itself can receive the tactile stimulation for the tactile control signal input by the other viewer 900. Further, in the present embodiment, for example, as shown in FIG. 20, it is preferable to display 880 the number of viewers 900 who can receive the tactile control signal for the viewer 900.

[0114] Next, an example of the information processing method according to this embodiment will be described with reference to FIGS. 21 and 22. FIGS. 21 and 22 are flowchart diagrams of an example of the information processing method according to this embodiment.

[0115] First, the information processing method at the time of setting will be described with reference to FIG. 21. As shown in FIG. 21, an example of the information processing method according to this embodiment includes a plurality of steps from step S101 to step S102. Details of each step included in an example of the information processing method according to this embodiment will be described below.

[0116] The information processing system 10 sets the validity / invalidity of receiving a tactile control signal of the tactile presentation device 100 of each distributor 800 or viewer 900 based on the setting information of whether the tactile control signal can be received or not from the distributor 800 and the viewer 900 (step S101). Next, the information processing system 10 causes the monitor 204 of the distributor 800 and the user terminal 700 of the viewer 900 to display the setting information (step S102). When the viewer 900 is viewing with a browser, it is preferable that the above setting is stored in the haptics server 300 as a setting value associated with the identification information of the viewer 900.

[0117] Next, the information processing method at the time of presenting a tactile stimulus will be described with reference to FIG. 22. As shown in FIG. 22, an example of the information processing method according to this embodiment includes a plurality of steps from step S201 to step S205. Details of each step included in an example of the information processing method according to this embodiment will be described below.

[0118] The haptics server 300 and the distribution data editing server 400 of the information processing system 10 receive the information of the stamp 850 input from the viewer 900 (the type of the stamp 850, the input viewer 900, the information of the distributor 800 to be transmitted, etc.) (step S201). In this embodiment, when the viewer 900 is browsing with a browser, etc., the haptics server 300 and the distribution data editing server 400 of the information processing system 10 may simultaneously receive the video, audio, and vibration data themselves. Next, the distribution data editing server 400 of the information processing system 10 overlays a video effect on the distribution screen of the corresponding distributor 800 or plays an audio effect according to the received information of the stamp 850 (step S202).

[0119] Next, the haptics server 300 of the information processing system 10 determines whether the tactile presentation device 100 of the distributor 800 or the viewer 900 corresponding to the information of the stamp 850 can receive the tactile control signal corresponding to the stamp 850 (step S203). If the haptics server 300 determines that it can be received (step S203: Yes), it proceeds to step S204, and if it determines that it cannot be received (step S203: No), the process ends.

[0120] The haptics server 300 of the information processing system 10 reads out each parameter of the tactile control signal associated with the stamp 850 (step S204). Next, the haptics server 300 outputs a tactile control signal from the tactile presentation device 100 according to each read parameter (step S205).

[0121] As described above, in the present embodiment, when the viewer 900 enjoys the distributions from a plurality of distributors 800 simultaneously, it is possible to easily select the distributor 800 that transmits the stamp 850 with a tactile stimulation effect. Also, in the present embodiment, the viewer 900 can experience the tactile stimulation presented to one or a plurality of distributors 800, or the viewers 900 can send stamps 850 with tactile stimulation effects to each other. That is, according to the present embodiment, the viewer 900 can obtain a real-time and highly interactive added-value experience.

[0122] <<5. Third Embodiment>> Next, a third embodiment of the present disclosure regarding the setting of each parameter of the tactile control signal and the setting method of the distributor 800 that transmits the stamp 850 with a tactile stimulation effect to the viewer 900 will be described with reference to FIG. 23. FIG. 23 is an explanatory diagram for explaining an input example according to the third embodiment of the present disclosure.

[0123] For example, in the present embodiment, as shown in FIG. 23, the viewer 900 can set the intensity or type of the tactile control signal presented to the distributor 800 by performing a pulling drag operation (specifically, after performing a drag operation from the touch operation start position of the finger toward the lower part of the screen and then releasing the operation) on the icon 890 displayed on the display unit 702 of the user terminal 700. In the present embodiment, the intensity or type of the tactile control signal presented to the distributor 800 is determined according to the distance, time, or speed at which the viewer 900 pulls the icon 890. Also, at this time, a tactile stimulation may be presented to the distributor 800 at the timing when the viewer 900 releases the finger from the icon 890. Further, at this time, a tactile stimulation may be presented to the distributor 800 associated with the position corresponding to the vector 910 opposite to the vector corresponding to the trajectory of the drag operation.

[0124] Next, an information processing method according to this embodiment will be described with reference to Fig. 24. Fig. 24 is a flow chart of an example of the information processing method according to this embodiment. As shown in Fig. 24, the example of the information processing method according to this embodiment includes a plurality of steps from step S301 to step S310. Each step included in the example of the information processing method according to this embodiment will be described in detail below.

[0125] The user terminal 700 of the information processing system 10 detects an input start position (a position where a touch operation is started) of the viewer 900 on the screen of the user terminal 700 (step S301). Next, the haptics server 300 measures the distance between the current input position and the input start position (step S302).

[0126] Next, the user terminal 700 of the information processing system 10 judges whether the distance measured in step S302 is equal to or greater than a predetermined distance (step S303). If the user terminal 700 judges that the distance is equal to or greater than the predetermined distance (step S303: Yes), it proceeds to step S304, and if the user terminal 700 judges that the distance is not equal to or greater than the predetermined distance (step S303: No), it ends the process.

[0127] Next, the user terminal 700 of the information processing system 10 transitions to a drag operation input mode (step S304). The user terminal 700 detects the input end position (the position where the finger is removed from the screen) and measures the distance between the input end position and the input start position (step S305).

[0128] Next, the user terminal 700 of the information processing system 10 judges whether the distance measured in step S305 is equal to or greater than a predetermined distance (step S306). If the user terminal 700 judges that the distance is equal to or greater than the predetermined distance (step S306: Yes), it proceeds to step S307, and if the user terminal 700 judges that the distance is not equal to or greater than the predetermined distance (step S306: No), it ends the process.

[0129] Next, the user terminal 700 of the information processing system 10 determines the distribution screen of the distributor 800 that is on the vector from the input end position toward the input start position (step S307). Then, the user terminal 700 transmits the screen information of the distributor 800, the information of the stamp 850, the operation distance information, etc. that are the targets to the haptics server 300 or the like (step S308). Next, the haptics server 300 determines the intensity of the tactile control signal according to the operation distance information (step S309). Next, the distribution data editing server 400 displays a video effect on the distribution screen of the target distributor 800, and the haptics server 300 outputs a tactile control signal from the tactile presentation device 100 (step S310).

[0130] Also, in the present embodiment, other setting methods may be used. Other setting methods will be described with reference to FIG. 25. FIG. 25 is an explanatory diagram for explaining an input example according to the present embodiment.

[0131] As shown in FIG. 25, the viewer 900 may perform a swipe operation (quickly swipe from the touch start position of the finger toward the upper part of the screen (or the distribution screen of the distributor) and then release) on the icon 890 displayed on the display unit 702 of the user terminal 700. Also in this case, the intensity of the tactile control signal presented by the distributor 800 is determined according to the distance or speed of the swipe operation.

[0132] Next, the information processing method according to the present embodiment will be described with reference to FIG. 26. FIG. 26 is a flowchart diagram of an example of the information processing method according to the present embodiment. As shown in FIG. 26, an example of the information processing method according to the present embodiment includes a plurality of steps from step S401 to step S410. Details of each step included in an example of the information processing method according to the present embodiment will be described below.

[0133] When the user terminal 700 of the information processing system 10 receives a predetermined trigger (such as a specific stamp 850 being input, the distributor 800 switching modes, the viewer 900 switching modes, etc.), it switches to the drag input mode (step S401). Next, the user terminal 700 detects an operation of touching the viewer 900's screen and starts the input (step S402).

[0134] The information processing system 10 (specifically, the user terminal 700 or the distribution data editing server 400) determines the length of the trajectory to be drawn and the number of stamps to be used for drawing according to the distance that the viewer 900 moves while touching the screen, and performs the drawing (step S403).

[0135] Next, the user terminal 700 of the information processing system 10 determines whether the non-operation time has elapsed for a predetermined time or more (step S404). If the user terminal 700 determines that it is a predetermined time or more (step S404: Yes), it proceeds to step S405. If it determines that it is not a predetermined time or more (step S404: No), it returns to the process of step S402.

[0136] The user terminal 700 of the information processing system 10 measures the total distance of the drawn trajectory (or the number of stamps used) (step S405).

[0137] Next, the user terminal 700 of the information processing system 10 determines whether the distance measured in step S405 is equal to or greater than a predetermined distance (or a predetermined number) (step S406). If the user terminal 700 determines that it is a predetermined distance or more (step S406: Yes), it proceeds to step S407. If it determines that it is not a predetermined distance or more (step S406: No), it proceeds to the process of step S408.

[0138] Based on the determination in step S406, the user terminal 700 or the haptics server 300 of the information processing system 10 switches the type of the haptic control signal to be presented (step S407). The haptics server 300 determines the haptic control signal to be presented and reads out each parameter of the haptic control signal (step S408). The haptics server 300 determines the intensity of the haptic control signal according to the distance of the trajectory or the number of stamps 850 used, etc. (step S409). Next, the haptics server 300 outputs a haptic control signal from the haptic presentation device 100 (step S410).

[0139] Also, in the present embodiment, the viewer 900 may set by tapping the back surface facing the display unit 702 of the user terminal 700. Specifically, the viewer 900 selects a stamp 850 by a tap operation on the display unit 702 and determines the timing for presenting a haptic stimulus by a tap operation on the back surface. Note that the tap operation on the back surface can be detected by an acceleration sensor 728 built in the user terminal 700. Also, in the present embodiment, the timing for presenting a haptic stimulus may be determined at a timing when tap operations by a plurality of viewers 900 are synchronized.

[0140] Also, in the present embodiment, not only can the haptic stimulus be edited and set by an operation on the display unit 702, but for example, various sensors and other devices built in the user terminal 700 may be used. In the present embodiment, for example, the viewer 900 may edit the waveform, intensity change, etc. of the haptic stimulus by performing a touch operation such as giving vibration or a motion operation such as changing the holding angle on the user terminal 700. Specifically, in the present embodiment, the motion operation is detected by a gyro sensor 726 or an acceleration sensor 728 built in the user terminal 700, and the waveform of the haptic stimulus, etc. can be determined based on the detected sensing data.

[0141] In addition, in this embodiment, it is also possible to recognize the shapes and characters (trajectories) drawn by the viewer 900, extract the text from the recognition result, and select the stamps 850 and tactile control signals associated with the extracted text. In that case, not only operations on the touch panel (not shown) of the user terminal 700 but also drawing characters and shapes in the air may be allowed. In this case, for example, an IMU sensor (not shown) attached to the arm of the viewer 900 may be used to recognize the characters and shapes.

[0142] Also, as shown in FIG. 27, tactile control signals can also be edited or the like by voice input. FIG. 27 is an explanatory diagram for explaining an input example according to this embodiment. For example, the viewer 900 speaks a voice with a predetermined rhythm to the user terminal 700, and the user terminal 700 detects the speech with a microphone 724 built therein. Then, the haptics server 300 may use the sensing data detected by the microphone 724 to set the frequency of the tactile control signal according to the rhythm of the voice. Further, in this embodiment, for example, as shown in FIG. 27, while the viewer 900 is touching the microphone icon 892, the voice input mode may be switched (the voice input mode ends when released), or the start and end of the input may be switched each time the microphone icon 892 is touched.

[0143] For example, when it is desired to continuously present the tactile stimuli associated with a plurality of stamps 850 to the distributor 800 in accordance with the rhythm of music, voice input can be an effective means because the timing of the presentation can be determined by voice instead of a tap operation. Also, when a predetermined music or rhythm is output toward the viewer 900 and the rhythm and timing of the music match those of the viewer 900's spoken voice, the intensity of the tactile control signal may increase or other parameters may change.

[0144] Also, in this embodiment, when viewer 900 utters onomatopoeia (such as "tsuntsun" (stuck-up), "gaan" (bummer), "zowazowa" (tremble), etc.) or the name of stamp 850 (such as "heart", "bouquet", etc.), the words may be extracted from the uttered voice, and stamp 850 or a tactile stimulus associated with the extracted words may be selected. At this time, the intensity of the tactile control signal may be increased or other parameters may be changed according to the sound pressure (volume) of the uttered voice.

[0145] As described above, in the embodiment of the present disclosure, by various input methods, viewer 900 can set the presentation timing of the tactile stimulus, the presenter 800 to be presented, and each parameter of the tactile control signal. Since viewer 900 can easily make settings in this way, according to this embodiment, viewer 900 can obtain a real-time and highly interactive added-value experience.

[0146] <<6. Fourth Embodiment>> Also, in the present disclosure, when a plurality of viewers 900 select a predetermined type of stamp 850 within a predetermined time, it may change to a new stamp 850 and a new tactile stimulus according to the combination of the stamps 850. The fourth embodiment of the present disclosure will be described with reference to FIG. 28. FIG. 28 is an explanatory diagram for explaining an input example according to this embodiment.

[0147] For example, as shown in FIG. 28, when viewer 1 selects the stamp 850a of the bun, the clock starts, and the user terminal 700 displays the remaining time. Next, if viewer 2 selects the stamp 850b of meat and further viewer 3 selects the stamp 850c of lettuce within a predetermined time, the stamps 850a, 850b, and 850c are combined into a hamburger stamp 850d. And the tactile stimulus associated with the stamp 850d is different from the tactile stimuli associated with the other stamps 850a, 850b, and 850c. That is, in the present embodiment, when a plurality of viewers 900 select stamps 850 of a predetermined type within a predetermined time and predetermined conditions are met, the video effect of the stamp 850 changes or the tactile stimulus presented to the distributor 800 changes.

[0148] Next, the information processing method according to the present embodiment will be described with reference to FIG. 29. FIG. 29 is a flowchart diagram of an example of the information processing method according to the present embodiment. As shown in FIG. 29, an example of the information processing method according to the present embodiment includes a plurality of steps from step S501 to step S507. Hereinafter, the details of each step included in the example of the information processing method according to the present embodiment will be described.

[0149] The distribution data editing server 400 of the information processing system 10 receives the information of the stamp 850 (identification information of the stamp 850) transmitted from the user terminal 700 (step S501).

[0150] Next, the distribution data editing server 400 of the information processing system 10 determines whether the selected stamp 850 corresponds to a composition (step S502). If the distribution data editing server 400 determines that it corresponds to a composition (step S502: Yes), it proceeds to step 503, and if it determines that it does not correspond to a composition (step S502: No), the process ends.

[0151] Next, the distribution data editing server 400 reads out and distributes the waiting time corresponding to the selected stamp 850 (step S503). At this time, the distributed user terminal 700 presents the remaining time to each viewer 900 by text-displaying it, displaying a time bar, or the like based on the waiting time.

[0152] Next, the distribution data editing server 400 determines whether the stamps 850 input from other viewers 900 within a predetermined time satisfy a predetermined synthesis condition (step S504). If the distribution data editing server 400 determines that the synthesis condition is satisfied (step S504: Yes), it proceeds to step 505. If it determines that the synthesis condition is not satisfied (step S504: No), the process ends.

[0153] When the distribution data editing server 400 of the information processing system 10 satisfies the synthesis condition, it distributes the identification information and video effects of the special stamp 850 to be displayed, and causes the user terminal 700 to display the special stamp 850 (step S505). At this time, the image of the previously input stamp 850 displayed on the user terminal 700 may be erased. Further, the haptics server 300 reads out the tactile control signal associated with the special stamp 850 selected in step S505 (step S506). Next, the haptics server 300 outputs a tactile control signal (a signal different from the tactile control signal corresponding to the stamp 850) from the tactile presentation device 100 (step S507). In the present embodiment, in step S506, the haptics server 300 may receive the identification information associated with the special stamp 850 selected in step S505, and read out the tactile control signal based on the received identification information.

[0154] As described above, in the embodiments of the present disclosure, it is possible to transmit a new stamp 850 and a tactile stimulus to the distributor 800 in cooperation with other viewers 900. Since such transmission can be performed, according to this embodiment, the viewer 900 can obtain a highly interactive and added-value experience in real time.

[0155] <<7. Fifth Embodiment>> By the way, when a large number of stamps 850 are simultaneously transmitted to a single distributor 800, the tactile stimulation unit 106 of the tactile presentation device 100 attached to the distributor 800 may generate heat due to its operation and may cause an abnormal operation. Therefore, in the following, as the fifth embodiment of the present disclosure, an embodiment will be described in which restrictions are imposed on the tactile stimuli to be reproduced simultaneously, and the burden on the tactile stimulation unit 106 of the tactile presentation device 100 can be avoided.

[0156] For example, in the present embodiment, the haptics server 300 of the information processing system 10 compares the number of tactile stimuli (stamps 850) to be reproduced simultaneously, the intensity of the tactile control signal, the presentation time, or the heat of the tactile stimulation unit 106, etc., with a threshold value determined in advance according to the characteristics of the tactile stimulation unit 106. Then, when the haptics server 300 determines that the threshold value has been exceeded, it restricts the tactile stimuli to be reproduced simultaneously. Further, when a restriction is imposed, the haptics server 300 may temporarily stop the presentation of the tactile stimuli, stack the tactile stimuli of each stamp 850, and resume the presentation of the tactile stimuli when the above parameters fall below the threshold value. In the present embodiment, the temperature of the tactile stimulation unit 106 is assumed to be transmitted from the tactile presentation device 100 to the haptics server 300.

[0157] Also, when temporarily stopping, it is preferable to notify the viewer 900 of the primary stop. For example, a display example of the notification will be described with reference to FIGS. 30 and 31. FIGS. 30 and 31 are explanatory diagrams for explaining a display example according to the present embodiment.

[0158] For example, as shown in FIG. 27, the user terminal 700 may only stop presenting the tactile stimulus and may continue to display the stamp 852 as it is. Also, at this time, the user terminal 700 may display an icon 870b indicating pause and may also display a message 850 indicating that there are no stamps that the viewer 900 can select.

[0159] Also, when not only the tactile stimulus but also the video display is stopped, the user terminal 700 may display a message as shown in FIG. 31. Also, at this time, the user terminal 700 may display an icon 872 indicating the stopped state or may display the number of stamps 850 or tactile stimuli in the stack as shown by the reference numeral 874.

[0160] Next, the information processing method according to the present embodiment will be described with reference to FIG. 32. FIG. 32 is a flowchart diagram of an example of the information processing method according to the present embodiment. As shown in FIG. 32, an example of the information processing method according to the present embodiment includes a plurality of steps from step S601 to step S607. Hereinafter, the details of each step included in the example of the information processing method according to the present embodiment will be described.

[0161] The haptics server 300 and the distribution data editing server 400 of the information processing system 10 receive the information (ID, etc.) of the stamp 850 transmitted from the user terminal 700 (step S601).

[0162] Next, the haptics server 300 of the information processing system 10 determines whether the number of stamps 850 being played back exceeds a threshold value (step S602). If the haptics server 300 determines that it exceeds the threshold (step S602: Yes), it proceeds to step S606, and if it determines that it does not exceed the threshold (step S602: No), it proceeds to step S603.

[0163] The distribution data editing server 400 of the information processing system 10 distributes the received stamp 850 to other viewers 900 and the distributor 800 (step S603). The haptics server 300 of the information processing system 10 reads out the parameters of the tactile control signal corresponding to the stamp 850 and outputs them from the tactile presentation device 100 (step S604). The distribution data editing server 400 of the information processing system 10 reads out and displays the video effect corresponding to the stamp 850 (step S605).

[0164] The haptics server 300 of the information processing system 10 includes information indicating that the tactile presentation device 100 is unable to present tactile stimuli in the distributed information (step S606). The user terminal 700 displays an icon or text indicating that the tactile presentation device 100 is unable to present tactile stimuli (step S607).

[0165] As described above, according to the present embodiment, it is possible to limit the tactile stimuli reproduced simultaneously and avoid imposing a burden on the tactile stimulation unit 106 of the tactile presentation device 100.

[0166] <<8. Sixth Embodiment>> By the way, in the present disclosure, for example, when the distributor 800 is not wearing the tactile presentation device 100, when the distributor 800 does not want to receive tactile stimuli, when the battery capacity of the tactile presentation device 100 is low, etc., instead of tactile stimuli, other modalities, for example, the vibration of tactile stimuli may be presented by video. Therefore, the presentation method in such a case will be described as the fifth embodiment of the present disclosure with reference to FIGS. 33 to 37. FIGS. 33 to 37 are explanatory diagrams for explaining the display examples according to the present embodiment.

[0167] In this embodiment, for example, as shown in FIG. 33, according to the frequency of the tactile control signal, a video effect may be used to reciprocate the distributed video 950 or a part of the video 950 to the left and right. Specifically, for example, the period of reciprocating the video 950 becomes shorter as the frequency of the tactile control signal is higher. Also, in this embodiment, the area of the video 950 to be reciprocated may be determined according to the frequency of the tactile control signal exceeding the threshold value. For example, as shown in FIG. 33, the lower the frequency of the tactile control signal, the more the video near the lower part of the video 950 is reciprocated.

[0168] Also, as shown in FIG. 34, when there are a plurality of frequencies of the tactile control signal exceeding the threshold value (for example, when two frequency regions, a relatively low frequency region and a high frequency region, exceed the threshold value), the lower part and the upper part of the distributed video 950 are reciprocated.

[0169] Also, in this embodiment, when reciprocating the video 950 to the left and right, it may be made continuous or discontinuous at the turning point, or the degree may be switched. For example, as shown in FIG. 35, when the vibration frequency component is relatively low, the turning point of the video deformation is made continuous (region 960b), and when it is relatively high, the turning point of the video deformation is made discontinuous (region 960a). By doing so, in the video 950, a video effect with a softer impression is applied as the frequency is lower. Note that in this embodiment, according to the frequency of the tactile control signal, not only the frequency but also the interval, waveform, etc. may change when reciprocating the video 950 to the left and right.

[0170] More specifically, the example of FIG. 35 will be described using FIG. 36 showing the position of the video for each unit time at the turning point of the video deformation. As shown in FIG. 36, in the example of FIG. 35, the video position changes in proportion to the passage of unit time, and when the video position reaches the end point, it starts to change in the reverse direction.

[0171] When the relatively high frequency shown on the left side of FIG. 36 (region 960a) is the case, the video position is determined according to the following formulas (1) and (2). First, the video position is determined according to the elapsed time t and formula (1), and when the end is reached, the video position x is determined according to the elapsed time t and formula (2). Note that α in formulas (1) and (2) is an arbitrary coefficient.

Number

[0172] When the relatively low frequency shown on the right side of FIG. 36 (region 960b) is the case, the video position x is determined according to the elapsed time t and formula (3). Note that α and β in formula (3) are arbitrary coefficients.

Number

[0173] In this embodiment, the determination of the video position in the folding of the video deformation can be performed based on the frequency component, intensity, type of video content, type of video effect, and metadata associated with the video content and video effect of the tactile control signal.

[0174] Also, in this embodiment, not only can the video 950 be reciprocated left and right over the entire area, but only a predetermined area 970 in the video 950 may be reciprocated left and right. For example, in FIG. 37, the area 970 where the distributor 800 is displayed may be reciprocated left and right. Alternatively, in this embodiment, only the image of the stamp 850 superimposed on the image of the distributor 800 may be reciprocated left and right.

[0175] Also, in this embodiment, when there is metadata associated with the video content, the type of the above video deformation may be determined based on the metadata. For example, when metadata such as "falling" is given to the stamp 850 having a video effect such that an object falls from above, the video 950 may be reciprocated up and down instead of left and right.

[0176] Next, the information processing method according to this embodiment will be described with reference to FIG. 38. FIG. 38 is a flowchart diagram of an example of the information processing method according to this embodiment. As shown in FIG. 38, an example of the information processing method according to this embodiment includes a plurality of steps from step S701 to step S706. Details of each step included in the example of the information processing method according to this embodiment will be described below.

[0177] The user terminal 700 of the information processing system 10 transmits the information of the stamp 850 to the haptics server 300 (step S701). The haptics server 300 reads out the tactile control signal corresponding to the stamp 850 (step S702).

[0178] Next, the haptics server 300 of the information processing system 10 determines whether the distributor 800 and the viewer 900 have the tactile presentation device 100 (step S703). If the haptics server 300 determines that they have it (step S703: Yes), it proceeds to step S704, and if it determines that they do not have it (step S703: No), it proceeds to step S705.

[0179] The haptics server 300 of the information processing system 10 outputs the tactile control signal corresponding to the stamp 850 from the tactile presentation device 100 (step S704).

[0180] The haptics server 300 of the information processing system 10 analyzes the frequency components and intensity included in the tactile control signal (step S705). The distribution data editing server 400 of the information processing system 10 modifies the video 950 based on the analysis result in step S705 (step S706).

[0181] Also, in this embodiment, not only the video 950 is shaken left and right, but also the audio data distributed to the viewer 900 and the distributor 800 may be processed to shake the sound image up and down or the like by the sound image localization technology.

[0182] In addition, in the present embodiment, when the distributor 800 is not wearing the tactile presentation device 100, a blower-type device may output air (for example, warm air, cold air, etc.), or the temperature of a temperature presentation device (equipped with a Peltier element, etc.) worn on the body may be changed. Further, in the present embodiment, an aromatic agent may be presented by a dispenser that discharges the aromatic agent.

[0183] As described above, in the present embodiment, even when the distributor 800 is not wearing the tactile presentation device 100 or the like, the vibration of the tactile stimulus can be presented by other modalities. Therefore, according to the present embodiment, the viewer 900 can obtain a real-time and highly interactive added-value experience.

[0184] <<9. Seventh Embodiment>> Next, referring to FIGS. 39 to 41, a seventh embodiment of the present disclosure will be described. FIGS. 39 to 41 are explanatory diagrams for explaining display examples according to the present embodiment.

[0185] In the present embodiment, for example, when the tactile presentation device 100 worn by the distributor 800 has a plurality of tactile stimulus units 106, each tactile stimulus unit 106 may be made exclusive to the viewer 900. That is, viewers 900 other than the viewer 900 who has exclusive use cannot present a tactile stimulus using the corresponding tactile stimulus unit 106. For example, in order to occupy the tactile stimulus unit 106, the viewer 900 may pay a predetermined amount, make a bulk purchase, or subscribe. Further, in the present embodiment, the presentation of the tactile stimulus unit 106 located around the tactile stimulus unit 106 designated by the high-value stamp 850 may be stopped. Also, when the viewers 900 are divided into several groups (divided by region, rank, etc.), the tactile stimulus units 106 may be assigned to each group.

[0186] For example, in FIG. 39, the left side shows a display example of the user terminal 700 of the viewer 900 that occupies a part of the tactile stimulation unit 106, and the right side shows a display example of the user terminal 700 of the viewer 900 that does not occupy a part of the tactile stimulation unit 106. In this case, the user terminal 700 of the viewer 900 that does not occupy a part of the tactile stimulation unit 106 displays that it cannot select the stamp 850 corresponding to the occupied tactile stimulation unit 106.

[0187] Also, for example, in FIG. 40, the left side shows a display example of the user terminal 700 of the viewer 900 that occupies a part of the tactile stimulation unit 106, and the right side shows a display example of the user terminal 700 of the viewer 900 that does not occupy a part of the tactile stimulation unit 106. In this case, the user terminal 700 of the viewer 900 that does not occupy a part of the tactile stimulation unit 106 displays that the occupied tactile stimulation unit 106 cannot be selected.

[0188] Also, for example, in FIG. 41, the left side shows a display example of the user terminal 700 of the viewer 900 that occupies all of the tactile stimulation unit 106, and the right side shows a display example of the user terminal 700 of the viewer 900 that does not occupy all of the tactile stimulation unit 106. In this case, the user terminal 700 of the viewer 900 that does not occupy all of the tactile stimulation unit 106 displays that the stamp 850 cannot be selected because all of the tactile stimulation units 106 are occupied. Alternatively, the display example in FIG. 41 may be such that the left side shows a display example of the user terminal 700 of the viewer 900 who has paid a high amount or has a high rank (e.g., premium course, gold rank, etc.), and the right side shows a display example of the user terminal 700 of another viewer 900 when the viewer 900 inputs a tactile control signal. In this case, after the presentation of the tactile stimulation related to the tactile control signal input by the viewer 900 who has paid a high amount or the like is completed, other viewers 900 can input a tactile control signal. Alternatively, in the present embodiment, immediately after a viewer 900 who has paid a low amount or has a low rank (e.g., basic course, bronze rank, etc.) inputs a tactile control signal, if a viewer 900 who has paid a high amount or has a high rank inputs a tactile control signal, the tactile control signal input by the viewer 900 who has paid a low amount or has a low rank may be immediately canceled.

[0189] Also, in the present embodiment, the tactile stimulation unit 106 capable of presenting tactile stimulation may be switched according to the category of the content being distributed. For example, in the case of makeup distribution or game live distribution, the tactile stimulation unit 106 corresponding to the hand or arm used in makeup or the game is set not to present tactile stimulation. Also, in the case of distribution related to a flea market (or live commerce), since the vest-type tactile presentation device 100 is removed to try on clothes, the tactile stimulation unit 106 corresponding to the torso is set not to present tactile stimulation. In this case, when the tactile presentation device 100 is worn, the stacked tactile stimulations may be presented in order.

[0190] In addition, in the present embodiment, in a competitive distribution or the like, when the types of the tactile presentation devices 100 worn by the distributor 800 and the viewer 900 are different (for example, the reproducible frequency bands are different or the number of mounted actuators is different), only the tactile stimulation unit 106, which is a common part (a common reproducible frequency region, the minimum value of the outputtable vibration intensity, a common actuator part), may be used to present the tactile stimulation.

[0191] Specifically, in order to match the tactile control signal for the tactile stimulation unit 106 optimized for the larger number of mounted tactile stimulation units 106 to the smaller number of mounted tactile stimulation units 106, the control signals given to each tactile stimulation unit 106 may be merged and output from a specific tactile stimulation unit 106. Specifically, the tactile stimulations planned to be presented on the shoulder and the arm are merged and output from one tactile stimulation unit 106 mounted on the wrist. In addition, when merging the tactile stimulations, priorities may be set in advance for each body part or each tactile stimulation unit 106 in order to determine the mixing ratio. Specifically, for example, when the priority of the shoulder is set to 4 and the priority of the arm is set to 1 (assuming that the larger the number, the higher the priority), the intensity of the tactile control signal output from the tactile stimulation unit 106 corresponding to the wrist is the intensity of the tactile control signal given to the shoulder × 0.8 + the intensity of the tactile control signal given to the arm × 0.2.

[0192] In addition, in the present embodiment, meanings (such as positive or negative) can be set in advance for each tactile stimulation unit 106, and by collating with the metadata associated with the stamp 850, the tactile stimulation unit 106 having the meaning corresponding to the metadata may be determined as the tactile stimulation unit 106 that presents the tactile stimulation.

[0193] Next, the information processing method according to the present embodiment will be described with reference to FIG. 42. FIG. 42 is a flowchart diagram of an example of the information processing method according to the present embodiment. As shown in FIG. 42, an example of the information processing method according to the present embodiment includes a plurality of steps from step S801 to step S806. Hereinafter, the details of each step included in the example of the information processing method according to the present embodiment will be described.

[0194] The haptics server 300 of the information processing system 10 detects the category of distribution (step S801). The haptics server 300 reads the information of the tactile stimulation unit 106 that cannot present tactile sensations for the detected category (step S802). Next, the haptics server 300 reads the information of the tactile stimulation unit 106 associated with the stamp 850 (step S803).

[0195] Next, the haptics server 300 of the information processing system 10 determines whether it includes the tactile stimulation unit 106 that cannot present tactile sensations (step S804). If the haptics server 300 determines that it includes it (step S804: Yes), it proceeds to step S805, and if it determines that it does not include it (step S804: No), the process ends.

[0196] The haptics server 300 of the information processing system 10 sets the stamp 850 to a non-selectable state and distributes the fact that it has become non-selectable (step S805). The user terminal 700 of the information processing system 10 changes to a display suggesting that the stamp 850 is non-selectable (step S806).

[0197] As described above, in the embodiment of the present disclosure, the viewer 900 can occupy the tactile stimulation unit 106, change the tactile stimulation unit 106 that presents tactile sensations according to the distributed content, or change the tactile stimulation according to the type of the tactile presentation device 100. By doing so, according to this embodiment, the viewer 900 can obtain a real-time and highly interactive added-value experience.

[0198] <<10. Eighth Embodiment>> In the eighth embodiment of the present disclosure, by viewers 900 inputting the stamp 850 simultaneously while matching the timing and rhythm, it may be switched to something different from the tactile stimulation associated with the stamp 850 and presented.

[0199] Also, in this embodiment, a time range (from what time to what time, for how many minutes) during which the stamp 850 can be selected and the number of throws within that time may be set. In this embodiment, for example, by bidding, each viewer 900 may determine the number of minutes during which the stamp 850 can be input.

[0200] <<11. Ninth Embodiment>> Also, in this embodiment, in the case of a distribution with a relatively small number of viewers 900 or a distribution where the number of transmissions of the stamp 850 is not active (the number of transmitted stamps 850 per unit time is less than the threshold value), the chat text or the like may be used as a trigger to present a tactile stimulus corresponding to the chat text to the distributor 800. By doing so, the motivation of the distributor 800 for the distribution can be enhanced. For example, when a comment including a keyword or emoji preset by the distributor 800 is posted by the viewer 900, it may be detected and a tactile stimulus may be presented. Further, depending on the number of matching keywords or the number of matching emojis in the comment, the type of video effect may be switched or the intensity of the presented tactile stimulus may be increased. In this case, as the timing for setting the keyword, it may be before starting the distribution or may be set appropriately during the distribution (for example, in a scene where a quiz is given, the correct keyword is set each time).

[0201] Also, in this embodiment, a tactile stimulus may be presented triggered by a new viewer 900 entering the distribution room of the distributor 800. Also, in this embodiment, a tactile stimulus may be presented triggered by the timing when a plurality of viewers 900 post the same comment, a comment including the same emoji, or the same stamp 850 (however, not corresponding to the tactile stimulus) simultaneously (within a certain time).

[0202] <<12. Summary>> As described above, according to each embodiment of the present disclosure, the viewer 900 can obtain a real-time and highly interactive added-value experience.

[0203] Note that the embodiments of the present disclosure are not only applicable to live distribution and the like as described above, but can also be applied to, for example, stamps 850 exchanged on an SNS (Social Networking Service). In that case, for example, instead of the tactile presentation device 100, the user terminal 700 can vibrate or the like to give a tactile stimulus to the person who sent the stamp 850.

[0204] <<13. Modification Example 1 of Information Processing System 10 of the Present Disclosure>> Furthermore, with reference to FIGS. 43 to 50, a modification example of the information processing system 10 according to the embodiment of the present disclosure will be described. FIGS. 43 to 50 are system diagrams showing schematic configuration examples of the information processing system 10 according to Modification Example 1 of the embodiment of the present disclosure.

[0205] First, in the information processing system 10 shown in FIG. 43, the haptics server 300a also undertakes the functions of the above-described distribution data editing server 400 and live distribution server 500. In this case, the management of the haptics server 300a can be performed by one operator.

[0206] Also, in the information processing system 10 shown in FIG. 44, a haptics server 600 may be included as undertaking part of the function of the storage unit 318 of the haptics server 300. In this case, the haptics server 600 is associated with the identification information (ID) of each stamp 850 and stores a pre-generated tactile data library (such as a vibration waveform pattern). Further, the haptics server 600 may store information such as the number, position, frequency characteristics, and maximum input voltage of the tactile stimulation units 106 of the tactile presentation device 100 as profile information of the tactile presentation device 100.

[0207] Also, in the information processing system 10 shown in FIG. 45, the haptics server 300 also undertakes the functions of the above-described distribution data editing server 400. In this case, the operator who manages the presentation for the tactile stimulus and the operator who manages the distribution can be different operators.

[0208] Also, in the information processing system 10 shown in FIG. 46, different from the example of FIG. 45, the information stored in the haptics server 600 is provided to the haptics server 300 via the live distribution server 500.

[0209] Also, in the information processing system 10 shown in FIG. 47, different from the example of FIG. 45, the information stored in the haptics server 600 is provided to the haptics server 300 via the live distribution server 500 and the distribution data editing server 400.

[0210] Also, in the information processing system 10 shown in FIG. 48, different from the example of FIG. 43, the haptics server 600 may be included as undertaking a part of the functions of the storage unit 318 of the haptics server 300.

[0211] Also, in the information processing system 10 shown in FIG. 49, the haptics server 600 may cooperate with the live distribution server 500 to perform distribution to the user terminal 700.

[0212] Also, in the information processing system 10 shown in FIG. 50, the live distribution server 500 acquires the information of the stamp 850 from the user terminal 700 and performs distribution of video and the like. Also, the haptics server 300 acquires the identification information (ID) of each stamp 850 from the live distribution server 500. Further, the haptics server 300 refers to the above identification information and generates a tactile control signal associated with each stamp 850 based on the information in the haptics server 600. In FIG. 50, it is assumed that the haptic data is the identification information of the tactile control signal (waveform data) corresponding to the stamp 850 or the tactile control signal itself.

[0213] <<14. Variation 2 of the information processing system 10 of the present disclosure>> The information processing system 10 according to an embodiment of the present disclosure may be applied to a system composed of a plurality of devices on the premise of connection to a network (or communication between devices), such as cloud computing (cloud). Therefore, variations in the positions of the respective servers on the network (cloud side or local side) in the information processing system 10 according to the present embodiment will be described with reference to FIGS. 51 to 54. FIGS. 51 to 54 are system diagrams showing schematic configuration examples of the information processing system 10 according to Variation 2 of the embodiment of the present disclosure. In these figures, the local side control device 350 corresponds to the haptics server 300 in the above-described embodiment of the present disclosure, and the live distribution server 500 corresponds to the live distribution server 500 and the distribution data editing server 400 in the above-described embodiment of the present disclosure. Further, in these figures, the haptic data is assumed to be identification information of a tactile control signal (waveform data) corresponding to the stamp 850 or the tactile control signal itself.

[0214]

[0215] ​Next, in the information processing system 10 shown in FIG. 52, unlike the information processing system 10 shown in FIG. 51, it includes a haptics server 600 arranged on the cloud side. The local side control device 350 is composed of a PC or the like, and can acquire the identification information of the tactile control signal (waveform data) associated with the identification information (stamp information) of the stamp 850 from the haptics server 600. In the information processing system 10 shown in FIG. 52, the haptics server 600 may be a local device installed on the distributor 800 side.

[0216] Next, in the information processing system 10 shown in FIG. 53, similar to the information processing system 10 shown in FIG. 52, it includes a haptics server 600 arranged on the cloud side. However, different from the information processing system 10 shown in FIG. 52, in the information processing system 10 shown in FIG. 53, the local side control device 350 acquires the identification information of the tactile control signal (waveform data) associated with the identification information (stamp information) of the stamp 850 from the haptics server 600 via the live distribution server 500.

[0217] Also, in the information processing system 10 shown in FIG. 54, the local side control device 350 is installed on the distributor 800 side. On the other hand, the live distribution server 500 and the haptics server 600 are arranged on the cloud side. Different from the information processing system 10 shown in FIG. 53, in the information processing system 10 shown in FIG. 54, the local side control device 350 acquires videos, audio signals, etc. associated with the identification information (stamp information) of the stamp 850 from the live distribution server 500 via the haptics server 600.

[0218] <<15. Output method of tactile stimulus>> In the embodiment of the present disclosure described above, the haptics server 300 refers to the identification information of the received stamp 850, and generates / outputs a tactile control signal corresponding to the stamp 850 based on the data associated with the stored identification information in advance. Therefore, a specific method for presenting a tactile stimulus in the embodiment of the present disclosure will be described with reference to FIG. 55. FIG. 55 is an explanatory diagram for explaining a method for presenting a tactile stimulus according to an embodiment of the present disclosure.

[0219] In the present embodiment, for example, as shown in FIG. 55, waveform data of a tactile stimulus associated with the identification information of the stamp 850 is prepared and stored in the haptics server 300 or the haptics server 600 so as to correspond to the tactile stimulus unit 106 that presents the tactile stimulus. Then, the haptics server 300 or the haptics server 600 transmits a plurality of waveform data associated with the identification information of the received stamp 850 to the drive amplifier / interface 200. Further, the drive amplifier / interface 200 can present a tactile stimulus by driving based on the received waveform data for the designated tactile stimulus units 106a to 106f.

[0220] Note that in the above description, it has been described that the waveform data is prepared so as to correspond to the tactile stimulus unit 106 that presents the tactile stimulus, but the present embodiment is not limited thereto. For example, waveform data of a tactile stimulus associated with the identification information of the stamp 850 is prepared and stored in the haptics server 300 or the haptics server 600 without being associated with the tactile stimulus unit 106 that presents the tactile stimulus. Further, the haptics server 300, the haptics server 600, or the drive amplifier / interface 200 acquires information on the specifications of the tactile presentation device 100 (the number, position, maximum vibration intensity given, etc. of the tactile stimulus units 106) from the tactile presentation device 100, or is assumed to have stored the information in advance.

[0221] Then, the haptics server 300 or the haptics server 600 associates (matches) each of the plurality of waveform data associated with the identification information of the received stamp 850 with each tactile stimulation unit 106 according to the specifications of the tactile presentation device 100. By doing so, for example, when the number of waveform data prepared in advance is not the same as the number of tactile stimulation units 106, the haptics server 300 or the haptics server 600 can transmit only the waveform data that matches the specifications of the tactile presentation device 100. Then, the haptics server 300 or the haptics server 600 transmits the waveform data associated with each tactile stimulation unit 106 to the drive amplifier / interface 200, and the drive amplifier / interface 200 can present a tactile stimulation by driving based on the waveform data received by each tactile stimulation unit 106.

[0222] <<16. Variations of Stamp Display>> Furthermore, with reference to FIG. 56, a display example according to a variation of the embodiment of the present disclosure will be described. FIG. 56 is an explanatory diagram for explaining a display example according to a variation of the embodiment of the present disclosure. In the embodiment of the present disclosure described so far, the viewer 900 has been described as selecting the stamp 850 with a tactile stimulation effect. However, in the present disclosure, the viewer 900 is not limited to this, and a stamp without a tactile stimulation effect may also be selectable together with the stamp 850 with a tactile stimulation effect.

[0223] For example, in FIG. 56, it is a display example of the user terminal 700 of the viewer 900, and in this display example, stamps 854a and 854b without a tactile stimulation effect are displayed together with the stamp 856 with a tactile stimulation effect. In this variation, by operating on such a display, the viewer 900 can select both the stamp 850 with a tactile stimulation effect and the stamp 854 without a tactile stimulation effect.

[0224] <<17. Hardware Configuration>> The information processing apparatus such as the haptics server 300 according to each of the above-described embodiments is realized by, for example, a computer 1000 having a configuration as shown in FIG. 57. Hereinafter, the haptics server 300 according to each embodiment of the present disclosure will be described as an example. FIG. 57 is a hardware configuration diagram showing an example of a computer that realizes the functions of the haptics server 300. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disc Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each part of the computer 1000 is connected by a bus 1050.

[0225] The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each part. For example, the CPU 1100 expands a program stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0226] The ROM 1300 stores a boot program such as a BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

[0227] The HDD 1400 is a computer-readable recording medium that non-temporarily records a program executed by the CPU 1100 and data used by such a program. Specifically, the HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0228] The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (for example, the Internet). For example, the CPU 1100 receives data from other devices or transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0229] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard and a mouse via the input / output interface 1600. Also, the CPU 1100 transmits data to output devices such as a display and a printer via the input / output interface 1600. Further, the input / output interface 1600 may function as a media interface for reading a program or the like recorded on a predetermined recording medium (media). The media is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory or the like.

[0230] For example, when the computer 1000 functions as the haptics server 300 according to the embodiment of the present disclosure, the CPU 1100 of the computer 1000 realizes functions such as generating a tactile control signal by executing an information processing program loaded on the RAM 1200. Also, an information processing program or the like according to the embodiment of the present disclosure is stored in the HDD 1400. Note that the CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, these programs may be acquired from other devices via the external network 1550.

[0231] Also, the information processing apparatus according to the present embodiment may be applied to a system including a plurality of devices on the premise of connection to a network (or communication between devices), such as cloud computing. That is, the information processing apparatus according to the present embodiment described above can also be realized as an information processing system that performs the processing according to the information processing method according to the present embodiment by a plurality of devices.

[0232] <<18. Supplementary Note>> Further, the embodiments described above can include, for example, a program for causing a computer to function as the information processing apparatus according to the present embodiment, and a non-transitory tangible medium on which the program is recorded. Further, the above program may be distributed via a communication line (including wireless communication) such as the Internet.

[0233] Furthermore, each step in the processing of each of the above-described embodiments does not necessarily have to be processed in the order described. For example, each step may be processed in a changed order as appropriate. Also, each step may be processed partially in parallel or individually instead of being processed in time series. Furthermore, the processing method of each step does not necessarily have to be processed in accordance with the method described, and for example, it may be processed by other functional blocks in other ways.

[0234] As described above, the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field of the present disclosure can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and it is naturally understood that these also belong to the technical scope of the present disclosure.

[0235] Also, the effects described in this specification are merely illustrative or exemplary and not restrictive. That is, the technology according to the present disclosure can exhibit other effects that are obvious to those skilled in the art from the description of this specification, together with or instead of the above effects.

[0236] Note that the present technology can have the following configuration. (1) a first acquisition unit that acquires each control command including presentation unit information for designating a presentation unit that presents a tactile stimulus by a tactile presentation device in response to an input from a first user, and form information for designating the form of the tactile stimulus; a generation unit that generates each tactile control signal for presenting the tactile stimulus to the presentation unit according to each of the control commands; A first distribution unit that distributes each of the tactile control signals to the tactile presentation device worn on the body of the second user; comprising When the inputs from the plurality of first users satisfy a predetermined condition, the first acquisition unit acquires a new control command different from each of the control commands corresponding to the inputs from the plurality of first users. An information processing apparatus. (2) The control command is pre-associated with a stamp displayed on a stamp selection screen. Each of the first users makes an input by selecting the stamp displayed on the stamp selection screen. The information processing apparatus according to (1) above. (3) By tapping an apparatus that displays the stamp selection screen, the first user sets a timing for presenting the tactile stimulus. The information processing apparatus according to (2) above. (4) Based on a timing at which the tap operations of the plurality of first users are synchronized, the tactile stimulus is presented. The information processing apparatus according to (3) above. (5) According to the number of inputs from the plurality of first users, the intensity or presentation time of the tactile stimulus corresponding to the new control command changes. The information processing apparatus according to (1) above. (6) When parameters of each of the tactile control signals generated within a predetermined time exceed a threshold value determined in advance based on the characteristics of the presentation unit, the first distribution unit temporarily stops distributing each of the tactile control signals. The information processing apparatus according to (1) above. (7) When temporarily stopping the distribution of each of the tactile control signals, the information processing apparatus according to (6) above further includes a notification unit that notifies the plurality of first users. (8) The information processing apparatus according to (1) above, wherein the tactile control signal designates at least one parameter among the presentation timing, frequency, interval, waveform, presentation time, and intensity of the tactile stimulus. (9) Further comprising a second distribution unit that distributes a predetermined image to a display device that displays the image toward the second user, When the tactile presentation device is not attached to the body of the second user, the second distribution unit distributes the predetermined image subjected to image processing in which at least a part of the predetermined image is vibrated in a predetermined direction according to the frequency. The information processing apparatus according to (8) above. (10) In the image processing, the information processing apparatus according to (9) above, wherein a region to be vibrated in the predetermined image is determined according to the frequency. (11) In the image processing, the information processing apparatus according to (9) or (10) above, wherein the predetermined image is amplitude-modulated, and the frequency, interval, and waveform change according to the parameter of the tactile stimulus. (12) Further comprising an audio distribution unit that distributes a predetermined audio to an audio output device that outputs the audio toward the second user, When the tactile presentation device is not attached to the body of the second user, the audio distribution unit distributes the predetermined audio subjected to audio processing in which the sound image of the predetermined audio is vibrated in a predetermined direction according to the frequency. The information processing apparatus according to (8) above. (13) The information processing apparatus according to (1) above, wherein the generation unit selects the presentation unit that presents the tactile stimulus according to the state of the second user. (14) The stamp selection screen is distributed from another information processing apparatus communicable with the display device that displays an image toward the first user to the display device. The information processing apparatus according to (2) above. (15) The other information processing device acquires command information according to the input of the first user. The information processing device according to (14) above. (16) A second acquisition unit that acquires a predetermined image according to an input from the first user, A third distribution unit that distributes the predetermined image to a display device that displays the image toward the first user, further comprising: The information processing device according to (1) above. (17) A first acquisition unit that acquires each identification information for specifying presentation unit information for specifying a presentation unit that presents a tactile stimulus by a tactile presentation device and form information for specifying the form of the tactile stimulus according to an input from the first user, A generation unit that generates each tactile control signal for presenting the tactile stimulus to the presentation unit according to each identification information, A first distribution unit that distributes each tactile control signal to the tactile presentation device worn on the body of the second user, comprising: When inputs from a plurality of the first users satisfy a predetermined condition, the first acquisition unit acquires other identification information that specifies the presentation unit information and the form information different from the presentation unit information and the form information specified by each identification information corresponding to each of the inputs from the plurality of the first users, Information processing device. (18) By an information processing device, acquiring each control command including presentation unit information for specifying a presentation unit that presents a tactile stimulus by a tactile presentation device and form information for specifying the form of the tactile stimulus according to an input from the first user; generating each tactile control signal for presenting the tactile stimulus to the presentation unit according to each control command; distributing each tactile control signal to the tactile presentation device worn on the body of the second user; including: When the inputs from the plurality of first users satisfy a predetermined condition, obtain a new control command different from each of the control commands corresponding to each of the inputs from the plurality of first users. Information processing method. (19) In a computer, a function of obtaining each control command including presentation unit information specifying a presentation unit that presents a tactile stimulus by a tactile presentation device and form information specifying the form of the tactile stimulus in response to an input from a first user; a function of generating each tactile control signal for presenting the tactile stimulus to the presentation unit according to each of the control commands; a function of distributing each of the tactile control signals to the tactile presentation device worn on the body of a second user; A program for realizing the above, When the inputs from the plurality of first users satisfy a predetermined condition, obtain a new control command different from each of the control commands corresponding to each of the inputs from the plurality of first users. Program. (20) An information processing system including an information processing device and a distribution device, wherein the information processing device has a first acquisition unit that acquires each control command including presentation unit information specifying a presentation unit that presents a tactile stimulus by a tactile presentation device and form information specifying the form of the tactile stimulus in response to an input from a first user, a generation unit that generates each tactile control signal for presenting the tactile stimulus to the presentation unit according to each of the control commands, a first distribution unit that distributes each of the tactile control signals to the tactile presentation device worn on the body of a second user, and the first acquisition unit acquires a new control command different from each of the control commands corresponding to each of the inputs from the plurality of first users when the inputs from the plurality of first users satisfy a predetermined condition, and the distribution device An image generation unit that superimposes a predetermined image generated based on the input on a real-space image distributed to the first user. Information processing system. (21) The distribution device is the information processing system according to (20) above, which is configured by a device on the cloud.

Explanation of symbols

[0237] 10 Information processing system 100 Tactile presentation device 102, 302, 402, 502, 708 Communication unit 104, 710 Control unit 106, 106a, 106b, 106c, 106d, 106e, 106f Tactile stimulation unit 108 Operation unit 200 Drive amplifier / interface 202, 706 Speaker 204 Monitor 206, 724 Microphone 208, 722 Camera 300, 300a Haptics server 304, 404 Camera image acquisition unit 306, 410 Microphone voice acquisition unit 308, 408, 506 Stamp acquisition unit 310 Tactile signal generation unit 312 Distributor status acquisition unit 314 Output image acquisition unit 316, 414 Output voice acquisition unit 318, 416, 518, 712 Storage unit 350 Local side control device 400 Distribution data editing server 406 Image generation unit 412 Voice generation unit 500 Live distribution server 504 GUI control unit 508 Voice data acquisition unit 510 Image data acquisition unit 512 Tactile signal acquisition unit 514 Viewer information acquisition unit 516 Distribution control unit 600 Haptics server 700 User terminal 702 Display unit 704 Operation input unit 720 Sensor unit 726 Gyro sensor 728 Acceleration sensor 800, 800a, 800b, 800c Distributors 840 Waveform image 842, 872, 874, 880, 980 Display 850, 850a, 850b, 850c, 850d, 852, 854, 854a, 854b, 856 Stamps 860 Comment 870a, 870b, 890, 892 Icons 900 Viewers 910 Vector 950 Display image 960a, 960b, 970 Regions

Claims

1. A first acquisition unit that acquires each control command including presentation unit information that designates a presentation unit that presents a tactile stimulus by a tactile presentation device in response to an input from a first user, and form information that designates the form of the tactile stimulus; A generation unit that generates each tactile control signal for presenting the tactile stimulus to the presentation unit according to each of the control commands; A first distribution unit that distributes each of the tactile control signals to the tactile presentation device worn on the body of a second user; comprising: When inputs from a plurality of the first users satisfy a predetermined condition, the first acquisition unit acquires a new control command different from each of the control commands corresponding to each of the inputs from the plurality of the first users. An information processing apparatus.

2. The control command is pre-associated with a stamp displayed on a stamp selection screen, Each of the first users makes an input by selecting the stamp displayed on the stamp selection screen. The information processing apparatus according to claim 1.

3. By tapping an operation on a device that displays the stamp selection screen, the first user sets a timing for presenting the tactile stimulus. The information processing apparatus according to claim 2.

4. Based on a timing at which the tap operations of the plurality of first users are synchronized, the tactile stimulus is presented. The information processing apparatus according to claim 3.

5. According to the number of inputs from the plurality of first users, the intensity or presentation time of the tactile stimulus corresponding to the new control command changes. The information processing apparatus according to claim 1.

6. When parameters of each of the tactile control signals generated within a predetermined time exceed a threshold value determined in advance based on characteristics of the presentation unit, the first distribution unit temporarily stops distribution of each of the tactile control signals. The information processing apparatus according to claim 1.

7. The information processing apparatus according to claim 6, further comprising a notification unit that notifies the plurality of first users when the distribution of each of the tactile control signals is temporarily stopped.

8. The tactile control signal specifies at least one parameter among the presentation timing, frequency, interval, waveform, presentation time, and intensity of the tactile stimulus, and the information processing apparatus according to claim 1.

9. Further comprising a second distribution unit that distributes a predetermined image to a display device that displays the image toward the second user, When the tactile presentation device is not attached to the body of the second user, the second distribution unit distributes the predetermined image subjected to image processing in which at least a part of the predetermined image is vibrated in a predetermined direction according to the frequency. The information processing apparatus according to claim 8.

10. In the image processing, the area to be vibrated in the predetermined image is determined according to the frequency, and the information processing apparatus according to claim 9.

11. In the image processing, the predetermined image is amplitude-modulated according to the parameter of the tactile stimulus, and the frequency, interval, and waveform change, and the information processing apparatus according to claim 9.

12. Further comprising an audio distribution unit that distributes a predetermined audio to an audio output device that outputs the audio toward the second user, When the tactile presentation device is not attached to the body of the second user, the audio distribution unit distributes the predetermined audio subjected to audio processing in which the sound image of the predetermined audio is vibrated in a predetermined direction according to the frequency. The information processing apparatus according to claim 8.

13. The generation unit selects the presentation unit that presents the tactile stimulus according to the state of the second user, and the information processing apparatus according to claim 1.

14. The stamp selection screen is distributed to the display device from another information processing device capable of communicating with the display device that displays an image for the first user, The information processing apparatus according to claim 2.

15. The other information processing device acquires command information according to an input from the first user. The information processing apparatus according to claim 14.

16. A second acquisition unit that acquires a predetermined image according to an input from the first user; A third distribution unit that distributes the predetermined image to a display device that displays an image for the first user; and further includes The information processing apparatus according to claim 1.

17. A first acquisition unit that acquires respective identification information for specifying presentation unit information for specifying a presentation unit that presents a tactile stimulus by a tactile presentation device and form information for specifying the form of the tactile stimulus according to an input from a first user; A generation unit that generates respective tactile control signals for presenting the tactile stimulus to the presentation unit according to the respective identification information; A first distribution unit that distributes the respective tactile control signals to the tactile presentation device worn on the body of the second user; and includes When inputs from a plurality of first users satisfy a predetermined condition, the first acquisition unit acquires other identification information that specifies the presentation unit information and the form information different from the presentation unit information and the form information specified by the respective identification information corresponding to each of the inputs from the plurality of first users. Information processing apparatus.

18. By an information processing apparatus, acquiring each control command including presentation unit information for specifying a presentation unit that presents a tactile stimulus by a tactile presentation device and form information for specifying the form of the tactile stimulus according to an input from a first user; Generating respective tactile control signals for presenting the tactile stimulus to the presentation unit according to each of the control commands; Delivering each of the tactile control signals to the tactile presentation device worn on the body of a second user; including; when inputs from a plurality of the first users satisfy a predetermined condition, obtaining a new control command different from each of the control commands corresponding to each of the inputs from the plurality of the first users, An information processing method.

19. causing a computer to obtain a function of obtaining each control command including presentation unit information specifying a presentation unit for presenting a tactile stimulus by a tactile presentation device and form information specifying a form of the tactile stimulus in response to an input from a first user; generate a function of generating respective tactile control signals for presenting the tactile stimulus to the presentation unit according to each of the control commands; deliver a function of delivering each of the tactile control signals to the tactile presentation device worn on the body of a second user; A program for realizing, when inputs from a plurality of the first users satisfy a predetermined condition, obtaining a new control command different from each of the control commands corresponding to each of the inputs from the plurality of the first users, Program.

20. An information processing system including an information processing device and a distribution device, wherein the information processing device a first acquisition unit that acquires each control command including presentation unit information specifying a presentation unit for presenting a tactile stimulus by a tactile presentation device and form information specifying a form of the tactile stimulus in response to an input from a first user; a generation unit that generates respective tactile control signals for presenting the tactile stimulus to the presentation unit according to each of the control commands; a first distribution unit that delivers each of the tactile control signals to the tactile presentation device worn on the body of a second user; having, when the inputs from the plurality of first users satisfy a predetermined condition, the first acquisition unit acquires a new control command different from each of the control commands corresponding to the inputs from the plurality of first users, the distribution device, has an image generation unit that superimposes a predetermined image generated based on the input on an image of the real space distributed to the first user, information processing system.

21. The information processing system according to claim 20, wherein the distribution device is configured by a device on the cloud.

Images