Information processing device, information processing method, and program

Abstract

[Problem] An information processing apparatus, method, and program are provided to assist hearing-impaired individuals in listening during motion picture reproduction. [Solution] The information processing apparatus includes a controller. The controller analyzes subtitle information, including audio subtitle data indicating sound effects with text information and audio subtitle data indicating human speech with text information, as well as sound information, in a motion picture file. Using the analysis results obtained from analyzing this subtitle information and sound information, the controller generates vibration signals corresponding to the audio subtitle data and / or the audio subtitle data based on the waveform of the sound data.

Description

Technical Field

[0001] The present technology relates to an information processing device, an information processing method, and a program for helping viewers to watch video files. Background Art

[0002] Subtitles for explaining the sound content are generally displayed in videos such as movies or dramas suitable for the hearing impaired to help the hearing impaired watch the video without sound information. Not only the human voice is displayed in the form of subtitles, but also the sound effects used to explain the scene are displayed. However, it is difficult to use text-based subtitles to express, for example, the pitch change and volume of the character's voice and the speed of the character's voice.

[0003] Here, Patent Document 1 discloses the use of an automatic haptic algorithm to generate a haptic effect. The automatic haptic algorithm performs analysis for performing pattern recognition on the audio track and video track of a movie to generate a haptic effect. In addition, the A / V receiver analyzes the closed caption text of a keyword. For example, when "explosion" is found as a keyword, the automatic haptic algorithm looks for an explosion pattern around the timestamp of the found keyword, and further, the automatic haptic algorithm generates a corresponding haptic effect, such as a strong vibration corresponding to the explosion. The A / V receiver outputs a haptic effect at a time corresponding to an event occurring in the movie.

[0004] Reference List

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2015-53048

[0007] Disclosure of the Invention

[0008] Technical issues

[0009] The technology disclosed in Patent Document 1 makes it possible to provide the user with a tactile effect corresponding to a so-called sound effect such as “explosion.” However, it is difficult for a hearing-impaired person to recognize, for example, changes in voice pitch and utterance speed when people are talking.

[0010] An object of the present technology is to provide an information processing device, an information processing method, and a program that make it possible to help a hearing-impaired person watch a video when playing back the video.

[0011] Solution to the problem

[0012] In order to achieve the above-mentioned object, an information processing device according to an embodiment of the present technology includes a controller.

[0013] The controller uses the results of analyzing the subtitle information and sound information included in the video file, and based on the waveform of the sound data, generates at least one of a vibration signal corresponding to the sound effect subtitle data and a vibration signal corresponding to the sound subtitle data, the sound effect subtitle data is used to represent the sound effect in the form of text information, and the sound subtitle data is used to represent the human sound in the form of text information, the sound effect subtitle data and the sound subtitle data are included in the subtitle data included in the subtitle information, and the sound data is included in the sound information.

[0014] Such a configuration makes it possible to use vibration to assist a viewer in viewing a video file, since a vibration signal corresponding to a sound effect or a sound utterance is generated based on the sound data.

[0015] The controller can divide the sound data into sound data corresponding to the sound effect subtitle data and sound data corresponding to the spoken subtitle data, and the controller can generate a vibration signal corresponding to the sound effect subtitle data based on the waveform of the sound data corresponding to the sound effect subtitle data, and can generate a vibration signal corresponding to the spoken subtitle data based on the waveform of the sound data corresponding to the spoken subtitle data.

[0016] The controller can use at least one of display position information, mark information, font information and color information included in the subtitle information to determine whether the subtitle data is sound effect subtitle data or voice subtitle data, the display position information is information about the position where the subtitle is displayed on the image, the mark information is information about the mark displayed on the image, the font information is information about the font of the text in the subtitle, and the color information is information about the color of the text in the subtitle.

[0017] In the case where the subtitle information includes subtitle information corresponding to a scene in which multiple persons are talking, the controller can use the subtitle information and segment the spoken subtitle data in the scene in which multiple persons are talking according to the order of speech in the conversation, and can generate a vibration signal corresponding to each piece of spoken subtitle data obtained by the segmentation.

[0018] The controller may generate a vibration signal corresponding to a sound effect and a vibration signal corresponding to a human vocalization such that there is a relative difference in amplitude between vibrations based on the vibration signal corresponding to the sound effect and vibrations based on the vibration signal corresponding to the human vocalization.

[0019] The video file may include content meta information, and the controller may generate the vibration signal taking into account the content meta information.

[0020] The controller may generate a vibration signal taking into account information about the user.

[0021] A vibration providing unit for providing vibration to a user based on the vibration signal may further be included.

[0022] A plurality of vibration providing portions may be included.

[0023] The controller can use the results of analyzing the subtitle information and the sound information to generate a vibration signal corresponding to the sound effect subtitle data and a vibration signal corresponding to the voiced subtitle data, and can output the generated vibration signal to different vibration providing units among the multiple vibration providing units.

[0024] In order to achieve the above-mentioned purpose, the information processing method according to the implementation mode of the present technology includes: using the results of analyzing the subtitle information and sound information included in the video file, based on the waveform of the sound data, generating at least one of a vibration signal corresponding to the sound effect subtitle data and a vibration signal corresponding to the sound subtitle data, the sound effect subtitle data is used to represent the sound effect in the form of text information, and the sound subtitle data is used to represent the human sound in the form of text information, the sound effect subtitle data and the sound subtitle data are included in the subtitle data included in the subtitle information, and the sound data is included in the sound information.

[0025] In order to achieve the above-mentioned purpose, the program according to the implementation mode of the present technology enables the information processing device to perform processing including the following operations: analyzing the subtitle information and sound information included in the video file, the subtitle information including subtitle data containing sound effect subtitle data and voice subtitle data, the sound information including sound data, the sound effect subtitle data is used to represent the sound effect in the form of text information, and the voice subtitle data is used to represent the human voice in the form of text information; and using the analysis results, based on the waveform of the sound data, generating at least one of a vibration signal corresponding to the sound effect subtitle data and a vibration signal corresponding to the voice subtitle data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 The configuration of the vibration providing system is schematically shown.

[0027] Figure 2 is a perspective view showing a state in which a user is using an information processing apparatus which is a part of a vibration providing system and includes a vibration providing device.

[0028] Figure 3 is a diagram of the basic flow of a method for generating vibrations.

[0029] Figure 4 is a diagram of an example flow of generating vibrations.

[0030] Figure 5An example of a scene included in a video and on which subtitles are displayed, a sound waveform of a person's vocalization and a sound waveform of a sound effect in the scene, and waveforms of vibration signals respectively generated based on the respective sound waveforms are shown.

[0031] Figure 6 An example of a scene included in another video and on which subtitles are displayed, a sound waveform of a person's utterance in the scene, and waveforms of vibration signals respectively generated based on the respective sound waveforms are shown.

[0032] Figure 7 An example of a scene included in still another video and on which subtitles are displayed, a sound waveform of a sound effect in the scene, and a waveform of a vibration signal generated based on the sound waveform are shown.

[0033] Figure 8 An example of a scene included in another video and on which a subtitle is displayed is shown. DETAILED DESCRIPTION

[0034] [Overview of vibration providing system]

[0035] use Figure 1 and Figure 2 A vibration providing system according to an embodiment of the present technology is described.

[0036] Figure 1 A functional block diagram showing the configuration of a vibration providing system and including an information processing device as a part of the vibration providing system.

[0037] Figure 2 : is a perspective view showing a state in which a user U is using an information processing apparatus including a vibration providing device.

[0038] The vibration providing system of the present embodiment helps a hearing-impaired user U (hereinafter referred to as a "viewer" in some cases) understand, for example, sound presentation when the user U consumes video content such as movies, TV dramas, and games.

[0039] More specifically, the sound information and subtitle information of the video file including the sound information, video information and subtitle information are used to provide the vibration corresponding to the sound to the user U in real time. In the present embodiment, the vibration corresponding to the sound effect and the vibration corresponding to the human voice are generated respectively. Therefore, for example, in the case of a movie, the viewer knows the pitch change and volume of the character's voice in the movie, the speed of the character's voice, the volume and duration of the sound effect, and the volume change through the vibration. This enables the viewer to understand the scene in the movie more deeply.

[0040] For example, when an "explosion sound" used to explain a sound effect is displayed on an image in the form of subtitles for explaining the sound content to help a hearing-impaired user U watch a video without sound information, the viewer does not know whether the explosion sound is a continuous low rumble of the earth or a momentary loud bang based on the subtitle information alone.

[0041] In this embodiment, a vibration signal is generated based on the waveform of the sound data of the sound effect, which makes it possible to provide the user with the type of explosion sound using vibration. This enables the user U to deeply understand the scene in the movie.

[0042] In addition, the following is another example. When displaying subtitles of a person's utterance, it may be difficult to understand the person's feelings using only subtitle information. In this embodiment, vibration is generated based on the waveform of the sound data of the person's utterance, which makes it possible to provide the pitch change and volume of the person's voice and the speed of the person's utterance using vibration. This enables the user U to deeply understand the scene in the movie.

[0043] This will be described in detail below.

[0044] like Figure 1 As shown, the vibration providing system 20 includes an information processing device 1 and a content distribution server 10 .

[0045] When a video file distributed by the content distribution server 10 is being played back, the vibration providing system 20 provides vibration to the viewer in real time based on subtitle information and sound information included in the video file.

[0046] In the vibration providing system 20, the information processing device 1 and the content distribution server 10 are connected to each other through a network such as the Internet to be able to communicate with each other. The information processing device 1 sends a request for obtaining list data and a request for a downloaded video (in some cases referred to as a "video file") to the content distribution server 10. In addition, the content distribution server 10 sends the list data and the video file to the information processing device 1, for example.

[0047] An example of providing vibration based on subtitle information and sound information included in a video file while a downloaded video is being played back is described below.

[0048] (Content Distribution Server)

[0049] The content distribution server 10 stores paid or free content data. The content distribution server 10 provides video files. When the information processing device 1 accesses the content distribution server 10, a content list is displayed on a display section 2 included in the information processing device 1 and described later. When the user U selects content, the video file of the content is downloaded from the content distribution server 10.

[0050] Video files include: content meta information, subtitle information, video information and sound information.

[0051] The content meta information includes, for example, category information on a video category, type information on a video type, and status information on a broadcast status.

[0052] Examples of video categories include movies, sports, animation, and news. Examples of video types include action, SF, and romance. Examples of broadcast states include during a program and during a commercial break.

[0053] The video information includes video data. Examples of information included in the video data include: object information about objects such as vehicles, trains, guns, and people; feeling information about human feelings such as joy, anger, sadness, and happiness; motion information about the movement of objects (e.g., stop and fast movement); flash information about flashes when explosions or bubbles appear; and sign language information.

[0054] The sound information includes, for example, sound data, the number of sound channels, sound source position information, and second audio channel information.

[0055] The sound data includes, for example, frequency information and information about sound pressure changes. The frequency information provides information about the pitch (such as high and low notes). The information about sound pressure changes provides information about the loudness of sounds such as explosions and powerful sound effects.

[0056] Examples of the second audio channel information include not only sound data of the language of the second audio channel in bilingual broadcasting but also voice-over for the visually impaired and the like.

[0057] The subtitle information includes, for example, subtitle data, display position information about the position where the subtitle is displayed on the image, mark information about a mark displayed on the image, font information about the font of text in the subtitle, and color information about the color of text in the subtitle.

[0058] The subtitle data is subtitle text data. The subtitle data includes two types of subtitle data, namely, spoken subtitle data for representing a person's voice in the form of text information and auxiliary subtitle data.

[0059] The auxiliary subtitle data is subtitle data used to help viewers understand the video content. The auxiliary subtitle data includes: sound effect subtitle data used to explain the sound effect in the form of text information and scene explanation subtitle data.

[0060] The audio effect subtitle data is essentially used to explain the sound that is not a human voice. Examples of sound effects include not only the actual environmental sounds emitted when playing back the video (such as the sound of a vehicle horn, the sound of an elevator arrival, the barking of a dog, the sound of a door opening and closing, and the sound of an explosion), but also include, for example, sounds emitted as a human voice when playing back the video, but not actually emitted by the human, for the human's feelings and narration.

[0061] The scenario interpretation subtitle data helps to understand the scenario in which no sound is made when playing back the video. For example, in a scene where multiple people are talking, the name of the person making the sound is displayed in the subtitles in parentheses or square brackets before the text corresponding to the person's sound. The person's name corresponds to the situation of the scenario interpretation subtitle data. Such scenario interpretation subtitle data enables the viewer to understand which sound is which person's sound. Note that when the text displayed in parentheses or square brackets is a person's name, the female name and the male name can be displayed in different colors, such as displaying the female name in red and the male name in blue, so that the person's gender can be determined. The color of the text displayed in parentheses or square brackets makes it possible to determine whether the subtitle data is scenario interpretation subtitle data.

[0062] (Configuration of Information Processing Device)

[0063] The information processing device 1 accesses the content distribution server 10 and downloads the video file. The information processing device 1 includes an application for generating a vibration signal of vibration provided to the user using the subtitle information and the sound information included in the video file.

[0064] For example, the information processing apparatus 1 may be a cellular phone, a personal digital assistant (PDA), or a portable game machine. Here, a cellular phone is described as an example of an information processing apparatus including a vibration device as a vibration providing section and a display section.

[0065] The information processing device 1 may include a communication function using a wireless communication system such as a wireless local area network (LAN) or a cellular phone. In addition, the information processing device 1 may be configured to communicate with a content distribution server by connecting to an external device using a wired cable such as a USB cable.

[0066] like Figure 2As shown, the information processing device 1 of the present embodiment includes a horizontally long display unit 2, and the user U holds the information processing device 1 with his / her hand in a so-called horizontally long holding state. The information processing device 1 includes a left-hand vibration device 4L and a right-hand vibration device 4R. In the state where the user U holds the information processing device 1, the left-hand vibration device 4L is positioned to correspond to the left hand of the user U, and the right-hand vibration device 4R is positioned to correspond to the right hand of the user U. The left-hand vibration device 4L is driven to vibrate based on a vibration signal output by a controller 6 described later. This causes vibration to be provided to the left hand of the user U. Similarly, the right-hand vibration device 4R is driven to vibrate based on a vibration signal output by the controller 6. This causes vibration to be provided to the right hand of the user U.

[0067] Here, when there is no need to particularly distinguish between the left-hand vibration device 4L and the right-hand vibration device 4R, the vibration devices are referred to as the vibration devices 4L and 4R.

[0068] The information processing device 1 of the present embodiment outputs the waveform of the input sound signal as vibration.

[0069] like Figure 1 As shown, the information processing apparatus 1 includes: a display unit 2, a sound output unit 3, a left-hand vibration device 4L, a right-hand vibration device 4R, a communication unit 5, a controller 6, an input unit 7, an amplifier 8 for a speaker, an amplifier 9L for driving the left-hand vibration device, and an amplifier 9R for driving the right-hand vibration device.

[0070] The display section 2 displays various information thereon as a test or in the form of images, such as video data of playback content and a content list. The display section 2 includes, for example, a cathode ray tube (CRT) display, a liquid crystal display (LCD), or an organic light emitting diode (OLED).

[0071] The display section 2 displays thereon an image based on the video data and subtitle data output by the controller 6 .

[0072] The sound output unit 3 is, for example, a speaker and an earphone. The sound output unit 3 changes the sound data output by the controller 6 into sound, and outputs the sound.

[0073] The vibration devices 4L, 4R are driven to provide vibration to the user based on a vibration signal output by the controller 6. The vibration signal is generated by a vibration generator 64 described later based on the waveform of the sound data of the video file.

[0074] A vibrator such as an electromagnetic vibrator capable of following vibrations in a frequency band of a sound signal up to about 20 kHz is used as the vibration device 4. As the vibration device 4, a known device such as an eccentric motor actuator or a linear resonance actuator can be used.

[0075] The information processing apparatus 1 of the present embodiment includes two vibration devices 4L and 4R.

[0076] The communication unit 5 sends information to and receives information from the external device. For example, the communication unit 5 is connected to the external device through, for example, a LAN, Bluetooth (registered trademark), or Wi-Fi to be able to communicate with the external device, and sends and receives information. The information processing device 1 of this embodiment receives a video file from a content distribution server 10 corresponding to the external device.

[0077] The controller 6 controls the information processing apparatus 1. The controller 6 generates a vibration signal for the vibration device 4 using the subtitle information and sound information included in the acquired video file, and outputs the vibration signal to the vibration device 4. In addition, the controller 6 outputs the video data and subtitle data to the display section 2, and outputs the sound data to the sound output section 3. The controller 6 will be described in detail later.

[0078] The input section 7 is an interface for performing input to the information processing apparatus 1. Through the input section 7, the user can input viewer information as user information.

[0079] The viewer information includes attribute information on attributes of a viewer who is a user and viewing environment information.

[0080] Examples of the attribute information regarding the attributes of the viewer include the hearing loss state, age, and vibration preference of the viewer.

[0081] The hearing loss status of the viewer is the degree of hearing loss of each of the left and right ears, such as hard of hearing in both ears, hard of hearing in only the right ear, and no hearing at all in the left ear.

[0082] The viewing environment information is environment information about the environment surrounding the viewer when the viewer views content using the information processing apparatus 1. Examples of the environment include indoors, outdoors, and during travel by, for example, a car or a train.

[0083] The magnitude of the vibration may be adjusted by performing comprehensive determination based on viewer information.

[0084] The type of the input section 7 is not limited, and the input section 7 may be any known input section. In the present embodiment, a touch panel is used as the input section 7, and the touch panel is provided on the display surface of the display section 2. Other examples of the input section 7 include a mouse, a keyboard, and a switch.

[0085] The amplifier 8 for the speaker amplifies the sound data outputted from the controller 6 , and outputs the amplified sound data to the sound output section 3 .

[0086] The amplifier 9L for driving the left-hand vibration device amplifies the left-hand vibration signal output by the controller 6 and outputs the amplified signal to the left-hand vibration device 4L.

[0087] The amplifier 9R for driving the right-hand vibration device amplifies the right-hand vibration signal output by the controller 6 and outputs the amplified signal to the right-hand vibration device 4R.

[0088] The controller 6 is described.

[0089] The controller 6 includes a memory 61 , an acquisition section 62 , an analyzer 63 , and a vibration generator 64 .

[0090] The acquisition section 62 acquires the video file received from the content distribution server 10 via the communication section 5 and the viewer information input via the input section 7 .

[0091] The memory 61 stores therein various data and programs used when processing is executed by the controller 6 and various data acquired from the outside, such as viewer information acquired by the acquisition section 62 .

[0092] The analyzer 63 analyzes subtitle information and sound information included in the video file.

[0093] More specifically, the analyzer 63 analyzes the subtitle information when playing back the video, and divides the subtitle data into voiced subtitle data and sound effect subtitle data using at least one of the following information: display position information about the position where the subtitle is displayed on the image, a mark state about a mark displayed on the image, font information about the font in the subtitle, or color information about the Chinese character color of the subtitle. In this case, the analyzer 63 performs analysis to determine that subtitle data displayed in an upright font, not in parentheses or square brackets, and displayed horizontally is voiced subtitle data, and subtitle data other than the voiced subtitle data is auxiliary subtitle data. In addition, the analyzer 63 performs analysis to determine that, among the auxiliary subtitle data, data of subtitles displayed using red or blue text in parentheses or square brackets is scene interpretation subtitle data, and subtitle data other than the scene interpretation subtitle data is sound effect subtitle data.

[0094] Here, referring to an example of displaying subtitles on a video, how to distinguish between sound effect subtitle data and spoken subtitle data is described using drawings.

[0095] Figure 5 (A) Figure 6 (A) Figure 7 (A) Figure 8 (A) to (C) of FIG. 1 respectively illustrate scenes in a video and are diagrams for describing an example of displaying a subtitle on an image.

[0096] In the case of videos such as Japanese movies, Japanese in which text can be written horizontally and vertically is mainly used for subtitles, it can be determined whether subtitle data is voiced subtitle data or sound effect subtitle data according to the position where the subtitles are displayed on the image.

[0097] For example, in a video in which Japanese subtitles are used, Figure 5 In the scene 40 of (A), the voice subtitle data 42 is displayed horizontally, and the sound effect subtitle data 41 for explaining the alarm sound corresponding to the sound effect is displayed vertically. As described above, whether the subtitle data is the voice subtitle data or the sound effect subtitle data can be determined according to the position where the subtitle is displayed on the image.

[0098] Furthermore, the subtitle data may be divided into spoken subtitle data and auxiliary subtitle data using a mark displayed on the image.

[0099] For example, in Figure 5 In scene 40 of the video shown in (A), audio effect subtitle data is displayed in square brackets, while symbols such as square brackets are not used for spoken subtitle data.

[0100] Furthermore, in the example corresponding to a video in which Japanese is used as a subtitle Figure 7 In the scene 45 of (A), subtitle data 46 for explaining the vehicle horn sound corresponding to the sound effect is horizontally displayed in parentheses.

[0101] Furthermore, in the example corresponding to a video such as a foreign language movie Figure 8 In scene 50 of (B), in which a language written horizontally is mainly used for subtitles, auxiliary subtitle data 51 for indicating an elevator sound corresponding to a sound effect is displayed in square brackets, while marks such as square brackets are not used for voiced subtitle data 52.

[0102] Furthermore, in the example of a video corresponding to a foreign language movie Figure 8 In scene 53 of (C), auxiliary subtitle data 54 for representing a person's laughter corresponding to the sound effect and sound effect subtitle data 56 for representing a creaking sound of a door corresponding to the sound effect are displayed in square brackets, while marks such as square brackets are not used for the vocal subtitle data 55.

[0103] As described above, the use of notations such as parentheses or square brackets makes it possible to distinguish between spoken subtitle data and auxiliary subtitle data. Figure 5 In the scene 40 of (A), in addition to being displayed in square brackets, the subtitle data can also be determined as audio effect subtitle data 41 due to being displayed vertically.

[0104] Furthermore, in addition to using parentheses or square brackets, the color of text in the subtitles in parentheses or square brackets (marks) displayed on the image may be used to divide the auxiliary subtitle data into audio effect subtitle data and scenario explanation subtitle data.

[0105] When the text in the round bracket or square bracket is red or blue, the text in the round bracket or square bracket can be determined as scene explanation subtitle data. When the text in the round bracket or square bracket is not red or blue, the text in the round bracket or square bracket can be determined as sound effect subtitle data.

[0106] exist Figure 7 (A) Figure 8 (B) and Figure 8 In each of the scenes 45, 50, and 53 shown in (C) of FIG. 1 , the characters in the parentheses or square brackets are displayed in a color other than red or blue. Therefore, the subtitles in the parentheses or square brackets can be determined as audio effect subtitle data.

[0107] On the other hand, videos in which Japanese is used in subtitles include Figure 6 Scene 43 of (A) is a scene in which two people are talking, and the names of the people are displayed in parentheses as subtitles so that the viewer understands which voice is which person's voice. Figure 6 In (A), "Naomi" in parentheses, one of the common names for Japanese women, is displayed in red, while "Ichiro" in parentheses, one of the common names for Japanese men, is displayed in blue. Therefore, since the color of the text in parentheses in the subtitles is red or blue, the subtitles in parentheses can be determined as situational interpretation subtitle data.

[0108] In addition, videos in which Japanese subtitles are used include Figure 8 In the subtitles of scene 47 of (A), the name of the person is displayed in parentheses, and the color of the person's name is blue. Therefore, since the color of the text of the subtitles in the parentheses is red or blue, the subtitles in the parentheses can be determined as situational interpretation subtitle data.

[0109] As described above, the subtitle data may be divided into the sound effect subtitle data and the scenario explanation subtitle data using the color of the characters in the subtitle.

[0110] Note that when the sound effect subtitle data and the scene explanation subtitle data are not distinguished from each other by color, the analyzer described later may perform analysis to determine whether the text displayed in the parentheses or square brackets is a person's name. In this case, for example, name data including the names of a plurality of people is pre-stored in the memory 61 described later, and the analyzer 63 described later may determine whether the text in the parentheses or square brackets is a person's name by referring to the name data. When the text in the parentheses or square brackets has been determined to be a person's name, the subtitle in the parentheses or square brackets may be determined as scene explanation subtitle data.

[0111] In addition, when the text in parentheses represents the name of a person in the scene 43 where multiple people are talking, the text corresponding to the person's utterance is displayed after the name of the person in parentheses. Figure 6 As shown in (A), the subtitles "(Naomi) Have you decided? (Ichiro) Yes" are displayed. In this case, the information about the parentheses (marking information) can be used to segment the voiced subtitle data according to the order of utterance in the conversation. Specifically, the voiced subtitle data can be distinguished into the voiced subtitle data "Have you decided?" and the voiced subtitle data "Yes". Therefore, multiple voice data of different people can be separated from each other.

[0112] Furthermore, the subtitle data may be divided into spoken subtitle data and auxiliary subtitle data using the fonts of the characters in the subtitles.

[0113] For example, in Figure 5 In scene 40 of (A), text corresponding to a person's utterance is displayed in an upright font, which is an upright font without an inclination. On the other hand, in a video in which Japanese is used in the subtitles, Figure 8 In scene 47 of (A), subtitle data 48 is displayed in an italic font. The subtitle data 48 is used to represent the sound that is emitted as a person's voice when playing back the video, but is not the actual sound of the person for human perception.

[0114] Therefore, the portion corresponding to the subtitles that are not in parentheses or square brackets and are in italic font can be determined as sound effect subtitle data. In addition, the portion corresponding to the subtitles that are not in parentheses or square brackets and are in upright font can be determined as voiced subtitle data.

[0115] As described above, a portion corresponding to a subtitle in an italic font may be determined as auxiliary subtitle data. A portion corresponding to a subtitle other than an italic font that is not in parentheses or square brackets may be determined as sound effect subtitle data. Subtitles in parentheses or square brackets and displayed in red or blue may be determined as scenario explanation subtitle data, and subtitles displayed in colors other than red and blue may be determined as sound effect subtitle data.

[0116] As described above, sound effect subtitle data and voice subtitle data can be distinguished from each other by the font of the subtitles. The sound effect subtitle data is used to represent sound effects corresponding to sounds used for human feelings or narration that are emitted as human voices when playing back videos but are not the actual voices of humans.

[0117] As described above, subtitle data may be divided into sound effect subtitle data and vocal subtitle data by analyzing subtitle information about subtitles of a video file by the analyzer 63 .

[0118] Furthermore, the analyzer 63 analyzes the sound data, and divides the sound data into sound data of human voice and sound data of sound effects.

[0119] By utilizing differences in volume and frequency, for example, the sound data of a person's utterance and the sound data of a sound effect can be separated from each other. In addition, the separation can be performed based on a database constructed by machine learning using a large amount of sound data of utterances and a large amount of sound data of sound effects.

[0120] The vibration generator 64 generates a vibration signal based on the result of the analysis performed by the analyzer 63. The vibration generator 64 generates a vibration signal corresponding to a person's utterance based on a waveform of the sound data corresponding to the utterance, and generates a vibration signal corresponding to a sound effect based on a waveform of the sound data corresponding to the sound effect.

[0121] Furthermore, in the present embodiment, the vibration generator 64 outputs a vibration signal corresponding to a person's vocalization to the right-hand vibration device 4R, and outputs a vibration signal corresponding to a sound effect to the left-hand vibration device 4L. When a video is being played back, that is, when sound data is being reproduced, the vibration generator 64 outputs a vibration signal corresponding to the sound data.

[0122] For example, the method for generating a vibration signal for driving a vibrator (vibration device) disclosed in Japanese Patent Application Laid-Open No. 2008-283305 can be used to generate a vibration signal. More specifically, from the time waveform of the sound signal for which sound data is being reproduced, a spectrogram of the sound signal at the same time is obtained. The spectrogram shows the temporal variation of the sound signal spectrum relative to the frequency, wherein the vertical axis represents the frequency and the horizontal axis represents the time. The timing of generating a high-intensity spectrum that occurs instantaneously within a wide frequency range of the spectrogram is extracted. Specifically, the time derivative of the spectrum is obtained, the timing at which the obtained value (i.e., the time variation of the spectrum) is large is extracted to obtain the time derivative value as the intensity degree, and a vibration signal having a waveform that vibrates with an amplitude corresponding to the intensity degree is generated. The vibration signal generated as described above is referred to as a vibration signal according to the basic setting.

[0123] As described above, a vibration signal is generated based on the waveform of the sound data. Therefore, for example, in the case of a vibration corresponding to a person's vocalization, the pitch change and volume of the person's vocalization, the speed of the vocalization, etc. are reflected in the vibration. Therefore, the vibration enables the viewer to easily imagine, for example, the person's feelings and to more deeply understand the situation in the scene. This makes it possible to help the viewer watch.

[0124] Likewise, in the case of vibration corresponding to sound effects, the volume of sound effects such as vehicle horns and explosions, changes in volume, duration of sound, etc. are reflected in the vibration that occurs due to the vibration signal generated based on the waveform of the sound data. Therefore, the vibration enables the viewer to understand the scene more deeply. This makes it possible to help the viewer watch.

[0125] In addition, the vibration generator 64 may adjust the vibration signal according to the basic setting based on the result of analyzing the subtitle information to generate a vibration signal.

[0126] The following is a specific example. The vibration generator 64 can adjust and generate a vibration signal so that the vibration corresponding to the human utterance is weaker than the vibration according to the basic setting in the vibration device 4. The vibration generator 64 can also adjust and generate a vibration signal so that the vibration corresponding to the sound effect is relatively weaker than the vibration corresponding to the human utterance. Generally, the human utterance time period tends to be relatively long, and if a strong vibration is provided for a long time accordingly, the viewer may feel uncomfortable. Therefore, for example, when the vibration signal generated according to the basic setting as described above is adjusted based on the sound data corresponding to the human utterance to generate a vibration signal so that the amplitude of the waveform becomes low, this makes it possible to provide more comfortable viewing.

[0127] In addition, the following is another specific example. The vibration generator 64 can adjust the vibration signal according to the basic setting to generate a vibration signal so that the vibration frequency corresponding to the female's voice is higher than the vibration frequency corresponding to the male's voice. Generally, the voice of an adult female is higher than that of an adult male. Therefore, it is possible to intuitively grasp which of the male's voice and the female's voice the vibration corresponds to in a scene where a male and a female are talking, based on whether the frequency is high.

[0128] Furthermore, in addition to using subtitle information and sound information, content meta information can also be used to generate vibration signals. For example, when the content meta information indicates action as the video type, vibration signals can be generated only for sound effects, not for human utterances. This makes it possible to provide dynamic vibration signals throughout the video that depend on the action scene.

[0129] [Method for generating vibration]

[0130] Next, use Figure 3 A method for generating vibrations performed by the controller 6 is described.

[0131] Figure 3 is a diagram of the basic flow of a method for generating vibrations.

[0132] First, the video file is acquired by the acquisition unit 62, and the video is played back (S1).

[0133] Next, analysis regarding the presence or absence of subtitle information in the video file is performed by the analyzer 63 (S2), and it is determined whether subtitle information exists (S3).

[0134] When analyzing the subtitle information, the presence or absence of the subtitle information is determined. In addition, when the subtitle information exists, the analyzer 63 divides the subtitle data into sound effect subtitle data and utterance subtitle data using the following information included in the subtitle information: display position information about the position where the subtitle is displayed on the image, mark information about the mark displayed on the image, font information about the font of the text in the subtitle, and color information about the color of the text in the subtitle. In addition, in the case of a scene where multiple people are talking, the utterance subtitle data is divided according to the order of utterances in the conversation based on the position of the round bracket or the square bracket.

[0135] When it is determined that there is no subtitle information (No), the process returns to S2, and the processes of S2 and S3 are repeated. When it is determined that there is subtitle information (Yes), the process moves to S4.

[0136] In S4, the sound information included in the video file is analyzed by the analyzer 63 (S4).

[0137] When the analyzer 63 determines that both sound effect subtitle data and voice subtitle data exist as a result of the analysis of the subtitle information, the sound data corresponding to the sound effect and the sound data corresponding to the human voice are separated from each other by analyzing the sound information, and these sound data are respectively associated with the sound effect subtitle data and the voice subtitle data.

[0138] Furthermore, when the result of the analysis of the subtitle information by the analyzer 63 determines that only one of the sound effect subtitle data and the vocal subtitle data exists, the sound data does not need to be separated, and the subtitle data is associated with the sound data.

[0139] Next, a vibration signal is generated by the vibration generator 64 based on the result of analyzing the subtitle information and the sound information, and the vibration signal is output to the vibration device (S5) by the vibration generator 64. In other words, a vibration signal corresponding to the sound effect is generated based on the waveform of the sound data corresponding to the sound effect, a vibration signal corresponding to the voicing is generated based on the waveform of the sound data corresponding to the voicing, and the vibration signal is output to each vibration device.

[0140] For example, the process of generating vibration is started when the viewer holds the information processing device 1 serving as the vibration providing device, and the process of generating vibration is terminated when the viewer removes his / her hand from the information processing device 1. In addition, when there are no subtitles on the screen, control is performed to prevent the vibration devices 4L, 4R from generating vibration.

[0141] Next, use Figure 4 A specific example of a method for generating a vibration signal is described.

[0142] Here, an example is described in which a vibration corresponding to a sound effect is output to the left-hand vibration device 4L and a vibration corresponding to a sound emission is output to the right-hand vibration device 4R. However, a vibration corresponding to a sound effect may be output to the right-hand vibration device 4R, and a vibration corresponding to a sound emission may be output to the left-hand vibration device 4L.

[0143] When vibrations corresponding to sound effects and vibrations corresponding to human vocalization are respectively generated by different vibration devices as described above, this enables the viewer to deeply understand the situation in the scene.

[0144] Figure 4 is a diagram of a flow chart of an example of a method for generating vibration when two vibration devices 4R and 4L are each used to provide vibration, and shows in more detail Figure 3 S5 processing. Figure 3 Similar steps are described by Figure 3 Step names similar to those used in are denoted by step names, and their descriptions are omitted. Figure 4 S51 to S54 in correspond to Figure 3 S5 in.

[0145] like Figure 4 As shown, after S4, a vibration signal is generated based on the waveform of the sound data of each of the sound effect and the human utterance by the vibration generator 64 using the analysis result (S51). More specifically, a left-hand vibration signal and a right-hand vibration signal are generated, the left-hand vibration signal being a vibration signal corresponding to the sound data corresponding to the sound effect, and the right-hand vibration signal being a vibration signal corresponding to the sound data corresponding to the utterance, the sound data being separated from each other by the analysis.

[0146] Next, it is determined by the vibration generator 64 whether the generated vibration signal corresponds to a sound effect (S52). When it is determined that the generated vibration signal corresponds to a sound effect (Yes), the process proceeds to S53, and the left-hand vibration signal as a vibration signal corresponding to the sound effect is output to the left-hand vibration device 4L. When it is determined that the generated vibration signal does not correspond to a sound effect (No), the process proceeds to S54, and the right-hand vibration signal as a vibration signal corresponding to the utterance of a person is output to the right-hand vibration device 4R.

[0147] Therefore, the left hand vibration device 4L is driven based on the input left hand vibration signal, and the vibration corresponding to the sound effect is provided to the user. Similarly, the right hand vibration device 4R is driven based on the input right hand vibration signal, and the vibration corresponding to the human utterance is provided to the user.

[0148] exist Figure 5 In the example of scenario 40 shown in (A), a vibration signal corresponding to the sound is generated based on the waveform of the sound data corresponding to the sound, and a vibration signal corresponding to the alarm corresponding to the sound effect is generated based on the waveform of the sound data corresponding to the alarm, such as Figure 5 The vibration devices 4R, 4L are driven based on the vibration signal, and provide vibration to the viewer.

[0149] Note that Figure 5 In (B), "sound" represents sound data, and "vibration" represents a vibration signal. The same applies to Figure 6 (B) and Figure 7 (B).

[0150] In addition, Figure 6 In the example of scene 43 shown in (A), a vibration signal corresponding to a male's utterance is generated based on the waveform of the sound data corresponding to the male's utterance, and a vibration signal corresponding to a female's utterance is generated based on the waveform of the sound data corresponding to the female's utterance, as shown in FIG. Figure 6 As shown in (B).

[0151] In addition, Figure 7 In the example of scene 45 shown in (A) of FIG. 4 , a vibration signal corresponding to the horn sound corresponding to the sound effect is generated based on the waveform of the sound data corresponding to the horn sound, such as Figure 7 As shown in (B).

[0152] When a vibration signal is generated based on the sound data corresponding to the utterance to provide vibration to the viewer as described above, this enables the viewer to understand by touch the content that is unreadable using only the subtitle information (such as the pitch change and volume of the utterance and the speed of the utterance). Therefore, the viewer can easily imagine, for example, the personality and feelings of the person, and can understand the situation more deeply. Similarly, in the case of sound effects, vibration enables the viewer to understand, for example, the volume of the sound, the change in volume, and the duration. Therefore, the viewer can understand the situation more deeply.

[0153] In addition, when generating vibration, a vibration signal can be generated by performing adjustment so that the vibration corresponding to the sound effect is relatively weaker than the vibration corresponding to the human utterance. As described above, this makes it possible to provide comfortable viewing.

[0154] In addition, when the gender of the person making the sound is determined as a result of analyzing the subtitle information when generating the vibration, the vibration generator 64 can generate a vibration signal by performing adjustment so that the vibration frequency corresponding to the sound of a woman is higher than the vibration frequency corresponding to the sound of a man. Therefore, when the viewer is watching a scene in which a man and a woman are talking, the viewer is provided with vibration, and this enables the viewer to easily and intuitively recognize which of the sound is the sound of a man and the sound of a woman.

[0155] In addition, for a scene in which multiple people are talking, when the voiced subtitle data is analyzed to determine whether to use parentheses or square brackets to divide the voiced subtitle data according to the order of utterances in the conversation, the vibration generator 64 can stop the vibration corresponding to the sound effects in the scene, can limit the vibration device that provides the vibration corresponding to the first sound, and can thereafter alternately vibrate the vibration device on the left and the vibration device on the right for each division.

[0156] For example, for a scene where two people are talking, left-hand and right-hand vibration signals are generated, such as outputting vibrations corresponding to a first utterance to the right-hand vibration device 4R and outputting vibrations corresponding to a second utterance to the left-hand vibration device 4L.

[0157] In addition, for a scenario in which three or more people are talking, a left-hand vibration signal and a right-hand vibration signal are generated so that the vibration device 4R and the vibration device 4L are driven alternately, for example, the vibration corresponding to the first sound is output to the right-hand vibration device 4R, the vibration corresponding to the second sound is output to the left-hand vibration device 4L, and the vibration corresponding to the third sound is output to the right-hand vibration device 4R.

[0158] Therefore, vibration enables the viewer to understand the pauses in the conversation and the speed at which each person speaks. As a result, the viewer can understand the context of the scene more deeply.

[0159] When it is determined by analyzing the subtitle information that a plurality of people in the scene are talking, as described above, a vibration signal for the talk mode may be generated, and in other cases, a vibration signal for the normal mode may be generated. In the talk mode, the vibration corresponding to the sound effect is stopped, and the vibration signal is alternately output to the left vibration device 4L and the right vibration device 4R per each talk segment. In the normal mode, the vibration signal corresponding to the sound effect is output to the left-hand vibration device, and the vibration signal corresponding to the utterance of a person is output to the right-hand vibration device.

[0160] Note that an example in which two vibration devices are present has been described above. However, the number of vibration devices may be one, and a single vibration device may be configured such that the single vibration device can vibrate based on a vibration signal corresponding to a sound emission and can vibrate based on a vibration signal corresponding to a sound effect.

[0161] In this case, if the vibration corresponding to the sound emission and the vibration corresponding to the sound effect occur at the same time, one of the vibration corresponding to the sound emission and the vibration corresponding to the sound effect can be stopped. For example, the content meta information can be used to determine which of the vibration corresponding to the sound emission and the vibration corresponding to the sound effect is stopped.

[0162] For example, when the content meta information indicates that the action is a video type, the vibration corresponding to the person's vocalization is stopped, and a vibration signal corresponding to a sound effect is output. This provides dynamic vibration according to the action scene and allows a deeper understanding of the scene.

[0163] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications may be made thereto without departing from the scope of the present technology.

[0164] In the above embodiment, a cellular phone including a display unit and a vibration device in addition to a controller has been described as an example of an information processing device. However, the information processing device is not limited to this. It is sufficient if the information processing device includes at least a controller and a communication unit for establishing a connection with an external device.

[0165] For example, the vibration providing system described above can be applied to a movie theater; and a screen used as a display unit and a chair or glasses or a vest including a vibration device, etc., can be used as an external device, wherein the glasses or the vest, etc. can be directly worn by the viewer. Thus, the information processing device can communicate with the external device. It is sufficient if the chair, glasses, vest, etc. include at least one vibration device, wherein at least one vibration device can provide vibration as in the case of the vibration device in the above embodiment.

[0166] Furthermore, the hearing aid may include a vibration device and may be used as an external device capable of communicating with an information processing device including a controller. This allows, for example, if the hearing of a user of the hearing aid is impaired, the user to be alerted to a sound by providing a vibration.

[0167] In addition, in the above embodiments, an example has been described in which it is assumed that the video file is, for example, a movie or a drama. However, the present technology can be applied to, for example, a video file of a game. For example, a video file of a role-playing game (RPG) or a simulation game includes subtitle information and sound information. Therefore, a vibration signal can be generated using the subtitle information and the sound information.

[0168] Furthermore, when playing back a video file including sound position information (eg, virtual reality (VR) content), the sound position information may also be used to determine, for example, which vibration signal to use to drive which vibration device among multiple vibration devices.

[0169] Furthermore, in the above embodiments, an example in which two vibration devices or a single vibration device is used has been described. However, three or more vibration devices may be used.

[0170] In addition, in the above embodiments, an example of generating vibration mainly using subtitle information and sound information has been described. However, in addition to using subtitle information and sound information, content meta information, video information, and viewer information can also be used to generate a vibration signal. This makes it possible to provide a vibration suitable for helping to understand the details of the scene.

[0171] By generating a vibration signal in consideration of content meta information, vibration according to the characteristics of the image content can be provided. For example, when playing back a video file whose video type is action, as described above, vibration can be generated only for sound effects, not for human voice.

[0172] Furthermore, by generating vibration signals taking into account video information, a deeper understanding of the details of the scene can be achieved. Figure 5 In the scene 40 shown in (A) of FIG. 4 , a person who buries his head in his hands is detected using video information, and the person is identified as sad. Based on the result of the identification, a vibration signal can be generated so that the vibration corresponding to the person's vocalization is weaker than the vibration signal according to the basic setting. Therefore, the vibration makes it possible to more deeply understand that the person is sad.

[0173] In addition, by generating a vibration signal in consideration of viewer information, a vibration that is more favorable to the viewer can be provided. As described above, the viewer information includes attribute information about the attributes of the viewer as a user and viewing environment information. Examples of the attribute information about the attributes of the viewer include the hearing loss state, age, and vibration preference of the viewer.

[0174] Depending on the hearing loss state of the viewer, the amplitude of the vibration can be adjusted so that, for example, the vibration is stronger in the case of complete inaudibility than in the case of partial inaudibility.

[0175] Furthermore, elderly people often experience hearing loss and a reduced ability to sense vibrations due to aging. Thus, using age information, the amplitude of the vibrations can be adjusted so that, for example, the vibrations are stronger when the viewer is elderly.

[0176] In addition, there are individual differences in preferences regarding the amplitude of vibration. Therefore, a more favorable vibration can be provided by the viewer setting his / her vibration preference.

[0177] In addition, for viewer environment information, outdoor environments are generally noisier than indoor environments. Therefore, for example, by performing adjustments so that vibrations in outdoor environments are stronger than vibrations in indoor environments, vibrations suitable for the viewing environment can be provided to the viewer.

[0178] The magnitude of the vibration may be adjusted by performing comprehensive determination based on viewer information.

[0179] In addition, in the above embodiment, it has been described that a vibration sensation is provided as an example of providing a skin sensation. However, a temperature sensation such as warmth and coldness or a pressure sensation such as a pressure sensation may be provided to the user as a skin sensation. In this case, a device for providing a sensation such as warmth, coolness or pressure is used.

[0180] For example, when video information is used in addition to subtitle information and sound information to provide a skin sensation, and when it has been determined by analyzing the subtitle information and sound information that words such as blazing fire, fire, or flame are included, and it has been determined by analyzing the video information that the image is red due to the flame, a device for providing a warm sensation can be used to provide a warm sensation of temperature to the user.

[0181] Furthermore, by providing wind or water to the user using, for example, an air blowing device or a device capable of draining water, a wind sensation or a water sensation can be provided to the user. This is an example of providing a skin sensation.

[0182] Furthermore, two or more sensations of vibration sensation, temperature sensation, pressure sensation, wind sensation, water sensation, etc. may be combined so that a variety of sensations may be provided.

[0183] Note that the present technology can also adopt the following configurations.

[0184] (1) An information processing device comprising:

[0185] A controller that uses the results of analyzing subtitle information and sound information included in a video file to generate at least one of a vibration signal corresponding to sound effect subtitle data and a vibration signal corresponding to vocal subtitle data based on the waveform of the sound data, wherein the sound effect subtitle data is used to represent sound effects in the form of text information, and the vocal subtitle data is used to represent human voice in the form of text information, the sound effect subtitle data and the vocal subtitle data are included in the subtitle data included in the subtitle information, and the sound data is included in the sound information.

[0186] (2) The information processing device according to (1), wherein:

[0187] The controller divides the sound data into sound data corresponding to the sound effect subtitle data and sound data corresponding to the voiced subtitle data, and

[0188] The controller generates a vibration signal corresponding to the audio effect subtitle data based on a waveform of sound data corresponding to the audio effect subtitle data, and generates a vibration signal corresponding to the spoken subtitle data based on a waveform of sound data corresponding to the spoken subtitle data.

[0189] (3) The information processing device according to (2), wherein:

[0190] The controller uses at least one of display position information, mark information, font information and color information included in the subtitle information to determine whether the subtitle data is the sound effect subtitle data or the voice subtitle data, the display position information is information about the position where the subtitle is displayed on the image, the mark information is information about the mark displayed on the image, the font information is information about the font of the text in the subtitle, and the color information is information about the color of the text in the subtitle.

[0191] (4) The information processing device according to (1) or (2), wherein:

[0192] In a case where the subtitle information includes subtitle information corresponding to a scene in which multiple persons are talking, the controller uses the subtitle information and segments the voiced subtitle data in the scene in which multiple persons are talking according to the order of voice in the conversation, and generates a vibration signal corresponding to each voiced subtitle data obtained by the segmentation.

[0193] (5) The information processing device according to (3), wherein:

[0194] The controller generates a vibration signal corresponding to the sound effect and a vibration signal corresponding to the person's vocalization, so that there is a relative difference in amplitude between the vibration based on the vibration signal corresponding to the sound effect and the vibration based on the vibration signal corresponding to the person's vocalization.

[0195] (6) The information processing device according to any one of (1) to (4), wherein:

[0196] The video file includes content meta information, and

[0197] The controller generates the vibration signal taking into account the content meta-information.

[0198] (7) The information processing device according to any one of (1) to (6), wherein:

[0199] The controller generates the vibration signal in consideration of information about the user.

[0200] (8) The information processing device according to any one of (1) to (7), further comprising:

[0201] A vibration providing unit provides vibration to the user based on the vibration signal.

[0202] (9) The information processing device according to (8), comprising:

[0203] A plurality of the vibration providing units.

[0204] (10) The information processing device according to (9), wherein:

[0205] The controller uses the results of analyzing the subtitle information and the sound information to generate a vibration signal corresponding to the sound effect subtitle data and a vibration signal corresponding to the voiced subtitle data, and outputs the generated vibration signal to different vibration providing units among the multiple vibration providing units.

[0206] (11) An information processing method comprising:

[0207] Using the results of analyzing the subtitle information and sound information included in the video file, at least one of a vibration signal corresponding to the sound effect subtitle data and a vibration signal corresponding to the sound subtitle data is generated based on the waveform of the sound data, the sound effect subtitle data is used to represent the sound effect in the form of text information, and the sound subtitle data is used to represent the human sound in the form of text information, the sound effect subtitle data and the sound subtitle data are included in the subtitle data included in the subtitle information, and the sound data is included in the sound information.

[0208] (12) A program causing an information processing device to execute a process including the following operations:

[0209] Analyzing subtitle information and sound information included in a video file, the subtitle information including subtitle data including sound effect subtitle data and voice subtitle data, the sound information including sound data, the sound effect subtitle data being used to represent sound effects in the form of text information, and the voice subtitle data being used to represent human voice in the form of text information; and

[0210] Using the result of the analysis, at least one of a vibration signal corresponding to the audio subtitle data and a vibration signal corresponding to the spoken subtitle data is generated based on the waveform of the sound data.

[0211] Reference numerals list

[0212] 1…Information processing device

[0213] 6…Controller

[0214] 42,44,52,55…voice subtitle data

[0215] 41,46,48,51,54,56…audio subtitle data

Claims

1. An information processing device, comprising: A controller, which uses the result of analyzing the subtitle information and sound information included in the video file, and generates at least one of a vibration signal corresponding to sound effect subtitle data and a vibration signal corresponding to vocal subtitle data based on the waveform of the sound data, wherein the sound effect subtitle data is used to represent the sound effect in the form of text information, and the vocal subtitle data is used to represent the human voice in the form of text information, the sound effect subtitle data and the vocal subtitle data are included in the subtitle data included in the subtitle information, and the sound data is included in the sound information, wherein the controller divides the sound data into sound data corresponding to the sound effect subtitle data and sound data corresponding to the voiced subtitle data, and The controller generates a vibration signal corresponding to the audio effect subtitle data based on a waveform of sound data corresponding to the audio effect subtitle data, and generates a vibration signal corresponding to the spoken subtitle data based on a waveform of sound data corresponding to the spoken subtitle data.

2. The information processing device according to claim 1, wherein: The controller uses at least one of display position information, mark information, font information and color information included in the subtitle information to determine whether the subtitle data is the sound effect subtitle data or the voice subtitle data, the display position information is information about the position where the subtitle is displayed on the image, the mark information is information about the mark displayed on the image, the font information is information about the font of the text in the subtitle, and the color information is information about the color of the text in the subtitle.

3. The information processing device according to claim 2, wherein: In a case where the subtitle information includes subtitle information corresponding to a scene in which multiple persons are talking, the controller uses the subtitle information and divides the voiced subtitle data in the scene in which multiple persons are talking according to the order of voice in the conversation, and generates a vibration signal corresponding to each voiced subtitle data obtained by the division.

4. The information processing device according to claim 2, wherein: The controller generates a vibration signal corresponding to the sound effect and a vibration signal corresponding to the person's vocalization, so that there is a relative difference in amplitude between the vibration based on the vibration signal corresponding to the sound effect and the vibration based on the vibration signal corresponding to the person's vocalization.

5. The information processing device according to claim 2, wherein: The video file includes content meta information, and The controller generates the vibration signal taking into account the content meta-information.

6. The information processing device according to claim 2, wherein: The controller generates the vibration signal in consideration of information about a user.

7. The information processing device according to claim 2, further comprising: A vibration providing unit provides vibration to a user based on the vibration signal.

8. The information processing device according to claim 7, comprising: A plurality of the vibration providing units.

9. The information processing device according to claim 8, wherein: The controller uses the results of analyzing the subtitle information and the sound information to generate a vibration signal corresponding to the sound effect subtitle data and a vibration signal corresponding to the voiced subtitle data, and outputs the generated vibration signal to different vibration providing units among the multiple vibration providing units.

10. An information processing method, comprising: Using the result of analyzing the subtitle information and sound information included in the video file, at least one of a vibration signal corresponding to the sound effect subtitle data and a vibration signal corresponding to the sound subtitle data is generated based on the waveform of the sound data, the sound effect subtitle data is used to represent the sound effect in the form of text information, the sound subtitle data is used to represent the sound of a person in the form of text information, the sound effect subtitle data and the sound subtitle data are included in the subtitle data included in the subtitle information, and the sound data is included in the sound information, The sound data is divided into sound data corresponding to the sound effect subtitle data and sound data corresponding to the voiced subtitle data, and A vibration signal corresponding to the audio effect subtitle data is generated based on a waveform of sound data corresponding to the audio effect subtitle data, and a vibration signal corresponding to the spoken subtitle data is generated based on a waveform of sound data corresponding to the spoken subtitle data.

11. A program product, the program product causing an information processing device to perform a process including the following operations: Analyzing subtitle information and sound information included in a video file, the subtitle information including subtitle data including sound effect subtitle data and voice subtitle data, the sound information including sound data, the sound effect subtitle data being used to represent sound effects in the form of text information, and the voice subtitle data being used to represent human voice in the form of text information; and Using the result of the analysis, at least one of a vibration signal corresponding to the audio subtitle data and a vibration signal corresponding to the spoken subtitle data is generated based on the waveform of the sound data, in, The sound data is divided into sound data corresponding to the sound effect subtitle data and sound data corresponding to the voiced subtitle data, and A vibration signal corresponding to the audio effect subtitle data is generated based on a waveform of sound data corresponding to the audio effect subtitle data, and a vibration signal corresponding to the spoken subtitle data is generated based on a waveform of sound data corresponding to the spoken subtitle data.

12. A recording medium having a program recorded thereon, the program being used to cause an information processing device to execute a process including the following operations: Analyzing subtitle information and sound information included in a video file, the subtitle information including subtitle data including sound effect subtitle data and voice subtitle data, the sound information including sound data, the sound effect subtitle data being used to represent sound effects in the form of text information, and the voice subtitle data being used to represent human voice in the form of text information; and Using the result of the analysis, at least one of a vibration signal corresponding to the audio subtitle data and a vibration signal corresponding to the spoken subtitle data is generated based on the waveform of the sound data, in, The sound data is divided into sound data corresponding to the sound effect subtitle data and sound data corresponding to the voiced subtitle data, and A vibration signal corresponding to the audio effect subtitle data is generated based on a waveform of sound data corresponding to the audio effect subtitle data, and a vibration signal corresponding to the spoken subtitle data is generated based on a waveform of sound data corresponding to the spoken subtitle data.

Images