System

Abstract

Systems according to embodiments aim to make movie watching a more realistic and immersive experience.SOLUTION: A system according to an embodiment includes a unit configured to analyze video, a unit configured to analyze audio, a conversion unit, and a provision unit. The video analyzing unit analyzes a video. The voice analyzing unit analyzes the voice. The conversion unit generates a vibration pattern based on results analyzed by the video analyzing unit and the audio analyzing unit. The provision unit provides the vibration pattern generated by the conversion unit to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies fall short in providing a realistic and immersive movie-watching experience, and there is room for improvement.

[0005] The system according to the embodiment aims to make watching movies a more realistic and immersive experience. [Means for solving the problem]

[0006] A system according to an embodiment includes a video analyzing unit, an audio analyzing unit, a conversion unit, and a providing unit. The video analyzing unit analyzes the video. The audio analyzing unit analyzes the audio. The conversion unit generates a vibration pattern based on the results of the analysis by the video analyzing unit and the audio analyzing unit. The providing unit provides the vibration pattern generated by the conversion unit to a user. [Effects of the Invention]

[0007] Systems according to embodiments can make watching movies a more realistic and immersive experience. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) An immersive home theater system according to an embodiment of the present invention generates vibration patterns based on analysis results of video and audio and provides them to a user. The immersive home theater system includes a video analysis unit, an audio analysis unit, a conversion unit that generates vibration patterns based on the analysis results, and a provision unit that provides the generated vibration patterns to a user. For example, the immersive home theater system analyzes movie scenes using video authentication technology and generates vibration patterns corresponding to specific scenes. The immersive home theater system also analyzes movie audio using audio authentication technology and generates vibration patterns corresponding to specific sounds. Furthermore, the immersive home theater system vibrates chairs or floors according to movie scenes and audio. This allows the immersive home theater system to make movie viewing more realistic and immersive. This allows the immersive home theater system to enjoy an attraction-like movie experience even at home. For example, by providing immersive sensations according to movie scenes, a new movie experience can be provided that fully utilizes the three senses of sight, hearing, and touch.

[0029] An immersive home theater system according to an embodiment includes a video analysis unit, an audio analysis unit, a conversion unit, and a providing unit. The video analysis unit analyzes movie scenes. For example, the video analysis unit may recognize specific scenes using image recognition technology. The video analysis unit may also analyze the content of a scene using frame analysis technology. The video analysis unit may also extract scene features using AI. For example, the video analysis unit may recognize specific scenes and generate a vibration pattern corresponding to the scene. The audio analysis unit analyzes movie audio. For example, the audio analysis unit may recognize specific sounds using audio recognition technology. The audio analysis unit may also analyze audio features using frequency analysis technology. The audio analysis unit may also extract audio features using AI. For example, the audio analysis unit may recognize specific sounds and generate a vibration pattern corresponding to the sounds. The conversion unit generates a vibration pattern based on the analysis results of the video and audio. For example, the conversion unit may cause a chair or floor to vibrate in accordance with the movie scenes and audio. The conversion unit can also generate a vibration pattern using AI. For example, the conversion unit causes a chair or floor to vibrate according to a scene in a movie. The provision unit provides the generated vibration pattern to a user. For example, the provision unit transmits the generated vibration pattern to a chair or floor. The provision unit can also provide a vibration pattern using AI. For example, the provision unit provides the generated vibration pattern to a user. In this way, the immersive home theater system according to the embodiment can generate a vibration pattern based on the analysis results of video and audio, and provide it to a user.

[0030] The authentication unit can recognize a specific scene and generate a vibration pattern corresponding to the scene. Specific scenes include, but are not limited to, action scenes and moving scenes. The authentication unit can recognize a specific scene using, for example, image recognition technology. The authentication unit can also analyze the content of a scene using frame analysis technology. Furthermore, the authentication unit can extract scene features using AI. For example, the authentication unit can recognize a specific scene and generate a vibration pattern corresponding to the scene. This allows for the generation of a vibration pattern corresponding to the specific scene, providing a more realistic experience. Some or all of the above-described processing in the authentication unit may be performed using AI, for example, or may be performed without using AI. For example, to recognize a specific scene, the authentication unit can input image data to a generation AI and have the generation AI recognize the scene.

[0031] The authentication unit can recognize a specific sound and generate a vibration pattern corresponding to that sound. Specific sounds include, but are not limited to, explosions, music, etc. The authentication unit can recognize the specific sound using, for example, voice recognition technology. The authentication unit can also analyze the characteristics of the sound using frequency analysis technology. Furthermore, the authentication unit can extract the characteristics of the sound using AI. For example, the authentication unit can recognize a specific sound and generate a vibration pattern corresponding to that sound. By generating a vibration pattern corresponding to the specific sound, a more realistic experience can be provided. Some or all of the above-mentioned processing in the authentication unit can be performed using, for example, AI, or can be performed without using AI. For example, to recognize a specific sound, the authentication unit can input voice data to a generation AI and have the generation AI recognize the sound.

[0032] The conversion unit can make a chair or floor vibrate according to a scene in a movie. Movie scenes include, but are not limited to, action scenes and moving scenes. For example, the conversion unit can make a chair or floor vibrate according to a scene in a movie. The conversion unit can also generate a vibration pattern using AI. For example, the conversion unit can make a chair or floor vibrate according to a scene in a movie. This allows the chair or floor to vibrate according to the scene and sound of the movie, providing a more realistic experience. Some or all of the above-described processing in the conversion unit may be performed using AI, for example, or may be performed without using AI. For example, the conversion unit can input scene data to a generation AI to generate a vibration pattern according to a scene in a movie, and cause the generation AI to generate a vibration pattern.

[0033] The providing unit can provide the generated vibration pattern to the user. The generated vibration pattern includes, for example, vibration intensity, rhythm, duration, etc., but is not limited to these examples. The providing unit, for example, transmits the generated vibration pattern to a chair or floor. The providing unit can also control a device for providing the generated vibration pattern to the user. Furthermore, the providing unit can provide the vibration pattern using AI. For example, the providing unit provides the generated vibration pattern to the user. By providing the generated vibration pattern to the user, a realistic sensation can be provided. Some or all of the above-described processing in the providing unit may be performed, for example, using AI or may be performed without using AI. For example, in order to provide the generated vibration pattern, the providing unit can input vibration data to a generating AI and cause the generating AI to provide vibrations.

[0034] The authentication unit that analyzes the video can apply different analysis algorithms depending on the genre of the movie. Movie genres include, but are not limited to, action, drama, and comedy. For example, in the case of an action movie, the authentication unit that analyzes the video can apply an analysis algorithm specialized for fast-moving scenes. In addition, in the case of a horror movie, the authentication unit that analyzes the video can apply an analysis algorithm that emphasizes dark scenes and sudden movements. In addition, in the case of a comedy movie, the authentication unit that analyzes the video can apply an analysis algorithm that emphasizes facial expressions and gestures. In this way, by applying an analysis algorithm depending on the genre of the movie, more appropriate video analysis can be performed. Some or all of the above-mentioned processing in the authentication unit that analyzes the video can be performed using, for example, AI, or without AI. For example, the authentication unit that analyzes the video can input movie genre data into the generation AI and cause the generation AI to apply an analysis algorithm depending on the genre.

[0035] The authentication unit that analyzes the video can improve the accuracy of the analysis by referring to past analysis results. Past analysis results include, but are not limited to, analysis results of the same movie scene and analysis results of similar scenes. The authentication unit that analyzes the video can improve the analysis accuracy by, for example, referring to previously analyzed scenes of the same movie. The authentication unit that analyzes the video can also improve the analysis accuracy of similar scenes based on data obtained from past analysis results. The authentication unit that analyzes the video can also learn from past analysis results and optimize the analysis algorithm. For example, the authentication unit that analyzes the video can improve the analysis accuracy by referring to previously analyzed scenes of the same movie. In this way, the analysis accuracy can be improved by referring to past analysis results. Some or all of the above-mentioned processing in the authentication unit that analyzes the video may be performed using, for example, AI, or may be performed without using AI. For example, the authentication unit that analyzes the video can input past analysis result data into the generation AI and cause the generation AI to improve the analysis accuracy.

[0036] The authentication unit that analyzes the video can recognize specific characters or objects and generate a corresponding vibration pattern. Specific characters or objects include, but are not limited to, main characters and explosions. The authentication unit that analyzes the video can recognize specific characters using, for example, facial recognition technology. The authentication unit that analyzes the video can also recognize specific objects using object detection technology. The authentication unit that analyzes the video can also extract characteristics of characters and objects using AI. For example, the authentication unit that analyzes the video can generate a vibration pattern when the main character fights in an action scene. The authentication unit that analyzes the video can also generate a vibration pattern when a specific object (e.g., an explosion) occurs. The authentication unit that analyzes the video can also generate a vibration pattern corresponding to the emotion of a character in an emotional scene. This allows for a more realistic experience by generating a vibration pattern corresponding to a specific character or object. Some or all of the above-described processing in the authentication unit that analyzes the video can be performed using, for example, AI, or without AI. For example, the authentication unit that analyzes the video can input data of a specific character or object into the generation AI, causing the generation AI to generate a vibration pattern.

[0037] The authentication unit that analyzes the video can identify important scenes using the user's gaze tracking data. The gaze tracking data includes, but is not limited to, gaze position and gaze movement, for example. The authentication unit that analyzes the video can, for example, identify a scene where the user gazes for a long time as an important scene. The authentication unit that analyzes the video can also identify a scene where the user frequently moves their gaze as an important scene. The authentication unit that analyzes the video can also identify a scene where the user focuses their gaze as an important scene. This makes it easier to identify important scenes by using the user's gaze tracking data. Some or all of the above-described processing in the authentication unit that analyzes the video can be performed using, for example, AI, or without AI. For example, the authentication unit that analyzes the video can input the gaze tracking data to a generation AI and cause the generation AI to identify important scenes.

[0038] The authentication unit that analyzes the video can analyze the subtitle information of the movie and complete the scene content. Subtitle information includes, but is not limited to, for example, character recognition and context analysis. The authentication unit that analyzes the video can, for example, analyze the subtitle information of the movie and complete the scene content. The authentication unit that analyzes the video can also analyze the lines of the characters from the subtitle information and complete the scene content. The authentication unit that analyzes the video can also complete the background information of the scene based on the subtitle information. This makes it easier to complete the scene content by analyzing the subtitle information of the movie. Some or all of the above-mentioned processing in the authentication unit that analyzes the video can be performed using, for example, AI, or can be performed without using AI. For example, the authentication unit that analyzes the video can input the subtitle information to a generation AI and have the generation AI complete the scene content.

[0039] The authentication unit that analyzes the video can customize the analysis method by reflecting the user's past viewing history. The past viewing history may include, but is not limited to, a list of movies viewed and viewing time. For example, the authentication unit that analyzes the video may prioritize analyzing favorite scenes based on the user's past viewing history. The authentication unit that analyzes the video may also analyze scenes that are likely to be of interest based on the user's past viewing history. The authentication unit that analyzes the video may also learn the user's past viewing history and optimize the analysis algorithm. For example, the authentication unit that analyzes the video may prioritize analyzing favorite scenes based on the user's past viewing history. This makes it easier to customize the analysis method by reflecting the user's past viewing history. Some or all of the above-described processing in the authentication unit that analyzes the video may be performed using, for example, AI, or may be performed without AI. For example, the authentication unit that analyzes the video may input past viewing history data into a generation AI and have the generation AI customize the analysis method.

[0040] The authentication unit that analyzes the audio can apply different analysis algorithms depending on the genre of the movie. Movie genres include, but are not limited to, action, drama, and comedy. For example, in the case of an action movie, the authentication unit that analyzes the audio can apply an analysis algorithm specialized for explosions and gunshots. In addition, in the case of a horror movie, the authentication unit that analyzes the audio can apply an analysis algorithm specialized for frightening sound effects. In addition, in the case of a comedy movie, the authentication unit that analyzes the audio can apply an analysis algorithm specialized for laughter and humorous sounds. In this way, by applying an analysis algorithm according to the genre of the movie, more appropriate audio analysis can be performed. Some or all of the above-mentioned processing in the authentication unit that analyzes the audio may be performed using, for example, AI, or may be performed without using AI. For example, the authentication unit that analyzes the audio can input movie genre data into the generation AI and cause the generation AI to apply an analysis algorithm according to the genre.

[0041] The authentication unit that analyzes the audio can improve the accuracy of the analysis by referring to past analysis results. Past analysis results include, but are not limited to, analysis results of audio from the same movie and analysis results of similar audio. The authentication unit that analyzes the audio can improve the analysis accuracy by, for example, referring to audio from the same movie that was analyzed in the past. The authentication unit that analyzes the audio can also improve the analysis accuracy of similar audio based on data obtained from past analysis results. The authentication unit that analyzes the audio can also learn from past analysis results and optimize the analysis algorithm. For example, the authentication unit that analyzes the audio can improve the analysis accuracy by referring to audio from the same movie that was analyzed in the past. In this way, the analysis accuracy can be improved by referring to past analysis results. Some or all of the above-mentioned processing in the authentication unit that analyzes the audio may be performed using, for example, AI, or may be performed without using AI. For example, the authentication unit that analyzes the audio can input past analysis result data into the generation AI and cause the generation AI to improve the analysis accuracy.

[0042] The authentication unit that analyzes the audio can recognize specific sound effects or music and generate a corresponding vibration pattern. Examples of specific sound effects or music include, but are not limited to, explosions and specific music. The authentication unit that analyzes the audio can recognize specific sound effects or music using, for example, voice recognition technology. The authentication unit that analyzes the audio can also analyze the characteristics of sound effects or music using frequency analysis technology. The authentication unit that analyzes the audio can also extract the characteristics of sound effects or music using AI. For example, the authentication unit that analyzes the audio can generate a vibration pattern when an explosion occurs. The authentication unit that analyzes the audio can also generate a vibration pattern corresponding to specific music in a scene where the music is playing. The authentication unit that analyzes the audio can also generate a vibration pattern corresponding to a character's scream or laughter. This allows for a more realistic experience by generating a vibration pattern corresponding to specific sound effects or music. Some or all of the above-described processing in the authentication unit that analyzes the audio may be performed using, for example, AI, or may be performed without using AI. For example, an authentication unit that analyzes voice can input specific sound effects or music data into a generation AI, causing the generation AI to generate vibration patterns.

[0043] The authentication unit that analyzes the voice can customize the analysis method by taking into account the user's hearing characteristics. The user's hearing characteristics include, but are not limited to, hearing test results and sound preferences. The authentication unit that analyzes the voice can, for example, adjust the voice analysis algorithm based on the user's hearing characteristics. The authentication unit that analyzes the voice can also learn the user's hearing characteristics and propose an optimal voice analysis method. The authentication unit that analyzes the voice can also apply an analysis method that emphasizes specific frequency bands depending on the user's hearing characteristics. This allows for more appropriate voice analysis by taking the user's hearing characteristics into account. Some or all of the above-described processing in the authentication unit that analyzes the voice can be performed using, for example, AI, or without AI. For example, the authentication unit that analyzes the voice can input the user's hearing characteristic data into a generation AI and have the generation AI customize the analysis method.

[0044] The authentication unit that analyzes audio can analyze dialogue information in a movie and complete the content of a scene. Dialogue information includes, but is not limited to, character recognition and context analysis. The authentication unit that analyzes audio can, for example, analyze dialogue information in a movie and complete the content of a scene. The authentication unit that analyzes audio can also analyze character emotions from the dialogue information and complete the content of a scene. The authentication unit that analyzes audio can also complete background information of a scene based on the dialogue information. This makes it easier to complete the content of a scene by analyzing the dialogue information in a movie. Some or all of the above-mentioned processing in the authentication unit that analyzes audio can be performed using, for example, AI, or can be performed without using AI. For example, the authentication unit that analyzes audio can input dialogue information to a generation AI and have the generation AI complete the content of the scene.

[0045] The authentication unit that analyzes audio can customize the analysis method by reflecting the user's past listening history. The past listening history includes, for example, a list of music listened to and listening time, but is not limited to these examples. For example, the authentication unit that analyzes audio prioritizes analyzing preferred audio based on the user's past listening history. The authentication unit that analyzes audio can also analyze audio that is likely to be of interest to the user based on the user's past listening history. The authentication unit that analyzes audio can also learn the user's past listening history and optimize the analysis algorithm. For example, the authentication unit that analyzes audio prioritizes analyzing preferred audio based on the user's past listening history. This makes it easier to customize the analysis method by reflecting the user's past listening history. Some or all of the above-described processing in the authentication unit that analyzes audio may be performed using, for example, AI, or may be performed without using AI. For example, the authentication unit that analyzes audio can input past listening history data into a generation AI and have the generation AI customize the analysis method.

[0046] The conversion unit that generates the vibration pattern can change the rhythm of the vibration according to the tension of the movie scene. Examples of tension in movie scenes include, but are not limited to, action scenes, romance scenes, and horror scenes. For example, the conversion unit that generates the vibration pattern can generate a vibration pattern with a fast rhythm for action scenes. The conversion unit that generates the vibration pattern can also generate a vibration pattern with a slow rhythm for romance scenes. The conversion unit that generates the vibration pattern can also generate a vibration pattern with an irregular rhythm for horror scenes. This allows the vibration rhythm to be changed according to the tension of the movie scene, providing a more realistic experience. Some or all of the above-described processing in the conversion unit that generates the vibration pattern may be performed using, or without, AI. For example, the conversion unit that generates the vibration pattern can input tension data of the scene into the generation AI and cause the generation AI to change the rhythm.

[0047] The conversion unit that generates a vibration pattern can improve the accuracy of generation by referring to data on past vibration patterns. Past vibration pattern data includes, for example, vibration patterns in the same scene, vibration patterns in similar scenes, etc., but is not limited to these examples. The conversion unit that generates a vibration pattern can improve the accuracy by referring to, for example, previously generated vibration patterns. The conversion unit that generates a vibration pattern can also optimize vibration patterns in similar scenes based on past vibration pattern data. The conversion unit that generates a vibration pattern can also learn past vibration pattern data and optimize the generation algorithm. For example, the conversion unit that generates a vibration pattern can improve the accuracy by referring to previously generated vibration patterns. In this way, the accuracy of generation can be improved by referring to past vibration pattern data. Some or all of the above-described processing in the conversion unit that generates a vibration pattern may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit that generates a vibration pattern can input past vibration pattern data to the generation AI and cause the generation AI to improve the generation accuracy.

[0048] The conversion unit that generates the vibration pattern can combine multiple vibration patterns according to specific scenes and sound effects. Specific scenes and sound effects include, but are not limited to, explosion sounds in action scenes and music in romance scenes. The conversion unit that generates the vibration pattern can combine different vibration patterns according to, for example, explosion sounds and gunfire in action scenes. The conversion unit that generates the vibration pattern can also combine multiple vibration patterns according to music and character movements in romance scenes. The conversion unit that generates the vibration pattern can also combine multiple vibration patterns according to sudden sound effects and visual effects in horror scenes. This allows for a more realistic experience by combining multiple vibration patterns according to specific scenes and sound effects. Some or all of the above-described processing in the conversion unit that generates the vibration pattern can be performed using, for example, AI, or without AI. For example, the conversion unit that generates the vibration pattern can input data on scenes and sound effects into a generation AI and have the generation AI execute a combination of multiple vibration patterns.

[0049] The conversion unit that generates the vibration pattern can generate an appropriate vibration pattern by taking into account the user's physical characteristics. Examples of the user's physical characteristics include, but are not limited to, weight, height, and sensory sensitivity. The conversion unit that generates the vibration pattern can generate an optimal vibration pattern based on, for example, the user's weight and height. The conversion unit that generates the vibration pattern can also learn the user's physical characteristics and suggest an optimal vibration pattern. The conversion unit that generates the vibration pattern can also emphasize a specific vibration pattern depending on the user's physical characteristics. This allows for a more appropriate vibration experience by taking the user's physical characteristics into consideration. Some or all of the above-described processing in the conversion unit that generates the vibration pattern can be performed using, for example, AI, or without AI. For example, the conversion unit that generates the vibration pattern can input the user's physical characteristic data into a generation AI and cause the generation AI to generate an optimal vibration pattern.

[0050] The conversion unit that generates the vibration pattern can adjust the duration of the vibration according to the length of a scene in a movie. The length of a scene in a movie includes, but is not limited to, the timing of the start and end of the scene and the duration of the scene. For example, the conversion unit that generates the vibration pattern can generate a continuous vibration pattern for a long scene. The conversion unit that generates the vibration pattern can also generate an instantaneous vibration pattern for a short scene. The conversion unit that generates the vibration pattern can also dynamically adjust the duration of the vibration according to the length of the scene. This allows for adjusting the duration of the vibration according to the length of the scene in a movie, thereby providing a more appropriate vibration experience. Some or all of the above-described processing in the conversion unit that generates the vibration pattern may be performed using, or without, AI. For example, the conversion unit that generates the vibration pattern can input scene length data to the generation AI and have the generation AI adjust the duration.

[0051] The conversion unit that generates the vibration pattern can customize the generation method by reflecting the user's past feedback. Past feedback includes, but is not limited to, user ratings and comments. The conversion unit that generates the vibration pattern can generate a preferred vibration pattern based on, for example, the user's past feedback. The conversion unit that generates the vibration pattern can also suggest an optimal vibration pattern based on the user's past feedback. The conversion unit that generates the vibration pattern can also learn the user's past feedback and optimize the generation algorithm. For example, the conversion unit that generates the vibration pattern can generate a preferred vibration pattern based on the user's past feedback. This makes it easier to customize the generation method by reflecting the user's past feedback. Some or all of the above-described processing in the conversion unit that generates the vibration pattern can be performed using, for example, AI, or can be performed without using AI. For example, the conversion unit that generates the vibration pattern can input past feedback data into a generation AI and have the generation AI customize the generation method.

[0052] The providing unit that provides the vibration pattern to the user can adjust the vibration intensity according to the user's seat position. The user's seat position includes, but is not limited to, for example, the seat arrangement and user position information. The providing unit that provides the vibration pattern to the user can, for example, increase the vibration intensity when the user is in a front seat. The providing unit that provides the vibration pattern to the user can also decrease the vibration intensity when the user is in a rear seat. The providing unit that provides the vibration pattern to the user can also dynamically adjust the vibration intensity according to the user's seat position. This makes it possible to provide a more appropriate vibration experience by adjusting the vibration intensity according to the user's seat position. Some or all of the above-described processing in the providing unit that provides the vibration pattern to the user may be performed using, for example, AI or without AI. For example, the providing unit that provides the vibration pattern to the user can input seat position data to a generation AI and cause the generation AI to adjust the intensity.

[0053] The providing unit that provides a vibration pattern to a user can optimize the providing method by referring to the user's past vibration experiences. Past vibration experiences include, but are not limited to, vibration intensity, rhythm, and duration. The providing unit that provides a vibration pattern to a user can, for example, suggest an optimal providing method based on the user's past vibration experiences. The providing unit that provides a vibration pattern to a user can also provide a preferred vibration pattern based on the user's past vibration experiences. The providing unit that provides a vibration pattern to a user can also learn the user's past vibration experiences and optimize the providing algorithm. For example, the providing unit that provides a vibration pattern to a user can suggest an optimal providing method based on the user's past vibration experiences. This makes it easier to optimize the providing method by referring to the user's past vibration experiences. Some or all of the above-described processing in the providing unit that provides a vibration pattern to a user may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit that provides a vibration pattern to a user can input past vibration experience data into a generation AI and cause the generation AI to optimize the providing method.

[0054] The providing unit that provides the vibration pattern to the user can provide different vibration patterns depending on the type of movie scene. Types of movie scenes include, but are not limited to, action scenes and emotional scenes. For example, the providing unit that provides the vibration pattern to the user can provide a strong vibration pattern for action scenes. The providing unit that provides the vibration pattern to the user can also provide a gentle vibration pattern for romance scenes. The providing unit that provides the vibration pattern to the user can also provide an irregular vibration pattern for horror scenes. In this way, by providing different vibration patterns depending on the type of movie scene, a more realistic experience can be provided. Some or all of the above-described processing in the providing unit that provides the vibration pattern to the user may be performed using, or without using, AI, for example. For example, the providing unit that provides the vibration pattern to the user can input scene type data to a generation AI and cause the generation AI to provide different vibration patterns.

[0055] The providing unit that provides a vibration pattern to a user can provide optimal vibrations by taking into account the user's physical characteristics. Examples of the user's physical characteristics include, but are not limited to, weight, height, and sensory sensitivity. The providing unit that provides a vibration pattern to a user can provide optimal vibrations based on, for example, the user's weight and height. The providing unit that provides a vibration pattern to a user can also learn the user's physical characteristics and suggest an optimal vibration method. The providing unit that provides a vibration pattern to a user can also emphasize a specific vibration pattern depending on the user's physical characteristics. This allows for a more appropriate vibration experience by taking the user's physical characteristics into consideration. Some or all of the above-described processing by the providing unit that provides a vibration pattern to a user may be performed using, or without, AI. For example, the providing unit that provides a vibration pattern to a user can input the user's physical characteristic data into a generation AI and cause the generation AI to provide optimal vibrations.

[0056] The providing unit that provides a vibration pattern to a user can provide an optimal vibration by taking into account the user's device information. The user's device information includes, for example, the type of device, the device's performance, etc., but is not limited to these examples. For example, if the user is using a smartphone, the providing unit that provides a vibration pattern to a user can utilize the device's vibration function. Furthermore, if the user is using a tablet, the providing unit that provides a vibration pattern to a user can also optimize the device's vibration function. Furthermore, if the user is using a smartwatch, the providing unit that provides a vibration pattern to a user can also utilize the device's vibration function. This allows for a more appropriate vibration experience by taking into account the user's device information. Some or all of the above-described processing by the providing unit that provides a vibration pattern to a user may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit that provides a vibration pattern to a user can input device information data to a generation AI and cause the generation AI to provide the optimal vibration.

[0057] The providing unit that provides a vibration pattern to a user can improve the providing method by reflecting user feedback. User feedback includes, but is not limited to, user ratings and comments. The providing unit that provides a vibration pattern to a user can, for example, improve the vibration providing method based on user feedback. The providing unit that provides a vibration pattern to a user can also provide an optimal vibration pattern based on user feedback. The providing unit that provides a vibration pattern to a user can also learn user feedback and optimize the providing algorithm. For example, the providing unit that provides a vibration pattern to a user improves the vibration providing method based on user feedback. This makes it easier to improve the providing method by reflecting user feedback. Some or all of the above-described processing in the providing unit that provides a vibration pattern to a user may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit that provides a vibration pattern to a user can input feedback data to a generation AI and cause the generation AI to improve the providing method.

[0058] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0059] The video analysis unit can identify important scenes by using the user's gaze tracking data. The gaze tracking data includes, but is not limited to, for example, the position of the gaze and the movement of the gaze. For example, the video analysis unit can identify a scene in which the user gazes for a long time as an important scene. The video analysis unit can also identify a scene in which the user frequently moves their gaze as an important scene. The video analysis unit can also identify a scene in which the user focuses their gaze as an important scene. This makes it easier to identify important scenes by using the user's gaze tracking data. Some or all of the above-described processing in the video analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the video analysis unit can input the gaze tracking data to a generation AI and cause the generation AI to identify important scenes.

[0060] The voice analysis unit can customize the analysis method by taking into account the user's hearing characteristics. The user's hearing characteristics include, but are not limited to, hearing test results, sound preferences, etc. The voice analysis unit can, for example, adjust the voice analysis algorithm based on the user's hearing characteristics. The voice analysis unit can also learn the user's hearing characteristics and propose an optimal voice analysis method. The voice analysis unit can also apply an analysis method that emphasizes specific frequency bands depending on the user's hearing characteristics. This allows for more appropriate voice analysis by taking the user's hearing characteristics into consideration. Some or all of the above-described processing in the voice analysis unit can be performed using, for example, AI, or without AI. For example, the voice analysis unit can input the user's hearing characteristic data into a generation AI and have the generation AI customize the analysis method.

[0061] The conversion unit can change the rhythm of the vibrations according to the tension of a movie scene. Examples of tension in movie scenes include, but are not limited to, action scenes, romance scenes, and horror scenes. For example, the conversion unit generates a vibration pattern with a fast rhythm for action scenes. The conversion unit can also generate a vibration pattern with a slow rhythm for romance scenes. The conversion unit can also generate a vibration pattern with an irregular rhythm for horror scenes. This allows the vibration rhythm to be changed according to the tension of a movie scene, providing a more realistic experience. Some or all of the above-described processing in the conversion unit may be performed using, or without, AI. For example, the conversion unit can input tension data for a scene into a generation AI and cause the generation AI to change the rhythm.

[0062] The providing unit can adjust the vibration intensity according to the user's seat position. The user's seat position includes, but is not limited to, for example, the seat arrangement and user position information. For example, the providing unit increases the vibration intensity when the user is in a front seat. The providing unit can also decrease the vibration intensity when the user is in a rear seat. The providing unit can also dynamically adjust the vibration intensity according to the user's seat position. This makes it possible to provide a more appropriate vibration experience by adjusting the vibration intensity according to the user's seat position. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input seat position data to the generating AI and cause the generating AI to adjust the intensity.

[0063] The video analysis unit can improve the accuracy of the analysis by referring to past analysis results. Past analysis results include, but are not limited to, analysis results of the same movie scene and analysis results of similar scenes. For example, the video analysis unit can improve the analysis accuracy by referring to previously analyzed scenes of the same movie. The video analysis unit can also improve the analysis accuracy of similar scenes based on data obtained from past analysis results. The video analysis unit can also learn from past analysis results and optimize the analysis algorithm. For example, the video analysis unit can improve the analysis accuracy by referring to previously analyzed scenes of the same movie. In this way, the analysis accuracy can be improved by referring to past analysis results. Some or all of the above-described processing in the video analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the video analysis unit can input past analysis result data into the generation AI and cause the generation AI to improve the analysis accuracy.

[0064] The audio analysis unit can recognize specific sound effects or music and generate a vibration pattern corresponding to the specific sound effects or music. Specific sound effects or music include, but are not limited to, explosions and specific music. The audio analysis unit can recognize specific sound effects or music using, for example, voice recognition technology. The audio analysis unit can also analyze characteristics of sound effects or music using frequency analysis technology. The audio analysis unit can also extract characteristics of sound effects or music using AI. For example, the audio analysis unit generates a vibration pattern when an explosion occurs. The audio analysis unit can also generate a vibration pattern corresponding to specific music in a scene where the music is playing. The audio analysis unit can also generate a vibration pattern corresponding to a character's scream or laughter. In this way, generating a vibration pattern corresponding to specific sound effects or music can provide a more realistic experience. Some or all of the above-described processing in the audio analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the audio analysis unit can input specific sound effects or music data into the generation AI and have the generation AI generate vibration patterns.

[0065] The conversion unit can improve the accuracy of generation by referring to past vibration pattern data. Past vibration pattern data includes, for example, vibration patterns in the same scene, vibration patterns in similar scenes, etc., but is not limited to these examples. The conversion unit, for example, improves accuracy by referring to vibration patterns generated in the past. The conversion unit can also optimize vibration patterns in similar scenes based on past vibration pattern data. The conversion unit can also learn past vibration pattern data and optimize the generation algorithm. For example, the conversion unit improves accuracy by referring to vibration patterns generated in the past. In this way, the accuracy of generation can be improved by referring to past vibration pattern data. Some or all of the above-mentioned processing in the conversion unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit can input past vibration pattern data to the generation AI and cause the generation AI to improve the generation accuracy.

[0066] The providing unit can optimize the delivery method by referring to the user's past vibration experiences. Past vibration experiences include, but are not limited to, vibration intensity, rhythm, and duration. The providing unit, for example, suggests an optimal delivery method based on the user's past vibration experiences. The providing unit can also provide a preferred vibration pattern based on the user's past vibration experiences. The providing unit can also learn the user's past vibration experiences and optimize the delivery algorithm. For example, the providing unit suggests an optimal delivery method based on the user's past vibration experiences. This makes it easier to optimize the delivery method by referring to the user's past vibration experiences. Some or all of the above-described processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input past vibration experience data into a generation AI and cause the generation AI to optimize the delivery method.

[0067] The video analysis unit can customize the analysis method by reflecting the user's past viewing history. The past viewing history includes, for example, a list of movies viewed and viewing time, but is not limited to these examples. For example, the video analysis unit prioritizes analysis of favorite scenes based on the user's past viewing history. The video analysis unit can also analyze scenes that are likely to be of interest based on the user's past viewing history. The video analysis unit can also learn the user's past viewing history and optimize the analysis algorithm. For example, the video analysis unit prioritizes analysis of favorite scenes based on the user's past viewing history. This makes it easier to customize the analysis method by reflecting the user's past viewing history. Some or all of the above-described processing in the video analysis unit may be performed using, or without, AI. For example, the video analysis unit can input past viewing history data into a generation AI and have the generation AI customize the analysis method.

[0068] The audio analysis unit can customize the analysis method by reflecting the user's past listening history. The past listening history includes, for example, a list of music listened to and listening time, but is not limited to these examples. For example, the audio analysis unit prioritizes analysis of preferred audio based on the user's past listening history. The audio analysis unit can also analyze audio that is likely to be of interest to the user from the user's past listening history. The audio analysis unit can also learn the user's past listening history and optimize the analysis algorithm. For example, the audio analysis unit prioritizes analysis of preferred audio based on the user's past listening history. This makes it easier to customize the analysis method by reflecting the user's past listening history. Some or all of the above-described processing in the audio analysis unit may be performed using, or without, AI. For example, the audio analysis unit can input past listening history data into a generation AI and have the generation AI customize the analysis method.

[0069] The conversion unit can combine multiple vibration patterns according to specific scenes and sound effects. Specific scenes and sound effects include, but are not limited to, explosion sounds in action scenes and music in romance scenes. For example, the conversion unit can combine different vibration patterns according to explosion sounds and gunfire in action scenes. The conversion unit can also combine multiple vibration patterns according to music and character movements in romance scenes. The conversion unit can also combine multiple vibration patterns according to sudden sound effects and visual effects in horror scenes. By combining multiple vibration patterns according to specific scenes and sound effects, a more realistic experience can be provided. Some or all of the above-described processing in the conversion unit can be performed using, for example, AI, or without AI. For example, the conversion unit can input data on scenes and sound effects into a generation AI and have the generation AI execute a combination of multiple vibration patterns.

[0070] The providing unit can provide different vibration patterns depending on the type of movie scene. Types of movie scenes include, but are not limited to, action scenes and emotional scenes. For example, the providing unit can provide a strong vibration pattern for action scenes. The providing unit can also provide a gentle vibration pattern for romance scenes. The providing unit can also provide an irregular vibration pattern for horror scenes. By providing different vibration patterns depending on the type of movie scene, a more realistic experience can be provided. Some or all of the above-described processing in the providing unit can be performed using, for example, AI, or can be performed without using AI. For example, the providing unit can input scene type data to the generating AI and cause the generating AI to provide different vibration patterns.

[0071] The processing flow of the first embodiment will be briefly explained below.

[0072] Step 1: The video analysis unit analyzes movie scenes. For example, the video analysis unit can use image recognition technology to recognize specific scenes. It can also use frame analysis technology to analyze the content of scenes. It can also use AI to extract scene features. Step 2: The audio analysis section analyzes the audio of the movie. For example, it uses voice recognition technology to recognize specific sounds. It can also analyze the characteristics of the audio using frequency analysis technology. Furthermore, it can also extract the audio characteristics using AI. Step 3: The conversion unit generates vibration patterns based on the results of the video and audio analysis. For example, a chair or floor can vibrate according to the scene or audio of a movie. The conversion unit can also generate vibration patterns using AI. Step 4: The providing unit provides the generated vibration pattern to the user. For example, the generated vibration pattern is transmitted to a chair or the floor. The providing unit can also provide the vibration pattern using AI.

[0073] (Example 2) An immersive home theater system according to an embodiment of the present invention generates vibration patterns based on analysis results of video and audio and provides them to a user. The immersive home theater system includes a video analysis unit, an audio analysis unit, a conversion unit that generates vibration patterns based on the analysis results, and a provision unit that provides the generated vibration patterns to a user. For example, the immersive home theater system analyzes movie scenes using video authentication technology and generates vibration patterns corresponding to specific scenes. The immersive home theater system also analyzes movie audio using audio authentication technology and generates vibration patterns corresponding to specific sounds. Furthermore, the immersive home theater system vibrates chairs or floors according to movie scenes and audio. This allows the immersive home theater system to make movie viewing more realistic and immersive. This allows the immersive home theater system to enjoy an attraction-like movie experience even at home. For example, by providing immersive sensations according to movie scenes, a new movie experience can be provided that fully utilizes the three senses of sight, hearing, and touch.

[0074] An immersive home theater system according to an embodiment includes a video analysis unit, an audio analysis unit, a conversion unit, and a providing unit. The video analysis unit analyzes movie scenes. For example, the video analysis unit may recognize specific scenes using image recognition technology. The video analysis unit may also analyze the content of a scene using frame analysis technology. The video analysis unit may also extract scene features using AI. For example, the video analysis unit may recognize specific scenes and generate a vibration pattern corresponding to the scene. The audio analysis unit analyzes movie audio. For example, the audio analysis unit may recognize specific sounds using audio recognition technology. The audio analysis unit may also analyze audio features using frequency analysis technology. The audio analysis unit may also extract audio features using AI. For example, the audio analysis unit may recognize specific sounds and generate a vibration pattern corresponding to the sounds. The conversion unit generates a vibration pattern based on the analysis results of the video and audio. For example, the conversion unit may cause a chair or floor to vibrate in accordance with the movie scenes and audio. The conversion unit can also generate a vibration pattern using AI. For example, the conversion unit causes a chair or floor to vibrate according to a scene in a movie. The provision unit provides the generated vibration pattern to a user. For example, the provision unit transmits the generated vibration pattern to a chair or floor. The provision unit can also provide a vibration pattern using AI. For example, the provision unit provides the generated vibration pattern to a user. In this way, the immersive home theater system according to the embodiment can generate a vibration pattern based on the analysis results of video and audio, and provide it to a user.

[0075] The authentication unit can recognize a specific scene and generate a vibration pattern corresponding to the scene. Specific scenes include, but are not limited to, action scenes and moving scenes. The authentication unit can recognize a specific scene using, for example, image recognition technology. The authentication unit can also analyze the content of a scene using frame analysis technology. Furthermore, the authentication unit can extract scene features using AI. For example, the authentication unit can recognize a specific scene and generate a vibration pattern corresponding to the scene. This allows for the generation of a vibration pattern corresponding to the specific scene, providing a more realistic experience. Some or all of the above-described processing in the authentication unit may be performed using AI, for example, or may be performed without using AI. For example, to recognize a specific scene, the authentication unit can input image data to a generation AI and have the generation AI recognize the scene.

[0076] The authentication unit can recognize a specific sound and generate a vibration pattern corresponding to that sound. Specific sounds include, but are not limited to, explosions, music, etc. The authentication unit can recognize the specific sound using, for example, voice recognition technology. The authentication unit can also analyze the characteristics of the sound using frequency analysis technology. Furthermore, the authentication unit can extract the characteristics of the sound using AI. For example, the authentication unit can recognize a specific sound and generate a vibration pattern corresponding to that sound. By generating a vibration pattern corresponding to the specific sound, a more realistic experience can be provided. Some or all of the above-mentioned processing in the authentication unit can be performed using, for example, AI, or can be performed without using AI. For example, to recognize a specific sound, the authentication unit can input voice data to a generation AI and have the generation AI recognize the sound.

[0077] The conversion unit can make a chair or floor vibrate according to a scene in a movie. Movie scenes include, but are not limited to, action scenes and moving scenes. For example, the conversion unit can make a chair or floor vibrate according to a scene in a movie. The conversion unit can also generate a vibration pattern using AI. For example, the conversion unit can make a chair or floor vibrate according to a scene in a movie. This allows the chair or floor to vibrate according to the scene and sound of the movie, providing a more realistic experience. Some or all of the above-described processing in the conversion unit may be performed using AI, for example, or may be performed without using AI. For example, the conversion unit can input scene data to a generation AI to generate a vibration pattern according to a scene in a movie, and cause the generation AI to generate a vibration pattern.

[0078] The providing unit can provide the generated vibration pattern to the user. The generated vibration pattern includes, for example, vibration intensity, rhythm, duration, etc., but is not limited to these examples. The providing unit, for example, transmits the generated vibration pattern to a chair or floor. The providing unit can also control a device for providing the generated vibration pattern to the user. Furthermore, the providing unit can provide the vibration pattern using AI. For example, the providing unit provides the generated vibration pattern to the user. By providing the generated vibration pattern to the user, a realistic sensation can be provided. Some or all of the above-described processing in the providing unit may be performed, for example, using AI or may be performed without using AI. For example, in order to provide the generated vibration pattern, the providing unit can input vibration data to a generating AI and cause the generating AI to provide vibrations.

[0079] The authentication unit that analyzes the video can estimate the user's emotions and adjust the accuracy of the video analysis based on the estimated user emotions. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The authentication unit that analyzes the video can estimate the user's emotions using, for example, facial expression recognition technology. The authentication unit that analyzes the video can also estimate the user's emotions using voice analysis technology. The authentication unit that analyzes the video can also estimate the user's emotions using AI. For example, if the user is excited, the authentication unit that analyzes the video can increase the accuracy of the video analysis and perform more detailed scene recognition. If the user is relaxed, the authentication unit that analyzes the video can moderately adjust the accuracy of the video analysis and eliminate excessive information. If the user is tense, the authentication unit that analyzes the video can maintain a medium level of accuracy in the video analysis to reduce visual strain. This allows for more appropriate video analysis by adjusting the accuracy of the video analysis according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or generative AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the authentication unit that analyzes the video may be performed using AI, or may be performed without using AI. For example, the authentication unit that analyzes the video may input user emotion data into the generation AI and cause the generation AI to adjust the accuracy of the video analysis based on emotion.

[0080] The authentication unit that analyzes the video can apply different analysis algorithms depending on the genre of the movie. Movie genres include, but are not limited to, action, drama, and comedy. For example, in the case of an action movie, the authentication unit that analyzes the video can apply an analysis algorithm specialized for fast-moving scenes. In addition, in the case of a horror movie, the authentication unit that analyzes the video can apply an analysis algorithm that emphasizes dark scenes and sudden movements. In addition, in the case of a comedy movie, the authentication unit that analyzes the video can apply an analysis algorithm that emphasizes facial expressions and gestures. In this way, by applying an analysis algorithm depending on the genre of the movie, more appropriate video analysis can be performed. Some or all of the above-mentioned processing in the authentication unit that analyzes the video can be performed using, for example, AI, or without AI. For example, the authentication unit that analyzes the video can input movie genre data into the generation AI and cause the generation AI to apply an analysis algorithm depending on the genre.

[0081] The authentication unit that analyzes the video can improve the accuracy of the analysis by referring to past analysis results. Past analysis results include, but are not limited to, analysis results of the same movie scene and analysis results of similar scenes. The authentication unit that analyzes the video can improve the analysis accuracy by, for example, referring to previously analyzed scenes of the same movie. The authentication unit that analyzes the video can also improve the analysis accuracy of similar scenes based on data obtained from past analysis results. The authentication unit that analyzes the video can also learn from past analysis results and optimize the analysis algorithm. For example, the authentication unit that analyzes the video can improve the analysis accuracy by referring to previously analyzed scenes of the same movie. In this way, the analysis accuracy can be improved by referring to past analysis results. Some or all of the above-mentioned processing in the authentication unit that analyzes the video may be performed using, for example, AI, or may be performed without using AI. For example, the authentication unit that analyzes the video can input past analysis result data into the generation AI and cause the generation AI to improve the analysis accuracy.

[0082] The authentication unit that analyzes the video can recognize specific characters or objects and generate a corresponding vibration pattern. Specific characters or objects include, but are not limited to, main characters and explosions. The authentication unit that analyzes the video can recognize specific characters using, for example, facial recognition technology. The authentication unit that analyzes the video can also recognize specific objects using object detection technology. The authentication unit that analyzes the video can also extract characteristics of characters and objects using AI. For example, the authentication unit that analyzes the video can generate a vibration pattern when the main character fights in an action scene. The authentication unit that analyzes the video can also generate a vibration pattern when a specific object (e.g., an explosion) occurs. The authentication unit that analyzes the video can also generate a vibration pattern corresponding to the emotion of a character in an emotional scene. This allows for a more realistic experience by generating a vibration pattern corresponding to a specific character or object. Some or all of the above-described processing in the authentication unit that analyzes the video can be performed using, for example, AI, or without AI. For example, the authentication unit that analyzes the video can input data of a specific character or object into the generation AI, causing the generation AI to generate a vibration pattern.

[0083] The authentication unit that analyzes the video can estimate the user's emotions and determine the priority of video analysis based on the estimated user emotions. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The authentication unit that analyzes the video can estimate the user's emotions using, for example, facial expression recognition technology. The authentication unit that analyzes the video can also estimate the user's emotions using voice analysis technology. The authentication unit that analyzes the video can also estimate the user's emotions using AI. For example, if the user is excited, the authentication unit that analyzes the video can prioritize analyzing action scenes. If the user is relaxed, the authentication unit that analyzes the video can also prioritize analyzing landscape scenes. If the user is tense, the authentication unit that analyzes the video can also prioritize analyzing scenes that relieve tension. This allows for more appropriate video analysis by determining the priority of video analysis based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generative AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the authentication unit that analyzes the video may be performed using AI, or may be performed without using AI. For example, the authentication unit that analyzes the video may input user emotion data into the generation AI and cause the generation AI to prioritize video analysis based on emotion.

[0084] The authentication unit that analyzes the video can identify important scenes using the user's gaze tracking data. The gaze tracking data includes, but is not limited to, gaze position and gaze movement, for example. The authentication unit that analyzes the video can, for example, identify a scene where the user gazes for a long time as an important scene. The authentication unit that analyzes the video can also identify a scene where the user frequently moves their gaze as an important scene. The authentication unit that analyzes the video can also identify a scene where the user focuses their gaze as an important scene. This makes it easier to identify important scenes by using the user's gaze tracking data. Some or all of the above-described processing in the authentication unit that analyzes the video can be performed using, for example, AI, or without AI. For example, the authentication unit that analyzes the video can input the gaze tracking data to a generation AI and cause the generation AI to identify important scenes.

[0085] The authentication unit that analyzes the video can analyze the subtitle information of the movie and complete the scene content. Subtitle information includes, but is not limited to, for example, character recognition and context analysis. The authentication unit that analyzes the video can, for example, analyze the subtitle information of the movie and complete the scene content. The authentication unit that analyzes the video can also analyze the lines of the characters from the subtitle information and complete the scene content. The authentication unit that analyzes the video can also complete the background information of the scene based on the subtitle information. This makes it easier to complete the scene content by analyzing the subtitle information of the movie. Some or all of the above-mentioned processing in the authentication unit that analyzes the video can be performed using, for example, AI, or can be performed without using AI. For example, the authentication unit that analyzes the video can input the subtitle information to a generation AI and have the generation AI complete the scene content.

[0086] The authentication unit that analyzes the video can customize the analysis method by reflecting the user's past viewing history. The past viewing history may include, but is not limited to, a list of movies viewed and viewing time. For example, the authentication unit that analyzes the video may prioritize analyzing favorite scenes based on the user's past viewing history. The authentication unit that analyzes the video may also analyze scenes that are likely to be of interest based on the user's past viewing history. The authentication unit that analyzes the video may also learn the user's past viewing history and optimize the analysis algorithm. For example, the authentication unit that analyzes the video may prioritize analyzing favorite scenes based on the user's past viewing history. This makes it easier to customize the analysis method by reflecting the user's past viewing history. Some or all of the above-described processing in the authentication unit that analyzes the video may be performed using, for example, AI, or may be performed without AI. For example, the authentication unit that analyzes the video may input past viewing history data into a generation AI and have the generation AI customize the analysis method.

[0087] The authentication unit that analyzes the voice can estimate the user's emotions and adjust the accuracy of the voice analysis based on the estimated user emotions. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The authentication unit that analyzes the voice can estimate the user's emotions using, for example, facial expression recognition technology. The authentication unit that analyzes the voice can also estimate the user's emotions using voice analysis technology. The authentication unit that analyzes the voice can also estimate the user's emotions using AI. For example, if the user is excited, the authentication unit that analyzes the voice can increase the accuracy of the voice analysis to perform more detailed voice recognition. If the user is relaxed, the authentication unit that analyzes the voice can also moderately adjust the accuracy of the voice analysis to eliminate excessive information. If the user is nervous, the authentication unit that analyzes the voice can maintain a medium level of accuracy to reduce auditory strain. This allows for more appropriate voice analysis by adjusting the accuracy of the voice analysis according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or generative AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the authentication unit that analyzes the voice may be performed using AI, or may be performed without using AI. For example, the authentication unit that analyzes the voice may input user emotion data into the generation AI and cause the generation AI to adjust the accuracy of the voice analysis based on emotion.

[0088] The authentication unit that analyzes the audio can apply different analysis algorithms depending on the genre of the movie. Movie genres include, but are not limited to, action, drama, and comedy. For example, in the case of an action movie, the authentication unit that analyzes the audio can apply an analysis algorithm specialized for explosions and gunshots. In addition, in the case of a horror movie, the authentication unit that analyzes the audio can apply an analysis algorithm specialized for frightening sound effects. In addition, in the case of a comedy movie, the authentication unit that analyzes the audio can apply an analysis algorithm specialized for laughter and humorous sounds. In this way, by applying an analysis algorithm according to the genre of the movie, more appropriate audio analysis can be performed. Some or all of the above-mentioned processing in the authentication unit that analyzes the audio may be performed using, for example, AI, or may be performed without using AI. For example, the authentication unit that analyzes the audio can input movie genre data into the generation AI and cause the generation AI to apply an analysis algorithm according to the genre.

[0089] The authentication unit that analyzes the audio can improve the accuracy of the analysis by referring to past analysis results. Past analysis results include, but are not limited to, analysis results of audio from the same movie and analysis results of similar audio. The authentication unit that analyzes the audio can improve the analysis accuracy by, for example, referring to audio from the same movie that was analyzed in the past. The authentication unit that analyzes the audio can also improve the analysis accuracy of similar audio based on data obtained from past analysis results. The authentication unit that analyzes the audio can also learn from past analysis results and optimize the analysis algorithm. For example, the authentication unit that analyzes the audio can improve the analysis accuracy by referring to audio from the same movie that was analyzed in the past. In this way, the analysis accuracy can be improved by referring to past analysis results. Some or all of the above-mentioned processing in the authentication unit that analyzes the audio may be performed using, for example, AI, or may be performed without using AI. For example, the authentication unit that analyzes the audio can input past analysis result data into the generation AI and cause the generation AI to improve the analysis accuracy.

[0090] The authentication unit that analyzes the audio can recognize specific sound effects or music and generate a corresponding vibration pattern. Examples of specific sound effects or music include, but are not limited to, explosions and specific music. The authentication unit that analyzes the audio can recognize specific sound effects or music using, for example, voice recognition technology. The authentication unit that analyzes the audio can also analyze the characteristics of sound effects or music using frequency analysis technology. The authentication unit that analyzes the audio can also extract the characteristics of sound effects or music using AI. For example, the authentication unit that analyzes the audio can generate a vibration pattern when an explosion occurs. The authentication unit that analyzes the audio can also generate a vibration pattern corresponding to specific music in a scene where the music is playing. The authentication unit that analyzes the audio can also generate a vibration pattern corresponding to a character's scream or laughter. This allows for a more realistic experience by generating a vibration pattern corresponding to specific sound effects or music. Some or all of the above-described processing in the authentication unit that analyzes the audio may be performed using, for example, AI, or may be performed without using AI. For example, an authentication unit that analyzes voice can input specific sound effects or music data into a generation AI, causing the generation AI to generate vibration patterns.

[0091] The authentication unit that analyzes the voice can estimate the user's emotions and determine the priority of the voice analysis based on the estimated user emotions. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The authentication unit that analyzes the voice can estimate the user's emotions using, for example, facial expression recognition technology. The authentication unit that analyzes the voice can also estimate the user's emotions using voice analysis technology. The authentication unit that analyzes the voice can also estimate the user's emotions using AI. For example, if the user is excited, the authentication unit that analyzes the voice can prioritize voice analysis of action scenes. If the user is relaxed, the authentication unit that analyzes the voice can also prioritize voice analysis of landscape scenes. If the user is nervous, the authentication unit that analyzes the voice can also prioritize voice analysis that relieves tension. This allows for more appropriate voice analysis by determining the priority of voice analysis according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or generative AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the authentication unit that analyzes the voice may be performed using, or without, AI. For example, the authentication unit that analyzes the voice may input user emotion data into the generation AI and cause the generation AI to prioritize the voice analysis based on emotion.

[0092] The authentication unit that analyzes the voice can customize the analysis method by taking into account the user's hearing characteristics. The user's hearing characteristics include, but are not limited to, hearing test results and sound preferences. The authentication unit that analyzes the voice can, for example, adjust the voice analysis algorithm based on the user's hearing characteristics. The authentication unit that analyzes the voice can also learn the user's hearing characteristics and propose an optimal voice analysis method. The authentication unit that analyzes the voice can also apply an analysis method that emphasizes specific frequency bands depending on the user's hearing characteristics. This allows for more appropriate voice analysis by taking the user's hearing characteristics into account. Some or all of the above-described processing in the authentication unit that analyzes the voice can be performed using, for example, AI, or without AI. For example, the authentication unit that analyzes the voice can input the user's hearing characteristic data into a generation AI and have the generation AI customize the analysis method.

[0093] The authentication unit that analyzes audio can analyze dialogue information in a movie and complete the content of a scene. Dialogue information includes, but is not limited to, character recognition and context analysis. The authentication unit that analyzes audio can, for example, analyze dialogue information in a movie and complete the content of a scene. The authentication unit that analyzes audio can also analyze character emotions from the dialogue information and complete the content of a scene. The authentication unit that analyzes audio can also complete background information of a scene based on the dialogue information. This makes it easier to complete the content of a scene by analyzing the dialogue information in a movie. Some or all of the above-mentioned processing in the authentication unit that analyzes audio can be performed using, for example, AI, or can be performed without using AI. For example, the authentication unit that analyzes audio can input dialogue information to a generation AI and have the generation AI complete the content of the scene.

[0094] The authentication unit that analyzes audio can customize the analysis method by reflecting the user's past listening history. The past listening history includes, for example, a list of music listened to and listening time, but is not limited to these examples. For example, the authentication unit that analyzes audio prioritizes analyzing preferred audio based on the user's past listening history. The authentication unit that analyzes audio can also analyze audio that is likely to be of interest to the user based on the user's past listening history. The authentication unit that analyzes audio can also learn the user's past listening history and optimize the analysis algorithm. For example, the authentication unit that analyzes audio prioritizes analyzing preferred audio based on the user's past listening history. This makes it easier to customize the analysis method by reflecting the user's past listening history. Some or all of the above-described processing in the authentication unit that analyzes audio may be performed using, for example, AI, or may be performed without using AI. For example, the authentication unit that analyzes audio can input past listening history data into a generation AI and have the generation AI customize the analysis method.

[0095] The conversion unit that generates the vibration pattern can estimate the user's emotion and adjust the intensity of the vibration pattern based on the estimated user's emotion. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The conversion unit that generates the vibration pattern can estimate the user's emotion using, for example, facial expression recognition technology. The conversion unit that generates the vibration pattern can also estimate the user's emotion using voice analysis technology. The conversion unit that generates the vibration pattern can also estimate the user's emotion using AI. For example, the conversion unit that generates the vibration pattern can increase the intensity of the vibration pattern when the user is excited. The conversion unit that generates the vibration pattern can also decrease the intensity of the vibration pattern when the user is relaxed. The conversion unit that generates the vibration pattern can also maintain the intensity of the vibration pattern at a medium level when the user is tense. This allows for a more appropriate vibration experience by adjusting the intensity of the vibration pattern according to the user's emotion. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. A part or all of the above-described processing in the conversion unit that generates the vibration pattern may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit that generates the vibration pattern may input user emotion data to the generation AI and cause the generation AI to adjust the intensity of the vibration pattern based on the emotion.

[0096] The conversion unit that generates the vibration pattern can change the rhythm of the vibration according to the tension of the movie scene. Examples of tension in movie scenes include, but are not limited to, action scenes, romance scenes, and horror scenes. For example, the conversion unit that generates the vibration pattern can generate a vibration pattern with a fast rhythm for action scenes. The conversion unit that generates the vibration pattern can also generate a vibration pattern with a slow rhythm for romance scenes. The conversion unit that generates the vibration pattern can also generate a vibration pattern with an irregular rhythm for horror scenes. This allows the vibration rhythm to be changed according to the tension of the movie scene, providing a more realistic experience. Some or all of the above-described processing in the conversion unit that generates the vibration pattern may be performed using, or without, AI. For example, the conversion unit that generates the vibration pattern can input tension data of the scene into the generation AI and cause the generation AI to change the rhythm.

[0097] The conversion unit that generates a vibration pattern can improve the accuracy of generation by referring to data on past vibration patterns. Past vibration pattern data includes, for example, vibration patterns in the same scene, vibration patterns in similar scenes, etc., but is not limited to these examples. The conversion unit that generates a vibration pattern can improve the accuracy by referring to, for example, previously generated vibration patterns. The conversion unit that generates a vibration pattern can also optimize vibration patterns in similar scenes based on past vibration pattern data. The conversion unit that generates a vibration pattern can also learn past vibration pattern data and optimize the generation algorithm. For example, the conversion unit that generates a vibration pattern can improve the accuracy by referring to previously generated vibration patterns. In this way, the accuracy of generation can be improved by referring to past vibration pattern data. Some or all of the above-described processing in the conversion unit that generates a vibration pattern may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit that generates a vibration pattern can input past vibration pattern data to the generation AI and cause the generation AI to improve the generation accuracy.

[0098] The conversion unit that generates the vibration pattern can combine multiple vibration patterns according to specific scenes and sound effects. Specific scenes and sound effects include, but are not limited to, explosion sounds in action scenes and music in romance scenes. The conversion unit that generates the vibration pattern can combine different vibration patterns according to, for example, explosion sounds and gunfire in action scenes. The conversion unit that generates the vibration pattern can also combine multiple vibration patterns according to music and character movements in romance scenes. The conversion unit that generates the vibration pattern can also combine multiple vibration patterns according to sudden sound effects and visual effects in horror scenes. This allows for a more realistic experience by combining multiple vibration patterns according to specific scenes and sound effects. Some or all of the above-described processing in the conversion unit that generates the vibration pattern can be performed using, for example, AI, or without AI. For example, the conversion unit that generates the vibration pattern can input data on scenes and sound effects into a generation AI and have the generation AI execute a combination of multiple vibration patterns.

[0099] The conversion unit that generates the vibration pattern can estimate the user's emotion and select a type of vibration pattern based on the estimated user's emotion. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The conversion unit that generates the vibration pattern can estimate the user's emotion using, for example, facial expression recognition technology. The conversion unit that generates the vibration pattern can also estimate the user's emotion using voice analysis technology. The conversion unit that generates the vibration pattern can also estimate the user's emotion using AI. For example, the conversion unit that generates the vibration pattern can select a strong vibration pattern if the user is excited. The conversion unit that generates the vibration pattern can also select a gentle vibration pattern if the user is relaxed. The conversion unit that generates the vibration pattern can also select a medium vibration pattern if the user is nervous. This allows for a more appropriate vibration experience by selecting a type of vibration pattern according to the user's emotion. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. A part or all of the above-described processing in the conversion unit that generates the vibration pattern may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit that generates the vibration pattern may input user emotion data to the generation AI and cause the generation AI to select a vibration pattern based on the emotion.

[0100] The conversion unit that generates the vibration pattern can generate an appropriate vibration pattern by taking into account the user's physical characteristics. Examples of the user's physical characteristics include, but are not limited to, weight, height, and sensory sensitivity. The conversion unit that generates the vibration pattern can generate an optimal vibration pattern based on, for example, the user's weight and height. The conversion unit that generates the vibration pattern can also learn the user's physical characteristics and suggest an optimal vibration pattern. The conversion unit that generates the vibration pattern can also emphasize a specific vibration pattern depending on the user's physical characteristics. This allows for a more appropriate vibration experience by taking the user's physical characteristics into consideration. Some or all of the above-described processing in the conversion unit that generates the vibration pattern can be performed using, for example, AI, or without AI. For example, the conversion unit that generates the vibration pattern can input the user's physical characteristic data into a generation AI and cause the generation AI to generate an optimal vibration pattern.

[0101] The conversion unit that generates the vibration pattern can adjust the duration of the vibration according to the length of a scene in a movie. The length of a scene in a movie includes, but is not limited to, the timing of the start and end of the scene and the duration of the scene. For example, the conversion unit that generates the vibration pattern can generate a continuous vibration pattern for a long scene. The conversion unit that generates the vibration pattern can also generate an instantaneous vibration pattern for a short scene. The conversion unit that generates the vibration pattern can also dynamically adjust the duration of the vibration according to the length of the scene. This allows for adjusting the duration of the vibration according to the length of the scene in a movie, thereby providing a more appropriate vibration experience. Some or all of the above-described processing in the conversion unit that generates the vibration pattern may be performed using, or without, AI. For example, the conversion unit that generates the vibration pattern can input scene length data to the generation AI and have the generation AI adjust the duration.

[0102] The conversion unit that generates the vibration pattern can customize the generation method by reflecting the user's past feedback. Past feedback includes, but is not limited to, user ratings and comments. The conversion unit that generates the vibration pattern can generate a preferred vibration pattern based on, for example, the user's past feedback. The conversion unit that generates the vibration pattern can also suggest an optimal vibration pattern based on the user's past feedback. The conversion unit that generates the vibration pattern can also learn the user's past feedback and optimize the generation algorithm. For example, the conversion unit that generates the vibration pattern can generate a preferred vibration pattern based on the user's past feedback. This makes it easier to customize the generation method by reflecting the user's past feedback. Some or all of the above-described processing in the conversion unit that generates the vibration pattern can be performed using, for example, AI, or can be performed without using AI. For example, the conversion unit that generates the vibration pattern can input past feedback data into a generation AI and have the generation AI customize the generation method.

[0103] The unit that provides a vibration pattern to a user can estimate the user's emotion and adjust the timing of vibration provision based on the estimated user's emotion. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The unit that provides a vibration pattern to a user can estimate the user's emotion using, for example, facial expression recognition technology. The unit that provides a vibration pattern to a user can also estimate the user's emotion using voice analysis technology. The unit that provides a vibration pattern to a user can also estimate the user's emotion using AI. For example, the unit that provides a vibration pattern to a user can advance the timing of vibration provision when the user is excited. The unit that provides a vibration pattern to a user can also delay the timing of vibration provision when the user is relaxed. The unit that provides a vibration pattern to a user can also maintain the timing of vibration provision at a medium level when the user is tense. This allows for a more appropriate vibration experience by adjusting the timing of vibration provision according to the user's emotion. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-described processing in the providing unit that provides the vibration pattern to the user may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit that provides the vibration pattern to the user may input the user's emotion data to the generation AI and cause the generation AI to adjust the timing of providing vibration based on the emotion.

[0104] The providing unit that provides the vibration pattern to the user can adjust the vibration intensity according to the user's seat position. The user's seat position includes, but is not limited to, for example, the seat arrangement and user position information. The providing unit that provides the vibration pattern to the user can, for example, increase the vibration intensity when the user is in a front seat. The providing unit that provides the vibration pattern to the user can also decrease the vibration intensity when the user is in a rear seat. The providing unit that provides the vibration pattern to the user can also dynamically adjust the vibration intensity according to the user's seat position. This makes it possible to provide a more appropriate vibration experience by adjusting the vibration intensity according to the user's seat position. Some or all of the above-described processing in the providing unit that provides the vibration pattern to the user may be performed using, for example, AI or without AI. For example, the providing unit that provides the vibration pattern to the user can input seat position data to a generation AI and cause the generation AI to adjust the intensity.

[0105] The providing unit that provides a vibration pattern to a user can optimize the providing method by referring to the user's past vibration experiences. Past vibration experiences include, but are not limited to, vibration intensity, rhythm, and duration. The providing unit that provides a vibration pattern to a user can, for example, suggest an optimal providing method based on the user's past vibration experiences. The providing unit that provides a vibration pattern to a user can also provide a preferred vibration pattern based on the user's past vibration experiences. The providing unit that provides a vibration pattern to a user can also learn the user's past vibration experiences and optimize the providing algorithm. For example, the providing unit that provides a vibration pattern to a user can suggest an optimal providing method based on the user's past vibration experiences. This makes it easier to optimize the providing method by referring to the user's past vibration experiences. Some or all of the above-described processing in the providing unit that provides a vibration pattern to a user may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit that provides a vibration pattern to a user can input past vibration experience data into a generation AI and cause the generation AI to optimize the providing method.

[0106] The providing unit that provides the vibration pattern to the user can provide different vibration patterns depending on the type of movie scene. Types of movie scenes include, but are not limited to, action scenes and emotional scenes. For example, the providing unit that provides the vibration pattern to the user can provide a strong vibration pattern for action scenes. The providing unit that provides the vibration pattern to the user can also provide a gentle vibration pattern for romance scenes. The providing unit that provides the vibration pattern to the user can also provide an irregular vibration pattern for horror scenes. In this way, by providing different vibration patterns depending on the type of movie scene, a more realistic experience can be provided. Some or all of the above-described processing in the providing unit that provides the vibration pattern to the user may be performed using, or without using, AI, for example. For example, the providing unit that provides the vibration pattern to the user can input scene type data to a generation AI and cause the generation AI to provide different vibration patterns.

[0107] The unit that provides a vibration pattern to a user can estimate the user's emotion and customize the vibration provision method based on the estimated user emotion. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The unit that provides a vibration pattern to a user can estimate the user's emotion using, for example, facial expression recognition technology. The unit that provides a vibration pattern to a user can also estimate the user's emotion using voice analysis technology. The unit that provides a vibration pattern to a user can also estimate the user's emotion using AI. For example, the unit that provides a vibration pattern to a user can provide strong vibrations when the user is excited. The unit that provides a vibration pattern to a user can also provide gentle vibrations when the user is relaxed. The unit that provides a vibration pattern to a user can also provide medium vibrations when the user is tense. This allows the vibration provision method to be customized according to the user's emotion, providing a more appropriate vibration experience. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. A part or all of the above-described processing in the providing unit that provides the vibration pattern to the user may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit that provides the vibration pattern to the user may input the user's emotion data to the generating AI and cause the generating AI to customize the method of providing vibration based on the emotion.

[0108] The providing unit that provides a vibration pattern to a user can provide optimal vibrations by taking into account the user's physical characteristics. Examples of the user's physical characteristics include, but are not limited to, weight, height, and sensory sensitivity. The providing unit that provides a vibration pattern to a user can provide optimal vibrations based on, for example, the user's weight and height. The providing unit that provides a vibration pattern to a user can also learn the user's physical characteristics and suggest an optimal vibration method. The providing unit that provides a vibration pattern to a user can also emphasize a specific vibration pattern depending on the user's physical characteristics. This allows for a more appropriate vibration experience by taking the user's physical characteristics into consideration. Some or all of the above-described processing by the providing unit that provides a vibration pattern to a user may be performed using, or without, AI. For example, the providing unit that provides a vibration pattern to a user can input the user's physical characteristic data into a generation AI and cause the generation AI to provide optimal vibrations.

[0109] The providing unit that provides a vibration pattern to a user can provide an optimal vibration by taking into account the user's device information. The user's device information includes, for example, the type of device, the device's performance, etc., but is not limited to these examples. For example, if the user is using a smartphone, the providing unit that provides a vibration pattern to a user can utilize the device's vibration function. Furthermore, if the user is using a tablet, the providing unit that provides a vibration pattern to a user can also optimize the device's vibration function. Furthermore, if the user is using a smartwatch, the providing unit that provides a vibration pattern to a user can also utilize the device's vibration function. This allows for a more appropriate vibration experience by taking into account the user's device information. Some or all of the above-described processing by the providing unit that provides a vibration pattern to a user may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit that provides a vibration pattern to a user can input device information data to a generation AI and cause the generation AI to provide the optimal vibration.

[0110] The providing unit that provides a vibration pattern to a user can improve the providing method by reflecting user feedback. User feedback includes, but is not limited to, user ratings and comments. The providing unit that provides a vibration pattern to a user can, for example, improve the vibration providing method based on user feedback. The providing unit that provides a vibration pattern to a user can also provide an optimal vibration pattern based on user feedback. The providing unit that provides a vibration pattern to a user can also learn user feedback and optimize the providing algorithm. For example, the providing unit that provides a vibration pattern to a user improves the vibration providing method based on user feedback. This makes it easier to improve the providing method by reflecting user feedback. Some or all of the above-described processing in the providing unit that provides a vibration pattern to a user may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit that provides a vibration pattern to a user can input feedback data to a generation AI and cause the generation AI to improve the providing method. === Hard Collateral 1-1 === Each of the multiple elements including the video analysis unit, audio analysis unit, conversion unit, and provision unit described above is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the video analysis unit is realized by the camera 42 and processor 46 of the smart device 14 and analyzes movie scenes. The audio analysis unit is realized by the microphone 38B and processor 46 of the smart device 14 and analyzes the audio of the movie. The conversion unit is realized by the specific processing unit 290 of the data processing device 12 and generates a vibration pattern based on the analysis results of the video and audio. The provision unit is realized by the control unit 46A of the smart device 14 and provides the generated vibration pattern to the user. === Hard Collateral 1-2 === Each of the multiple elements including the above-mentioned video analyzing unit, audio analyzing unit, conversion unit, and providing unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the video analyzing unit is realized by the camera 42 and processor 46 of the smart glasses 214 and analyzes movie scenes. The audio analyzing unit is realized by the microphone 238 and processor 46 of the smart glasses 214 and analyzes movie audio. The conversion unit is realized by the specific processing unit 290 of the data processing device 12 and generates a vibration pattern based on the analysis results of the video and audio. The providing unit is realized by the control unit 46A of the smart glasses 214 and provides the generated vibration pattern to the user. === Hard Collateral 1-3 === Each of the multiple elements including the above-mentioned video analyzing unit, audio analyzing unit, conversion unit, and providing unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the video analyzing unit is realized by the camera 42 and processor 46 of the headset type terminal 314 and analyzes movie scenes. The audio analyzing unit is realized by the microphone 238 and processor 46 of the headset type terminal 314 and analyzes the audio of the movie. The conversion unit is realized by the specific processing unit 290 of the data processing device 12 and generates a vibration pattern based on the analysis results of the video and audio. The providing unit is realized by the control unit 46A of the headset type terminal 314 and provides the generated vibration pattern to the user. === Hard Collateral 1-4 === Each of the multiple elements including the above-mentioned video analyzing unit, audio analyzing unit, conversion unit, and providing unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the video analyzing unit is realized by the camera 42 and processor 46 of the robot 414 and analyzes movie scenes. The audio analyzing unit is realized by the microphone 238 and processor 46 of the robot 414 and analyzes the audio of the movie. The conversion unit is realized by the specific processing unit 290 of the data processing device 12 and generates a vibration pattern based on the analysis results of the video and audio. The providing unit is realized by the control unit 46A of the robot 414 and provides the generated vibration pattern to the user.

[0111] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0112] The video analysis unit can identify important scenes by using the user's gaze tracking data. The gaze tracking data includes, but is not limited to, for example, the position of the gaze and the movement of the gaze. For example, the video analysis unit can identify a scene in which the user gazes for a long time as an important scene. The video analysis unit can also identify a scene in which the user frequently moves their gaze as an important scene. The video analysis unit can also identify a scene in which the user focuses their gaze as an important scene. This makes it easier to identify important scenes by using the user's gaze tracking data. Some or all of the above-described processing in the video analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the video analysis unit can input the gaze tracking data to a generation AI and cause the generation AI to identify important scenes.

[0113] The voice analysis unit can customize the analysis method by taking into account the user's hearing characteristics. The user's hearing characteristics include, but are not limited to, hearing test results, sound preferences, etc. The voice analysis unit can, for example, adjust the voice analysis algorithm based on the user's hearing characteristics. The voice analysis unit can also learn the user's hearing characteristics and propose an optimal voice analysis method. The voice analysis unit can also apply an analysis method that emphasizes specific frequency bands depending on the user's hearing characteristics. This allows for more appropriate voice analysis by taking the user's hearing characteristics into consideration. Some or all of the above-described processing in the voice analysis unit can be performed using, for example, AI, or without AI. For example, the voice analysis unit can input the user's hearing characteristic data into a generation AI and have the generation AI customize the analysis method.

[0114] The conversion unit can change the rhythm of the vibrations according to the tension of a movie scene. Examples of tension in movie scenes include, but are not limited to, action scenes, romance scenes, and horror scenes. For example, the conversion unit generates a vibration pattern with a fast rhythm for action scenes. The conversion unit can also generate a vibration pattern with a slow rhythm for romance scenes. The conversion unit can also generate a vibration pattern with an irregular rhythm for horror scenes. This allows the vibration rhythm to be changed according to the tension of a movie scene, providing a more realistic experience. Some or all of the above-described processing in the conversion unit may be performed using, or without, AI. For example, the conversion unit can input tension data for a scene into a generation AI and cause the generation AI to change the rhythm.

[0115] The providing unit can adjust the vibration intensity according to the user's seat position. The user's seat position includes, but is not limited to, for example, the seat arrangement and user position information. For example, the providing unit increases the vibration intensity when the user is in a front seat. The providing unit can also decrease the vibration intensity when the user is in a rear seat. The providing unit can also dynamically adjust the vibration intensity according to the user's seat position. This makes it possible to provide a more appropriate vibration experience by adjusting the vibration intensity according to the user's seat position. Some or all of the above-described processing by the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input seat position data to the generating AI and cause the generating AI to adjust the intensity.

[0116] The video analysis unit can estimate the user's emotions and adjust the accuracy of the video analysis based on the estimated user's emotions. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The video analysis unit can estimate the user's emotions using, for example, facial expression recognition technology. The video analysis unit can also estimate the user's emotions using voice analysis technology. The video analysis unit can also estimate the user's emotions using AI. For example, if the user is excited, the video analysis unit can increase the accuracy of the video analysis and perform more detailed scene recognition. If the user is relaxed, the video analysis unit can moderately adjust the accuracy of the video analysis and eliminate excessive information. If the user is tense, the video analysis unit can maintain a medium level of accuracy in the video analysis to reduce visual strain. This allows for more appropriate video analysis by adjusting the accuracy of the video analysis according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generative AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the video analyzing unit may be performed using AI, or may be performed without using AI. For example, the video analyzing unit may input user emotion data into the generation AI and cause the generation AI to adjust the accuracy of the video analysis based on the emotion.

[0117] The voice analysis unit can estimate the user's emotions and adjust the accuracy of the voice analysis based on the estimated user's emotions. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The voice analysis unit can estimate the user's emotions using, for example, facial expression recognition technology. The voice analysis unit can also estimate the user's emotions using voice analysis technology. Furthermore, the voice analysis unit can estimate the user's emotions using AI. For example, if the user is excited, the voice analysis unit can increase the accuracy of the voice analysis to perform more detailed voice recognition. If the user is relaxed, the voice analysis unit can moderately adjust the accuracy of the voice analysis to eliminate excessive information. If the user is nervous, the voice analysis unit can maintain a medium level of accuracy to reduce auditory strain. This allows for more appropriate voice analysis by adjusting the accuracy of the voice analysis according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generative AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the voice analysis unit may be performed using AI, or may be performed without using AI. For example, the voice analysis unit may input user emotion data into the generation AI and cause the generation AI to adjust the accuracy of the voice analysis based on the emotion.

[0118] The conversion unit can estimate the user's emotion and adjust the intensity of the vibration pattern based on the estimated user's emotion. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The conversion unit can estimate the user's emotion using, for example, facial expression recognition technology. The conversion unit can also estimate the user's emotion using voice analysis technology. Furthermore, the conversion unit can estimate the user's emotion using AI. For example, the conversion unit can increase the intensity of the vibration pattern when the user is excited. The conversion unit can also decrease the intensity of the vibration pattern when the user is relaxed. The conversion unit can also maintain the intensity of the vibration pattern at a medium level when the user is tense. This allows for a more appropriate vibration experience by adjusting the intensity of the vibration pattern according to the user's emotion. The emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the conversion unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit may input user emotion data to the generation AI and cause the generation AI to adjust the intensity of the vibration pattern based on the emotion.

[0119] The providing unit can estimate the user's emotion and adjust the timing of providing vibrations based on the estimated user's emotion. Examples of user emotions include, but are not limited to, excitement, relaxation, and tension. The providing unit can estimate the user's emotion using, for example, facial expression recognition technology. The providing unit can also estimate the user's emotion using voice analysis technology. The providing unit can also estimate the user's emotion using AI. For example, the providing unit can advance the timing of providing vibrations when the user is excited. The providing unit can also delay the timing of providing vibrations when the user is relaxed. The providing unit can also maintain the timing of providing vibrations at a moderate level when the user is tense. This allows for a more appropriate vibration experience by adjusting the timing of providing vibrations according to the user's emotion. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, for example, a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to these examples. Some or all of the above-described processing in the providing unit can be performed using, for example, AI, or without AI. For example, the providing unit can input the user's emotion data into the generating AI and cause the generating AI to adjust the timing of providing vibrations based on the emotion.

[0120] The video analysis unit can improve the accuracy of the analysis by referring to past analysis results. Past analysis results include, but are not limited to, analysis results of the same movie scene and analysis results of similar scenes. For example, the video analysis unit can improve the analysis accuracy by referring to previously analyzed scenes of the same movie. The video analysis unit can also improve the analysis accuracy of similar scenes based on data obtained from past analysis results. The video analysis unit can also learn from past analysis results and optimize the analysis algorithm. For example, the video analysis unit can improve the analysis accuracy by referring to previously analyzed scenes of the same movie. In this way, the analysis accuracy can be improved by referring to past analysis results. Some or all of the above-described processing in the video analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the video analysis unit can input past analysis result data into the generation AI and cause the generation AI to improve the analysis accuracy.

[0121] The audio analysis unit can recognize specific sound effects or music and generate a vibration pattern corresponding to the specific sound effects or music. Specific sound effects or music include, but are not limited to, explosions and specific music. The audio analysis unit can recognize specific sound effects or music using, for example, voice recognition technology. The audio analysis unit can also analyze characteristics of sound effects or music using frequency analysis technology. The audio analysis unit can also extract characteristics of sound effects or music using AI. For example, the audio analysis unit generates a vibration pattern when an explosion occurs. The audio analysis unit can also generate a vibration pattern corresponding to specific music in a scene where the music is playing. The audio analysis unit can also generate a vibration pattern corresponding to a character's scream or laughter. In this way, generating a vibration pattern corresponding to specific sound effects or music can provide a more realistic experience. Some or all of the above-described processing in the audio analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the audio analysis unit can input specific sound effects or music data into the generation AI and have the generation AI generate vibration patterns.

[0122] The conversion unit can improve the accuracy of generation by referring to past vibration pattern data. Past vibration pattern data includes, for example, vibration patterns in the same scene, vibration patterns in similar scenes, etc., but is not limited to these examples. The conversion unit, for example, improves accuracy by referring to vibration patterns generated in the past. The conversion unit can also optimize vibration patterns in similar scenes based on past vibration pattern data. The conversion unit can also learn past vibration pattern data and optimize the generation algorithm. For example, the conversion unit improves accuracy by referring to vibration patterns generated in the past. In this way, the accuracy of generation can be improved by referring to past vibration pattern data. Some or all of the above-mentioned processing in the conversion unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversion unit can input past vibration pattern data to the generation AI and cause the generation AI to improve the generation accuracy.

[0123] The providing unit can optimize the delivery method by referring to the user's past vibration experiences. Past vibration experiences include, but are not limited to, vibration intensity, rhythm, and duration. The providing unit, for example, suggests an optimal delivery method based on the user's past vibration experiences. The providing unit can also provide a preferred vibration pattern based on the user's past vibration experiences. The providing unit can also learn the user's past vibration experiences and optimize the delivery algorithm. For example, the providing unit suggests an optimal delivery method based on the user's past vibration experiences. This makes it easier to optimize the delivery method by referring to the user's past vibration experiences. Some or all of the above-described processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input past vibration experience data into a generation AI and cause the generation AI to optimize the delivery method.

[0124] The video analysis unit can customize the analysis method by reflecting the user's past viewing history. The past viewing history includes, for example, a list of movies viewed and viewing time, but is not limited to these examples. For example, the video analysis unit prioritizes analysis of favorite scenes based on the user's past viewing history. The video analysis unit can also analyze scenes that are likely to be of interest based on the user's past viewing history. The video analysis unit can also learn the user's past viewing history and optimize the analysis algorithm. For example, the video analysis unit prioritizes analysis of favorite scenes based on the user's past viewing history. This makes it easier to customize the analysis method by reflecting the user's past viewing history. Some or all of the above-described processing in the video analysis unit may be performed using, or without, AI. For example, the video analysis unit can input past viewing history data into a generation AI and have the generation AI customize the analysis method.

[0125] The audio analysis unit can customize the analysis method by reflecting the user's past listening history. The past listening history includes, for example, a list of music listened to and listening time, but is not limited to these examples. For example, the audio analysis unit prioritizes analysis of preferred audio based on the user's past listening history. The audio analysis unit can also analyze audio that is likely to be of interest to the user from the user's past listening history. The audio analysis unit can also learn the user's past listening history and optimize the analysis algorithm. For example, the audio analysis unit prioritizes analysis of preferred audio based on the user's past listening history. This makes it easier to customize the analysis method by reflecting the user's past listening history. Some or all of the above-described processing in the audio analysis unit may be performed using, or without, AI. For example, the audio analysis unit can input past listening history data into a generation AI and have the generation AI customize the analysis method.

[0126] The conversion unit can combine multiple vibration patterns according to specific scenes and sound effects. Specific scenes and sound effects include, but are not limited to, explosion sounds in action scenes and music in romance scenes. For example, the conversion unit can combine different vibration patterns according to explosion sounds and gunfire in action scenes. The conversion unit can also combine multiple vibration patterns according to music and character movements in romance scenes. The conversion unit can also combine multiple vibration patterns according to sudden sound effects and visual effects in horror scenes. By combining multiple vibration patterns according to specific scenes and sound effects, a more realistic experience can be provided. Some or all of the above-described processing in the conversion unit can be performed using, for example, AI, or without AI. For example, the conversion unit can input data on scenes and sound effects into a generation AI and have the generation AI execute a combination of multiple vibration patterns.

[0127] The providing unit can provide different vibration patterns depending on the type of movie scene. Types of movie scenes include, but are not limited to, action scenes and emotional scenes. For example, the providing unit can provide a strong vibration pattern for action scenes. The providing unit can also provide a gentle vibration pattern for romance scenes. The providing unit can also provide an irregular vibration pattern for horror scenes. By providing different vibration patterns depending on the type of movie scene, a more realistic experience can be provided. Some or all of the above-described processing in the providing unit can be performed using, for example, AI, or can be performed without using AI. For example, the providing unit can input scene type data to the generating AI and cause the generating AI to provide different vibration patterns.

[0128] The processing flow of the second embodiment will be briefly explained below.

[0129] Step 1: The video analysis unit analyzes movie scenes. For example, the video analysis unit can use image recognition technology to recognize specific scenes. It can also use frame analysis technology to analyze the content of scenes. It can also use AI to extract scene features. Step 2: The audio analysis section analyzes the audio of the movie. For example, it uses voice recognition technology to recognize specific sounds. It can also analyze the characteristics of the audio using frequency analysis technology. Furthermore, it can also extract the audio characteristics using AI. Step 3: The conversion unit generates vibration patterns based on the results of the video and audio analysis. For example, a chair or floor can vibrate according to the scene or audio of a movie. The conversion unit can also generate vibration patterns using AI. Step 4: The providing unit provides the generated vibration pattern to the user. For example, the generated vibration pattern is transmitted to a chair or the floor. The providing unit can also provide the vibration pattern using AI.

[0130] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0131] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0132] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0133] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0134] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0135] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0136] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0137] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0138] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0139] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0140] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0141] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0142] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0143] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0144] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0145] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0146] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0147] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0148] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0149] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0150] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0151] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0152] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0153] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0154] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0155] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0156] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0157] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0158] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0159] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0160] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0161] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0162] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0163] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0164] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0165] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0166] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0167] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0168] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0169] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0170] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0171] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0172] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0173] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0174] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0175] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0176] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0177] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0178] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0179] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0180] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0181] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0182] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0183] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0184] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0185] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0186] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0187] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0188] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0189] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0190] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0191] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0192] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0193] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0194] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0195] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0196] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0197] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0198] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0199] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0200] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0201] [Explanation of symbols]

[0202] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A video analysis department and a part for analyzing audio; a conversion unit that generates a vibration pattern based on the analysis result by the conversion unit; a providing unit that provides a user with the vibration pattern generated by the converting unit; Equipped with A system characterized by:

2. The authentication unit Recognizes specific scenes and generates vibration patterns that correspond to those scenes 2. The system of claim 1.

3. The authentication unit Recognizes specific sounds and generates vibration patterns in response to those sounds 2. The system of claim 1.

4. The conversion unit Make chairs and floors vibrate according to movie scenes 2. The system of claim 1.

5. The providing unit Provide the generated vibration pattern to the user 2. The system of claim 1.

6. The authentication unit that analyzes the video includes: Estimate the user's emotions and adjust the accuracy of video analysis based on the estimated user emotions.

2. The system of claim 1.

7. The authentication unit that analyzes the video includes: Applying different analysis algorithms depending on the movie genre 2. The system of claim 1.

8. The authentication unit that analyzes the video includes: Improve the accuracy of your analysis by referencing past analysis results 2. The system of claim 1.

Images