System

Abstract

An object of a system according to an embodiment is to perform information presentation utilizing multiple senses using generated AI.SOLUTION: A system includes a visual information presentation part, an auditory information presentation part, a tactile information presentation part, a taste information presentation part, and an olfactory information presentation part. The visual information presenting unit presents visual information by using the generated AI. The audio information presenting unit presents audio information by using the generated AI. The tactile information presenting unit presents tactile information by using the generated AI. The taste information presenter presents taste information by using the generated AI. The olfactory information presenting unit presents olfactory information by using the generated AI.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies have had issues with low efficiency in information input and output, particularly in the lack of multi-sensory information presentation.

[0005] The system according to the embodiment aims to present information using multi-sensory technology using generative AI. [Means for solving the problem]

[0006] The system according to the embodiment includes a visual information presentation unit, an auditory information presentation unit, a tactile information presentation unit, a taste information presentation unit, and an olfactory information presentation unit. The visual information presentation unit presents visual information using a generation AI. The auditory information presentation unit presents auditory information using a generation AI. The tactile information presentation unit presents tactile information using a generation AI. The taste information presentation unit presents taste information using a generation AI. The olfactory information presentation unit presents olfactory information using a generation AI. [Effects of the Invention]

[0007] The system according to the embodiment can present information using multi-sensory technology by using generation AI. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The information input system according to an embodiment of the present invention is a system that utilizes generative AI to input information into the brain through the five senses (sight, hearing, touch, taste, and smell). This makes it possible for the information input system to improve the efficiency of information storage and recall, thereby realizing an information revolution.

[0029] An information input system according to an embodiment includes a visual information presentation unit, an auditory information presentation unit, a tactile information presentation unit, a taste information presentation unit, and an olfactory information presentation unit. The visual information presentation unit presents visual information using a generation AI. For example, the generation AI analyzes visual information and presents it in an optimal form to the user. The generation AI converts learning content into graphs and diagrams that are visually easy to understand. Input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates visual learning materials based on the prompt. For example, the generation AI converts learning content into graphs and diagrams that are visually easy to understand to make information input easier. The auditory information presentation unit presents auditory information using the generation AI. For example, the generation AI analyzes auditory information and presents it in an optimal form to the user. The generation AI generates audio learning materials that explain the learning content aloud. Input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates audio learning materials based on the prompt. For example, the generation AI generates audio learning materials that explain the learning content aloud to make information input easier. The tactile information presentation unit uses a generation AI to present tactile information. For example, the generation AI analyzes tactile information and presents it in an optimal form to the user. The generation AI controls a device that provides tactile feedback that allows the user to understand the learning content tactilely. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates tactile feedback based on the prompt. For example, the generation AI controls a device that provides tactile feedback that allows the user to understand the learning content tactilely, making it easier to input information tactilely. The taste information presentation unit uses a generation AI to present taste information. For example, the generation AI analyzes taste information and presents it in an optimal form to the user. The generation AI controls a device that provides a taste experience related to the learning content. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates taste feedback based on the prompt. For example, the generation AI controls a device that provides a taste experience related to the learning content, making it easier to input information tactilely. The olfactory information presentation unit uses a generation AI to present olfactory information.For example, the generation AI analyzes olfactory information and presents it to the user in an optimal form. The generation AI controls a device that provides a scent related to the learning content. The input to the generation AI is a prompt containing instructions on the content the user wants to learn, and the generation AI generates olfactory feedback based on the prompt. For example, the generation AI controls a device that provides a scent related to the learning content to make it easier to input information olfactorily. As a result, the information input system according to the embodiment can input information using the five senses, thereby making it possible to efficiently memorize and recall information.

[0030] The visual information presentation unit can analyze gaze tracking data and automatically adjust the most visually effective information presentation method. For example, the visual information presentation unit uses an gaze tracking device while the user is studying, and the generation AI analyzes the gaze data in real time. For example, if the user focuses their gaze on a specific area, that area is highlighted. The visual information presentation unit also automatically adjusts the information presentation method based on the gaze tracking data to make it easier for the user to understand visually. For example, if the user frequently moves their gaze, the information is presented in parts. The visual information presentation unit also analyzes the gaze tracking data using the generation AI to identify areas where the user is having visual difficulty and reconstructs and presents those parts. For example, a complex graph can be simplified and displayed. This can increase the effectiveness of visual information input by optimizing visual information based on the user's gaze data.

[0031] The visual information presentation unit can learn visual preferences and learning history and generate individually optimized visual teaching materials. For example, the generation AI of the visual information presentation unit analyzes the user's past learning history and generates teaching materials that are visually easy to understand. For example, new teaching materials are created based on the style of teaching materials that the user has previously preferred. The visual information presentation unit also learns the user's visual preferences, and the generation AI customizes the teaching materials based on that. For example, it adjusts the color usage and font size to suit the user's preferences. The visual information presentation unit also suggests visually effective teaching materials based on the user's learning history. For example, it reuses the format of teaching materials that have previously shown high learning effectiveness. This makes it possible to increase the effectiveness of visual information input by optimizing visual teaching materials based on the user's preferences and learning history.

[0032] The visual information presentation unit can present visual information as a 3D hologram, allowing the user to understand the information in three dimensions. For example, the generation AI in the visual information presentation unit generates learning content as a 3D hologram, allowing the user to understand the information in three dimensions. For example, a scientific experiment is reproduced in a 3D hologram. The visual information presentation unit also allows the user to learn while operating the 3D hologram. For example, a historical building can be displayed in a 3D hologram, allowing the user to freely explore it. The generation AI in the visual information presentation unit also dynamically adjusts the content of the 3D hologram according to the user's learning progress. For example, the viewpoint of the hologram can be changed to make it easier for the user to understand. In this way, presenting visual information as a 3D hologram enables three-dimensional information input.

[0033] The visual information presentation unit can combine visual information with AR technology to provide a learning experience that blends the real world and digital information. For example, the generation AI in the visual information presentation unit overlays learning content onto the real world using AR technology. For example, a historical event may be recreated in an actual location in a history class. The visual information presentation unit also allows the user to use an AR device to provide a learning experience that blends the real world and digital information. For example, the growth process of a plant may be displayed using AR. The generation AI in the visual information presentation unit also dynamically adjusts the AR content according to the user's learning progress. For example, it changes the placement of AR objects to make it easier for the user to understand. In this way, by combining visual information with AR technology, a learning experience that blends the real world and digital information is possible.

[0034] The auditory information presentation unit can learn the auditory preferences and learning history and generate individually optimized audio learning materials. For example, the generation AI of the auditory information presentation unit analyzes the user's past learning history and generates audio learning materials that are easy to understand auditorily. For example, new learning materials are created based on the user's preferred audio style. The auditory information presentation unit also learns the user's auditory preferences, and the generation AI customizes the audio learning materials based on that. For example, the tone and speed of the audio are adjusted to suit the user's preferences. The auditory information presentation unit also suggests auditorily effective audio learning materials based on the user's learning history. For example, an audio style that has proven highly effective in the past is reused. This allows the effectiveness of auditory information input to be increased by optimizing the audio learning materials based on the user's preferences and learning history.

[0035] The auditory information presentation unit can analyze the reaction speed to auditory information and automatically adjust the optimal audio presentation speed. For example, the generation AI of the auditory information presentation unit analyzes the user's reaction speed to auditory information in real time and automatically adjusts the optimal audio presentation speed. For example, the audio is played at a speed that is easy for the user to understand. In addition, the generation AI of the auditory information presentation unit dynamically adjusts the speed of the audio teaching material according to the user's reaction speed. For example, if the user can understand it quickly, the speed is increased, and if the user can understand it slowly, the speed is decreased. In addition, the generation AI of the auditory information presentation unit optimizes the presentation method of the audio teaching material based on the user's reaction speed data. For example, appropriate pauses are inserted between audio segments to make it easier for the user to understand. In this way, the audio presentation speed can be optimized based on the user's reaction speed, thereby enhancing the effectiveness of auditory information input.

[0036] The auditory information presentation unit can present auditory information as binaural audio, allowing the user to experience sound in three dimensions. For example, the auditory information presentation unit uses a generation AI to generate learning content as binaural audio, allowing the user to experience sound in three dimensions. For example, the audio of a natural environment is reproduced using binaural audio. The auditory information presentation unit also allows the user to use the binaural audio to have a learning experience with three-dimensional sound. For example, a historical event may be reproduced using audio, making the user feel as if they were actually there. The auditory information presentation unit also allows the generation AI to dynamically adjust the content of the binaural audio according to the user's learning progress. For example, the position of the sound may be changed to make it easier for the user to understand. As a result, presenting auditory information as binaural audio enables a learning experience with three-dimensional sound.

[0037] The auditory information presentation unit can combine auditory information with VR technology to provide an audio learning experience within a virtual space. For example, the auditory information presentation unit uses a generation AI to recreate learning content within a virtual space using VR technology, allowing the user to experience the audio learning. For example, it provides audio lectures in a virtual classroom. The auditory information presentation unit also allows the user to use a VR device to experience the audio learning within the virtual space. For example, it provides an audio guide in a virtual museum. The auditory information presentation unit also allows the generation AI to dynamically adjust the VR content according to the user's learning progress. For example, it changes the audio placement within the virtual space to make it easier for the user to understand. In this way, by combining auditory information with VR technology, an audio learning experience within a virtual space is possible.

[0038] The tactile information presentation unit can learn tactile preferences and learning history and generate individually optimized tactile feedback. For example, the generation AI of the tactile information presentation unit analyzes the user's past learning history and generates feedback that is easy to understand tactilely. For example, new feedback is created based on the user's preferred tactile device settings. The tactile information presentation unit also learns the user's tactile preferences, and the generation AI customizes the tactile feedback based on that. For example, the vibration intensity and pattern are adjusted to suit the user's preferences. The tactile information presentation unit also suggests tactilely effective feedback based on the user's learning history. For example, the generation AI reuses a tactile feedback format that has previously proven highly effective for learning. This allows the effectiveness of tactile information input to be enhanced by optimizing tactile feedback based on the user's preferences and learning history.

[0039] The tactile information presentation unit can analyze the reaction speed to tactile information and automatically adjust the optimal tactile presentation speed. For example, the generation AI of the tactile information presentation unit analyzes the user's reaction speed to tactile information in real time and automatically adjusts the optimal tactile presentation speed. For example, the tactile feedback is provided at a speed that is easy for the user to understand. Furthermore, the generation AI of the tactile information presentation unit dynamically adjusts the speed of the tactile feedback according to the user's reaction speed. For example, if the user can understand it quickly, the speed is increased, and if the user can understand it slowly, the speed is decreased. Furthermore, the generation AI of the tactile information presentation unit optimizes the presentation method of the tactile feedback based on the user's reaction speed data. For example, an appropriate pause is inserted between tactile feedback to make it easy for the user to understand. In this way, the tactile presentation speed can be optimized based on the user's reaction speed, thereby enhancing the effectiveness of tactile information input.

[0040] The tactile information presentation unit can combine tactile information with VR technology to provide a tactile learning experience within a virtual space. For example, the generation AI in the tactile information presentation unit uses VR technology to recreate learning content within a virtual space, allowing the user to have a tactile learning experience. For example, the tactile information presentation unit provides a lesson in a virtual classroom with tactile feedback. The tactile information presentation unit also allows the user to use a VR device to have a tactile learning experience within the virtual space. For example, the tactile information presentation unit provides a tactile guide in a virtual museum. The generation AI in the tactile information presentation unit also dynamically adjusts the VR content according to the user's learning progress. For example, the tactile placement within the virtual space can be changed to make it easier for the user to understand. In this way, combining tactile information with VR technology enables a tactile learning experience within a virtual space.

[0041] The tactile information presentation unit can combine tactile information with AR technology to provide a tactile experience that blends the real world and digital information. For example, the generation AI in the tactile information presentation unit overlays learning content onto the real world using AR technology. For example, in a history class, historical events are recreated in actual locations with tactile feedback. The tactile information presentation unit also allows users to use an AR device to provide a tactile experience that blends the real world and digital information. For example, the growth process of a plant is displayed in AR and tactile feedback is provided. The generation AI in the tactile information presentation unit also dynamically adjusts the AR content according to the user's learning progress. For example, the tactile placement of an AR object is changed to make it easier for the user to understand. In this way, combining tactile information with AR technology enables a tactile experience that blends the real world and digital information.

[0042] The taste information presentation unit can learn taste preferences and learning history and generate individually optimized taste feedback. For example, the generation AI in the taste information presentation unit analyzes the user's past learning history and generates feedback that is easy to understand from a taste perspective. For example, new feedback is created based on the user's preferred settings for a taste device. The taste information presentation unit also learns the user's taste preferences, and the generation AI customizes the taste feedback based on that. For example, the intensity and type of flavor are adjusted to suit the user's preferences. The taste information presentation unit also suggests taste-effective feedback based on the user's learning history. For example, the generation AI reuses a taste feedback format that has previously shown high learning effects. This makes it possible to increase the effectiveness of taste information input by optimizing taste feedback based on the user's preferences and learning history.

[0043] The taste information presentation unit can analyze the reaction speed to taste information and automatically adjust the optimal taste presentation speed. For example, the generation AI of the taste information presentation unit analyzes the user's reaction speed to taste information in real time and automatically adjusts the optimal taste presentation speed. For example, it provides taste feedback at a speed that is easy for the user to understand. Furthermore, the generation AI of the taste information presentation unit dynamically adjusts the speed of the taste feedback according to the user's reaction speed. For example, it increases the speed if the user can understand it quickly and decreases the speed if the user can understand it slowly. Furthermore, the generation AI of the taste information presentation unit optimizes the presentation method of taste feedback based on the user's reaction speed data. For example, it inserts appropriate pauses between taste feedback to make it easy for the user to understand. In this way, the effectiveness of taste information input can be enhanced by optimizing the taste presentation speed based on the user's reaction speed.

[0044] The taste information presentation unit can combine taste information with VR technology to provide a taste learning experience in a virtual space. For example, the taste information presentation unit uses a generation AI to recreate learning content in a virtual space using VR technology, allowing the user to have a taste learning experience. For example, it provides a lesson in a virtual classroom with taste feedback. The taste information presentation unit also allows the user to use a VR device to have a taste learning experience in a virtual space. For example, it provides a taste guide in a virtual museum. The taste information presentation unit also allows the generation AI to dynamically adjust the VR content according to the user's learning progress. For example, it changes the taste arrangement in the virtual space to make it easier for the user to understand. In this way, by combining taste information with VR technology, a taste learning experience in a virtual space is possible.

[0045] The taste information presentation unit can combine taste information with AR technology to provide a taste experience that blends the real world and digital information. For example, the generation AI in the taste information presentation unit overlays learning content onto the real world using AR technology. For example, in a history class, historical events are recreated in actual locations with taste feedback. The taste information presentation unit also allows the user to use an AR device to provide a taste experience that blends the real world and digital information. For example, the growth process of a plant is displayed in AR and taste feedback is provided. The generation AI in the taste information presentation unit also dynamically adjusts the AR content according to the user's learning progress. For example, the taste placement of an AR object is changed to make it easier for the user to understand. In this way, by combining taste information with AR technology, a taste experience that blends the real world and digital information is possible.

[0046] The olfactory information presentation unit can learn olfactory preferences and learning history and generate individually optimized olfactory feedback. For example, the generation AI in the olfactory information presentation unit analyzes the user's past learning history and generates feedback that is easy to understand olfactorily. For example, new feedback is created based on the user's preferred olfactory device settings. The olfactory information presentation unit also learns the user's olfactory preferences, and the generation AI customizes the olfactory feedback based on that. For example, the strength and type of scent are adjusted to suit the user's preferences. The olfactory information presentation unit also suggests olfactorily effective feedback based on the user's learning history. For example, the generation AI reuses a format of olfactory feedback that has previously shown high learning effects. This allows the effectiveness of olfactory information input to be enhanced by optimizing olfactory feedback based on the user's preferences and learning history.

[0047] The olfactory information presentation unit can analyze the reaction speed to olfactory information and automatically adjust the optimal olfactory presentation speed. For example, the generation AI of the olfactory information presentation unit analyzes the user's reaction speed to olfactory information in real time and automatically adjusts the optimal olfactory presentation speed. For example, the olfactory feedback is provided at a speed that is easy for the user to understand. Furthermore, the generation AI of the olfactory information presentation unit dynamically adjusts the speed of the olfactory feedback according to the user's reaction speed. For example, if the user can understand it quickly, the speed is increased, and if the user can understand it slowly, the speed is decreased. Furthermore, the generation AI of the olfactory information presentation unit optimizes the presentation method of the olfactory feedback based on the user's reaction speed data. For example, an appropriate pause is inserted between olfactory feedback to make it easy for the user to understand. In this way, the olfactory presentation speed can be optimized based on the user's reaction speed, thereby enhancing the effectiveness of olfactory information input.

[0048] The olfactory information presentation unit can combine olfactory information with VR technology to provide an olfactory learning experience in a virtual space. In the olfactory information presentation unit, for example, a generation AI recreates learning content in a virtual space using VR technology, allowing the user to have an olfactory learning experience. For example, the olfactory information presentation unit provides olfactory feedback for a lesson in a virtual classroom. The olfactory information presentation unit also allows the user to use a VR device to have an olfactory learning experience in a virtual space. For example, the olfactory information presentation unit provides an olfactory guide in a virtual museum. The olfactory information presentation unit also allows the generation AI to dynamically adjust the VR content according to the user's learning progress. For example, the olfactory placement in the virtual space can be changed to make it easier for the user to understand. In this way, by combining olfactory information with VR technology, an olfactory learning experience in a virtual space is possible.

[0049] The olfactory information presentation unit can combine olfactory information with AR technology to provide an olfactory experience that blends the real world and digital information. For example, the generation AI in the olfactory information presentation unit overlays learning content onto the real world using AR technology. For example, in a history class, historical events are recreated in actual locations using olfactory feedback. The olfactory information presentation unit also allows the user to use an AR device to provide an olfactory experience that blends the real world and digital information. For example, the growth process of a plant is displayed using AR and olfactory feedback is provided. The generation AI in the olfactory information presentation unit also dynamically adjusts the AR content according to the user's learning progress. For example, the olfactory placement of an AR object is changed to make it easier for the user to understand. In this way, by combining olfactory information with AR technology, an olfactory experience that blends the real world and digital information is possible.

[0050] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0051] The information input system can further include a learning style analysis unit that analyzes the user's learning style and suggests the most suitable learning method. For example, the learning style analysis unit analyzes the learning materials and learning methods used by the user in the past and suggests the most suitable learning method for the user. For example, if the user prefers visual learning, the visual information presentation unit can be enhanced. The learning style analysis unit can also monitor the user's learning progress and dynamically adjust the learning method. For example, if there is a part that the user finds difficult to understand, it can suggest a method to focus on studying that part. This can improve learning effectiveness by providing the most suitable learning method based on the user's learning style.

[0052] The information input system can further include a health monitoring unit that monitors the user's health condition and provides an optimal learning environment. For example, the health monitoring unit monitors the user's heart rate and stress level and adjusts the learning environment. For example, if the user is feeling stressed, it provides a relaxing environment. The health monitoring unit also optimizes study time and break times based on the user's health data. For example, if the user is tired, it sends a notification encouraging them to take a break. This allows for an optimal learning environment based on the user's health condition, thereby improving learning effectiveness.

[0053] The information input system may further include a learning goal setting unit that sets learning goals for the user and monitors the degree of achievement of those goals. For example, the learning goal setting unit creates a learning plan based on the learning goals set by the user. For example, if the user's goal is to pass a specific exam, the learning goal setting unit proposes a learning plan for that purpose. The learning goal setting unit also monitors the user's learning progress and evaluates the degree of goal achievement. For example, it displays in real time whether the user is approaching the goal. The learning goal setting unit also provides advice to help the user achieve the goal. For example, it suggests areas for improvement in learning methods. This makes it possible to improve learning effectiveness by providing an optimal learning plan based on the user's learning goals.

[0054] The information input system can further include a community formation unit that forms a learning community of users and promotes collaborative learning. For example, the community formation unit matches users with the same learning goals and provides opportunities for collaborative learning. For example, it may suggest online discussions or group projects. The community formation unit also promotes feedback between users to improve learning effectiveness. For example, it may provide a mechanism for users to evaluate the learning progress of other users. The community formation unit also proposes events and activities to maintain the user learning community. For example, it may hold regular online seminars or workshops. In this way, a learning community of users is formed, collaborative learning is promoted, and learning effectiveness can be improved.

[0055] The information input system can further include a data analysis unit that analyzes the user's learning data and visualizes the progress and effectiveness of learning. For example, the data analysis unit analyzes the user's learning history and test results and displays the learning progress in graphs and charts. For example, it visually shows in which areas the user has progressed. The data analysis unit also evaluates the user's learning effectiveness and suggests areas for improvement. For example, it identifies areas in which the user is weak and proposes a study plan that focuses on those areas. The data analysis unit also provides advice to maximize the effectiveness of learning based on the user's learning data. For example, it suggests optimizing study time and effective study methods. In this way, the learning effectiveness can be improved by analyzing the user's learning data and visualizing the progress and effectiveness of learning.

[0056] The processing flow of the first embodiment will be briefly explained below.

[0057] Step 1: The visual information presentation unit uses the generation AI to present visual information. The generation AI analyzes the visual information and presents it in the most appropriate form for the user. For example, it converts the learning content into graphs and charts that are easy to understand visually. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates visual learning materials based on that prompt. Step 2: The auditory information presentation unit uses the generation AI to present the auditory information. The generation AI analyzes the auditory information and presents it in the most appropriate form for the user. For example, it generates audio learning materials that explain the learning content aloud. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates the audio learning materials based on that prompt. Step 3: The tactile information presentation unit uses the generation AI to present tactile information. The generation AI analyzes the tactile information and presents it in the most appropriate form for the user. For example, it controls a device that provides tactile feedback that allows the user to understand the learning content tactilely. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates tactile feedback based on that prompt. Step 4: The taste information presentation unit uses the generation AI to present taste information. The generation AI analyzes the taste information and presents it in the most appropriate form for the user. For example, it controls a device that provides a taste experience related to the learning content. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates taste feedback based on the prompt. Step 5: The olfactory information presentation unit uses the generation AI to present olfactory information. The generation AI analyzes the olfactory information and presents it in the most optimal form for the user. For example, it controls a device that provides scents related to the learning content. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates olfactory feedback based on the prompt.

[0058] (Example 2) The information input system according to an embodiment of the present invention is a system that utilizes generative AI to input information into the brain through the five senses (sight, hearing, touch, taste, and smell). This makes it possible for the information input system to improve the efficiency of information storage and recall, thereby realizing an information revolution.

[0059] An information input system according to an embodiment includes a visual information presentation unit, an auditory information presentation unit, a tactile information presentation unit, a taste information presentation unit, and an olfactory information presentation unit. The visual information presentation unit presents visual information using a generation AI. For example, the generation AI analyzes visual information and presents it in an optimal form to the user. The generation AI converts learning content into graphs and diagrams that are visually easy to understand. Input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates visual learning materials based on the prompt. For example, the generation AI converts learning content into graphs and diagrams that are visually easy to understand to make information input easier. The auditory information presentation unit presents auditory information using the generation AI. For example, the generation AI analyzes auditory information and presents it in an optimal form to the user. The generation AI generates audio learning materials that explain the learning content aloud. Input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates audio learning materials based on the prompt. For example, the generation AI generates audio learning materials that explain the learning content aloud to make information input easier. The tactile information presentation unit uses a generation AI to present tactile information. For example, the generation AI analyzes tactile information and presents it in an optimal form to the user. The generation AI controls a device that provides tactile feedback that allows the user to understand the learning content tactilely. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates tactile feedback based on the prompt. For example, the generation AI controls a device that provides tactile feedback that allows the user to understand the learning content tactilely, making it easier to input information tactilely. The taste information presentation unit uses a generation AI to present taste information. For example, the generation AI analyzes taste information and presents it in an optimal form to the user. The generation AI controls a device that provides a taste experience related to the learning content. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates taste feedback based on the prompt. For example, the generation AI controls a device that provides a taste experience related to the learning content, making it easier to input information tactilely. The olfactory information presentation unit uses a generation AI to present olfactory information.For example, the generation AI analyzes olfactory information and presents it to the user in an optimal form. The generation AI controls a device that provides a scent related to the learning content. The input to the generation AI is a prompt containing instructions on the content the user wants to learn, and the generation AI generates olfactory feedback based on the prompt. For example, the generation AI controls a device that provides a scent related to the learning content to make it easier to input information olfactorily. As a result, the information input system according to the embodiment can input information using the five senses, thereby making it possible to efficiently memorize and recall information.

[0060] The visual information presentation unit can analyze gaze tracking data and automatically adjust the most visually effective information presentation method. For example, the visual information presentation unit uses an gaze tracking device while the user is studying, and the generation AI analyzes the gaze data in real time. For example, if the user focuses their gaze on a specific area, that area is highlighted. The visual information presentation unit also automatically adjusts the information presentation method based on the gaze tracking data to make it easier for the user to understand visually. For example, if the user frequently moves their gaze, the information is presented in parts. The visual information presentation unit also analyzes the gaze tracking data using the generation AI to identify areas where the user is having visual difficulty and reconstructs and presents those parts. For example, a complex graph can be simplified and displayed. This can increase the effectiveness of visual information input by optimizing visual information based on the user's gaze data.

[0061] The visual information presentation unit can learn visual preferences and learning history and generate individually optimized visual teaching materials. For example, the generation AI of the visual information presentation unit analyzes the user's past learning history and generates teaching materials that are visually easy to understand. For example, new teaching materials are created based on the style of teaching materials that the user has previously preferred. The visual information presentation unit also learns the user's visual preferences, and the generation AI customizes the teaching materials based on that. For example, it adjusts the color usage and font size to suit the user's preferences. The visual information presentation unit also suggests visually effective teaching materials based on the user's learning history. For example, it reuses the format of teaching materials that have previously shown high learning effectiveness. This makes it possible to increase the effectiveness of visual information input by optimizing visual teaching materials based on the user's preferences and learning history.

[0062] The visual information presentation unit can use the emotion estimation function to analyze the user's emotions toward visual information and generate visual teaching materials that elicit positive emotions. For example, the generation AI of the visual information presentation unit analyzes the user's facial expressions and gaze data to estimate the user's emotions toward the visual information. For example, it highlights areas that the user is interested in. The visual information presentation unit also uses the emotion estimation function to generate teaching materials that elicit positive emotions toward the visual information. For example, it incorporates colors and designs that the user prefers. The generation AI of the visual information presentation unit also dynamically adjusts the visual teaching materials based on the user's emotion data. For example, if the user is tired, it presents visually relaxing content. This makes it possible to enhance the effectiveness of visual information input by optimizing the visual teaching materials based on the user's emotions.

[0063] The visual information presentation unit can present visual information as a 3D hologram, allowing the user to understand the information in three dimensions. For example, the generation AI in the visual information presentation unit generates learning content as a 3D hologram, allowing the user to understand the information in three dimensions. For example, a scientific experiment is reproduced in a 3D hologram. The visual information presentation unit also allows the user to learn while operating the 3D hologram. For example, a historical building can be displayed in a 3D hologram, allowing the user to freely explore it. The generation AI in the visual information presentation unit also dynamically adjusts the content of the 3D hologram according to the user's learning progress. For example, the viewpoint of the hologram can be changed to make it easier for the user to understand. In this way, presenting visual information as a 3D hologram enables three-dimensional information input.

[0064] The visual information presentation unit can combine visual information with AR technology to provide a learning experience that blends the real world and digital information. For example, the generation AI in the visual information presentation unit overlays learning content onto the real world using AR technology. For example, a historical event may be recreated in an actual location in a history class. The visual information presentation unit also allows the user to use an AR device to provide a learning experience that blends the real world and digital information. For example, the growth process of a plant may be displayed using AR. The generation AI in the visual information presentation unit also dynamically adjusts the AR content according to the user's learning progress. For example, it changes the placement of AR objects to make it easier for the user to understand. In this way, by combining visual information with AR technology, a learning experience that blends the real world and digital information is possible.

[0065] The visual information presentation unit uses an emotion estimation function to monitor emotional changes in real time while viewing visual information and dynamically present optimal visual information. For example, the generation AI of the visual information presentation unit analyzes the user's facial expressions and gaze data in real time to monitor emotional changes. For example, it highlights areas in which the user is interested. The visual information presentation unit also uses the emotion estimation function to dynamically adjust the information so that the user feels positive emotions toward the visual information. For example, it incorporates colors and designs that the user prefers. The generation AI of the visual information presentation unit also dynamically presents visual information based on the user's emotional data. For example, if the user is tired, it presents content that is visually relaxing. This allows the effectiveness of visual information input to be enhanced by monitoring the user's emotional changes in real time and dynamically presenting optimal visual information.

[0066] The auditory information presentation unit can learn the auditory preferences and learning history and generate individually optimized audio learning materials. For example, the generation AI of the auditory information presentation unit analyzes the user's past learning history and generates audio learning materials that are easy to understand auditorily. For example, new learning materials are created based on the user's preferred audio style. The auditory information presentation unit also learns the user's auditory preferences, and the generation AI customizes the audio learning materials based on that. For example, the tone and speed of the audio are adjusted to suit the user's preferences. The auditory information presentation unit also suggests auditorily effective audio learning materials based on the user's learning history. For example, an audio style that has proven highly effective in the past is reused. This allows the effectiveness of auditory information input to be increased by optimizing the audio learning materials based on the user's preferences and learning history.

[0067] The auditory information presentation unit can analyze the reaction speed to auditory information and automatically adjust the optimal audio presentation speed. For example, the generation AI of the auditory information presentation unit analyzes the user's reaction speed to auditory information in real time and automatically adjusts the optimal audio presentation speed. For example, the audio is played at a speed that is easy for the user to understand. In addition, the generation AI of the auditory information presentation unit dynamically adjusts the speed of the audio teaching material according to the user's reaction speed. For example, if the user can understand it quickly, the speed is increased, and if the user can understand it slowly, the speed is decreased. In addition, the generation AI of the auditory information presentation unit optimizes the presentation method of the audio teaching material based on the user's reaction speed data. For example, appropriate pauses are inserted between audio segments to make it easier for the user to understand. In this way, the audio presentation speed can be optimized based on the user's reaction speed, thereby enhancing the effectiveness of auditory information input.

[0068] The auditory information presentation unit can use the emotion estimation function to analyze the emotions felt in response to auditory information and generate audio learning materials that elicit positive emotions. For example, the generation AI of the auditory information presentation unit analyzes the user's tone of voice and facial expression to estimate the emotions felt in response to auditory information. For example, it highlights parts that the user is interested in. The auditory information presentation unit also uses the emotion estimation function to generate audio learning materials that elicit positive emotions in response to auditory information. For example, it incorporates music and sound effects that the user likes. The auditory information presentation unit also dynamically adjusts the audio learning materials based on the user's emotional data. For example, it uses an audio tone that relaxes the user. This allows the audio learning materials to be optimized based on the user's emotions, thereby enhancing the effectiveness of auditory information input.

[0069] The auditory information presentation unit can present auditory information as binaural audio, allowing the user to experience sound in three dimensions. For example, the auditory information presentation unit uses a generation AI to generate learning content as binaural audio, allowing the user to experience sound in three dimensions. For example, the audio of a natural environment is reproduced using binaural audio. The auditory information presentation unit also allows the user to use the binaural audio to have a learning experience with three-dimensional sound. For example, a historical event may be reproduced using audio, making the user feel as if they were actually there. The auditory information presentation unit also allows the generation AI to dynamically adjust the content of the binaural audio according to the user's learning progress. For example, the position of the sound may be changed to make it easier for the user to understand. As a result, presenting auditory information as binaural audio enables a learning experience with three-dimensional sound.

[0070] The auditory information presentation unit can combine auditory information with VR technology to provide an audio learning experience within a virtual space. For example, the auditory information presentation unit uses a generation AI to recreate learning content within a virtual space using VR technology, allowing the user to experience the audio learning. For example, it provides audio lectures in a virtual classroom. The auditory information presentation unit also allows the user to use a VR device to experience the audio learning within the virtual space. For example, it provides an audio guide in a virtual museum. The auditory information presentation unit also allows the generation AI to dynamically adjust the VR content according to the user's learning progress. For example, it changes the audio placement within the virtual space to make it easier for the user to understand. In this way, by combining auditory information with VR technology, an audio learning experience within a virtual space is possible.

[0071] The auditory information presentation unit uses an emotion estimation function to monitor emotional changes in real time while listening to auditory information and dynamically present optimal auditory information. For example, the generation AI of the auditory information presentation unit analyzes the user's tone of voice and facial expressions in real time to monitor emotional changes. For example, it may highlight parts of the audio that show the user's interest. The auditory information presentation unit also uses the emotion estimation function to dynamically adjust the information so that the user feels positive emotions toward the auditory information. For example, it may incorporate music and sound effects that the user likes. The auditory information presentation unit also dynamically presents auditory information based on the user's emotional data. For example, it may use an audio tone that helps the user relax. This allows the unit to monitor the user's emotional changes in real time and dynamically present optimal auditory information, thereby enhancing the effectiveness of auditory information input.

[0072] The tactile information presentation unit can learn tactile preferences and learning history and generate individually optimized tactile feedback. For example, the generation AI of the tactile information presentation unit analyzes the user's past learning history and generates feedback that is easy to understand tactilely. For example, new feedback is created based on the user's preferred tactile device settings. The tactile information presentation unit also learns the user's tactile preferences, and the generation AI customizes the tactile feedback based on that. For example, the vibration intensity and pattern are adjusted to suit the user's preferences. The tactile information presentation unit also suggests tactilely effective feedback based on the user's learning history. For example, the generation AI reuses a tactile feedback format that has previously proven highly effective for learning. This allows the effectiveness of tactile information input to be enhanced by optimizing tactile feedback based on the user's preferences and learning history.

[0073] The tactile information presentation unit can analyze the reaction speed to tactile information and automatically adjust the optimal tactile presentation speed. For example, the generation AI of the tactile information presentation unit analyzes the user's reaction speed to tactile information in real time and automatically adjusts the optimal tactile presentation speed. For example, the tactile feedback is provided at a speed that is easy for the user to understand. Furthermore, the generation AI of the tactile information presentation unit dynamically adjusts the speed of the tactile feedback according to the user's reaction speed. For example, if the user can understand it quickly, the speed is increased, and if the user can understand it slowly, the speed is decreased. Furthermore, the generation AI of the tactile information presentation unit optimizes the presentation method of the tactile feedback based on the user's reaction speed data. For example, an appropriate pause is inserted between tactile feedback to make it easy for the user to understand. In this way, the tactile presentation speed can be optimized based on the user's reaction speed, thereby enhancing the effectiveness of tactile information input.

[0074] The tactile information presentation unit can use the emotion estimation function to analyze the user's emotions toward the tactile information and generate tactile feedback that elicits positive emotions. For example, the generation AI of the tactile information presentation unit analyzes the user's facial expressions and reactions to estimate the user's emotions toward the tactile information. For example, it emphasizes tactile feedback that shows the user's interest. The tactile information presentation unit also uses the emotion estimation function to generate feedback that makes the user feel positive emotions toward the tactile information. For example, it incorporates tactile patterns that the user prefers. The generation AI of the tactile information presentation unit also dynamically adjusts the tactile feedback based on the user's emotion data. For example, it provides tactile feedback that helps the user relax. This makes it possible to enhance the effectiveness of tactile information input by optimizing tactile feedback based on the user's emotions.

[0075] The tactile information presentation unit can combine tactile information with VR technology to provide a tactile learning experience within a virtual space. For example, the generation AI in the tactile information presentation unit uses VR technology to recreate learning content within a virtual space, allowing the user to have a tactile learning experience. For example, the tactile information presentation unit provides a lesson in a virtual classroom with tactile feedback. The tactile information presentation unit also allows the user to use a VR device to have a tactile learning experience within the virtual space. For example, the tactile information presentation unit provides a tactile guide in a virtual museum. The generation AI in the tactile information presentation unit also dynamically adjusts the VR content according to the user's learning progress. For example, the tactile placement within the virtual space can be changed to make it easier for the user to understand. In this way, combining tactile information with VR technology enables a tactile learning experience within a virtual space.

[0076] The tactile information presentation unit can combine tactile information with AR technology to provide a tactile experience that blends the real world and digital information. For example, the generation AI in the tactile information presentation unit overlays learning content onto the real world using AR technology. For example, in a history class, historical events are recreated in actual locations with tactile feedback. The tactile information presentation unit also allows users to use an AR device to provide a tactile experience that blends the real world and digital information. For example, the growth process of a plant is displayed in AR and tactile feedback is provided. The generation AI in the tactile information presentation unit also dynamically adjusts the AR content according to the user's learning progress. For example, the tactile placement of an AR object is changed to make it easier for the user to understand. In this way, combining tactile information with AR technology enables a tactile experience that blends the real world and digital information.

[0077] The tactile information presentation unit uses an emotion estimation function to monitor emotional changes in real time when the user is feeling tactile information and dynamically present optimal tactile information. For example, the generation AI of the tactile information presentation unit analyzes the user's facial expressions and reactions in real time and monitors emotional changes. For example, it emphasizes tactile feedback that the user is interested in. The tactile information presentation unit also uses the emotion estimation function to dynamically adjust the information so that the user feels positive emotions toward the tactile information. For example, it incorporates tactile patterns that the user prefers. The tactile information presentation unit also dynamically presents tactile information based on the user's emotional data using the generation AI. For example, it provides tactile feedback that helps the user relax. This allows the effectiveness of tactile information input to be enhanced by monitoring the user's emotional changes in real time and dynamically presenting optimal tactile information.

[0078] The tactile information presentation unit uses an emotion estimation function to monitor emotional changes in real time when the user is feeling tactile information and dynamically present optimal tactile information. For example, the generation AI of the tactile information presentation unit analyzes the user's facial expressions and reactions in real time and monitors emotional changes. For example, it emphasizes tactile feedback that the user is interested in. The tactile information presentation unit also uses the emotion estimation function to dynamically adjust the information so that the user feels positive emotions toward the tactile information. For example, it incorporates tactile patterns that the user prefers. The tactile information presentation unit also dynamically presents tactile information based on the user's emotional data using the generation AI. For example, it provides tactile feedback that helps the user relax. This allows the effectiveness of tactile information input to be enhanced by monitoring the user's emotional changes in real time and dynamically presenting optimal tactile information.

[0079] The taste information presentation unit can learn taste preferences and learning history and generate individually optimized taste feedback. For example, the generation AI in the taste information presentation unit analyzes the user's past learning history and generates feedback that is easy to understand from a taste perspective. For example, new feedback is created based on the user's preferred settings for a taste device. The taste information presentation unit also learns the user's taste preferences, and the generation AI customizes the taste feedback based on that. For example, the intensity and type of flavor are adjusted to suit the user's preferences. The taste information presentation unit also suggests taste-effective feedback based on the user's learning history. For example, the generation AI reuses a taste feedback format that has previously shown high learning effects. This makes it possible to increase the effectiveness of taste information input by optimizing taste feedback based on the user's preferences and learning history.

[0080] The taste information presentation unit can analyze the reaction speed to taste information and automatically adjust the optimal taste presentation speed. For example, the generation AI of the taste information presentation unit analyzes the user's reaction speed to taste information in real time and automatically adjusts the optimal taste presentation speed. For example, it provides taste feedback at a speed that is easy for the user to understand. Furthermore, the generation AI of the taste information presentation unit dynamically adjusts the speed of the taste feedback according to the user's reaction speed. For example, it increases the speed if the user can understand it quickly and decreases the speed if the user can understand it slowly. Furthermore, the generation AI of the taste information presentation unit optimizes the presentation method of taste feedback based on the user's reaction speed data. For example, it inserts appropriate pauses between taste feedback to make it easy for the user to understand. In this way, the effectiveness of taste information input can be enhanced by optimizing the taste presentation speed based on the user's reaction speed.

[0081] The taste information presentation unit can use the emotion estimation function to analyze the user's emotions toward the taste information and generate taste feedback that elicits positive emotions. For example, the generation AI of the taste information presentation unit analyzes the user's facial expressions and reactions to estimate the user's emotions toward the taste information. For example, it emphasizes taste feedback that the user is interested in. The taste information presentation unit also uses the emotion estimation function to generate feedback that makes the user feel positive emotions toward the taste information. For example, it incorporates the user's preferred taste patterns. The generation AI of the taste information presentation unit also dynamically adjusts the taste feedback based on the user's emotion data. For example, it provides taste feedback that helps the user relax. This makes it possible to enhance the effectiveness of taste information input by optimizing taste feedback based on the user's emotions.

[0082] The taste information presentation unit can combine taste information with VR technology to provide a taste learning experience in a virtual space. For example, the taste information presentation unit uses a generation AI to recreate learning content in a virtual space using VR technology, allowing the user to have a taste learning experience. For example, it provides a lesson in a virtual classroom with taste feedback. The taste information presentation unit also allows the user to use a VR device to have a taste learning experience in a virtual space. For example, it provides a taste guide in a virtual museum. The taste information presentation unit also allows the generation AI to dynamically adjust the VR content according to the user's learning progress. For example, it changes the taste arrangement in the virtual space to make it easier for the user to understand. In this way, by combining taste information with VR technology, a taste learning experience in a virtual space is possible.

[0083] The taste information presentation unit can combine taste information with AR technology to provide a taste experience that blends the real world and digital information. For example, the generation AI in the taste information presentation unit overlays learning content onto the real world using AR technology. For example, in a history class, historical events are recreated in actual locations with taste feedback. The taste information presentation unit also allows the user to use an AR device to provide a taste experience that blends the real world and digital information. For example, the growth process of a plant is displayed in AR and taste feedback is provided. The generation AI in the taste information presentation unit also dynamically adjusts the AR content according to the user's learning progress. For example, the taste placement of an AR object is changed to make it easier for the user to understand. In this way, by combining taste information with AR technology, a taste experience that blends the real world and digital information is possible.

[0084] The taste information presentation unit uses the emotion estimation function to monitor emotional changes in real time when the user is experiencing taste information and dynamically present optimal taste information. For example, the generation AI of the taste information presentation unit analyzes the user's facial expressions and reactions in real time and monitors emotional changes. For example, it emphasizes taste feedback that the user is interested in. The taste information presentation unit also uses the emotion estimation function to dynamically adjust the information so that the user feels positive emotions toward the taste information. For example, it incorporates taste patterns that the user prefers. The generation AI of the taste information presentation unit also dynamically presents taste information based on the user's emotional data. For example, it provides taste feedback that helps the user relax. This allows the user's emotional changes to be monitored in real time and optimal taste information to be dynamically presented, thereby enhancing the effectiveness of taste information input.

[0085] The olfactory information presentation unit can learn olfactory preferences and learning history and generate individually optimized olfactory feedback. For example, the generation AI in the olfactory information presentation unit analyzes the user's past learning history and generates feedback that is easy to understand olfactorily. For example, new feedback is created based on the user's preferred olfactory device settings. The olfactory information presentation unit also learns the user's olfactory preferences, and the generation AI customizes the olfactory feedback based on that. For example, the strength and type of scent are adjusted to suit the user's preferences. The olfactory information presentation unit also suggests olfactorily effective feedback based on the user's learning history. For example, the generation AI reuses a format of olfactory feedback that has previously shown high learning effects. This allows the effectiveness of olfactory information input to be enhanced by optimizing olfactory feedback based on the user's preferences and learning history.

[0086] The olfactory information presentation unit can analyze the reaction speed to olfactory information and automatically adjust the optimal olfactory presentation speed. For example, the generation AI of the olfactory information presentation unit analyzes the user's reaction speed to olfactory information in real time and automatically adjusts the optimal olfactory presentation speed. For example, the olfactory feedback is provided at a speed that is easy for the user to understand. Furthermore, the generation AI of the olfactory information presentation unit dynamically adjusts the speed of the olfactory feedback according to the user's reaction speed. For example, if the user can understand it quickly, the speed is increased, and if the user can understand it slowly, the speed is decreased. Furthermore, the generation AI of the olfactory information presentation unit optimizes the presentation method of the olfactory feedback based on the user's reaction speed data. For example, an appropriate pause is inserted between olfactory feedback to make it easy for the user to understand. In this way, the olfactory presentation speed can be optimized based on the user's reaction speed, thereby enhancing the effectiveness of olfactory information input.

[0087] The olfactory information presentation unit can use the emotion estimation function to analyze the user's emotions in response to olfactory information and generate olfactory feedback that elicits positive emotions. For example, the generation AI in the olfactory information presentation unit analyzes the user's facial expressions and reactions to estimate the user's emotions in response to the olfactory information. For example, it can emphasize olfactory feedback that the user is interested in. The olfactory information presentation unit also uses the emotion estimation function to generate feedback that causes the user to feel positive emotions in response to the olfactory information. For example, it can incorporate a scent that the user prefers. The generation AI in the olfactory information presentation unit also dynamically adjusts the olfactory feedback based on the user's emotional data. For example, it can provide olfactory feedback that helps the user relax. This can enhance the effectiveness of olfactory information input by optimizing the olfactory feedback based on the user's emotions.

[0088] The olfactory information presentation unit can combine olfactory information with VR technology to provide an olfactory learning experience in a virtual space. In the olfactory information presentation unit, for example, a generation AI recreates learning content in a virtual space using VR technology, allowing the user to have an olfactory learning experience. For example, the olfactory information presentation unit provides olfactory feedback for a lesson in a virtual classroom. The olfactory information presentation unit also allows the user to use a VR device to have an olfactory learning experience in a virtual space. For example, the olfactory information presentation unit provides an olfactory guide in a virtual museum. The olfactory information presentation unit also allows the generation AI to dynamically adjust the VR content according to the user's learning progress. For example, the olfactory placement in the virtual space can be changed to make it easier for the user to understand. In this way, by combining olfactory information with VR technology, an olfactory learning experience in a virtual space is possible.

[0089] The olfactory information presentation unit can combine olfactory information with AR technology to provide an olfactory experience that blends the real world and digital information. For example, the generation AI in the olfactory information presentation unit overlays learning content onto the real world using AR technology. For example, in a history class, historical events are recreated in actual locations using olfactory feedback. The olfactory information presentation unit also allows the user to use an AR device to provide an olfactory experience that blends the real world and digital information. For example, the growth process of a plant is displayed using AR and olfactory feedback is provided. The generation AI in the olfactory information presentation unit also dynamically adjusts the AR content according to the user's learning progress. For example, the olfactory placement of an AR object is changed to make it easier for the user to understand. In this way, by combining olfactory information with AR technology, an olfactory experience that blends the real world and digital information is possible.

[0090] The olfactory information presentation unit uses an emotion estimation function to monitor emotional changes in real time when sensing olfactory information and dynamically present optimal olfactory information. For example, the generation AI of the olfactory information presentation unit analyzes the user's facial expressions and reactions in real time and monitors emotional changes. For example, it emphasizes olfactory feedback that the user is interested in. The olfactory information presentation unit also uses the emotion estimation function to dynamically adjust the information so that the user feels positive emotions toward the olfactory information. For example, it incorporates a scent that the user prefers. The olfactory information presentation unit also dynamically presents olfactory information based on the user's emotional data using the generation AI. For example, it provides olfactory feedback that helps the user relax. This allows the effectiveness of olfactory information input to be enhanced by monitoring the user's emotional changes in real time and dynamically presenting optimal olfactory information.

[0091] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0092] The information input system can further include a learning style analysis unit that analyzes the user's learning style and suggests the most suitable learning method. For example, the learning style analysis unit analyzes the learning materials and learning methods used by the user in the past and suggests the most suitable learning method for the user. For example, if the user prefers visual learning, the visual information presentation unit can be enhanced. The learning style analysis unit can also monitor the user's learning progress and dynamically adjust the learning method. For example, if there is a part that the user finds difficult to understand, it can suggest a method to focus on studying that part. This can improve learning effectiveness by providing the most suitable learning method based on the user's learning style.

[0093] The information input system can further include a health monitoring unit that monitors the user's health condition and provides an optimal learning environment. For example, the health monitoring unit monitors the user's heart rate and stress level and adjusts the learning environment. For example, if the user is feeling stressed, it provides a relaxing environment. The health monitoring unit also optimizes study time and break times based on the user's health data. For example, if the user is tired, it sends a notification encouraging them to take a break. This allows for an optimal learning environment based on the user's health condition, thereby improving learning effectiveness.

[0094] The information input system may further include a learning goal setting unit that sets learning goals for the user and monitors the degree of achievement of those goals. For example, the learning goal setting unit creates a learning plan based on the learning goals set by the user. For example, if the user's goal is to pass a specific exam, the learning goal setting unit proposes a learning plan for that purpose. The learning goal setting unit also monitors the user's learning progress and evaluates the degree of goal achievement. For example, it displays in real time whether the user is approaching the goal. The learning goal setting unit also provides advice to help the user achieve the goal. For example, it suggests areas for improvement in learning methods. This makes it possible to improve learning effectiveness by providing an optimal learning plan based on the user's learning goals.

[0095] The information input system can further include a community formation unit that forms a learning community of users and promotes collaborative learning. For example, the community formation unit matches users with the same learning goals and provides opportunities for collaborative learning. For example, it may suggest online discussions or group projects. The community formation unit also promotes feedback between users to improve learning effectiveness. For example, it may provide a mechanism for users to evaluate the learning progress of other users. The community formation unit also proposes events and activities to maintain the user learning community. For example, it may hold regular online seminars or workshops. In this way, a learning community of users is formed, collaborative learning is promoted, and learning effectiveness can be improved.

[0096] The information input system can further include a data analysis unit that analyzes the user's learning data and visualizes the progress and effectiveness of learning. For example, the data analysis unit analyzes the user's learning history and test results and displays the learning progress in graphs and charts. For example, it visually shows in which areas the user has progressed. The data analysis unit also evaluates the user's learning effectiveness and suggests areas for improvement. For example, it identifies areas in which the user is weak and proposes a study plan that focuses on those areas. The data analysis unit also provides advice to maximize the effectiveness of learning based on the user's learning data. For example, it suggests optimizing study time and effective study methods. In this way, the learning effectiveness can be improved by analyzing the user's learning data and visualizing the progress and effectiveness of learning.

[0097] The information input system may further include a motivation maintenance unit that estimates the user's emotions and provides feedback to maintain motivation for learning. For example, the motivation maintenance unit may analyze the user's facial expressions and tone of voice to estimate emotions regarding learning. For example, if the user is tired, it may display an encouraging message. The motivation maintenance unit may also provide feedback to encourage the user to have positive emotions regarding learning. For example, it may provide a reward for the user's achieved goals. The motivation maintenance unit may also provide advice to maintain motivation for learning based on the user's emotional data. For example, it may make suggestions to adjust the pace of learning. This may improve learning effectiveness by maintaining motivation for learning based on the user's emotions.

[0098] The information input system may further include a relaxation providing unit that estimates the user's emotions and provides relaxation content to reduce learning stress. For example, the relaxation providing unit may analyze the user's facial expressions and heart rate to estimate the user's stress level related to learning. For example, if the user is feeling high stress, the relaxation providing unit may play relaxation music. The relaxation providing unit may also dynamically adjust content based on the emotional data to help the user relax. For example, the relaxation providing unit may suggest a relaxation method preferred by the user. The relaxation providing unit may also provide relaxation content between study sessions based on the user's emotional data. For example, the relaxation providing unit may suggest a short meditation session. This may improve learning effectiveness by reducing learning stress based on the user's emotions.

[0099] The information input system may further include a gamification providing unit that estimates a user's emotions and provides gamification elements to enhance the enjoyment of learning. For example, the gamification providing unit analyzes the user's facial expressions and tone of voice to estimate the user's emotions toward learning. For example, if the user is enjoying learning, points or badges may be provided. The gamification providing unit may also dynamically adjust game elements based on the emotion data so that the user feels positive about learning. For example, it may incorporate a game style preferred by the user. The gamification providing unit may also provide rewards according to the user's learning progress based on the emotion data. For example, a special reward may be provided when a specific goal is achieved. This may enhance the enjoyment of learning based on the user's emotions, thereby improving the learning effect.

[0100] The information input system may further include a concentration improvement unit that estimates the user's emotions and adjusts the environment to improve the user's concentration while studying. For example, the concentration improvement unit analyzes the user's facial expressions and gaze data to estimate the user's concentration while studying. For example, if the user is concentrating, notifications may be restricted. The concentration improvement unit may also dynamically adjust the environment based on the emotional data so that the user feels positive about studying. For example, it may play environmental sounds that the user prefers. The concentration improvement unit may also provide advice on improving concentration according to the user's progress while studying, based on the user's emotional data. For example, it may suggest taking a short break. This may improve the learning effect by improving the user's concentration while studying based on the user's emotions.

[0101] The information input system may further include a sense of accomplishment improvement unit that estimates the user's emotions and provides feedback to improve the user's sense of accomplishment in learning. For example, the sense of accomplishment improvement unit analyzes the user's facial expressions and tone of voice to estimate the user's emotions regarding learning. For example, if the user feels a sense of accomplishment, it displays words of praise. The sense of accomplishment improvement unit also dynamically adjusts the feedback based on the emotion data so that the user feels positive about learning. For example, it incorporates a feedback style preferred by the user. The sense of accomplishment improvement unit also provides advice to improve the user's sense of accomplishment in accordance with the user's learning progress based on the user's emotion data. For example, it suggests setting the next goal. This can improve the user's sense of accomplishment in learning based on the user's emotions, thereby improving the learning effect.

[0102] The processing flow of the second embodiment will be briefly explained below.

[0103] Step 1: The visual information presentation unit uses the generation AI to present visual information. The generation AI analyzes the visual information and presents it in the most appropriate form for the user. For example, it converts the learning content into graphs and charts that are easy to understand visually. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates visual learning materials based on that prompt. Step 2: The auditory information presentation unit uses the generation AI to present the auditory information. The generation AI analyzes the auditory information and presents it in the most appropriate form for the user. For example, it generates audio learning materials that explain the learning content aloud. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates the audio learning materials based on that prompt. Step 3: The tactile information presentation unit uses the generation AI to present tactile information. The generation AI analyzes the tactile information and presents it in the most appropriate form for the user. For example, it controls a device that provides tactile feedback that allows the user to understand the learning content tactilely. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates tactile feedback based on that prompt. Step 4: The taste information presentation unit uses the generation AI to present taste information. The generation AI analyzes the taste information and presents it in the most appropriate form for the user. For example, it controls a device that provides a taste experience related to the learning content. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates taste feedback based on the prompt. Step 5: The olfactory information presentation unit uses the generation AI to present olfactory information. The generation AI analyzes the olfactory information and presents it in the most optimal form for the user. For example, it controls a device that provides scents related to the learning content. The input to the generation AI is a prompt containing instructions on what the user wants to learn, and the generation AI generates olfactory feedback based on the prompt.

[0104] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0105] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0106] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0107] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0108] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0109] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0110] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0111] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0112] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0113] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0114] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0115] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0116] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0117] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0118] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0119] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0120] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0121] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0122] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0123] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0124] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0125] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0126] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0127] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0128] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0129] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0130] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0131] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0132] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 may also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0133] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0134] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0135] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0136] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0137] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0138] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0139] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0140] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0141] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0142] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0143] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0144] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0145] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0146] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0147] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0148] In the robot 414, the processor 46 performs the identification process. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 may have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59.

[0149] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0150] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0151] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0152] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0153] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0154] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0155] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0156] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0157] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0158] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0159] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0160] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0161] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0162] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0163] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0164] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0165] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0166] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0167] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0168] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0169] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0170] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0171] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a visual information presentation unit that presents visual information using a generation AI; an auditory information presentation unit that presents auditory information; a tactile information presentation unit that presents tactile information; a taste information presentation unit that presents taste information; an olfactory information presentation unit that presents olfactory information; A system characterized by:

2. The visual information presentation unit The visual information is presented as a 3D hologram, allowing users to understand the information in three dimensions.

2. The system of claim 1.

3. The auditory information presentation unit Learns auditory preferences and learning history to generate individually optimized audio learning materials 2. The system of claim 1.

4. The tactile information presentation unit Learns tactile preferences and learning history to generate individually optimized tactile feedback 2. The system of claim 1.

5. The taste information presentation unit Learns taste preferences and learning history to generate individually optimized taste feedback 2. The system of claim 1.

6. The olfactory information presentation unit Learns olfactory preferences and learning history to generate individually optimized olfactory feedback 2. The system of claim 1.

7. The visual information presentation unit Analyze the emotions felt in relation to the visual information and generate visual materials that evoke positive emotions.

2. The system of claim 1.

Images