system

Abstract

The system according to this embodiment aims to visually display the contents of a textbook in 3D images, thereby making it easier to attract students' interest in learning. [Solution] The system according to the embodiment comprises a generation unit and a display unit. The generation unit displays a story or diagram from a textbook as a 3D image. The display unit displays the 3D image generated by the generation unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, there is a problem that it is difficult to visually understand the content of textbooks and it is difficult to arouse the interest in learning.

[0005] The system according to the embodiment aims to visually display the content of textbooks in 3D videos and make it easier to arouse the interest in learning.

Means for Solving the Problems

[0006] The system according to the embodiment includes a generation unit and a display unit. The generation unit shows the stories or pictures in textbooks in 3D videos. The display unit displays the 3D videos generated by the generation unit.

Effects of the Invention

[0007] The system according to this embodiment can visually display the contents of a textbook in 3D images, making it easier to attract students' interest in learning. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage device 32. The processor 28, the RAM 30, and the storage device 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage device 50. The processor 46, the RAM 48, and the storage device 50 are connected to a bus 52. Also, the reception device 38, the output device 40, and the camera 42 are connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) An educational support system according to an embodiment of the present invention is a system that displays the contents of a textbook as 3D images. This system can display stories and diagrams from a textbook as 3D images. For example, when studying ancient castles in a history class, students can virtually walk through the castle or view paintings on the walls up close using 3D images. In chemistry experiments, students can see what happens by mixing and heating on the screen without actually using glass tubes or chemicals. Using this program makes it easier for teachers to explain and for children to learn with more interest. Even parts that are difficult to understand with just a regular textbook become easier to visualize when viewed in 3D images. For example, when studying ancient castles in a history class, students can virtually walk through the castle or view paintings on the walls up close using 3D images. In chemistry experiments, students can see what happens by mixing and heating on the screen without actually using glass tubes or chemicals. As a result, the educational support system can deepen learning understanding by displaying the contents of textbooks as 3D images.

[0029] The educational support system according to this embodiment comprises a generation unit and a display unit. The generation unit generates 3D images of stories and diagrams in a textbook. For example, the generation unit generates a 3D image of an ancient castle in a history lesson. The generation unit generates a 3D image of an ancient castle and displays it. This image is generated by the generation unit, which includes AI processing, and is displayed by the display unit. The generation unit generates 3D images of glass tubes and chemicals in a chemistry experiment. The generation unit generates a 3D image of glass tubes and chemicals and displays it. This image is also generated by the generation unit, which includes AI processing, and is displayed by the display unit. The display unit displays the 3D images generated by the generation unit. For example, the display unit displays a 3D image of an ancient castle generated by the generation unit. The display unit displays a 3D image of glass tubes and chemicals generated by the generation unit. As a result, the educational support system can deepen learning by displaying the content of textbooks as 3D images.

[0030] The generation unit creates 3D images from stories and diagrams in textbooks. Specifically, the generation unit analyzes the text and illustrations in the textbook and generates a 3D model based on that analysis. For example, when generating a 3D image of an ancient castle in a history lesson, the generation unit analyzes the structure and characteristics of the castle described in the textbook and creates a detailed 3D model based on that. This 3D model can include not only the exterior of the castle but also its interior structure and surrounding environment. The generation unit uses AI technology to automatically analyze the content of the textbook, extract the necessary information, and generate a 3D model. For example, it uses natural language processing technology to analyze the text of the textbook and extract important keywords and phrases. It also uses image recognition technology to analyze the illustrations in the textbook and reflect the content of the illustrations in the 3D model. Furthermore, when generating 3D images of glass tubes and chemicals in a chemistry experiment, the generation unit analyzes the experimental procedure and the structure of the apparatus described in the textbook and creates a detailed 3D model based on that analysis. For example, it can realistically reproduce the shape of the glass tube, the color of the chemicals, and the reaction process. The generation unit uses AI technology to automatically analyze the content of textbooks, extract necessary information, and generate 3D models. This allows the generation unit to visually represent the textbook content as 3D images, deepening learners' understanding.

[0031] The display unit displays 3D images generated by the generation unit. Specifically, the display unit uses a high-resolution display or projector to display the generated 3D images in real time. For example, the display unit can display a 3D image of an ancient castle generated by the generation unit, allowing learners to observe the castle's exterior and interior structure in detail. The display unit also has the functionality to interactively manipulate the 3D images, allowing learners to rotate the images and zoom in and out. This enables learners to gain a deeper understanding of the textbook content. The display unit can also display 3D images of glass tubes and chemicals generated by the generation unit, allowing learners to visually understand the experimental procedures and reaction processes. The display unit also has the functionality to update the 3D images in real time, allowing the images to change according to the progress of the experiment. For example, it can display in real time the color change of chemicals as they react or the flow of liquid inside the glass tube. This allows learners to visually understand the experimental process and gain a deeper understanding of the experimental principles and results. Furthermore, the display unit can utilize a wide-viewing-angle display or projector, allowing multiple learners to simultaneously view 3D images. This supports learning activities throughout the classroom and promotes cooperation and exchange of ideas among learners.

[0032] The generation unit can generate 3D images of ancient castles for use in history lessons. For example, the generation unit can generate 3D images of ancient castles. The generation unit can target castles from specific eras and regions, such as medieval European castles or castles from Japan's Sengoku period. This allows for a deeper understanding of the subject matter by generating 3D images of ancient castles in history lessons. Some or all of the processing described above in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can use an AI model to recreate the structure and decorations of ancient castles in order to generate 3D images of them.

[0033] The display unit can display a 3D image of an ancient castle generated by the generation unit. The display unit can, for example, display a 3D image of an ancient castle generated by the generation unit. The display unit can, for example, display the 3D image of the ancient castle in high resolution, allowing the user to walk inside the castle or view the paintings on the walls up close. This deepens the learning understanding by displaying the 3D image of the ancient castle. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can use an AI model to improve the quality of the 3D image generated by the generation unit.

[0034] The generation unit can generate 3D images of glass tubes and chemicals in chemistry experiments. For example, the generation unit can generate 3D images of glass tubes and chemicals. The generation unit can target specific glass tubes and chemicals, such as test tubes, beakers, and specific chemicals. This allows for a deeper understanding of learning by generating 3D images of glass tubes and chemicals in chemistry experiments. Some or all of the above-described processes in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can use an AI model to reproduce chemical reactions and experimental procedures in order to generate 3D images of glass tubes and chemicals.

[0035] The display unit can display 3D images of glass tubes and chemicals generated by the generation unit. The display unit can, for example, display 3D images of glass tubes and chemicals generated by the generation unit. The display unit can, for example, display 3D images of glass tubes and chemicals in high resolution, allowing the user to simulate the experiment. This deepens the understanding of the learning by displaying 3D images of glass tubes and chemicals. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can use an AI model to improve the quality of the 3D images generated by the generation unit.

[0036] The generation unit can generate 3D images tailored to specific learning objectives based on the content of a textbook. For example, the generation unit can generate 3D images tailored to specific learning objectives based on the content of a textbook. For example, in a history class, the generation unit can generate 3D images focusing on a specific era or event. For example, in a chemistry class, the generation unit can generate 3D images based on a specific chemical reaction or experimental procedure. For example, in a biology class, the generation unit can generate 3D images related to the ecology or evolution of a specific organism. By generating 3D images tailored to specific learning objectives, the effectiveness of learning is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of a textbook into a generation AI and have the generation AI perform the generation of 3D images tailored to learning objectives.

[0037] The generation unit can add interactive elements to the 3D images it generates, allowing users to interact with them. For example, the generation unit can add interactive elements to the 3D images it generates, allowing users to interact with them. For example, the generation unit can allow users to control a character in the 3D image and explore specific locations. For example, the generation unit can allow users to click on objects in the 3D image to display detailed information. For example, the generation unit can allow users to simulate experiments in the 3D image and see the results. This improves the learning effect by adding interactive elements. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input interactive elements into a generation AI and have the generation AI generate user-operable 3D images.

[0038] The generation unit can generate 3D videos that correspond to different grade levels and learning levels based on the content of textbooks. For example, the generation unit can generate 3D videos that correspond to different grade levels and learning levels based on the content of textbooks. For example, the generation unit can generate simple and visually easy-to-understand 3D videos for elementary school students. For example, the generation unit can generate 3D videos that include detailed explanations and annotations for junior high school students. For example, the generation unit can generate 3D videos that include specialized knowledge and theories for high school students. By generating 3D videos that correspond to different grade levels and learning levels, the effectiveness of learning is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of textbooks into a generation AI and have the generation AI perform the generation of 3D videos that correspond to grade levels and learning levels.

[0039] The generation unit can add audio guides and subtitles to the generated 3D images, supporting learning through both sight and hearing. For example, the generation unit can add audio guides and subtitles to the generated 3D images, supporting learning through both sight and hearing. For example, the generation unit can add audio guides to 3D images, providing information through both sight and hearing. For example, the generation unit can add subtitles to 3D images, accommodating users with hearing impairments. For example, the generation unit can add multilingual audio guides and subtitles to 3D images, accommodating users who speak different languages. This allows for learning to be supported through both sight and hearing by adding audio guides and subtitles. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the audio guides and subtitles into a generation AI, causing the generation AI to generate the audio guides and subtitles.

[0040] The display unit can have the function of adding annotations and supplementary information to the displayed 3D image in real time. For example, the display unit can have the function of adding annotations and supplementary information to the displayed 3D image in real time. For example, the display unit can display annotations in real time for specific objects in the image. For example, the display unit can display supplementary information in real time for specific scenes in the image. For example, the display unit can add annotations and supplementary information in real time in response to user operations. This improves the learning effect by adding annotations and supplementary information in real time. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the content of annotations and supplementary information into a generating AI and have the generating AI perform the addition of annotations and supplementary information in real time.

[0041] The display unit can add customization functions to the 3D images it displays based on the user's operation history. For example, the display unit can add customization functions to the 3D images it displays based on the user's operation history. For example, the display unit can customize the way the images are displayed based on what the user has done in the past. For example, the display unit can prioritize displaying content of interest to the user based on their operation history. For example, the display unit can analyze the user's operation history and suggest the optimal display method. This improves the learning effect by customizing based on the user's operation history. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user operation history data into a generating AI and have the generating AI execute the customization functions.

[0042] The display unit can be equipped with multi-user functionality that allows multiple users to access the displayed 3D image simultaneously. For example, the display unit can be equipped with multi-user functionality that allows multiple users to access the displayed 3D image simultaneously. For example, the display unit can enable multiple users to view and collaboratively manipulate the 3D image simultaneously. For example, the display unit can enable multiple users to view the 3D image simultaneously and individually add annotations or supplementary information. For example, the display unit can enable multiple users to view the 3D image simultaneously and communicate in real time. This enables collaborative learning through simultaneous access by multiple users. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can have a generating AI perform the execution of the multi-user functionality.

[0043] The display unit can display progress bars and achievement meters that reflect the user's learning progress on the displayed 3D image. For example, the display unit can display progress bars and achievement meters that reflect the user's learning progress on the displayed 3D image. For example, the display unit can display progress bars according to the user's learning progress. For example, the display unit can display achievement meters according to the user's achievement level. For example, the display unit can update and display the user's learning progress and achievement level in real time. This improves the effectiveness of learning by displaying learning progress and achievement level. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user learning progress data into a generating AI and have the generating AI execute the display of progress bars and achievement meters.

[0044] The generation unit can add historical background and related episodes to 3D images of ancient castles. For example, the generation unit can add historical background and related episodes to 3D images of ancient castles. For example, the generation unit can add the construction process and historical background of ancient castles to 3D images. For example, the generation unit can add historical episodes and people related to ancient castles to 3D images. For example, the generation unit can add the changes and restoration process of ancient castles to 3D images. This improves the learning effect by adding historical background and episodes. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of historical background and episodes into a generation AI and have the generation AI perform the addition to the 3D images.

[0045] The generation unit can add interactive elements that users can explore to a 3D image of an ancient castle. For example, the generation unit can add interactive elements that users can explore to a 3D image of an ancient castle. For example, the generation unit can allow users to freely walk around inside the ancient castle. For example, the generation unit can allow users to click on specific locations in the ancient castle to display detailed information. For example, the generation unit can allow users to manipulate specific objects inside the ancient castle. By adding interactive elements, the learning effect is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the interactive elements into a generation AI and have the generation AI add them to the 3D image.

[0046] The generation unit can add a comparison function to the 3D image of the ancient castle with other historical buildings. For example, the generation unit can add a comparison function to the 3D image of the ancient castle with other historical buildings. For example, the generation unit can display the ancient castle and other historical buildings side by side so that they can be compared. For example, the generation unit can compare a specific part of the ancient castle with the same part of another building. For example, the generation unit can compare the architectural style and technology of the ancient castle with other buildings. By adding a comparison function with other historical buildings, the effectiveness of learning is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the comparison function into the generation AI and have the generation AI perform the addition to the 3D image.

[0047] The generation unit can add an interactive guide function to the 3D image of the ancient castle, allowing the user to ask questions. For example, the generation unit can add an interactive guide function to the 3D image of the ancient castle, allowing the user to ask questions. For example, the generation unit can allow the user to input questions about the ancient castle and receive answers in real time. For example, the generation unit can allow the user to ask questions about specific parts of the ancient castle and display detailed information. For example, the generation unit can allow the user to ask questions about the historical background of the ancient castle and display information about relevant episodes and people. This improves the learning effect by adding an interactive guide function. Some or all of the above processing in the generation unit may be performed using AI, for example, or not using AI. For example, the generation unit can input the content of the guide function into a generation AI and have the generation AI add it to the 3D image.

[0048] The display unit can have the function of adding annotations and supplementary information to 3D images of ancient castles in real time. For example, the display unit can have the function of adding annotations and supplementary information to 3D images of ancient castles in real time. For example, the display unit can display annotations in real time for specific objects in the image. For example, the display unit can display supplementary information in real time for specific scenes in the image. For example, the display unit can add annotations and supplementary information in real time in response to user operations. This improves the learning effect by adding annotations and supplementary information in real time. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the content of annotations and supplementary information into a generating AI and have the generating AI perform the addition of annotations and supplementary information in real time.

[0049] The display unit can add customization features to the 3D image of the ancient castle based on the user's operation history. For example, the display unit can add customization features to the 3D image of the ancient castle based on the user's operation history. For example, the display unit can customize the way the image is displayed based on what the user has done in the past. For example, the display unit can prioritize displaying content of interest based on the user's operation history. For example, the display unit can analyze the user's operation history and suggest the optimal display method. This improves the learning effect by customizing based on the user's operation history. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user operation history data into a generating AI and have the generating AI execute the customization features.

[0050] The display unit can be equipped with multi-user functionality that allows multiple users to access the 3D image of the ancient castle simultaneously. For example, the display unit can enable multiple users to view and collaboratively manipulate the 3D image simultaneously. For example, the display unit can enable multiple users to view the 3D image simultaneously and individually add annotations and supplementary information. For example, the display unit can enable multiple users to view the 3D image simultaneously and communicate in real time. This allows for collaborative learning through simultaneous access by multiple users. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can have a generative AI perform the execution of the multi-user functionality.

[0051] The display unit can display a progress bar and achievement meter that reflect the user's learning progress on a 3D image of an ancient castle. For example, the display unit can display a progress bar and achievement meter that reflect the user's learning progress on a 3D image of an ancient castle. For example, the display unit can display a progress bar according to the user's learning progress. For example, the display unit can display an achievement meter according to the user's achievement level. For example, the display unit can update and display the user's learning progress and achievement level in real time. This improves the effectiveness of learning by displaying learning progress and achievement level. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the user's learning progress data into a generating AI and have the generating AI execute the display of the progress bar and achievement meter.

[0052] The generation unit can add the theoretical background and related knowledge of a chemical experiment to a 3D video of the experiment. For example, the generation unit can add the theoretical background and related knowledge of a chemical experiment to a 3D video of a chemical experiment. For example, the generation unit can add the theoretical background of a chemical reaction to a 3D video. For example, the generation unit can add chemical knowledge and information related to the experiment to a 3D video. For example, the generation unit can add the experimental procedure and precautions to a 3D video. By adding the theoretical background and knowledge, the effectiveness of learning is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the theoretical background and knowledge into a generation AI and have the generation AI perform the addition to the 3D video.

[0053] The generation unit can add user-operable interactive elements to 3D images of chemical experiments. For example, the generation unit can add user-operable interactive elements to 3D images of chemical experiments. For example, the generation unit can enable users to operate experimental equipment within the 3D image and simulate the experiment. For example, the generation unit can enable users to mix or heat chemicals within the 3D image to observe the reaction. For example, the generation unit can enable users to analyze experimental results within the 3D image and acquire data. By adding interactive elements, the learning effect is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the interactive elements into a generation AI and have the generation AI perform the addition to the 3D image.

[0054] The generation unit can add a comparison function to 3D images of chemical experiments. For example, the generation unit can add a comparison function to 3D images of chemical experiments. For example, the generation unit can make it possible to compare the results of the same chemical reaction performed under different conditions. For example, the generation unit can display different chemical reactions side by side and make them comparable. For example, the generation unit can make it possible to compare the procedures and results of an experiment with those of other experiments. By adding a comparison function with other experiments, the effectiveness of learning is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the comparison function into a generation AI and have the generation AI perform the addition to the 3D images.

[0055] The generation unit can add an interactive guide function to 3D video of a chemistry experiment, allowing the user to ask questions. For example, the generation unit can add an interactive guide function to 3D video of a chemistry experiment, allowing the user to ask questions. For example, the generation unit can enable the user to input questions about the chemistry experiment and receive answers in real time. For example, the generation unit can enable the user to ask questions about specific parts of the chemistry experiment and display detailed information. For example, the generation unit can enable the user to ask questions about the theoretical background of the chemistry experiment and display relevant knowledge and information. By adding this interactive guide function, the learning effect is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the guide function into a generation AI and have the generation AI perform the addition to the 3D video.

[0056] The display unit can have the function of adding annotations and supplementary information to 3D images of chemical experiments in real time. For example, the display unit can have the function of adding annotations and supplementary information to 3D images of chemical experiments in real time. For example, the display unit can display annotations in real time for specific objects in the image. For example, the display unit can display supplementary information in real time for specific scenes in the image. For example, the display unit can add annotations and supplementary information in real time in response to user operations. This improves the effectiveness of learning by adding annotations and supplementary information in real time. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the content of annotations and supplementary information into a generating AI and have the generating AI perform the addition of annotations and supplementary information in real time.

[0057] The display unit can add customization functions to 3D images of chemical experiments based on the user's operation history. For example, the display unit can add customization functions to 3D images of chemical experiments based on the user's operation history. For example, the display unit can customize the display method of the images based on what the user has done in the past. For example, the display unit can prioritize displaying content of interest based on the user's operation history. For example, the display unit can analyze the user's operation history and suggest the optimal display method. This improves the learning effect by customizing based on the user's operation history. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user operation history data into a generating AI and have the generating AI execute the customization functions.

[0058] The display unit can be equipped with multi-user functionality that allows multiple users to access 3D images of chemical experiments simultaneously. For example, the display unit can enable multiple users to view and collaboratively manipulate 3D images simultaneously. For example, it can enable multiple users to view 3D images simultaneously and add annotations or supplementary information individually. For example, it can enable multiple users to view 3D images simultaneously and communicate in real time. This allows for collaborative learning through simultaneous access by multiple users. Some or all of the above-described processes in the display unit may be performed using AI, for example, or without AI. For example, the display unit can have a generative AI perform the execution of the multi-user functionality.

[0059] The display unit can display progress bars and achievement meters that reflect the user's learning progress on a 3D video of a chemistry experiment. For example, the display unit can display progress bars and achievement meters that reflect the user's learning progress on a 3D video of a chemistry experiment. For example, the display unit can display progress bars according to the user's learning progress. For example, the display unit can display achievement meters according to the user's achievement level. For example, the display unit can update and display the user's learning progress and achievement level in real time. This improves the effectiveness of learning by displaying learning progress and achievement level. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the user's learning progress data into a generating AI and have the generating AI display progress bars and achievement meters.

[0060] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0061] The educational support system can also be equipped with a voice recognition unit. The voice recognition unit can recognize the user's voice commands and send instructions to the generation and display units. For example, if the user says, "Show me the inside of the castle," the generation unit can generate a 3D image of the castle's interior based on that command, and the display unit can display that image. Also, if the user says, "Start the next experiment," the generation unit can generate a 3D image of the next chemistry experiment, and the display unit can display that image. As a result, by using voice recognition, users can operate the system more intuitively, improving the effectiveness of their learning.

[0062] The educational support system can also include a feedback unit. This feedback unit can evaluate the user's learning progress and understanding, and provide appropriate feedback. For example, after a user views a 3D image of an ancient castle in a history lesson, the feedback unit can present the user with a quiz and evaluate their understanding based on their answers. Similarly, after a user manipulates a 3D image in a chemistry experiment, the feedback unit can provide feedback on the experiment's procedure and results. This allows for improved learning effectiveness through feedback.

[0063] The educational support system can also include a data analysis unit. This unit can collect and analyze user learning data to evaluate learning trends and effectiveness. For example, it can collect and analyze data such as which 3D images users watched, for how long, and where they struggled. Furthermore, it can compare data from multiple users to identify common challenges and effective learning methods. This enables data-driven optimization of learning.

[0064] The educational support system can also include a customization section. This customization section allows users to customize the content and display method of 3D images according to their learning style and preferences. For example, if a user prefers visual learning, the customization section can add numerous visual effects to the images. If a user prefers detailed information, annotations and supplementary information can be added to the images. This enables learning tailored to the individual needs of each user.

[0065] The educational support system can also include a collaborative learning section. This section allows multiple users to simultaneously view 3D images and engage in collaborative learning activities. For example, multiple users can simultaneously view a 3D image of an ancient castle while exchanging opinions via chat or voice calls. Users can also collaboratively manipulate a 3D image of a chemistry experiment and share the results. This promotes interaction among users through collaborative learning, improving the effectiveness of the learning process.

[0066] The following briefly describes the processing flow for example form 1.

[0067] Step 1: The generation unit generates 3D images from stories and diagrams in textbooks. For example, the generation unit generates 3D images of ancient castles in history class and 3D images of glass tubes and chemicals in chemistry experiments. These images are generated by the generation unit, which includes AI processing. Step 2: The display unit displays the 3D images generated by the generation unit. For example, the display unit displays 3D images of an ancient castle, a glass tube, or chemicals, all generated by the generation unit.

[0068] (Example of form 2) An educational support system according to an embodiment of the present invention is a system that displays the contents of a textbook as 3D images. This system can display stories and diagrams from a textbook as 3D images. For example, when studying ancient castles in a history class, students can virtually walk through the castle or view paintings on the walls up close using 3D images. In chemistry experiments, students can see what happens by mixing and heating on the screen without actually using glass tubes or chemicals. Using this program makes it easier for teachers to explain and for children to learn with more interest. Even parts that are difficult to understand with just a regular textbook become easier to visualize when viewed in 3D images. For example, when studying ancient castles in a history class, students can virtually walk through the castle or view paintings on the walls up close using 3D images. In chemistry experiments, students can see what happens by mixing and heating on the screen without actually using glass tubes or chemicals. As a result, the educational support system can deepen learning understanding by displaying the contents of textbooks as 3D images.

[0069] The educational support system according to this embodiment comprises a generation unit and a display unit. The generation unit generates 3D images of stories and diagrams in a textbook. For example, the generation unit generates a 3D image of an ancient castle in a history lesson. The generation unit generates a 3D image of an ancient castle and displays it. This image is generated by the generation unit, which includes AI processing, and is displayed by the display unit. The generation unit generates 3D images of glass tubes and chemicals in a chemistry experiment. The generation unit generates a 3D image of glass tubes and chemicals and displays it. This image is also generated by the generation unit, which includes AI processing, and is displayed by the display unit. The display unit displays the 3D images generated by the generation unit. For example, the display unit displays a 3D image of an ancient castle generated by the generation unit. The display unit displays a 3D image of glass tubes and chemicals generated by the generation unit. As a result, the educational support system can deepen learning by displaying the content of textbooks as 3D images.

[0070] The generation unit creates 3D images from stories and diagrams in textbooks. Specifically, the generation unit analyzes the text and illustrations in the textbook and generates a 3D model based on that analysis. For example, when generating a 3D image of an ancient castle in a history lesson, the generation unit analyzes the structure and characteristics of the castle described in the textbook and creates a detailed 3D model based on that. This 3D model can include not only the exterior of the castle but also its interior structure and surrounding environment. The generation unit uses AI technology to automatically analyze the content of the textbook, extract the necessary information, and generate a 3D model. For example, it uses natural language processing technology to analyze the text of the textbook and extract important keywords and phrases. It also uses image recognition technology to analyze the illustrations in the textbook and reflect the content of the illustrations in the 3D model. Furthermore, when generating 3D images of glass tubes and chemicals in a chemistry experiment, the generation unit analyzes the experimental procedure and the structure of the apparatus described in the textbook and creates a detailed 3D model based on that analysis. For example, it can realistically reproduce the shape of the glass tube, the color of the chemicals, and the reaction process. The generation unit uses AI technology to automatically analyze the content of textbooks, extract necessary information, and generate 3D models. This allows the generation unit to visually represent the textbook content as 3D images, deepening learners' understanding.

[0071] The display unit displays 3D images generated by the generation unit. Specifically, the display unit uses a high-resolution display or projector to display the generated 3D images in real time. For example, the display unit can display a 3D image of an ancient castle generated by the generation unit, allowing learners to observe the castle's exterior and interior structure in detail. The display unit also has the functionality to interactively manipulate the 3D images, allowing learners to rotate the images and zoom in and out. This enables learners to gain a deeper understanding of the textbook content. The display unit can also display 3D images of glass tubes and chemicals generated by the generation unit, allowing learners to visually understand the experimental procedures and reaction processes. The display unit also has the functionality to update the 3D images in real time, allowing the images to change according to the progress of the experiment. For example, it can display in real time the color change of chemicals as they react or the flow of liquid inside the glass tube. This allows learners to visually understand the experimental process and gain a deeper understanding of the experimental principles and results. Furthermore, the display unit can utilize a wide-viewing-angle display or projector, allowing multiple learners to simultaneously view 3D images. This supports learning activities throughout the classroom and promotes cooperation and exchange of ideas among learners.

[0072] The generation unit can generate 3D images of ancient castles for use in history lessons. For example, the generation unit can generate 3D images of ancient castles. The generation unit can target castles from specific eras and regions, such as medieval European castles or castles from Japan's Sengoku period. This allows for a deeper understanding of the subject matter by generating 3D images of ancient castles in history lessons. Some or all of the processing described above in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can use an AI model to recreate the structure and decorations of ancient castles in order to generate 3D images of them.

[0073] The display unit can display a 3D image of an ancient castle generated by the generation unit. The display unit can, for example, display a 3D image of an ancient castle generated by the generation unit. The display unit can, for example, display the 3D image of the ancient castle in high resolution, allowing the user to walk inside the castle or view the paintings on the walls up close. This deepens the learning understanding by displaying the 3D image of the ancient castle. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can use an AI model to improve the quality of the 3D image generated by the generation unit.

[0074] The generation unit can generate 3D images of glass tubes and chemicals in chemistry experiments. For example, the generation unit can generate 3D images of glass tubes and chemicals. The generation unit can target specific glass tubes and chemicals, such as test tubes, beakers, and specific chemicals. This allows for a deeper understanding of learning by generating 3D images of glass tubes and chemicals in chemistry experiments. Some or all of the above-described processes in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can use an AI model to reproduce chemical reactions and experimental procedures in order to generate 3D images of glass tubes and chemicals.

[0075] The display unit can display 3D images of glass tubes and chemicals generated by the generation unit. The display unit can, for example, display 3D images of glass tubes and chemicals generated by the generation unit. The display unit can, for example, display 3D images of glass tubes and chemicals in high resolution, allowing the user to simulate the experiment. This deepens the understanding of the learning by displaying 3D images of glass tubes and chemicals. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can use an AI model to improve the quality of the 3D images generated by the generation unit.

[0076] The generation unit can estimate the user's emotions and adjust the content of the generated 3D video based on the estimated user emotions. For example, the generation unit estimates the user's emotions and adjusts the content of the generated 3D video based on the estimated user emotions. For example, if the user is excited, the generation unit adds dynamic scenes or effects to the video. For example, if the user is relaxed, the generation unit adds calm scenes or tones to the video. For example, if the user is focused, the generation unit adds detailed information or annotations to the video. This improves the learning effect by adjusting the content of the 3D video based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the generation unit may be performed using AI, for example, or not using AI. For example, the generation unit can input user emotion data into the generation AI and have the generation AI perform emotion-based video adjustments.

[0077] The generation unit can generate 3D images tailored to specific learning objectives based on the content of a textbook. For example, the generation unit can generate 3D images tailored to specific learning objectives based on the content of a textbook. For example, in a history class, the generation unit can generate 3D images focusing on a specific era or event. For example, in a chemistry class, the generation unit can generate 3D images based on a specific chemical reaction or experimental procedure. For example, in a biology class, the generation unit can generate 3D images related to the ecology or evolution of a specific organism. By generating 3D images tailored to specific learning objectives, the effectiveness of learning is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of a textbook into a generation AI and have the generation AI perform the generation of 3D images tailored to learning objectives.

[0078] The generation unit can add interactive elements to the 3D images it generates, allowing users to interact with them. For example, the generation unit can add interactive elements to the 3D images it generates, allowing users to interact with them. For example, the generation unit can allow users to control a character in the 3D image and explore specific locations. For example, the generation unit can allow users to click on objects in the 3D image to display detailed information. For example, the generation unit can allow users to simulate experiments in the 3D image and see the results. This improves the learning effect by adding interactive elements. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input interactive elements into a generation AI and have the generation AI generate user-operable 3D images.

[0079] The generation unit can estimate the user's emotions and adjust the difficulty level of the 3D video it generates based on the estimated emotions. For example, the generation unit estimates the user's emotions and adjusts the difficulty level of the 3D video it generates based on the estimated emotions. For example, if the user is stressed, the generation unit generates a simple 3D video. For example, if the user is relaxed, the generation unit generates a detailed and complex 3D video. For example, if the user is focused, the generation unit generates a challenging 3D video. This improves the learning effect by adjusting the difficulty level of the 3D video based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input user emotion data into the generation AI and have the generation AI perform emotion-based difficulty level adjustments for the video.

[0080] The generation unit can generate 3D videos that correspond to different grade levels and learning levels based on the content of textbooks. For example, the generation unit can generate 3D videos that correspond to different grade levels and learning levels based on the content of textbooks. For example, the generation unit can generate simple and visually easy-to-understand 3D videos for elementary school students. For example, the generation unit can generate 3D videos that include detailed explanations and annotations for junior high school students. For example, the generation unit can generate 3D videos that include specialized knowledge and theories for high school students. By generating 3D videos that correspond to different grade levels and learning levels, the effectiveness of learning is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of textbooks into a generation AI and have the generation AI perform the generation of 3D videos that correspond to grade levels and learning levels.

[0081] The generation unit can add audio guides and subtitles to the generated 3D images, supporting learning through both sight and hearing. For example, the generation unit can add audio guides and subtitles to the generated 3D images, supporting learning through both sight and hearing. For example, the generation unit can add audio guides to 3D images, providing information through both sight and hearing. For example, the generation unit can add subtitles to 3D images, accommodating users with hearing impairments. For example, the generation unit can add multilingual audio guides and subtitles to 3D images, accommodating users who speak different languages. This allows for learning to be supported through both sight and hearing by adding audio guides and subtitles. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the audio guides and subtitles into a generation AI, causing the generation AI to generate the audio guides and subtitles.

[0082] The display unit can estimate the user's emotions and adjust the viewpoint and angle of the 3D image displayed based on the estimated user emotions. For example, the display unit can estimate the user's emotions and adjust the viewpoint and angle of the 3D image displayed based on the estimated user emotions. For example, if the user is excited, the display unit can display the image with a dynamic viewpoint and angle. For example, if the user is relaxed, the display unit can display the image with a calm viewpoint and angle. For example, if the user is focused, the display unit can display the image with a detailed viewpoint and angle. This improves the learning effect by adjusting the viewpoint and angle based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user emotion data into the generative AI and have the generative AI perform the viewpoint and angle adjustment based on the emotions.

[0083] The display unit can have the function of adding annotations and supplementary information to the displayed 3D image in real time. For example, the display unit can have the function of adding annotations and supplementary information to the displayed 3D image in real time. For example, the display unit can display annotations in real time for specific objects in the image. For example, the display unit can display supplementary information in real time for specific scenes in the image. For example, the display unit can add annotations and supplementary information in real time in response to user operations. This improves the learning effect by adding annotations and supplementary information in real time. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the content of annotations and supplementary information into a generating AI and have the generating AI perform the addition of annotations and supplementary information in real time.

[0084] The display unit can add customization functions to the 3D images it displays based on the user's operation history. For example, the display unit can add customization functions to the 3D images it displays based on the user's operation history. For example, the display unit can customize the way the images are displayed based on what the user has done in the past. For example, the display unit can prioritize displaying content of interest to the user based on their operation history. For example, the display unit can analyze the user's operation history and suggest the optimal display method. This improves the learning effect by customizing based on the user's operation history. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user operation history data into a generating AI and have the generating AI execute the customization functions.

[0085] The display unit can estimate the user's emotions and adjust the color tone and brightness of the 3D image displayed based on the estimated emotions. For example, the display unit can estimate the user's emotions and adjust the color tone and brightness of the 3D image displayed based on the estimated emotions. For example, if the user is excited, the display unit can display the image with vivid colors and brightness. For example, if the user is relaxed, the display unit can display the image with calm colors and brightness. For example, if the user is focused, the display unit can display the image with highly visible colors and brightness. By adjusting the color tone and brightness based on the user's emotions, the learning effect is improved. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user emotion data into a generating AI, which can then perform adjustments to the color tone and brightness based on those emotions.

[0086] The display unit can be equipped with multi-user functionality that allows multiple users to access the displayed 3D image simultaneously. For example, the display unit can be equipped with multi-user functionality that allows multiple users to access the displayed 3D image simultaneously. For example, the display unit can enable multiple users to view and collaboratively manipulate the 3D image simultaneously. For example, the display unit can enable multiple users to view the 3D image simultaneously and individually add annotations or supplementary information. For example, the display unit can enable multiple users to view the 3D image simultaneously and communicate in real time. This enables collaborative learning through simultaneous access by multiple users. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can have a generating AI perform the execution of the multi-user functionality.

[0087] The display unit can display progress bars and achievement meters that reflect the user's learning progress on the displayed 3D image. For example, the display unit can display progress bars and achievement meters that reflect the user's learning progress on the displayed 3D image. For example, the display unit can display progress bars according to the user's learning progress. For example, the display unit can display achievement meters according to the user's achievement level. For example, the display unit can update and display the user's learning progress and achievement level in real time. This improves the effectiveness of learning by displaying learning progress and achievement level. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user learning progress data into a generating AI and have the generating AI execute the display of progress bars and achievement meters.

[0088] The generation unit can estimate the user's emotions and adjust the level of detail of the 3D image of the ancient castle based on the estimated emotions. For example, the generation unit can estimate the user's emotions and adjust the level of detail of the 3D image of the ancient castle based on the estimated emotions. For example, if the user is excited, the generation unit can add detailed decorations and effects. For example, if the user is relaxed, the generation unit can use calm colors and a simple design. For example, if the user is focused, the generation unit can add historical annotations and detailed information. This improves the learning effect by adjusting the level of detail based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the generation unit may be performed using AI, for example, or not using AI. For example, the generation unit can input user emotion data into the generative AI and have the generative AI perform the emotion-based level of detail adjustment.

[0089] The generation unit can add historical background and related episodes to 3D images of ancient castles. For example, the generation unit can add historical background and related episodes to 3D images of ancient castles. For example, the generation unit can add the construction process and historical background of ancient castles to 3D images. For example, the generation unit can add historical episodes and people related to ancient castles to 3D images. For example, the generation unit can add the changes and restoration process of ancient castles to 3D images. This improves the learning effect by adding historical background and episodes. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of historical background and episodes into a generation AI and have the generation AI perform the addition to the 3D images.

[0090] The generation unit can add interactive elements that users can explore to a 3D image of an ancient castle. For example, the generation unit can add interactive elements that users can explore to a 3D image of an ancient castle. For example, the generation unit can allow users to freely walk around inside the ancient castle. For example, the generation unit can allow users to click on specific locations in the ancient castle to display detailed information. For example, the generation unit can allow users to manipulate specific objects inside the ancient castle. By adding interactive elements, the learning effect is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the interactive elements into a generation AI and have the generation AI add them to the 3D image.

[0091] The generation unit can estimate the user's emotions and adjust the viewpoint of the 3D image of the ancient castle based on the estimated user emotions. For example, the generation unit estimates the user's emotions and adjusts the viewpoint of the 3D image of the ancient castle based on the estimated user emotions. For example, if the user is excited, the generation unit displays the image with a dynamic viewpoint. For example, if the user is relaxed, the generation unit displays the image with a calm viewpoint. For example, if the user is focused, the generation unit displays the image with a detailed viewpoint. This improves the learning effect by adjusting the viewpoint based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input user emotion data into the generative AI and have the generative AI perform the emotion-based viewpoint adjustment.

[0092] The generation unit can add a comparison function to the 3D image of the ancient castle with other historical buildings. For example, the generation unit can add a comparison function to the 3D image of the ancient castle with other historical buildings. For example, the generation unit can display the ancient castle and other historical buildings side by side so that they can be compared. For example, the generation unit can compare a specific part of the ancient castle with the same part of another building. For example, the generation unit can compare the architectural style and technology of the ancient castle with other buildings. By adding a comparison function with other historical buildings, the effectiveness of learning is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the comparison function into the generation AI and have the generation AI perform the addition to the 3D image.

[0093] The generation unit can add an interactive guide function to the 3D image of the ancient castle, allowing the user to ask questions. For example, the generation unit can add an interactive guide function to the 3D image of the ancient castle, allowing the user to ask questions. For example, the generation unit can allow the user to input questions about the ancient castle and receive answers in real time. For example, the generation unit can allow the user to ask questions about specific parts of the ancient castle and display detailed information. For example, the generation unit can allow the user to ask questions about the historical background of the ancient castle and display information about relevant episodes and people. This improves the learning effect by adding an interactive guide function. Some or all of the above processing in the generation unit may be performed using AI, for example, or not using AI. For example, the generation unit can input the content of the guide function into a generation AI and have the generation AI add it to the 3D image.

[0094] The display unit can estimate the user's emotions and adjust the display method of the 3D image of the ancient castle based on the estimated user emotions. For example, the display unit can estimate the user's emotions and adjust the display method of the 3D image of the ancient castle based on the estimated user emotions. For example, if the user is excited, the display unit can display with dynamic viewpoints and effects. For example, if the user is relaxed, the display unit can display with calm colors and viewpoints. For example, if the user is focused, the display unit can display with detailed information and annotations. This improves the learning effect by adjusting the display method based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user emotion data into the generative AI and have the generative AI perform emotion-based adjustments to the display method.

[0095] The display unit can have the function of adding annotations and supplementary information to 3D images of ancient castles in real time. For example, the display unit can have the function of adding annotations and supplementary information to 3D images of ancient castles in real time. For example, the display unit can display annotations in real time for specific objects in the image. For example, the display unit can display supplementary information in real time for specific scenes in the image. For example, the display unit can add annotations and supplementary information in real time in response to user operations. This improves the learning effect by adding annotations and supplementary information in real time. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the content of annotations and supplementary information into a generating AI and have the generating AI perform the addition of annotations and supplementary information in real time.

[0096] The display unit can add customization features to the 3D image of the ancient castle based on the user's operation history. For example, the display unit can add customization features to the 3D image of the ancient castle based on the user's operation history. For example, the display unit can customize the way the image is displayed based on what the user has done in the past. For example, the display unit can prioritize displaying content of interest based on the user's operation history. For example, the display unit can analyze the user's operation history and suggest the optimal display method. This improves the learning effect by customizing based on the user's operation history. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user operation history data into a generating AI and have the generating AI execute the customization features.

[0097] The display unit can estimate the user's emotions and adjust the color tone and brightness of the 3D image of the ancient castle based on the estimated emotions. For example, the display unit can estimate the user's emotions and adjust the color tone and brightness of the 3D image of the ancient castle based on the estimated emotions. For example, if the user is excited, the display unit can display the image with vivid colors and brightness. For example, if the user is relaxed, the display unit can display the image with calm colors and brightness. For example, if the user is focused, the display unit can display the image with highly visible colors and brightness. By adjusting the color tone and brightness based on the user's emotions, the learning effect is improved. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user emotion data into a generating AI, which can then perform adjustments to the color tone and brightness based on those emotions.

[0098] The display unit can be equipped with multi-user functionality that allows multiple users to access the 3D image of the ancient castle simultaneously. For example, the display unit can enable multiple users to view and collaboratively manipulate the 3D image simultaneously. For example, the display unit can enable multiple users to view the 3D image simultaneously and individually add annotations and supplementary information. For example, the display unit can enable multiple users to view the 3D image simultaneously and communicate in real time. This allows for collaborative learning through simultaneous access by multiple users. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can have a generative AI perform the execution of the multi-user functionality.

[0099] The display unit can display a progress bar and achievement meter that reflect the user's learning progress on a 3D image of an ancient castle. For example, the display unit can display a progress bar and achievement meter that reflect the user's learning progress on a 3D image of an ancient castle. For example, the display unit can display a progress bar according to the user's learning progress. For example, the display unit can display an achievement meter according to the user's achievement level. For example, the display unit can update and display the user's learning progress and achievement level in real time. This improves the effectiveness of learning by displaying learning progress and achievement level. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the user's learning progress data into a generating AI and have the generating AI execute the display of the progress bar and achievement meter.

[0100] The generation unit can estimate the user's emotions and adjust the level of detail of the 3D video of the chemical experiment based on the estimated emotions. For example, the generation unit can estimate the user's emotions and adjust the level of detail of the 3D video of the chemical experiment based on the estimated emotions. For example, if the user is excited, the generation unit can add detailed reactions and effects. For example, if the user is relaxed, the generation unit can use calm colors and a simple design. For example, if the user is focused, the generation unit can add theoretical annotations and detailed information. This improves the learning effect by adjusting the level of detail based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the generation unit may be performed using AI or not using AI. For example, the generation unit can input user emotion data into the generative AI and have the generative AI perform the emotion-based level of detail adjustment.

[0101] The generation unit can add the theoretical background and related knowledge of a chemical experiment to a 3D video of the experiment. For example, the generation unit can add the theoretical background and related knowledge of a chemical experiment to a 3D video of a chemical experiment. For example, the generation unit can add the theoretical background of a chemical reaction to a 3D video. For example, the generation unit can add chemical knowledge and information related to the experiment to a 3D video. For example, the generation unit can add the experimental procedure and precautions to a 3D video. By adding the theoretical background and knowledge, the effectiveness of learning is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the theoretical background and knowledge into a generation AI and have the generation AI perform the addition to the 3D video.

[0102] The generation unit can add user-operable interactive elements to 3D images of chemical experiments. For example, the generation unit can add user-operable interactive elements to 3D images of chemical experiments. For example, the generation unit can enable users to operate experimental equipment within the 3D image and simulate the experiment. For example, the generation unit can enable users to mix or heat chemicals within the 3D image to observe the reaction. For example, the generation unit can enable users to analyze experimental results within the 3D image and acquire data. By adding interactive elements, the learning effect is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the interactive elements into a generation AI and have the generation AI perform the addition to the 3D image.

[0103] The generation unit can estimate the user's emotions and adjust the viewpoint of the 3D video of the chemical experiment based on the estimated user emotions. For example, the generation unit can estimate the user's emotions and adjust the viewpoint of the 3D video of the chemical experiment based on the estimated user emotions. For example, if the user is excited, the generation unit can display the video with a dynamic viewpoint. For example, if the user is relaxed, the generation unit can display the video with a calm viewpoint. For example, if the user is focused, the generation unit can display the video with a detailed viewpoint. This improves the learning effect by adjusting the viewpoint based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input user emotion data into the generative AI and have the generative AI perform the viewpoint adjustment based on emotions.

[0104] The generation unit can add a comparison function to 3D images of chemical experiments. For example, the generation unit can add a comparison function to 3D images of chemical experiments. For example, the generation unit can make it possible to compare the results of the same chemical reaction performed under different conditions. For example, the generation unit can display different chemical reactions side by side and make them comparable. For example, the generation unit can make it possible to compare the procedures and results of an experiment with those of other experiments. By adding a comparison function with other experiments, the effectiveness of learning is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the comparison function into a generation AI and have the generation AI perform the addition to the 3D images.

[0105] The generation unit can add an interactive guide function to 3D video of a chemistry experiment, allowing the user to ask questions. For example, the generation unit can add an interactive guide function to 3D video of a chemistry experiment, allowing the user to ask questions. For example, the generation unit can enable the user to input questions about the chemistry experiment and receive answers in real time. For example, the generation unit can enable the user to ask questions about specific parts of the chemistry experiment and display detailed information. For example, the generation unit can enable the user to ask questions about the theoretical background of the chemistry experiment and display relevant knowledge and information. By adding this interactive guide function, the learning effect is improved. Some or all of the above processing in the generation unit may be performed using AI, for example, or without AI. For example, the generation unit can input the content of the guide function into a generation AI and have the generation AI perform the addition to the 3D video.

[0106] The display unit can estimate the user's emotions and adjust the display method of the 3D chemical experiment based on the estimated user emotions. For example, the display unit can estimate the user's emotions and adjust the display method of the 3D chemical experiment based on the estimated user emotions. For example, if the user is excited, the display unit can add dynamic viewpoints and effects. For example, if the user is relaxed, the display unit can display with calm colors and viewpoints. For example, if the user is focused, the display unit can add detailed information and annotations. This improves the learning effect by adjusting the display method based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the display unit may be performed using AI or not using AI. For example, the display unit can input user emotion data into the generative AI and have the generative AI perform emotion-based adjustments to the display method.

[0107] The display unit can have the function of adding annotations and supplementary information to 3D images of chemical experiments in real time. For example, the display unit can have the function of adding annotations and supplementary information to 3D images of chemical experiments in real time. For example, the display unit can display annotations in real time for specific objects in the image. For example, the display unit can display supplementary information in real time for specific scenes in the image. For example, the display unit can add annotations and supplementary information in real time in response to user operations. This improves the effectiveness of learning by adding annotations and supplementary information in real time. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the content of annotations and supplementary information into a generating AI and have the generating AI perform the addition of annotations and supplementary information in real time.

[0108] The display unit can add customization functions to 3D images of chemical experiments based on the user's operation history. For example, the display unit can add customization functions to 3D images of chemical experiments based on the user's operation history. For example, the display unit can customize the display method of the images based on what the user has done in the past. For example, the display unit can prioritize displaying content of interest based on the user's operation history. For example, the display unit can analyze the user's operation history and suggest the optimal display method. This improves the learning effect by customizing based on the user's operation history. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user operation history data into a generating AI and have the generating AI execute the customization functions.

[0109] The display unit can estimate the user's emotions and adjust the color tone and brightness of the 3D video of the chemical experiment based on the estimated emotions. For example, the display unit estimates the user's emotions and adjusts the color tone and brightness of the 3D video of the chemical experiment based on the estimated emotions. For example, if the user is excited, the display unit displays the video with vivid colors and brightness. For example, if the user is relaxed, the display unit displays the video with calm colors and brightness. For example, if the user is focused, the display unit displays the video with highly visible colors and brightness. By adjusting the color tone and brightness based on the user's emotions, the learning effect is improved. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input user emotion data into a generating AI, which can then perform adjustments to the color tone and brightness based on those emotions.

[0110] The display unit can be equipped with multi-user functionality that allows multiple users to access 3D images of chemical experiments simultaneously. For example, the display unit can enable multiple users to view and collaboratively manipulate 3D images simultaneously. For example, it can enable multiple users to view 3D images simultaneously and add annotations or supplementary information individually. For example, it can enable multiple users to view 3D images simultaneously and communicate in real time. This allows for collaborative learning through simultaneous access by multiple users. Some or all of the above-described processes in the display unit may be performed using AI, for example, or without AI. For example, the display unit can have a generative AI perform the execution of the multi-user functionality.

[0111] The display unit can display progress bars and achievement meters that reflect the user's learning progress on a 3D video of a chemistry experiment. For example, the display unit can display progress bars and achievement meters that reflect the user's learning progress on a 3D video of a chemistry experiment. For example, the display unit can display progress bars according to the user's learning progress. For example, the display unit can display achievement meters according to the user's achievement level. For example, the display unit can update and display the user's learning progress and achievement level in real time. This improves the effectiveness of learning by displaying learning progress and achievement level. Some or all of the above processing in the display unit may be performed using AI, for example, or without AI. For example, the display unit can input the user's learning progress data into a generating AI and have the generating AI display progress bars and achievement meters.

[0112] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0113] The educational support system can also be equipped with a voice recognition unit. The voice recognition unit can recognize the user's voice commands and send instructions to the generation and display units. For example, if the user says, "Show me the inside of the castle," the generation unit can generate a 3D image of the castle's interior based on that command, and the display unit can display that image. Also, if the user says, "Start the next experiment," the generation unit can generate a 3D image of the next chemistry experiment, and the display unit can display that image. As a result, by using voice recognition, users can operate the system more intuitively, improving the effectiveness of their learning.

[0114] The educational support system can also include a feedback unit. This feedback unit can evaluate the user's learning progress and understanding, and provide appropriate feedback. For example, after a user views a 3D image of an ancient castle in a history lesson, the feedback unit can present the user with a quiz and evaluate their understanding based on their answers. Similarly, after a user manipulates a 3D image in a chemistry experiment, the feedback unit can provide feedback on the experiment's procedure and results. This allows for improved learning effectiveness through feedback.

[0115] The educational support system can also include a data analysis unit. This unit can collect and analyze user learning data to evaluate learning trends and effectiveness. For example, it can collect and analyze data such as which 3D images users watched, for how long, and where they struggled. Furthermore, it can compare data from multiple users to identify common challenges and effective learning methods. This enables data-driven optimization of learning.

[0116] The educational support system can also include a customization section. This customization section allows users to customize the content and display method of 3D images according to their learning style and preferences. For example, if a user prefers visual learning, the customization section can add numerous visual effects to the images. If a user prefers detailed information, annotations and supplementary information can be added to the images. This enables learning tailored to the individual needs of each user.

[0117] The educational support system can also include a collaborative learning section. This section allows multiple users to simultaneously view 3D images and engage in collaborative learning activities. For example, multiple users can simultaneously view a 3D image of an ancient castle while exchanging opinions via chat or voice calls. Users can also collaboratively manipulate a 3D image of a chemistry experiment and share the results. This promotes interaction among users through collaborative learning, improving the effectiveness of the learning process.

[0118] The educational support system can also be equipped with an emotion estimation unit. This unit can estimate the user's emotions from their facial expressions and voice, and adjust the learning content based on those emotions. For example, if the user is excited, the emotion estimation unit can add dynamic scenes or effects to the video. Conversely, if the user is relaxed, it can add calm scenes or tones to the video. This provides a learning experience tailored to the user's emotions, improving the effectiveness of the learning.

[0119] The educational support system can also be equipped with an emotional feedback unit. This unit can estimate the user's emotions and provide feedback based on those emotions. For example, if the user is feeling stressed, the emotional feedback unit can suggest relaxation advice or breaks. Conversely, if the user is focused, it can offer more challenging tasks. This provides feedback tailored to the user's emotions, improving learning effectiveness.

[0120] The educational support system can also be equipped with an emotion monitoring unit. This unit can monitor the user's emotions in real time and adjust the learning content based on that data. For example, if the user is excited, the emotion monitoring unit can add dynamic scenes or effects to the video. Conversely, if the user is relaxed, it can add calm scenes or tones to the video. This provides a learning experience tailored to the user's emotions, improving learning effectiveness.

[0121] The educational support system can also be equipped with an emotion adaptation unit. This unit can estimate the user's emotions and adjust the learning pace based on those emotions. For example, if the user is feeling stressed, the unit can slow down the learning pace. Conversely, if the user is focused, it can speed up the learning pace. This provides a learning pace that is tailored to the user's emotions, improving the effectiveness of the learning.

[0122] The educational support system can also be equipped with an emotion analysis unit. This unit can analyze the user's emotional data and optimize learning content based on the results. For example, if the user is excited, the emotion analysis unit can add dynamic scenes or effects to the video. Conversely, if the user is relaxed, it can add calming scenes or tones. This provides a learning experience tailored to the user's emotions, improving learning effectiveness.

[0123] The following briefly describes the processing flow for example form 2.

[0124] Step 1: The generation unit generates 3D images from stories and diagrams in textbooks. For example, the generation unit generates 3D images of ancient castles in history class and 3D images of glass tubes and chemicals in chemistry experiments. These images are generated by the generation unit, which includes AI processing. Step 2: The display unit displays the 3D images generated by the generation unit. For example, the display unit displays 3D images of an ancient castle, a glass tube, or chemicals, all generated by the generation unit.

[0125] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0126] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0127] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0128] For example, each of the multiple elements, including the generation unit and the display unit, is implemented by at least one of the smart device 14 and the data processing device 12. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates the contents of the textbook as a 3D image. The display unit is implemented by the display 40A of the smart device 14 and displays the generated 3D image. The generation unit may also be implemented by the control unit 46A of the smart device 14, for example. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0129] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0130] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0131] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0132] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0133] The microphone 238 receives voice commands and other instructions from the user by receiving voice signals. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0134] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0135] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0136] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0137] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0138] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0139] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0140] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0141] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0142] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0143] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0144] For example, each of the multiple elements, including the generation unit and the display unit, is implemented by at least one of the smart glasses 214 and the data processing device 12. For example, the generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates the contents of the textbook as a 3D image. The display unit is implemented by the display of the smart glasses 214 and displays the generated 3D image. The generation unit may also be implemented by the control unit 46A of the smart glasses 214, for example. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0145] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0146] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0147] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0148] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0149] The microphone 238 receives voice commands and other instructions from the user by receiving voice signals. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0150] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0151] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0152] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0153] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0154] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0155] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0156] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0157] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0158] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0159] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0160] For example, each of the multiple elements, including the generation unit and the display unit, is implemented by at least one of the headset terminal 314 and the data processing unit 12. For example, the generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates the contents of the textbook as a 3D image. The display unit is implemented by the display 343 of the headset terminal 314 and displays the generated 3D image. The generation unit may also be implemented by the control unit 46A of the headset terminal 314, for example. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0161] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0162] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0163] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0164] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0165] The microphone 238 receives voice commands and other instructions from the user by receiving voice signals. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0166] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0167] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0168] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0169] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0170] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0171] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0172] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0173] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0174] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0175] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0176] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0177] For example, each of the multiple elements, including the generation unit and the display unit, is implemented by at least one of the robot 414 and the data processing unit 12. For example, the generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates the contents of the textbook as a 3D image. The display unit is implemented by the display of the robot 414 and displays the generated 3D image. The generation unit may also be implemented by the control unit 46A of the robot 414, for example. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0178] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0179] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0180] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0181] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0182] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0183] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0184] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0185] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0186] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0187] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0188] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0189] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0190] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0191] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0192] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0193] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0194] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0195] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0196] (Note 1) A generation unit that generates 3D images from stories or diagrams in a textbook, The system includes a display unit that displays the 3D image generated by the generation unit. A system characterized by the following features. (Note 2) The generating unit is In history class, we generate 3D images of ancient castles. The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned display unit is The 3D image of the ancient castle generated by the aforementioned generation unit is displayed. The system described in Appendix 1, characterized by the features described herein. (Note 4) The generating unit is Generate 3D images of glass tubes or chemicals in chemistry experiments. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned display unit is The 3D image of the glass tube or chemicals produced by the aforementioned generation unit is displayed. The system described in Appendix 1, characterized by the features described herein. (Note 6) The generating unit is It estimates the user's emotions and adjusts the content of the generated 3D video based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 7) The generating unit is Based on the textbook content, it generates 3D images tailored to specific learning objectives. The system described in Appendix 1, characterized by the features described herein. (Note 8) The generating unit is This adds interactive elements to the generated 3D images, allowing users to interact with them within the video. The system described in Appendix 1, characterized by the features described herein. (Note 9) The generating unit is It estimates the user's emotions and adjusts the difficulty level of the generated 3D images based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 10) The generating unit is Based on the textbook content, it generates 3D images that correspond to different grade levels and learning levels. The system described in Appendix 1, characterized by the features described herein. (Note 11) The generating unit is The generated 3D images can be enhanced with audio guides and subtitles to support learning through both visual and auditory means. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned display unit is It estimates the user's emotions and adjusts the viewpoint and angle of the displayed 3D image based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned display unit is The system will have a function to add annotations and supplementary information to the displayed 3D image in real time. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned display unit is Add a customization feature to the displayed 3D images based on the user's operation history. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned display unit is It estimates the user's emotions and adjusts the color tone and brightness of the displayed 3D image based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned display unit is Add a multi-user feature that allows multiple users to access the displayed 3D image simultaneously. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned display unit is The displayed 3D image will show a progress bar and achievement meter that reflect the user's learning progress. The system described in Appendix 1, characterized by the features described herein. (Note 18) The generating unit is The system estimates the user's emotions and adjusts the level of detail in the 3D image of the ancient castle based on those emotions. The system described in Appendix 2, characterized by the features described herein. (Note 19) The generating unit is Add historical background and related episodes to 3D images of ancient castles. The system described in Appendix 2, characterized by the features described herein. (Note 20) The generating unit is Add interactive elements that users can explore to a 3D image of an ancient castle. The system described in Appendix 2, characterized by the features described herein. (Note 21) The generating unit is It estimates the user's emotions and adjusts the viewpoint of the 3D image of the ancient castle based on the estimated user emotions. The system described in Appendix 2, characterized by the features described herein. (Note 22) The generating unit is Add a feature to compare 3D images of ancient castles with other historical buildings. The system described in Appendix 2, characterized by the features described herein. (Note 23) The generating unit is Add an interactive guide feature to the 3D video of the ancient castle, allowing users to ask questions. The system described in Appendix 2, characterized by the features described herein. (Note 24) The aforementioned display unit is The system estimates the user's emotions and adjusts how the 3D image of the ancient castle is displayed based on those estimated emotions. The system described in Appendix 3, characterized by the features described herein. (Note 25) The aforementioned display unit is The system will have a feature that allows users to add annotations and supplementary information to 3D images of ancient castles in real time. The system described in Appendix 3, characterized by the features described herein. (Note 26) The aforementioned display unit is Add a customization feature to the 3D model of ancient castles based on the user's operation history. The system described in Appendix 3, characterized by the features described herein. (Note 27) The aforementioned display unit is The system estimates the user's emotions and adjusts the color tone and brightness of the 3D image of the ancient castle based on those estimated emotions. The system described in Appendix 3, characterized by the features described herein. (Note 28) The aforementioned display unit is Adding multi-user support functionality that allows multiple users to access 3D images of ancient castles simultaneously. The system described in Appendix 3, characterized by the features described herein. (Note 29) The aforementioned display unit is The 3D model of the ancient castle displays a progress bar and achievement meter that reflect the user's learning progress. The system described in Appendix 3, characterized by the features described herein. (Note 30) The generating unit is The system estimates the user's emotions and adjusts the level of detail in the 3D video of the chemical experiment based on those emotions. The system described in Appendix 4, characterized by the features described herein. (Note 31) The generating unit is Add the theoretical background and related knowledge of the experiment to 3D videos of chemical experiments. The system described in Appendix 4, characterized by the features described herein. (Note 32) The generating unit is Add interactive elements to 3D videos of chemistry experiments that users can control. The system described in Appendix 4, characterized by the features described herein. (Note 33) The generating unit is It estimates the user's emotions and adjusts the viewpoint of the 3D video of the chemical experiment based on the estimated user emotions. The system described in Appendix 4, characterized by the features described herein. (Note 34) The generating unit is Add a comparison function with other experiments to 3D videos of chemistry experiments. The system described in Appendix 4, characterized by the features described herein. (Note 35) The generating unit is Add an interactive guide feature to 3D videos of chemistry experiments, allowing users to ask questions. The system described in Appendix 4, characterized by the features described herein. (Note 36) The display unit is The system estimates the user's emotions and adjusts how the 3D images of the chemical experiment are displayed based on those emotions. The system described in Appendix 5, characterized by the features described herein. (Note 37) The aforementioned display unit is The system will have a feature that allows for real-time addition of annotations and supplementary information to 3D videos of chemical experiments. The system described in Appendix 5, characterized by the features described herein. (Note 38) The aforementioned display unit is Add a customization feature to 3D videos of chemistry experiments based on the user's operation history. The system described in Appendix 5, characterized by the features described herein. (Note 39) The display unit is It estimates the user's emotions and adjusts the color tone and brightness of the 3D video of the chemical experiment based on the estimated user emotions. The system described in Appendix 5, characterized by the features described herein. (Note 40) The display unit is Add a multi-user feature that allows multiple users to access 3D videos of chemistry experiments simultaneously. The system described in Appendix 5, characterized by the features described herein. (Note 41) The display unit is The 3D video of the chemistry experiment displays a progress bar and achievement meter that reflect the user's learning progress. The system described in Appendix 5, characterized by the features described herein. [Explanation of Symbols]

[0197] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A generation unit that generates 3D images from stories or diagrams in a textbook, The system includes a display unit that displays the 3D image generated by the generation unit. A system characterized by the following features.

2. The generating unit is In history class, we generate 3D images of ancient castles. The system according to feature 1.

3. The aforementioned display unit is The 3D image of the ancient castle generated by the aforementioned generation unit is displayed. The system according to feature 1.

4. The generating unit is Generate 3D images of glass tubes or chemicals in chemistry experiments. The system according to feature 1.

5. The aforementioned display unit is The 3D image of the glass tube or chemicals produced by the aforementioned generation unit is displayed. The system according to feature 1.

6. The generating unit is It estimates the user's emotions and adjusts the content of the generated 3D video based on the estimated user emotions. The system according to feature 1.

7. The generating unit is Based on the textbook content, it generates 3D images tailored to specific learning objectives. The system according to feature 1.

8. The generating unit is This adds interactive elements to the generated 3D images, allowing users to interact with them within the video. The system according to feature 1.

9. The generating unit is It estimates the user's emotions and adjusts the difficulty level of the generated 3D images based on those estimated emotions. The system according to feature 1.

10. The generating unit is Based on the textbook content, it generates 3D images that correspond to different grade levels and learning levels. The system according to feature 1.

Images