system

Abstract

We provide the system. [Solution] A means of capturing learning information in educational institutions, means for converting the aforementioned learning information into an appropriate learning format, A means for providing the information converted into the aforementioned learning format to the learner via a communication path, A means for receiving inquiry information from the learner and generating an answer based on said inquiry information, A means for presenting the generated answer to the learner, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In educational institutions, for learners who have missed classes due to poor health or other reasons, the conventional methods may not effectively convey the class content, resulting in a loss of learning opportunities. This problem is caused by the fact that it is difficult for absent learners to receive the class content in real time and that question - and - answer sessions are not directly conducted, which may lead to insufficient learning comprehension. As a result, there is a problem that the educational achievements of learners are not fully achieved.

Means for Solving the Problems

[0005] This invention provides a system that uses information processing technology to record lessons in educational institutions, converts that lesson information into an appropriate format, and provides it to learners. Specifically, it summarizes the captured learning information within a time frame specified by the learner and distributes it via a network. Furthermore, this system receives inquiry information from learners, generates answers corresponding to the content using information processing technology, and presents them to the learners, thereby supplementing their understanding of the lesson content. In addition, it enables the system to respond to individual learning needs by generating additional learning information according to the learner's level of understanding.

[0006] An "educational institution" is an organization or facility whose purpose is to impart knowledge and skills to learners.

[0007] "Learning information" refers to data such as lesson content and educational materials provided by educational institutions.

[0008] "Capture" refers to the process of recording and saving information.

[0009] A "learning format" refers to the form of information that is organized in a way that is easy for learners to understand.

[0010] "Communication path" refers to the internet and other communication infrastructure used to send and receive information.

[0011] "Learners" refer to individuals who are in an educational institution with the aim of acquiring knowledge and skills.

[0012] "Inquiry information" refers to information that learners use to raise questions or concerns about the lesson content.

[0013] "Answer" refers to the response information generated based on the inquiry information.

[0014] "To present" means the act of displaying or providing information.

[0015] "Information processing technology" refers to the technology for operating and processing data using a computer.

Brief Explanation of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Best Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] The system according to the present invention appropriately provides lesson content to learners who have missed classes at educational institutions. This system consists of a server, terminals, and users.

[0038] The server receives recorded lesson information as classes are held and stores this information in a database. The stored lesson information is then converted into a learning format accessible to learners. For example, the server captures a math class in video format and stores the data in the database. Furthermore, the server converts and provides the learning format in response to user requests to watch the class at a desired time.

[0039] Users can access a dedicated portal using their device to search for information about classes they missed and specify their preferred viewing time. For example, if a user requests a class starting at 3 PM, the server will customize and provide the user with appropriately summarized class information.

[0040] Furthermore, users can input questions that arise while watching the lessons into their devices. The server receives this inquiry information and uses an AI engine to generate answers to the questions. The results are then presented to the user via their device to support their understanding.

[0041] As described above, this system has a structure that supports learners who miss classes in a way that allows them to flexibly access learning opportunities as if they were participating in real-time classes. In this way, it is possible to minimize the loss of learning opportunities due to absence and provide an efficient learning experience. For example, if a user has difficulty understanding a particular mathematical formula and asks a question, the AI ​​will explain the breakdown process of the formula step by step, deepening the user's understanding.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The server receives real-time video footage of the lessons from the designated camera system at the start of each class at the educational institution.

[0045] Step 2:

[0046] The server converts the received lecture video data into a standard format (e.g., MP4 format), adds metadata (such as the instructor, lecture date, and subject name), and stores it in the database.

[0047] Step 3:

[0048] Users log in to a dedicated portal via their device and operate an interface to select the classes they wish to watch from a list of missed classes.

[0049] Step 4:

[0050] The user specifies their preferred viewing time for the selected class (e.g., one hour starting at 3 PM) and sends this information to the server.

[0051] Step 5:

[0052] Based on the requested viewing time slot, the server sends the saved lesson data to the AI ​​engine and generates a learning format that summarizes the key points to fit within the given time frame.

[0053] Step 6:

[0054] The server prepares to stream the summarized lesson data generated by the AI ​​engine to the user and begins receiving the data on the user's device.

[0055] Step 7:

[0056] If a user encounters a part they don't understand while watching, they can submit their question to the server by entering it through a question form installed on their device.

[0057] Step 8:

[0058] The server forwards the question received from the user to the AI ​​engine and executes the process of generating an appropriate answer to that question.

[0059] Step 9:

[0060] The server sends the answer received from the AI ​​engine to the user's device and displays the answer on the screen.

[0061] Step 10:

[0062] Users refer to the provided answers and continue watching the lessons while deepening their understanding of the learning material.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] In modern educational institutions, a challenge exists in the learning environment: learners may miss learning opportunities due to absences from classes. Furthermore, there is a lack of mechanisms to effectively review missed lessons, ensuring proper understanding of the material and resolving any questions. Flexible learning support tailored to each learner's level of comprehension is also needed.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes means for recording learning activities, means for storing information in a storage device, and information processing means for converting the stored information into an appropriate format. This allows learners to efficiently review the content of missed classes and to immediately answer questions they face using a live artificial intelligence model. Furthermore, appropriate learning support is achieved by appropriately summarizing information according to the learner's level of understanding.

[0068] "Learning activities" refer to lectures, classes, and all related educational activities within educational institutions.

[0069] "Means of recording" refers to the technology used to collect information in audio, video, or digital data format and to store that information.

[0070] A "storage device" refers to an electronic device or system that holds recorded data and makes it accessible at a later date.

[0071] "Information processing means" refers to programs, algorithms, or technologies used to perform data transformation, analysis, and modification.

[0072] "Communication means" refers to the technology or infrastructure used for sending and receiving information, and includes the internet and network communication technologies.

[0073] "User" refers to the subject who receives learning content using this system, i.e., the learner.

[0074] "Inquiry content" refers to questions or concerns that users have about the system, and it also refers to data that is processed within the system.

[0075] A "generative artificial intelligence model" refers to a computer program that uses machine learning techniques to analyze, understand, and generate responses from data.

[0076] "Means of generating responses" refers to computational methods or digital technologies for automatically constructing and outputting responses based on the content of inquiries.

[0077] This invention provides a system that records information on learning activities in educational institutions as digital data and provides an environment in which learners can access and learn from that information. The system mainly consists of a server, terminals, and users.

[0078] The server captures learning activities as digital data using equipment designed to record lectures and classes at educational institutions. Video capture software such as "OBS Studio" is used for this recording. The recorded data is processed into an appropriate format and stored in a database. Database management systems such as "MySQL®" and "PostgreSQL" are used for database management.

[0079] The server processes stored lesson data to shorten and summarize it in response to user requests. This process uses software such as "FFmpeg" to trim and summarize videos. Furthermore, the server uses a generative AI model to create answers to user inquiries. For example, the "GPT model from OpenAI®" is used as the "generative AI model."

[0080] Users can access a web portal from a dedicated terminal or personal computer to search for missed class content and specify their desired viewing time. If a user enters "I would like to view the class from 3 PM" via the terminal, the server will adjust the content to fit the specified time and provide it to the user. Furthermore, users can enter questions via the terminal while viewing the class, and the server will derive answers through a generative AI model, displaying the results on the terminal. An example of a prompt message at this time is, "Please explain the decomposition process of the following equation step by step: x^2 + y^2 = z^2".

[0081] Thus, the present invention provides an environment that ensures learners do not miss learning opportunities in real time, and realizes an efficient learning experience tailored to the individual needs of each learner.

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] The server records classroom sessions in real time. Video capture software is used for recording, converting the video feed into digital data. This digital data is temporarily stored in a format that includes both video and audio.

[0085] Step 2:

[0086] The server stores the recorded digital data in a database. Here, the stored data is organized, and metadata such as the ID, title, date, and instructor's name for each lesson is added. This metadata serves as input data for users to search later.

[0087] Step 3:

[0088] Users access the web portal from their device to search for information about classes they missed. Users send queries based on criteria such as class ID and date, and the portal sends data requests to the server based on that input.

[0089] Step 4:

[0090] The server selects lesson data according to the user's request. The selected data is shortened and summarized using video processing software. The input is the full video data, and by applying a summarization algorithm, it outputs a shortened video that extracts only the important parts.

[0091] Step 5:

[0092] Users watch summarized lecture videos. A play command is entered from the terminal, and the summarized video, which is output data from the server, can be streamed.

[0093] Step 6:

[0094] When a user enters a question while watching a lesson, their device sends this question to the server. The question is entered as text data, and the server analyzes it using a generative AI model.

[0095] Step 7:

[0096] The server generates answers to questions using a generative AI model. It takes "Answer the following question: User's question" as input and outputs the answer generated by the AI.

[0097] Step 8:

[0098] The generated answers are then presented to the user again via the device. Through this output, the user can resolve their questions and deepen their understanding of the material.

[0099] (Application Example 1)

[0100] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0101] Educational institutions need a way to effectively supplement the learning content of students who miss classes, without losing necessary learning opportunities. In this context, the challenge is to provide a system that efficiently catches up on missed classes and can flexibly adapt to the student's level of understanding.

[0102] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0103] In this invention, the server includes means for acquiring learning information from an educational institution, means for converting the learning information into an appropriate information format, and means for providing the converted information to a user via a communication path. This makes it possible to efficiently supplement the content of classes that learners have missed and to realize a learning environment that allows them to view the content according to their own schedule.

[0104] "Learning information" refers to data related to the content of lessons and educational resources at educational institutions.

[0105] "Means of acquisition" refers to technical means for capturing or collecting learning information.

[0106] "Information format" refers to the data format in which acquired learning information is converted into a form that is easy for users to understand.

[0107] A "communication path" refers to the network or infrastructure used to transmit information.

[0108] A "user" is a student who has missed classes at an educational institution or a beneficiary of the system.

[0109] "Inquiry information" refers to data related to questions and requests that users enter into the system.

[0110] "Means of generating a response" refers to the technologies and processes used to create an appropriate answer based on the inquiry information.

[0111] An "interface" is a screen or platform that users use to operate a system and view information.

[0112] "Means for setting a viewing schedule" refers to functions or methods that allow users to specify and plan the amount of time they will spend viewing information.

[0113] To implement this invention, a server, a terminal, and a user must each fulfill their respective roles. The server records lessons at educational institutions and stores the data in a database. In doing so, the server converts the recorded data into an appropriate information format and prepares it for later access by users through a predefined communication path. This typically involves using a cloud server or a large-capacity storage system.

[0114] A terminal is a device used by users to access data stored on a server. Typically, this refers to a mobile device such as a smartphone or tablet, with a dedicated application installed. Using this terminal, users can log in to their account and search, select, and view information about missed classes.

[0115] Furthermore, users can input questions that arise during class on their devices. These questions are sent to a server, which uses a generative AI model to generate appropriate answers. The generated answers are returned to the device in real time and presented to the user. This allows learners not only to catch up on missed lessons but also to deepen their understanding.

[0116] As a concrete example, if a user enters the prompt "I want to know more about how this formula is derived" while watching a recording of a calculus lesson, the server will use a generative AI model to explain the derivation process of the formula step by step and display it on the user's device. An example of such a prompt would be, "Please explain in detail how to calculate differentiation." This system allows learners to access learning opportunities flexibly and minimizes the loss of learning opportunities due to absences.

[0117] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0118] Step 1:

[0119] The server records classes at educational institutions and stores the data in a database. This process saves the recorded video and audio files to the database in a specialized compressed format. At this point, the input is the raw class data, and the output is the compressed recorded data.

[0120] Step 2:

[0121] The user logs into a dedicated application using their terminal and searches the database for the necessary class information. In this case, the input consists of the user's login information and search query, and the output is the recorded data of the corresponding class.

[0122] Step 3:

[0123] The device plays back the received recorded data at a time specified by the user and displays the learning content. At this point, the input is the recorded data and the user's viewing schedule, and the output is the playback of video and audio.

[0124] Step 4:

[0125] Users enter questions that arise during viewing into a question input form within the application. The input here is the prompt text entered by the user, and the output is the transmission of the question data to the server.

[0126] Step 5:

[0127] The server generates answers using a generative AI model based on questions received from the user. This process analyzes the input question through natural language processing and generates answers from a relevant knowledge database. The output is the generated answer text.

[0128] Step 6:

[0129] The terminal displays the response sent from the server to the user. The input here is the response generated by the server, and the output is the response text displayed on the user's screen.

[0130] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0131] This invention combines a system that provides class information to learners who have missed classes in educational institutions with an emotion engine that recognizes learners' emotions and adjusts learning information accordingly. This system consists of a server, terminals, and an emotion engine, and provides a flexible learning experience tailored to the individual needs of learners.

[0132] The server retrieves lesson information from educational institutions in real time and converts it into a standard format using information processing technology. This information is stored in a database and later provided to learners in a summarized and edited format. Users can use their terminals to view videos of lessons they missed and specify their desired viewing time. The server then streams a video of the lesson, summarizing the key points within the specified time.

[0133] The emotion engine analyzes camera footage and sensor information acquired from the user's device to evaluate the learner's emotional state in real time. This evaluation result is sent to the server and used to dynamically adjust the way learning content is delivered. For example, if the server determines that the user is confused, it can provide detailed explanations or navigation using friendly language, following the instructions of the emotion engine.

[0134] Furthermore, the emotion engine assists the server in generating appropriate communication styles by considering the user's emotional state in response to questions entered by the user on the device. In addition, the emotion engine inserts encouraging messages and feedback that gives a sense of accomplishment as needed to maintain or improve the user's motivation.

[0135] In this way, the present invention aims not merely to provide educational information, but to improve learning efficiency and comprehension by understanding the learner's emotional state and providing a personalized learning experience. For example, if the system determines that the user is tired, it can explain the learning content in simpler terms and send a message encouraging them to take a break, thus enabling a learner-friendly system.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The server acquires video and audio data from classes held at educational institutions in real time and converts it into a standard format. This allows the class information to be stored in an organized manner in a database.

[0139] Step 2:

[0140] Users log in to a dedicated portal via their device and select the class they wish to view from a list of missed classes. They then specify the time slot they wish to view the class and send this information to the server.

[0141] Step 3:

[0142] The server summarizes saved lesson information based on the time frame specified by the user and converts key points into a learning format. Using an AI engine, it generates lesson content that is adjusted to fit within the time frame.

[0143] Step 4:

[0144] The device uses its built-in camera and sensors to capture the user's facial expressions and movements, and sends this data to the emotion engine. Meanwhile, the server prepares to stream the summarized lesson video.

[0145] Step 5:

[0146] The emotion engine analyzes emotional data received from the user to identify the user's emotional state. For example, it evaluates states such as being focused, tired, or confused.

[0147] Step 6:

[0148] The server adjusts the delivery format of learning content based on the user's emotional state identified by the emotion engine. If it determines that the user is confused, it provides additional explanations and examples to improve the viewing experience.

[0149] Step 7:

[0150] If a user has any questions while watching a lesson, they can enter them through the terminal's interface. These questions are then sent to the server.

[0151] Step 8:

[0152] The server sends the user's question to the emotion engine and initiates the process of generating an answer in a communication style appropriate to the user's emotional state.

[0153] Step 9:

[0154] The server receives responses from the emotion engine and delivers them to the user's device. The user then refers to these responses to ask further questions or continue the lesson.

[0155] Step 10:

[0156] The emotion engine supports the entire learning experience by sending encouraging and feedback messages at appropriate times to maintain user motivation as learning progresses.

[0157] (Example 2)

[0158] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] Traditional educational information systems struggled to adequately address the understanding of absent learners. Furthermore, they lacked mechanisms for providing adaptive educational content tailored to learners' emotional states, resulting in insufficient flexibility to meet individual learner needs. This led to a decline in the quality of the learning experience and hindered improvements in learner comprehension and motivation.

[0160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0161] In this invention, the server includes means for acquiring educational information at an educational facility, means for converting the educational information into a unified format, and means for providing the information converted into the unified format to learners via a communication channel. This enables the analysis of learners' emotional states and adaptive adjustment of educational content according to individual needs, thereby improving each learner's understanding and motivation.

[0162] "Educational institutions" is a general term for institutions and organizations that provide education and training to learners.

[0163] "Educational information" is a general term for information related to learning, including the content of lessons, teaching materials, lecture materials, and other information.

[0164] "Unified format" refers to a state where information from different formats or styles has been converted into a consistent format.

[0165] A "communication channel" is a general term for a path or medium used to send and receive information, and is used to transmit digital data through a network.

[0166] A "learner" is a general term for individuals who acquire knowledge and skills through educational institutions and systems.

[0167] "Inquiry information" refers to information and data that express the questions and doubts that learners have.

[0168] "Response" refers to information including answers, explanations, and feedback in response to inquiries.

[0169] "Emotional state" refers to a person's psychological or emotional state, which is usually determined by the analysis of sensory information.

[0170] "Adaptive educational content" refers to educational materials and information that are dynamically adjusted and modified according to the needs and circumstances of learners.

[0171] This invention is a system for providing educational information to learners in a flexible and adaptive manner within educational facilities. The system is primarily constructed using a server, terminals, and an emotion engine.

[0172] The server has the function of acquiring lesson information from educational institutions in real time and converting it into a unified format using information processing technology. The hardware used includes a database server and a network interface. The database stores the saved, converted educational information and performs any necessary editing. Software used includes a database management system and video conversion software.

[0173] The device provides learners with an environment that makes it easy to access educational content. Learners can watch lesson videos through the device and set their desired viewing time. The server then provides summarized educational content according to the specified time. Furthermore, the device is equipped with a camera and sensors, which are used by an emotion engine to analyze the learner's emotional state.

[0174] The emotion engine analyzes data acquired from the device and evaluates the learner's psychological state in real time. For example, if it determines that the learner is confused, the server is instructed to provide a detailed explanation. This analysis process utilizes emotion analysis algorithms and machine learning techniques, which are applied by the server.

[0175] For example, if the emotion engine determines that a user is tired, the server will translate the learning content into more easily understandable language and send a message encouraging them to take a break. It is also possible to provide encouraging messages to the user using a generative AI model. An example of a prompt is shown below: "Generate a message for a tired learner that briefly explains the learning content and encourages them to take a break."

[0176] In this way, the invention aims to improve the quality of education by enabling learning experiences tailored to the individual needs of learners.

[0177] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0178] Step 1:

[0179] The server retrieves lesson information from educational institutions in real time. It receives lesson information from the educational institutions' databases as input and processes it to convert it into a unified format. Specifically, it standardizes different video formats and summarizes text information using natural language processing tools to obtain output that stores the converted educational information in the database.

[0180] Step 2:

[0181] The server prepares to provide the converted educational information to learners at the appropriate time. As input, it receives the specified time slots for the lessons to be viewed from the learners, and uses this information to extract the necessary video fragments from the database. As output, it provides summarized video and text summaries within the specified time range. The server then performs the specific actions to make this information available in streaming format.

[0182] Step 3:

[0183] The terminal displays educational content retrieved from the server to the learner. It receives summary videos and text from the server as input. The terminal delivers this in a format suitable for the user interface, allowing the user to freely manipulate the content. As output, it performs specific actions to display the educational content on the screen, making it viewable by the learner.

[0184] Step 4:

[0185] The emotion engine analyzes the learner's emotional state through the device's camera and sensors. It acquires user facial expression data and voice tone in real time as input. The emotion engine uses a machine learning model to analyze this data and evaluate the user's emotional state. It then sends the results to the server as output.

[0186] Step 5:

[0187] The server dynamically adjusts the educational content based on evaluation results received from the emotion engine. It receives learner emotional state information as input and optimizes the educational content based on this. For example, if it determines that the user is confused, it adds more detailed explanations. As output, it performs specific actions such as sending the adjusted educational content to the device.

[0188] (Application Example 2)

[0189] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0190] In today's educational environment, there is a need to efficiently supplement learning content when students miss classes. Furthermore, flexibly adjusting learning content according to individual students' emotions and levels of understanding is essential for enhancing learning effectiveness. However, traditional systems have difficulty providing learning information that takes students' emotional states into account, resulting in the limitation of only providing fixed content. Moreover, because answers to questions do not reflect students' emotions, it has been difficult to maintain their interest and motivation.

[0191] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0192] In this invention, the server includes a device for collecting learning information at an educational institution, a device for converting the learning information into an appropriate educational format, and a device for supplying the converted educational information to learners via a communication channel. This makes it possible for learners to receive efficient and flexible education by providing learning information optimized according to their individual emotional state, even if they are absent from class.

[0193] "Educational institutions" refer to organizations and facilities that provide learning, and include schools and universities.

[0194] "Learning information" refers to information that learners should acquire, such as lesson content and teaching materials provided by educational institutions.

[0195] "Collection devices" refer to hardware or software used to receive and store learning information from educational institutions.

[0196] A "device for converting to an educational format" refers to a computer system that processes collected learning information into a format that is easy for learners to understand.

[0197] "Communication channel" refers to the network path used to deliver learning information to a device, and includes the internet and dedicated lines.

[0198] "Learner" refers to an individual who receives information from an educational institution, and includes students.

[0199] A "device for detecting emotional states" refers to a system that uses facial recognition technology and sensors to analyze a learner's emotions and determine their emotions in real time.

[0200] A "dynamically adjusting device" refers to a system that modifies or reconfigures learning information in real time based on the learner's emotional state.

[0201] "Inquiry information" refers to question data that learners send to the system to resolve points of confusion or doubts.

[0202] A "generating device" refers to a system equipped with an algorithm for creating answers based on inquiry information and providing them to learners.

[0203] The system implementing this invention is configured to provide educational information and consists of a server, a terminal, and an emotion engine. The server collects learning information from educational institutions and stores it in a database. This information is then converted into a standard format to prepare it for appropriate provision to learners.

[0204] The user (learner) can use their device to watch videos of missed lessons, and by specifying a particular viewing time slot, they receive a streaming video of the lesson with key points summarized. Furthermore, an emotion engine analyzes camera footage and sensor information sent from the device to recognize and evaluate the learner's emotions in real time. The server analyzes this emotional state and dynamically adjusts the content and delivery method of educational information. For example, if it determines that the learner is confused, it may increase the amount of detailed explanations or present lesson content that emphasizes explanations in accessible language.

[0205] The emotion engine employs a communication style that considers the learner's emotions, even when generating answers based on questions, and provides appropriate feedback. In addition, it inserts encouraging words and feedback that promotes a sense of accomplishment to maintain the user's motivation. For example, a home robot supporting a child's learning might, through emotion recognition, advise the child that they may be feeling fatigued, saying, "You might need a short break."

[0206] The program development utilizes OpenCV for facial recognition and TENSORFLOW® for sentiment analysis. The Flask framework is used for data analysis and response generation, and the Pygame library is helpful for providing visual and auditory feedback. The generative AI model generates content using prompts such as "Generate a gentle message to encourage a child who is confused."

[0207] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0208] Step 1:

[0209] The server collects learning information from educational institutions. The input is course data from educational institutions, a resource obtained online. The server converts this data into a standard format and stores it in a database. The output is the stored, standardized learning information.

[0210] Step 2:

[0211] The user begins watching a missed class using their device. The input is the user's viewing request and the specified time slot. Based on this information, the server processes the data to generate a summarized class video. The output is video data streamed to the user's device.

[0212] Step 3:

[0213] The emotion engine acquires video from the device's camera and receives user face and voice data as input. Face recognition is performed using OpenCV, and emotion analysis is performed using TensorFlow. As a result of the data calculations, the user's emotional state (e.g., confused, interested) is output.

[0214] Step 4:

[0215] The server dynamically adjusts learning information based on emotional states. It receives the results of sentiment analysis as input and uses the Flask framework to adjust the educational content. The output is emotionally adjusted learning content. Specifically, for users in a confused state, additional explanations and more approachable language are selected.

[0216] Step 5:

[0217] The user enters a question into the terminal. The server receives this inquiry information and begins analyzing it as input. Using a generative AI model, it generates an emotionally sensitive response based on the prompt text. The output is the generated response text, which is returned to the user.

[0218] Step 6:

[0219] The server monitors the user's learning progress and motivation, and provides timely feedback. Considering learning records and sentiment data as input, the AI ​​model generates encouraging messages and break suggestions using prompts. The output is a message displayed on the device, including specific action suggestions and positive feedback.

[0220] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0221] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0222] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0223] [Second Embodiment]

[0224] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0225] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0226] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0227] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0228] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0229] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0230] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0231] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0232] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0233] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0234] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0235] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0236] The system according to the present invention appropriately provides lesson content to learners who have missed classes at educational institutions. This system consists of a server, terminals, and users.

[0237] The server receives recorded lesson information as classes are held and stores this information in a database. The stored lesson information is then converted into a learning format accessible to learners. For example, the server captures a math class in video format and stores the data in the database. Furthermore, the server converts and provides the learning format in response to user requests to watch the class at a desired time.

[0238] Users can access a dedicated portal using their device to search for information about classes they missed and specify their preferred viewing time. For example, if a user requests a class starting at 3 PM, the server will customize and provide the user with appropriately summarized class information.

[0239] Furthermore, users can input questions that arise while watching the lessons into their devices. The server receives this inquiry information and uses an AI engine to generate answers to the questions. The results are then presented to the user via their device to support their understanding.

[0240] As described above, this system has a structure that supports learners who miss classes in a way that allows them to flexibly access learning opportunities as if they were participating in real-time classes. In this way, it is possible to minimize the loss of learning opportunities due to absence and provide an efficient learning experience. For example, if a user has difficulty understanding a particular mathematical formula and asks a question, the AI ​​will explain the breakdown process of the formula step by step, deepening the user's understanding.

[0241] The following describes the processing flow.

[0242] Step 1:

[0243] The server receives real-time video footage of the lessons from the designated camera system at the start of each class at the educational institution.

[0244] Step 2:

[0245] The server converts the received lecture video data into a standard format (e.g., MP4 format), adds metadata (such as the instructor, lecture date, and subject name), and stores it in the database.

[0246] Step 3:

[0247] Users log in to a dedicated portal via their device and operate an interface to select the classes they wish to watch from a list of missed classes.

[0248] Step 4:

[0249] The user specifies their preferred viewing time for the selected class (e.g., one hour starting at 3 PM) and sends this information to the server.

[0250] Step 5:

[0251] Based on the requested viewing time slot, the server sends the saved lesson data to the AI ​​engine and generates a learning format that summarizes the key points to fit within the given time frame.

[0252] Step 6:

[0253] The server prepares to stream the summarized lesson data generated by the AI ​​engine to the user and begins receiving the data on the user's device.

[0254] Step 7:

[0255] If a user encounters a part they don't understand while watching, they can submit their question to the server by entering it through a question form installed on their device.

[0256] Step 8:

[0257] The server forwards the question received from the user to the AI ​​engine and executes the process of generating an appropriate answer to that question.

[0258] Step 9:

[0259] The server sends the answer received from the AI ​​engine to the user's device and displays the answer on the screen.

[0260] Step 10:

[0261] Users refer to the provided answers and continue watching the lessons while deepening their understanding of the learning material.

[0262] (Example 1)

[0263] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0264] In modern educational institutions, a challenge exists in the learning environment: learners may miss learning opportunities due to absences from classes. Furthermore, there is a lack of mechanisms to effectively review missed lessons, ensuring proper understanding of the material and resolving any questions. Flexible learning support tailored to each learner's level of comprehension is also needed.

[0265] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0266] In this invention, the server includes means for recording learning activities, means for storing information in a storage device, and information processing means for converting the stored information into an appropriate format. This allows learners to efficiently review the content of missed classes and to immediately answer questions they face using a live artificial intelligence model. Furthermore, appropriate learning support is achieved by appropriately summarizing information according to the learner's level of understanding.

[0267] "Learning activities" refer to lectures, classes, and all related educational activities within educational institutions.

[0268] "Means of recording" refers to the technology used to collect information in audio, video, or digital data format and to store that information.

[0269] A "storage device" refers to an electronic device or system that holds recorded data and makes it accessible at a later date.

[0270] "Information processing means" refers to programs, algorithms, or technologies used to perform data transformation, analysis, and modification.

[0271] "Communication means" refers to the technology or infrastructure used for sending and receiving information, and includes the internet and network communication technologies.

[0272] "User" refers to the subject who receives learning content using this system, i.e., the learner.

[0273] "Inquiry content" refers to questions or concerns that users have about the system, and it also refers to data that is processed within the system.

[0274] A "generative artificial intelligence model" refers to a computer program that uses machine learning techniques to analyze, understand, and generate responses from data.

[0275] "Means of generating responses" refers to computational methods or digital technologies for automatically constructing and outputting responses based on the content of inquiries.

[0276] This invention provides a system that records information on learning activities in educational institutions as digital data and provides an environment in which learners can access and learn from that information. The system mainly consists of a server, terminals, and users.

[0277] The server captures learning activities as digital data using equipment designed to record lectures and classes at educational institutions. Video capture software such as "OBS Studio" is used for this recording. The recorded data is processed into an appropriate format and stored in a database. Database management systems such as "MySQL" or "PostgreSQL" are used for database management.

[0278] The server performs information processing to shorten and summarize the stored class data in response to requests from users. In this process, software such as "FFmpeg" is used to trim and summarize videos. Furthermore, the server uses a generative AI model to create answers to the inquiry content received from the user. For example, the "OpenAI GPT model" is used as the "generative AI model".

[0279] Users can access the web portal from a dedicated terminal or personal computer, search for the class content they missed, and specify the desired viewing time. When the user inputs "I want to watch the class starting at 3 pm" via the terminal, the server adjusts the content according to the specified time and provides it to the user. Also, the user can input questions through the terminal during class viewing, and the server derives answers via the generative AI model and displays the results on the terminal. An example of the prompt text at this time is "Please explain the decomposition process of the following mathematical formula step by step: x^2 + y^2 = z^2".

[0280] In this way, the present invention provides an environment where learning opportunities are not missed in real time and realizes an efficient learning experience tailored to the individual needs of learners.

[0281] The flow of the specific process in Example 1 will be described using FIG. 11.

[0282] Step 1:

[0283] The server records the class content in real time at an educational institution. Video capture software is used for recording, and the video feed is converted into digital data as input. This digital data is temporarily stored in a format that includes video and audio.

[0284] Step 2:

[0285] The server stores the recorded digital data in a database. Here, the stored data is organized, and metadata such as the ID, title, date, and instructor name of each class is added. This metadata serves as input data when the user searches later.

[0286] Step 3:

[0287] The user accesses the web portal from the terminal and searches for information on missed classes. The user sends a query based on conditions such as the class ID and date, and the portal sends a data request to the server based on that input.

[0288] Step 4:

[0289] The server selects the class data according to the user's request. The selected data is shortened and summarized using video processing software. The input is full video data, and by applying a summarization algorithm, a shortened video that extracts only the important parts is output.

[0290] Step 5:

[0291] The user watches the summarized class video. A playback command is input from the terminal, and the summarized video, which is the output data from the server, can be streamed and played back.

[0292] Step 6:

[0293] If the user enters a question during class viewing, the terminal sends this question to the server. The question text is input as text data, and the server analyzes it through a generative AI model.

[0294] Step 7:

[0295] The server generates an answer to the question using the generative AI model. The prompt text "Please answer the following question: User's question" is used as the input, and the answer generated by the AI is output.

[0296] Step 8:

[0297] The generated answers are then presented to the user again via the device. Through this output, the user can resolve their questions and deepen their understanding of the material.

[0298] (Application Example 1)

[0299] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0300] Educational institutions need a way to effectively supplement the learning content of students who miss classes, without losing necessary learning opportunities. In this context, the challenge is to provide a system that efficiently catches up on missed classes and can flexibly adapt to the student's level of understanding.

[0301] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0302] In this invention, the server includes means for acquiring learning information from an educational institution, means for converting the learning information into an appropriate information format, and means for providing the converted information to a user via a communication path. This makes it possible to efficiently supplement the content of classes that learners have missed and to realize a learning environment that allows them to view the content according to their own schedule.

[0303] "Learning information" refers to data related to the content of lessons and educational resources at educational institutions.

[0304] "Means of acquisition" refers to technical means for capturing or collecting learning information.

[0305] "Information format" refers to the data format in which acquired learning information is converted into a form that is easy for users to understand.

[0306] A "communication path" is a network or infrastructure for transmitting information.

[0307] A "user" is a learner who has missed classes at an educational institution or a beneficiary of the system.

[0308] "Inquiry information" is data related to questions or requests entered by the user into the system.

[0309] "Means for generating a response" is a technology or process for creating an appropriate answer based on the inquiry information.

[0310] An "interface" is a screen or platform used by the user to operate the system and view information.

[0311] "Means for setting a viewing schedule" is a function or method for the user to specify and plan the viewing time of information.

[0312] To implement this invention, the server, terminal, and user each need to play their respective roles. The server records classes at an educational institution and stores the data in a database. At this time, the server converts the recorded data into an appropriate information format and prepares it so that the user can access it later through a predefined communication path. Usually, a cloud server or a large-capacity storage system is used.

[0313] A terminal is a device for the user to access the data stored by the server. Usually, it refers to a portable terminal such as a smartphone or a tablet, and a dedicated application is installed. Using this terminal, the user can log in to their account and search, select, and view information on the missed classes.

[0314] Furthermore, users can input questions that arise during class on their devices. These questions are sent to a server, which uses a generative AI model to generate appropriate answers. The generated answers are returned to the device in real time and presented to the user. This allows learners not only to catch up on missed lessons but also to deepen their understanding.

[0315] As a concrete example, if a user enters the prompt "I want to know more about how this formula is derived" while watching a recording of a calculus lesson, the server will use a generative AI model to explain the derivation process of the formula step by step and display it on the user's device. An example of such a prompt would be, "Please explain in detail how to calculate differentiation." This system allows learners to access learning opportunities flexibly and minimizes the loss of learning opportunities due to absences.

[0316] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0317] Step 1:

[0318] The server records classes at educational institutions and stores the data in a database. This process saves the recorded video and audio files to the database in a specialized compressed format. At this point, the input is the raw class data, and the output is the compressed recorded data.

[0319] Step 2:

[0320] The user logs into a dedicated application using their terminal and searches the database for the necessary class information. In this case, the input consists of the user's login information and search query, and the output is the recorded data of the corresponding class.

[0321] Step 3:

[0322] The device plays back the received recorded data at a time specified by the user and displays the learning content. At this point, the input is the recorded data and the user's viewing schedule, and the output is the playback of video and audio.

[0323] Step 4:

[0324] Users enter questions that arise during viewing into a question input form within the application. The input here is the prompt text entered by the user, and the output is the transmission of the question data to the server.

[0325] Step 5:

[0326] The server generates answers using a generative AI model based on questions received from the user. This process analyzes the input question through natural language processing and generates answers from a relevant knowledge database. The output is the generated answer text.

[0327] Step 6:

[0328] The terminal displays the response sent from the server to the user. The input here is the response generated by the server, and the output is the response text displayed on the user's screen.

[0329] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0330] This invention combines a system that provides class information to learners who have missed classes in educational institutions with an emotion engine that recognizes learners' emotions and adjusts learning information accordingly. This system consists of a server, terminals, and an emotion engine, and provides a flexible learning experience tailored to the individual needs of learners.

[0331] The server retrieves lesson information from educational institutions in real time and converts it into a standard format using information processing technology. This information is stored in a database and later provided to learners in a summarized and edited format. Users can use their terminals to view videos of lessons they missed and specify their desired viewing time. The server then streams a video of the lesson, summarizing the key points within the specified time.

[0332] The emotion engine analyzes camera footage and sensor information acquired from the user's device to evaluate the learner's emotional state in real time. This evaluation result is sent to the server and used to dynamically adjust the way learning content is delivered. For example, if the server determines that the user is confused, it can provide detailed explanations or navigation using friendly language, following the instructions of the emotion engine.

[0333] Furthermore, the emotion engine assists the server in generating appropriate communication styles by considering the user's emotional state in response to questions entered by the user on the device. In addition, the emotion engine inserts encouraging messages and feedback that gives a sense of accomplishment as needed to maintain or improve the user's motivation.

[0334] In this way, the present invention aims not merely to provide educational information, but to improve learning efficiency and comprehension by understanding the learner's emotional state and providing a personalized learning experience. For example, if the system determines that the user is tired, it can explain the learning content in simpler terms and send a message encouraging them to take a break, thus enabling a learner-friendly system.

[0335] The following describes the processing flow.

[0336] Step 1:

[0337] The server acquires video and audio data from classes held at educational institutions in real time and converts it into a standard format. This allows the class information to be stored in an organized manner in a database.

[0338] Step 2:

[0339] Users log in to a dedicated portal via their device and select the class they wish to view from a list of missed classes. They then specify the time slot they wish to view the class and send this information to the server.

[0340] Step 3:

[0341] The server summarizes saved lesson information based on the time frame specified by the user and converts key points into a learning format. Using an AI engine, it generates lesson content that is adjusted to fit within the time frame.

[0342] Step 4:

[0343] The device uses its built-in camera and sensors to capture the user's facial expressions and movements, and sends this data to the emotion engine. Meanwhile, the server prepares to stream the summarized lesson video.

[0344] Step 5:

[0345] The emotion engine analyzes emotional data received from the user to identify the user's emotional state. For example, it evaluates states such as being focused, tired, or confused.

[0346] Step 6:

[0347] The server adjusts the delivery format of learning content based on the user's emotional state identified by the emotion engine. If it determines that the user is confused, it provides additional explanations and examples to improve the viewing experience.

[0348] Step 7:

[0349] If a user has any questions while watching a lesson, they can enter them through the terminal's interface. These questions are then sent to the server.

[0350] Step 8:

[0351] The server sends the user's question to the emotion engine and initiates the process of generating an answer in a communication style appropriate to the user's emotional state.

[0352] Step 9:

[0353] The server receives responses from the emotion engine and delivers them to the user's device. The user then refers to these responses to ask further questions or continue the lesson.

[0354] Step 10:

[0355] The emotion engine supports the entire learning experience by sending encouraging and feedback messages at appropriate times to maintain user motivation as learning progresses.

[0356] (Example 2)

[0357] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0358] Traditional educational information systems struggled to adequately address the understanding of absent learners. Furthermore, they lacked mechanisms for providing adaptive educational content tailored to learners' emotional states, resulting in insufficient flexibility to meet individual learner needs. This led to a decline in the quality of the learning experience and hindered improvements in learner comprehension and motivation.

[0359] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0360] In this invention, the server includes means for acquiring educational information at an educational facility, means for converting the educational information into a unified format, and means for providing the information converted into the unified format to learners via a communication channel. This enables the analysis of learners' emotional states and adaptive adjustment of educational content according to individual needs, thereby improving each learner's understanding and motivation.

[0361] "Educational institutions" is a general term for institutions and organizations that provide education and training to learners.

[0362] "Educational information" is a general term for information related to learning, including the content of lessons, teaching materials, lecture materials, and other information.

[0363] "Unified format" refers to a state where information from different formats or styles has been converted into a consistent format.

[0364] A "communication channel" is a general term for a path or medium used to send and receive information, and is used to transmit digital data through a network.

[0365] A "learner" is a general term for individuals who acquire knowledge and skills through educational institutions and systems.

[0366] "Inquiry information" refers to information and data that express the questions and doubts that learners have.

[0367] "Response" refers to information including answers, explanations, and feedback in response to inquiries.

[0368] "Emotional state" refers to a person's psychological or emotional state, which is usually determined by the analysis of sensory information.

[0369] "Adaptive educational content" refers to educational materials and information that are dynamically adjusted and modified according to the needs and circumstances of learners.

[0370] This invention is a system for providing educational information to learners in a flexible and adaptive manner within educational facilities. The system is primarily constructed using a server, terminals, and an emotion engine.

[0371] The server has the function of acquiring lesson information from educational institutions in real time and converting it into a unified format using information processing technology. The hardware used includes a database server and a network interface. The database stores the saved, converted educational information and performs any necessary editing. Software used includes a database management system and video conversion software.

[0372] The device provides learners with an environment that makes it easy to access educational content. Learners can watch lesson videos through the device and set their desired viewing time. The server then provides summarized educational content according to the specified time. Furthermore, the device is equipped with a camera and sensors, which are used by an emotion engine to analyze the learner's emotional state.

[0373] The emotion engine analyzes data acquired from the device and evaluates the learner's psychological state in real time. For example, if it determines that the learner is confused, the server is instructed to provide a detailed explanation. This analysis process utilizes emotion analysis algorithms and machine learning techniques, which are applied by the server.

[0374] For example, if the emotion engine determines that a user is tired, the server will translate the learning content into more easily understandable language and send a message encouraging them to take a break. It is also possible to provide encouraging messages to the user using a generative AI model. An example of a prompt is shown below: "Generate a message for a tired learner that briefly explains the learning content and encourages them to take a break."

[0375] In this way, the invention aims to improve the quality of education by enabling learning experiences tailored to the individual needs of learners.

[0376] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0377] Step 1:

[0378] The server retrieves lesson information from educational institutions in real time. It receives lesson information from the educational institutions' databases as input and processes it to convert it into a unified format. Specifically, it standardizes different video formats and summarizes text information using natural language processing tools to obtain output that stores the converted educational information in the database.

[0379] Step 2:

[0380] The server prepares to provide the converted educational information to learners at the appropriate time. As input, it receives the specified time slots for the lessons to be viewed from the learners, and uses this information to extract the necessary video fragments from the database. As output, it provides summarized video and text summaries within the specified time range. The server then performs the specific actions to make this information available in streaming format.

[0381] Step 3:

[0382] The terminal displays educational content retrieved from the server to the learner. It receives summary videos and text from the server as input. The terminal delivers this in a format suitable for the user interface, allowing the user to freely manipulate the content. As output, it performs specific actions to display the educational content on the screen, making it viewable by the learner.

[0383] Step 4:

[0384] The emotion engine analyzes the learner's emotional state through the device's camera and sensors. It acquires user facial expression data and voice tone in real time as input. The emotion engine uses a machine learning model to analyze this data and evaluate the user's emotional state. It then sends the results to the server as output.

[0385] Step 5:

[0386] The server dynamically adjusts the educational content based on evaluation results received from the emotion engine. It receives learner emotional state information as input and optimizes the educational content based on this. For example, if it determines that the user is confused, it adds more detailed explanations. As output, it performs specific actions such as sending the adjusted educational content to the device.

[0387] (Application Example 2)

[0388] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0389] In today's educational environment, there is a need to efficiently supplement learning content when students miss classes. Furthermore, flexibly adjusting learning content according to individual students' emotions and levels of understanding is essential for enhancing learning effectiveness. However, traditional systems have difficulty providing learning information that takes students' emotional states into account, resulting in the limitation of only providing fixed content. Moreover, because answers to questions do not reflect students' emotions, it has been difficult to maintain their interest and motivation.

[0390] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0391] In this invention, the server includes a device for collecting learning information at an educational institution, a device for converting the learning information into an appropriate educational format, and a device for supplying the converted educational information to learners via a communication channel. This makes it possible for learners to receive efficient and flexible education by providing learning information optimized according to their individual emotional state, even if they are absent from class.

[0392] "Educational institutions" refer to organizations and facilities that provide learning, and include schools and universities.

[0393] "Learning information" refers to information that learners should acquire, such as lesson content and teaching materials provided by educational institutions.

[0394] "Collection devices" refer to hardware or software used to receive and store learning information from educational institutions.

[0395] A "device for converting to an educational format" refers to a computer system that processes collected learning information into a format that is easy for learners to understand.

[0396] "Communication channel" refers to the network path used to deliver learning information to a device, and includes the internet and dedicated lines.

[0397] "Learner" refers to an individual who receives information from an educational institution, and includes students.

[0398] A "device for detecting emotional states" refers to a system that uses facial recognition technology and sensors to analyze a learner's emotions and determine their emotions in real time.

[0399] A "dynamically adjusting device" refers to a system that modifies or reconfigures learning information in real time based on the learner's emotional state.

[0400] "Inquiry information" refers to question data that learners send to the system to resolve points of confusion or doubts.

[0401] A "generating device" refers to a system equipped with an algorithm for creating answers based on inquiry information and providing them to learners.

[0402] The system implementing this invention is configured to provide educational information and consists of a server, a terminal, and an emotion engine. The server collects learning information from educational institutions and stores it in a database. This information is then converted into a standard format to prepare it for appropriate provision to learners.

[0403] The user (learner) can use their device to watch videos of missed lessons, and by specifying a particular viewing time slot, they receive a streaming video of the lesson with key points summarized. Furthermore, an emotion engine analyzes camera footage and sensor information sent from the device to recognize and evaluate the learner's emotions in real time. The server analyzes this emotional state and dynamically adjusts the content and delivery method of educational information. For example, if it determines that the learner is confused, it may increase the amount of detailed explanations or present lesson content that emphasizes explanations in accessible language.

[0404] The emotion engine employs a communication style that considers the learner's emotions, even when generating answers based on questions, and provides appropriate feedback. In addition, it inserts encouraging words and feedback that promotes a sense of accomplishment to maintain the user's motivation. For example, a home robot supporting a child's learning might, through emotion recognition, advise the child that they may be feeling fatigued, saying, "You might need a short break."

[0405] The program development uses OpenCV for facial recognition and TensorFlow for sentiment analysis. The Flask framework is used for data analysis and response generation, and the Pygame library is useful for providing visual and auditory feedback. The generative AI model generates content using prompts such as "Generate a kind message to encourage a child who is confused."

[0406] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0407] Step 1:

[0408] The server collects learning information from educational institutions. The input is course data from educational institutions, a resource obtained online. The server converts this data into a standard format and stores it in a database. The output is the stored, standardized learning information.

[0409] Step 2:

[0410] The user begins watching a missed class using their device. The input is the user's viewing request and the specified time slot. Based on this information, the server processes the data to generate a summarized class video. The output is video data streamed to the user's device.

[0411] Step 3:

[0412] The emotion engine acquires video from the device's camera and receives user face and voice data as input. Face recognition is performed using OpenCV, and emotion analysis is performed using TensorFlow. As a result of the data calculations, the user's emotional state (e.g., confused, interested) is output.

[0413] Step 4:

[0414] The server dynamically adjusts learning information based on emotional states. It receives the results of sentiment analysis as input and uses the Flask framework to adjust the educational content. The output is emotionally adjusted learning content. Specifically, for users in a confused state, additional explanations and more approachable language are selected.

[0415] Step 5:

[0416] The user enters a question into the terminal. The server receives this inquiry information and begins analyzing it as input. Using a generative AI model, it generates an emotionally sensitive response based on the prompt text. The output is the generated response text, which is returned to the user.

[0417] Step 6:

[0418] The server monitors the user's learning progress and motivation, and provides timely feedback. Considering learning records and sentiment data as input, the AI ​​model generates encouraging messages and break suggestions using prompts. The output is a message displayed on the device, including specific action suggestions and positive feedback.

[0419] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0420] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0421] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0422] [Third Embodiment]

[0423] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0424] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0425] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0426] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0427] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0429] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0430] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0431] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0432] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0433] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0434] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0435] The system according to the present invention appropriately provides lesson content to learners who have missed classes at educational institutions. This system consists of a server, terminals, and users.

[0436] The server receives recorded lesson information as classes are held and stores this information in a database. The stored lesson information is then converted into a learning format accessible to learners. For example, the server captures a math class in video format and stores the data in the database. Furthermore, the server converts and provides the learning format in response to user requests to watch the class at a desired time.

[0437] Users can access a dedicated portal using their device to search for information about classes they missed and specify their preferred viewing time. For example, if a user requests a class starting at 3 PM, the server will customize and provide the user with appropriately summarized class information.

[0438] Furthermore, users can input questions that arise while watching the lessons into their devices. The server receives this inquiry information and uses an AI engine to generate answers to the questions. The results are then presented to the user via their device to support their understanding.

[0439] As described above, this system has a structure that supports learners who miss classes in a way that allows them to flexibly access learning opportunities as if they were participating in real-time classes. In this way, it is possible to minimize the loss of learning opportunities due to absence and provide an efficient learning experience. For example, if a user has difficulty understanding a particular mathematical formula and asks a question, the AI ​​will explain the breakdown process of the formula step by step, deepening the user's understanding.

[0440] The following describes the processing flow.

[0441] Step 1:

[0442] The server receives real-time video footage of the lessons from the designated camera system at the start of each class at the educational institution.

[0443] Step 2:

[0444] The server converts the received lecture video data into a standard format (e.g., MP4 format), adds metadata (such as the instructor, lecture date, and subject name), and stores it in the database.

[0445] Step 3:

[0446] Users log in to a dedicated portal via their device and operate an interface to select the classes they wish to watch from a list of missed classes.

[0447] Step 4:

[0448] The user specifies their preferred viewing time for the selected class (e.g., one hour starting at 3 PM) and sends this information to the server.

[0449] Step 5:

[0450] Based on the requested viewing time slot, the server sends the saved lesson data to the AI ​​engine and generates a learning format that summarizes the key points to fit within the given time frame.

[0451] Step 6:

[0452] The server prepares to stream the summarized lesson data generated by the AI ​​engine to the user and begins receiving the data on the user's device.

[0453] Step 7:

[0454] If a user encounters a part they don't understand while watching, they can submit their question to the server by entering it through a question form installed on their device.

[0455] Step 8:

[0456] The server forwards the question received from the user to the AI ​​engine and executes the process of generating an appropriate answer to that question.

[0457] Step 9:

[0458] The server sends the answer received from the AI ​​engine to the user's device and displays the answer on the screen.

[0459] Step 10:

[0460] Users refer to the provided answers and continue watching the lessons while deepening their understanding of the learning material.

[0461] (Example 1)

[0462] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0463] In modern educational institutions, a challenge exists in the learning environment: learners may miss learning opportunities due to absences from classes. Furthermore, there is a lack of mechanisms to effectively review missed lessons, ensuring proper understanding of the material and resolving any questions. Flexible learning support tailored to each learner's level of comprehension is also needed.

[0464] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0465] In this invention, the server includes means for recording learning activities, means for storing information in a storage device, and information processing means for converting the stored information into an appropriate format. This allows learners to efficiently review the content of missed classes and to immediately answer questions they face using a live artificial intelligence model. Furthermore, appropriate learning support is achieved by appropriately summarizing information according to the learner's level of understanding.

[0466] "Learning activities" refer to lectures, classes, and all related educational activities within educational institutions.

[0467] "Means of recording" refers to the technology used to collect information in audio, video, or digital data format and to store that information.

[0468] A "storage device" refers to an electronic device or system that holds recorded data and makes it accessible at a later date.

[0469] "Information processing means" refers to programs, algorithms, or technologies used to perform data transformation, analysis, and modification.

[0470] "Communication means" refers to the technology or infrastructure used for sending and receiving information, and includes the internet and network communication technologies.

[0471] "User" refers to the subject who receives learning content using this system, i.e., the learner.

[0472] "Inquiry content" refers to questions or concerns that users have about the system, and it also refers to data that is processed within the system.

[0473] A "generative artificial intelligence model" refers to a computer program that uses machine learning techniques to analyze, understand, and generate responses from data.

[0474] "Means of generating responses" refers to computational methods or digital technologies for automatically constructing and outputting responses based on the content of inquiries.

[0475] This invention provides a system that records information on learning activities in educational institutions as digital data and provides an environment in which learners can access and learn from that information. The system mainly consists of a server, terminals, and users.

[0476] The server captures learning activities as digital data using equipment designed to record lectures and classes at educational institutions. Video capture software such as "OBS Studio" is used for this recording. The recorded data is processed into an appropriate format and stored in a database. Database management systems such as "MySQL" or "PostgreSQL" are used for database management.

[0477] The server processes stored lesson data to shorten and summarize it in response to user requests. This process uses software such as "FFmpeg" to trim and summarize videos. Furthermore, the server uses a generative AI model to create answers to user inquiries. For example, "OpenAI's GPT model" is used as the "generative AI model."

[0478] Users can access a web portal from a dedicated terminal or personal computer to search for missed class content and specify their desired viewing time. If a user enters "I would like to view the class from 3 PM" via the terminal, the server will adjust the content to fit the specified time and provide it to the user. Furthermore, users can enter questions via the terminal while viewing the class, and the server will derive answers through a generative AI model, displaying the results on the terminal. An example of a prompt message at this time is, "Please explain the decomposition process of the following equation step by step: x^2 + y^2 = z^2".

[0479] Thus, the present invention provides an environment that ensures learners do not miss learning opportunities in real time, and realizes an efficient learning experience tailored to the individual needs of each learner.

[0480] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0481] Step 1:

[0482] The server records classroom sessions in real time. Video capture software is used for recording, converting the video feed into digital data. This digital data is temporarily stored in a format that includes both video and audio.

[0483] Step 2:

[0484] The server stores the recorded digital data in a database. Here, the stored data is organized, and metadata such as the ID, title, date, and instructor's name for each lesson is added. This metadata serves as input data for users to search later.

[0485] Step 3:

[0486] Users access the web portal from their device to search for information about classes they missed. Users send queries based on criteria such as class ID and date, and the portal sends data requests to the server based on that input.

[0487] Step 4:

[0488] The server selects lesson data according to the user's request. The selected data is shortened and summarized using video processing software. The input is the full video data, and by applying a summarization algorithm, it outputs a shortened video that extracts only the important parts.

[0489] Step 5:

[0490] Users watch summarized lecture videos. A play command is entered from the terminal, and the summarized video, which is output data from the server, can be streamed.

[0491] Step 6:

[0492] When a user enters a question while watching a lesson, their device sends this question to the server. The question is entered as text data, and the server analyzes it using a generative AI model.

[0493] Step 7:

[0494] The server generates answers to questions using a generative AI model. It takes "Answer the following question: User's question" as input and outputs the answer generated by the AI.

[0495] Step 8:

[0496] The generated answers are then presented to the user again via the device. Through this output, the user can resolve their questions and deepen their understanding of the material.

[0497] (Application Example 1)

[0498] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0499] Educational institutions need a way to effectively supplement the learning content of students who miss classes, without losing necessary learning opportunities. In this context, the challenge is to provide a system that efficiently catches up on missed classes and can flexibly adapt to the student's level of understanding.

[0500] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0501] In this invention, the server includes means for acquiring learning information from an educational institution, means for converting the learning information into an appropriate information format, and means for providing the converted information to a user via a communication path. This makes it possible to efficiently supplement the content of classes that learners have missed and to realize a learning environment that allows them to view the content according to their own schedule.

[0502] "Learning information" refers to data related to the content of lessons and educational resources at educational institutions.

[0503] "Means of acquisition" refers to technical means for capturing or collecting learning information.

[0504] "Information format" refers to the data format in which acquired learning information is converted into a form that is easy for users to understand.

[0505] A "communication path" refers to the network or infrastructure used to transmit information.

[0506] A "user" is a student who has missed classes at an educational institution or a beneficiary of the system.

[0507] "Inquiry information" refers to data related to questions and requests that users enter into the system.

[0508] "Means of generating a response" refers to the technologies and processes used to create an appropriate answer based on the inquiry information.

[0509] An "interface" is a screen or platform that users use to operate a system and view information.

[0510] "Means for setting a viewing schedule" refers to functions or methods that allow users to specify and plan the amount of time they will spend viewing information.

[0511] To implement this invention, a server, a terminal, and a user must each fulfill their respective roles. The server records lessons at educational institutions and stores the data in a database. In doing so, the server converts the recorded data into an appropriate information format and prepares it for later access by users through a predefined communication path. This typically involves using a cloud server or a large-capacity storage system.

[0512] A terminal is a device used by users to access data stored on a server. Typically, this refers to a mobile device such as a smartphone or tablet, with a dedicated application installed. Using this terminal, users can log in to their account and search, select, and view information about missed classes.

[0513] Furthermore, users can input questions that arise during class on their devices. These questions are sent to a server, which uses a generative AI model to generate appropriate answers. The generated answers are returned to the device in real time and presented to the user. This allows learners not only to catch up on missed lessons but also to deepen their understanding.

[0514] As a concrete example, if a user enters the prompt "I want to know more about how this formula is derived" while watching a recording of a calculus lesson, the server will use a generative AI model to explain the derivation process of the formula step by step and display it on the user's device. An example of such a prompt would be, "Please explain in detail how to calculate differentiation." This system allows learners to access learning opportunities flexibly and minimizes the loss of learning opportunities due to absences.

[0515] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0516] Step 1:

[0517] The server records classes at educational institutions and stores the data in a database. This process saves the recorded video and audio files to the database in a specialized compressed format. At this point, the input is the raw class data, and the output is the compressed recorded data.

[0518] Step 2:

[0519] The user logs into a dedicated application using their terminal and searches the database for the necessary class information. In this case, the input consists of the user's login information and search query, and the output is the recorded data of the corresponding class.

[0520] Step 3:

[0521] The device plays back the received recorded data at a time specified by the user and displays the learning content. At this point, the input is the recorded data and the user's viewing schedule, and the output is the playback of video and audio.

[0522] Step 4:

[0523] Users enter questions that arise during viewing into a question input form within the application. The input here is the prompt text entered by the user, and the output is the transmission of the question data to the server.

[0524] Step 5:

[0525] The server generates answers using a generative AI model based on questions received from the user. This process analyzes the input question through natural language processing and generates answers from a relevant knowledge database. The output is the generated answer text.

[0526] Step 6:

[0527] The terminal displays the response sent from the server to the user. The input here is the response generated by the server, and the output is the response text displayed on the user's screen.

[0528] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0529] This invention combines a system that provides class information to learners who have missed classes in educational institutions with an emotion engine that recognizes learners' emotions and adjusts learning information accordingly. This system consists of a server, terminals, and an emotion engine, and provides a flexible learning experience tailored to the individual needs of learners.

[0530] The server retrieves lesson information from educational institutions in real time and converts it into a standard format using information processing technology. This information is stored in a database and later provided to learners in a summarized and edited format. Users can use their terminals to view videos of lessons they missed and specify their desired viewing time. The server then streams a video of the lesson, summarizing the key points within the specified time.

[0531] The emotion engine analyzes camera footage and sensor information acquired from the user's device to evaluate the learner's emotional state in real time. This evaluation result is sent to the server and used to dynamically adjust the way learning content is delivered. For example, if the server determines that the user is confused, it can provide detailed explanations or navigation using friendly language, following the instructions of the emotion engine.

[0532] Furthermore, the emotion engine assists the server in generating appropriate communication styles by considering the user's emotional state in response to questions entered by the user on the device. In addition, the emotion engine inserts encouraging messages and feedback that gives a sense of accomplishment as needed to maintain or improve the user's motivation.

[0533] In this way, the present invention aims not merely to provide educational information, but to improve learning efficiency and comprehension by understanding the learner's emotional state and providing a personalized learning experience. For example, if the system determines that the user is tired, it can explain the learning content in simpler terms and send a message encouraging them to take a break, thus enabling a learner-friendly system.

[0534] The following describes the processing flow.

[0535] Step 1:

[0536] The server acquires video and audio data from classes held at educational institutions in real time and converts it into a standard format. This allows the class information to be stored in an organized manner in a database.

[0537] Step 2:

[0538] Users log in to a dedicated portal via their device and select the class they wish to view from a list of missed classes. They then specify the time slot they wish to view the class and send this information to the server.

[0539] Step 3:

[0540] The server summarizes saved lesson information based on the time frame specified by the user and converts key points into a learning format. Using an AI engine, it generates lesson content that is adjusted to fit within the time frame.

[0541] Step 4:

[0542] The device uses its built-in camera and sensors to capture the user's facial expressions and movements, and sends this data to the emotion engine. Meanwhile, the server prepares to stream the summarized lesson video.

[0543] Step 5:

[0544] The emotion engine analyzes emotional data received from the user to identify the user's emotional state. For example, it evaluates states such as being focused, tired, or confused.

[0545] Step 6:

[0546] The server adjusts the delivery format of learning content based on the user's emotional state identified by the emotion engine. If it determines that the user is confused, it provides additional explanations and examples to improve the viewing experience.

[0547] Step 7:

[0548] If a user has any questions while watching a lesson, they can enter them through the terminal's interface. These questions are then sent to the server.

[0549] Step 8:

[0550] The server sends the user's question to the emotion engine and initiates the process of generating an answer in a communication style appropriate to the user's emotional state.

[0551] Step 9:

[0552] The server receives responses from the emotion engine and delivers them to the user's device. The user then refers to these responses to ask further questions or continue the lesson.

[0553] Step 10:

[0554] The emotion engine supports the entire learning experience by sending encouraging and feedback messages at appropriate times to maintain user motivation as learning progresses.

[0555] (Example 2)

[0556] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0557] Traditional educational information systems struggled to adequately address the understanding of absent learners. Furthermore, they lacked mechanisms for providing adaptive educational content tailored to learners' emotional states, resulting in insufficient flexibility to meet individual learner needs. This led to a decline in the quality of the learning experience and hindered improvements in learner comprehension and motivation.

[0558] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0559] In this invention, the server includes means for acquiring educational information at an educational facility, means for converting the educational information into a unified format, and means for providing the information converted into the unified format to learners via a communication channel. This enables the analysis of learners' emotional states and adaptive adjustment of educational content according to individual needs, thereby improving each learner's understanding and motivation.

[0560] "Educational institutions" is a general term for institutions and organizations that provide education and training to learners.

[0561] "Educational information" is a general term for information related to learning, including the content of lessons, teaching materials, lecture materials, and other information.

[0562] "Unified format" refers to a state where information from different formats or styles has been converted into a consistent format.

[0563] A "communication channel" is a general term for a path or medium used to send and receive information, and is used to transmit digital data through a network.

[0564] A "learner" is a general term for individuals who acquire knowledge and skills through educational institutions and systems.

[0565] "Inquiry information" refers to information and data that express the questions and doubts that learners have.

[0566] "Response" refers to information including answers, explanations, and feedback in response to inquiries.

[0567] "Emotional state" refers to a person's psychological or emotional state, which is usually determined by the analysis of sensory information.

[0568] "Adaptive educational content" refers to educational materials and information that are dynamically adjusted and modified according to the needs and circumstances of learners.

[0569] This invention is a system for providing educational information to learners in a flexible and adaptive manner within educational facilities. The system is primarily constructed using a server, terminals, and an emotion engine.

[0570] The server has the function of acquiring lesson information from educational institutions in real time and converting it into a unified format using information processing technology. The hardware used includes a database server and a network interface. The database stores the saved, converted educational information and performs any necessary editing. Software used includes a database management system and video conversion software.

[0571] The device provides learners with an environment that makes it easy to access educational content. Learners can watch lesson videos through the device and set their desired viewing time. The server then provides summarized educational content according to the specified time. Furthermore, the device is equipped with a camera and sensors, which are used by an emotion engine to analyze the learner's emotional state.

[0572] The emotion engine analyzes data acquired from the device and evaluates the learner's psychological state in real time. For example, if it determines that the learner is confused, the server is instructed to provide a detailed explanation. This analysis process utilizes emotion analysis algorithms and machine learning techniques, which are applied by the server.

[0573] For example, if the emotion engine determines that a user is tired, the server will translate the learning content into more easily understandable language and send a message encouraging them to take a break. It is also possible to provide encouraging messages to the user using a generative AI model. An example of a prompt is shown below: "Generate a message for a tired learner that briefly explains the learning content and encourages them to take a break."

[0574] In this way, the invention aims to improve the quality of education by enabling learning experiences tailored to the individual needs of learners.

[0575] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0576] Step 1:

[0577] The server retrieves lesson information from educational institutions in real time. It receives lesson information from the educational institutions' databases as input and processes it to convert it into a unified format. Specifically, it standardizes different video formats and summarizes text information using natural language processing tools to obtain output that stores the converted educational information in the database.

[0578] Step 2:

[0579] The server prepares to provide the converted educational information to learners at the appropriate time. As input, it receives the specified time slots for the lessons to be viewed from the learners, and uses this information to extract the necessary video fragments from the database. As output, it provides summarized video and text summaries within the specified time range. The server then performs the specific actions to make this information available in streaming format.

[0580] Step 3:

[0581] The terminal displays educational content retrieved from the server to the learner. It receives summary videos and text from the server as input. The terminal delivers this in a format suitable for the user interface, allowing the user to freely manipulate the content. As output, it performs specific actions to display the educational content on the screen, making it viewable by the learner.

[0582] Step 4:

[0583] The emotion engine analyzes the learner's emotional state through the device's camera and sensors. It acquires user facial expression data and voice tone in real time as input. The emotion engine uses a machine learning model to analyze this data and evaluate the user's emotional state. It then sends the results to the server as output.

[0584] Step 5:

[0585] The server dynamically adjusts the educational content based on evaluation results received from the emotion engine. It receives learner emotional state information as input and optimizes the educational content based on this. For example, if it determines that the user is confused, it adds more detailed explanations. As output, it performs specific actions such as sending the adjusted educational content to the device.

[0586] (Application Example 2)

[0587] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0588] In today's educational environment, there is a need to efficiently supplement learning content when students miss classes. Furthermore, flexibly adjusting learning content according to individual students' emotions and levels of understanding is essential for enhancing learning effectiveness. However, traditional systems have difficulty providing learning information that takes students' emotional states into account, resulting in the limitation of only providing fixed content. Moreover, because answers to questions do not reflect students' emotions, it has been difficult to maintain their interest and motivation.

[0589] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0590] In this invention, the server includes a device for collecting learning information at an educational institution, a device for converting the learning information into an appropriate educational format, and a device for supplying the converted educational information to learners via a communication channel. This makes it possible for learners to receive efficient and flexible education by providing learning information optimized according to their individual emotional state, even if they are absent from class.

[0591] "Educational institutions" refer to organizations and facilities that provide learning, and include schools and universities.

[0592] "Learning information" refers to information that learners should acquire, such as lesson content and teaching materials provided by educational institutions.

[0593] "Collection devices" refer to hardware or software used to receive and store learning information from educational institutions.

[0594] A "device for converting to an educational format" refers to a computer system that processes collected learning information into a format that is easy for learners to understand.

[0595] "Communication channel" refers to the network path used to deliver learning information to a device, and includes the internet and dedicated lines.

[0596] "Learner" refers to an individual who receives information from an educational institution, and includes students.

[0597] A "device for detecting emotional states" refers to a system that uses facial recognition technology and sensors to analyze a learner's emotions and determine their emotions in real time.

[0598] A "dynamically adjusting device" refers to a system that modifies or reconfigures learning information in real time based on the learner's emotional state.

[0599] "Inquiry information" refers to question data that learners send to the system to resolve points of confusion or doubts.

[0600] A "generating device" refers to a system equipped with an algorithm for creating answers based on inquiry information and providing them to learners.

[0601] The system implementing this invention is configured to provide educational information and consists of a server, a terminal, and an emotion engine. The server collects learning information from educational institutions and stores it in a database. This information is then converted into a standard format to prepare it for appropriate provision to learners.

[0602] The user (learner) can use their device to watch videos of missed lessons, and by specifying a particular viewing time slot, they receive a streaming video of the lesson with key points summarized. Furthermore, an emotion engine analyzes camera footage and sensor information sent from the device to recognize and evaluate the learner's emotions in real time. The server analyzes this emotional state and dynamically adjusts the content and delivery method of educational information. For example, if it determines that the learner is confused, it may increase the amount of detailed explanations or present lesson content that emphasizes explanations in accessible language.

[0603] The emotion engine employs a communication style that considers the learner's emotions, even when generating answers based on questions, and provides appropriate feedback. In addition, it inserts encouraging words and feedback that promotes a sense of accomplishment to maintain the user's motivation. For example, a home robot supporting a child's learning might, through emotion recognition, advise the child that they may be feeling fatigued, saying, "You might need a short break."

[0604] The program development uses OpenCV for facial recognition and TensorFlow for sentiment analysis. The Flask framework is used for data analysis and response generation, and the Pygame library is useful for providing visual and auditory feedback. The generative AI model generates content using prompts such as "Generate a kind message to encourage a child who is confused."

[0605] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0606] Step 1:

[0607] The server collects learning information from educational institutions. The input is course data from educational institutions, a resource obtained online. The server converts this data into a standard format and stores it in a database. The output is the stored, standardized learning information.

[0608] Step 2:

[0609] The user begins watching a missed class using their device. The input is the user's viewing request and the specified time slot. Based on this information, the server processes the data to generate a summarized class video. The output is video data streamed to the user's device.

[0610] Step 3:

[0611] The emotion engine acquires video from the device's camera and receives user face and voice data as input. Face recognition is performed using OpenCV, and emotion analysis is performed using TensorFlow. As a result of the data calculations, the user's emotional state (e.g., confused, interested) is output.

[0612] Step 4:

[0613] The server dynamically adjusts learning information based on emotional states. It receives the results of sentiment analysis as input and uses the Flask framework to adjust the educational content. The output is emotionally adjusted learning content. Specifically, for users in a confused state, additional explanations and more approachable language are selected.

[0614] Step 5:

[0615] The user enters a question into the terminal. The server receives this inquiry information and begins analyzing it as input. Using a generative AI model, it generates an emotionally sensitive response based on the prompt text. The output is the generated response text, which is returned to the user.

[0616] Step 6:

[0617] The server monitors the user's learning progress and motivation, and provides timely feedback. Considering learning records and sentiment data as input, the AI ​​model generates encouraging messages and break suggestions using prompts. The output is a message displayed on the device, including specific action suggestions and positive feedback.

[0618] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0619] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0620] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0621] [Fourth Embodiment]

[0622] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0623] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0624] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0625] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0626] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0627] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0628] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0629] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0630] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0631] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0632] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0633] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0634] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0635] The system according to the present invention appropriately provides lesson content to learners who have missed classes at educational institutions. This system consists of a server, terminals, and users.

[0636] The server receives recorded lesson information as classes are held and stores this information in a database. The stored lesson information is then converted into a learning format accessible to learners. For example, the server captures a math class in video format and stores the data in the database. Furthermore, the server converts and provides the learning format in response to user requests to watch the class at a desired time.

[0637] Users can access a dedicated portal using their device to search for information about classes they missed and specify their preferred viewing time. For example, if a user requests a class starting at 3 PM, the server will customize and provide the user with appropriately summarized class information.

[0638] Furthermore, users can input questions that arise while watching the lessons into their devices. The server receives this inquiry information and uses an AI engine to generate answers to the questions. The results are then presented to the user via their device to support their understanding.

[0639] As described above, this system has a structure that supports learners who miss classes in a way that allows them to flexibly access learning opportunities as if they were participating in real-time classes. In this way, it is possible to minimize the loss of learning opportunities due to absence and provide an efficient learning experience. For example, if a user has difficulty understanding a particular mathematical formula and asks a question, the AI ​​will explain the breakdown process of the formula step by step, deepening the user's understanding.

[0640] The following describes the processing flow.

[0641] Step 1:

[0642] The server receives real-time video footage of the lessons from the designated camera system at the start of each class at the educational institution.

[0643] Step 2:

[0644] The server converts the received lecture video data into a standard format (e.g., MP4 format), adds metadata (such as the instructor, lecture date, and subject name), and stores it in the database.

[0645] Step 3:

[0646] Users log in to a dedicated portal via their device and operate an interface to select the classes they wish to watch from a list of missed classes.

[0647] Step 4:

[0648] The user specifies their preferred viewing time for the selected class (e.g., one hour starting at 3 PM) and sends this information to the server.

[0649] Step 5:

[0650] Based on the requested viewing time slot, the server sends the saved lesson data to the AI ​​engine and generates a learning format that summarizes the key points to fit within the given time frame.

[0651] Step 6:

[0652] The server prepares to stream the summarized lesson data generated by the AI ​​engine to the user and begins receiving the data on the user's device.

[0653] Step 7:

[0654] If a user encounters a part they don't understand while watching, they can submit their question to the server by entering it through a question form installed on their device.

[0655] Step 8:

[0656] The server forwards the question received from the user to the AI ​​engine and executes the process of generating an appropriate answer to that question.

[0657] Step 9:

[0658] The server sends the answer received from the AI ​​engine to the user's device and displays the answer on the screen.

[0659] Step 10:

[0660] Users refer to the provided answers and continue watching the lessons while deepening their understanding of the learning material.

[0661] (Example 1)

[0662] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0663] In modern educational institutions, a challenge exists in the learning environment: learners may miss learning opportunities due to absences from classes. Furthermore, there is a lack of mechanisms to effectively review missed lessons, ensuring proper understanding of the material and resolving any questions. Flexible learning support tailored to each learner's level of comprehension is also needed.

[0664] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0665] In this invention, the server includes means for recording learning activities, means for storing information in a storage device, and information processing means for converting the stored information into an appropriate format. This allows learners to efficiently review the content of missed classes and to immediately answer questions they face using a live artificial intelligence model. Furthermore, appropriate learning support is achieved by appropriately summarizing information according to the learner's level of understanding.

[0666] "Learning activities" refer to lectures, classes, and all related educational activities within educational institutions.

[0667] "Means of recording" refers to the technology used to collect information in audio, video, or digital data format and to store that information.

[0668] A "storage device" refers to an electronic device or system that holds recorded data and makes it accessible at a later date.

[0669] "Information processing means" refers to programs, algorithms, or technologies used to perform data transformation, analysis, and modification.

[0670] "Communication means" refers to the technology or infrastructure used for sending and receiving information, and includes the internet and network communication technologies.

[0671] "User" refers to the subject who receives learning content using this system, i.e., the learner.

[0672] "Inquiry content" refers to questions or concerns that users have about the system, and it also refers to data that is processed within the system.

[0673] A "generative artificial intelligence model" refers to a computer program that uses machine learning techniques to analyze, understand, and generate responses from data.

[0674] "Means of generating responses" refers to computational methods or digital technologies for automatically constructing and outputting responses based on the content of inquiries.

[0675] This invention provides a system that records information on learning activities in educational institutions as digital data and provides an environment in which learners can access and learn from that information. The system mainly consists of a server, terminals, and users.

[0676] The server captures learning activities as digital data using equipment designed to record lectures and classes at educational institutions. Video capture software such as "OBS Studio" is used for this recording. The recorded data is processed into an appropriate format and stored in a database. Database management systems such as "MySQL" or "PostgreSQL" are used for database management.

[0677] The server processes stored lesson data to shorten and summarize it in response to user requests. This process uses software such as "FFmpeg" to trim and summarize videos. Furthermore, the server uses a generative AI model to create answers to user inquiries. For example, "OpenAI's GPT model" is used as the "generative AI model."

[0678] Users can access a web portal from a dedicated terminal or personal computer to search for missed class content and specify their desired viewing time. If a user enters "I would like to view the class from 3 PM" via the terminal, the server will adjust the content to fit the specified time and provide it to the user. Furthermore, users can enter questions via the terminal while viewing the class, and the server will derive answers through a generative AI model, displaying the results on the terminal. An example of a prompt message at this time is, "Please explain the decomposition process of the following equation step by step: x^2 + y^2 = z^2".

[0679] Thus, the present invention provides an environment that ensures learners do not miss learning opportunities in real time, and realizes an efficient learning experience tailored to the individual needs of each learner.

[0680] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0681] Step 1:

[0682] The server records classroom sessions in real time. Video capture software is used for recording, converting the video feed into digital data. This digital data is temporarily stored in a format that includes both video and audio.

[0683] Step 2:

[0684] The server stores the recorded digital data in a database. Here, the stored data is organized, and metadata such as the ID, title, date, and instructor's name for each lesson is added. This metadata serves as input data for users to search later.

[0685] Step 3:

[0686] Users access the web portal from their device to search for information about classes they missed. Users send queries based on criteria such as class ID and date, and the portal sends data requests to the server based on that input.

[0687] Step 4:

[0688] The server selects lesson data according to the user's request. The selected data is shortened and summarized using video processing software. The input is the full video data, and by applying a summarization algorithm, it outputs a shortened video that extracts only the important parts.

[0689] Step 5:

[0690] Users watch summarized lecture videos. A play command is entered from the terminal, and the summarized video, which is output data from the server, can be streamed.

[0691] Step 6:

[0692] When a user enters a question while watching a lesson, their device sends this question to the server. The question is entered as text data, and the server analyzes it using a generative AI model.

[0693] Step 7:

[0694] The server generates answers to questions using a generative AI model. It takes "Answer the following question: User's question" as input and outputs the answer generated by the AI.

[0695] Step 8:

[0696] The generated answers are then presented to the user again via the device. Through this output, the user can resolve their questions and deepen their understanding of the material.

[0697] (Application Example 1)

[0698] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0699] Educational institutions need a way to effectively supplement the learning content of students who miss classes, without losing necessary learning opportunities. In this context, the challenge is to provide a system that efficiently catches up on missed classes and can flexibly adapt to the student's level of understanding.

[0700] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0701] In this invention, the server includes means for acquiring learning information from an educational institution, means for converting the learning information into an appropriate information format, and means for providing the converted information to a user via a communication path. This makes it possible to efficiently supplement the content of classes that learners have missed and to realize a learning environment that allows them to view the content according to their own schedule.

[0702] "Learning information" refers to data related to the content of lessons and educational resources at educational institutions.

[0703] "Means of acquisition" refers to technical means for capturing or collecting learning information.

[0704] "Information format" refers to the data format in which acquired learning information is converted into a form that is easy for users to understand.

[0705] A "communication path" refers to the network or infrastructure used to transmit information.

[0706] A "user" is a student who has missed classes at an educational institution or a beneficiary of the system.

[0707] "Inquiry information" refers to data related to questions and requests that users enter into the system.

[0708] "Means of generating a response" refers to the technologies and processes used to create an appropriate answer based on the inquiry information.

[0709] An "interface" is a screen or platform that users use to operate a system and view information.

[0710] "Means for setting a viewing schedule" refers to functions or methods that allow users to specify and plan the amount of time they will spend viewing information.

[0711] To implement this invention, a server, a terminal, and a user must each fulfill their respective roles. The server records lessons at educational institutions and stores the data in a database. In doing so, the server converts the recorded data into an appropriate information format and prepares it for later access by users through a predefined communication path. This typically involves using a cloud server or a large-capacity storage system.

[0712] A terminal is a device used by users to access data stored on a server. Typically, this refers to a mobile device such as a smartphone or tablet, with a dedicated application installed. Using this terminal, users can log in to their account and search, select, and view information about missed classes.

[0713] Furthermore, users can input questions that arise during class on their devices. These questions are sent to a server, which uses a generative AI model to generate appropriate answers. The generated answers are returned to the device in real time and presented to the user. This allows learners not only to catch up on missed lessons but also to deepen their understanding.

[0714] As a concrete example, if a user enters the prompt "I want to know more about how this formula is derived" while watching a recording of a calculus lesson, the server will use a generative AI model to explain the derivation process of the formula step by step and display it on the user's device. An example of such a prompt would be, "Please explain in detail how to calculate differentiation." This system allows learners to access learning opportunities flexibly and minimizes the loss of learning opportunities due to absences.

[0715] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0716] Step 1:

[0717] The server records classes at educational institutions and stores the data in a database. This process saves the recorded video and audio files to the database in a specialized compressed format. At this point, the input is the raw class data, and the output is the compressed recorded data.

[0718] Step 2:

[0719] The user logs into a dedicated application using their terminal and searches the database for the necessary class information. In this case, the input consists of the user's login information and search query, and the output is the recorded data of the corresponding class.

[0720] Step 3:

[0721] The device plays back the received recorded data at a time specified by the user and displays the learning content. At this point, the input is the recorded data and the user's viewing schedule, and the output is the playback of video and audio.

[0722] Step 4:

[0723] Users enter questions that arise during viewing into a question input form within the application. The input here is the prompt text entered by the user, and the output is the transmission of the question data to the server.

[0724] Step 5:

[0725] The server generates answers using a generative AI model based on questions received from the user. This process analyzes the input question through natural language processing and generates answers from a relevant knowledge database. The output is the generated answer text.

[0726] Step 6:

[0727] The terminal displays the response sent from the server to the user. The input here is the response generated by the server, and the output is the response text displayed on the user's screen.

[0728] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0729] This invention combines a system that provides class information to learners who have missed classes in educational institutions with an emotion engine that recognizes learners' emotions and adjusts learning information accordingly. This system consists of a server, terminals, and an emotion engine, and provides a flexible learning experience tailored to the individual needs of learners.

[0730] The server retrieves lesson information from educational institutions in real time and converts it into a standard format using information processing technology. This information is stored in a database and later provided to learners in a summarized and edited format. Users can use their terminals to view videos of lessons they missed and specify their desired viewing time. The server then streams a video of the lesson, summarizing the key points within the specified time.

[0731] The emotion engine analyzes camera footage and sensor information acquired from the user's device to evaluate the learner's emotional state in real time. This evaluation result is sent to the server and used to dynamically adjust the way learning content is delivered. For example, if the server determines that the user is confused, it can provide detailed explanations or navigation using friendly language, following the instructions of the emotion engine.

[0732] Furthermore, the emotion engine assists the server in generating appropriate communication styles by considering the user's emotional state in response to questions entered by the user on the device. In addition, the emotion engine inserts encouraging messages and feedback that gives a sense of accomplishment as needed to maintain or improve the user's motivation.

[0733] In this way, the present invention aims not merely to provide educational information, but to improve learning efficiency and comprehension by understanding the learner's emotional state and providing a personalized learning experience. For example, if the system determines that the user is tired, it can explain the learning content in simpler terms and send a message encouraging them to take a break, thus enabling a learner-friendly system.

[0734] The following describes the processing flow.

[0735] Step 1:

[0736] The server acquires video and audio data from classes held at educational institutions in real time and converts it into a standard format. This allows the class information to be stored in an organized manner in a database.

[0737] Step 2:

[0738] Users log in to a dedicated portal via their device and select the class they wish to view from a list of missed classes. They then specify the time slot they wish to view the class and send this information to the server.

[0739] Step 3:

[0740] The server summarizes saved lesson information based on the time frame specified by the user and converts key points into a learning format. Using an AI engine, it generates lesson content that is adjusted to fit within the time frame.

[0741] Step 4:

[0742] The device uses its built-in camera and sensors to capture the user's facial expressions and movements, and sends this data to the emotion engine. Meanwhile, the server prepares to stream the summarized lesson video.

[0743] Step 5:

[0744] The emotion engine analyzes emotional data received from the user to identify the user's emotional state. For example, it evaluates states such as being focused, tired, or confused.

[0745] Step 6:

[0746] The server adjusts the delivery format of learning content based on the user's emotional state identified by the emotion engine. If it determines that the user is confused, it provides additional explanations and examples to improve the viewing experience.

[0747] Step 7:

[0748] If a user has any questions while watching a lesson, they can enter them through the terminal's interface. These questions are then sent to the server.

[0749] Step 8:

[0750] The server sends the user's question to the emotion engine and initiates the process of generating an answer in a communication style appropriate to the user's emotional state.

[0751] Step 9:

[0752] The server receives responses from the emotion engine and delivers them to the user's device. The user then refers to these responses to ask further questions or continue the lesson.

[0753] Step 10:

[0754] The emotion engine supports the entire learning experience by sending encouraging and feedback messages at appropriate times to maintain user motivation as learning progresses.

[0755] (Example 2)

[0756] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0757] Traditional educational information systems struggled to adequately address the understanding of absent learners. Furthermore, they lacked mechanisms for providing adaptive educational content tailored to learners' emotional states, resulting in insufficient flexibility to meet individual learner needs. This led to a decline in the quality of the learning experience and hindered improvements in learner comprehension and motivation.

[0758] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0759] In this invention, the server includes means for acquiring educational information at an educational facility, means for converting the educational information into a unified format, and means for providing the information converted into the unified format to learners via a communication channel. This enables the analysis of learners' emotional states and adaptive adjustment of educational content according to individual needs, thereby improving each learner's understanding and motivation.

[0760] "Educational institutions" is a general term for institutions and organizations that provide education and training to learners.

[0761] "Educational information" is a general term for information related to learning, including the content of lessons, teaching materials, lecture materials, and other information.

[0762] "Unified format" refers to a state where information from different formats or styles has been converted into a consistent format.

[0763] A "communication channel" is a general term for a path or medium used to send and receive information, and is used to transmit digital data through a network.

[0764] A "learner" is a general term for individuals who acquire knowledge and skills through educational institutions and systems.

[0765] "Inquiry information" refers to information and data that express the questions and doubts that learners have.

[0766] "Response" refers to information including answers, explanations, and feedback in response to inquiries.

[0767] "Emotional state" refers to a person's psychological or emotional state, which is usually determined by the analysis of sensory information.

[0768] "Adaptive educational content" refers to educational materials and information that are dynamically adjusted and modified according to the needs and circumstances of learners.

[0769] This invention is a system for providing educational information to learners in a flexible and adaptive manner within educational facilities. The system is primarily constructed using a server, terminals, and an emotion engine.

[0770] The server has the function of acquiring lesson information from educational institutions in real time and converting it into a unified format using information processing technology. The hardware used includes a database server and a network interface. The database stores the saved, converted educational information and performs any necessary editing. Software used includes a database management system and video conversion software.

[0771] The device provides learners with an environment that makes it easy to access educational content. Learners can watch lesson videos through the device and set their desired viewing time. The server then provides summarized educational content according to the specified time. Furthermore, the device is equipped with a camera and sensors, which are used by an emotion engine to analyze the learner's emotional state.

[0772] The emotion engine analyzes data acquired from the device and evaluates the learner's psychological state in real time. For example, if it determines that the learner is confused, the server is instructed to provide a detailed explanation. This analysis process utilizes emotion analysis algorithms and machine learning techniques, which are applied by the server.

[0773] For example, if the emotion engine determines that a user is tired, the server will translate the learning content into more easily understandable language and send a message encouraging them to take a break. It is also possible to provide encouraging messages to the user using a generative AI model. An example of a prompt is shown below: "Generate a message for a tired learner that briefly explains the learning content and encourages them to take a break."

[0774] In this way, the invention aims to improve the quality of education by enabling learning experiences tailored to the individual needs of learners.

[0775] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0776] Step 1:

[0777] The server retrieves lesson information from educational institutions in real time. It receives lesson information from the educational institutions' databases as input and processes it to convert it into a unified format. Specifically, it standardizes different video formats and summarizes text information using natural language processing tools to obtain output that stores the converted educational information in the database.

[0778] Step 2:

[0779] The server prepares to provide the converted educational information to learners at the appropriate time. As input, it receives the specified time slots for the lessons to be viewed from the learners, and uses this information to extract the necessary video fragments from the database. As output, it provides summarized video and text summaries within the specified time range. The server then performs the specific actions to make this information available in streaming format.

[0780] Step 3:

[0781] The terminal displays educational content retrieved from the server to the learner. It receives summary videos and text from the server as input. The terminal delivers this in a format suitable for the user interface, allowing the user to freely manipulate the content. As output, it performs specific actions to display the educational content on the screen, making it viewable by the learner.

[0782] Step 4:

[0783] The emotion engine analyzes the learner's emotional state through the device's camera and sensors. It acquires user facial expression data and voice tone in real time as input. The emotion engine uses a machine learning model to analyze this data and evaluate the user's emotional state. It then sends the results to the server as output.

[0784] Step 5:

[0785] The server dynamically adjusts the educational content based on evaluation results received from the emotion engine. It receives learner emotional state information as input and optimizes the educational content based on this. For example, if it determines that the user is confused, it adds more detailed explanations. As output, it performs specific actions such as sending the adjusted educational content to the device.

[0786] (Application Example 2)

[0787] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0788] In today's educational environment, there is a need to efficiently supplement learning content when students miss classes. Furthermore, flexibly adjusting learning content according to individual students' emotions and levels of understanding is essential for enhancing learning effectiveness. However, traditional systems have difficulty providing learning information that takes students' emotional states into account, resulting in the limitation of only providing fixed content. Moreover, because answers to questions do not reflect students' emotions, it has been difficult to maintain their interest and motivation.

[0789] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0790] In this invention, the server includes a device for collecting learning information at an educational institution, a device for converting the learning information into an appropriate educational format, and a device for supplying the converted educational information to learners via a communication channel. This makes it possible for learners to receive efficient and flexible education by providing learning information optimized according to their individual emotional state, even if they are absent from class.

[0791] "Educational institutions" refer to organizations and facilities that provide learning, and include schools and universities.

[0792] "Learning information" refers to information that learners should acquire, such as lesson content and teaching materials provided by educational institutions.

[0793] "Collection devices" refer to hardware or software used to receive and store learning information from educational institutions.

[0794] A "device for converting to an educational format" refers to a computer system that processes collected learning information into a format that is easy for learners to understand.

[0795] "Communication channel" refers to the network path used to deliver learning information to a device, and includes the internet and dedicated lines.

[0796] "Learner" refers to an individual who receives information from an educational institution, and includes students.

[0797] A "device for detecting emotional states" refers to a system that uses facial recognition technology and sensors to analyze a learner's emotions and determine their emotions in real time.

[0798] A "dynamically adjusting device" refers to a system that modifies or reconfigures learning information in real time based on the learner's emotional state.

[0799] "Inquiry information" refers to question data that learners send to the system to resolve points of confusion or doubts.

[0800] A "generating device" refers to a system equipped with an algorithm for creating answers based on inquiry information and providing them to learners.

[0801] The system implementing this invention is configured to provide educational information and consists of a server, a terminal, and an emotion engine. The server collects learning information from educational institutions and stores it in a database. This information is then converted into a standard format to prepare it for appropriate provision to learners.

[0802] The user (learner) can use their device to watch videos of missed lessons, and by specifying a particular viewing time slot, they receive a streaming video of the lesson with key points summarized. Furthermore, an emotion engine analyzes camera footage and sensor information sent from the device to recognize and evaluate the learner's emotions in real time. The server analyzes this emotional state and dynamically adjusts the content and delivery method of educational information. For example, if it determines that the learner is confused, it may increase the amount of detailed explanations or present lesson content that emphasizes explanations in accessible language.

[0803] The emotion engine employs a communication style that considers the learner's emotions, even when generating answers based on questions, and provides appropriate feedback. In addition, it inserts encouraging words and feedback that promotes a sense of accomplishment to maintain the user's motivation. For example, a home robot supporting a child's learning might, through emotion recognition, advise the child that they may be feeling fatigued, saying, "You might need a short break."

[0804] The program development uses OpenCV for facial recognition and TensorFlow for sentiment analysis. The Flask framework is used for data analysis and response generation, and the Pygame library is useful for providing visual and auditory feedback. The generative AI model generates content using prompts such as "Generate a kind message to encourage a child who is confused."

[0805] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0806] Step 1:

[0807] The server collects learning information from educational institutions. The input is course data from educational institutions, a resource obtained online. The server converts this data into a standard format and stores it in a database. The output is the stored, standardized learning information.

[0808] Step 2:

[0809] The user begins watching a missed class using their device. The input is the user's viewing request and the specified time slot. Based on this information, the server processes the data to generate a summarized class video. The output is video data streamed to the user's device.

[0810] Step 3:

[0811] The emotion engine acquires video from the device's camera and receives user face and voice data as input. Face recognition is performed using OpenCV, and emotion analysis is performed using TensorFlow. As a result of the data calculations, the user's emotional state (e.g., confused, interested) is output.

[0812] Step 4:

[0813] The server dynamically adjusts learning information based on emotional states. It receives the results of sentiment analysis as input and uses the Flask framework to adjust the educational content. The output is emotionally adjusted learning content. Specifically, for users in a confused state, additional explanations and more approachable language are selected.

[0814] Step 5:

[0815] The user enters a question into the terminal. The server receives this inquiry information and begins analyzing it as input. Using a generative AI model, it generates an emotionally sensitive response based on the prompt text. The output is the generated response text, which is returned to the user.

[0816] Step 6:

[0817] The server monitors the user's learning progress and motivation, and provides timely feedback. Considering learning records and sentiment data as input, the AI ​​model generates encouraging messages and break suggestions using prompts. The output is a message displayed on the device, including specific action suggestions and positive feedback.

[0818] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0819] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0820] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0821] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0822] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0823] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0824] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0825] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0826] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0827] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0828] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0829] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0830] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0831] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0832] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0833] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0834] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0835] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0836] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0837] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0838] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0839] The following is further disclosed regarding the embodiments described above.

[0840] (Claim 1)

[0841] A means of capturing learning information in educational institutions,

[0842] means for converting the aforementioned learning information into an appropriate learning format,

[0843] A means for providing the information converted into the aforementioned learning format to the learner via a communication path,

[0844] A means for receiving inquiry information from the learner and generating an answer based on said inquiry information,

[0845] A means for presenting the generated answer to the learner,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, further comprising means for generating additional learning information according to the learner's level of understanding based on the aforementioned inquiry information.

[0849] (Claim 3)

[0850] The system according to claim 1, wherein the conversion means uses information processing technology to summarize the key points within a time frame specified by the learner.

[0851] "Example 1"

[0852] (Claim 1)

[0853] Means for recording learning activities,

[0854] Means for storing the aforementioned learning activities in a storage device,

[0855] Information processing means for converting the stored information into an appropriate format,

[0856] A means of providing the converted information to the user via a communication means,

[0857] A means for processing the content of inquiries received from the user's terminal,

[0858] A means for generating an answer using a generative artificial intelligence model based on the aforementioned inquiry content,

[0859] A means of distributing the generated responses to users,

[0860] A system that includes this.

[0861] (Claim 2)

[0862] The system according to claim 1, further comprising an information processing device for generating additional learning materials according to the user's level of understanding.

[0863] (Claim 3)

[0864] The system according to claim 1, which applies advanced information processing technology to summarize learning content to fit within a time frame specified by the user.

[0865] "Application Example 1"

[0866] (Claim 1)

[0867] Means of obtaining learning information in educational institutions,

[0868] Means for converting the aforementioned learning information into an appropriate information format,

[0869] Means for providing the information converted to the aforementioned information format to the user via a communication path,

[0870] A means for receiving inquiry information from the user and generating a response based on said inquiry information,

[0871] Means for presenting the generated response to the user,

[0872] A means of setting a viewing schedule through an interface,

[0873] A system that includes this.

[0874] (Claim 2)

[0875] The system according to claim 1, further comprising means for generating additional information according to the user's level of understanding based on the aforementioned inquiry information.

[0876] (Claim 3)

[0877] The system according to claim 1, wherein the conversion means uses information processing technology to summarize the key points within a time frame specified by the user.

[0878] "Example 2 of combining an emotion engine"

[0879] (Claim 1)

[0880] Means of obtaining educational information in educational institutions,

[0881] A means for converting the aforementioned educational information into a unified format,

[0882] A means for providing the information converted to the aforementioned unified format to the learner via a communication channel,

[0883] A means for receiving inquiry information from learners and generating a response based on said inquiry information,

[0884] Means for presenting the generated response to the learner,

[0885] To analyze the emotional state of learners, a means of collecting and evaluating data from a device,

[0886] Means for adaptively adjusting learning content according to the aforementioned emotional state,

[0887] A system that includes this.

[0888] (Claim 2)

[0889] The system according to claim 1, further comprising means for generating additional educational information corresponding to the learner's level of understanding and emotional state based on the aforementioned inquiry information.

[0890] (Claim 3)

[0891] The system according to claim 1, wherein the conversion means uses information processing technology to summarize and provide key points within a time frame specified by the learner.

[0892] "Application example 2 when combining with an emotional engine"

[0893] (Claim 1)

[0894] A device for collecting learning information in educational institutions,

[0895] A device for converting the aforementioned learning information into an appropriate educational format,

[0896] A device that supplies information converted into the aforementioned educational format to learners via a communication channel,

[0897] The device for detecting the emotional state of the learner,

[0898] A device that dynamically adjusts learning information based on the aforementioned emotional state,

[0899] A device that receives inquiry information from the learner and generates an answer based on the inquiry information,

[0900] A device for displaying the generated answer to the learner,

[0901] A system that includes this.

[0902] (Claim 2)

[0903] The system according to claim 1, further comprising a device that generates additional educational information corresponding to the learner's level of understanding and emotions, based on the aforementioned inquiry information and the learner's emotional state.

[0904] (Claim 3)

[0905] The system according to claim 1, wherein the transforming device uses an information processing method to summarize the key points within a time frame specified by the learner and to adjust the details according to the learner's emotional state. [Explanation of Symbols]

[0906] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of capturing learning information in educational institutions, means for converting the aforementioned learning information into an appropriate learning format, A means for providing the information converted into the aforementioned learning format to the learner via a communication path, A means for receiving inquiry information from the learner and generating an answer based on said inquiry information, A means for presenting the generated answer to the learner, A system that includes this.

2. The system according to claim 1, further comprising means for generating additional learning information according to the learner's level of understanding based on the aforementioned inquiry information.

3. The system according to claim 1, wherein the conversion means uses information processing technology to summarize the key points within a time frame specified by the learner.

Images